tg3 and Netgear GA302T x 2 locks machine

tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

Got my two new Netgear GA302t nics today.  They seem to use the
tg3 driver....

I put them into the 64/66 slots on my Tyan dual amd motherboard..
Running kernel 2.4.20-pre8

eth2 is plugged into eth3 over cross-over cable.

The tg3 found and used them, mostly:

tg3.c:v1.1 (Aug 30, 2002)
eth2: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:20:56
eth3: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:66MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:22:fd
tg3: eth3: Link is up at 1000 Mbps, full duplex.
tg3: eth3: Flow control is off for TX and off for RX.
...
tg3: eth3: Link is up at 1000 Mbps, full duplex.
tg3: eth3: Flow control is off for TX and off for RX.
tg3: eth3: Link is down.
tg3: eth3: Link is up at 1000 Mbps, full duplex.
tg3: eth3: Flow control is on for TX and on for RX.

I tried to start traffic with pktgen, and the machine hangs solid.
The last thing I see is:

[root@localhost root]# tg3: eth2: transmit timed out, resetting


Any ideas?  Is this NIC supposed to work with tg3?

Thanks,
Ben


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

Ben Greear wrote:
> Got my two new Netgear GA302t nics today.  They seem to use the
> tg3 driver....

Changed the PCI slot of eth2 to a 32-bit slot, and now it works better...

It passed packets for about 20 seconds, then spit out a bunch of these:

tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled

... Then the machine locked hard


Ben

-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: tg3 and Netgear GA302T x 2 locks machine

[James Morris]

On Thu, 3 Oct 2002, Ben Greear wrote:

> Changed the PCI slot of eth2 to a 32-bit slot, and now it works
> better...
> 
> It passed packets for about 20 seconds, then spit out a bunch of these:
> 

Which kernel version?  There was a lockup problem for the GA302T just 
after NAPI went in (which was fixed soon after).


- James
-- 
James Morris
<jmorris@intercode.com.au>

Re: tg3 and Netgear GA302T x 2 locks machine

[David S. Miller]

   From: Ben Greear <greearb@candelatech.com>
   Date: Thu, 03 Oct 2002 21:19:37 -0700

   Got my two new Netgear GA302t nics today.  They seem to use the
   tg3 driver....
   
   I put them into the 64/66 slots on my Tyan dual amd motherboard..
   Running kernel 2.4.20-pre8
   
You reported the other week problems with two Acenic's in
this same machine right?  The second Acenic wouldn't even probe
properly.  And the two Acenic's were identical.

I see a pattern developing... :-)

Re: tg3 and Netgear GA302T x 2 locks machine

[David S. Miller]

   From: James Morris <jmorris@intercode.com.au>
   Date: Sat, 5 Oct 2002 11:20:06 +1000 (EST)

   On Fri, 4 Oct 2002, David S. Miller wrote:
   
   > You reported the other week problems with two Acenic's in
   > this same machine right?  The second Acenic wouldn't even probe
   > properly.  And the two Acenic's were identical.
   
   FWIW, my GA302T seems fine with the kernel he originally reported 
   (2.4.20-pre8).

Yes but are you running parallel pktgen streams on two
tg3's? :-)

Ben, by chance, does reverting the patch below cure your problems?

# This is a BitKeeper generated patch for the following project:
# Project Name: Linux kernel tree
# This patch format is intended for GNU patch command version 2.5 or higher.
# This patch includes the following deltas:
#	           ChangeSet	1.670   -> 1.671  
#	   drivers/net/tg3.c	1.35    -> 1.36   
#
# The following is the BitKeeper ChangeSet Log
# --------------------------------------------
# 02/09/17	davem@nuts.ninka.net	1.671
# [TIGON3]: Optimize NAPI irq masking a bit.
# --------------------------------------------
#
diff -Nru a/drivers/net/tg3.c b/drivers/net/tg3.c
--- a/drivers/net/tg3.c	Fri Oct  4 18:18:26 2002
+++ b/drivers/net/tg3.c	Fri Oct  4 18:18:26 2002
@@ -234,9 +234,23 @@
 		tw32(GRC_LOCAL_CTRL,
 		     tp->grc_local_ctrl | GRC_LCLCTRL_SETINT);
 	}
-#if 0
 	tr32(MAILBOX_INTERRUPT_0 + TG3_64BIT_REG_LOW);
-#endif
+}
+
+static inline void tg3_mask_ints(struct tg3 *tp)
+{
+	tw32(TG3PCI_MISC_HOST_CTRL,
+	     (tp->misc_host_ctrl | MISC_HOST_CTRL_MASK_PCI_INT));
+}
+
+static inline void tg3_unmask_ints(struct tg3 *tp)
+{
+	tw32(TG3PCI_MISC_HOST_CTRL,
+	     (tp->misc_host_ctrl & ~MISC_HOST_CTRL_MASK_PCI_INT));
+	if (tp->hw_status->status & SD_STATUS_UPDATED) {
+		tw32(GRC_LOCAL_CTRL,
+		     tp->grc_local_ctrl | GRC_LCLCTRL_SETINT);
+	}
 }
 
 static void tg3_switch_clocks(struct tg3 *tp)
@@ -2093,7 +2107,7 @@
 
 	if (done) {
 		netif_rx_complete(netdev);
-		tg3_enable_ints(tp);
+		tg3_unmask_ints(tp);
 	}
 
 	spin_unlock_irq(&tp->lock);
@@ -2120,11 +2134,10 @@
 		return;
 
 	if (netif_rx_schedule_prep(dev)) {
-		/* NOTE: This write is posted by the readback of
+		/* NOTE: These writes are posted by the readback of
 		 *       the mailbox register done by our caller.
 		 */
-		tw32(TG3PCI_MISC_HOST_CTRL,
-		     (tp->misc_host_ctrl | MISC_HOST_CTRL_MASK_PCI_INT));
+		tg3_mask_ints(tp);
 		__netif_rx_schedule(dev);
 	} else {
 		printk(KERN_ERR PFX "%s: Error, poll already scheduled\n",

Re: tg3 and Netgear GA302T x 2 locks machine

[James Morris]

On Fri, 4 Oct 2002, David S. Miller wrote:

>    From: Ben Greear <greearb@candelatech.com>
>    Date: Thu, 03 Oct 2002 21:19:37 -0700
> 
>    Got my two new Netgear GA302t nics today.  They seem to use the
>    tg3 driver....
>    
>    I put them into the 64/66 slots on my Tyan dual amd motherboard..
>    Running kernel 2.4.20-pre8
>    
> You reported the other week problems with two Acenic's in
> this same machine right?  The second Acenic wouldn't even probe
> properly.  And the two Acenic's were identical.
> 

FWIW, my GA302T seems fine with the kernel he originally reported 
(2.4.20-pre8).


- James
-- 
James Morris
<jmorris@intercode.com.au>

Re: tg3 and Netgear GA302T x 2 locks machine

[Matthew N. Andrews]

James Morris wrote:

>On Fri, 4 Oct 2002, David S. Miller wrote:
>
>  
>
>>   From: Ben Greear <greearb@candelatech.com>
>>   Date: Thu, 03 Oct 2002 21:19:37 -0700
>>
>>   Got my two new Netgear GA302t nics today.  They seem to use the
>>   tg3 driver....
>>   
>>   I put them into the 64/66 slots on my Tyan dual amd motherboard..
>>   Running kernel 2.4.20-pre8
>>   
>>You reported the other week problems with two Acenic's in
>>this same machine right?  The second Acenic wouldn't even probe
>>properly.  And the two Acenic's were identical.
>>
>>    
>>
>
>FWIW, my GA302T seems fine with the kernel he originally reported 
>(2.4.20-pre8).
>
>
>- James
>  
>
What version of the bios does your motherboard have? I had problems with 
a dual syskonnect board on this motherboard(tyan 2466) where it would 
not would not initialize properly under the older bios.

-Matt

Re: tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

David S. Miller wrote:
>    From: Ben Greear <greearb@candelatech.com>
>    Date: Thu, 03 Oct 2002 21:19:37 -0700
> 
>    Got my two new Netgear GA302t nics today.  They seem to use the
>    tg3 driver....
>    
>    I put them into the 64/66 slots on my Tyan dual amd motherboard..
>    Running kernel 2.4.20-pre8
>    
> You reported the other week problems with two Acenic's in
> this same machine right?  The second Acenic wouldn't even probe
> properly.  And the two Acenic's were identical.
> 
> I see a pattern developing... :-)
> 

Yep, one of my 64-bit PCI slots only seems to work with Intel NICs.
Go figure.

At any rate, I have 4 32-bit slots, and they seem to work, at
least long enough for the driver to puke messages and hard-lock my
machine ;)

If you think this NIC should work with the tg3, then I can change
them to a completely other machine, btw.

Ben

-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

James Morris wrote:
> On Thu, 3 Oct 2002, Ben Greear wrote:
> 
> 
>>Changed the PCI slot of eth2 to a 32-bit slot, and now it works
>>better...
>>
>>It passed packets for about 20 seconds, then spit out a bunch of these:
>>
> 
> 
> Which kernel version?  There was a lockup problem for the GA302T just 
> after NAPI went in (which was fixed soon after).

This was 2.4.20-pre8, ie quite recent.  I didn't see any mention of
tg3 in the pre9 patch that just came out.

I'm beginning to think SMP may not be worth it...will try in my trusty
P-IV single-processor cheap-o box next.

Ben

> 
> 
> - James


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

David S. Miller wrote:
>    From: James Morris <jmorris@intercode.com.au>
>    Date: Sat, 5 Oct 2002 11:20:06 +1000 (EST)
> 
>    On Fri, 4 Oct 2002, David S. Miller wrote:
>    
>    > You reported the other week problems with two Acenic's in
>    > this same machine right?  The second Acenic wouldn't even probe
>    > properly.  And the two Acenic's were identical.
>    
>    FWIW, my GA302T seems fine with the kernel he originally reported 
>    (2.4.20-pre8).
> 
> Yes but are you running parallel pktgen streams on two
> tg3's? :-)

Heh, tis true, I'm abusing them horribly. ;)

So far, the tulip driver and DFE-570tx & Phobos NICs are the only
thing I've found that can stand up to my tests...  (e1000 pukes,
old acenic pukes (after quite a while), tg3 pukes, at least currently...

I'll try this this weekend.  Thanks for the suggestion.

Ben

> 
> Ben, by chance, does reverting the patch below cure your problems?

[patch snipped]


-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

David S. Miller wrote:

> Yes but are you running parallel pktgen streams on two
> tg3's? :-)
> 
> Ben, by chance, does reverting the patch below cure your problems?

No, didn't fix it, maybe made it worse.

With pktgen:  Locked up as soon as I started it (or less than .5 seconds afterwards).

With raw ethernet packets, sent from user-space, at around 40Mbps bi-directional,
I see loads of these messages:


tg3: eth3: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth2: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled
tg3: eth3: Error, poll already scheduled

...

After about 1 minute, the whole machine locked.

Ben

-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear

Re: tg3 and Netgear GA302T x 2 locks machine

[David S. Miller]

   From: Ben Greear <greearb@candelatech.com>
   Date: Sat, 05 Oct 2002 13:08:02 -0700
   
   With raw ethernet packets, sent from user-space, at around 40Mbps bi-directional,
   I see loads of these messages:
   
   tg3: eth3: Error, poll already scheduled

This, frankly, isn't possible.

When we get the first interrupt, we hold the spinlock and have IRQs
disabled, in that environment we invoke netif_rx_schedule_prep(dev)
and then disable device interrupts....

is the tg3 sharing it's IRQ with something else?  That might be
an important clue.  In that case what you report might be possible.

Otherwise the message you see appears to be totally impossible.

Re: tg3 and Netgear GA302T x 2 locks machine

[Ben Greear]

David S. Miller wrote:
>    From: Ben Greear <greearb@candelatech.com>
>    Date: Sat, 05 Oct 2002 13:08:02 -0700
>    
>    With raw ethernet packets, sent from user-space, at around 40Mbps bi-directional,
>    I see loads of these messages:
>    
>    tg3: eth3: Error, poll already scheduled
> 
> This, frankly, isn't possible.
> 
> When we get the first interrupt, we hold the spinlock and have IRQs
> disabled, in that environment we invoke netif_rx_schedule_prep(dev)
> and then disable device interrupts....
> 
> is the tg3 sharing it's IRQ with something else?  That might be
> an important clue.  In that case what you report might be possible.

Here is what the interrupts look like:

[root@localhost lanforge]# cat /proc/interrupts
            CPU0       CPU1
   0:      14734      19562    IO-APIC-edge  timer
   1:          3          1    IO-APIC-edge  keyboard
   2:          0          0          XT-PIC  cascade
   4:       1163       1124    IO-APIC-edge  serial
   5:          0          0   IO-APIC-level  eth1
   8:          0          1    IO-APIC-edge  rtc
   9:        558          0   IO-APIC-level  eth2, eth4
  10:          0          0   IO-APIC-level  usb-ohci
  11:       1385       1454   IO-APIC-level  eth0, eth3, eth5
  12:         20         12    IO-APIC-edge  PS/2 Mouse
  14:       3958       4169    IO-APIC-edge  ide0
  15:          7         13    IO-APIC-edge  ide1
NMI:          0          0
LOC:      34138      34183
ERR:          0
MIS:          0
[root@localhost lanforge]#


eth0-1 is the 3com built-in nics
eth2-3 is the e1000 dual nic
eth4-5 is the tg3

Oct  5 21:42:10 localhost kernel: tg3.c:v1.1 (Aug 30, 2002)
Oct  5 21:42:11 localhost kernel: eth4: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:33MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:22:fd
Oct  5 21:42:11 localhost kernel: eth5: Tigon3 [partno(AC91002A1) rev 0105 PHY(5701)] (PCI:33MHz:32-bit) 10/100/1000BaseT Ethernet 00:40:f4:47:20:56


Upon starting a user-space TCP connection to myself (no traffic running on eth2-3,
eth1 is not plugged in to a cable.  eth0 is handling a small amount of traffic,
no more than about 25 packets per second on average):

[root@localhost lanforge]# tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled
tg3: eth5: Error, poll already scheduled


Please let me know what other debugging info I can get you.

> Otherwise the message you see appears to be totally impossible.

I told the machine that...but it only blinked it's little leds in mirth!  ;)


Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>       <Ben_Greear AT excite.com>
President of Candela Technologies Inc      http://www.candelatech.com
ScryMUD:  http://scry.wanfear.com     http://scry.wanfear.com/~greear