* Problems with tg3 on BCM5720
@ 2012-09-26 13:48 Dirkjan Ochtman
2012-09-26 21:13 ` Nithin Nayak Sujir
2012-09-26 21:40 ` Michael Chan
0 siblings, 2 replies; 5+ messages in thread
From: Dirkjan Ochtman @ 2012-09-26 13:48 UTC (permalink / raw)
To: netdev
Hi,
I hope this is the right list for this; please let me know if there is
some more appropriate venue for this kind of problem.
We have bought a Dell PowerEdge server recently with Broadcom ethernet
controllers:
01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
Gigabit Ethernet PCIe
01:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
Gigabit Ethernet PCIe
02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
Gigabit Ethernet PCIe
02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
Gigabit Ethernet PCIe
Using the tg3 driver as provided by the 3.4.9 kernel (it's a distro
kernel, but I verified that there are no relevant patches on top of
the vanilla 3.4.9), networking works fine on a small VLAN. However,
we've run into trouble connecting one of the interfaces to switch run
by our data center vendor (the switch side works fine when connecting
with a laptop). The driver keeps erroring out with "transmit timed
out, resetting" (full syslog output is at [1]). This also keeps
happening when I use the latest tg3.{c,h} from torvalds/linux-2.6
(though I've had to revert a few things to make it compile inside the
3.4.9 kernel).
I've also run some tests with ethtool:
djc@jansky ~ $ sudo ethtool --test eth0
The test result is FAIL
The test extra info:
nvram test (online) 0
link test (online) 1
register test (offline) 0
memory test (offline) 0
mac loopback test (offline) 7
phy loopback test (offline) 7
ext loopback test (offline) 0
interrupt test (offline) 1
While I'd be happy to do more digging, at this point I'm out of ideas
on how to go about that.
Again, sorry if this is off-topic for this list, any help would be appreciated.
Cheers,
Dirkjan
[1] http://dirkjan.ochtman.nl/files/tg3-fail.txt
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: Problems with tg3 on BCM5720
2012-09-26 13:48 Problems with tg3 on BCM5720 Dirkjan Ochtman
@ 2012-09-26 21:13 ` Nithin Nayak Sujir
2012-09-27 6:45 ` Dirkjan Ochtman
2012-09-26 21:40 ` Michael Chan
1 sibling, 1 reply; 5+ messages in thread
From: Nithin Nayak Sujir @ 2012-09-26 21:13 UTC (permalink / raw)
To: Dirkjan Ochtman; +Cc: netdev
Hi Dirkjan,
A few questions -
1. Can you tell me the last patch that is included in the tg3 driver in 3.4.9 on your distro?
2. Can you give more info about the working setup?
3. Was there any system reset or driver reload between the working and not working setups? Or was it
just a cable switch?
4. Please give the output of
ethtool eth0
ethtool -i eth0
ethtool -k eth0
5. Can you run ethtool --test in the working setup?
6. I noticed in the syslog, the link is coming up at 100 Mbps. Is this expected?
7. Does it fail immediately on connect to the data center switch? Or is it after some traffic goes
through?
Thanks,
Nithin.
On 09/26/2012 06:48 AM, Dirkjan Ochtman wrote:
> Hi,
>
> I hope this is the right list for this; please let me know if there is
> some more appropriate venue for this kind of problem.
>
> We have bought a Dell PowerEdge server recently with Broadcom ethernet
> controllers:
>
> 01:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
> Gigabit Ethernet PCIe
> 01:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
> Gigabit Ethernet PCIe
> 02:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
> Gigabit Ethernet PCIe
> 02:00.1 Ethernet controller: Broadcom Corporation NetXtreme BCM5720
> Gigabit Ethernet PCIe
>
> Using the tg3 driver as provided by the 3.4.9 kernel (it's a distro
> kernel, but I verified that there are no relevant patches on top of
> the vanilla 3.4.9), networking works fine on a small VLAN. However,
> we've run into trouble connecting one of the interfaces to switch run
> by our data center vendor (the switch side works fine when connecting
> with a laptop). The driver keeps erroring out with "transmit timed
> out, resetting" (full syslog output is at [1]). This also keeps
> happening when I use the latest tg3.{c,h} from torvalds/linux-2.6
> (though I've had to revert a few things to make it compile inside the
> 3.4.9 kernel).
>
> I've also run some tests with ethtool:
>
> djc@jansky ~ $ sudo ethtool --test eth0
> The test result is FAIL
> The test extra info:
> nvram test (online) 0
> link test (online) 1
> register test (offline) 0
> memory test (offline) 0
> mac loopback test (offline) 7
> phy loopback test (offline) 7
> ext loopback test (offline) 0
> interrupt test (offline) 1
>
> While I'd be happy to do more digging, at this point I'm out of ideas
> on how to go about that.
>
> Again, sorry if this is off-topic for this list, any help would be appreciated.
>
> Cheers,
>
> Dirkjan
>
> [1] http://dirkjan.ochtman.nl/files/tg3-fail.txt
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
^ permalink raw reply [flat|nested] 5+ messages in thread* Re: Problems with tg3 on BCM5720
2012-09-26 21:13 ` Nithin Nayak Sujir
@ 2012-09-27 6:45 ` Dirkjan Ochtman
2012-09-27 16:38 ` Nithin Sujir
0 siblings, 1 reply; 5+ messages in thread
From: Dirkjan Ochtman @ 2012-09-27 6:45 UTC (permalink / raw)
To: Nithin Nayak Sujir; +Cc: netdev
On Wed, Sep 26, 2012 at 11:13 PM, Nithin Nayak Sujir
<nsujir@broadcom.com> wrote:
> 1. Can you tell me the last patch that is included in the tg3 driver in
> 3.4.9 on your distro?
There are no tg3-specific patches in my distro's 3.4.9 package.
> 2. Can you give more info about the working setup?
The working setup is a simple small VLAN with a 192.168.1.0/24 subnet
and a few other Linux boxes on it (some of them also have BCM5720,
others have BCM5722 or BCM5709 networking). Not sure what other
information you'd want about this?
> 3. Was there any system reset or driver reload between the working and not
> working setups? Or was it just a cable switch?
Just a cable switch suffices to reproduce the problem we're seeing.
> 4. Please give the output of
> ethtool eth0
> ethtool -i eth0
> ethtool -k eth0
djc@jansky ~ $ sudo ethtool eth0
Settings for eth0:
Supported ports: [ TP ]
Supported link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Supported pause frame use: No
Supports auto-negotiation: Yes
Advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
1000baseT/Half 1000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: Yes
Link partner advertised link modes: 10baseT/Half 10baseT/Full
100baseT/Half 100baseT/Full
Link partner advertised pause frame use: No
Link partner advertised auto-negotiation: Yes
Speed: 100Mb/s
Duplex: Full
Port: Twisted Pair
PHYAD: 1
Transceiver: internal
Auto-negotiation: on
MDI-X: off
Supports Wake-on: g
Wake-on: d
Current message level: 0x000000ff (255)
drv probe link timer ifdown ifup rx_err tx_err
Link detected: yes
djc@jansky ~ $ sudo ethtool -i eth0
driver: tg3
version: 3.124
firmware-version: FFV7.2.14 bc 5720-v1.25
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: yes
supports-eeprom-access: yes
supports-register-dump: yes
djc@jansky ~ $ sudo ethtool -k eth0
Offload parameters for eth0:
rx-checksumming: on
tx-checksumming: on
scatter-gather: off
tcp-segmentation-offload: off
udp-fragmentation-offload: off
generic-segmentation-offload: off
generic-receive-offload: on
large-receive-offload: off
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: off
(Note that this is with my hacked up driver from the 3.6 tree, taken
from 185d4c8bf579322e1c2835d70729bc30f6f80f55, with
8d4057a938481351dc690fbe23e8c72af08d5890,
d3836f21b0af5513ef55701dd3f50b8c42e44c7a,
a1e8b307986ab27b7608f107aec71d3569650f46,
118008784965003307ea164370094c7d0810546e,
3f84749004925dd1e94025292fed5c76ce418516 reverted to make it compile
on 3.4.9.)
> 5. Can you run ethtool --test in the working setup?
Here's the ethtool --test result from eth1, which is currently plugged
into the VLAN (eth0 was plugged into it before; we also tried plugging
the external network into eth1, but that gave the same results as
plugging it into eth0).
djc@jansky ~ $ sudo ethtool --test eth1
The test result is PASS
The test extra info:
nvram test (online) 0
link test (online) 0
register test (offline) 0
memory test (offline) 0
mac loopback test (offline) 0
phy loopback test (offline) 0
ext loopback test (offline) 0
interrupt test (offline) 0
> 6. I noticed in the syslog, the link is coming up at 100 Mbps. Is this
> expected?
No, I don't think so, it should be a Gbit line.
> 7. Does it fail immediately on connect to the data center switch? Or is it
> after some traffic goes through?
The vendor whose switch we're connecting to says they see that the
link is up, but they don't see a MAC attached. One of the things that
look weird to me is the ifconfig output saying "RX packets:513
errors:0 dropped:0 overruns:0 frame:0" but also "TX packets:0 errors:0
dropped:0 overruns:0 carrier:0".
On Wed, Sep 26, 2012 at 11:40 PM, Michael Chan <mchan@broadcom.com> wrote:
> It is most likely that the device eth0 is down. The device needs to be
> up in order to perform all the tests that failed. Please bring up the
> device and run the test again. Thanks.
Right, sorry about that. Here's the results again, with the interface up:
djc@jansky ~ $ sudo ethtool --test eth0
The test result is FAIL
The test extra info:
nvram test (online) 0
link test (online) 0
register test (offline) 0
memory test (offline) 0
mac loopback test (offline) 0
phy loopback test (offline) 5
ext loopback test (offline) 0
interrupt test (offline) 0
Hope that helps,
Dirkjan
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Problems with tg3 on BCM5720
2012-09-27 6:45 ` Dirkjan Ochtman
@ 2012-09-27 16:38 ` Nithin Sujir
0 siblings, 0 replies; 5+ messages in thread
From: Nithin Sujir @ 2012-09-27 16:38 UTC (permalink / raw)
To: Dirkjan Ochtman; +Cc: netdev, Michael Chan
On Wednesday 26 September 2012 11:45 PM, Dirkjan Ochtman wrote:
> On Wed, Sep 26, 2012 at 11:13 PM, Nithin Nayak Sujir
> <nsujir@broadcom.com> wrote:
>> 1. Can you tell me the last patch that is included in the tg3 driver in
>> 3.4.9 on your distro?
> There are no tg3-specific patches in my distro's 3.4.9 package.
Ok. I was trying to make my setup here as close to your code to try and
see the problem. Would you say that the linux-stable tree at 3.4.9 has
the same tg3 code as your distro?
I'm not able to see a problem with 3.4.9 at gigabit or 100mbit on my
setup here.
>> 2. Can you give more info about the working setup?
> The working setup is a simple small VLAN with a 192.168.1.0/24 subnet
> and a few other Linux boxes on it (some of them also have BCM5720,
> others have BCM5722 or BCM5709 networking). Not sure what other
> information you'd want about this?
Asking more about the physical setup to see what is different with the
failing one w.r.t the working case. Is this a gigabit router/switch/hub?
>> 6. I noticed in the syslog, the link is coming up at 100 Mbps. Is this
>> expected?
> No, I don't think so, it should be a Gbit line.
Probably stupid question but have you have tried a different cable?
You mentioned that the switch side works with a laptop? What is the link
speed on the laptop at that time?
>
>> 7. Does it fail immediately on connect to the data center switch? Or is it
>> after some traffic goes through?
> The vendor whose switch we're connecting to says they see that the
> link is up, but they don't see a MAC attached. One of the things that
> look weird to me is the ifconfig output saying "RX packets:513
> errors:0 dropped:0 overruns:0 frame:0" but also "TX packets:0 errors:0
> dropped:0 overruns:0 carrier:0".
>
> On Wed, Sep 26, 2012 at 11:40 PM, Michael Chan <mchan@broadcom.com> wrote:
>> It is most likely that the device eth0 is down. The device needs to be
>> up in order to perform all the tests that failed. Please bring up the
>> device and run the test again. Thanks.
> Right, sorry about that. Here's the results again, with the interface up:
>
> djc@jansky ~ $ sudo ethtool --test eth0
> The test result is FAIL
> The test extra info:
> nvram test (online) 0
> link test (online) 0
> register test (offline) 0
> memory test (offline) 0
> mac loopback test (offline) 0
> phy loopback test (offline) 5
> ext loopback test (offline) 0
> interrupt test (offline) 0
>
> Hope that helps,
>
> Dirkjan
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Problems with tg3 on BCM5720
2012-09-26 13:48 Problems with tg3 on BCM5720 Dirkjan Ochtman
2012-09-26 21:13 ` Nithin Nayak Sujir
@ 2012-09-26 21:40 ` Michael Chan
1 sibling, 0 replies; 5+ messages in thread
From: Michael Chan @ 2012-09-26 21:40 UTC (permalink / raw)
To: Dirkjan Ochtman; +Cc: netdev
On Wed, 2012-09-26 at 15:48 +0200, Dirkjan Ochtman wrote:
> I've also run some tests with ethtool:
>
> djc@jansky ~ $ sudo ethtool --test eth0
> The test result is FAIL
> The test extra info:
> nvram test (online) 0
> link test (online) 1
> register test (offline) 0
> memory test (offline) 0
> mac loopback test (offline) 7
> phy loopback test (offline) 7
> ext loopback test (offline) 0
> interrupt test (offline) 1
It is most likely that the device eth0 is down. The device needs to be
up in order to perform all the tests that failed. Please bring up the
device and run the test again. Thanks.
>
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-09-27 16:39 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-09-26 13:48 Problems with tg3 on BCM5720 Dirkjan Ochtman
2012-09-26 21:13 ` Nithin Nayak Sujir
2012-09-27 6:45 ` Dirkjan Ochtman
2012-09-27 16:38 ` Nithin Sujir
2012-09-26 21:40 ` Michael Chan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.