* question: tg3 driver/nics and inconsistent RX ring count @ 2016-07-26 19:52 Michal Soltys 2016-07-26 20:06 ` Alexander Duyck 0 siblings, 1 reply; 5+ messages in thread From: Michal Soltys @ 2016-07-26 19:52 UTC (permalink / raw) To: Linux Netdev List Hi, I have a few of BCM5720 and BCM5719 kinds sitting in Dell R320 and R520 servers - and all of them have certain peculiarity: they claim to have up to 4 TX and RX rings (and this can be set/verified just fine through ethtool -l/-L, with driver defaulting to 4 rings), indirection table (ethtool -x) looks fine as well: RX flow hash indirection table for eth1b with 3 RX ring(s): 0: 0 1 2 3 0 1 2 3 8: 0 1 2 3 0 1 2 3 ...... But this "3 RX ring(s)" is actually a real limit if I try to adjust anything, for example all those commands would fail: ethtool -X eth1b equal 4 ethtool -X eth1b weight 1 1 1 1 But would work fine for 3 and less rings. This was quickly tested with different kernel/ethtool combinations, from old 3.16 to 4.6.y with same exact results. Nothing fancier (-N/-U) is supported either. Any hints/comments about the cause of this and/or possible workarounds ? ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: question: tg3 driver/nics and inconsistent RX ring count 2016-07-26 19:52 question: tg3 driver/nics and inconsistent RX ring count Michal Soltys @ 2016-07-26 20:06 ` Alexander Duyck 2016-07-26 20:32 ` Michal Soltys 0 siblings, 1 reply; 5+ messages in thread From: Alexander Duyck @ 2016-07-26 20:06 UTC (permalink / raw) To: Michal Soltys; +Cc: Linux Netdev List On Tue, Jul 26, 2016 at 12:52 PM, Michal Soltys <soltys@ziu.info> wrote: > Hi, > > I have a few of BCM5720 and BCM5719 kinds sitting in Dell R320 and R520 > servers - and all of them have certain peculiarity: they claim to have > up to 4 TX and RX rings (and this can be set/verified just fine through > ethtool -l/-L, with driver defaulting to 4 rings), indirection table > (ethtool -x) looks fine as well: > > RX flow hash indirection table for eth1b with 3 RX ring(s): > 0: 0 1 2 3 0 1 2 3 > 8: 0 1 2 3 0 1 2 3 > ...... > > But this "3 RX ring(s)" is actually a real limit if I try to adjust > anything, for example all those commands would fail: > > ethtool -X eth1b equal 4 > ethtool -X eth1b weight 1 1 1 1 > > But would work fine for 3 and less rings. This was quickly tested with > different kernel/ethtool combinations, from old 3.16 to 4.6.y with same > exact results. Nothing fancier (-N/-U) is supported either. > > Any hints/comments about the cause of this and/or possible workarounds ? Well a quick look at the driver code makes it seem the problem lies in tg3_get_rxnfc. I suspect the bug is related to the following lines: /* The first interrupt vector only * handles link interrupts. */ info->data -= 1; I'm not sure what the number of interrupt vectors has to do with the number of rings. Perhaps someone more familiar with the driver can point out why you would subtract 1 from tp->rxq_cnt to get the number of queues when it seems like it should be tp->rxq_cnt. Hope that helps. - Alex ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: question: tg3 driver/nics and inconsistent RX ring count 2016-07-26 20:06 ` Alexander Duyck @ 2016-07-26 20:32 ` Michal Soltys 2016-07-26 22:55 ` Michael Chan [not found] ` <CAApE=TPAZA8Y-buM=9VMaGDTV5gLr-UuremMx3rmYNLiOUFihQ@mail.gmail.com> 0 siblings, 2 replies; 5+ messages in thread From: Michal Soltys @ 2016-07-26 20:32 UTC (permalink / raw) To: Alexander Duyck; +Cc: Linux Netdev List, mcarlson On 2016-07-26 22:06, Alexander Duyck wrote: > On Tue, Jul 26, 2016 at 12:52 PM, Michal Soltys <soltys@ziu.info> wrote: >> Hi, >> >> I have a few of BCM5720 and BCM5719 kinds sitting in Dell R320 and R520 >> servers - and all of them have certain peculiarity: they claim to have >> up to 4 TX and RX rings (and this can be set/verified just fine through >> ethtool -l/-L, with driver defaulting to 4 rings), indirection table >> (ethtool -x) looks fine as well: >> >> RX flow hash indirection table for eth1b with 3 RX ring(s): >> 0: 0 1 2 3 0 1 2 3 >> 8: 0 1 2 3 0 1 2 3 >> ...... >> >> But this "3 RX ring(s)" is actually a real limit if I try to adjust >> anything, for example all those commands would fail: >> >> ethtool -X eth1b equal 4 >> ethtool -X eth1b weight 1 1 1 1 >> >> But would work fine for 3 and less rings. This was quickly tested with >> different kernel/ethtool combinations, from old 3.16 to 4.6.y with same >> exact results. Nothing fancier (-N/-U) is supported either. >> >> Any hints/comments about the cause of this and/or possible workarounds ? > > Well a quick look at the driver code makes it seem the problem lies in > tg3_get_rxnfc. I suspect the bug is related to the following lines: > > /* The first interrupt vector only > * handles link interrupts. > */ > info->data -= 1; > > I'm not sure what the number of interrupt vectors has to do with the > number of rings. Perhaps someone more familiar with the driver can > point out why you would subtract 1 from tp->rxq_cnt to get the number > of queues when it seems like it should be tp->rxq_cnt. > > Hope that helps. > > - Alex > Ah thanks, seems to be the case then. Quick git blame shows it's been since the very introduction of RSS indirection configurability (ca. 2011) in this driver, 90415477bf1356f72acc34063ff52441fc10a754. I've CCed the author, maybe he will be able to shed some light. ^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: question: tg3 driver/nics and inconsistent RX ring count 2016-07-26 20:32 ` Michal Soltys @ 2016-07-26 22:55 ` Michael Chan [not found] ` <CAApE=TPAZA8Y-buM=9VMaGDTV5gLr-UuremMx3rmYNLiOUFihQ@mail.gmail.com> 1 sibling, 0 replies; 5+ messages in thread From: Michael Chan @ 2016-07-26 22:55 UTC (permalink / raw) To: Michal Soltys, Siva Reddy Kallam Cc: Alexander Duyck, Linux Netdev List, mcarlson On Tue, Jul 26, 2016 at 1:32 PM, Michal Soltys <soltys@ziu.info> wrote: > On 2016-07-26 22:06, Alexander Duyck wrote: >> On Tue, Jul 26, 2016 at 12:52 PM, Michal Soltys <soltys@ziu.info> wrote: >>> Hi, >>> >>> I have a few of BCM5720 and BCM5719 kinds sitting in Dell R320 and R520 >>> servers - and all of them have certain peculiarity: they claim to have >>> up to 4 TX and RX rings (and this can be set/verified just fine through >>> ethtool -l/-L, with driver defaulting to 4 rings), indirection table >>> (ethtool -x) looks fine as well: >>> >>> RX flow hash indirection table for eth1b with 3 RX ring(s): >>> 0: 0 1 2 3 0 1 2 3 >>> 8: 0 1 2 3 0 1 2 3 >>> ...... >>> >>> But this "3 RX ring(s)" is actually a real limit if I try to adjust >>> anything, for example all those commands would fail: >>> >>> ethtool -X eth1b equal 4 >>> ethtool -X eth1b weight 1 1 1 1 >>> >>> But would work fine for 3 and less rings. This was quickly tested with >>> different kernel/ethtool combinations, from old 3.16 to 4.6.y with same >>> exact results. Nothing fancier (-N/-U) is supported either. >>> >>> Any hints/comments about the cause of this and/or possible workarounds ? >> >> Well a quick look at the driver code makes it seem the problem lies in >> tg3_get_rxnfc. I suspect the bug is related to the following lines: >> >> /* The first interrupt vector only >> * handles link interrupts. >> */ >> info->data -= 1; >> >> I'm not sure what the number of interrupt vectors has to do with the >> number of rings. Perhaps someone more familiar with the driver can >> point out why you would subtract 1 from tp->rxq_cnt to get the number >> of queues when it seems like it should be tp->rxq_cnt. >> >> Hope that helps. >> >> - Alex >> > > Ah thanks, seems to be the case then. Quick git blame shows it's been > since the very introduction of RSS indirection configurability (ca. > 2011) in this driver, 90415477bf1356f72acc34063ff52441fc10a754. > > I've CCed the author, maybe he will be able to shed some light. Matt is no longer working here. The driver should support up to 5 MSIX vectors and 4 RSS rings. It looks like the code to subtract 1 in tg3_get_rxnfc() is not correct. Siva will look further into this. Thanks for reporting the issue. ^ permalink raw reply [flat|nested] 5+ messages in thread
[parent not found: <CAApE=TPAZA8Y-buM=9VMaGDTV5gLr-UuremMx3rmYNLiOUFihQ@mail.gmail.com>]
[parent not found: <CACKFLinjzVHWcDuYyr5P5oW=muYfkmwO_=BFkNLBZTi2Prq-Ew@mail.gmail.com>]
[parent not found: <CAApE=TPeRLDEev5XGmeq61b=khZVq_PDaf7RtLvLT=K3y55Ujw@mail.gmail.com>]
[parent not found: <CAMet4B4o4PMbPxNaf=6BB=70zfVpM1DY3fb-StJvi0BWz_dutg@mail.gmail.com>]
* Re: question: tg3 driver/nics and inconsistent RX ring count [not found] ` <CAMet4B4o4PMbPxNaf=6BB=70zfVpM1DY3fb-StJvi0BWz_dutg@mail.gmail.com> @ 2016-07-29 10:37 ` Siva Reddy Kallam 0 siblings, 0 replies; 5+ messages in thread From: Siva Reddy Kallam @ 2016-07-29 10:37 UTC (permalink / raw) To: Michael Chan; +Cc: Michal Soltys, Alexander Duyck, Linux Netdev List, mcarlson On Wed, Jul 27, 2016 at 4:25 AM, Michael Chan <michael.chan@broadcom.com> wrote: > On Tue, Jul 26, 2016 at 1:32 PM, Michal Soltys <soltys@ziu.info> wrote: >> On 2016-07-26 22:06, Alexander Duyck wrote: >>> On Tue, Jul 26, 2016 at 12:52 PM, Michal Soltys <soltys@ziu.info> wrote: >>>> Hi, >>>> >>>> I have a few of BCM5720 and BCM5719 kinds sitting in Dell R320 and R520 >>>> servers - and all of them have certain peculiarity: they claim to have >>>> up to 4 TX and RX rings (and this can be set/verified just fine through >>>> ethtool -l/-L, with driver defaulting to 4 rings), indirection table >>>> (ethtool -x) looks fine as well: >>>> >>>> RX flow hash indirection table for eth1b with 3 RX ring(s): >>>> 0: 0 1 2 3 0 1 2 3 >>>> 8: 0 1 2 3 0 1 2 3 >>>> ...... >>>> >>>> But this "3 RX ring(s)" is actually a real limit if I try to adjust >>>> anything, for example all those commands would fail: >>>> >>>> ethtool -X eth1b equal 4 >>>> ethtool -X eth1b weight 1 1 1 1 >>>> >>>> But would work fine for 3 and less rings. This was quickly tested with >>>> different kernel/ethtool combinations, from old 3.16 to 4.6.y with same >>>> exact results. Nothing fancier (-N/-U) is supported either. >>>> >>>> Any hints/comments about the cause of this and/or possible workarounds ? >>> >>> Well a quick look at the driver code makes it seem the problem lies in >>> tg3_get_rxnfc. I suspect the bug is related to the following lines: >>> >>> /* The first interrupt vector only >>> * handles link interrupts. >>> */ >>> info->data -= 1; >>> >>> I'm not sure what the number of interrupt vectors has to do with the >>> number of rings. Perhaps someone more familiar with the driver can >>> point out why you would subtract 1 from tp->rxq_cnt to get the number >>> of queues when it seems like it should be tp->rxq_cnt. >>> >>> Hope that helps. >>> >>> - Alex >>> >> >> Ah thanks, seems to be the case then. Quick git blame shows it's been >> since the very introduction of RSS indirection configurability (ca. >> 2011) in this driver, 90415477bf1356f72acc34063ff52441fc10a754. >> >> I've CCed the author, maybe he will be able to shed some light. > > Matt is no longer working here. The driver should support up to 5 > MSIX vectors and 4 RSS rings. It looks like the code to subtract 1 in > tg3_get_rxnfc() is not correct. Siva will look further into this. > Thanks for reporting the issue. Yes, the code to subtract 1 in tg3_get_rxnfc looks incorrect. we will upstream the patch for removing this code. ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-07-29 10:37 UTC | newest] Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2016-07-26 19:52 question: tg3 driver/nics and inconsistent RX ring count Michal Soltys 2016-07-26 20:06 ` Alexander Duyck 2016-07-26 20:32 ` Michal Soltys 2016-07-26 22:55 ` Michael Chan [not found] ` <CAApE=TPAZA8Y-buM=9VMaGDTV5gLr-UuremMx3rmYNLiOUFihQ@mail.gmail.com> [not found] ` <CACKFLinjzVHWcDuYyr5P5oW=muYfkmwO_=BFkNLBZTi2Prq-Ew@mail.gmail.com> [not found] ` <CAApE=TPeRLDEev5XGmeq61b=khZVq_PDaf7RtLvLT=K3y55Ujw@mail.gmail.com> [not found] ` <CAMet4B4o4PMbPxNaf=6BB=70zfVpM1DY3fb-StJvi0BWz_dutg@mail.gmail.com> 2016-07-29 10:37 ` Siva Reddy Kallam
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.