Re: [EXT] Re: [PATCH net-next 3/5] net: mvpp2: cls: Use RSS contexts to handle RSS tables

[Matteo Croce <mcroce@redhat.com>]

On Sat, May 9, 2020 at 4:49 PM Russell King - ARM Linux admin
<linux@armlinux.org.uk> wrote:
>
> On Sat, May 09, 2020 at 02:51:05PM +0100, Russell King - ARM Linux admin wrote:
> > On Sat, May 09, 2020 at 03:14:05PM +0200, Matteo Croce wrote:
> > > On Sat, May 9, 2020 at 1:45 PM Russell King - ARM Linux admin
> > > <linux@armlinux.org.uk> wrote:
> > > >
> > > > On Sat, May 09, 2020 at 11:15:58AM +0000, Stefan Chulski wrote:
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Matteo Croce <mcroce@redhat.com>
> > > > > > Sent: Saturday, May 9, 2020 3:13 AM
> > > > > > To: David S . Miller <davem@davemloft.net>
> > > > > > Cc: Maxime Chevallier <maxime.chevallier@bootlin.com>; netdev
> > > > > > <netdev@vger.kernel.org>; LKML <linux-kernel@vger.kernel.org>; Antoine
> > > > > > Tenart <antoine.tenart@bootlin.com>; Thomas Petazzoni
> > > > > > <thomas.petazzoni@bootlin.com>; gregory.clement@bootlin.com;
> > > > > > miquel.raynal@bootlin.com; Nadav Haklai <nadavh@marvell.com>; Stefan
> > > > > > Chulski <stefanc@marvell.com>; Marcin Wojtas <mw@semihalf.com>; Linux
> > > > > > ARM <linux-arm-kernel@lists.infradead.org>; Russell King - ARM Linux admin
> > > > > > <linux@armlinux.org.uk>
> > > > > > Subject: [EXT] Re: [PATCH net-next 3/5] net: mvpp2: cls: Use RSS contexts to
> > > > > > handle RSS tables
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > What do you think about temporarily disabling it like this?
> > > > > >
> > > > > > --- a/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> > > > > > +++ b/drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c
> > > > > > @@ -5775,7 +5775,8 @@ static int mvpp2_port_probe(struct platform_device
> > > > > > *pdev,
> > > > > >                             NETIF_F_HW_VLAN_CTAG_FILTER;
> > > > > >
> > > > > >         if (mvpp22_rss_is_supported()) {
> > > > > > -               dev->hw_features |= NETIF_F_RXHASH;
> > > > > > +               if (port->phy_interface != PHY_INTERFACE_MODE_SGMII)
> > > > > > +                       dev->hw_features |= NETIF_F_RXHASH;
> > > > > >                 dev->features |= NETIF_F_NTUPLE;
> > > > > >         }
> > > > > >
> > > > > >
> > > > > > David, is this "workaround" too bad to get accepted?
> > > > >
> > > > > Not sure that RSS related to physical interface(SGMII), better just remove NETIF_F_RXHASH as "workaround".
> > > >
> > > > Hmm, I'm not sure this is the right way forward.  This patch has the
> > > > effect of disabling:
> > > >
> > > > d33ec4525007 ("net: mvpp2: add an RSS classification step for each flow")
> > > >
> > > > but the commit you're pointing at which caused the regression is:
> > > >
> > > > 895586d5dc32 ("net: mvpp2: cls: Use RSS contexts to handle RSS tables")
> > > >
> > > >
> > >
> > > Hi,
> > >
> > > When git bisect pointed to 895586d5dc32 ("net: mvpp2: cls: Use RSS
> > > contexts to handle RSS tables"), which was merged
> > > almost an year after d33ec4525007 ("net: mvpp2: add an RSS
> > > classification step for each flow"), so I assume that between these
> > > two commits either the feature was working or it was disable and we
> > > didn't notice
> > >
> > > Without knowing what was happening, which commit should my Fixes tag point to?
> >
> > Let me make sure that I get this clear:
> >
> > - Prior to 895586d5dc32, you can turn on and off rxhash without issue
> >   on any port.
> > - After 895586d5dc32, turning rxhash on eth2 prevents reception.
> >
> > Prior to 895586d5dc32, with rxhash on, it looks like hashing using
> > CRC32 is supported but only one context.  So, if it's possible to
> > enable rxhash on any port on the mcbin without 895586d5dc32, and the
> > port continues to work, I'd say the bug was introduced by
> > 895586d5dc32.
> >
> > Of course, that would be reinforced if there was a measurable
> > difference in performance due to rxhash on each port.
>
> I've just run this test, but I can detect no difference in performance
> with or without 895586d5dc32 on eth0 or eth2 on the mcbin (apart from
> eth2 stopping working with 895586d5dc32 applied.)  I tested this by
> reverting almost all changes to the mvpp2 driver between 5.6 and that
> commit.
>
> That's not too surprising; I'm using my cex7 platform with the Mellanox
> card in for one end of the 10G link, and that platform doesn't seem to
> be able to saturdate a 10G link - it only seems to manage around 4Gbps.
>
> --
> RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
> FTTC broadband for 0.8mile line in suburbia: sync at 10.2Mbps down 587kbps up
>

Well it depends on the traffic type. I used to generate 5k flows with
T-Rex and an Intel X710 card.
This way t-rex changes the UDP port of every packet:

root@macchiatobin:~# tcpdump -tnni eth0
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
IP 16.0.0.18.9874 > 48.0.0.81.2001: UDP, length 18
IP 16.0.0.248.56289 > 48.0.192.56.2001: UDP, length 18
IP 16.0.0.154.44965 > 48.0.128.26.2001: UDP, length 18
IP 16.0.0.23.31363 > 48.0.0.86.2001: UDP, length 18
IP 16.0.0.192.1674 > 48.0.192.63.2001: UDP, length 18
IP 16.0.0.155.62370 > 48.0.128.27.2001: UDP, length 18
IP 16.0.0.30.22126 > 48.0.0.93.2001: UDP, length 18
IP 16.0.0.195.51329 > 48.0.192.66.2001: UDP, length 18
IP 16.0.0.160.18323 > 48.0.128.32.2001: UDP, length 18
IP 16.0.0.199.55413 > 48.0.192.70.2001: UDP, length 18

And here RX hash gives a huge performance gain.

root@macchiatobin:~# utraf eth0
tx: 0 bps 0 pps rx: 425.5 Mbps 886.5 Kpps
tx: 0 bps 0 pps rx: 426.0 Mbps 887.6 Kpps
tx: 0 bps 0 pps rx: 425.3 Mbps 886.1 Kpps
tx: 0 bps 0 pps rx: 425.2 Mbps 885.8 Kpps
root@macchiatobin:~# ethtool -K eth0 rxhash on
root@macchiatobin:~# utraf eth0
tx: 0 bps 0 pps rx: 1595 Mbps 3323 Kpps
tx: 0 bps 0 pps rx: 1593 Mbps 3319 Kpps
tx: 0 bps 0 pps rx: 1595 Mbps 3323 Kpps
tx: 0 bps 0 pps rx: 1594 Mbps 3320 Kpps

utraf is just a tool which reads netlink statistics, packets are
dropped with a tc rule.

Regards,
-- 
Matteo Croce
per aspera ad upstream