Re: [dpdk-dev] testpmd / SR-IOV RX packets, but TX-errors

From: Bill Michalowski <bmichalo@redhat.com>
To: Lance Richardson <lance.richardson@broadcom.com>
Cc: dev@dpdk.org, users@dpdk.org
Subject: Re: [dpdk-dev] testpmd / SR-IOV RX packets, but TX-errors
Date: Tue, 15 Oct 2019 11:03:48 -0400	[thread overview]
Message-ID: <CAO8xzOsSTdZCEkOEGXX3QEw8L6uO3YUYVw+zZtA4da73hGBP1A@mail.gmail.com> (raw)
In-Reply-To: <CADyeNECfYNgA7B3ihnyCvVnRhEmN+JHqiTtHdD4hzk2=NSGu8w@mail.gmail.com>

Thank you for the suggestion Lance, but that didn't correct the problem.

That being said, I would think that testpmd in "io" mode would just forward
the packets in the SR-IOV case just as it successfully does with testpmd
across the PFs (non SR-IOV) - but I could be wrong.

On Tue, Oct 15, 2019 at 10:26 AM Lance Richardson <
lance.richardson@broadcom.com> wrote:

> Hi Bill,
>
> This is just a guess, but perhaps the transmit drops are due to MAC
> anti-spoofing checks?
>
> You could probably prove/disprove by making the packet source MAC
> match the transmit VF's MAC.
>
>     Lance
>
>
> On Tue, Oct 15, 2019 at 10:01 AM Bill Michalowski <bmichalo@redhat.com>
> wrote:
> >
> > I'm trying run testpmd with SR-IOV, specifically across two VFs.  I see
> > packets being received successfully by the application but they are not
> > being transmitted.  They are dropped with the 'TX-errors' counters
> > incrementing.
> >
> > However, if I generate the same traffic and the DUT isn't programmed with
> > SR-IOV, but with testpmd just across PFs, the packets are transmitted
> just
> > fine and I don't have any TX-errors.  I'm hoping someone can help me
> figure
> > out why the TX packets are being dropped in the SR-IOV case.
> >
> > testpmd is running in the default "io" mode.
> >
> > The DUT is a two port Intel xxv710 25G card.  To simplify things, I'm not
> > actually using a guest, just testpmd across VFs.
> >
> > *# rpm -qa | grep dpdk*
> > dpdk-18.11.2-1.el7.x86_64
> >
> > *# uname -r*
> > 3.10.0-1062.1.2.el7.x86_64
> >
> > *# cat /etc/redhat-release *
> > Red Hat Enterprise Linux Server release 7.7 (Maipo)
> >
> > PFs:
> >
> > #* ethtool -i p7p1*
> > driver: i40e
> > version: 2.10.19.30
> > firmware-version: 6.80 0x80003d17 18.8.9
> > expansion-rom-version:
> > bus-info: 0000:86:00.0
> >
> > *# ethtool -i p7p2*
> > driver: i40e
> > version: 2.10.19.30
> > firmware-version: 6.80 0x80003d17 18.8.9
> > expansion-rom-version:
> > bus-info: 0000:86:00.1
> >
> > Procedure:
> >
> > Create VFs:
> >
> > *#echo 1 > /sys/class/net/p7p1/device/sriov_numvfs*
> >
> > *#echo 1 > /sys/class/net/p7p2/device/sriov_numvfs*
> >
> > *# ip l*
> > 4: p7p1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> mode
> > DEFAULT group default qlen 1000
> >     link/ether 3c:fd:fe:b9:c8:a0 brd ff:ff:ff:ff:ff:ff
> >     vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust
> > off
> > 5: p7p2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
> mode
> > DEFAULT group default qlen 1000
> >     link/ether 3c:fd:fe:b9:c8:a1 brd ff:ff:ff:ff:ff:ff
> >     vf 0 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust
> > off
> > 13: p7p1_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT group default qlen 1000
> >     link/ether 6a:ff:a3:55:23:4e brd ff:ff:ff:ff:ff:ff
> > 14: p7p2_0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode
> > DEFAULT group default qlen 1000
> >     link/ether da:a1:1c:c4:6c:e7 brd ff:ff:ff:ff:ff:ff
> >
> > *# ethtool -i p7p1_0*
> > driver: iavf
> > version: 3.7.53
> > firmware-version: N/A
> > expansion-rom-version:
> > bus-info: 0000:86:02.0
> >
> > *# ethtool -i p7p2_0*
> > driver: iavf
> > version: 3.7.53
> > firmware-version: N/A
> > expansion-rom-version:
> > bus-info: 0000:86:0a.0
> >
> > Set MAC addresses:
> >
> > *#ip link set p7p1 vf 0 mac a0:b0:c0:d0:e0:f0*
> > *#ip link set p7p2 vf 0 mac 66:55:44:33:22:11*
> >
> > Set trust mode:
> >
> >
> >
> > *#ip link set p7p1 vf 0 trust on#ip link set p7p2 vf 0 trust on*
> >
> > Unbind iavf from VFs and bind vfio-pci to them:
> >
> > *# modprobe vfio-pci*
> > *# dpdk-devbind -u 0000:86:02.0 0000:86:0a.0*
> > *# dpdk-devbind -b vfio-pci 0000:86:02.0 0000:86:0a.0*
> > *# dpdk-devbind -s*
> >
> > Network devices using DPDK-compatible driver
> > ============================================
> > 0000:86:02.0 'Ethernet Virtual Function 700 Series 154c' drv=vfio-pci
> > unused=iavf
> > 0000:86:0a.0 'Ethernet Virtual Function 700 Series 154c' drv=vfio-pci
> > unused=iavf
> >
> > Start testpmd:
> >
> > #*/usr/bin/testpmd -l 1,3,35 --socket-mem 1024,1024 -n 4 --proc-type auto
> > --file-prefix testpmd0 -w 0000:86:02.0 -w 0000:86:0a.0 -- --nb-cores=2
> > --nb-ports=2 --portmask=3 --auto-start --rxq=1 --txq=1 --rxd=2048
> > --txd=2048 -i*
> >
> > Starting traffic at the far end, packets are RX successfully on the DUT
> but
> > not TX, with many TX-errors:
> >
> > *testpmd> show port stats all*
> >
> >   ######################## NIC statistics for port 0
> >  ########################
> >   RX-packets: 144132155  RX-missed: 0          RX-bytes:  9224457088
> >   RX-errors: 0
> >   RX-nombuf:  0
> >   TX-packets: 0          TX-errors: 137664400  TX-bytes:  0
> >
> >   Throughput (since last show)
> >   Rx-pps:     19000875
> >   Tx-pps:            0
> >
> >
> ############################################################################
> >
> >   ######################## NIC statistics for port 1
> >  ########################
> >   RX-packets: 144324307  RX-missed: 0          RX-bytes:  9236754880
> >   RX-errors: 0
> >   RX-nombuf:  0
> >   TX-packets: 0          TX-errors: 137873118  TX-bytes:  0
> >
> >   Throughput (since last show)
> >   Rx-pps:     19000772
> >   Tx-pps:            0
> >
> >
> ############################################################################
> >
> > Thanks for any help.
> >
> > - Bill
>