Re: [PATCH bpf-next v2 0/3] xsk: build skb by page

["Michael S. Tsirkin" <mst@redhat.com>]

On Tue, Jan 19, 2021 at 06:53:29PM +0800, Xuan Zhuo wrote:
> On Tue, 19 Jan 2021 04:50:30 -0500, Michael S. Tsirkin <mst@redhat.com> wrote:
> > On Tue, Jan 19, 2021 at 05:45:09PM +0800, Xuan Zhuo wrote:
> > > v2:
> > >     1. add priv_flags IFF_TX_SKB_NO_LINEAR instead of netdev_feature
> > >     2. split the patch to three:
> > >         a. add priv_flags IFF_TX_SKB_NO_LINEAR
> > >         b. virtio net add priv_flags IFF_TX_SKB_NO_LINEAR
> > >         c. When there is support this flag, construct skb without linear space
> > >     3. use ERR_PTR() and PTR_ERR() to handle the err
> > >
> > >
> > > v1 message log:
> > > ---------------
> > >
> > > This patch is used to construct skb based on page to save memory copy
> > > overhead.
> > >
> > > This has one problem:
> > >
> > > We construct the skb by fill the data page as a frag into the skb. In
> > > this way, the linear space is empty, and the header information is also
> > > in the frag, not in the linear space, which is not allowed for some
> > > network cards. For example, Mellanox Technologies MT27710 Family
> > > [ConnectX-4 Lx] will get the following error message:
> > >
> > >     mlx5_core 0000:3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb, opcode 0xd, syndrome 0x1, vendor syndrome 0x68
> > >     00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > >     00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > >     00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > >     00000030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2
> > >     WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64
> > >     00000000: 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00
> > >     00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > >     00000020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00
> > >     00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> > >     mlx5_core 0000:3b:00.1 eth1: ERR CQE on SQ: 0x1dbb
> > >
> > > I also tried to use build_skb to construct skb, but because of the
> > > existence of skb_shinfo, it must be behind the linear space, so this
> > > method is not working. We can't put skb_shinfo on desc->addr, it will be
> > > exposed to users, this is not safe.
> > >
> > > Finally, I added a feature NETIF_F_SKB_NO_LINEAR to identify whether the
> > > network card supports the header information of the packet in the frag
> > > and not in the linear space.
> > >
> > > ---------------- Performance Testing ------------
> > >
> > > The test environment is Aliyun ECS server.
> > > Test cmd:
> > > ```
> > > xdpsock -i eth0 -t  -S -s <msg size>
> > > ```
> > >
> > > Test result data:
> > >
> > > size    64      512     1024    1500
> > > copy    1916747 1775988 1600203 1440054
> > > page    1974058 1953655 1945463 1904478
> > > percent 3.0%    10.0%   21.58%  32.3%
> >
> > Just making sure, are these test results with v2?
> 
> The data was tested at v1,
> but v2 did not modify the performance-related code.
> 
> Thanks.

Looks like v1 wouldn't even build, or did I miss anything?
It would be nicer if you retested it ...

> 
> >
> > >
> > > Xuan Zhuo (3):
> > >   net: add priv_flags for allow tx skb without linear
> > >   virtio-net: support IFF_TX_SKB_NO_LINEAR
> > >   xsk: build skb by page
> > >
> > >  drivers/net/virtio_net.c  |   3 +-
> > >  include/linux/netdevice.h |   3 ++
> > >  net/xdp/xsk.c             | 112 ++++++++++++++++++++++++++++++++++++++--------
> > >  3 files changed, 99 insertions(+), 19 deletions(-)
> > >
> > > --
> > > 1.8.3.1
> >