From: Jason Wang <jasowang@redhat.com>
To: Jesper Dangaard Brouer <brouer@redhat.com>
Cc: sameehj@amazon.com, netdev@vger.kernel.org, bpf@vger.kernel.org,
zorik@amazon.com, akiyano@amazon.com, gtzalik@amazon.com,
"Toke Høiland-Jørgensen" <toke@redhat.com>,
"Daniel Borkmann" <borkmann@iogearbox.net>,
"Alexei Starovoitov" <alexei.starovoitov@gmail.com>,
"John Fastabend" <john.fastabend@gmail.com>,
"Alexander Duyck" <alexander.duyck@gmail.com>,
"Jeff Kirsher" <jeffrey.t.kirsher@intel.com>,
"David Ahern" <dsahern@gmail.com>,
"Willem de Bruijn" <willemdebruijn.kernel@gmail.com>,
"Ilias Apalodimas" <ilias.apalodimas@linaro.org>,
"Lorenzo Bianconi" <lorenzo@kernel.org>,
"Saeed Mahameed" <saeedm@mellanox.com>,
steffen.klassert@secunet.com
Subject: Re: [PATCH net-next 21/33] virtio_net: add XDP frame size in two code paths
Date: Wed, 6 May 2020 14:38:11 +0800 [thread overview]
Message-ID: <a197ea65-eb78-23aa-eab3-406f95edc199@redhat.com> (raw)
In-Reply-To: <d939445b-9713-2b2f-9830-38c2867bb963@redhat.com>
On 2020/4/28 下午5:50, Jason Wang wrote:
>
> On 2020/4/27 下午10:32, Jesper Dangaard Brouer wrote:
>> On Mon, 27 Apr 2020 15:21:02 +0800
>> Jason Wang<jasowang@redhat.com> wrote:
>>
>>> On 2020/4/23 上午12:09, Jesper Dangaard Brouer wrote:
>>>> The virtio_net driver is running inside the guest-OS. There are two
>>>> XDP receive code-paths in virtio_net, namely receive_small() and
>>>> receive_mergeable(). The receive_big() function does not support XDP.
>>>>
>>>> In receive_small() the frame size is available in buflen. The buffer
>>>> backing these frames are allocated in add_recvbuf_small() with same
>>>> size, except for the headroom, but tailroom have reserved room for
>>>> skb_shared_info. The headroom is encoded in ctx pointer as a value.
>>>>
>>>> In receive_mergeable() the frame size is more dynamic. There are two
>>>> basic cases: (1) buffer size is based on a exponentially weighted
>>>> moving average (see DECLARE_EWMA) of packet length. Or (2) in case
>>>> virtnet_get_headroom() have any headroom then buffer size is
>>>> PAGE_SIZE. The ctx pointer is this time used for encoding two values;
>>>> the buffer len "truesize" and headroom. In case (1) if the rx buffer
>>>> size is underestimated, the packet will have been split over more
>>>> buffers (num_buf info in virtio_net_hdr_mrg_rxbuf placed in top of
>>>> buffer area). If that happens the XDP path does a xdp_linearize_page
>>>> operation.
>>>>
>>>> Cc: Jason Wang<jasowang@redhat.com>
>>>> Signed-off-by: Jesper Dangaard Brouer<brouer@redhat.com>
>>>> ---
>>>> drivers/net/virtio_net.c | 15 ++++++++++++---
>>>> 1 file changed, 12 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
>>>> index 11f722460513..1df3676da185 100644
>>>> --- a/drivers/net/virtio_net.c
>>>> +++ b/drivers/net/virtio_net.c
>>>> @@ -689,6 +689,7 @@ static struct sk_buff *receive_small(struct
>>>> net_device *dev,
>>>> xdp.data_end = xdp.data + len;
>>>> xdp.data_meta = xdp.data;
>>>> xdp.rxq = &rq->xdp_rxq;
>>>> + xdp.frame_sz = buflen;
>>>> orig_data = xdp.data;
>>>> act = bpf_prog_run_xdp(xdp_prog, &xdp);
>>>> stats->xdp_packets++;
>>>> @@ -797,10 +798,11 @@ static struct sk_buff
>>>> *receive_mergeable(struct net_device *dev,
>>>> int offset = buf - page_address(page);
>>>> struct sk_buff *head_skb, *curr_skb;
>>>> struct bpf_prog *xdp_prog;
>>>> - unsigned int truesize;
>>>> + unsigned int truesize = mergeable_ctx_to_truesize(ctx);
>>>> unsigned int headroom = mergeable_ctx_to_headroom(ctx);
>>>> - int err;
>>>> unsigned int metasize = 0;
>>>> + unsigned int frame_sz;
>>>> + int err;
>>>> head_skb = NULL;
>>>> stats->bytes += len - vi->hdr_len;
>>>> @@ -821,6 +823,11 @@ static struct sk_buff
>>>> *receive_mergeable(struct net_device *dev,
>>>> if (unlikely(hdr->hdr.gso_type))
>>>> goto err_xdp;
>>>> + /* Buffers with headroom use PAGE_SIZE as alloc size,
>>>> + * see add_recvbuf_mergeable() + get_mergeable_buf_len()
>>>> + */
>>>> + frame_sz = headroom ? PAGE_SIZE : truesize;
>>>> +
>>>> /* This happens when rx buffer size is underestimated
>>>> * or headroom is not enough because of the buffer
>>>> * was refilled before XDP is set. This should only
>>>> @@ -834,6 +841,8 @@ static struct sk_buff *receive_mergeable(struct
>>>> net_device *dev,
>>>> page, offset,
>>>> VIRTIO_XDP_HEADROOM,
>>>> &len);
>>>> + frame_sz = PAGE_SIZE;
>>> Should this be PAGE_SIZE - SKB_DATA_ALIGN(sizeof(struct
>>> skb_shared_info))?
>> No, frame_sz include the SKB_DATA_ALIGN(sizeof(struct
>> skb_shared_info)) length.
>
>
> Ok, consider mergeable buffer path depends on the truesize which is
> encoded in ctx.
>
> It looks to the the calculation in add_recvfbuf_mergeable() is wrong,
> we need count both headroom and tailroom there.
>
> We probably need the attached 2 patches to fix this.
>
> (untested, will test it tomorrow).
Sorry for the late reply. I gave a test and post the attached two
patches (with minor tweaks).
It looks to me they are required for this patch to work since
data_hard_start excludes vnet hdr len without the attached patches which
means PAGE_SIZE could not be used as frame_sz.
Thanks
>
> Thanks
next prev parent reply other threads:[~2020-05-06 6:38 UTC|newest]
Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <158757160439.1370371.13213378122947426220.stgit@firesoul>
2020-04-22 16:07 ` [PATCH net-next 01/33] xdp: add frame size to xdp_buff Jesper Dangaard Brouer
2020-04-24 14:00 ` Toke Høiland-Jørgensen
2020-04-28 16:06 ` Jesper Dangaard Brouer
2020-04-22 16:07 ` [PATCH net-next 02/33] bnxt: add XDP frame size to driver Jesper Dangaard Brouer
2020-04-22 16:07 ` [PATCH net-next 03/33] sfc: add XDP frame size Jesper Dangaard Brouer
2020-04-22 16:07 ` [PATCH net-next 04/33] mvneta: add XDP frame size to driver Jesper Dangaard Brouer
2020-04-22 16:07 ` [PATCH net-next 05/33] net: netsec: Add support for XDP frame size Jesper Dangaard Brouer
2020-04-22 16:07 ` [PATCH net-next 06/33] net: XDP-generic determining " Jesper Dangaard Brouer
2020-04-24 14:03 ` Toke Høiland-Jørgensen
2020-04-22 16:07 ` [PATCH net-next 07/33] xdp: xdp_frame add member frame_sz and handle in convert_to_xdp_frame Jesper Dangaard Brouer
2020-04-24 14:04 ` Toke Høiland-Jørgensen
2020-04-25 3:24 ` Toshiaki Makita
2020-04-27 15:20 ` Jesper Dangaard Brouer
2020-04-22 16:08 ` [PATCH net-next 08/33] xdp: cpumap redirect use frame_sz and increase skb_tailroom Jesper Dangaard Brouer
2020-04-24 14:04 ` Toke Høiland-Jørgensen
2020-04-22 16:08 ` [PATCH net-next 09/33] veth: adjust hard_start offset on redirect XDP frames Jesper Dangaard Brouer
2020-04-24 14:05 ` Toke Høiland-Jørgensen
2020-04-22 16:08 ` [PATCH net-next 10/33] veth: xdp using frame_sz in veth driver Jesper Dangaard Brouer
2020-04-24 14:07 ` Toke Høiland-Jørgensen
2020-04-25 3:10 ` Toshiaki Makita
2020-04-22 16:08 ` [PATCH net-next 11/33] dpaa2-eth: add XDP frame size Jesper Dangaard Brouer
2020-04-22 16:08 ` [PATCH net-next 12/33] hv_netvsc: add XDP frame size to driver Jesper Dangaard Brouer
2020-04-22 16:57 ` Haiyang Zhang
2020-04-22 16:08 ` [PATCH net-next 13/33] qlogic/qede: " Jesper Dangaard Brouer
2020-04-22 16:08 ` [PATCH net-next 14/33] net: ethernet: ti: add XDP frame size to driver cpsw Jesper Dangaard Brouer
2020-04-22 20:28 ` Grygorii Strashko
2020-04-22 16:08 ` [PATCH net-next 15/33] ena: add XDP frame size to amazon NIC driver Jesper Dangaard Brouer
2020-04-22 16:08 ` [PATCH net-next 16/33] mlx4: add XDP frame size and adjust max XDP MTU Jesper Dangaard Brouer
2020-04-22 16:08 ` [PATCH net-next 17/33] net: thunderx: add XDP frame size Jesper Dangaard Brouer
2020-04-22 16:08 ` [PATCH net-next 18/33] nfp: add XDP frame size to netronome driver Jesper Dangaard Brouer
2020-04-23 2:43 ` Jakub Kicinski
2020-04-22 16:08 ` [PATCH net-next 19/33] tun: add XDP frame size Jesper Dangaard Brouer
2020-04-27 5:51 ` Jason Wang
2020-05-06 20:30 ` Michael S. Tsirkin
2020-04-22 16:09 ` [PATCH net-next 20/33] vhost_net: also populate " Jesper Dangaard Brouer
2020-04-27 5:50 ` Jason Wang
2020-04-30 9:54 ` Jesper Dangaard Brouer
2020-05-06 6:43 ` Jason Wang
2020-04-22 16:09 ` [PATCH net-next 21/33] virtio_net: add XDP frame size in two code paths Jesper Dangaard Brouer
2020-04-27 7:21 ` Jason Wang
2020-04-27 14:32 ` Jesper Dangaard Brouer
2020-04-28 9:50 ` Jason Wang
2020-04-30 10:14 ` Jesper Dangaard Brouer
2020-05-06 6:38 ` Jason Wang [this message]
2020-04-22 16:09 ` [PATCH net-next 22/33] ixgbe: fix XDP redirect on archs with PAGE_SIZE above 4K Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 23/33] ixgbe: add XDP frame size to driver Jesper Dangaard Brouer
2020-04-27 19:51 ` Daniel Borkmann
2020-04-22 16:09 ` [PATCH net-next 24/33] ixgbevf: add XDP frame size to VF driver Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 25/33] i40e: add XDP frame size to driver Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 26/33] ice: " Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 27/33] xdp: for Intel AF_XDP drivers add XDP frame_sz Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 28/33] mlx5: rx queue setup time determine frame_sz for XDP Jesper Dangaard Brouer
2020-04-25 0:58 ` Alexei Starovoitov
2020-04-27 20:22 ` Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 29/33] xdp: allow bpf_xdp_adjust_tail() to grow packet size Jesper Dangaard Brouer
2020-04-24 14:09 ` Toke Høiland-Jørgensen
2020-04-27 19:01 ` Daniel Borkmann
2020-04-28 16:37 ` Jesper Dangaard Brouer
2020-04-28 19:36 ` Daniel Borkmann
2020-04-22 16:09 ` [PATCH net-next 30/33] xdp: clear grow memory in bpf_xdp_adjust_tail() Jesper Dangaard Brouer
2020-04-24 14:09 ` Toke Høiland-Jørgensen
2020-04-27 5:26 ` John Fastabend
2020-04-28 14:50 ` Jesper Dangaard Brouer
2020-04-22 16:09 ` [PATCH net-next 31/33] bpf: add xdp.frame_sz in bpf_prog_test_run_xdp() Jesper Dangaard Brouer
2020-04-22 16:10 ` [PATCH net-next 32/33] selftests/bpf: adjust BPF selftest for xdp_adjust_tail Jesper Dangaard Brouer
2020-04-22 16:10 ` [PATCH net-next 33/33] selftests/bpf: xdp_adjust_tail add grow tail tests Jesper Dangaard Brouer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=a197ea65-eb78-23aa-eab3-406f95edc199@redhat.com \
--to=jasowang@redhat.com \
--cc=akiyano@amazon.com \
--cc=alexander.duyck@gmail.com \
--cc=alexei.starovoitov@gmail.com \
--cc=borkmann@iogearbox.net \
--cc=bpf@vger.kernel.org \
--cc=brouer@redhat.com \
--cc=dsahern@gmail.com \
--cc=gtzalik@amazon.com \
--cc=ilias.apalodimas@linaro.org \
--cc=jeffrey.t.kirsher@intel.com \
--cc=john.fastabend@gmail.com \
--cc=lorenzo@kernel.org \
--cc=netdev@vger.kernel.org \
--cc=saeedm@mellanox.com \
--cc=sameehj@amazon.com \
--cc=steffen.klassert@secunet.com \
--cc=toke@redhat.com \
--cc=willemdebruijn.kernel@gmail.com \
--cc=zorik@amazon.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).