From: Lorenzo Bianconi <lorenzo@kernel.org>
To: Alexander Duyck <alexander.duyck@gmail.com>
Cc: Jesper Dangaard Brouer <jbrouer@redhat.com>,
brouer@redhat.com, Yunsheng Lin <linyunsheng@huawei.com>,
davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com,
netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
Jesper Dangaard Brouer <hawk@kernel.org>,
Ilias Apalodimas <ilias.apalodimas@linaro.org>,
Eric Dumazet <edumazet@google.com>,
Maryam Tahhan <mtahhan@redhat.com>, bpf <bpf@vger.kernel.org>
Subject: Re: [PATCH net-next v3 3/4] page_pool: introduce page_pool_alloc() API
Date: Sun, 18 Jun 2023 17:05:35 +0200 [thread overview]
Message-ID: <ZI8dP5+guKdR7IFE@lore-desk> (raw)
In-Reply-To: <CAKgT0UcNOYwxRP_zkaBaZh-VBL-CriL8dFG-VY7-FUyzxfHDWw@mail.gmail.com>
[-- Attachment #1: Type: text/plain, Size: 2610 bytes --]
[...]
> >
> > Yes, precisely.
> > I distinctly remember what I tried to poke you and Eric on this approach
> > earlier, but I cannot find a link to that email.
> >
> > I would really appreciate, if you Alex, could give the approach in
> > veth_convert_skb_to_xdp_buff() some review, as I believe that is a huge
> > potential for improvements that will lead to large performance
> > improvements. (I'm sure Maryam will be eager to help re-test performance
> > for her use-cases).
>
> Well just looking at it the quick and dirty answer would be to look at
> making use of something like page_frag_cache. I won't go into details
> since it isn't too different from the frag allocator, but it is much
> simpler since it is just doing reference count hacks instead of having
> to do the extra overhead to keep the DMA mapping in place. The veth
> would then just be sitting on at most an order 3 page while it is
> waiting to fully consume it rather than waiting on a full pool of
> pages.
Hi,
I did some experiments using page_frag_cache/page_frag_alloc() instead of
page_pools in a simple environment I used to test XDP for veth driver.
In particular, I allocate a new buffer in veth_convert_skb_to_xdp_buff() from
the page_frag_cache in order to copy the full skb in the new one, actually
"linearizing" the packet (since we know the original skb length).
I run an iperf TCP connection over a veth pair where the
remote device runs the xdp_rxq_info sample (available in the kernel source
tree, with action XDP_PASS):
TCP clietn -- v0 === v1 (xdp_rxq_info) -- TCP server
net-next (page_pool):
- MTU 1500B: ~ 7.5 Gbps
- MTU 8000B: ~ 15.3 Gbps
net-next + page_frag_alloc:
- MTU 1500B: ~ 8.4 Gbps
- MTU 8000B: ~ 14.7 Gbps
It seems there is no a clear "win" situation here (at least in this environment
and we this simple approach). Moreover:
- can the linearization introduce any issue whenever we perform XDP_REDIRECT
into a destination device?
- can the page_frag_cache introduce more memory fragmentation (IIRC we were
experiencing this issue in mt76 before switching to page_pools).
What do you think?
Regards,
Lorenzo
>
> Alternatively it could do something similar to page_frag_alloc_align
> itself and just bypass doing a custom allocator. If it went that route
> it could do something almost like a ring buffer and greatly improve
> the throughput since it would be able to allocate a higher order page
> and just copy the entire skb in so the entire thing would be linear
> rather than having to allocate a bunch of single pages.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]
next prev parent reply other threads:[~2023-06-18 15:05 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-09 13:17 [PATCH net-next v3 0/4] introduce page_pool_alloc() API Yunsheng Lin
[not found] ` <20230609131740.7496-4-linyunsheng@huawei.com>
[not found] ` <CAKgT0UfVwQ=ri7ZDNnsATH2RQpEz+zDBBb6YprvniMEWGdw+dQ@mail.gmail.com>
[not found] ` <36366741-8df2-1137-0dd9-d498d0f770e4@huawei.com>
[not found] ` <CAKgT0UdXTSv1fDHBX4UC6Ok9NXKMJ_9F88CEv5TK+mpzy0N21g@mail.gmail.com>
[not found] ` <c06f6f59-6c35-4944-8f7a-7f6f0e076649@huawei.com>
[not found] ` <CAKgT0UccmDe+CE6=zDYQHi1=3vXf5MptzDo+BsPrKdmP5j9kgQ@mail.gmail.com>
2023-06-15 16:19 ` [PATCH net-next v3 3/4] page_pool: " Jesper Dangaard Brouer
2023-06-16 11:57 ` Yunsheng Lin
2023-06-16 16:31 ` Jesper Dangaard Brouer
2023-06-16 17:34 ` Alexander Duyck
2023-06-16 18:41 ` Jesper Dangaard Brouer
2023-06-16 18:47 ` Jakub Kicinski
2023-06-16 19:50 ` Alexander Duyck
2023-06-18 15:05 ` Lorenzo Bianconi [this message]
2023-06-20 16:19 ` Alexander Duyck
2023-06-20 21:16 ` Lorenzo Bianconi
2023-06-21 11:55 ` Jesper Dangaard Brouer
2023-06-24 14:44 ` Yunsheng Lin
2023-06-17 12:47 ` Yunsheng Lin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZI8dP5+guKdR7IFE@lore-desk \
--to=lorenzo@kernel.org \
--cc=alexander.duyck@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=brouer@redhat.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=hawk@kernel.org \
--cc=ilias.apalodimas@linaro.org \
--cc=jbrouer@redhat.com \
--cc=kuba@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linyunsheng@huawei.com \
--cc=mtahhan@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).