From: Jesper Dangaard Brouer <brouer@redhat.com>
To: "Waskiewicz, PJ" <PJ.Waskiewicz@netapp.com>
Cc: "lsf@lists.linux-foundation.org" <lsf@lists.linux-foundation.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"bblanco@plumgrid.com" <bblanco@plumgrid.com>,
"alexei.starovoitov@gmail.com" <alexei.starovoitov@gmail.com>,
"James.Bottomley@HansenPartnership.com"
<James.Bottomley@HansenPartnership.com>,
"tom@herbertland.com" <tom@herbertland.com>,
"lsf-pc@lists.linux-foundation.org"
<lsf-pc@lists.linux-foundation.org>,
brouer@redhat.com
Subject: Re: [Lsf] [LSF/MM TOPIC] Generic page-pool recycle facility?
Date: Thu, 7 Apr 2016 22:38:53 +0200 [thread overview]
Message-ID: <20160407223853.6f4c7dbd@redhat.com> (raw)
In-Reply-To: <1460058531.13579.12.camel@netapp.com>
On Thu, 7 Apr 2016 19:48:50 +0000
"Waskiewicz, PJ" <PJ.Waskiewicz@netapp.com> wrote:
> On Thu, 2016-04-07 at 16:17 +0200, Jesper Dangaard Brouer wrote:
> > (Topic proposal for MM-summit)
> >
> > Network Interface Cards (NIC) drivers, and increasing speeds stress
> > the page-allocator (and DMA APIs). A number of driver specific
> > open-coded approaches exists that work-around these bottlenecks in
> > the
> > page allocator and DMA APIs. E.g. open-coded recycle mechanisms, and
> > allocating larger pages and handing-out page "fragments".
> >
> > I'm proposing a generic page-pool recycle facility, that can cover
> > the
> > driver use-cases, increase performance and open up for zero-copy RX.
>
> Is this based on the page recycle stuff from ixgbe that used to be in
> the driver? If so I'd really like to be part of the discussion.
Okay, so it is not part of the driver any-longer? I've studied the
current ixgbe driver (and other NIC drivers) closely. Do you have some
code pointers, to this older code?
The likely-fastest recycle code I've see is in the bnx2x driver. If
you are interested see: bnx2x_reuse_rx_data(). Again is it a bit
open-coded produce/consumer ring queue (which would be nice to also
cleanup).
To amortize the cost of allocating a single page, most other drivers
use the trick of allocating a larger (compound) page, and partition
this page into smaller "fragments". Which also amortize the cost of
dma_map/unmap (important on non-x86).
This is actually problematic performance wise, because packet-data
(in these page fragments) only get DMA_sync'ed, and is thus considered
"read-only". As netstack need to write packet headers, yet-another
(writable) memory area is allocated per packet (plus the SKB meta-data
struct).
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
Author of http://www.iptv-analyzer.org
LinkedIn: http://www.linkedin.com/in/brouer
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2016-04-07 20:38 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <1460034425.20949.7.camel@HansenPartnership.com>
2016-04-07 14:17 ` [LSF/MM TOPIC] Generic page-pool recycle facility? Jesper Dangaard Brouer
2016-04-07 14:38 ` [Lsf-pc] " Christoph Hellwig
2016-04-07 15:11 ` [Lsf] " Bart Van Assche
2016-04-10 18:45 ` Sagi Grimberg
2016-04-11 21:41 ` Jesper Dangaard Brouer
2016-04-11 22:02 ` Alexander Duyck
2016-04-12 6:28 ` Jesper Dangaard Brouer
2016-04-12 15:37 ` Alexander Duyck
2016-04-11 22:21 ` Alexei Starovoitov
2016-04-12 6:16 ` Jesper Dangaard Brouer
2016-04-12 17:20 ` Alexei Starovoitov
2016-04-07 15:48 ` Chuck Lever
2016-04-07 16:14 ` [Lsf-pc] [Lsf] " Rik van Riel
2016-04-07 19:43 ` [Lsf] [Lsf-pc] " Jesper Dangaard Brouer
2016-04-07 15:18 ` Eric Dumazet
2016-04-09 9:11 ` [Lsf] " Jesper Dangaard Brouer
2016-04-09 12:34 ` Eric Dumazet
2016-04-11 20:23 ` Jesper Dangaard Brouer
2016-04-11 21:27 ` Eric Dumazet
2016-04-07 19:48 ` Waskiewicz, PJ
2016-04-07 20:38 ` Jesper Dangaard Brouer [this message]
2016-04-08 16:12 ` Alexander Duyck
2016-04-11 8:58 ` [Lsf-pc] " Mel Gorman
2016-04-11 12:26 ` Jesper Dangaard Brouer
2016-04-11 13:08 ` Mel Gorman
2016-04-11 16:19 ` [Lsf] " Jesper Dangaard Brouer
2016-04-11 16:53 ` Eric Dumazet
2016-04-11 19:47 ` Jesper Dangaard Brouer
2016-04-11 21:14 ` Eric Dumazet
2016-04-11 18:07 ` Mel Gorman
2016-04-11 19:26 ` Jesper Dangaard Brouer
2016-04-11 16:20 ` Matthew Wilcox
2016-04-11 17:46 ` Thadeu Lima de Souza Cascardo
2016-04-11 18:37 ` Jesper Dangaard Brouer
2016-04-11 18:53 ` Bart Van Assche
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160407223853.6f4c7dbd@redhat.com \
--to=brouer@redhat.com \
--cc=James.Bottomley@HansenPartnership.com \
--cc=PJ.Waskiewicz@netapp.com \
--cc=alexei.starovoitov@gmail.com \
--cc=bblanco@plumgrid.com \
--cc=linux-mm@kvack.org \
--cc=lsf-pc@lists.linux-foundation.org \
--cc=lsf@lists.linux-foundation.org \
--cc=netdev@vger.kernel.org \
--cc=tom@herbertland.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).