From: Magnus Karlsson <magnus.karlsson@gmail.com>
To: Alexei Starovoitov <alexei.starovoitov@gmail.com>
Cc: "Kevin Laatz" <kevin.laatz@intel.com>,
"Jakub Kicinski" <jakub.kicinski@netronome.com>,
"Daniel Borkmann" <daniel@iogearbox.net>,
"Network Development" <netdev@vger.kernel.org>,
ciara.loftus@intel.com, "Alexei Starovoitov" <ast@kernel.org>,
intel-wired-lan <intel-wired-lan@lists.osuosl.org>,
"Jonathan Lemon" <jonathan.lemon@gmail.com>,
bruce.richardson@intel.com, bpf <bpf@vger.kernel.org>,
"Björn Töpel" <bjorn.topel@intel.com>,
"Karlsson, Magnus" <magnus.karlsson@intel.com>
Subject: Re: [Intel-wired-lan] [PATCH v2 00/10] XDP unaligned chunk placement support
Date: Wed, 24 Jul 2019 15:25:32 +0200 [thread overview]
Message-ID: <CAJ8uoz26Byzhc4My70h3nmfd3pQB1ahFf=ZzN2heN4asrJ-NQg@mail.gmail.com> (raw)
In-Reply-To: <CAADnVQLEdJwT7DRCpp2zuKWTg0uj=WKQkFc2LZ4o+1fDgnEFLg@mail.gmail.com>
On Tue, Jul 23, 2019 at 11:08 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> Johnathan, Bjorn, Jakub,
> Please review!
> The patch set has been pending for a week.
There is a v3 coming out shortly so I suggest we wait for that. It
will have Mellanox support for this feature too and some clean ups. I
refrained from posting a review on the mailing list due to the merge
window being closed last week, but maybe that was not correct. Should
I still post reviews for new features submitted during the closed
merge window period? I am happy to do it since I do not have any other
tasks during that time. It is a quite period for me. Just let me know.
/Magnus
> On Tue, Jul 16, 2019 at 4:21 AM Kevin Laatz <kevin.laatz@intel.com> wrote:
> >
> > This patch set adds the ability to use unaligned chunks in the XDP umem.
> >
> > Currently, all chunk addresses passed to the umem are masked to be chunk
> > size aligned (default is 2k, max is PAGE_SIZE). This limits where we can
> > place chunks within the umem as well as limiting the packet sizes that are
> > supported.
> >
> > The changes in this patch set removes these restrictions, allowing XDP to
> > be more flexible in where it can place a chunk within a umem. By relaxing
> > where the chunks can be placed, it allows us to use an arbitrary buffer
> > size and place that wherever we have a free address in the umem. These
> > changes add the ability to support arbitrary frame sizes up to 4k
> > (PAGE_SIZE) and make it easy to integrate with other existing frameworks
> > that have their own memory management systems, such as DPDK.
> >
> > Since we are now dealing with arbitrary frame sizes, we need also need to
> > update how we pass around addresses. Currently, the addresses can simply be
> > masked to 2k to get back to the original address. This becomes less trivial
> > when using frame sizes that are not a 'power of 2' size. This patch set
> > modifies the Rx/Tx descriptor format to use the upper 16-bits of the addr
> > field for an offset value, leaving the lower 48-bits for the address (this
> > leaves us with 256 Terabytes, which should be enough!). We only need to use
> > the upper 16-bits to store the offset when running in unaligned mode.
> > Rather than adding the offset (headroom etc) to the address, we will store
> > it in the upper 16-bits of the address field. This way, we can easily add
> > the offset to the address where we need it, using some bit manipulation and
> > addition, and we can also easily get the original address wherever we need
> > it (for example in i40e_zca_free) by simply masking to get the lower
> > 48-bits of the address field.
> >
> > The numbers below were recorded with the following set up:
> > - Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz
> > - Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
> > - Driver: i40e
> > - Application: xdpsock with l2fwd (single interface)
> >
> > These are solely for comparing performance with and without the patches.
> > The largest drop was ~1% (in zero-copy mode).
> >
> > +-------------------------+------------+-----------------+-------------+
> > | Buffer size: 2048 | SKB mode | Zero-copy | Copy |
> > +-------------------------+------------+-----------------+-------------+
> > | Aligned (baseline) | 1.7 Mpps | 15.3 Mpps | 2.08 Mpps |
> > +-------------------------+------------+-----------------+-------------+
> > | Aligned (with patches) | 1.7 Mpps | 15.1 Mpps | 2.08 Mpps |
> > +-------------------------+------------+-----------------+-------------+
> > | Unaligned | 1.7 Mpps | 14.5 Mpps | 2.08 Mpps |
> > +-------------------------+------------+-----------------+-------------+
> >
> > NOTE: We are currently working on the changes required in the Mellanox
> > driver. We will include these in the v3.
> >
> > Structure of the patchset:
> > Patch 1:
> > - Remove unnecessary masking and headroom addition during zero-copy Rx
> > buffer recycling in i40e. This change is required in order for the
> > buffer recycling to work in the unaligned chunk mode.
> >
> > Patch 2:
> > - Remove unnecessary masking and headroom addition during
> > zero-copy Rx buffer recycling in ixgbe. This change is required in
> > order for the buffer recycling to work in the unaligned chunk mode.
> >
> > Patch 3:
> > - Add infrastructure for unaligned chunks. Since we are dealing with
> > unaligned chunks that could potentially cross a physical page boundary,
> > we add checks to keep track of that information. We can later use this
> > information to correctly handle buffers that are placed at an address
> > where they cross a page boundary. This patch also modifies the
> > existing Rx and Tx functions to use the new descriptor format. To
> > handle addresses correctly, we need to mask appropriately based on
> > whether we are in aligned or unaligned mode.
> >
> > Patch 4:
> > - This patch updates the i40e driver to make use of the new descriptor
> > format. The new format is particularly useful here since we can now
> > retrieve the original address in places like i40e_zca_free with ease.
> > This saves us doing various calculations to get the original address
> > back.
> >
> > Patch 5:
> > - This patch updates the ixgbe driver to make use of the new descriptor
> > format. The new format is particularly useful here since we can now
> > retrieve the original address in places like ixgbe_zca_free with ease.
> > This saves us doing various calculations to get the original address
> > back.
> >
> > Patch 6:
> > - Add flags for umem configuration to libbpf
> >
> > Patch 7:
> > - Modify xdpsock application to add a command line option for
> > unaligned chunks
> >
> > Patch 8:
> > - Since we can now run the application in unaligned chunk mode, we need
> > to make sure we recycle the buffers appropriately.
> >
> > Patch 9:
> > - Adds hugepage support to the xdpsock application
> >
> > Patch 10:
> > - Documentation update to include the unaligned chunk scenario. We need
> > to explicitly state that the incoming addresses are only masked in the
> > aligned chunk mode and not the unaligned chunk mode.
> >
> > ---
> > v2:
> > - fixed checkpatch issues
> > - fixed Rx buffer recycling for unaligned chunks in xdpsock
> > - removed unused defines
> > - fixed how chunk_size is calculated in xsk_diag.c
> > - added some performance numbers to cover letter
> > - modified descriptor format to make it easier to retrieve original
> > address
> > - removed patch adding off_t off to the zero copy allocator. This is no
> > longer needed with the new descriptor format.
> >
> > Kevin Laatz (10):
> > i40e: simplify Rx buffer recycle
> > ixgbe: simplify Rx buffer recycle
> > xsk: add support to allow unaligned chunk placement
> > i40e: modify driver for handling offsets
> > ixgbe: modify driver for handling offsets
> > libbpf: add flags to umem config
> > samples/bpf: add unaligned chunks mode support to xdpsock
> > samples/bpf: add buffer recycling for unaligned chunks to xdpsock
> > samples/bpf: use hugepages in xdpsock app
> > doc/af_xdp: include unaligned chunk case
> >
> > Documentation/networking/af_xdp.rst | 10 ++-
> > drivers/net/ethernet/intel/i40e/i40e_xsk.c | 39 +++++----
> > drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c | 39 +++++----
> > include/net/xdp_sock.h | 2 +
> > include/uapi/linux/if_xdp.h | 9 ++
> > net/xdp/xdp_umem.c | 17 ++--
> > net/xdp/xsk.c | 89 ++++++++++++++++----
> > net/xdp/xsk_diag.c | 2 +-
> > net/xdp/xsk_queue.h | 70 +++++++++++++--
> > samples/bpf/xdpsock_user.c | 61 ++++++++++----
> > tools/include/uapi/linux/if_xdp.h | 4 +
> > tools/lib/bpf/xsk.c | 3 +
> > tools/lib/bpf/xsk.h | 2 +
> > 13 files changed, 266 insertions(+), 81 deletions(-)
> >
> > --
> > 2.17.1
> >
> _______________________________________________
> Intel-wired-lan mailing list
> Intel-wired-lan@osuosl.org
> https://lists.osuosl.org/mailman/listinfo/intel-wired-lan
next prev parent reply other threads:[~2019-07-24 13:25 UTC|newest]
Thread overview: 136+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-20 9:09 [PATCH 00/11] XDP unaligned chunk placement support Kevin Laatz
2019-06-20 9:09 ` [PATCH 01/11] i40e: simplify Rx buffer recycle Kevin Laatz
2019-06-24 14:29 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 02/11] ixgbe: " Kevin Laatz
2019-06-24 14:30 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 03/11] xdp: add offset param to zero_copy_allocator Kevin Laatz
2019-06-24 14:31 ` Björn Töpel
2019-06-24 19:23 ` Jakub Kicinski
2019-06-25 13:14 ` Laatz, Kevin
2019-06-20 9:09 ` [PATCH 04/11] i40e: add offset to zca_free Kevin Laatz
2019-06-24 14:32 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 05/11] ixgbe: " Kevin Laatz
2019-06-24 14:32 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 06/11] xsk: add support to allow unaligned chunk placement Kevin Laatz
2019-06-24 15:29 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 07/11] libbpf: add flags to umem config Kevin Laatz
2019-06-24 15:30 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 08/11] samples/bpf: add unaligned chunks mode support to xdpsock Kevin Laatz
2019-06-24 15:31 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 09/11] samples/bpf: add buffer recycling for unaligned chunks " Kevin Laatz
2019-06-24 15:35 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 10/11] samples/bpf: use hugepages in xdpsock app Kevin Laatz
2019-06-24 15:36 ` Björn Töpel
2019-06-20 9:09 ` [PATCH 11/11] doc/af_xdp: include unaligned chunk case Kevin Laatz
2019-06-24 15:34 ` Björn Töpel
2019-07-16 3:06 ` [PATCH v2 00/10] XDP unaligned chunk placement support Kevin Laatz
2019-07-16 3:06 ` [PATCH v2 01/10] i40e: simplify Rx buffer recycle Kevin Laatz
2019-07-19 17:19 ` [Intel-wired-lan] " Bowers, AndrewX
2019-07-16 3:06 ` [PATCH v2 02/10] ixgbe: " Kevin Laatz
2019-07-19 17:20 ` [Intel-wired-lan] " Bowers, AndrewX
2019-07-16 3:06 ` [PATCH v2 03/10] xsk: add support to allow unaligned chunk placement Kevin Laatz
2019-07-19 17:21 ` [Intel-wired-lan] " Bowers, AndrewX
2019-07-16 3:06 ` [PATCH v2 04/10] i40e: modify driver for handling offsets Kevin Laatz
2019-07-19 17:22 ` [Intel-wired-lan] " Bowers, AndrewX
2019-07-16 3:06 ` [PATCH v2 05/10] ixgbe: " Kevin Laatz
2019-07-19 17:22 ` [Intel-wired-lan] " Bowers, AndrewX
2019-07-16 3:06 ` [PATCH v2 06/10] libbpf: add flags to umem config Kevin Laatz
2019-07-16 3:06 ` [PATCH v2 07/10] samples/bpf: add unaligned chunks mode support to xdpsock Kevin Laatz
2019-07-16 3:06 ` [PATCH v2 08/10] samples/bpf: add buffer recycling for unaligned chunks " Kevin Laatz
2019-07-16 3:06 ` [PATCH v2 09/10] samples/bpf: use hugepages in xdpsock app Kevin Laatz
2019-07-16 3:06 ` [PATCH v2 10/10] doc/af_xdp: include unaligned chunk case Kevin Laatz
2019-07-23 21:08 ` [PATCH v2 00/10] XDP unaligned chunk placement support Alexei Starovoitov
2019-07-24 13:25 ` Magnus Karlsson [this message]
2019-07-25 15:43 ` [Intel-wired-lan] " Jonathan Lemon
2019-07-24 5:10 ` [PATCH bpf-next v3 00/11] " Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 01/11] i40e: simplify Rx buffer recycle Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 02/11] ixgbe: " Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 03/11] xsk: add support to allow unaligned chunk placement Kevin Laatz
2019-07-25 2:22 ` Jakub Kicinski
2019-07-25 17:01 ` Laatz, Kevin
2019-07-25 9:27 ` Maxim Mikityanskiy
2019-07-25 17:00 ` Laatz, Kevin
2019-07-25 10:08 ` Maxim Mikityanskiy
2019-07-25 15:39 ` Jonathan Lemon
2019-07-24 5:10 ` [PATCH bpf-next v3 04/11] i40e: modify driver for handling offsets Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 05/11] ixgbe: " Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 06/11] mlx5e: " Kevin Laatz
2019-07-25 10:15 ` Maxim Mikityanskiy
2019-07-25 17:00 ` Laatz, Kevin
2019-07-24 5:10 ` [PATCH bpf-next v3 07/11] libbpf: add flags to umem config Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 08/11] samples/bpf: add unaligned chunks mode support to xdpsock Kevin Laatz
2019-07-25 9:43 ` Maxim Mikityanskiy
2019-07-25 17:00 ` Laatz, Kevin
2019-07-24 5:10 ` [PATCH bpf-next v3 09/11] samples/bpf: add buffer recycling for unaligned chunks " Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 10/11] samples/bpf: use hugepages in xdpsock app Kevin Laatz
2019-07-24 5:10 ` [PATCH bpf-next v3 11/11] doc/af_xdp: include unaligned chunk case Kevin Laatz
2019-07-25 15:39 ` [PATCH bpf-next v3 00/11] XDP unaligned chunk placement support Jonathan Lemon
2019-07-25 15:56 ` Richardson, Bruce
2019-07-25 17:30 ` Jonathan Lemon
2019-07-26 8:41 ` Bruce Richardson
2019-07-30 8:53 ` [PATCH bpf-next v4 " Kevin Laatz
2019-07-30 8:53 ` [PATCH bpf-next v4 01/11] i40e: simplify Rx buffer recycle Kevin Laatz
2019-07-30 8:53 ` [PATCH bpf-next v4 02/11] ixgbe: " Kevin Laatz
2019-07-30 8:53 ` [PATCH bpf-next v4 03/11] libbpf: add flags to umem config Kevin Laatz
2019-07-31 12:45 ` [Intel-wired-lan] " Björn Töpel
2019-07-31 14:25 ` Björn Töpel
2019-08-01 6:59 ` Andrii Nakryiko
2019-08-01 7:34 ` Björn Töpel
2019-08-02 7:19 ` Andrii Nakryiko
2019-08-02 7:26 ` Björn Töpel
2019-07-30 8:53 ` [PATCH bpf-next v4 04/11] xsk: add support to allow unaligned chunk placement Kevin Laatz
2019-07-31 18:11 ` Jonathan Lemon
2019-07-30 8:53 ` [PATCH bpf-next v4 05/11] i40e: modify driver for handling offsets Kevin Laatz
2019-07-30 8:53 ` [PATCH bpf-next v4 06/11] ixgbe: " Kevin Laatz
2019-07-30 8:53 ` [PATCH bpf-next v4 07/11] mlx5e: " Kevin Laatz
2019-07-31 18:10 ` Jonathan Lemon
2019-08-01 10:05 ` Maxim Mikityanskiy
2019-08-19 14:36 ` Maxim Mikityanskiy
2019-08-19 14:47 ` Laatz, Kevin
2019-07-30 8:53 ` [PATCH bpf-next v4 08/11] samples/bpf: add unaligned chunks mode support to xdpsock Kevin Laatz
2019-07-30 8:53 ` [PATCH bpf-next v4 09/11] samples/bpf: add buffer recycling for unaligned chunks " Kevin Laatz
2019-07-31 18:26 ` Jonathan Lemon
2019-07-30 8:53 ` [PATCH bpf-next v4 10/11] samples/bpf: use hugepages in xdpsock app Kevin Laatz
2019-07-30 8:54 ` [PATCH bpf-next v4 11/11] doc/af_xdp: include unaligned chunk case Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 00/11] XDP unaligned chunk placement support Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 01/11] i40e: simplify Rx buffer recycle Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 02/11] ixgbe: " Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 03/11] xsk: add support to allow unaligned chunk placement Kevin Laatz
2019-08-22 18:43 ` Jonathan Lemon
2019-08-23 13:35 ` Laatz, Kevin
2019-08-27 7:36 ` Maxim Mikityanskiy
2019-08-22 1:44 ` [PATCH bpf-next v5 04/11] i40e: modify driver for handling offsets Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 05/11] ixgbe: " Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 06/11] mlx5e: " Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 07/11] libbpf: add flags to umem config Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 08/11] samples/bpf: add unaligned chunks mode support to xdpsock Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 09/11] samples/bpf: add buffer recycling for unaligned chunks " Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 10/11] samples/bpf: use hugepages in xdpsock app Kevin Laatz
2019-08-22 1:44 ` [PATCH bpf-next v5 11/11] doc/af_xdp: include unaligned chunk case Kevin Laatz
2019-08-27 2:25 ` [PATCH bpf-next v6 00/12] XDP unaligned chunk placement support Kevin Laatz
2019-08-27 2:25 ` [PATCH bpf-next v6 01/12] i40e: simplify Rx buffer recycle Kevin Laatz
2019-08-30 15:37 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 02/12] ixgbe: " Kevin Laatz
2019-08-30 15:39 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 03/12] xsk: add support to allow unaligned chunk placement Kevin Laatz
2019-08-30 15:41 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 04/12] i40e: modify driver for handling offsets Kevin Laatz
2019-08-30 15:42 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 05/12] ixgbe: " Kevin Laatz
2019-08-30 15:42 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 06/12] mlx5e: " Kevin Laatz
2019-08-30 15:43 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 07/12] net/mlx5e: Allow XSK frames smaller than a page Kevin Laatz
2019-08-30 15:45 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 08/12] libbpf: add flags to umem config Kevin Laatz
2019-08-30 15:46 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 09/12] samples/bpf: add unaligned chunks mode support to xdpsock Kevin Laatz
2019-08-30 15:47 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 10/12] samples/bpf: add buffer recycling for unaligned chunks " Kevin Laatz
2019-08-30 15:49 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 11/12] samples/bpf: use hugepages in xdpsock app Kevin Laatz
2019-08-30 15:51 ` Jonathan Lemon
2019-08-27 2:25 ` [PATCH bpf-next v6 12/12] doc/af_xdp: include unaligned chunk case Kevin Laatz
2019-08-30 15:51 ` Jonathan Lemon
2019-08-30 15:52 ` [PATCH bpf-next v6 00/12] XDP unaligned chunk placement support Jonathan Lemon
2019-08-30 23:29 ` Daniel Borkmann
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAJ8uoz26Byzhc4My70h3nmfd3pQB1ahFf=ZzN2heN4asrJ-NQg@mail.gmail.com' \
--to=magnus.karlsson@gmail.com \
--cc=alexei.starovoitov@gmail.com \
--cc=ast@kernel.org \
--cc=bjorn.topel@intel.com \
--cc=bpf@vger.kernel.org \
--cc=bruce.richardson@intel.com \
--cc=ciara.loftus@intel.com \
--cc=daniel@iogearbox.net \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jakub.kicinski@netronome.com \
--cc=jonathan.lemon@gmail.com \
--cc=kevin.laatz@intel.com \
--cc=magnus.karlsson@intel.com \
--cc=netdev@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).