From: Magnus Karlsson <magnus.karlsson@gmail.com>
To: "Gaul, Maximilian" <maximilian.gaul@hm.edu>
Cc: Xdp <xdp-newbies@vger.kernel.org>
Subject: Re: Shared Umem and reducing ksoftirqd-Load
Date: Wed, 25 Mar 2020 10:41:10 +0100 [thread overview]
Message-ID: <CAJ8uoz1Vh+zj6msumTNWgtZY6jdOeYgoyDBoizfna1dzhC465A@mail.gmail.com> (raw)
In-Reply-To: <018e8071725b48399141cc46b63641e1@hm.edu>
On Wed, Mar 25, 2020 at 10:04 AM Gaul, Maximilian
<maximilian.gaul@hm.edu> wrote:
>
> I am running a Multi-AF-XDP-Socket approach per RX-Queue (using Shared Umem).
>
> Unfortunately I am noticing, that at around 650k pps, the *ksoftirqd*-thread of that RX-Queue ramps up to 100% thus leading to packet loss.
> I tried setting *XDP_USE_NEED_WAKEUP* on *xsk_socket_cfg.bind_flags* but those bind_flags are only taken into account if *umem->refcount > 1* (libbpf/xsk.c - xsk_socket__create()).
> As far as I understand this correctly, only the first socket is able to set *XDP_USE_NEED_WAKEUP* because for all sockets after, *umem->refcount* is going to be at least 2.
Yes, the other sockets just inherit the settings of the first one.
Are you using the SKB mode? What is your packet size? Sounds like a
low number unless you have large packets and are using the SKB mode.
> I didn't observe a dramatic change as I've hoped to. Are there some other ways to reduce interrupt load (user-space application and ksoftirq are already running on different CPUs)?
The need_wakeup flag has a big impact when you run the softirq and the
application thread on the same core. When using two cores for this, it
has less of an impact.
/Magnus
> NIC: Mellanox Technologies MT27800
>
> Best regards
>
> Max
>
next prev parent reply other threads:[~2020-03-25 9:41 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-25 9:04 Shared Umem and reducing ksoftirqd-Load Gaul, Maximilian
2020-03-25 9:41 ` Magnus Karlsson [this message]
2020-03-25 10:02 ` AW: " Gaul, Maximilian
2020-03-25 10:24 ` Magnus Karlsson
2020-03-25 10:45 ` AW: " Gaul, Maximilian
2020-03-25 11:04 ` Magnus Karlsson
2020-03-25 12:40 ` AW: " Gaul, Maximilian
2020-03-25 13:36 ` Magnus Karlsson
2020-03-26 12:30 ` AW: " Gaul, Maximilian
2020-03-26 12:48 ` Magnus Karlsson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJ8uoz1Vh+zj6msumTNWgtZY6jdOeYgoyDBoizfna1dzhC465A@mail.gmail.com \
--to=magnus.karlsson@gmail.com \
--cc=maximilian.gaul@hm.edu \
--cc=xdp-newbies@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).