All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: Eric Dumazet <eric.dumazet@gmail.com>
Cc: Felix Fietkau <nbd@nbd.name>,
	netdev@vger.kernel.org, Hillf Danton <hdanton@sina.com>
Subject: Re: [PATCH v2] net: add support for threaded NAPI polling
Date: Thu, 6 Aug 2020 12:57:08 -0700	[thread overview]
Message-ID: <20200806125708.6492ebfe@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <2ce6de8b-f520-3f09-746a-caf2ecab428a@gmail.com>

On Thu, 6 Aug 2020 12:25:08 -0700 Eric Dumazet wrote:
> On 8/6/20 11:55 AM, Jakub Kicinski wrote:
> > I'm still trying to wrap my head around this.
> > 
> > Am I understanding correctly that you have one IRQ and multiple NAPI
> > instances?
> > 
> > Are we not going to end up with pretty terrible cache locality here if
> > the scheduler starts to throw rx and tx completions around to random
> > CPUs?
> > 
> > I understand that implementing separate kthreads would be more LoC, but
> > we do have ksoftirqs already... maybe we should make the NAPI ->
> > ksoftirq mapping more flexible, and improve the logic which decides to
> > load ksoftirq rather than make $current() pay?
> > 
> > Sorry for being slow.
> 
> Issue with ksoftirqd is that
> - it is bound to a cpu

Do you envision the scheduler balancing or work stealing being
advantageous in some configurations?

I was guessing that for compute workloads having ksoftirq bound will
actually make things more predictable/stable.

For pure routers (where we expect multiple cores to reach 100% just
doing packet forwarding) as long as there is an API to re-balance NAPIs
to cores - a simple specialized user space daemon would probably do a
better job as it can consult packet drop metrics etc.

Obviously I have no data to back up these claims..

> - Its nice value is 0, meaning that user threads can sometime compete too much with it.

True, I thought we could assume user level tuning.

> - It handles all kinds of softirqs, so messing with it might hurt some other layer.

Right, I have no data on how much this hurts in practice.

> Note that the patch is using a dedicate work queue. It is going to be not practical
> in case you need to handle two different NIC, and want separate pools for each of them.
> 
> Ideally, having one kthread per queue would be nice, but then there is more plumbing
> work to let these kthreads being visible in a convenient way (/sys/class/net/ethX/queues/..../kthread)

Is context switching cost negligible?

ksoftirq-like thread replicates all the NAPI budget-level mixing we
already do today.

  reply	other threads:[~2020-08-06 19:57 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-08-06  9:55 [PATCH v2] net: add support for threaded NAPI polling Felix Fietkau
2020-08-06 11:51 ` kernel test robot
2020-08-06 11:51   ` kernel test robot
2020-08-06 11:51 ` [RFC PATCH] net: dev_attr_napi_threaded can be static kernel test robot
2020-08-06 11:51   ` kernel test robot
2020-08-06 17:39 ` [PATCH v2] net: add support for threaded NAPI polling Eric Dumazet
2020-08-06 18:55 ` Jakub Kicinski
2020-08-06 19:25   ` Eric Dumazet
2020-08-06 19:57     ` Jakub Kicinski [this message]
2020-08-06 21:18       ` Eric Dumazet
2020-08-06 22:48 ` Wei Wang

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200806125708.6492ebfe@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com \
    --to=kuba@kernel.org \
    --cc=eric.dumazet@gmail.com \
    --cc=hdanton@sina.com \
    --cc=nbd@nbd.name \
    --cc=netdev@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.