All of lore.kernel.org
 help / color / mirror / Atom feed
From: Jakub Kicinski <kuba@kernel.org>
To: Andrew Lunn <andrew@lunn.ch>
Cc: Rakesh Pillai <pillair@codeaurora.org>,
	ath10k@lists.infradead.org, linux-wireless@vger.kernel.org,
	linux-kernel@vger.kernel.org, kvalo@codeaurora.org,
	johannes@sipsolutions.net, davem@davemloft.net,
	netdev@vger.kernel.org, dianders@chromium.org,
	evgreen@chromium.org, Eric Dumazet <edumazet@google.com>
Subject: Re: [RFC 0/7] Add support to process rx packets in thread
Date: Wed, 22 Jul 2020 09:20:01 -0700	[thread overview]
Message-ID: <20200722092001.62f3772c@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <20200721172514.GT1339445@lunn.ch>

On Tue, 21 Jul 2020 19:25:14 +0200 Andrew Lunn wrote:
> On Tue, Jul 21, 2020 at 10:44:19PM +0530, Rakesh Pillai wrote:
> > NAPI gets scheduled on the CPU core which got the
> > interrupt. The linux scheduler cannot move it to a
> > different core, even if the CPU on which NAPI is running
> > is heavily loaded. This can lead to degraded wifi
> > performance when running traffic at peak data rates.
> > 
> > A thread on the other hand can be moved to different
> > CPU cores, if the one on which its running is heavily
> > loaded. During high incoming data traffic, this gives
> > better performance, since the thread can be moved to a
> > less loaded or sometimes even a more powerful CPU core
> > to account for the required CPU performance in order
> > to process the incoming packets.
> > 
> > This patch series adds the support to use a high priority
> > thread to process the incoming packets, as opposed to
> > everything being done in NAPI context.  
> 
> I don't see why this problem is limited to the ath10k driver. I expect
> it applies to all drivers using NAPI. So shouldn't you be solving this
> in the NAPI core? Allow a driver to request the NAPI core uses a
> thread?

Agreed, this is a problem we have with all drivers today.
We see seriously sub-optimal behavior in data center workloads, 
because kernel overloads the cores doing packet processing.

I think the fix may actually be in the scheduler. AFAIU the scheduler
counts the softIRQ time towards the interrupted process, and on top
of that tries to move processes to the cores handling their IO. In the
end the configuration which works somewhat okay is when each core has
its own IRQ and queues, which is seriously sub-optimal.

WARNING: multiple messages have this Message-ID
From: Jakub Kicinski <kuba@kernel.org>
To: Andrew Lunn <andrew@lunn.ch>
Cc: netdev@vger.kernel.org, linux-wireless@vger.kernel.org,
	linux-kernel@vger.kernel.org, ath10k@lists.infradead.org,
	dianders@chromium.org, Eric Dumazet <edumazet@google.com>,
	Rakesh Pillai <pillair@codeaurora.org>,
	evgreen@chromium.org, johannes@sipsolutions.net,
	davem@davemloft.net, kvalo@codeaurora.org
Subject: Re: [RFC 0/7] Add support to process rx packets in thread
Date: Wed, 22 Jul 2020 09:20:01 -0700	[thread overview]
Message-ID: <20200722092001.62f3772c@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com> (raw)
In-Reply-To: <20200721172514.GT1339445@lunn.ch>

On Tue, 21 Jul 2020 19:25:14 +0200 Andrew Lunn wrote:
> On Tue, Jul 21, 2020 at 10:44:19PM +0530, Rakesh Pillai wrote:
> > NAPI gets scheduled on the CPU core which got the
> > interrupt. The linux scheduler cannot move it to a
> > different core, even if the CPU on which NAPI is running
> > is heavily loaded. This can lead to degraded wifi
> > performance when running traffic at peak data rates.
> > 
> > A thread on the other hand can be moved to different
> > CPU cores, if the one on which its running is heavily
> > loaded. During high incoming data traffic, this gives
> > better performance, since the thread can be moved to a
> > less loaded or sometimes even a more powerful CPU core
> > to account for the required CPU performance in order
> > to process the incoming packets.
> > 
> > This patch series adds the support to use a high priority
> > thread to process the incoming packets, as opposed to
> > everything being done in NAPI context.  
> 
> I don't see why this problem is limited to the ath10k driver. I expect
> it applies to all drivers using NAPI. So shouldn't you be solving this
> in the NAPI core? Allow a driver to request the NAPI core uses a
> thread?

Agreed, this is a problem we have with all drivers today.
We see seriously sub-optimal behavior in data center workloads, 
because kernel overloads the cores doing packet processing.

I think the fix may actually be in the scheduler. AFAIU the scheduler
counts the softIRQ time towards the interrupted process, and on top
of that tries to move processes to the cores handling their IO. In the
end the configuration which works somewhat okay is when each core has
its own IRQ and queues, which is seriously sub-optimal.

_______________________________________________
ath10k mailing list
ath10k@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/ath10k

  parent reply	other threads:[~2020-07-22 16:20 UTC|newest]

Thread overview: 96+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-21 17:14 Rakesh Pillai
2020-07-21 17:14 ` Rakesh Pillai
2020-07-21 17:14 ` [RFC 1/7] mac80211: Add check for napi handle before WARN_ON Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-22 12:56   ` Johannes Berg
2020-07-22 12:56     ` Johannes Berg
2020-07-23 18:26     ` Rakesh Pillai
2020-07-23 18:26       ` Rakesh Pillai
2020-07-23 20:06       ` Johannes Berg
2020-07-23 20:06         ` Johannes Berg
2020-07-24  6:21         ` Rakesh Pillai
2020-07-24  6:21           ` Rakesh Pillai
2020-07-26 16:19         ` Rakesh Pillai
2020-07-26 16:19           ` Rakesh Pillai
2020-07-30 12:40           ` Johannes Berg
2020-07-30 12:40             ` Johannes Berg
2020-07-21 17:14 ` [RFC 2/7] ath10k: Add support to process rx packet in thread Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-21 21:53   ` Rajkumar Manoharan
2020-07-21 21:53     ` Rajkumar Manoharan
2020-07-22 12:27     ` Felix Fietkau
2020-07-22 12:27       ` Felix Fietkau
2020-07-22 12:55       ` Johannes Berg
2020-07-22 12:55         ` Johannes Berg
2020-07-22 13:00         ` Felix Fietkau
2020-07-22 13:00           ` Felix Fietkau
2020-07-23  6:09           ` Rajkumar Manoharan
2020-07-23  6:09             ` Rajkumar Manoharan
2021-03-22 23:57           ` Ben Greear
2021-03-22 23:57             ` Ben Greear
2021-03-23  1:20             ` Brian Norris
2021-03-23  1:20               ` Brian Norris
2021-03-23  3:01               ` Ben Greear
2021-03-23  3:01                 ` Ben Greear
2021-03-23  7:45                 ` Felix Fietkau
2021-03-23  7:45                   ` Felix Fietkau
2021-03-25  9:45                   ` Rakesh Pillai
2021-03-25  9:45                     ` Rakesh Pillai
2021-03-25 10:33                     ` Felix Fietkau
2021-03-25 10:33                       ` Felix Fietkau
2020-07-23 18:25     ` Rakesh Pillai
2020-07-23 18:25       ` Rakesh Pillai
2020-07-24 23:11       ` Jacob Keller
2020-07-24 23:11         ` Jacob Keller
2020-07-21 17:14 ` [RFC 3/7] ath10k: Add module param to enable rx thread Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-21 17:14 ` [RFC 4/7] ath10k: Do not exhaust budget on process tx completion Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-21 17:14 ` [RFC 5/7] ath10k: Handle the rx packet processing in thread Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-21 17:14 ` [RFC 6/7] ath10k: Add deliver to stack from thread context Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-21 17:14 ` [RFC 7/7] ath10k: Handle rx thread suspend and resume Rakesh Pillai
2020-07-21 17:14   ` Rakesh Pillai
2020-07-23 23:06   ` Sebastian Gottschall
2020-07-23 23:06     ` Sebastian Gottschall
2020-07-24  6:19     ` Rakesh Pillai
2020-07-24  6:19       ` Rakesh Pillai
2020-07-21 17:25 ` [RFC 0/7] Add support to process rx packets in thread Andrew Lunn
2020-07-21 17:25   ` Andrew Lunn
2020-07-21 18:05   ` Florian Fainelli
2020-07-21 18:05     ` Florian Fainelli
2020-07-23 18:21     ` Rakesh Pillai
2020-07-23 18:21       ` Rakesh Pillai
2020-07-23 19:02       ` Florian Fainelli
2020-07-23 19:02         ` Florian Fainelli
2020-07-24  6:20         ` Rakesh Pillai
2020-07-24  6:20           ` Rakesh Pillai
2020-07-24 22:28           ` Florian Fainelli
2020-07-24 22:28             ` Florian Fainelli
2020-07-22  9:12   ` David Laight
2020-07-22  9:12     ` David Laight
2020-07-25  8:16     ` Hillf Danton
2020-07-25 10:38       ` Sebastian Gottschall
2020-07-25 10:38         ` Sebastian Gottschall
2020-07-25 12:25         ` Hillf Danton
2020-07-25 14:08         ` Sebastian Gottschall
2020-07-25 14:08           ` Sebastian Gottschall
2020-07-25 14:57           ` Hillf Danton
2020-07-25 15:41             ` Sebastian Gottschall
2020-07-25 15:41               ` Sebastian Gottschall
2020-07-26 11:16               ` David Laight
2020-07-26 11:16                 ` David Laight
2020-07-28 16:59                 ` Rakesh Pillai
2020-07-28 16:59                   ` Rakesh Pillai
2020-07-29  1:34                   ` Hillf Danton
2020-07-25 17:57       ` Felix Fietkau
2020-07-25 17:57         ` Felix Fietkau
2020-07-26  1:22         ` Hillf Danton
2020-07-26  8:10           ` Felix Fietkau
2020-07-26  8:10             ` Felix Fietkau
2020-07-26  8:32             ` Hillf Danton
2020-07-26  8:59               ` Felix Fietkau
2020-07-26  8:59                 ` Felix Fietkau
2020-07-22 16:20   ` Jakub Kicinski [this message]
2020-07-22 16:20     ` Jakub Kicinski

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200722092001.62f3772c@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com \
    --to=kuba@kernel.org \
    --cc=andrew@lunn.ch \
    --cc=ath10k@lists.infradead.org \
    --cc=davem@davemloft.net \
    --cc=dianders@chromium.org \
    --cc=edumazet@google.com \
    --cc=evgreen@chromium.org \
    --cc=johannes@sipsolutions.net \
    --cc=kvalo@codeaurora.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-wireless@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=pillair@codeaurora.org \
    --subject='Re: [RFC 0/7] Add support to process rx packets in thread' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.