linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Nitesh Narayan Lal <nitesh@redhat.com>
To: Marcelo Tosatti <mtosatti@redhat.com>,
	Robin Murphy <robin.murphy@arm.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	"frederic@kernel.org" <frederic@kernel.org>
Cc: linux-kernel@vger.kernel.org, linux-api@vger.kernel.org,
	juri.lelli@redhat.com, abelits@marvell.com, bhelgaas@google.com,
	linux-pci@vger.kernel.org, rostedt@goodmis.org, mingo@kernel.org,
	peterz@infradead.org, davem@davemloft.net,
	akpm@linux-foundation.org, sfr@canb.auug.org.au,
	stephen@networkplumber.org, rppt@linux.vnet.ibm.com,
	jinyuqi@huawei.com, zhangshaokun@hisilicon.com
Subject: Re: [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs
Date: Wed, 27 Jan 2021 09:16:37 -0500	[thread overview]
Message-ID: <7b810c5a-c732-4df3-1034-c2959ae86e65@redhat.com> (raw)
In-Reply-To: <20210127130925.GA64740@fuller.cnet>


On 1/27/21 8:09 AM, Marcelo Tosatti wrote:
> On Wed, Jan 27, 2021 at 12:36:30PM +0000, Robin Murphy wrote:
>> On 2021-01-27 12:19, Marcelo Tosatti wrote:
>>> On Wed, Jan 27, 2021 at 11:57:16AM +0000, Robin Murphy wrote:
>>>> Hi,
>>>>
>>>> On 2020-06-25 23:34, Nitesh Narayan Lal wrote:
>>>>> From: Alex Belits <abelits@marvell.com>
>>>>>
>>>>> The current implementation of cpumask_local_spread() does not respect the
>>>>> isolated CPUs, i.e., even if a CPU has been isolated for Real-Time task,
>>>>> it will return it to the caller for pinning of its IRQ threads. Having
>>>>> these unwanted IRQ threads on an isolated CPU adds up to a latency
>>>>> overhead.
>>>>>
>>>>> Restrict the CPUs that are returned for spreading IRQs only to the
>>>>> available housekeeping CPUs.
>>>>>
>>>>> Signed-off-by: Alex Belits <abelits@marvell.com>
>>>>> Signed-off-by: Nitesh Narayan Lal <nitesh@redhat.com>
>>>>> ---
>>>>>    lib/cpumask.c | 16 +++++++++++-----
>>>>>    1 file changed, 11 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/lib/cpumask.c b/lib/cpumask.c
>>>>> index fb22fb266f93..85da6ab4fbb5 100644
>>>>> --- a/lib/cpumask.c
>>>>> +++ b/lib/cpumask.c
>>>>> @@ -6,6 +6,7 @@
>>>>>    #include <linux/export.h>
>>>>>    #include <linux/memblock.h>
>>>>>    #include <linux/numa.h>
>>>>> +#include <linux/sched/isolation.h>
>>>>>    /**
>>>>>     * cpumask_next - get the next cpu in a cpumask
>>>>> @@ -205,22 +206,27 @@ void __init free_bootmem_cpumask_var(cpumask_var_t mask)
>>>>>     */
>>>>>    unsigned int cpumask_local_spread(unsigned int i, int node)
>>>>>    {
>>>>> -	int cpu;
>>>>> +	int cpu, hk_flags;
>>>>> +	const struct cpumask *mask;
>>>>> +	hk_flags = HK_FLAG_DOMAIN | HK_FLAG_MANAGED_IRQ;
>>>>> +	mask = housekeeping_cpumask(hk_flags);
>>>> AFAICS, this generally resolves to something based on cpu_possible_mask
>>>> rather than cpu_online_mask as before, so could now potentially return an
>>>> offline CPU. Was that an intentional change?
>>> Robin,
>>>
>>> AFAICS online CPUs should be filtered.
>> Apologies if I'm being thick, but can you explain how? In the case of
>> isolation being disabled or compiled out, housekeeping_cpumask() is
>> literally just "return cpu_possible_mask;". If we then iterate over that
>> with for_each_cpu() and just return the i'th possible CPU (e.g. in the
>> NUMA_NO_NODE case), what guarantees that CPU is actually online?
>>
>> Robin.
> Nothing, but that was the situation before 1abdfe706a579a702799fce465bceb9fb01d407c
> as well.

Marcelo, before the commit cpumask_local_spread, was in fact, relying on
cpu_online_mask as Robin mentioned.
The problem here is with housekeeping_cpumask which always relied on the
cpu_possible_mask.

>
> cpumask_local_spread() should probably be disabling CPU hotplug.


Yes and this should also be done at several other places in the drivers
which don't take CPU hotplug into account eg. at the time of vector
allocation.


-- 
Thanks
Nitesh


  parent reply	other threads:[~2021-01-27 14:20 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-06-25 22:34 [PATCH v4 0/3] Preventing job distribution to isolated CPUs Nitesh Narayan Lal
2020-06-25 22:34 ` [Patch v4 1/3] lib: Restrict cpumask_local_spread to houskeeping CPUs Nitesh Narayan Lal
2020-06-29 16:11   ` Nitesh Narayan Lal
2020-07-01  0:32     ` Andrew Morton
2020-07-01  0:47       ` Nitesh Narayan Lal
2021-01-27 11:57   ` Robin Murphy
2021-01-27 12:19     ` Marcelo Tosatti
2021-01-27 12:36       ` Robin Murphy
2021-01-27 13:09         ` Marcelo Tosatti
2021-01-27 13:49           ` Robin Murphy
2021-01-27 14:16           ` Nitesh Narayan Lal [this message]
2021-01-28 15:56           ` Thomas Gleixner
2021-01-28 16:33             ` Marcelo Tosatti
     [not found]             ` <02ac9d85-7ddd-96da-1252-4663feea7c9f@marvell.com>
2021-02-01 17:50               ` [EXT] " Marcelo Tosatti
2021-01-28 16:02       ` Thomas Gleixner
2021-01-28 16:59         ` Marcelo Tosatti
2021-01-28 17:35           ` Nitesh Narayan Lal
2021-01-28 20:01           ` Thomas Gleixner
     [not found]             ` <d2a4dc97-a9ed-e0e7-3b9c-c56ae46f6608@redhat.com>
     [not found]               ` <20210129142356.GB40876@fuller.cnet>
2021-01-29 17:34                 ` [EXT] " Alex Belits
     [not found]                 ` <18584612-868c-0f88-5de2-dc93c8638816@redhat.com>
2021-02-05 19:56                   ` Thomas Gleixner
2021-02-04 18:15             ` Marcelo Tosatti
2021-02-04 18:47               ` Nitesh Narayan Lal
2021-02-04 19:06                 ` Marcelo Tosatti
2021-02-04 19:17                   ` Nitesh Narayan Lal
2021-02-05 22:23                     ` Thomas Gleixner
2021-02-05 22:26                       ` Thomas Gleixner
2021-02-07  0:43                       ` Nitesh Narayan Lal
2021-02-11 15:55                         ` Nitesh Narayan Lal
2021-03-04 18:15                           ` Nitesh Narayan Lal
     [not found]                             ` <faa8d84e-db67-7fbe-891e-f4987f106b20@marvell.com>
2021-03-04 23:23                               ` [EXT] " Nitesh Narayan Lal
2021-04-06 17:22                             ` Jesse Brandeburg
2021-04-07 15:18                               ` Nitesh Narayan Lal
2021-04-08 18:49                                 ` Nitesh Narayan Lal
2021-04-14 16:11                                 ` Jesse Brandeburg
2021-04-15 22:11                                   ` Nitesh Narayan Lal
2021-04-29 21:44                                     ` Nitesh Lal
2021-04-30  1:48                                       ` Jesse Brandeburg
2021-04-30 13:10                                         ` Nitesh Lal
2021-04-30  7:10                                       ` Thomas Gleixner
2021-04-30 16:14                                         ` Nitesh Lal
2021-04-30 18:21                                           ` Thomas Gleixner
2021-04-30 21:07                                             ` Nitesh Lal
2021-05-01  2:21                                               ` Jesse Brandeburg
2021-05-03 13:15                                                 ` Nitesh Lal
2020-06-25 22:34 ` [Patch v4 2/3] PCI: Restrict probe functions to housekeeping CPUs Nitesh Narayan Lal
2020-06-25 22:34 ` [Patch v4 3/3] net: Restrict receive packets queuing " Nitesh Narayan Lal
2020-06-26 11:14   ` Peter Zijlstra
2020-06-26 17:20     ` David Miller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=7b810c5a-c732-4df3-1034-c2959ae86e65@redhat.com \
    --to=nitesh@redhat.com \
    --cc=abelits@marvell.com \
    --cc=akpm@linux-foundation.org \
    --cc=bhelgaas@google.com \
    --cc=davem@davemloft.net \
    --cc=frederic@kernel.org \
    --cc=jinyuqi@huawei.com \
    --cc=juri.lelli@redhat.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=mtosatti@redhat.com \
    --cc=peterz@infradead.org \
    --cc=robin.murphy@arm.com \
    --cc=rostedt@goodmis.org \
    --cc=rppt@linux.vnet.ibm.com \
    --cc=sfr@canb.auug.org.au \
    --cc=stephen@networkplumber.org \
    --cc=tglx@linutronix.de \
    --cc=zhangshaokun@hisilicon.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).