LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Thomas Gleixner <tglx@linutronix.de>
To: Marc Zyngier <maz@kernel.org>, John Garry <john.garry@huawei.com>
Cc: linux-kernel@vger.kernel.org, Jason Cooper <jason@lakedaemon.net>,
	Ming Lei <ming.lei@redhat.com>,
	"chenxiang \(M\)" <chenxiang66@hisilicon.com>
Subject: Re: [PATCH] irqchip/gic-v3-its: Balance initial LPI affinity across CPUs
Date: Mon, 20 Jan 2020 20:24:35 +0100
Message-ID: <87h80q2aoc.fsf@nanos.tec.linutronix.de> (raw)
In-Reply-To: <7dc37b35d8ec6c78e75969d8c6c2d2e9@kernel.org>

Marc,

Marc Zyngier <maz@kernel.org> writes:
> We're stuck between a rock and a hard place here:
>
> (1) We place all interrupts on the least loaded CPU that matches
>      the affinity -> results in performance issues on some funky
>      HW (like D05's SAS controller).
>
> (2) We place managed interrupts on the least loaded CPU that matches
>      the affinity -> we have artificial load on NUMA boundaries, and
>      reduced spread of overlapping managed interrupts.
>
> (3) We don't account for non-managed LPIs, and we run the risk of
>      unpredictable performance because we don't really know where
>      the *other* interrupts are.
>
> My personal preference would be to go for (1), as in my original post.
> I find (3) the least appealing, because we don't track things anymore.
> (2) feels like "the least of all evils", as it is a decent performance
> gain, seems to give predictable performance, and doesn't regress lesser
> systems...
>
> I'm definitely open to suggestions here.

The way x86 does it and that's mostly ok except for some really broken
setups is:

1) Non-managed interrupts:

   If the interrupt is bound to a node, then we try to find a target

     I)  in the intersection of affinity mask and node mask.

     II) in the nodemask itself

         Yes we ignore affinity mask there because that's pretty much
         the same as if the given affinity does not contain an online
         CPU.

     If all of that fails then we try the nodeless mode

   If the interrupt is not bound to a node, then we try to find a target

     I)  in the intersection of affinity mask and online mask.

     II) in the onlinemask itself

  Each step searches for the CPU in the searched mask which has the
  least number of total interrupts assigned.

2) Managed interrupts

  For managed interrupts we just search in the intersection of assigned
  mask and online CPUs for the CPU with the least number of managed
  interrupts.

  If no CPU is online then the interrupt is shutdown anyway, so no
  fallback required.

Don't know whether that's something you can map to ARM64, but I assume
the principle of trying to enforce NUMA locality plus balancing the
number of interrupts makes sense in general.

Thanks,

        tglx



  reply index

Thread overview: 9+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-19 19:05 Marc Zyngier
2020-01-20 16:13 ` John Garry
2020-01-20 17:42   ` Marc Zyngier
2020-01-20 18:21     ` John Garry
2020-01-20 18:45       ` Marc Zyngier
2020-01-20 19:24         ` Thomas Gleixner [this message]
2020-01-21 10:46           ` John Garry
2020-03-10 11:33           ` John Garry
2020-03-11 13:18             ` Marc Zyngier

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87h80q2aoc.fsf@nanos.tec.linutronix.de \
    --to=tglx@linutronix.de \
    --cc=chenxiang66@hisilicon.com \
    --cc=jason@lakedaemon.net \
    --cc=john.garry@huawei.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=maz@kernel.org \
    --cc=ming.lei@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git