From: Ingo Molnar <mingo@kernel.org>
To: Alexander Gordeev <agordeev@redhat.com>,
Arjan van de Ven <arjan@infradead.org>
Cc: linux-kernel@vger.kernel.org, x86@kernel.org,
Suresh Siddha <suresh.b.siddha@intel.com>,
Cyrill Gorcunov <gorcunov@openvz.org>,
Yinghai Lu <yinghai@kernel.org>,
Linus Torvalds <torvalds@linux-foundation.org>
Subject: Re: [PATCH 2/3] x86: x2apic/cluster: Make use of lowest priority delivery mode
Date: Mon, 21 May 2012 14:40:26 +0200 [thread overview]
Message-ID: <20120521124025.GC17065@gmail.com> (raw)
In-Reply-To: <20120521093648.GC28930@dhcp-26-207.brq.redhat.com>
* Alexander Gordeev <agordeev@redhat.com> wrote:
> > So, in theory, prior the patch you should be seeing irqs go
> > to only one CPU, while after the patch they are spread out
> > amongst the CPU. If it's using LowestPrio delivery then we
> > depend on the hardware doing this for us - how does this
> > work out in practice, are the target CPUs round-robin-ed,
> > with a new CPU for every new IRQ delivered?
>
> That is exactly what I can observe.
>
> As of 'target CPUs round-robin-ed' and 'with a new CPU for
> every new IRQ delivered' -- that is something we can not
> control as you noted. Nor do we care to my understanding.
>
> I can not commit on every h/w out there obviously, but on my
> PowerEdge M910 with some half-dozen clusters with six CPU per
> each, the interrupts are perfectly balanced among those ones
> present in IRTEs.
But that is not 'perfectly balanced' in many cases.
When the hardware round-robins the interrupts then each
interrupt will go to a 'cache cold' CPU in essence. This is
pretty much the worst thing possible thing to do in most cases:
while it's "perfectly balanced" in the sense of distributing
cycles evenly between CPUs, each interrupt handler execution
will generate an avalance of cachemisses, for cachelines there
were modified in the previous invocation of the irq.
One notable exception is when the CPUs are SMT/Hyperthreading
siblings, in that case they are sharing even the L1 cache, so
there's very little cost to round-robining the IRQs within the
CPU mask.
But AFAICS irqbalanced will spread irqs on wider masks than SMT
sibling boundaries, exposing us to the above performance
problem.
So I think we need to tread carefully here.
Thanks,
Ingo
next prev parent reply other threads:[~2012-05-21 12:40 UTC|newest]
Thread overview: 69+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-05-18 10:26 [PATCH 2/3] x86: x2apic/cluster: Make use of lowest priority delivery mode Alexander Gordeev
2012-05-18 14:41 ` Cyrill Gorcunov
2012-05-18 15:42 ` Alexander Gordeev
2012-05-18 15:51 ` Cyrill Gorcunov
2012-05-19 10:47 ` Cyrill Gorcunov
2012-05-21 7:11 ` Alexander Gordeev
2012-05-21 9:46 ` Cyrill Gorcunov
2012-05-19 20:53 ` Yinghai Lu
2012-05-21 8:13 ` Alexander Gordeev
2012-05-21 23:02 ` Yinghai Lu
2012-05-21 23:33 ` Yinghai Lu
2012-05-22 9:36 ` Alexander Gordeev
2012-05-21 23:44 ` Suresh Siddha
2012-05-21 23:58 ` [PATCH 1/2] x86, irq: update irq_cfg domain unless the new affinity is a subset of the current domain Suresh Siddha
2012-05-21 23:58 ` [PATCH 2/2] x2apic, cluster: use all the members of one cluster specified in the smp_affinity mask for the interrupt desintation Suresh Siddha
2012-05-22 7:04 ` Ingo Molnar
2012-05-22 7:34 ` Cyrill Gorcunov
2012-05-22 17:21 ` Suresh Siddha
2012-05-22 17:39 ` Cyrill Gorcunov
2012-05-22 17:42 ` Suresh Siddha
2012-05-22 17:45 ` Cyrill Gorcunov
2012-05-22 20:03 ` Yinghai Lu
2012-06-06 15:04 ` [tip:x86/apic] x86/x2apic/cluster: Use all the members of one cluster specified in the smp_affinity mask for the interrupt destination tip-bot for Suresh Siddha
2012-06-06 22:21 ` Yinghai Lu
2012-06-06 23:14 ` Suresh Siddha
2012-06-06 15:03 ` [tip:x86/apic] x86/irq: Update irq_cfg domain unless the new affinity is a subset of the current domain tip-bot for Suresh Siddha
2012-08-07 15:31 ` Robert Richter
2012-08-07 15:41 ` do_IRQ: 1.55 No irq handler for vector (irq -1) Borislav Petkov
2012-08-07 16:24 ` Suresh Siddha
2012-08-07 17:28 ` Robert Richter
2012-08-07 17:47 ` Suresh Siddha
2012-08-07 17:45 ` Eric W. Biederman
2012-08-07 20:57 ` Borislav Petkov
2012-08-07 22:39 ` Suresh Siddha
2012-08-08 8:58 ` Robert Richter
2012-08-08 11:04 ` Borislav Petkov
2012-08-08 19:16 ` Suresh Siddha
2012-08-14 17:02 ` [tip:x86/urgent] x86, apic: fix broken legacy interrupts in the logical apic mode tip-bot for Suresh Siddha
2012-06-06 17:20 ` [PATCH 1/2] x86, irq: update irq_cfg domain unless the new affinity is a subset of the current domain Alexander Gordeev
2012-06-06 23:02 ` Suresh Siddha
2012-06-16 0:25 ` Suresh Siddha
2012-06-18 9:17 ` Alexander Gordeev
2012-06-19 0:51 ` Suresh Siddha
2012-06-19 23:43 ` [PATCH 1/2] x86, apic: optimize cpu traversal in __assign_irq_vector() using domain membership Suresh Siddha
2012-06-19 23:43 ` [PATCH 2/2] x86, x2apic: limit the vector reservation to the user specified mask Suresh Siddha
2012-06-20 5:56 ` Yinghai Lu
2012-06-21 9:04 ` Alexander Gordeev
2012-06-21 21:51 ` Suresh Siddha
2012-06-20 5:53 ` [PATCH 1/2] x86, apic: optimize cpu traversal in __assign_irq_vector() using domain membership Yinghai Lu
2012-06-21 8:31 ` Alexander Gordeev
2012-06-21 21:53 ` Suresh Siddha
2012-06-20 0:18 ` [PATCH 1/2] x86, irq: update irq_cfg domain unless the new affinity is a subset of the current domain Suresh Siddha
2012-06-21 11:00 ` Alexander Gordeev
2012-06-21 21:58 ` Suresh Siddha
2012-05-22 10:12 ` [PATCH 2/3] x86: x2apic/cluster: Make use of lowest priority delivery mode Alexander Gordeev
2012-05-21 8:22 ` Ingo Molnar
2012-05-21 9:36 ` Alexander Gordeev
2012-05-21 12:40 ` Ingo Molnar [this message]
2012-05-21 14:48 ` Alexander Gordeev
2012-05-21 14:59 ` Ingo Molnar
2012-05-21 15:22 ` Alexander Gordeev
2012-05-21 15:34 ` Cyrill Gorcunov
2012-05-21 15:36 ` Linus Torvalds
2012-05-21 18:07 ` Suresh Siddha
2012-05-21 18:18 ` Linus Torvalds
2012-05-21 18:37 ` Suresh Siddha
2012-05-21 19:30 ` Ingo Molnar
2012-05-21 19:15 ` Ingo Molnar
2012-05-21 19:56 ` Suresh Siddha
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120521124025.GC17065@gmail.com \
--to=mingo@kernel.org \
--cc=agordeev@redhat.com \
--cc=arjan@infradead.org \
--cc=gorcunov@openvz.org \
--cc=linux-kernel@vger.kernel.org \
--cc=suresh.b.siddha@intel.com \
--cc=torvalds@linux-foundation.org \
--cc=x86@kernel.org \
--cc=yinghai@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.