From: "Michael S. Tsirkin" <mst@redhat.com>
To: Laurent Vivier <lvivier@redhat.com>
Cc: linux-kernel@vger.kernel.org,
Thomas Gleixner <tglx@linutronix.de>,
Paul Mackerras <paulus@samba.org>,
Benjamin Herrenschmidt <benh@kernel.crashing.org>,
Michael Ellerman <mpe@ellerman.id.au>,
linux-block@vger.kernel.org, Marc Zyngier <maz@kernel.org>,
linuxppc-dev@lists.ozlabs.org, linux-pci@vger.kernel.org,
Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH 0/2] powerpc/pseries: fix MSI/X IRQ affinity on pseries
Date: Tue, 24 Nov 2020 16:08:06 -0500 [thread overview]
Message-ID: <20201124160747-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <20201124200308.1110744-1-lvivier@redhat.com>
On Tue, Nov 24, 2020 at 09:03:06PM +0100, Laurent Vivier wrote:
> With virtio, in multiqueue case, each queue IRQ is normally
> bound to a different CPU using the affinity mask.
>
> This works fine on x86_64 but totally ignored on pseries.
>
> This is not obvious at first look because irqbalance is doing
> some balancing to improve that.
>
> It appears that the "managed" flag set in the MSI entry
> is never copied to the system IRQ entry.
>
> This series passes the affinity mask from rtas_setup_msi_irqs()
> to irq_domain_alloc_descs() by adding an affinity parameter to
> irq_create_mapping().
>
> The first patch adds the parameter (no functional change), the
> second patch passes the actual affinity mask to irq_create_mapping()
> in rtas_setup_msi_irqs().
>
> For instance, with 32 CPUs VM and 32 queues virtio-scsi interface:
>
> ... -smp 32 -device virtio-scsi-pci,id=virtio_scsi_pci0,num_queues=32
>
> for IRQ in $(grep virtio2-request /proc/interrupts |cut -d: -f1); do
> for file in /proc/irq/$IRQ/ ; do
> echo -n "IRQ: $(basename $file) CPU: " ; cat $file/smp_affinity_list
> done
> done
>
> Without the patch (and without irqbalanced)
>
> IRQ: 268 CPU: 0-31
> IRQ: 269 CPU: 0-31
> IRQ: 270 CPU: 0-31
> IRQ: 271 CPU: 0-31
> IRQ: 272 CPU: 0-31
> IRQ: 273 CPU: 0-31
> IRQ: 274 CPU: 0-31
> IRQ: 275 CPU: 0-31
> IRQ: 276 CPU: 0-31
> IRQ: 277 CPU: 0-31
> IRQ: 278 CPU: 0-31
> IRQ: 279 CPU: 0-31
> IRQ: 280 CPU: 0-31
> IRQ: 281 CPU: 0-31
> IRQ: 282 CPU: 0-31
> IRQ: 283 CPU: 0-31
> IRQ: 284 CPU: 0-31
> IRQ: 285 CPU: 0-31
> IRQ: 286 CPU: 0-31
> IRQ: 287 CPU: 0-31
> IRQ: 288 CPU: 0-31
> IRQ: 289 CPU: 0-31
> IRQ: 290 CPU: 0-31
> IRQ: 291 CPU: 0-31
> IRQ: 292 CPU: 0-31
> IRQ: 293 CPU: 0-31
> IRQ: 294 CPU: 0-31
> IRQ: 295 CPU: 0-31
> IRQ: 296 CPU: 0-31
> IRQ: 297 CPU: 0-31
> IRQ: 298 CPU: 0-31
> IRQ: 299 CPU: 0-31
>
> With the patch:
>
> IRQ: 265 CPU: 0
> IRQ: 266 CPU: 1
> IRQ: 267 CPU: 2
> IRQ: 268 CPU: 3
> IRQ: 269 CPU: 4
> IRQ: 270 CPU: 5
> IRQ: 271 CPU: 6
> IRQ: 272 CPU: 7
> IRQ: 273 CPU: 8
> IRQ: 274 CPU: 9
> IRQ: 275 CPU: 10
> IRQ: 276 CPU: 11
> IRQ: 277 CPU: 12
> IRQ: 278 CPU: 13
> IRQ: 279 CPU: 14
> IRQ: 280 CPU: 15
> IRQ: 281 CPU: 16
> IRQ: 282 CPU: 17
> IRQ: 283 CPU: 18
> IRQ: 284 CPU: 19
> IRQ: 285 CPU: 20
> IRQ: 286 CPU: 21
> IRQ: 287 CPU: 22
> IRQ: 288 CPU: 23
> IRQ: 289 CPU: 24
> IRQ: 290 CPU: 25
> IRQ: 291 CPU: 26
> IRQ: 292 CPU: 27
> IRQ: 293 CPU: 28
> IRQ: 294 CPU: 29
> IRQ: 295 CPU: 30
> IRQ: 299 CPU: 31
>
> This matches what we have on an x86_64 system.
Makes sense to me. FWIW
Acked-by: Michael S. Tsirkin <mst@redhat.com>
> Laurent Vivier (2):
> genirq: add an affinity parameter to irq_create_mapping()
> powerpc/pseries: pass MSI affinity to irq_create_mapping()
>
> arch/arc/kernel/intc-arcv2.c | 4 ++--
> arch/arc/kernel/mcip.c | 2 +-
> arch/arm/common/sa1111.c | 2 +-
> arch/arm/mach-s3c/irq-s3c24xx.c | 3 ++-
> arch/arm/plat-orion/gpio.c | 2 +-
> arch/mips/ath25/ar2315.c | 4 ++--
> arch/mips/ath25/ar5312.c | 4 ++--
> arch/mips/lantiq/irq.c | 2 +-
> arch/mips/pci/pci-ar2315.c | 3 ++-
> arch/mips/pic32/pic32mzda/time.c | 2 +-
> arch/mips/ralink/irq.c | 2 +-
> arch/powerpc/kernel/pci-common.c | 2 +-
> arch/powerpc/kvm/book3s_xive.c | 2 +-
> arch/powerpc/platforms/44x/ppc476.c | 4 ++--
> arch/powerpc/platforms/cell/interrupt.c | 4 ++--
> arch/powerpc/platforms/cell/iommu.c | 3 ++-
> arch/powerpc/platforms/cell/pmu.c | 2 +-
> arch/powerpc/platforms/cell/spider-pic.c | 2 +-
> arch/powerpc/platforms/cell/spu_manage.c | 6 +++---
> arch/powerpc/platforms/maple/pci.c | 2 +-
> arch/powerpc/platforms/pasemi/dma_lib.c | 5 +++--
> arch/powerpc/platforms/pasemi/msi.c | 2 +-
> arch/powerpc/platforms/pasemi/setup.c | 4 ++--
> arch/powerpc/platforms/powermac/pci.c | 2 +-
> arch/powerpc/platforms/powermac/pic.c | 2 +-
> arch/powerpc/platforms/powermac/smp.c | 2 +-
> arch/powerpc/platforms/powernv/opal-irqchip.c | 5 +++--
> arch/powerpc/platforms/powernv/pci.c | 2 +-
> arch/powerpc/platforms/powernv/vas.c | 2 +-
> arch/powerpc/platforms/ps3/interrupt.c | 2 +-
> arch/powerpc/platforms/pseries/ibmebus.c | 2 +-
> arch/powerpc/platforms/pseries/msi.c | 2 +-
> arch/powerpc/sysdev/fsl_mpic_err.c | 2 +-
> arch/powerpc/sysdev/fsl_msi.c | 2 +-
> arch/powerpc/sysdev/mpic.c | 3 ++-
> arch/powerpc/sysdev/mpic_u3msi.c | 2 +-
> arch/powerpc/sysdev/xics/xics-common.c | 2 +-
> arch/powerpc/sysdev/xive/common.c | 2 +-
> arch/sh/boards/mach-se/7343/irq.c | 2 +-
> arch/sh/boards/mach-se/7722/irq.c | 2 +-
> arch/sh/boards/mach-x3proto/gpio.c | 2 +-
> arch/xtensa/kernel/perf_event.c | 2 +-
> arch/xtensa/kernel/smp.c | 2 +-
> arch/xtensa/kernel/time.c | 2 +-
> drivers/ata/pata_macio.c | 2 +-
> drivers/base/regmap/regmap-irq.c | 2 +-
> drivers/bus/moxtet.c | 2 +-
> drivers/clocksource/ingenic-timer.c | 2 +-
> drivers/clocksource/timer-riscv.c | 2 +-
> drivers/extcon/extcon-max8997.c | 3 ++-
> drivers/gpio/gpio-bcm-kona.c | 2 +-
> drivers/gpio/gpio-brcmstb.c | 2 +-
> drivers/gpio/gpio-davinci.c | 2 +-
> drivers/gpio/gpio-em.c | 3 ++-
> drivers/gpio/gpio-grgpio.c | 2 +-
> drivers/gpio/gpio-mockup.c | 2 +-
> drivers/gpio/gpio-mpc8xxx.c | 2 +-
> drivers/gpio/gpio-mvebu.c | 2 +-
> drivers/gpio/gpio-tb10x.c | 2 +-
> drivers/gpio/gpio-tegra.c | 2 +-
> drivers/gpio/gpio-wm831x.c | 2 +-
> drivers/gpio/gpiolib.c | 2 +-
> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 3 ++-
> drivers/gpu/ipu-v3/ipu-common.c | 2 +-
> drivers/hid/hid-rmi.c | 2 +-
> drivers/i2c/busses/i2c-cht-wc.c | 2 +-
> drivers/i2c/i2c-core-base.c | 2 +-
> drivers/i2c/muxes/i2c-mux-pca954x.c | 2 +-
> drivers/ide/pmac.c | 2 +-
> drivers/iio/dummy/iio_dummy_evgen.c | 3 ++-
> drivers/input/rmi4/rmi_bus.c | 2 +-
> drivers/irqchip/irq-ath79-misc.c | 3 ++-
> drivers/irqchip/irq-bcm2835.c | 3 ++-
> drivers/irqchip/irq-csky-mpintc.c | 2 +-
> drivers/irqchip/irq-eznps.c | 2 +-
> drivers/irqchip/irq-mips-gic.c | 8 +++++---
> drivers/irqchip/irq-mmp.c | 4 ++--
> drivers/irqchip/irq-versatile-fpga.c | 2 +-
> drivers/irqchip/irq-vic.c | 2 +-
> drivers/macintosh/macio_asic.c | 2 +-
> drivers/memory/omap-gpmc.c | 2 +-
> drivers/mfd/ab8500-core.c | 2 +-
> drivers/mfd/arizona-irq.c | 5 +++--
> drivers/mfd/db8500-prcmu.c | 2 +-
> drivers/mfd/mfd-core.c | 2 +-
> drivers/mfd/stmpe.c | 5 +++--
> drivers/mfd/tc3589x.c | 2 +-
> drivers/mfd/tps6586x.c | 2 +-
> drivers/mfd/wm8994-irq.c | 5 +++--
> drivers/misc/cxl/irq.c | 2 +-
> drivers/misc/ocxl/afu_irq.c | 2 +-
> drivers/misc/ocxl/link.c | 2 +-
> drivers/net/dsa/mv88e6xxx/chip.c | 2 +-
> drivers/net/dsa/mv88e6xxx/global2.c | 2 +-
> drivers/net/dsa/qca/ar9331.c | 2 +-
> drivers/net/dsa/rtl8366rb.c | 3 ++-
> drivers/net/ethernet/ibm/ibmvnic.c | 4 ++--
> drivers/net/usb/lan78xx.c | 2 +-
> drivers/pci/controller/pci-ftpci100.c | 2 +-
> drivers/pci/controller/pci-tegra.c | 2 +-
> drivers/pci/controller/pcie-rcar-host.c | 2 +-
> drivers/pci/controller/pcie-xilinx-cpm.c | 4 ++--
> drivers/pci/controller/pcie-xilinx.c | 2 +-
> drivers/pinctrl/mediatek/mtk-eint.c | 2 +-
> drivers/pinctrl/nomadik/pinctrl-abx500.c | 3 ++-
> drivers/pinctrl/pinctrl-at91-pio4.c | 3 ++-
> drivers/pinctrl/pinctrl-rockchip.c | 2 +-
> drivers/pinctrl/samsung/pinctrl-samsung.c | 2 +-
> drivers/pinctrl/sunxi/pinctrl-sunxi.c | 2 +-
> drivers/power/supply/lp8788-charger.c | 2 +-
> drivers/rtc/rtc-lp8788.c | 2 +-
> drivers/rtc/rtc-max8997.c | 3 ++-
> drivers/rtc/rtc-max8998.c | 3 ++-
> drivers/scsi/cxlflash/ocxl_hw.c | 2 +-
> drivers/ssb/driver_gpio.c | 4 ++--
> drivers/staging/hikey9xx/hi6421-spmi-pmic.c | 2 +-
> drivers/staging/octeon-usb/octeon-hcd.c | 2 +-
> drivers/tty/hvc/hvsi.c | 2 +-
> drivers/tty/serial/pmac_zilog.c | 6 +++---
> drivers/watchdog/octeon-wdt-main.c | 2 +-
> include/linux/irqdomain.h | 3 ++-
> kernel/irq/irqdomain.c | 8 +++++---
> sound/soc/codecs/rt5677.c | 2 +-
> 123 files changed, 171 insertions(+), 146 deletions(-)
>
> --
> 2.28.0
>
prev parent reply other threads:[~2020-11-24 21:08 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-24 20:03 [PATCH 0/2] powerpc/pseries: fix MSI/X IRQ affinity on pseries Laurent Vivier
2020-11-24 20:03 ` [PATCH 1/2] genirq: add an affinity parameter to irq_create_mapping() Laurent Vivier
2020-11-24 22:19 ` Thomas Gleixner
2020-11-25 7:30 ` Laurent Vivier
2020-11-24 22:50 ` kernel test robot
2020-11-25 1:03 ` kernel test robot
2020-11-24 20:03 ` [PATCH 2/2] powerpc/pseries: pass MSI affinity " Laurent Vivier
2020-11-24 22:35 ` Thomas Gleixner
2020-11-24 21:08 ` Michael S. Tsirkin [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201124160747-mutt-send-email-mst@kernel.org \
--to=mst@redhat.com \
--cc=benh@kernel.crashing.org \
--cc=hch@lst.de \
--cc=linux-block@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linuxppc-dev@lists.ozlabs.org \
--cc=lvivier@redhat.com \
--cc=maz@kernel.org \
--cc=mpe@ellerman.id.au \
--cc=paulus@samba.org \
--cc=tglx@linutronix.de \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).