From: David Woodhouse <dwmw2@infradead.org> To: x86 <x86@kernel.org> Cc: iommu <iommu@lists.linux-foundation.org>, kvm <kvm@vger.kernel.org>, linux-hyperv@vger.kernel.org, Paolo Bonzini <pbonzini@redhat.com> Subject: [PATCH 0/13] Fix per-domain IRQ affinity, allow >255 CPUs on x86 without IRQ remapping Date: Mon, 05 Oct 2020 16:28:24 +0100 [thread overview] Message-ID: <77e64f977f559412f62b467fd062d051ea288f14.camel@infradead.org> (raw) [-- Attachment #1: Type: text/plain, Size: 3253 bytes --] Linux currently refuses to use >255 CPUs on x86 unless it has interrupt remapping. This is a bit gratuitous because it could use those extra CPUs just fine; it just can't target external interrupts at them. The only problem is that our generic IRQ domain code cann't cope with the concept of domains which can only target a subset of CPUs. The hyperv-iommu IRQ remapping driver works around this — not by actually doing any remapping, but just returning -EINVAL if the affinity is ever set to an unreachable CPU. This almost works, but ends up being a bit late because irq_set_affinity_locked() doesn't call into the irqchip driver immediately; the error only happens later. This patch series implements a per-domain "maximum affinity" set and uses it for the non-remapped IOAPIC and MSI domains on x86. As well as allowing more CPUs to be used without interrupt remapping, this also fixes the case where some IOAPICs or PCI devices aren't actually in scope of any active IOMMU and are operating without remapping. While we're at it, recognise that the 8-bit limit is a bit gratuitous and a hypervisor could offer at least 15 bits of APIC ID in the IOAPIC RTE and MSI address bits 11-5 without even needing to use remapping. David Woodhouse (13): x86/apic: Use x2apic in guest kernels even with unusable CPUs. x86/msi: Only use high bits of MSI address for DMAR unit x86/ioapic: Handle Extended Destination ID field in RTE x86/apic: Support 15 bits of APIC ID in IOAPIC/MSI where available genirq: Prepare for default affinity to be passed to __irq_alloc_descs() genirq: Add default_affinity argument to __irq_alloc_descs() irqdomain: Add max_affinity argument to irq_domain_alloc_descs() genirq: Add irq_domain_set_affinity() x86/irq: Add x86_non_ir_cpumask x86/irq: Limit IOAPIC and MSI domains' affinity without IR x86/smp: Allow more than 255 CPUs even without interrupt remapping iommu/irq_remapping: Kill most of hyperv-iommu.c now it's redundant x86/kvm: Add KVM_FEATURE_MSI_EXT_DEST_ID Documentation/virt/kvm/cpuid.rst | 4 + arch/x86/include/asm/apic.h | 1 + arch/x86/include/asm/io_apic.h | 3 +- arch/x86/include/asm/mpspec.h | 2 + arch/x86/include/asm/x86_init.h | 2 + arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kernel/apic/apic.c | 41 +++++++++- arch/x86/kernel/apic/io_apic.c | 23 ++++-- arch/x86/kernel/apic/msi.c | 44 +++++++++-- arch/x86/kernel/apic/x2apic_phys.c | 9 +++ arch/x86/kernel/kvm.c | 6 ++ arch/x86/kernel/x86_init.c | 1 + drivers/iommu/hyperv-iommu.c | 149 +---------------------------------- include/linux/interrupt.h | 2 + include/linux/irq.h | 10 ++- include/linux/irqdomain.h | 7 +- kernel/irq/devres.c | 8 +- kernel/irq/ipi.c | 2 +- kernel/irq/irqdesc.c | 29 ++++--- kernel/irq/irqdomain.c | 69 ++++++++++++++-- kernel/irq/manage.c | 19 ++++- 21 files changed, 240 insertions(+), 192 deletions(-) [-- Attachment #2: smime.p7s --] [-- Type: application/x-pkcs7-signature, Size: 5174 bytes --]
WARNING: multiple messages have this Message-ID (diff)
From: David Woodhouse <dwmw2@infradead.org> To: x86 <x86@kernel.org> Cc: Paolo Bonzini <pbonzini@redhat.com>, iommu <iommu@lists.linux-foundation.org>, linux-hyperv@vger.kernel.org, kvm <kvm@vger.kernel.org> Subject: [PATCH 0/13] Fix per-domain IRQ affinity, allow >255 CPUs on x86 without IRQ remapping Date: Mon, 05 Oct 2020 16:28:24 +0100 [thread overview] Message-ID: <77e64f977f559412f62b467fd062d051ea288f14.camel@infradead.org> (raw) [-- Attachment #1.1: Type: text/plain, Size: 3253 bytes --] Linux currently refuses to use >255 CPUs on x86 unless it has interrupt remapping. This is a bit gratuitous because it could use those extra CPUs just fine; it just can't target external interrupts at them. The only problem is that our generic IRQ domain code cann't cope with the concept of domains which can only target a subset of CPUs. The hyperv-iommu IRQ remapping driver works around this — not by actually doing any remapping, but just returning -EINVAL if the affinity is ever set to an unreachable CPU. This almost works, but ends up being a bit late because irq_set_affinity_locked() doesn't call into the irqchip driver immediately; the error only happens later. This patch series implements a per-domain "maximum affinity" set and uses it for the non-remapped IOAPIC and MSI domains on x86. As well as allowing more CPUs to be used without interrupt remapping, this also fixes the case where some IOAPICs or PCI devices aren't actually in scope of any active IOMMU and are operating without remapping. While we're at it, recognise that the 8-bit limit is a bit gratuitous and a hypervisor could offer at least 15 bits of APIC ID in the IOAPIC RTE and MSI address bits 11-5 without even needing to use remapping. David Woodhouse (13): x86/apic: Use x2apic in guest kernels even with unusable CPUs. x86/msi: Only use high bits of MSI address for DMAR unit x86/ioapic: Handle Extended Destination ID field in RTE x86/apic: Support 15 bits of APIC ID in IOAPIC/MSI where available genirq: Prepare for default affinity to be passed to __irq_alloc_descs() genirq: Add default_affinity argument to __irq_alloc_descs() irqdomain: Add max_affinity argument to irq_domain_alloc_descs() genirq: Add irq_domain_set_affinity() x86/irq: Add x86_non_ir_cpumask x86/irq: Limit IOAPIC and MSI domains' affinity without IR x86/smp: Allow more than 255 CPUs even without interrupt remapping iommu/irq_remapping: Kill most of hyperv-iommu.c now it's redundant x86/kvm: Add KVM_FEATURE_MSI_EXT_DEST_ID Documentation/virt/kvm/cpuid.rst | 4 + arch/x86/include/asm/apic.h | 1 + arch/x86/include/asm/io_apic.h | 3 +- arch/x86/include/asm/mpspec.h | 2 + arch/x86/include/asm/x86_init.h | 2 + arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kernel/apic/apic.c | 41 +++++++++- arch/x86/kernel/apic/io_apic.c | 23 ++++-- arch/x86/kernel/apic/msi.c | 44 +++++++++-- arch/x86/kernel/apic/x2apic_phys.c | 9 +++ arch/x86/kernel/kvm.c | 6 ++ arch/x86/kernel/x86_init.c | 1 + drivers/iommu/hyperv-iommu.c | 149 +---------------------------------- include/linux/interrupt.h | 2 + include/linux/irq.h | 10 ++- include/linux/irqdomain.h | 7 +- kernel/irq/devres.c | 8 +- kernel/irq/ipi.c | 2 +- kernel/irq/irqdesc.c | 29 ++++--- kernel/irq/irqdomain.c | 69 ++++++++++++++-- kernel/irq/manage.c | 19 ++++- 21 files changed, 240 insertions(+), 192 deletions(-) [-- Attachment #1.2: smime.p7s --] [-- Type: application/x-pkcs7-signature, Size: 5174 bytes --] [-- Attachment #2: Type: text/plain, Size: 156 bytes --] _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next reply other threads:[~2020-10-05 15:28 UTC|newest] Thread overview: 102+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-10-05 15:28 David Woodhouse [this message] 2020-10-05 15:28 ` [PATCH 0/13] Fix per-domain IRQ affinity, allow >255 CPUs on x86 without IRQ remapping David Woodhouse 2020-10-05 15:28 ` [PATCH 01/13] x86/apic: Use x2apic in guest kernels even with unusable CPUs David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 02/13] x86/msi: Only use high bits of MSI address for DMAR unit David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 20:45 ` Thomas Gleixner 2020-10-06 20:45 ` Thomas Gleixner 2020-10-05 15:28 ` [PATCH 03/13] x86/ioapic: Handle Extended Destination ID field in RTE David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 04/13] x86/apic: Support 15 bits of APIC ID in IOAPIC/MSI where available David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 05/13] genirq: Prepare for default affinity to be passed to __irq_alloc_descs() David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 21:01 ` Thomas Gleixner 2020-10-06 21:01 ` Thomas Gleixner 2020-10-06 21:07 ` David Woodhouse 2020-10-06 21:07 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 06/13] genirq: Add default_affinity argument " David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 21:06 ` Thomas Gleixner 2020-10-06 21:06 ` Thomas Gleixner 2020-10-05 15:28 ` [PATCH 07/13] irqdomain: Add max_affinity argument to irq_domain_alloc_descs() David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 21:26 ` Thomas Gleixner 2020-10-06 21:26 ` Thomas Gleixner 2020-10-07 7:19 ` David Woodhouse 2020-10-07 7:19 ` David Woodhouse 2020-10-07 13:37 ` Thomas Gleixner 2020-10-07 13:37 ` Thomas Gleixner 2020-10-07 14:10 ` David Woodhouse 2020-10-07 14:10 ` David Woodhouse 2020-10-07 15:57 ` Thomas Gleixner 2020-10-07 15:57 ` Thomas Gleixner 2020-10-07 16:11 ` David Woodhouse 2020-10-07 16:11 ` David Woodhouse 2020-10-07 20:53 ` Thomas Gleixner 2020-10-07 20:53 ` Thomas Gleixner 2020-10-08 7:21 ` David Woodhouse 2020-10-08 7:21 ` David Woodhouse 2020-10-08 9:34 ` Thomas Gleixner 2020-10-08 9:34 ` Thomas Gleixner 2020-10-08 11:10 ` David Woodhouse 2020-10-08 11:10 ` David Woodhouse 2020-10-08 12:40 ` Thomas Gleixner 2020-10-08 12:40 ` Thomas Gleixner 2020-10-09 7:54 ` David Woodhouse 2020-10-09 7:54 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 08/13] genirq: Add irq_domain_set_affinity() David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 21:32 ` Thomas Gleixner 2020-10-06 21:32 ` Thomas Gleixner 2020-10-07 7:22 ` David Woodhouse 2020-10-07 7:22 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 09/13] x86/irq: Add x86_non_ir_cpumask David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 21:42 ` Thomas Gleixner 2020-10-06 21:42 ` Thomas Gleixner 2020-10-07 7:25 ` David Woodhouse 2020-10-07 7:25 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 10/13] x86/irq: Limit IOAPIC and MSI domains' affinity without IR David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-06 21:54 ` Thomas Gleixner 2020-10-06 21:54 ` Thomas Gleixner 2020-10-07 7:48 ` David Woodhouse 2020-10-07 7:48 ` David Woodhouse 2020-10-07 12:59 ` Thomas Gleixner 2020-10-07 12:59 ` Thomas Gleixner 2020-10-07 13:08 ` David Woodhouse 2020-10-07 13:08 ` David Woodhouse 2020-10-07 14:05 ` Thomas Gleixner 2020-10-07 14:05 ` Thomas Gleixner 2020-10-07 14:23 ` David Woodhouse 2020-10-07 14:23 ` David Woodhouse 2020-10-07 16:02 ` Thomas Gleixner 2020-10-07 16:02 ` Thomas Gleixner 2020-10-07 16:15 ` David Woodhouse 2020-10-07 16:15 ` David Woodhouse 2020-10-07 15:05 ` David Woodhouse 2020-10-07 15:05 ` David Woodhouse 2020-10-07 15:25 ` Thomas Gleixner 2020-10-07 15:25 ` Thomas Gleixner 2020-10-07 15:46 ` David Woodhouse 2020-10-07 15:46 ` David Woodhouse 2020-10-07 17:23 ` Thomas Gleixner 2020-10-07 17:23 ` Thomas Gleixner 2020-10-07 17:34 ` David Woodhouse 2020-10-07 17:34 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 11/13] x86/smp: Allow more than 255 CPUs even without interrupt remapping David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 12/13] iommu/irq_remapping: Kill most of hyperv-iommu.c now it's redundant David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-05 15:28 ` [PATCH 13/13] x86/kvm: Add KVM_FEATURE_MSI_EXT_DEST_ID David Woodhouse 2020-10-05 15:28 ` David Woodhouse 2020-10-07 8:14 ` Paolo Bonzini 2020-10-07 8:14 ` Paolo Bonzini 2020-10-07 8:59 ` David Woodhouse 2020-10-07 8:59 ` David Woodhouse 2020-10-07 11:15 ` Paolo Bonzini 2020-10-07 11:15 ` Paolo Bonzini 2020-10-07 12:04 ` David Woodhouse 2020-10-07 12:04 ` David Woodhouse
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=77e64f977f559412f62b467fd062d051ea288f14.camel@infradead.org \ --to=dwmw2@infradead.org \ --cc=iommu@lists.linux-foundation.org \ --cc=kvm@vger.kernel.org \ --cc=linux-hyperv@vger.kernel.org \ --cc=pbonzini@redhat.com \ --cc=x86@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.