All of lore.kernel.org
 help / color / mirror / Atom feed
From: Laurent Vivier <lvivier@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: Thomas Gleixner <tglx@linutronix.de>,
	Paul Mackerras <paulus@samba.org>,
	Benjamin Herrenschmidt <benh@kernel.crashing.org>,
	Michael Ellerman <mpe@ellerman.id.au>,
	linux-block@vger.kernel.org,
	"Michael S . Tsirkin" <mst@redhat.com>,
	Marc Zyngier <maz@kernel.org>,
	linuxppc-dev@lists.ozlabs.org, linux-pci@vger.kernel.org,
	Christoph Hellwig <hch@lst.de>,
	Laurent Vivier <lvivier@redhat.com>
Subject: [PATCH 2/2] powerpc/pseries: pass MSI affinity to irq_create_mapping()
Date: Tue, 24 Nov 2020 21:03:08 +0100	[thread overview]
Message-ID: <20201124200308.1110744-3-lvivier@redhat.com> (raw)
In-Reply-To: <20201124200308.1110744-1-lvivier@redhat.com>

With virtio multiqueue, normally each queue IRQ is mapped to a CPU.

But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity")
this is broken on pseries.

The affinity is correctly computed in msi_desc but this is not applied
to the system IRQs.

It appears the affinity is correctly passed to rtas_setup_msi_irqs() but
lost at this point and never passed to irq_domain_alloc_descs()
(see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation"))
because irq_create_mapping() doesn't take an affinity parameter.

As the previous patch has added the affinity parameter to
irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs()
to irq_domain_alloc_descs().

With this change, the virtqueues are correctly dispatched between the CPUs
on pseries.

This problem cannot be shown on x86_64 for two reasons:

- the call path traverses arch_setup_msi_irqs() that is arch specific:

   virtscsi_probe()
      virtscsi_init()
         vp_modern_find_vqs()
            vp_find_vqs()
               vp_find_vqs_msix()
                  pci_alloc_irq_vectors_affinity()
                     __pci_enable_msix_range()
                        pci_msi_setup_msi_irqs()
                           arch_setup_msi_irqs()
                              rtas_setup_msi_irqs()
                                 irq_create_mapping()
                                    irq_domain_alloc_descs()
                                      __irq_alloc_descs()

- and x86_64 has CONFIG_PCI_MSI_IRQ_DOMAIN that uses another path:

   virtscsi_probe()
      virtscsi_init()
         vp_modern_find_vqs()
            vp_find_vqs()
               vp_find_vqs_msix()
                  pci_alloc_irq_vectors_affinity()
                     __pci_enable_msix_range()
                        __msi_domain_alloc_irqs()
                           __irq_domain_alloc_irqs()
                              __irq_alloc_descs()

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
---
 arch/powerpc/platforms/pseries/msi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
index 42ba08eaea91..58197f92c6a2 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -458,7 +458,7 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int nvec_in, int type)
 			return hwirq;
 		}
 
-		virq = irq_create_mapping(NULL, hwirq, NULL);
+		virq = irq_create_mapping(NULL, hwirq, entry->affinity);
 
 		if (!virq) {
 			pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq);
-- 
2.28.0


WARNING: multiple messages have this Message-ID (diff)
From: Laurent Vivier <lvivier@redhat.com>
To: linux-kernel@vger.kernel.org
Cc: Laurent Vivier <lvivier@redhat.com>,
	Marc Zyngier <maz@kernel.org>,
	"Michael S . Tsirkin" <mst@redhat.com>,
	linux-pci@vger.kernel.org, linux-block@vger.kernel.org,
	Paul Mackerras <paulus@samba.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	linuxppc-dev@lists.ozlabs.org, Christoph Hellwig <hch@lst.de>
Subject: [PATCH 2/2] powerpc/pseries: pass MSI affinity to irq_create_mapping()
Date: Tue, 24 Nov 2020 21:03:08 +0100	[thread overview]
Message-ID: <20201124200308.1110744-3-lvivier@redhat.com> (raw)
In-Reply-To: <20201124200308.1110744-1-lvivier@redhat.com>

With virtio multiqueue, normally each queue IRQ is mapped to a CPU.

But since commit 0d9f0a52c8b9f ("virtio_scsi: use virtio IRQ affinity")
this is broken on pseries.

The affinity is correctly computed in msi_desc but this is not applied
to the system IRQs.

It appears the affinity is correctly passed to rtas_setup_msi_irqs() but
lost at this point and never passed to irq_domain_alloc_descs()
(see commit 06ee6d571f0e ("genirq: Add affinity hint to irq allocation"))
because irq_create_mapping() doesn't take an affinity parameter.

As the previous patch has added the affinity parameter to
irq_create_mapping() we can forward the affinity from rtas_setup_msi_irqs()
to irq_domain_alloc_descs().

With this change, the virtqueues are correctly dispatched between the CPUs
on pseries.

This problem cannot be shown on x86_64 for two reasons:

- the call path traverses arch_setup_msi_irqs() that is arch specific:

   virtscsi_probe()
      virtscsi_init()
         vp_modern_find_vqs()
            vp_find_vqs()
               vp_find_vqs_msix()
                  pci_alloc_irq_vectors_affinity()
                     __pci_enable_msix_range()
                        pci_msi_setup_msi_irqs()
                           arch_setup_msi_irqs()
                              rtas_setup_msi_irqs()
                                 irq_create_mapping()
                                    irq_domain_alloc_descs()
                                      __irq_alloc_descs()

- and x86_64 has CONFIG_PCI_MSI_IRQ_DOMAIN that uses another path:

   virtscsi_probe()
      virtscsi_init()
         vp_modern_find_vqs()
            vp_find_vqs()
               vp_find_vqs_msix()
                  pci_alloc_irq_vectors_affinity()
                     __pci_enable_msix_range()
                        __msi_domain_alloc_irqs()
                           __irq_domain_alloc_irqs()
                              __irq_alloc_descs()

Signed-off-by: Laurent Vivier <lvivier@redhat.com>
---
 arch/powerpc/platforms/pseries/msi.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/powerpc/platforms/pseries/msi.c b/arch/powerpc/platforms/pseries/msi.c
index 42ba08eaea91..58197f92c6a2 100644
--- a/arch/powerpc/platforms/pseries/msi.c
+++ b/arch/powerpc/platforms/pseries/msi.c
@@ -458,7 +458,7 @@ static int rtas_setup_msi_irqs(struct pci_dev *pdev, int nvec_in, int type)
 			return hwirq;
 		}
 
-		virq = irq_create_mapping(NULL, hwirq, NULL);
+		virq = irq_create_mapping(NULL, hwirq, entry->affinity);
 
 		if (!virq) {
 			pr_debug("rtas_msi: Failed mapping hwirq %d\n", hwirq);
-- 
2.28.0


  parent reply	other threads:[~2020-11-24 20:03 UTC|newest]

Thread overview: 20+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-11-24 20:03 [PATCH 0/2] powerpc/pseries: fix MSI/X IRQ affinity on pseries Laurent Vivier
2020-11-24 20:03 ` Laurent Vivier
2020-11-24 20:03 ` [PATCH 1/2] genirq: add an affinity parameter to irq_create_mapping() Laurent Vivier
2020-11-24 20:03   ` Laurent Vivier
2020-11-24 22:19   ` Thomas Gleixner
2020-11-24 22:19     ` Thomas Gleixner
2020-11-25  7:30     ` Laurent Vivier
2020-11-25  7:30       ` Laurent Vivier
2020-11-24 22:50   ` kernel test robot
2020-11-24 22:50     ` kernel test robot
2020-11-24 22:50     ` kernel test robot
2020-11-25  1:03   ` kernel test robot
2020-11-25  1:03     ` kernel test robot
2020-11-25  1:03     ` kernel test robot
2020-11-24 20:03 ` Laurent Vivier [this message]
2020-11-24 20:03   ` [PATCH 2/2] powerpc/pseries: pass MSI affinity " Laurent Vivier
2020-11-24 22:35   ` Thomas Gleixner
2020-11-24 22:35     ` Thomas Gleixner
2020-11-24 21:08 ` [PATCH 0/2] powerpc/pseries: fix MSI/X IRQ affinity on pseries Michael S. Tsirkin
2020-11-24 21:08   ` Michael S. Tsirkin

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20201124200308.1110744-3-lvivier@redhat.com \
    --to=lvivier@redhat.com \
    --cc=benh@kernel.crashing.org \
    --cc=hch@lst.de \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linuxppc-dev@lists.ozlabs.org \
    --cc=maz@kernel.org \
    --cc=mpe@ellerman.id.au \
    --cc=mst@redhat.com \
    --cc=paulus@samba.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.