From: Anup Patel <apatel@ventanamicro.com> To: Palmer Dabbelt <palmer@dabbelt.com>, Paul Walmsley <paul.walmsley@sifive.com>, Thomas Gleixner <tglx@linutronix.de>, Marc Zyngier <maz@kernel.org>, Rob Herring <robh+dt@kernel.org>, Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>, Robin Murphy <robin.murphy@arm.com>, Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>, Frank Rowand <frowand.list@gmail.com> Cc: Atish Patra <atishp@atishpatra.org>, Andrew Jones <ajones@ventanamicro.com>, Anup Patel <anup@brainfault.org>, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux.dev, Anup Patel <apatel@ventanamicro.com>, Vincent Chen <vincent.chen@sifive.com> Subject: [PATCH v3 07/11] irqchip/riscv-imsic: Improve IOMMU DMA support Date: Mon, 8 May 2023 19:58:38 +0530 [thread overview] Message-ID: <20230508142842.854564-8-apatel@ventanamicro.com> (raw) In-Reply-To: <20230508142842.854564-1-apatel@ventanamicro.com> We have a separate RISC-V IMSIC MSI address for each CPU so changing MSI (or IRQ) affinity results in re-programming of MSI address in the PCIe (or platform) device. Currently, the iommu_dma_prepare_msi() is called only once at the time of IRQ allocation so IOMMU DMA domain will only have mapping for one MSI page. This means iommu_dma_compose_msi_msg() called by imsic_irq_compose_msi_msg() will always use the same MSI page irrespective to target CPU MSI address. In other words, changing MSI (or IRQ) affinity for device using IOMMU DMA domain will not work. To address above issue, we do the following: 1) Map MSI pages for all CPUs in imsic_irq_domain_alloc() using iommu_dma_prepare_msi(). 2) Add a new iommu_dma_select_msi() API to select a specific MSI page from a set of already mapped MSI pages. 3) Use iommu_dma_select_msi() to select a specific MSI page before calling iommu_dma_compose_msi_msg() in imsic_irq_compose_msi_msg(). Reported-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> --- drivers/iommu/dma-iommu.c | 38 +++++++++++++++++++++++++++++++ drivers/irqchip/irq-riscv-imsic.c | 27 ++++++++++++---------- include/linux/iommu.h | 6 +++++ 3 files changed, 59 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..07782c77a6eb 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1677,6 +1677,44 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) return 0; } +/** + * iommu_dma_select_msi() - Select a MSI page from a set of + * already mapped MSI pages in the IOMMU domain. + * + * @desc: MSI descriptor prepared by iommu_dma_prepare_msi() + * @msi_addr: physical address of the MSI page to be selected + * + * Return: 0 on success or negative error code if the select failed. + */ +int iommu_dma_select_msi(struct msi_desc *desc, phys_addr_t msi_addr) +{ + struct device *dev = msi_desc_to_dev(desc); + struct iommu_domain *domain = iommu_get_domain_for_dev(dev); + const struct iommu_dma_msi_page *msi_page; + struct iommu_dma_cookie *cookie; + + if (!domain || !domain->iova_cookie) { + desc->iommu_cookie = NULL; + return 0; + } + + cookie = domain->iova_cookie; + msi_addr &= ~(phys_addr_t)(cookie_msi_granule(cookie) - 1); + + msi_page = msi_desc_get_iommu_cookie(desc); + if (msi_page && msi_page->phys == msi_addr) + return 0; + + list_for_each_entry(msi_page, &cookie->msi_page_list, list) { + if (msi_page->phys == msi_addr) { + msi_desc_set_iommu_cookie(desc, msi_page); + return 0; + } + } + + return -ENOENT; +} + /** * iommu_dma_compose_msi_msg() - Apply translation to an MSI message * @desc: MSI descriptor prepared by iommu_dma_prepare_msi() diff --git a/drivers/irqchip/irq-riscv-imsic.c b/drivers/irqchip/irq-riscv-imsic.c index 30247c84a6b0..ec61c599e0c5 100644 --- a/drivers/irqchip/irq-riscv-imsic.c +++ b/drivers/irqchip/irq-riscv-imsic.c @@ -446,6 +446,10 @@ static void imsic_irq_compose_msi_msg(struct irq_data *d, if (WARN_ON(err)) return; + err = iommu_dma_select_msi(desc, msi_addr); + if (WARN_ON(err)) + return; + msg->address_hi = upper_32_bits(msi_addr); msg->address_lo = lower_32_bits(msi_addr); msg->data = d->hwirq; @@ -493,11 +497,18 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, int i, hwirq, err = 0; unsigned int cpu; - err = imsic_get_cpu(&imsic->lmask, false, &cpu); - if (err) - return err; + /* Map MSI address of all CPUs */ + for_each_cpu(cpu, &imsic->lmask) { + err = imsic_cpu_page_phys(cpu, 0, &msi_addr); + if (err) + return err; - err = imsic_cpu_page_phys(cpu, 0, &msi_addr); + err = iommu_dma_prepare_msi(info->desc, msi_addr); + if (err) + return err; + } + + err = imsic_get_cpu(&imsic->lmask, false, &cpu); if (err) return err; @@ -505,10 +516,6 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, if (hwirq < 0) return hwirq; - err = iommu_dma_prepare_msi(info->desc, msi_addr); - if (err) - goto fail; - for (i = 0; i < nr_irqs; i++) { imsic_id_set_target(hwirq + i, cpu); irq_domain_set_info(domain, virq + i, hwirq + i, @@ -528,10 +535,6 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, } return 0; - -fail: - imsic_ids_free(hwirq, get_count_order(nr_irqs)); - return err; } static void imsic_irq_domain_free(struct irq_domain *domain, diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e8c9a7da1060..41e8613832ab 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -1117,6 +1117,7 @@ void iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 dma_limit); int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base); int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr); +int iommu_dma_select_msi(struct msi_desc *desc, phys_addr_t msi_addr); void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg); #else /* CONFIG_IOMMU_DMA */ @@ -1138,6 +1139,11 @@ static inline int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_a return 0; } +static inline int iommu_dma_select_msi(struct msi_desc *desc, phys_addr_t msi_addr) +{ + return 0; +} + static inline void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg) { } -- 2.34.1
WARNING: multiple messages have this Message-ID (diff)
From: Anup Patel <apatel@ventanamicro.com> To: Palmer Dabbelt <palmer@dabbelt.com>, Paul Walmsley <paul.walmsley@sifive.com>, Thomas Gleixner <tglx@linutronix.de>, Marc Zyngier <maz@kernel.org>, Rob Herring <robh+dt@kernel.org>, Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>, Robin Murphy <robin.murphy@arm.com>, Joerg Roedel <joro@8bytes.org>, Will Deacon <will@kernel.org>, Frank Rowand <frowand.list@gmail.com> Cc: Atish Patra <atishp@atishpatra.org>, Andrew Jones <ajones@ventanamicro.com>, Anup Patel <anup@brainfault.org>, linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org, iommu@lists.linux.dev, Anup Patel <apatel@ventanamicro.com>, Vincent Chen <vincent.chen@sifive.com> Subject: [PATCH v3 07/11] irqchip/riscv-imsic: Improve IOMMU DMA support Date: Mon, 8 May 2023 19:58:38 +0530 [thread overview] Message-ID: <20230508142842.854564-8-apatel@ventanamicro.com> (raw) In-Reply-To: <20230508142842.854564-1-apatel@ventanamicro.com> We have a separate RISC-V IMSIC MSI address for each CPU so changing MSI (or IRQ) affinity results in re-programming of MSI address in the PCIe (or platform) device. Currently, the iommu_dma_prepare_msi() is called only once at the time of IRQ allocation so IOMMU DMA domain will only have mapping for one MSI page. This means iommu_dma_compose_msi_msg() called by imsic_irq_compose_msi_msg() will always use the same MSI page irrespective to target CPU MSI address. In other words, changing MSI (or IRQ) affinity for device using IOMMU DMA domain will not work. To address above issue, we do the following: 1) Map MSI pages for all CPUs in imsic_irq_domain_alloc() using iommu_dma_prepare_msi(). 2) Add a new iommu_dma_select_msi() API to select a specific MSI page from a set of already mapped MSI pages. 3) Use iommu_dma_select_msi() to select a specific MSI page before calling iommu_dma_compose_msi_msg() in imsic_irq_compose_msi_msg(). Reported-by: Vincent Chen <vincent.chen@sifive.com> Signed-off-by: Anup Patel <apatel@ventanamicro.com> --- drivers/iommu/dma-iommu.c | 38 +++++++++++++++++++++++++++++++ drivers/irqchip/irq-riscv-imsic.c | 27 ++++++++++++---------- include/linux/iommu.h | 6 +++++ 3 files changed, 59 insertions(+), 12 deletions(-) diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c index 7a9f0b0bddbd..07782c77a6eb 100644 --- a/drivers/iommu/dma-iommu.c +++ b/drivers/iommu/dma-iommu.c @@ -1677,6 +1677,44 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr) return 0; } +/** + * iommu_dma_select_msi() - Select a MSI page from a set of + * already mapped MSI pages in the IOMMU domain. + * + * @desc: MSI descriptor prepared by iommu_dma_prepare_msi() + * @msi_addr: physical address of the MSI page to be selected + * + * Return: 0 on success or negative error code if the select failed. + */ +int iommu_dma_select_msi(struct msi_desc *desc, phys_addr_t msi_addr) +{ + struct device *dev = msi_desc_to_dev(desc); + struct iommu_domain *domain = iommu_get_domain_for_dev(dev); + const struct iommu_dma_msi_page *msi_page; + struct iommu_dma_cookie *cookie; + + if (!domain || !domain->iova_cookie) { + desc->iommu_cookie = NULL; + return 0; + } + + cookie = domain->iova_cookie; + msi_addr &= ~(phys_addr_t)(cookie_msi_granule(cookie) - 1); + + msi_page = msi_desc_get_iommu_cookie(desc); + if (msi_page && msi_page->phys == msi_addr) + return 0; + + list_for_each_entry(msi_page, &cookie->msi_page_list, list) { + if (msi_page->phys == msi_addr) { + msi_desc_set_iommu_cookie(desc, msi_page); + return 0; + } + } + + return -ENOENT; +} + /** * iommu_dma_compose_msi_msg() - Apply translation to an MSI message * @desc: MSI descriptor prepared by iommu_dma_prepare_msi() diff --git a/drivers/irqchip/irq-riscv-imsic.c b/drivers/irqchip/irq-riscv-imsic.c index 30247c84a6b0..ec61c599e0c5 100644 --- a/drivers/irqchip/irq-riscv-imsic.c +++ b/drivers/irqchip/irq-riscv-imsic.c @@ -446,6 +446,10 @@ static void imsic_irq_compose_msi_msg(struct irq_data *d, if (WARN_ON(err)) return; + err = iommu_dma_select_msi(desc, msi_addr); + if (WARN_ON(err)) + return; + msg->address_hi = upper_32_bits(msi_addr); msg->address_lo = lower_32_bits(msi_addr); msg->data = d->hwirq; @@ -493,11 +497,18 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, int i, hwirq, err = 0; unsigned int cpu; - err = imsic_get_cpu(&imsic->lmask, false, &cpu); - if (err) - return err; + /* Map MSI address of all CPUs */ + for_each_cpu(cpu, &imsic->lmask) { + err = imsic_cpu_page_phys(cpu, 0, &msi_addr); + if (err) + return err; - err = imsic_cpu_page_phys(cpu, 0, &msi_addr); + err = iommu_dma_prepare_msi(info->desc, msi_addr); + if (err) + return err; + } + + err = imsic_get_cpu(&imsic->lmask, false, &cpu); if (err) return err; @@ -505,10 +516,6 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, if (hwirq < 0) return hwirq; - err = iommu_dma_prepare_msi(info->desc, msi_addr); - if (err) - goto fail; - for (i = 0; i < nr_irqs; i++) { imsic_id_set_target(hwirq + i, cpu); irq_domain_set_info(domain, virq + i, hwirq + i, @@ -528,10 +535,6 @@ static int imsic_irq_domain_alloc(struct irq_domain *domain, } return 0; - -fail: - imsic_ids_free(hwirq, get_count_order(nr_irqs)); - return err; } static void imsic_irq_domain_free(struct irq_domain *domain, diff --git a/include/linux/iommu.h b/include/linux/iommu.h index e8c9a7da1060..41e8613832ab 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -1117,6 +1117,7 @@ void iommu_setup_dma_ops(struct device *dev, u64 dma_base, u64 dma_limit); int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base); int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr); +int iommu_dma_select_msi(struct msi_desc *desc, phys_addr_t msi_addr); void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg); #else /* CONFIG_IOMMU_DMA */ @@ -1138,6 +1139,11 @@ static inline int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_a return 0; } +static inline int iommu_dma_select_msi(struct msi_desc *desc, phys_addr_t msi_addr) +{ + return 0; +} + static inline void iommu_dma_compose_msi_msg(struct msi_desc *desc, struct msi_msg *msg) { } -- 2.34.1 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv
next prev parent reply other threads:[~2023-05-08 14:30 UTC|newest] Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top 2023-05-08 14:28 [PATCH v3 00/11] Linux RISC-V AIA Support Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 01/11] RISC-V: Add riscv_fw_parent_hartid() function Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-10 12:45 ` Conor Dooley 2023-05-10 12:45 ` Conor Dooley 2023-06-13 8:05 ` Anup Patel 2023-06-13 8:05 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 02/11] of/irq: Set FWNODE_FLAG_BEST_EFFORT for the interrupt controller DT nodes Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-06-08 18:28 ` Rob Herring 2023-06-08 18:28 ` Rob Herring 2023-06-08 20:04 ` Saravana Kannan 2023-06-08 20:04 ` Saravana Kannan 2023-06-09 11:40 ` Anup Patel 2023-06-09 11:40 ` Anup Patel 2023-06-09 21:17 ` Saravana Kannan 2023-06-09 21:17 ` Saravana Kannan 2023-06-13 4:42 ` Anup Patel 2023-06-13 4:42 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 03/11] irqchip/riscv-intc: Add support for RISC-V AIA Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 04/11] dt-bindings: interrupt-controller: Add RISC-V incoming MSI controller Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-10 12:16 ` Conor Dooley 2023-05-10 12:16 ` Conor Dooley 2023-06-13 8:18 ` Anup Patel 2023-06-13 8:18 ` Anup Patel 2023-05-11 9:49 ` Krzysztof Kozlowski 2023-05-11 9:49 ` Krzysztof Kozlowski 2023-05-08 14:28 ` [PATCH v3 05/11] irqchip: Add RISC-V incoming MSI controller driver Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 06/11] irqchip/riscv-imsic: Add support for PCI MSI irqdomain Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-08 14:28 ` Anup Patel [this message] 2023-05-08 14:28 ` [PATCH v3 07/11] irqchip/riscv-imsic: Improve IOMMU DMA support Anup Patel 2023-05-10 10:48 ` Robin Murphy 2023-05-10 10:48 ` Robin Murphy 2023-05-10 15:12 ` Anup Patel 2023-05-10 15:12 ` Anup Patel 2023-05-15 12:53 ` Jason Gunthorpe 2023-05-15 12:53 ` Jason Gunthorpe 2023-06-13 7:55 ` Anup Patel 2023-06-13 7:55 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 08/11] dt-bindings: interrupt-controller: Add RISC-V advanced PLIC Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-10 15:41 ` Conor Dooley 2023-05-10 15:41 ` Conor Dooley 2023-06-13 10:37 ` Anup Patel 2023-06-13 10:37 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 09/11] irqchip: Add RISC-V advanced PLIC driver Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-08 14:28 ` [PATCH v3 10/11] RISC-V: Select APLIC and IMSIC drivers Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-10 12:20 ` Conor Dooley 2023-05-10 12:20 ` Conor Dooley 2023-05-08 14:28 ` [PATCH v3 11/11] MAINTAINERS: Add entry for RISC-V AIA drivers Anup Patel 2023-05-08 14:28 ` Anup Patel 2023-05-10 10:24 ` [PATCH v3 00/11] Linux RISC-V AIA Support Conor Dooley 2023-05-10 10:24 ` Conor Dooley
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20230508142842.854564-8-apatel@ventanamicro.com \ --to=apatel@ventanamicro.com \ --cc=ajones@ventanamicro.com \ --cc=anup@brainfault.org \ --cc=atishp@atishpatra.org \ --cc=devicetree@vger.kernel.org \ --cc=frowand.list@gmail.com \ --cc=iommu@lists.linux.dev \ --cc=joro@8bytes.org \ --cc=krzysztof.kozlowski+dt@linaro.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-riscv@lists.infradead.org \ --cc=maz@kernel.org \ --cc=palmer@dabbelt.com \ --cc=paul.walmsley@sifive.com \ --cc=robh+dt@kernel.org \ --cc=robin.murphy@arm.com \ --cc=tglx@linutronix.de \ --cc=vincent.chen@sifive.com \ --cc=will@kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.