From: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
To: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Cc: "Rob Herring" <robh@kernel.org>,
"Bjorn Helgaas" <bhelgaas@google.com>,
"Lorenzo Pieralisi" <lorenzo.pieralisi@arm.com>,
"Jingoo Han" <jingoohan1@gmail.com>,
"Gustavo Pimentel" <gustavo.pimentel@synopsys.com>,
"Lorenzo Pieralisi" <lpieralisi@kernel.org>,
"Krzysztof Wilczyński" <kw@linux.com>,
"Serge Semin" <fancer.lancer@gmail.com>,
"Alexey Malahov" <Alexey.Malahov@baikalelectronics.ru>,
"Pavel Parkhomenko" <Pavel.Parkhomenko@baikalelectronics.ru>,
"Frank Li" <Frank.Li@nxp.com>, "Rob Herring" <robh+dt@kernel.org>,
linux-pci@vger.kernel.org, devicetree@vger.kernel.org,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host
Date: Mon, 1 Aug 2022 19:30:47 +0530 [thread overview]
Message-ID: <20220801140047.GO93763@thinkpad> (raw)
In-Reply-To: <20220624143947.8991-16-Sergey.Semin@baikalelectronics.ru>
On Fri, Jun 24, 2022 at 05:39:47PM +0300, Serge Semin wrote:
> In accordance with the generic PCIe Root Port DT-bindings the "dma-ranges"
> property has the same format as the "ranges" property. The only difference
> is in their semantics. The "dma-ranges" property describes the PCIe-to-CPU
> memory mapping in opposite to the CPU-to-PCIe mapping of the "ranges"
> property. Even though the DW PCIe controllers are normally equipped with
> the internal Address Translation Unit which inbound and outbound tables
> can be used to implement both properties semantics, it was surprising for
> me to discover that the host-related part of the DW PCIe driver currently
> supports the "ranges" property only while the "dma-ranges" windows are
> just ignored. Having the "dma-ranges" supported in the driver would be
> very handy for the platforms, that don't tolerate the 1:1 CPU-PCIe memory
> mapping and require a customized PCIe memory layout. So let's fix that by
> introducing the "dma-ranges" property support.
>
> First of all we suggest to rename the dw_pcie_prog_inbound_atu() method to
> dw_pcie_prog_ep_inbound_atu() and create a new version of the
> dw_pcie_prog_inbound_atu() function. Thus we'll have two methods for the
> RC and EP controllers respectively in the same way as it has been
> developed for the outbound ATU setup methods.
>
> Secondly aside with the memory window index and type the new
> dw_pcie_prog_inbound_atu() function will accept CPU address, PCIe address
> and size as its arguments. These parameters define the PCIe and CPU memory
> ranges which will be used to setup the respective inbound ATU mapping. The
> passed parameters need to be verified against the ATU ranges constraints
> in the same way as it is done for the outbound ranges.
>
> Finally the DMA-ranges detected for the PCIe controller need to be
> converted to the inbound ATU entries during the host controller
> initialization procedure. It will be done in the framework of the
> dw_pcie_iatu_setup() method. Note before setting the inbound ranges up we
> need to disable all the inbound ATU entries in order to prevent unexpected
> PCIe TLPs translations defined by some third party software like
> bootloaders.
>
> Signed-off-by: Serge Semin <Sergey.Semin@baikalelectronics.ru>
Small nitpick below,
Reviewed-by: Manivannan Sadhasivam <manivannan.sadhasivam@linaro.org>
Thanks,
Mani
> Reviewed-by: Rob Herring <robh@kernel.org>
>
> ---
>
> Changelog v3:
> - Drop inbound iATU window size alignment constraint. (@Manivannan)
> ---
> .../pci/controller/dwc/pcie-designware-ep.c | 4 +-
> .../pci/controller/dwc/pcie-designware-host.c | 32 ++++++++++-
> drivers/pci/controller/dwc/pcie-designware.c | 56 ++++++++++++++++++-
> drivers/pci/controller/dwc/pcie-designware.h | 6 +-
> 4 files changed, 89 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/pci/controller/dwc/pcie-designware-ep.c b/drivers/pci/controller/dwc/pcie-designware-ep.c
> index 627c4b69878c..441feff1917a 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-ep.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-ep.c
> @@ -167,8 +167,8 @@ static int dw_pcie_ep_inbound_atu(struct dw_pcie_ep *ep, u8 func_no, int type,
> return -EINVAL;
> }
>
> - ret = dw_pcie_prog_inbound_atu(pci, func_no, free_win, type,
> - cpu_addr, bar);
> + ret = dw_pcie_prog_ep_inbound_atu(pci, func_no, free_win, type,
> + cpu_addr, bar);
> if (ret < 0) {
> dev_err(pci->dev, "Failed to program IB window\n");
> return ret;
> diff --git a/drivers/pci/controller/dwc/pcie-designware-host.c b/drivers/pci/controller/dwc/pcie-designware-host.c
> index 6993ce9e856d..2fbe9dc11634 100644
> --- a/drivers/pci/controller/dwc/pcie-designware-host.c
> +++ b/drivers/pci/controller/dwc/pcie-designware-host.c
> @@ -581,12 +581,15 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
> }
>
> /*
> - * Ensure all outbound windows are disabled before proceeding with
> - * the MEM/IO ranges setups.
> + * Ensure all out/inbound windows are disabled before proceeding with
> + * the MEM/IO (dma-)ranges setups.
> */
> for (i = 0; i < pci->num_ob_windows; i++)
> dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_OB, i);
>
> + for (i = 0; i < pci->num_ib_windows; i++)
> + dw_pcie_disable_atu(pci, PCIE_ATU_REGION_DIR_IB, i);
> +
> i = 0;
> resource_list_for_each_entry(entry, &pp->bridge->windows) {
> if (resource_type(entry->res) != IORESOURCE_MEM)
> @@ -623,9 +626,32 @@ static int dw_pcie_iatu_setup(struct dw_pcie_rp *pp)
> }
>
> if (pci->num_ob_windows <= i)
> - dev_warn(pci->dev, "Resources exceed number of ATU entries (%d)\n",
> + dev_warn(pci->dev, "Ranges exceed outbound iATU size (%d)\n",
> pci->num_ob_windows);
>
I think a comment here explaining what's going on below would be helpful in the
future.
> + i = 0;
> + resource_list_for_each_entry(entry, &pp->bridge->dma_ranges) {
> + if (resource_type(entry->res) != IORESOURCE_MEM)
> + continue;
> +
> + if (pci->num_ib_windows <= i)
> + break;
> +
> + ret = dw_pcie_prog_inbound_atu(pci, i++, PCIE_ATU_TYPE_MEM,
> + entry->res->start,
> + entry->res->start - entry->offset,
> + resource_size(entry->res));
> + if (ret) {
> + dev_err(pci->dev, "Failed to set DMA range %pr\n",
> + entry->res);
> + return ret;
> + }
> + }
> +
> + if (pci->num_ib_windows <= i)
> + dev_warn(pci->dev, "Dma-ranges exceed inbound iATU size (%u)\n",
"dma-ranges"
> + pci->num_ib_windows);
> +
> return 0;
> }
>
> diff --git a/drivers/pci/controller/dwc/pcie-designware.c b/drivers/pci/controller/dwc/pcie-designware.c
> index 9c622b635fdd..7a5be3c4f8e0 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.c
> +++ b/drivers/pci/controller/dwc/pcie-designware.c
> @@ -396,8 +396,60 @@ static inline void dw_pcie_writel_atu_ib(struct dw_pcie *pci, u32 index, u32 reg
> dw_pcie_writel_atu(pci, PCIE_ATU_REGION_DIR_IB, index, reg, val);
> }
>
> -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> - int type, u64 cpu_addr, u8 bar)
> +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> + u64 cpu_addr, u64 pci_addr, u64 size)
> +{
> + u64 limit_addr = pci_addr + size - 1;
> + u32 retries, val;
> +
> + if ((limit_addr & ~pci->region_limit) != (pci_addr & ~pci->region_limit) ||
> + !IS_ALIGNED(cpu_addr, pci->region_align) ||
> + !IS_ALIGNED(pci_addr, pci->region_align) || !size) {
> + return -EINVAL;
> + }
> +
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_BASE,
> + lower_32_bits(pci_addr));
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_BASE,
> + upper_32_bits(pci_addr));
> +
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LIMIT,
> + lower_32_bits(limit_addr));
> + if (dw_pcie_ver_is_ge(pci, 460A))
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_LIMIT,
> + upper_32_bits(limit_addr));
> +
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_LOWER_TARGET,
> + lower_32_bits(cpu_addr));
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_UPPER_TARGET,
> + upper_32_bits(cpu_addr));
> +
> + val = type;
> + if (upper_32_bits(limit_addr) > upper_32_bits(pci_addr) &&
> + dw_pcie_ver_is_ge(pci, 460A))
> + val |= PCIE_ATU_INCREASE_REGION_SIZE;
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL1, val);
> + dw_pcie_writel_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2, PCIE_ATU_ENABLE);
> +
> + /*
> + * Make sure ATU enable takes effect before any subsequent config
> + * and I/O accesses.
> + */
> + for (retries = 0; retries < LINK_WAIT_MAX_IATU_RETRIES; retries++) {
> + val = dw_pcie_readl_atu_ib(pci, index, PCIE_ATU_REGION_CTRL2);
> + if (val & PCIE_ATU_ENABLE)
> + return 0;
> +
> + mdelay(LINK_WAIT_IATU);
> + }
> +
> + dev_err(pci->dev, "Inbound iATU is not being enabled\n");
> +
> + return -ETIMEDOUT;
> +}
> +
> +int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> + int type, u64 cpu_addr, u8 bar)
> {
> u32 retries, val;
>
> diff --git a/drivers/pci/controller/dwc/pcie-designware.h b/drivers/pci/controller/dwc/pcie-designware.h
> index c3e73ed9aff5..5954e8cf9eec 100644
> --- a/drivers/pci/controller/dwc/pcie-designware.h
> +++ b/drivers/pci/controller/dwc/pcie-designware.h
> @@ -308,8 +308,10 @@ int dw_pcie_prog_outbound_atu(struct dw_pcie *pci, int index, int type,
> u64 cpu_addr, u64 pci_addr, u64 size);
> int dw_pcie_prog_ep_outbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> int type, u64 cpu_addr, u64 pci_addr, u64 size);
> -int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> - int type, u64 cpu_addr, u8 bar);
> +int dw_pcie_prog_inbound_atu(struct dw_pcie *pci, int index, int type,
> + u64 cpu_addr, u64 pci_addr, u64 size);
> +int dw_pcie_prog_ep_inbound_atu(struct dw_pcie *pci, u8 func_no, int index,
> + int type, u64 cpu_addr, u8 bar);
> void dw_pcie_disable_atu(struct dw_pcie *pci, u32 dir, int index);
> void dw_pcie_setup(struct dw_pcie *pci);
> void dw_pcie_iatu_detect(struct dw_pcie *pci);
> --
> 2.35.1
>
--
மணிவண்ணன் சதாசிவம்
next prev parent reply other threads:[~2022-08-01 14:01 UTC|newest]
Thread overview: 53+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-06-24 14:39 [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support Serge Semin
2022-06-24 14:39 ` [PATCH RESEND v4 01/15] PCI: dwc: Add more verbose link-up message Serge Semin
2022-08-01 12:59 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 02/15] PCI: dwc: Detect iATU settings after getting "addr_space" resource Serge Semin
2022-07-28 14:38 ` Bjorn Helgaas
2022-08-01 13:01 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 03/15] PCI: dwc: Convert to using native IP-core versions representation Serge Semin
2022-07-28 15:24 ` Bjorn Helgaas
2022-07-28 15:31 ` Ben Dooks
2022-07-28 16:34 ` Bjorn Helgaas
2022-07-28 18:53 ` Serge Semin
2022-08-01 13:07 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 04/15] PCI: dwc: Add IP-core version detection procedure Serge Semin
2022-08-01 13:12 ` Manivannan Sadhasivam
2022-08-01 20:06 ` Bjorn Helgaas
2022-08-02 7:31 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 05/15] PCI: dwc: Introduce Synopsys IP-core versions/types interface Serge Semin
2022-08-01 13:21 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 06/15] PCI: intel-gw: Drop manual DW PCIe controller version setup Serge Semin
2022-08-01 13:28 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 07/15] PCI: tegra194: " Serge Semin
2022-06-27 7:59 ` Vidya Sagar
2022-06-27 22:31 ` Serge Semin
2022-08-01 13:29 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 08/15] PCI: dwc: Add host de-initialization callback Serge Semin
2022-08-01 13:33 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 09/15] PCI: dwc: Drop inbound iATU types enumeration - dw_pcie_as_type Serge Semin
2022-08-01 13:36 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 10/15] PCI: dwc: Drop iATU regions enumeration - dw_pcie_region_type Serge Semin
2022-08-01 13:37 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 11/15] PCI: dwc: Simplify in/outbound iATU setup methods Serge Semin
2022-08-01 13:50 ` Manivannan Sadhasivam
2022-08-01 20:11 ` Bjorn Helgaas
2022-06-24 14:39 ` [PATCH RESEND v4 12/15] PCI: dwc: Add iATU regions size detection procedure Serge Semin
2022-08-01 13:52 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 13/15] PCI: dwc: Verify in/out regions against iATU constraints Serge Semin
2022-08-01 13:53 ` Manivannan Sadhasivam
2022-06-24 14:39 ` [PATCH RESEND v4 14/15] PCI: dwc: Check iATU in/outbound ranges setup methods status Serge Semin
2022-08-01 13:54 ` Manivannan Sadhasivam
2022-08-01 20:17 ` Bjorn Helgaas
2022-06-24 14:39 ` [PATCH RESEND v4 15/15] PCI: dwc: Introduce dma-ranges property support for RC-host Serge Semin
2022-07-28 22:11 ` Bjorn Helgaas
2022-07-29 4:52 ` Serge Semin
2022-07-29 11:33 ` Bjorn Helgaas
2022-07-29 14:38 ` Serge Semin
2022-08-01 14:00 ` Manivannan Sadhasivam [this message]
2022-08-09 20:07 ` Serge Semin
2022-07-11 18:48 ` [PATCH RESEND v4 00/15] PCI: dwc: Add hw version and dma-ranges support Serge Semin
2022-07-27 22:49 ` Bjorn Helgaas
2022-07-28 12:21 ` Serge Semin
2022-07-29 2:36 ` Bjorn Helgaas
2022-09-28 8:10 ` Lorenzo Pieralisi
2022-09-28 10:53 ` Serge Semin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220801140047.GO93763@thinkpad \
--to=manivannan.sadhasivam@linaro.org \
--cc=Alexey.Malahov@baikalelectronics.ru \
--cc=Frank.Li@nxp.com \
--cc=Pavel.Parkhomenko@baikalelectronics.ru \
--cc=Sergey.Semin@baikalelectronics.ru \
--cc=bhelgaas@google.com \
--cc=devicetree@vger.kernel.org \
--cc=fancer.lancer@gmail.com \
--cc=gustavo.pimentel@synopsys.com \
--cc=jingoohan1@gmail.com \
--cc=kw@linux.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=lpieralisi@kernel.org \
--cc=robh+dt@kernel.org \
--cc=robh@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).