From: Bjorn Helgaas <helgaas@kernel.org>
To: Jim Quinlan <james.quinlan@broadcom.com>
Cc: "open list:PCI NATIVE HOST BRIDGE AND ENDPOINT DRIVERS"
<linux-pci@vger.kernel.org>, Christoph Hellwig <hch@lst.de>,
Nicolas Saenz Julienne <nsaenzjulienne@suse.de>,
"maintainer:BROADCOM BCM7XXX ARM ARCHITECTURE"
<bcm-kernel-feedback-list@broadcom.com>,
Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
Rob Herring <robh@kernel.org>,
Bjorn Helgaas <bhelgaas@google.com>,
Florian Fainelli <f.fainelli@gmail.com>,
"moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE"
<linux-rpi-kernel@lists.infradead.org>,
"moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE"
<linux-arm-kernel@lists.infradead.org>,
open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v5 09/12] PCI: brcmstb: Set internal memory viewport sizes
Date: Wed, 17 Jun 2020 22:01:41 -0500 [thread overview]
Message-ID: <20200618030141.GA2041805@bjorn-Precision-5520> (raw)
In-Reply-To: <CA+-6iNx1j5uK=nL-H32qthxEwZe+KOxtqCG4TPJxD+WdzMQFrA@mail.gmail.com>
On Wed, Jun 17, 2020 at 01:28:12PM -0400, Jim Quinlan wrote:
> Hello Bjorn,
>
> On Tue, Jun 16, 2020 at 6:05 PM Bjorn Helgaas <helgaas@kernel.org> wrote:
> >
> > On Tue, Jun 16, 2020 at 04:55:16PM -0400, Jim Quinlan wrote:
> > > BrcmSTB PCIe controllers are intimately connected to the memory
> > > controller(s) on the SOC. There is a "viewport" for each memory controller
> > > that allows inbound accesses to CPU memory. Each viewport's size must be
> > > set to a power of two, and that size must be equal to or larger than the
> > > amount of memory each controller supports.
> >
> > This describes some requirements, but doesn't actually say what this
> > patch *does*.
> >
> > I *think* it reads the viewport sizes from the "brcm,scb-sizes" DT
> > property instead of computing something from "dma-ranges". Looks like
> > it also adds support for SCB1 and SCB2.
> >
> > Those seem interesting, but don't really come through in the subject
> > or even the commit log.
> >
> > If I understand correctly, this is all for DMA ("inbound accesses to
> > CPU memory"). I think it would be worth mentioning "DMA", since
> > that's the common term for this.
>
>
> I have changed the commit message to the text below. Please let me
> know if it requires more work
> Thanks, Jim
>
> PCI: brcmstb: Set internal memory DMA viewport sizes
Did you not set the viewport sizes before?
> BrcmSTB PCIe controllers are intimately connected to the memory
> controller(s) on the SOC. There is a "viewport" for each memory controller
> that allows inbound DMA acceses to CPU memory. Each viewport's size must
> be set to a power of two, and that size must be equal to or larger than the
> amount of memory each controller supports. Unfortunately the viewport
> sizes cannot be ascertained from the "dma-ranges" property so they have
> their own property, "brcm,scb-sizes".
s/inbound DMA acceses to CPU memory/DMA/
"Accesses" is redundant since the "A" in "DMA" stands for "access".
I'm not sure "inbound" adds anything and might confuse since DMA may
be either a read or write of CPU memory.
I assume *all* drivers need to know the address and size of regions in
"dma-ranges". Is there something special about this device that means
it needs something different?
I guess it's the base/extension split? That couldn't be described as
two separate DMA ranges?
Could/should the new property have a name somehow related to
"dma-ranges"?
Should "dma-ranges" be documented in
Documentation/devicetree/bindings/pci/pci.txt instead of the
individual device bindings?
> There may be one to three memory controllers; they are indicated by the
> term SCBi. Each controller has a base region and an optional extension
> region. In physical memory, the base and extension regions are not
> adjacent, but in PCIe-space they are. Further, the 1-3 viewports are also
> adjacent in PCIe-space.
>
> The SCB settings work in conjunction with the "dma-ranges' offsets to
> enable non-identity mappings between system memory and PCIe space.
s/ranges'/ranges"/ (mismatched quotes)
This describes the hardware, but still doesn't actually say what this
patch *does*.
If I'm a user, why do I want this patch? Does it fix something that
didn't work before? Does it increase the amount of DMA-able memory?
What does this mean in terms of backwards compatibility with old DTs?
Does this work with old DTs that don't have "brcm,scb-sizes"? Maybe
this is all related to specific devices that weren't supported before,
so there *are* no old DTs for them? I can't tell from the binding
update or the patch that this is related to specific devices.
> > > Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> > > Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> > > ---
> > > drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
> > > 1 file changed, 50 insertions(+), 18 deletions(-)
> > >
> > > diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> > > index 9189406fd35c..39f77709c6a2 100644
> > > --- a/drivers/pci/controller/pcie-brcmstb.c
> > > +++ b/drivers/pci/controller/pcie-brcmstb.c
> > > @@ -57,6 +57,8 @@
> > > #define PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_MASK 0x300000
> > > #define PCIE_MISC_MISC_CTRL_MAX_BURST_SIZE_128 0x0
> > > #define PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK 0xf8000000
> > > +#define PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK 0x07c00000
> > > +#define PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK 0x0000001f
> > >
> > > #define PCIE_MISC_CPU_2_PCIE_MEM_WIN0_LO 0x400c
> > > #define PCIE_MEM_WIN0_LO(win) \
> > > @@ -154,6 +156,7 @@
> > > #define SSC_STATUS_OFFSET 0x1
> > > #define SSC_STATUS_SSC_MASK 0x400
> > > #define SSC_STATUS_PLL_LOCK_MASK 0x800
> > > +#define PCIE_BRCM_MAX_MEMC 3
> > >
> > > #define IDX_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_INDEX])
> > > #define DATA_ADDR(pcie) (pcie->reg_offsets[EXT_CFG_DATA])
> > > @@ -260,6 +263,8 @@ struct brcm_pcie {
> > > const int *reg_field_info;
> > > enum pcie_type type;
> > > struct reset_control *rescal;
> > > + int num_memc;
> > > + u64 memc_size[PCIE_BRCM_MAX_MEMC];
> > > };
> > >
> > > /*
> > > @@ -715,22 +720,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
> > > u64 *rc_bar2_offset)
> > > {
> > > struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> > > - struct device *dev = pcie->dev;
> > > struct resource_entry *entry;
> > > + struct device *dev = pcie->dev;
> > > + u64 lowest_pcie_addr = ~(u64)0;
> > > + int ret, i = 0;
> > > + u64 size = 0;
> > >
> > > - entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> > > - if (!entry)
> > > - return -ENODEV;
> > > + resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> > > + u64 pcie_beg = entry->res->start - entry->offset;
> > >
> > > + size += entry->res->end - entry->res->start + 1;
> > > + if (pcie_beg < lowest_pcie_addr)
> > > + lowest_pcie_addr = pcie_beg;
> > > + }
> > >
> > > - /*
> > > - * The controller expects the inbound window offset to be calculated as
> > > - * the difference between PCIe's address space and CPU's. The offset
> > > - * provided by the firmware is calculated the opposite way, so we
> > > - * negate it.
> > > - */
> > > - *rc_bar2_offset = -entry->offset;
> > > - *rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> > > + if (lowest_pcie_addr == ~(u64)0) {
> > > + dev_err(dev, "DT node has no dma-ranges\n");
> > > + return -EINVAL;
> > > + }
> > > +
> > > + ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> > > + PCIE_BRCM_MAX_MEMC);
> > > +
> > > + if (ret <= 0) {
> > > + /* Make an educated guess */
> > > + pcie->num_memc = 1;
> > > + pcie->memc_size[0] = 1 << fls64(size - 1);
> > > + } else {
> > > + pcie->num_memc = ret;
> > > + }
> > > +
> > > + /* Each memc is viewed through a "port" that is a power of 2 */
> > > + for (i = 0, size = 0; i < pcie->num_memc; i++)
> > > + size += pcie->memc_size[i];
> > > +
> > > + /* System memory starts at this address in PCIe-space */
> > > + *rc_bar2_offset = lowest_pcie_addr;
> > > + /* The sum of all memc views must also be a power of 2 */
> > > + *rc_bar2_size = 1ULL << fls64(size - 1);
> > >
> > > /*
> > > * We validate the inbound memory view even though we should trust
> > > @@ -782,12 +809,11 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
> > > void __iomem *base = pcie->base;
> > > struct device *dev = pcie->dev;
> > > struct resource_entry *entry;
> > > - unsigned int scb_size_val;
> > > bool ssc_good = false;
> > > struct resource *res;
> > > int num_out_wins = 0;
> > > u16 nlw, cls, lnksta;
> > > - int i, ret;
> > > + int i, ret, memc;
> > > u32 tmp, aspm_support;
> > >
> > > /* Reset the bridge */
> > > @@ -824,11 +850,17 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
> > > writel(upper_32_bits(rc_bar2_offset),
> > > base + PCIE_MISC_RC_BAR2_CONFIG_HI);
> > >
> > > - scb_size_val = rc_bar2_size ?
> > > - ilog2(rc_bar2_size) - 15 : 0xf; /* 0xf is 1GB */
> > > tmp = readl(base + PCIE_MISC_MISC_CTRL);
> > > - u32p_replace_bits(&tmp, scb_size_val,
> > > - PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> > > + for (memc = 0; memc < pcie->num_memc; memc++) {
> > > + u32 scb_size_val = ilog2(pcie->memc_size[memc]) - 15;
> > > +
> > > + if (memc == 0)
> > > + u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB0_SIZE_MASK);
> > > + else if (memc == 1)
> > > + u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB1_SIZE_MASK);
> > > + else if (memc == 2)
> > > + u32p_replace_bits(&tmp, scb_size_val, PCIE_MISC_MISC_CTRL_SCB2_SIZE_MASK);
> > > + }
> > > writel(tmp, base + PCIE_MISC_MISC_CTRL);
> > >
> > > /*
> > > --
> > > 2.17.1
> > >
next prev parent reply other threads:[~2020-06-18 3:01 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-06-16 20:55 [PATCH v5 00/12] PCI: brcmstb: enable PCIe for STB chips Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 01/12] PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 02/12] ata: ahci_brcm: Fix use of BCM7216 reset controller Jim Quinlan
2020-06-16 22:04 ` Bjorn Helgaas
2020-06-16 20:55 ` [PATCH v5 03/12] dt-bindings: PCI: Add bindings for more Brcmstb chips Jim Quinlan
2020-06-16 22:05 ` Bjorn Helgaas
2020-06-16 20:55 ` [PATCH v5 04/12] PCI: brcmstb: Add bcm7278 register info Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 05/12] PCI: brcmstb: Add suspend and resume pm_ops Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 06/12] PCI: brcmstb: Add bcm7278 PERST support Jim Quinlan
2020-06-16 22:05 ` Bjorn Helgaas
2020-06-16 20:55 ` [PATCH v5 07/12] PCI: brcmstb: Add control of rescal reset Jim Quinlan
2020-06-16 22:05 ` Bjorn Helgaas
2020-06-17 17:32 ` Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 09/12] PCI: brcmstb: Set internal memory viewport sizes Jim Quinlan
2020-06-16 22:05 ` Bjorn Helgaas
2020-06-17 17:28 ` Jim Quinlan
2020-06-18 3:01 ` Bjorn Helgaas [this message]
2020-06-18 14:56 ` Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 10/12] PCI: brcmstb: Accommodate MSI for older chips Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 11/12] PCI: brcmstb: Set bus max burst size by chip type Jim Quinlan
2020-06-16 20:55 ` [PATCH v5 12/12] PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list Jim Quinlan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200618030141.GA2041805@bjorn-Precision-5520 \
--to=helgaas@kernel.org \
--cc=bcm-kernel-feedback-list@broadcom.com \
--cc=bhelgaas@google.com \
--cc=f.fainelli@gmail.com \
--cc=hch@lst.de \
--cc=james.quinlan@broadcom.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-pci@vger.kernel.org \
--cc=linux-rpi-kernel@lists.infradead.org \
--cc=lorenzo.pieralisi@arm.com \
--cc=nsaenzjulienne@suse.de \
--cc=robh@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).