Linux-PCI Archive on lore.kernel.org
 help / color / Atom feed
From: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
To: Jim Quinlan <james.quinlan@broadcom.com>,
	linux-pci@vger.kernel.org, Christoph Hellwig <hch@lst.de>,
	Robin Murphy <robin.murphy@arm.com>,
	bcm-kernel-feedback-list@broadcom.com
Cc: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Rob Herring <robh@kernel.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Florian Fainelli <f.fainelli@gmail.com>,
	"moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" 
	<linux-rpi-kernel@lists.infradead.org>,
	"moderated list:BROADCOM BCM2711/BCM2835 ARM ARCHITECTURE" 
	<linux-arm-kernel@lists.infradead.org>,
	open list <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v9 09/12] PCI: brcmstb: Set additional internal memory DMA viewport sizes
Date: Sat, 01 Aug 2020 19:39:32 +0200
Message-ID: <e37ff97826cf006bf1c9a3e0a134847f8030c79a.camel@suse.de> (raw)
In-Reply-To: <20200724203407.16972-10-james.quinlan@broadcom.com>

Hi Jim,

On Fri, 2020-07-24 at 16:33 -0400, Jim Quinlan wrote:
> The Raspberry Pi (RPI) is currently the only chip using this driver
> (pcie-brcmstb.c).  There, only one memory controller is used, without an
> extension region, and the SCB0 viewport size is set to the size of the
> first and only dma-range region.  Other BrcmSTB SOCs have more complicated
> memory configurations that require setting additional viewport sizes.
> 
> BrcmSTB PCIe controllers are intimately connected to the memory
> controller(s) on the SOC.  The SOC may have one to three memory
> controllers; they are indicated by the term SCBi.  Each controller has a
> base region and an optional extension region.  In physical memory, the base
> and extension regions of a controller are not adjacent, but in PCIe-space
> they are.
> 
> There is a "viewport" for each memory controller that allows DMA from
> endpoint devices.  Each viewport's size must be set to a power of two, and
> that size must be equal to or larger than the amount of memory each
> controller supports which is the sum of base region and its optional
> extension.  Further, the 1-3 viewports are also adjacent in PCIe-space.
> 
> Unfortunately the viewport sizes cannot be ascertained from the
> "dma-ranges" property so they have their own property, "brcm,scb-sizes".
> This is because dma-range information does not indicate what memory
> controller it is associated.  For example, consider the following case
> where the size of one dma-range is 2GB and the second dma-range is 1GB:
> 
>     /* Case 1: SCB0 size set to 4GB */
>     dma-range0: 2GB (from memc0-base)
>     dma-range1: 1GB (from memc0-extension)
> 
>     /* Case 2: SCB0 size set to 2GB, SCB1 size set to 1GB */
>     dma-range0: 2GB (from memc0-base)
>     dma-range1: 1GB (from memc0-extension)
> 
> By just looking at the dma-ranges information, one cannot tell which
> situation applies. That is why an additional property is needed.  Its
> length indicates the number of memory controllers being used and each value
> indicates the viewport size.
> 
> Note that the RPI DT does not have a "brcm,scb-sizes" property value,
> as it is assumed that it only requires one memory controller and no
> extension.  So the optional use of "brcm,scb-sizes" will be backwards
> compatible.
> 
> One last layer of complexity exists: all of the viewports sizes must be
> added and rounded up to a power of two to determine what the "BAR" size is.
> Further, an offset must be given that indicates the base PCIe address of
> this "BAR".  The use of the term BAR is typically associated with endpoint
> devices, and the term is used here because the PCIe HW may be used as an RC
> or an EP.  In the former case, all of the system memory appears in a single
> "BAR" region in PCIe memory.  As it turns out, BrcmSTB PCIe HW is rarely
> used in the EP role and its system of mapping memory is an artifact that
> requires multiple dma-ranges regions.
> 
> Signed-off-by: Jim Quinlan <james.quinlan@broadcom.com>
> Acked-by: Florian Fainelli <f.fainelli@gmail.com>
> ---
>  drivers/pci/controller/pcie-brcmstb.c | 68 ++++++++++++++++++++-------
>  1 file changed, 50 insertions(+), 18 deletions(-)
> 
> diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
> index 8dacb9d3b7b6..3ef2d37cc43b 100644
> --- a/drivers/pci/controller/pcie-brcmstb.c
> +++ b/drivers/pci/controller/pcie-brcmstb.c
> @@ -715,22 +720,44 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,
>  							u64 *rc_bar2_offset)
>  {
>  	struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
> -	struct device *dev = pcie->dev;
>  	struct resource_entry *entry;
> +	struct device *dev = pcie->dev;
> +	u64 lowest_pcie_addr = ~(u64)0;
> +	int ret, i = 0;
> +	u64 size = 0;
>  
> -	entry = resource_list_first_type(&bridge->dma_ranges, IORESOURCE_MEM);
> -	if (!entry)
> -		return -ENODEV;
> +	resource_list_for_each_entry(entry, &bridge->dma_ranges) {
> +		u64 pcie_beg = entry->res->start - entry->offset;
>  
> +		size += entry->res->end - entry->res->start + 1;
> +		if (pcie_beg < lowest_pcie_addr)
> +			lowest_pcie_addr = pcie_beg;
> +	}
>  
> -	/*
> -	 * The controller expects the inbound window offset to be calculated as
> -	 * the difference between PCIe's address space and CPU's. The offset
> -	 * provided by the firmware is calculated the opposite way, so we
> -	 * negate it.
> -	 */
> -	*rc_bar2_offset = -entry->offset;
> -	*rc_bar2_size = 1ULL << fls64(entry->res->end - entry->res->start);
> +	if (lowest_pcie_addr == ~(u64)0) {
> +		dev_err(dev, "DT node has no dma-ranges\n");
> +		return -EINVAL;
> +	}
> +
> +	ret = of_property_read_variable_u64_array(pcie->np, "brcm,scb-sizes", pcie->memc_size, 1,
> +						  PCIE_BRCM_MAX_MEMC);
> +
> +	if (ret <= 0) {
> +		/* Make an educated guess */
> +		pcie->num_memc = 1;
> +		pcie->memc_size[0] = 1 << fls64(size - 1);

You need to 1ULL here.

Regards,
Nicolas

> +	} else {
> +		pcie->num_memc = ret;
> +	}
> +
> +	/* Each memc is viewed through a "port" that is a power of 2 */
> +	for (i = 0, size = 0; i < pcie->num_memc; i++)
> +		size += pcie->memc_size[i];
> +
> +	/* System memory starts at this address in PCIe-space */
> +	*rc_bar2_offset = lowest_pcie_addr;
> +	/* The sum of all memc views must also be a power of 2 */
> +	*rc_bar2_size = 1ULL << fls64(size - 1);
>  
>  	/*
>  	 * We validate the inbound memory view even though we should trust


  reply index

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-24 20:33 [PATCH v9 00/12] PCI: brcmstb: enable PCIe for STB chips Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 01/12] PCI: brcmstb: PCIE_BRCMSTB depends on ARCH_BRCMSTB Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 02/12] ata: ahci_brcm: Fix use of BCM7216 reset controller Jim Quinlan
2020-07-24 21:02   ` Florian Fainelli
2020-07-25  9:18   ` Sergei Shtylyov
2020-07-27  6:58   ` Philipp Zabel
2020-07-24 20:33 ` [PATCH v9 03/12] dt-bindings: PCI: Add bindings for more Brcmstb chips Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 04/12] PCI: brcmstb: Add bcm7278 register info Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 05/12] PCI: brcmstb: Add suspend and resume pm_ops Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 06/12] PCI: brcmstb: Add bcm7278 PERST# support Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 07/12] PCI: brcmstb: Add control of rescal reset Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 09/12] PCI: brcmstb: Set additional internal memory DMA viewport sizes Jim Quinlan
2020-08-01 17:39   ` Nicolas Saenz Julienne [this message]
2020-08-03 12:40     ` Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 10/12] PCI: brcmstb: Accommodate MSI for older chips Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 11/12] PCI: brcmstb: Set bus max burst size by chip type Jim Quinlan
2020-07-24 20:33 ` [PATCH v9 12/12] PCI: brcmstb: Add bcm7211, bcm7216, bcm7445, bcm7278 to match list Jim Quinlan

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=e37ff97826cf006bf1c9a3e0a134847f8030c79a.camel@suse.de \
    --to=nsaenzjulienne@suse.de \
    --cc=bcm-kernel-feedback-list@broadcom.com \
    --cc=bhelgaas@google.com \
    --cc=f.fainelli@gmail.com \
    --cc=hch@lst.de \
    --cc=james.quinlan@broadcom.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-rpi-kernel@lists.infradead.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=robh@kernel.org \
    --cc=robin.murphy@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-PCI Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-pci/0 linux-pci/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-pci linux-pci/ https://lore.kernel.org/linux-pci \
		linux-pci@vger.kernel.org
	public-inbox-index linux-pci

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-pci


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git