dmaengine Archive on lore.kernel.org
 help / color / Atom feed
From: Florian Fainelli <f.fainelli@gmail.com>
To: Robin Murphy <robin.murphy@arm.com>,
	Nicolas Saenz Julienne <nsaenzjulienne@suse.de>,
	Rob Herring <robh+dt@kernel.org>
Cc: "moderated list:ARM/FREESCALE IMX / MXC ARM ARCHITECTURE" 
	<linux-arm-kernel@lists.infradead.org>,
	devicetree@vger.kernel.org, Matthias Brugger <mbrugger@suse.com>,
	Frank Rowand <frowand.list@gmail.com>,
	linux-arm-msm <linux-arm-msm@vger.kernel.org>,
	linux-wireless <linux-wireless@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	dri-devel <dri-devel@lists.freedesktop.org>,
	etnaviv@lists.freedesktop.org,
	"open list:DMA GENERIC OFFLOAD ENGINE SUBSYSTEM" 
	<dmaengine@vger.kernel.org>, Stefan Wahren <wahrenst@gmx.net>,
	james.quinlan@broadcom.com, linux-pci@vger.kernel.org,
	linux-tegra@vger.kernel.org, xen-devel@lists.xenproject.org,
	Dan Williams <dan.j.williams@intel.com>,
	freedreno <freedreno@lists.freedesktop.org>,
	Linux Media Mailing List <linux-media@vger.kernel.org>
Subject: Re: [PATCH 00/11] of: Fix DMA configuration for non-DT masters
Date: Wed, 2 Oct 2019 11:28:06 -0700
Message-ID: <fbae48ca-fbd4-e32b-e874-92b5bba5df4d@gmail.com> (raw)
In-Reply-To: <c27a51e1-1adf-ae6a-dc67-ae76222a1163@arm.com>



On 9/26/2019 4:20 AM, Robin Murphy wrote:
> On 2019-09-26 11:44 am, Nicolas Saenz Julienne wrote:
>>>>>> Robin, have you looked into supporting multiple dma-ranges? It's the
>>>>>> next thing
>>>>>> we need for BCM STB's PCIe. I'll have a go at it myself if nothing
>>>>>> is in
>>>>>> the
>>>>>> works already.
>>>>>
>>>>> Multiple dma-ranges as far as configuring inbound windows should work
>>>>> already other than the bug when there's any parent translation. But if
>>>>> you mean supporting multiple DMA offsets and masks per device in the
>>>>> DMA API, there's nothing in the works yet.
>>
>> Sorry, I meant supporting multiple DMA offsets[1]. I think I could
>> still make
>> it with a single DMA mask though.
> 
> The main problem for supporting that case in general is the disgusting
> carving up of the physical memory map you may have to do to guarantee
> that a single buffer allocation cannot ever span two windows with
> different offsets. I don't think we ever reached a conclusion on whether
> that was even achievable in practice.

It is with the Broadcom STB SoCs which have between 1 and 3 memory
controllers depending on the SoC, and multiple dma-ranges cells for PCIe
as a consequence.

Each memory controller has a different physical address aperture in the
CPU's physical address map (e.g.: MEMC0 is 0x0 - 0x3fff_ffff, MEMC1
0x4000_0000 - 0x7ffff_ffff and MEMC2 0x8000_0000 - 0xbfff_ffff, not
counting the extension regions above 4GB), and while the CPU is
scheduled and arbitrated the same way across all memory controllers
(thus making it virtually UMA, almost) having a buffer span two memory
controllers would be problematic because the memory controllers do not
know how to guarantee the transaction ordering and buffer data
consistency in both DRAM itself and for other memory controller clients,
like PCIe.

We historically had to reserve the last 4KB of each memory controller to
avoid problematic controllers like EHCI to prefetch beyond the end of a
memory controller's populated memory and that also incidentally takes
care of never having a buffer cross a controller boundary. Either you
can allocate the entire buffer on a given memory controller, or you
cannot allocate memory at all on that zone/region and another one must
be found (or there is no more memory and there is a genuine OOM).

The way we reserve memory right now is based on the first patch
submitted by Jim:

https://lore.kernel.org/patchwork/patch/988469/

whereby we read the memory node's "reg" property and we map the physical
addresses to the memory controller configuration read from the specific
registers in the CPU's Bus Interface Unit (where the memory controller
apertures are architecturally defined) and then we use that to call
memblock_reserve() (not part of that patch, it should be though).
-- 
Florian

  reply index

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-24 18:12 Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 01/11] of: address: clean-up unused variable in of_dma_get_range() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 02/11] of: base: introduce __of_n_*_cells_parent() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 03/11] of: address: use parent DT node in bus->count_cells() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 04/11] of: address: introduce of_translate_dma_address_parent() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 05/11] of: expose __of_get_dma_parent() to OF subsystem Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 06/11] of: address: use parent OF node in of_dma_get_range() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 07/11] dts: arm64: layerscape: add dma-ranges property to qoric-mc node Nicolas Saenz Julienne
2019-10-14  8:28   ` Shawn Guo
2019-10-14 10:00     ` Nicolas Saenz Julienne
2019-10-14 11:09       ` Shawn Guo
2019-09-24 18:12 ` [PATCH 08/11] dts: arm64: layerscape: add dma-ranges property to pcie nodes Nicolas Saenz Julienne
2019-10-14  8:29   ` Shawn Guo
2019-09-24 18:12 ` [PATCH 09/11] of: device: remove comment in of_dma_configure() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 10/11] of: device: introduce of_dma_configure_parent() Nicolas Saenz Julienne
2019-09-24 18:12 ` [PATCH 11/11] of: simplify of_dma_config()'s arguments Nicolas Saenz Julienne
2019-09-24 21:59 ` [PATCH 00/11] of: Fix DMA configuration for non-DT masters Rob Herring
2019-09-25 14:52   ` Nicolas Saenz Julienne
2019-09-25 15:09     ` Robin Murphy
2019-09-25 15:30       ` Nicolas Saenz Julienne
2019-09-25 16:16         ` Rob Herring
2019-09-25 16:52           ` Robin Murphy
2019-09-25 21:33             ` Rob Herring
2019-09-26 10:44               ` Nicolas Saenz Julienne
2019-09-26 11:20                 ` Robin Murphy
2019-10-02 18:28                   ` Florian Fainelli [this message]
2019-09-25 16:07     ` Rob Herring

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=fbae48ca-fbd4-e32b-e874-92b5bba5df4d@gmail.com \
    --to=f.fainelli@gmail.com \
    --cc=dan.j.williams@intel.com \
    --cc=devicetree@vger.kernel.org \
    --cc=dmaengine@vger.kernel.org \
    --cc=dri-devel@lists.freedesktop.org \
    --cc=etnaviv@lists.freedesktop.org \
    --cc=freedreno@lists.freedesktop.org \
    --cc=frowand.list@gmail.com \
    --cc=james.quinlan@broadcom.com \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-arm-msm@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-media@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-tegra@vger.kernel.org \
    --cc=linux-wireless@vger.kernel.org \
    --cc=mbrugger@suse.com \
    --cc=nsaenzjulienne@suse.de \
    --cc=robh+dt@kernel.org \
    --cc=robin.murphy@arm.com \
    --cc=wahrenst@gmx.net \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

dmaengine Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/dmaengine/0 dmaengine/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 dmaengine dmaengine/ https://lore.kernel.org/dmaengine \
		dmaengine@vger.kernel.org
	public-inbox-index dmaengine

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.dmaengine


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git