linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
From: Robin Murphy <robin.murphy@arm.com>
To: Thierry Reding <thierry.reding@gmail.com>,
	Christoph Hellwig <hch@infradead.org>
Cc: Rob Herring <robh@kernel.org>,
	Florian Fainelli <f.fainelli@gmail.com>,
	Geert Uytterhoeven <geert+renesas@glider.be>,
	Arnd Bergmann <arnd@arndb.de>, DTML <devicetree@vger.kernel.org>,
	linux-pci <linux-pci@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	Marek Vasut <marek.vasut@gmail.com>,
	Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>,
	Oza Pawandeep <oza.oza@broadcom.com>,
	Nicolas Saenz Julienne <nsaenzjulienne@suse.de>,
	linux-tegra@vger.kernel.org,
	Simon Horman <horms+renesas@verge.net.au>,
	Frank Rowand <frowand.list@gmail.com>,
	Linux ARM <linux-arm-kernel@lists.infradead.org>,
	Stefan Wahren <wahrenst@gmx.net>
Subject: Re: [PATCH 00/11] of: dma-ranges fixes and improvements
Date: Mon, 30 Sep 2019 10:55:15 +0100	[thread overview]
Message-ID: <89e33aae-bc96-53c3-2a4e-e879e9a3c73e@arm.com> (raw)
In-Reply-To: <20190930085606.GG1518582@ulmo>

On 2019-09-30 9:56 am, Thierry Reding wrote:
> On Mon, Sep 30, 2019 at 01:20:55AM -0700, Christoph Hellwig wrote:
>> On Sun, Sep 29, 2019 at 01:16:20PM +0200, Arnd Bergmann wrote:
>>> On a semi-related note, Thierry asked about one aspect of the dma-ranges
>>> property recently, which is the behavior of dma_set_mask() and related
>>> functions when a driver sets a mask that is larger than the memory
>>> area in the bus-ranges but smaller than the available physical RAM.
>>> As I understood Thierry's problem and the current code, the generic
>>> dma_set_mask() will either reject the new mask entirely or override
>>> the mask set by of_dma_configure, but it fails to set a correct mask
>>> within the limitations of the parent bus in this case.
>>
>> There days dma_set_mask will only reject a mask if it is too small
>> to be supported by the hardware.  Larger than required masks are now
>> always accepted.
> 
> Summarizing why this came up: the memory subsystem on Tegra194 has a
> mechanism controlled by bit 39 of physical addresses. This is used to
> support two variants of sector ordering for block linear formats. The
> GPU uses a slightly different ordering than other MSS clients, so the
> drivers have to set this bit depending on who they interoperate with.
> 
> I was running into this as I was adding support for IOMMU support for
> the Ethernet controller on Tegra194. The controller has a HW feature
> register that contains how many address bits it supports. This is 40
> for Tegra194, corresponding to the number of address bits to the MSS.
> Without IOMMU support that's not a problem because there are no systems
> with 40 bits of system memory. However, if we enable IOMMU support, the
> DMA/IOMMU code will allocate from the top of a 48-bit (constrained to
> 40 bits via the DMA mask) input address space. This causes bit 39 to be
> set, which in turn will make the MSS reorder sectors and break network
> communications.
> 
> Since this reordering takes place at the MSS level, this applies to all
> MSS clients. Most of these clients always want bit 39 to be 0, whereas
> the clients that can and want to make use of the reordering always want
> bit 39 to be under their control, so they can control in a fine-grained
> way when to set it.
> 
> This means that effectively all MSS clients can only address 39 bits, so
> instead of hard-coding that for each driver I thought it'd make sense to
> have a central place to configure this, so that all devices by default
> are restricted to 39-bit addressing. However, with the current DMA API
> implementation this causes a problem because the default 39-bit DMA mask
> would get overwritten by the driver (as in the example of the Ethernet
> controller setting a 40-bit DMA mask because that's what the hardware
> supports).
> 
> I realize that this is somewhat exotic. On one hand it is correct for a
> driver to say that the hardware supports 40-bit addressing (i.e. the
> Ethernet controller can address bit 39), but from a system integration
> point of view, using bit 39 is wrong, except in a very restricted set of
> cases.
> 
> If I understand correctly, describing this with a dma-ranges property is
> the right thing to do, but it wouldn't work with the current
> implementation because drivers can still override a lower DMA mask with
> a higher one.

But that sounds like exactly the situation for which we introduced 
bus_dma_mask. If "dma-ranges" is found, then we should initialise that 
to reflect the limitation. Drivers may subsequently set a larger mask 
based on what the device is natively capable of, but the DMA API 
internals should quietly clamp that down to the bus mask wherever it 
matters.

Since that change, the initial value of dma_mask and coherent_dma_mask 
doesn't really matter much, as we expect drivers to reset them anyway 
(and in general they have to be able to enlarge them from a 32-bit 
default value).

As far as I'm aware this has been working fine (albeit in equivalent 
ACPI form) for at least one SoC with 64-bit device masks, a 48-bit 
IOMMU, and a 44-bit interconnect in between - indeed if I avoid 
distraction long enough to set up the big new box under my desk, the 
sending of future emails will depend on it ;)

Robin.

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

  reply	other threads:[~2019-09-30  9:55 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-09-27  0:24 [PATCH 00/11] of: dma-ranges fixes and improvements Rob Herring
2019-09-27  0:24 ` [PATCH 01/11] of: Remove unused of_find_matching_node_by_address() Rob Herring
2019-09-27  9:17   ` Geert Uytterhoeven
2019-09-30 12:49   ` Christoph Hellwig
2019-09-27  0:24 ` [PATCH 02/11] of: Make of_dma_get_range() private Rob Herring
2019-09-27  9:18   ` Geert Uytterhoeven
2019-09-30 12:53   ` Christoph Hellwig
2019-09-27  0:24 ` [PATCH 03/11] of: address: Report of_dma_get_range() errors meaningfully Rob Herring
2019-09-27  9:15   ` Geert Uytterhoeven
2019-09-30 12:54   ` Christoph Hellwig
2019-09-27  0:24 ` [PATCH 04/11] of/unittest: Add dma-ranges address translation tests Rob Herring
2019-09-27  0:24 ` [PATCH 05/11] of: Ratify of_dma_configure() interface Rob Herring
2019-09-30 12:57   ` Christoph Hellwig
2019-09-30 13:32     ` Nicolas Saenz Julienne
2019-09-30 21:24       ` Rob Herring
2019-10-01 15:43         ` Nicolas Saenz Julienne
2019-10-04  1:53           ` Rob Herring
2019-10-07 17:51             ` Nicolas Saenz Julienne
2019-09-27  0:24 ` [PATCH 06/11] of/address: Introduce of_get_next_dma_parent() helper Rob Herring
2019-09-27  9:24   ` Geert Uytterhoeven
2019-09-27  0:24 ` [PATCH 07/11] of: address: Follow DMA parent for "dma-coherent" Rob Herring
2019-09-27  0:24 ` [PATCH 08/11] of: Factor out #{addr,size}-cells parsing Rob Herring
2019-09-27  9:27   ` Geert Uytterhoeven
2019-09-27  0:24 ` [PATCH 09/11] of: Make of_dma_get_range() work on bus nodes Rob Herring
2019-09-27  0:24 ` [PATCH 10/11] of/address: Translate 'dma-ranges' for parent nodes missing 'dma-ranges' Rob Herring
2019-09-27  0:24 ` [PATCH 11/11] of/address: Fix of_pci_range_parser_one translation of DMA addresses Rob Herring
2019-09-29 11:16 ` [PATCH 00/11] of: dma-ranges fixes and improvements Arnd Bergmann
2019-09-30  8:20   ` Christoph Hellwig
2019-09-30  8:56     ` Thierry Reding
2019-09-30  9:55       ` Robin Murphy [this message]
2019-09-30 13:35         ` Thierry Reding
2019-09-30  9:20 ` Nicolas Saenz Julienne
2019-09-30 12:40 ` Marek Vasut
2019-09-30 12:52   ` Robin Murphy
2019-09-30 12:54     ` Marek Vasut
2019-09-30 13:05       ` Robin Murphy

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=89e33aae-bc96-53c3-2a4e-e879e9a3c73e@arm.com \
    --to=robin.murphy@arm.com \
    --cc=arnd@arndb.de \
    --cc=devicetree@vger.kernel.org \
    --cc=f.fainelli@gmail.com \
    --cc=frowand.list@gmail.com \
    --cc=geert+renesas@glider.be \
    --cc=hch@infradead.org \
    --cc=horms+renesas@verge.net.au \
    --cc=linux-arm-kernel@lists.infradead.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=linux-tegra@vger.kernel.org \
    --cc=lorenzo.pieralisi@arm.com \
    --cc=marek.vasut@gmail.com \
    --cc=nsaenzjulienne@suse.de \
    --cc=oza.oza@broadcom.com \
    --cc=robh@kernel.org \
    --cc=thierry.reding@gmail.com \
    --cc=wahrenst@gmx.net \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).