From: Laurentiu Tudor <laurentiu.tudor@nxp.com>
To: Robin Murphy <robin.murphy@arm.com>,
"devicetree@vger.kernel.org" <devicetree@vger.kernel.org>,
"netdev@vger.kernel.org" <netdev@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>
Cc: Roy Pledge <roy.pledge@nxp.com>, Leo Li <leoyang.li@nxp.com>,
"shawnguo@kernel.org" <shawnguo@kernel.org>,
"davem@davemloft.net" <davem@davemloft.net>,
Madalin-cristian Bucur <madalin.bucur@nxp.com>
Subject: Re: [PATCH 00/21] SMMU enablement for NXP LS1043A and LS1046A
Date: Thu, 20 Sep 2018 14:33:14 +0000 [thread overview]
Message-ID: <bb973ba9-ca45-c9f1-38d6-67e9bc05e429@nxp.com> (raw)
In-Reply-To: <c2acecc8-5357-6cbe-f93c-00d71470dff5@arm.com>
On 20.09.2018 14:49, Robin Murphy wrote:
> On 20/09/18 11:38, Laurentiu Tudor wrote:
>>
>>
>> On 19.09.2018 17:37, Robin Murphy wrote:
>>> On 19/09/18 15:18, Laurentiu Tudor wrote:
>>>> Hi Robin,
>>>>
>>>> On 19.09.2018 16:25, Robin Murphy wrote:
>>>>> Hi Laurentiu,
>>>>>
>>>>> On 19/09/18 13:35, laurentiu.tudor@nxp.com wrote:
>>>>>> From: Laurentiu Tudor <laurentiu.tudor@nxp.com>
>>>>>>
>>>>>> This patch series adds SMMU support for NXP LS1043A and LS1046A chips
>>>>>> and consists mostly in important driver fixes and the required device
>>>>>> tree updates. It touches several subsystems and consists of three
>>>>>> main
>>>>>> parts:
>>>>>> - changes in soc/drivers/fsl/qbman drivers adding iommu
>>>>>> mapping of
>>>>>> reserved memory areas, fixes and defered probe support
>>>>>> - changes in drivers/net/ethernet/freescale/dpaa_eth drivers
>>>>>> consisting in misc dma mapping related fixes and probe ordering
>>>>>> - addition of the actual arm smmu device tree node together with
>>>>>> various adjustments to the device trees
>>>>>>
>>>>>> Performance impact
>>>>>>
>>>>>> Running iperf benchmarks in a back-to-back setup (both sides
>>>>>> having smmu enabled) on a 10GBps port show an important
>>>>>> networking performance degradation of around %40 (9.48Gbps
>>>>>> linerate vs 5.45Gbps). If you need performance but without
>>>>>> SMMU support you can use "iommu.passthrough=1" to disable
>>>>>> SMMU.
>
> I should have said before - thanks for the numbers there as well. Always
> good to add another datapoint to my collection. If you're interested
> I've added SMMUv2 support to the "non-strict mode" series (of which I
> should be posting v8 soon), so it might be fun to see how well that
> works on MMU-500 in the real world.
Hmm, I think I gave those a try some weeks ago and vaguely remember that
I did see improvements. Can't remember the numbers off the top of my
head but I'll re-test with the latest spin and update the numbers.
>>>>>>
>>>>>> USB issue and workaround
>>>>>>
>>>>>> There's a problem with the usb controllers in these chips
>>>>>> generating smaller, 40-bit wide dma addresses instead of the
>>>>>> 48-bit
>>>>>> supported at the smmu input. So you end up in a situation
>>>>>> where the
>>>>>> smmu is mapped with 48-bit address translations, but the
>>>>>> device
>>>>>> generates transactions with clipped 40-bit addresses, thus
>>>>>> smmu
>>>>>> context faults are triggered. I encountered a similar
>>>>>> situation for
>>>>>> mmc that I managed to fix in software [1] however for USB I
>>>>>> did not
>>>>>> find a proper place in the code to add a similar fix. The only
>>>>>> workaround I found was to add this kernel parameter which
>>>>>> limits the
>>>>>> usb dma to 32-bit size: "xhci-hcd.quirks=0x800000".
>>>>>> This workaround if far from ideal, so any suggestions for a
>>>>>> code
>>>>>> based workaround in this area would be greatly appreciated.
>>>>>
>>>>> If you have a nominally-64-bit device with a
>>>>> narrower-than-the-main-interconnect link in front of it, that should
>>>>> already be fixed in 4.19-rc by bus_dma_mask picking up DT dma-ranges,
>>>>> provided the interconnect hierarchy can be described appropriately (or
>>>>> at least massaged sufficiently to satisfy the binding), e.g.:
>>>>>
>>>>> / {
>>>>> ...
>>>>>
>>>>> soc {
>>>>> ranges;
>>>>> dma-ranges = <0 0 10000 0>;
>>>>>
>>>>> dev_48bit { ... };
>>>>>
>>>>> periph_bus {
>>>>> ranges;
>>>>> dma-ranges = <0 0 100 0>;
>>>>>
>>>>> dev_40bit { ... };
>>>>> };
>>>>> };
>>>>> };
>>>>>
>>>>> and if that fails to work as expected (except for PCI hosts where
>>>>> handling dma-ranges properly still needs sorting out), please do
>>>>> let us
>>>>> know ;)
>>>>>
>>>>
>>>> Just to confirm, Is this [1] the change I was supposed to test?
>>>
>>> Not quite - dma-ranges is only valid for nodes representing a bus, so
>>> putting it directly in the USB device nodes doesn't work (FWIW that's
>>> why PCI is broken, because the parser doesn't expect the
>>> bus-as-leaf-node case). That's teh point of that intermediate simple-bus
>>> node represented by "periph_bus" in my example (sorry, I should have put
>>> compatibles in to make it clearer) - often that's actually true to life
>>> (i.e. "soc" is something like a CCI and "periph_bus" is something like
>>> an AXI NIC gluing a bunch of lower-bandwidth DMA masters to one of the
>>> CCI ports) but at worst it's just a necessary evil to make the binding
>>> happy (if it literally only represents the point-to-point link between
>>> the device master port and interconnect slave port).
>>>
>>
>> Quick update: so I adjusted to device tree according to your example and
>> it works so now I can get rid of that nasty kernel arg based workaround,
>> yey! :-)
>
> Cool! In fact, judging by the block diagrams on the website, the "basic
> peripherals and interconnect" section hanging off the side of the CCI
> implies that probably is true to the real topology as I imagined, so it
> doesn't even count as a horrible hack :)
Indeed, on this chip there's a NoC lumping behind it several low-speed
devices such as usb, sata, esdhc.
>> Thanks a lot, that was really helpful.
>
> No problem. FWIW if you ever come to doing ACPI support for these SoCs,
> the equivalent is merely a case of setting the device memory address
> size limit field appropriately for all the named components.
>
Thanks, I'll keep this in mind. If i remember correctly, there are
people over here working on UEFI + ACPI support for some LS chips but
progress appears to be slow.
---
Best Regards, Laurentiu
next prev parent reply other threads:[~2018-09-20 14:33 UTC|newest]
Thread overview: 34+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-09-19 12:35 [PATCH 00/21] SMMU enablement for NXP LS1043A and LS1046A laurentiu.tudor
2018-09-19 12:35 ` [PATCH 01/21] soc/fsl/qman: fixup liodns only on ppc targets laurentiu.tudor
2018-09-19 12:35 ` [PATCH 02/21] soc/fsl/bman: map FBPR area in the iommu laurentiu.tudor
2018-09-19 12:35 ` [PATCH 03/21] soc/fsl/qman: map FQD and PFDR areas " laurentiu.tudor
2018-09-19 12:35 ` [PATCH 04/21] soc/fsl/qman-portal: map CENA area " laurentiu.tudor
2018-09-19 12:35 ` [PATCH 05/21] soc/fsl/qbman: add APIs to retrieve the probing status laurentiu.tudor
2018-09-19 12:35 ` [PATCH 06/21] soc/fsl/qman_portals: defer probe after qman's probe laurentiu.tudor
2018-09-19 12:35 ` [PATCH 07/21] soc/fsl/bman_portals: defer probe after bman's probe laurentiu.tudor
2018-09-19 12:36 ` [PATCH 08/21] soc/fsl/qbman_portals: add APIs to retrieve the probing status laurentiu.tudor
2018-09-19 12:36 ` [PATCH 09/21] fsl/fman: backup and restore ICID registers laurentiu.tudor
2018-09-19 12:36 ` [PATCH 10/21] fsl/fman: add API to get the device behind a fman port laurentiu.tudor
2018-09-19 12:36 ` [PATCH 11/21] dpaa_eth: defer probing after qbman laurentiu.tudor
2018-09-19 12:36 ` [PATCH 12/21] dpaa_eth: base dma mappings on the fman rx port laurentiu.tudor
2018-09-19 12:36 ` [PATCH 13/21] dpaa_eth: fix iova handling for contiguous frames laurentiu.tudor
2018-09-19 12:36 ` [PATCH 14/21] dpaa_eth: fix iova handling for sg frames laurentiu.tudor
2018-09-19 12:36 ` [PATCH 15/21] dpaa_eth: fix SG frame cleanup laurentiu.tudor
2018-09-19 12:36 ` [PATCH 16/21] arm64: dts: ls1046a: add smmu node laurentiu.tudor
2018-09-19 13:30 ` Robin Murphy
2018-09-19 13:51 ` Laurentiu Tudor
2018-09-19 12:36 ` [PATCH 17/21] arm64: dts: ls1043a: " laurentiu.tudor
2018-09-19 12:36 ` [PATCH 18/21] arm64: dts: ls104xa: set mask to drop TBU ID from StreamID laurentiu.tudor
2018-09-19 13:41 ` Robin Murphy
2018-09-19 14:06 ` Laurentiu Tudor
2018-09-19 12:36 ` [PATCH 19/21] arm64: dts: ls104x: add missing dma ranges property laurentiu.tudor
2018-09-19 12:36 ` [PATCH 20/21] arm64: dts: ls104x: add iommu-map to pci controllers laurentiu.tudor
2018-09-19 12:36 ` [PATCH 21/21] arm64: dts: ls104x: make dma-coherent global to the SoC laurentiu.tudor
2018-09-19 13:25 ` [PATCH 00/21] SMMU enablement for NXP LS1043A and LS1046A Robin Murphy
2018-09-19 14:18 ` Laurentiu Tudor
2018-09-19 14:37 ` Robin Murphy
2018-09-20 10:38 ` Laurentiu Tudor
2018-09-20 11:49 ` Robin Murphy
2018-09-20 14:33 ` Laurentiu Tudor [this message]
2018-09-20 19:07 ` Li Yang
2018-09-21 7:32 ` Laurentiu Tudor
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=bb973ba9-ca45-c9f1-38d6-67e9bc05e429@nxp.com \
--to=laurentiu.tudor@nxp.com \
--cc=davem@davemloft.net \
--cc=devicetree@vger.kernel.org \
--cc=leoyang.li@nxp.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=madalin.bucur@nxp.com \
--cc=netdev@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=roy.pledge@nxp.com \
--cc=shawnguo@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).