From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.1 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, UNPARSEABLE_RELAY autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D916DC2B9F2 for ; Sat, 22 May 2021 13:59:37 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3F4C2610A6 for ; Sat, 22 May 2021 13:59:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3F4C2610A6 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=xs4all.nl Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:MIME-Version:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Message-ID:References:Subject:In-Reply-To:Cc:To:From :Date:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=wLRCph+7r13CqSyUMmJbY60HPOHD1gsVPu6Oum18Rcg=; b=g7yaMH3HEBaJlv7EWEuIWZRwiz a+QwHitp27kr7HrEhpnFAQYvenom43AT8uOM/RT+cdLc6jPZ5RAvnnD2GBfRJkhPS6k2W6bTjjMXs C5LWHYk/S5ijgHnR9AWLuGXnyPEsoXX68Yy55KCxOUJ+1BnjeE+apWvxLpQMtxcBknHd+UWp40vsb yUJipk/lEPV1NiYIZHpEH0wrFS+5K/33cZBjTcALcZmsudp6ABaIb5Gq/HTtTBuptytF/RQorN5IW vJxT53u4s/8o2Yw562B3eqkrrB2ib2EJ6OmP9VfJ4+u0H4cvDHaTIlNvBIqSXF8kOOXceYnAnoui0 4w2FyhLQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1lkS8L-002ZsM-Ki; Sat, 22 May 2021 13:57:41 +0000 Received: from bombadil.infradead.org ([2607:7c80:54:e::133]) by desiato.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1lkS8G-002ZsC-JP for linux-arm-kernel@desiato.infradead.org; Sat, 22 May 2021 13:57:36 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Message-ID:References:Subject: In-Reply-To:Cc:To:From:Date:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description; bh=IaFIA649Js6d4rKIFWzsts8ET2TnsuYcWDJjP8/szuw=; b=Bq5LICSYbDH6iBnJZ0tKozdMbS TOg9DPn7lVQBVQpowB18MKdw/GzvLPdMXzGqsmNhhYVSKkJzoVvviGi3UGz91dVuKE8wc714153ZK e3mwqaqIQ5wOtv8916wdGcY4LFOjES1Q24f0dljv7UZKRftTr/wHy2pZ/2olmFma6VDVnutbHYjCP 86438/qJLuaPf136z8jfIOypVvUpwi4EVX9rIUrv1rA3RSExKh34K+ugDzSUT8yeixiM3hjfG6eZX jIzlWpzcx0S3a3CeGfd/VaQMzJcbXTgv33kWlbdFqdd8TXRGQuyqiizsowbJOD72y0sCQscx1HQUl VNtF65Sg==; Received: from sibelius.xs4all.nl ([83.163.83.176]) by bombadil.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lkS8C-0007kt-UW for linux-arm-kernel@lists.infradead.org; Sat, 22 May 2021 13:57:35 +0000 Received: from localhost (bloch.sibelius.xs4all.nl [local]) by bloch.sibelius.xs4all.nl (OpenSMTPD) with ESMTPA id c4a205f5; Sat, 22 May 2021 15:57:27 +0200 (CEST) Date: Sat, 22 May 2021 15:57:27 +0200 (CEST) From: Mark Kettenis To: Robin Murphy , sven@svenpeter.dev Cc: devicetree@vger.kernel.org, maz@kernel.org, arnd@arndb.de, kettenis@openbsd.org, marcan@marcan.st, bhelgaas@google.com, robh+dt@kernel.org, linux-arm-kernel@lists.infradead.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org In-Reply-To: (message from Robin Murphy on Tue, 18 May 2021 15:10:01 +0100) Subject: Re: [PATCH 1/2] dt-bindings: pci: Add DT bindings for apple,pcie References: <20210516211851.74921-1-mark.kettenis@xs4all.nl> <20210516211851.74921-2-mark.kettenis@xs4all.nl> Message-ID: <5612ef8f8dd80e4d@bloch.sibelius.xs4all.nl> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210522_065733_327399_2FA1E84B X-CRM114-Status: GOOD ( 27.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org > From: Robin Murphy > Date: Tue, 18 May 2021 15:10:01 +0100 Hi Robin, > On 2021-05-16 22:18, Mark Kettenis wrote: > > From: Mark Kettenis > > > > The Apple PCIe host controller is a PCIe host controller with > > multiple root ports present in Apple ARM SoC platforms, including > > various iPhone and iPad devices and the "Apple Silicon" Macs. > > > > Signed-off-by: Mark Kettenis > > --- > > .../devicetree/bindings/pci/apple,pcie.yaml | 150 ++++++++++++++++++ > > MAINTAINERS | 1 + > > 2 files changed, 151 insertions(+) > > create mode 100644 Documentation/devicetree/bindings/pci/apple,pcie.yaml > > > > diff --git a/Documentation/devicetree/bindings/pci/apple,pcie.yaml b/Documentation/devicetree/bindings/pci/apple,pcie.yaml > > new file mode 100644 > > index 000000000000..af3c9f64e380 > > --- /dev/null > > +++ b/Documentation/devicetree/bindings/pci/apple,pcie.yaml > > @@ -0,0 +1,150 @@ > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) > > +%YAML 1.2 > > +--- > > +$id: http://devicetree.org/schemas/pci/apple,pcie.yaml# > > +$schema: http://devicetree.org/meta-schemas/core.yaml# > > + > > +title: Apple PCIe host controller > > + > > +maintainers: > > + - Mark Kettenis > > + > > +description: | > > + The Apple PCIe host controller is a PCIe host controller with > > + multiple root ports present in Apple ARM SoC platforms, including > > + various iPhone and iPad devices and the "Apple Silicon" Macs. > > + > > +allOf: > > + - $ref: /schemas/pci/pci-bus.yaml# > > + > > +properties: > > + compatible: > > + items: > > + - const: apple,t8103-pcie > > + - const: apple,pcie > > + > > + reg: > > + minItems: 4 > > + maxItems: 6 > > + > > + reg-names: > > + minItems: 4 > > + maxItems: 7 > > + items: > > + - const: ecam > > + - const: rc > > + - const: phy > > + - const: port0 > > + - const: port1 > > + - const: port2 > > + > > + ranges: > > + minItems: 2 > > + maxItems: 2 > > + > > + interrupts: > > + minItems: 3 > > + maxItems: 3 > > + > > + msi-ranges: > > + description: > > + A list of pairs , where "intid" is the first > > + interrupt number that can be used as an MSI, and "span" the size > > + of that range. > > + $ref: /schemas/types.yaml#/definitions/uint32-matrix > > + items: > > + minItems: 2 > > + maxItems: 2 > > + > > +required: > > + - compatible > > + - reg > > + - reg-names > > + - bus-range > > + - interrupts > > + - msi-controller > > + - msi-parent > > + - msi-ranges > > + > > +unevaluatedProperties: false > > + > > +examples: > > + - | > > + #include > > + #include > > + > > + soc { > > + #address-cells = <2>; > > + #size-cells = <2>; > > + > > + pcie0: pcie@690000000 { > > + compatible = "apple,t8103-pcie", "apple,pcie"; > > + device_type = "pci"; > > + > > + reg = <0x6 0x90000000 0x0 0x1000000>, > > + <0x6 0x80000000 0x0 0x4000>, > > + <0x6 0x8c000000 0x0 0x4000>, > > + <0x6 0x81000000 0x0 0x8000>, > > + <0x6 0x82000000 0x0 0x8000>, > > + <0x6 0x83000000 0x0 0x8000>; > > + reg-names = "ecam", "rc", "phy", "port0", "port1", "port2"; > > + > > + interrupt-parent = <&aic>; > > + interrupts = , > > + , > > + ; > > + > > + msi-controller; > > + msi-parent = <&pcie0>; > > + msi-ranges = <704 32>; > > + > > + iommu-map = <0x0 &dart0 0x8000 0x100>, > > + <0x100 &dart0 0x100 0x100>, > > + <0x200 &dart1 0x200 0x100>, > > + <0x300 &dart2 0x300 0x100>; > > + iommu-map-mask = <0xff00>; > > This doesn't quite add up - if the mask is ignoring the bottom 8 bits, > then each of those map entries is describing one single ID mapping, not 256. > > > + bus-range = <0 7>; > > Given that the iommu-map only covers buses 0-3, what happens to traffic > from buses 4-7? Yes, that probably needs a little bit of thought. The hardware is somewhat "interesting". The PCIe host bridge has (up to) three ports. Each port is associated with its own IOMMU/DART. Each port provides mapping logic that maps the RID to an SID. There are 16 mapping registers for the PCIe host bridge that connects the onboard devices and 64 mapping registers for the PCIe host bridges that are asociated with the Thunderbolt ports. If no mappings are enabled, it seems that all RIDs get mapped to SID 0. The Apple firmware doesn't enable any mappings and my U-Boot code doesn't change it either. The Corellium folks in their port chose a 1:1 mapping from bus number to SID and that is what the example above came from. Both my U-Boot driver and my OpenBSD actually ignore the SID and install the same IOMMU translation table for all the SIDs. That's probably good enough for U-Boot as long as we don't enable the Thunderbolt ports. But for the OS itself a bit more control is certainly desirable. Would it be reasonable to allow the device tree some flexibility in specifying the desired iommu mapping and let the OS PCIe host bride driver program the RID to SID mappings to match what's specified in the "iommu-map" and "iommu-map-mask" properties? Or is it better to just pick a mapping scheme like Corellium did and make that part of the DT binding? Thanks, Mark > > + #address-cells = <3>; > > + #size-cells = <2>; > > + ranges = <0x43000000 0x6 0xa0000000 0x6 0xa0000000 0x0 0x20000000>, > > + <0x02000000 0x0 0xc0000000 0x6 0xc0000000 0x0 0x40000000>; > > + > > + clocks = <&pcie_core_clk>, <&pcie_aux_clk>, <&pcie_ref_clk>; > > + pinctrl-0 = <&pcie_pins>; > > + pinctrl-names = "default"; > > + > > + pci@0,0 { > > + device_type = "pci"; > > + reg = <0x0 0x0 0x0 0x0 0x0>; > > + reset-gpios = <&pinctrl_ap 152 0>; > > + max-link-speed = <2>; > > + > > + #address-cells = <3>; > > + #size-cells = <2>; > > + ranges; > > + }; > > + > > + pci@1,0 { > > + device_type = "pci"; > > + reg = <0x800 0x0 0x0 0x0 0x0>; > > + reset-gpios = <&pinctrl_ap 153 0>; > > + max-link-speed = <2>; > > + > > + #address-cells = <3>; > > + #size-cells = <2>; > > + ranges; > > + }; > > + > > + pci@2,0 { > > + device_type = "pci"; > > + reg = <0x1000 0x0 0x0 0x0 0x0>; > > + reset-gpios = <&pinctrl_ap 33 0>; > > + max-link-speed = <1>; > > + > > + #address-cells = <3>; > > + #size-cells = <2>; > > + ranges; > > + }; > > + }; > > + }; > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 7327c9b778f1..789d79315485 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -1654,6 +1654,7 @@ C: irc://chat.freenode.net/asahi-dev > > T: git https://github.com/AsahiLinux/linux.git > > F: Documentation/devicetree/bindings/arm/apple.yaml > > F: Documentation/devicetree/bindings/interrupt-controller/apple,aic.yaml > > +F: Documentation/devicetree/bindings/pci/apple,pcie.yaml > > F: Documentation/devicetree/bindings/pinctrl/apple,pinctrl.yaml > > F: arch/arm64/boot/dts/apple/ > > F: drivers/irqchip/irq-apple-aic.c > > > _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel