From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5427EC433EF for ; Fri, 3 Sep 2021 14:36:50 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2F224610E7 for ; Fri, 3 Sep 2021 14:36:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233092AbhICOht (ORCPT ); Fri, 3 Sep 2021 10:37:49 -0400 Received: from mail.kernel.org ([198.145.29.99]:35400 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230332AbhICOhs (ORCPT ); Fri, 3 Sep 2021 10:37:48 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 40EBF610FC; Fri, 3 Sep 2021 14:36:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1630679808; bh=2B45XJbOwgTgbzqy60XQ3eiBE1S6v8PBW/gObbwsud8=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=NUibM1cBrIb7xdP+a9IlAobPtiDnSeLPkUNNE/lMoNDHxvqJXuhHyX0+YqqSAyqt1 T/3SNDXBa+RLpbi5W9qRBqErcWUpZNsja3UBQRrcZL3hSRqOpJVBC9DQ50e3Lk6MXu QmM7H63IU+ca7ViSkifp6z4FkxGxMs3FrVMofPJtis+mvUAXr+PVmsKuW0MGk52KSg hzm12Sdu6fCSB1clN/hJBmStmXHHmqxQGw8urgUwoy1TRNHHdyz6pY1aFP+kGpTXft wz9ydocqRJdhizSw8Fg9CWdkT/rtpk4zCqXBwy9k3p+fqY094SHO5sRuOjg+ISg2T9 6EkxA81Sb4O5w== Received: by mail-ed1-f52.google.com with SMTP id g21so8255925edw.4; Fri, 03 Sep 2021 07:36:48 -0700 (PDT) X-Gm-Message-State: AOAM533ixilmySdhA+JIvRSJoz7Xn282i6nPd3EpvK4yNNUEb6mCnFE7 EaTEJ6w5pf1qJKHTMr76PmgMvCAzgian4wJm0g== X-Google-Smtp-Source: ABdhPJwT59/l+PqQGnGKvEzHeMnfgEAWnMmugUDNzDEHJ13EnpYQ3GZ2MWlaaOqiAHgzarYc+upjNQe9xqFSFdjQb5c= X-Received: by 2002:a50:ed09:: with SMTP id j9mr4323997eds.164.1630679806621; Fri, 03 Sep 2021 07:36:46 -0700 (PDT) MIME-Version: 1.0 References: <20210423163234.3651547-1-thierry.reding@gmail.com> <20210423163234.3651547-2-thierry.reding@gmail.com> <20210520220306.GA1976116@robh.at.kernel.org> <7995b0ed-a277-ced1-b3d0-e0e7e02817a6@gmail.com> In-Reply-To: From: Rob Herring Date: Fri, 3 Sep 2021 09:36:33 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 1/5] dt-bindings: reserved-memory: Document memory region specifier To: Thierry Reding Cc: Alyssa Rosenzweig , Sven Peter , Dmitry Osipenko , Joerg Roedel , Will Deacon , Robin Murphy , Nicolin Chen , Krishna Reddy , devicetree@vger.kernel.org, Linux IOMMU , linux-tegra , dri-devel Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Precedence: bulk List-ID: X-Mailing-List: linux-tegra@vger.kernel.org On Fri, Sep 3, 2021 at 8:52 AM Thierry Reding wr= ote: > > On Fri, Sep 03, 2021 at 08:20:55AM -0500, Rob Herring wrote: > > On Wed, Sep 1, 2021 at 9:13 AM Thierry Reding wrote: > > > > > > On Fri, Jul 02, 2021 at 05:16:25PM +0300, Dmitry Osipenko wrote: > > > > 01.07.2021 21:14, Thierry Reding =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > > > > > On Tue, Jun 08, 2021 at 06:51:40PM +0200, Thierry Reding wrote: > > > > >> On Fri, May 28, 2021 at 06:54:55PM +0200, Thierry Reding wrote: > > > > >>> On Thu, May 20, 2021 at 05:03:06PM -0500, Rob Herring wrote: > > > > >>>> On Fri, Apr 23, 2021 at 06:32:30PM +0200, Thierry Reding wrote= : > > > > >>>>> From: Thierry Reding > > > > >>>>> > > > > >>>>> Reserved memory region phandle references can be accompanied = by a > > > > >>>>> specifier that provides additional information about how that= specific > > > > >>>>> reference should be treated. > > > > >>>>> > > > > >>>>> One use-case is to mark a memory region as needing an identit= y mapping > > > > >>>>> in the system's IOMMU for the device that references the regi= on. This is > > > > >>>>> needed for example when the bootloader has set up hardware (s= uch as a > > > > >>>>> display controller) to actively access a memory region (e.g. = a boot > > > > >>>>> splash screen framebuffer) during boot. The operating system = can use the > > > > >>>>> identity mapping flag from the specifier to make sure an IOMM= U identity > > > > >>>>> mapping is set up for the framebuffer before IOMMU translatio= ns are > > > > >>>>> enabled for the display controller. > > > > >>>>> > > > > >>>>> Signed-off-by: Thierry Reding > > > > >>>>> --- > > > > >>>>> .../reserved-memory/reserved-memory.txt | 21 +++++++++= ++++++++++ > > > > >>>>> include/dt-bindings/reserved-memory.h | 8 +++++++ > > > > >>>>> 2 files changed, 29 insertions(+) > > > > >>>>> create mode 100644 include/dt-bindings/reserved-memory.h > > > > >>>> > > > > >>>> Sorry for being slow on this. I have 2 concerns. > > > > >>>> > > > > >>>> First, this creates an ABI issue. A DT with cells in 'memory-r= egion' > > > > >>>> will not be understood by an existing OS. I'm less concerned a= bout this > > > > >>>> if we address that with a stable fix. (Though I'm pretty sure = we've > > > > >>>> naively added #?-cells in the past ignoring this issue.) > > > > >>> > > > > >>> A while ago I had proposed adding memory-region*s* as an altern= ative > > > > >>> name for memory-region to make the naming more consistent with = other > > > > >>> types of properties (think clocks, resets, gpios, ...). If we a= dded > > > > >>> that, we could easily differentiate between the "legacy" cases = where > > > > >>> no #memory-region-cells was allowed and the new cases where it = was. > > > > >>> > > > > >>>> Second, it could be the bootloader setting up the reserved reg= ion. If a > > > > >>>> node already has 'memory-region', then adding more regions is = more > > > > >>>> complicated compared to adding new properties. And defining wh= at each > > > > >>>> memory-region entry is or how many in schemas is impossible. > > > > >>> > > > > >>> It's true that updating the property gets a bit complicated, bu= t it's > > > > >>> not exactly rocket science. We really just need to splice the a= rray. I > > > > >>> have a working implemention for this in U-Boot. > > > > >>> > > > > >>> For what it's worth, we could run into the same issue with any = new > > > > >>> property that we add. Even if we renamed this to iommu-memory-r= egion, > > > > >>> it's still possible that a bootloader may have to update this p= roperty > > > > >>> if it already exists (it could be hard-coded in DT, or it could= have > > > > >>> been added by some earlier bootloader or firmware). > > > > >>> > > > > >>>> Both could be addressed with a new property. Perhaps something= like > > > > >>>> 'iommu-memory-region =3D <&phandle>;'. I think the 'iommu' pre= fix is > > > > >>>> appropriate given this is entirely because of the IOMMU being = in the > > > > >>>> mix. I might feel differently if we had other uses for cells, = but I > > > > >>>> don't really see it in this case. > > > > >>> > > > > >>> I'm afraid that down the road we'll end up with other cases and= then we > > > > >>> might proliferate a number of *-memory-region properties with v= arying > > > > >>> prefixes. > > > > >>> > > > > >>> I am aware of one other case where we might need something like= this: on > > > > >>> some Tegra SoCs we have audio processors that will access memor= y buffers > > > > >>> using a DMA engine. These processors are booted from early firm= ware > > > > >>> using firmware from system memory. In order to avoid trashing t= he > > > > >>> firmware, we need to reserve memory. We can do this using reser= ved > > > > >>> memory nodes. However, the audio DMA engine also uses the SMMU,= so we > > > > >>> need to make sure that the firmware memory is marked as reserve= d within > > > > >>> the SMMU. This is similar to the identity mapping case, but not= exactly > > > > >>> the same. Instead of creating a 1:1 mapping, we just want that = IOVA > > > > >>> region to be reserved (i.e. IOMMU_RESV_RESERVED instead of > > > > >>> IOMMU_RESV_DIRECT{,_RELAXABLE}). > > > > >>> > > > > >>> That would also fall into the IOMMU domain, but we can't reuse = the > > > > >>> iommu-memory-region property for that because then we don't hav= e enough > > > > >>> information to decide which type of reservation we need. > > > > >>> > > > > >>> We could obviously make iommu-memory-region take a specifier, b= ut we > > > > >>> could just as well use memory-regions in that case since we hav= e > > > > >>> something more generic anyway. > > > > >>> > > > > >>> With the #memory-region-cells proposal, we can easily extend th= e cell in > > > > >>> the specifier with an additional MEMORY_REGION_IOMMU_RESERVE fl= ag to > > > > >>> take that other use case into account. If we than also change t= o the new > > > > >>> memory-regions property name, we avoid the ABI issue (and we ga= in a bit > > > > >>> of consistency while at it). > > > > >> > > > > >> Ping? Rob, do you want me to add this second use-case to the pat= ch > > > > >> series to make it more obvious that this isn't just a one-off th= ing? Or > > > > >> how do we proceed? > > > > > > > > > > Rob, given that additional use-case, do you want me to run with t= his > > > > > proposal and send out an updated series? > > > > > > > > > > > > What about variant with a "descriptor" properties that will describ= e > > > > each region: > > > > > > > > fb_desc: display-framebuffer-memory-descriptor { > > > > needs-identity-mapping; > > > > } > > > > > > > > display@52400000 { > > > > memory-region =3D <&fb ...>; > > > > memory-region-descriptor =3D <&fb_desc ...>; > > > > }; > > > > > > > > It could be a more flexible/extendible variant. > > > > > > This problem recently came up on #dri-devel again. Adding Alyssa and > > > Sven who are facing a similar challenge on their work on Apple M1 (if= I > > > understood correctly). Also adding dri-devel for visibility since thi= s > > > is a very common problem for display in particular. > > > > > > On M1 the situation is slightly more complicated: the firmware will > > > allocate a couple of buffers (including the framebuffer) in high memo= ry > > > (> 4 GiB) and use the IOMMU to map that into an IOVA region below 4 G= iB > > > so that the display hardware can access it. This makes it impossible = to > > > bypass the IOMMU like we do on other chips (in particular to work aro= und > > > the fault-by-default policy of the ARM SMMU driver). It also means th= at > > > in addition to the simple reserved regions I mentioned we need for au= dio > > > use-cases and identity mapping use-cases we need for display on Tegra= , > > > we now also need to be able to convey physical to IOVA mappings. > > > > > > Fitting the latter into the original proposal sounds difficult. A qui= ck > > > fix would've been to generate a mapping table in memory and pass that= to > > > the kernel using a reserved-memory node (similar to what's done for > > > example on Tegra for the EMC frequency table on Tegra210) and mark it= as > > > such using a special flag. But that then involves two layers of parsi= ng, > > > which seems a bit suboptimal. Another way to shoehorn that into the > > > original proposal would've been to add flags for physical and virtual > > > address regions and use pairs to pass them using special flags. Again= , > > > this is a bit wonky because it needs these to be carefully parsed and > > > matched up. > > > > > > Another downside is that we now have a situation where some of these > > > regions are no longer "reserved-memory regions" in the traditional > > > sense. This would require an additional flag in the reserved-memory > > > region nodes to prevent the IOVA regions from being reserved. By the > > > way, this is something that would also be needed for the audio use-ca= se > > > I mentioned before, because the physical memory at that address can > > > still be used by an operating system. > > > > > > A more general solution would be to draw a bit from Dmitry's proposal > > > and introduce a new top-level "iov-reserved-memory" node. This could = be > > > modelled on the existing reserved-memory node, except that the physic= al > > > memory pages for regions represented by child nodes would not be mark= ed > > > as reserved. Only the IOVA range described by the region would be > > > reserved subsequently by the IOMMU framework and/or IOMMU driver. > > > > > > The simplest case where we just want to reserve some IOVA region coul= d > > > then be done like this: > > > > > > iov-reserved-memory { > > > /* > > > * Probably safest to default to <2>, <2> here given > > > * that most IOMMUs support either > 32 bits of IAS > > > * or OAS. > > > */ > > > #address-cells =3D <2>; > > > #size-cells =3D <2>; > > > > > > firmware: firmware@80000000 { > > > reg =3D <0 0x80000000 0 0x01000000>; > > > }; > > > }; > > > > > > audio@30000000 { > > > ... > > > iov-memory-regions =3D <&firmware>; > > > ... > > > }; > > > > > > Mappings could be represented by an IOV reserved region taking a > > > reference to the reserved-region that they map: > > > > > > reserved-memory { > > > #address-cells =3D <2>; > > > #size-cells =3D <2>; > > > > > > /* 16 MiB of framebuffer at top-of-memory */ > > > framebuffer: framebuffer@1,ff000000 { > > > reg =3D <0x1 0xff000000 0 0x01000000>; > > > no-map; > > > }; > > > }; > > > > > > iov-reserved-memory { > > > /* IOMMU supports only 32-bit output address space */ > > > #address-cells =3D <1>; > > > #size-cells =3D <1>; > > > > > > /* 16 MiB of framebuffer mapped to top of IOVA */ > > > fb: fb@ff000000 { > > > reg =3D <0 0xff000000 0 0x01000000>; > > > memory-region =3D <&framebuffer>; > > > }; > > > }; > > > > > > display@40000000 { > > > ... > > > /* optional? */ > > > memory-region =3D <&framebuffer>; > > > iov-memory-regions =3D <&fb>; > > > ... > > > }; > > > > > > It's interesting how identity mapped regions now become a trivial > > > special case of mappings. All that is needed is to make the reg prope= rty > > > of the IOV reserved region correspond to the reg property of the norm= al > > > reserved region. Alternatively, as a small optimization for lazy peop= le > > > like me, we could just allow these cases to omit the reg property and > > > instead inherit it from the referenced reserved region. > > > > > > As the second example shows it might be convenient if memory-region > > > could be derived from iov-memory-regions. This could be useful for ca= ses > > > where the driver wants to do something with the physical pages of the > > > reserved region (such as mapping them and copying out the framebuffer > > > data to another buffer so that the reserved memory can be recycled). = If > > > we have the IOV reserved region, we could provide an API to extract t= he > > > physical reserved region (if it exists). That way we could avoid > > > referencing it twice in DT. Then again, there's something elegant abo= ut > > > the explicit second reference to. It indicates the intent that we may > > > want to use the region for something other than just the IOV mapping. > > > > > > Anyway, this has been long enough. Let me know what you think. Alyssa= , > > > Sven, it'd be interesting to hear if you think this could work as a > > > solution to the problem on M1. > > > > > > Rob, I think you might like this alternative because it basically get= s > > > rid of all the points in the original proposal that you were concerne= d > > > about. Let me know what you think. > > > > Couldn't we keep this all in /reserved-memory? Just add an iova > > version of reg. Perhaps abuse 'assigned-address' for this purpose. The > > issue I see would be handling reserved iova areas without a physical > > area. That can be handled with just a iova and no reg. We already have > > a no reg case. > > I had thought about that initially. One thing I'm worried about is that > every child node in /reserved-memory will effectively cause the memory > that it described to be reserved. But we don't want that for regions > that are "virtual only" (i.e. IOMMU reservations). By virtual only, you mean no physical mapping, just a region of virtual space, right? For that we'd have no 'reg' and therefore no (physical) reservation by the OS. It's similar to non-static regions. You need a specific handler for them. We'd probably want a compatible as well for these virtual reservations. Are these being global in DT going to be a problem? Presumably we have a virtual space per IOMMU. We'd know which IOMMU based on a device's 'iommus' and 'memory-region' properties, but within /reserved-memory we wouldn't be able to distinguish overlapping addresses from separate address spaces. Or we could have 2 different IOVAs for 1 physical space. That could be solved with something like this: iommu-addresses =3D <&iommu1
>; Or the other way to do this is reuse 'iommus' property to define the mapping of each address entry to iommu. > Obviously we can fix that in Linux, but what about other operating > systems? Currently "reg" is a required property for statically allocated > regions (which all of these would be). Do you have an idea of how widely > that's used? What about other OSes, or bootloaders, what if they > encounter these nodes that don't have a "reg" property? Without 'reg', there must be a compatible that the client understands or the node should be ignored. My suspicion is that /reserved-memory is abused for all sorts of things downstream, but that's not really relevant here. Rob From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 531B6C433EF for ; Fri, 3 Sep 2021 14:36:53 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 08643610E5 for ; Fri, 3 Sep 2021 14:36:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 08643610E5 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id C59E461583; Fri, 3 Sep 2021 14:36:52 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id dD81Yois0sj5; Fri, 3 Sep 2021 14:36:51 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [140.211.9.56]) by smtp3.osuosl.org (Postfix) with ESMTPS id 23C4560644; Fri, 3 Sep 2021 14:36:51 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id E97A9C0010; Fri, 3 Sep 2021 14:36:50 +0000 (UTC) Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) by lists.linuxfoundation.org (Postfix) with ESMTP id F0473C000E for ; Fri, 3 Sep 2021 14:36:49 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id D0D3960644 for ; Fri, 3 Sep 2021 14:36:49 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0bs0Pq4ZmVy2 for ; Fri, 3 Sep 2021 14:36:48 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp3.osuosl.org (Postfix) with ESMTPS id AD0D6605C8 for ; Fri, 3 Sep 2021 14:36:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 3AC1D610E9 for ; Fri, 3 Sep 2021 14:36:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1630679808; bh=2B45XJbOwgTgbzqy60XQ3eiBE1S6v8PBW/gObbwsud8=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=NUibM1cBrIb7xdP+a9IlAobPtiDnSeLPkUNNE/lMoNDHxvqJXuhHyX0+YqqSAyqt1 T/3SNDXBa+RLpbi5W9qRBqErcWUpZNsja3UBQRrcZL3hSRqOpJVBC9DQ50e3Lk6MXu QmM7H63IU+ca7ViSkifp6z4FkxGxMs3FrVMofPJtis+mvUAXr+PVmsKuW0MGk52KSg hzm12Sdu6fCSB1clN/hJBmStmXHHmqxQGw8urgUwoy1TRNHHdyz6pY1aFP+kGpTXft wz9ydocqRJdhizSw8Fg9CWdkT/rtpk4zCqXBwy9k3p+fqY094SHO5sRuOjg+ISg2T9 6EkxA81Sb4O5w== Received: by mail-ed1-f47.google.com with SMTP id l6so8265818edb.7 for ; Fri, 03 Sep 2021 07:36:48 -0700 (PDT) X-Gm-Message-State: AOAM530Ul/i6BMo59asPxglvnAQ9TLM/kMGAEJ2uqFmHDpgxcAgtDZ9K 62hQvZBHQKI4PrUc+IifvsTigAv+qrkQ/PjDng== X-Google-Smtp-Source: ABdhPJwT59/l+PqQGnGKvEzHeMnfgEAWnMmugUDNzDEHJ13EnpYQ3GZ2MWlaaOqiAHgzarYc+upjNQe9xqFSFdjQb5c= X-Received: by 2002:a50:ed09:: with SMTP id j9mr4323997eds.164.1630679806621; Fri, 03 Sep 2021 07:36:46 -0700 (PDT) MIME-Version: 1.0 References: <20210423163234.3651547-1-thierry.reding@gmail.com> <20210423163234.3651547-2-thierry.reding@gmail.com> <20210520220306.GA1976116@robh.at.kernel.org> <7995b0ed-a277-ced1-b3d0-e0e7e02817a6@gmail.com> In-Reply-To: From: Rob Herring Date: Fri, 3 Sep 2021 09:36:33 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 1/5] dt-bindings: reserved-memory: Document memory region specifier To: Thierry Reding Cc: devicetree@vger.kernel.org, Robin Murphy , Linux IOMMU , dri-devel , Alyssa Rosenzweig , linux-tegra , Dmitry Osipenko , Will Deacon X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gRnJpLCBTZXAgMywgMjAyMSBhdCA4OjUyIEFNIFRoaWVycnkgUmVkaW5nIDx0aGllcnJ5LnJl ZGluZ0BnbWFpbC5jb20+IHdyb3RlOgo+Cj4gT24gRnJpLCBTZXAgMDMsIDIwMjEgYXQgMDg6MjA6 NTVBTSAtMDUwMCwgUm9iIEhlcnJpbmcgd3JvdGU6Cj4gPiBPbiBXZWQsIFNlcCAxLCAyMDIxIGF0 IDk6MTMgQU0gVGhpZXJyeSBSZWRpbmcgPHRoaWVycnkucmVkaW5nQGdtYWlsLmNvbT4gd3JvdGU6 Cj4gPiA+Cj4gPiA+IE9uIEZyaSwgSnVsIDAyLCAyMDIxIGF0IDA1OjE2OjI1UE0gKzAzMDAsIERt aXRyeSBPc2lwZW5rbyB3cm90ZToKPiA+ID4gPiAwMS4wNy4yMDIxIDIxOjE0LCBUaGllcnJ5IFJl ZGluZyDQv9C40YjQtdGCOgo+ID4gPiA+ID4gT24gVHVlLCBKdW4gMDgsIDIwMjEgYXQgMDY6NTE6 NDBQTSArMDIwMCwgVGhpZXJyeSBSZWRpbmcgd3JvdGU6Cj4gPiA+ID4gPj4gT24gRnJpLCBNYXkg MjgsIDIwMjEgYXQgMDY6NTQ6NTVQTSArMDIwMCwgVGhpZXJyeSBSZWRpbmcgd3JvdGU6Cj4gPiA+ ID4gPj4+IE9uIFRodSwgTWF5IDIwLCAyMDIxIGF0IDA1OjAzOjA2UE0gLTA1MDAsIFJvYiBIZXJy aW5nIHdyb3RlOgo+ID4gPiA+ID4+Pj4gT24gRnJpLCBBcHIgMjMsIDIwMjEgYXQgMDY6MzI6MzBQ TSArMDIwMCwgVGhpZXJyeSBSZWRpbmcgd3JvdGU6Cj4gPiA+ID4gPj4+Pj4gRnJvbTogVGhpZXJy eSBSZWRpbmcgPHRyZWRpbmdAbnZpZGlhLmNvbT4KPiA+ID4gPiA+Pj4+Pgo+ID4gPiA+ID4+Pj4+ IFJlc2VydmVkIG1lbW9yeSByZWdpb24gcGhhbmRsZSByZWZlcmVuY2VzIGNhbiBiZSBhY2NvbXBh bmllZCBieSBhCj4gPiA+ID4gPj4+Pj4gc3BlY2lmaWVyIHRoYXQgcHJvdmlkZXMgYWRkaXRpb25h bCBpbmZvcm1hdGlvbiBhYm91dCBob3cgdGhhdCBzcGVjaWZpYwo+ID4gPiA+ID4+Pj4+IHJlZmVy ZW5jZSBzaG91bGQgYmUgdHJlYXRlZC4KPiA+ID4gPiA+Pj4+Pgo+ID4gPiA+ID4+Pj4+IE9uZSB1 c2UtY2FzZSBpcyB0byBtYXJrIGEgbWVtb3J5IHJlZ2lvbiBhcyBuZWVkaW5nIGFuIGlkZW50aXR5 IG1hcHBpbmcKPiA+ID4gPiA+Pj4+PiBpbiB0aGUgc3lzdGVtJ3MgSU9NTVUgZm9yIHRoZSBkZXZp Y2UgdGhhdCByZWZlcmVuY2VzIHRoZSByZWdpb24uIFRoaXMgaXMKPiA+ID4gPiA+Pj4+PiBuZWVk ZWQgZm9yIGV4YW1wbGUgd2hlbiB0aGUgYm9vdGxvYWRlciBoYXMgc2V0IHVwIGhhcmR3YXJlIChz dWNoIGFzIGEKPiA+ID4gPiA+Pj4+PiBkaXNwbGF5IGNvbnRyb2xsZXIpIHRvIGFjdGl2ZWx5IGFj Y2VzcyBhIG1lbW9yeSByZWdpb24gKGUuZy4gYSBib290Cj4gPiA+ID4gPj4+Pj4gc3BsYXNoIHNj cmVlbiBmcmFtZWJ1ZmZlcikgZHVyaW5nIGJvb3QuIFRoZSBvcGVyYXRpbmcgc3lzdGVtIGNhbiB1 c2UgdGhlCj4gPiA+ID4gPj4+Pj4gaWRlbnRpdHkgbWFwcGluZyBmbGFnIGZyb20gdGhlIHNwZWNp ZmllciB0byBtYWtlIHN1cmUgYW4gSU9NTVUgaWRlbnRpdHkKPiA+ID4gPiA+Pj4+PiBtYXBwaW5n IGlzIHNldCB1cCBmb3IgdGhlIGZyYW1lYnVmZmVyIGJlZm9yZSBJT01NVSB0cmFuc2xhdGlvbnMg YXJlCj4gPiA+ID4gPj4+Pj4gZW5hYmxlZCBmb3IgdGhlIGRpc3BsYXkgY29udHJvbGxlci4KPiA+ ID4gPiA+Pj4+Pgo+ID4gPiA+ID4+Pj4+IFNpZ25lZC1vZmYtYnk6IFRoaWVycnkgUmVkaW5nIDx0 cmVkaW5nQG52aWRpYS5jb20+Cj4gPiA+ID4gPj4+Pj4gLS0tCj4gPiA+ID4gPj4+Pj4gIC4uLi9y ZXNlcnZlZC1tZW1vcnkvcmVzZXJ2ZWQtbWVtb3J5LnR4dCAgICAgICB8IDIxICsrKysrKysrKysr KysrKysrKysKPiA+ID4gPiA+Pj4+PiAgaW5jbHVkZS9kdC1iaW5kaW5ncy9yZXNlcnZlZC1tZW1v cnkuaCAgICAgICAgIHwgIDggKysrKysrKwo+ID4gPiA+ID4+Pj4+ICAyIGZpbGVzIGNoYW5nZWQs IDI5IGluc2VydGlvbnMoKykKPiA+ID4gPiA+Pj4+PiAgY3JlYXRlIG1vZGUgMTAwNjQ0IGluY2x1 ZGUvZHQtYmluZGluZ3MvcmVzZXJ2ZWQtbWVtb3J5LmgKPiA+ID4gPiA+Pj4+Cj4gPiA+ID4gPj4+ PiBTb3JyeSBmb3IgYmVpbmcgc2xvdyBvbiB0aGlzLiBJIGhhdmUgMiBjb25jZXJucy4KPiA+ID4g PiA+Pj4+Cj4gPiA+ID4gPj4+PiBGaXJzdCwgdGhpcyBjcmVhdGVzIGFuIEFCSSBpc3N1ZS4gQSBE VCB3aXRoIGNlbGxzIGluICdtZW1vcnktcmVnaW9uJwo+ID4gPiA+ID4+Pj4gd2lsbCBub3QgYmUg dW5kZXJzdG9vZCBieSBhbiBleGlzdGluZyBPUy4gSSdtIGxlc3MgY29uY2VybmVkIGFib3V0IHRo aXMKPiA+ID4gPiA+Pj4+IGlmIHdlIGFkZHJlc3MgdGhhdCB3aXRoIGEgc3RhYmxlIGZpeC4gKFRo b3VnaCBJJ20gcHJldHR5IHN1cmUgd2UndmUKPiA+ID4gPiA+Pj4+IG5haXZlbHkgYWRkZWQgIz8t Y2VsbHMgaW4gdGhlIHBhc3QgaWdub3JpbmcgdGhpcyBpc3N1ZS4pCj4gPiA+ID4gPj4+Cj4gPiA+ ID4gPj4+IEEgd2hpbGUgYWdvIEkgaGFkIHByb3Bvc2VkIGFkZGluZyBtZW1vcnktcmVnaW9uKnMq IGFzIGFuIGFsdGVybmF0aXZlCj4gPiA+ID4gPj4+IG5hbWUgZm9yIG1lbW9yeS1yZWdpb24gdG8g bWFrZSB0aGUgbmFtaW5nIG1vcmUgY29uc2lzdGVudCB3aXRoIG90aGVyCj4gPiA+ID4gPj4+IHR5 cGVzIG9mIHByb3BlcnRpZXMgKHRoaW5rIGNsb2NrcywgcmVzZXRzLCBncGlvcywgLi4uKS4gSWYg d2UgYWRkZWQKPiA+ID4gPiA+Pj4gdGhhdCwgd2UgY291bGQgZWFzaWx5IGRpZmZlcmVudGlhdGUg YmV0d2VlbiB0aGUgImxlZ2FjeSIgY2FzZXMgd2hlcmUKPiA+ID4gPiA+Pj4gbm8gI21lbW9yeS1y ZWdpb24tY2VsbHMgd2FzIGFsbG93ZWQgYW5kIHRoZSBuZXcgY2FzZXMgd2hlcmUgaXQgd2FzLgo+ ID4gPiA+ID4+Pgo+ID4gPiA+ID4+Pj4gU2Vjb25kLCBpdCBjb3VsZCBiZSB0aGUgYm9vdGxvYWRl ciBzZXR0aW5nIHVwIHRoZSByZXNlcnZlZCByZWdpb24uIElmIGEKPiA+ID4gPiA+Pj4+IG5vZGUg YWxyZWFkeSBoYXMgJ21lbW9yeS1yZWdpb24nLCB0aGVuIGFkZGluZyBtb3JlIHJlZ2lvbnMgaXMg bW9yZQo+ID4gPiA+ID4+Pj4gY29tcGxpY2F0ZWQgY29tcGFyZWQgdG8gYWRkaW5nIG5ldyBwcm9w ZXJ0aWVzLiBBbmQgZGVmaW5pbmcgd2hhdCBlYWNoCj4gPiA+ID4gPj4+PiBtZW1vcnktcmVnaW9u IGVudHJ5IGlzIG9yIGhvdyBtYW55IGluIHNjaGVtYXMgaXMgaW1wb3NzaWJsZS4KPiA+ID4gPiA+ Pj4KPiA+ID4gPiA+Pj4gSXQncyB0cnVlIHRoYXQgdXBkYXRpbmcgdGhlIHByb3BlcnR5IGdldHMg YSBiaXQgY29tcGxpY2F0ZWQsIGJ1dCBpdCdzCj4gPiA+ID4gPj4+IG5vdCBleGFjdGx5IHJvY2tl dCBzY2llbmNlLiBXZSByZWFsbHkganVzdCBuZWVkIHRvIHNwbGljZSB0aGUgYXJyYXkuIEkKPiA+ ID4gPiA+Pj4gaGF2ZSBhIHdvcmtpbmcgaW1wbGVtZW50aW9uIGZvciB0aGlzIGluIFUtQm9vdC4K PiA+ID4gPiA+Pj4KPiA+ID4gPiA+Pj4gRm9yIHdoYXQgaXQncyB3b3J0aCwgd2UgY291bGQgcnVu IGludG8gdGhlIHNhbWUgaXNzdWUgd2l0aCBhbnkgbmV3Cj4gPiA+ID4gPj4+IHByb3BlcnR5IHRo YXQgd2UgYWRkLiBFdmVuIGlmIHdlIHJlbmFtZWQgdGhpcyB0byBpb21tdS1tZW1vcnktcmVnaW9u LAo+ID4gPiA+ID4+PiBpdCdzIHN0aWxsIHBvc3NpYmxlIHRoYXQgYSBib290bG9hZGVyIG1heSBo YXZlIHRvIHVwZGF0ZSB0aGlzIHByb3BlcnR5Cj4gPiA+ID4gPj4+IGlmIGl0IGFscmVhZHkgZXhp c3RzIChpdCBjb3VsZCBiZSBoYXJkLWNvZGVkIGluIERULCBvciBpdCBjb3VsZCBoYXZlCj4gPiA+ ID4gPj4+IGJlZW4gYWRkZWQgYnkgc29tZSBlYXJsaWVyIGJvb3Rsb2FkZXIgb3IgZmlybXdhcmUp Lgo+ID4gPiA+ID4+Pgo+ID4gPiA+ID4+Pj4gQm90aCBjb3VsZCBiZSBhZGRyZXNzZWQgd2l0aCBh IG5ldyBwcm9wZXJ0eS4gUGVyaGFwcyBzb21ldGhpbmcgbGlrZQo+ID4gPiA+ID4+Pj4gJ2lvbW11 LW1lbW9yeS1yZWdpb24gPSA8JnBoYW5kbGU+OycuIEkgdGhpbmsgdGhlICdpb21tdScgcHJlZml4 IGlzCj4gPiA+ID4gPj4+PiBhcHByb3ByaWF0ZSBnaXZlbiB0aGlzIGlzIGVudGlyZWx5IGJlY2F1 c2Ugb2YgdGhlIElPTU1VIGJlaW5nIGluIHRoZQo+ID4gPiA+ID4+Pj4gbWl4LiBJIG1pZ2h0IGZl ZWwgZGlmZmVyZW50bHkgaWYgd2UgaGFkIG90aGVyIHVzZXMgZm9yIGNlbGxzLCBidXQgSQo+ID4g PiA+ID4+Pj4gZG9uJ3QgcmVhbGx5IHNlZSBpdCBpbiB0aGlzIGNhc2UuCj4gPiA+ID4gPj4+Cj4g PiA+ID4gPj4+IEknbSBhZnJhaWQgdGhhdCBkb3duIHRoZSByb2FkIHdlJ2xsIGVuZCB1cCB3aXRo IG90aGVyIGNhc2VzIGFuZCB0aGVuIHdlCj4gPiA+ID4gPj4+IG1pZ2h0IHByb2xpZmVyYXRlIGEg bnVtYmVyIG9mICotbWVtb3J5LXJlZ2lvbiBwcm9wZXJ0aWVzIHdpdGggdmFyeWluZwo+ID4gPiA+ ID4+PiBwcmVmaXhlcy4KPiA+ID4gPiA+Pj4KPiA+ID4gPiA+Pj4gSSBhbSBhd2FyZSBvZiBvbmUg b3RoZXIgY2FzZSB3aGVyZSB3ZSBtaWdodCBuZWVkIHNvbWV0aGluZyBsaWtlIHRoaXM6IG9uCj4g PiA+ID4gPj4+IHNvbWUgVGVncmEgU29DcyB3ZSBoYXZlIGF1ZGlvIHByb2Nlc3NvcnMgdGhhdCB3 aWxsIGFjY2VzcyBtZW1vcnkgYnVmZmVycwo+ID4gPiA+ID4+PiB1c2luZyBhIERNQSBlbmdpbmUu IFRoZXNlIHByb2Nlc3NvcnMgYXJlIGJvb3RlZCBmcm9tIGVhcmx5IGZpcm13YXJlCj4gPiA+ID4g Pj4+IHVzaW5nIGZpcm13YXJlIGZyb20gc3lzdGVtIG1lbW9yeS4gSW4gb3JkZXIgdG8gYXZvaWQg dHJhc2hpbmcgdGhlCj4gPiA+ID4gPj4+IGZpcm13YXJlLCB3ZSBuZWVkIHRvIHJlc2VydmUgbWVt b3J5LiBXZSBjYW4gZG8gdGhpcyB1c2luZyByZXNlcnZlZAo+ID4gPiA+ID4+PiBtZW1vcnkgbm9k ZXMuIEhvd2V2ZXIsIHRoZSBhdWRpbyBETUEgZW5naW5lIGFsc28gdXNlcyB0aGUgU01NVSwgc28g d2UKPiA+ID4gPiA+Pj4gbmVlZCB0byBtYWtlIHN1cmUgdGhhdCB0aGUgZmlybXdhcmUgbWVtb3J5 IGlzIG1hcmtlZCBhcyByZXNlcnZlZCB3aXRoaW4KPiA+ID4gPiA+Pj4gdGhlIFNNTVUuIFRoaXMg aXMgc2ltaWxhciB0byB0aGUgaWRlbnRpdHkgbWFwcGluZyBjYXNlLCBidXQgbm90IGV4YWN0bHkK PiA+ID4gPiA+Pj4gdGhlIHNhbWUuIEluc3RlYWQgb2YgY3JlYXRpbmcgYSAxOjEgbWFwcGluZywg d2UganVzdCB3YW50IHRoYXQgSU9WQQo+ID4gPiA+ID4+PiByZWdpb24gdG8gYmUgcmVzZXJ2ZWQg KGkuZS4gSU9NTVVfUkVTVl9SRVNFUlZFRCBpbnN0ZWFkIG9mCj4gPiA+ID4gPj4+IElPTU1VX1JF U1ZfRElSRUNUeyxfUkVMQVhBQkxFfSkuCj4gPiA+ID4gPj4+Cj4gPiA+ID4gPj4+IFRoYXQgd291 bGQgYWxzbyBmYWxsIGludG8gdGhlIElPTU1VIGRvbWFpbiwgYnV0IHdlIGNhbid0IHJldXNlIHRo ZQo+ID4gPiA+ID4+PiBpb21tdS1tZW1vcnktcmVnaW9uIHByb3BlcnR5IGZvciB0aGF0IGJlY2F1 c2UgdGhlbiB3ZSBkb24ndCBoYXZlIGVub3VnaAo+ID4gPiA+ID4+PiBpbmZvcm1hdGlvbiB0byBk ZWNpZGUgd2hpY2ggdHlwZSBvZiByZXNlcnZhdGlvbiB3ZSBuZWVkLgo+ID4gPiA+ID4+Pgo+ID4g PiA+ID4+PiBXZSBjb3VsZCBvYnZpb3VzbHkgbWFrZSBpb21tdS1tZW1vcnktcmVnaW9uIHRha2Ug YSBzcGVjaWZpZXIsIGJ1dCB3ZQo+ID4gPiA+ID4+PiBjb3VsZCBqdXN0IGFzIHdlbGwgdXNlIG1l bW9yeS1yZWdpb25zIGluIHRoYXQgY2FzZSBzaW5jZSB3ZSBoYXZlCj4gPiA+ID4gPj4+IHNvbWV0 aGluZyBtb3JlIGdlbmVyaWMgYW55d2F5Lgo+ID4gPiA+ID4+Pgo+ID4gPiA+ID4+PiBXaXRoIHRo ZSAjbWVtb3J5LXJlZ2lvbi1jZWxscyBwcm9wb3NhbCwgd2UgY2FuIGVhc2lseSBleHRlbmQgdGhl IGNlbGwgaW4KPiA+ID4gPiA+Pj4gdGhlIHNwZWNpZmllciB3aXRoIGFuIGFkZGl0aW9uYWwgTUVN T1JZX1JFR0lPTl9JT01NVV9SRVNFUlZFIGZsYWcgdG8KPiA+ID4gPiA+Pj4gdGFrZSB0aGF0IG90 aGVyIHVzZSBjYXNlIGludG8gYWNjb3VudC4gSWYgd2UgdGhhbiBhbHNvIGNoYW5nZSB0byB0aGUg bmV3Cj4gPiA+ID4gPj4+IG1lbW9yeS1yZWdpb25zIHByb3BlcnR5IG5hbWUsIHdlIGF2b2lkIHRo ZSBBQkkgaXNzdWUgKGFuZCB3ZSBnYWluIGEgYml0Cj4gPiA+ID4gPj4+IG9mIGNvbnNpc3RlbmN5 IHdoaWxlIGF0IGl0KS4KPiA+ID4gPiA+Pgo+ID4gPiA+ID4+IFBpbmc/IFJvYiwgZG8geW91IHdh bnQgbWUgdG8gYWRkIHRoaXMgc2Vjb25kIHVzZS1jYXNlIHRvIHRoZSBwYXRjaAo+ID4gPiA+ID4+ IHNlcmllcyB0byBtYWtlIGl0IG1vcmUgb2J2aW91cyB0aGF0IHRoaXMgaXNuJ3QganVzdCBhIG9u ZS1vZmYgdGhpbmc/IE9yCj4gPiA+ID4gPj4gaG93IGRvIHdlIHByb2NlZWQ/Cj4gPiA+ID4gPgo+ ID4gPiA+ID4gUm9iLCBnaXZlbiB0aGF0IGFkZGl0aW9uYWwgdXNlLWNhc2UsIGRvIHlvdSB3YW50 IG1lIHRvIHJ1biB3aXRoIHRoaXMKPiA+ID4gPiA+IHByb3Bvc2FsIGFuZCBzZW5kIG91dCBhbiB1 cGRhdGVkIHNlcmllcz8KPiA+ID4gPgo+ID4gPiA+Cj4gPiA+ID4gV2hhdCBhYm91dCB2YXJpYW50 IHdpdGggYSAiZGVzY3JpcHRvciIgcHJvcGVydGllcyB0aGF0IHdpbGwgZGVzY3JpYmUKPiA+ID4g PiBlYWNoIHJlZ2lvbjoKPiA+ID4gPgo+ID4gPiA+IGZiX2Rlc2M6IGRpc3BsYXktZnJhbWVidWZm ZXItbWVtb3J5LWRlc2NyaXB0b3Igewo+ID4gPiA+ICAgICAgIG5lZWRzLWlkZW50aXR5LW1hcHBp bmc7Cj4gPiA+ID4gfQo+ID4gPiA+Cj4gPiA+ID4gZGlzcGxheUA1MjQwMDAwMCB7Cj4gPiA+ID4g ICAgICAgbWVtb3J5LXJlZ2lvbiA9IDwmZmIgLi4uPjsKPiA+ID4gPiAgICAgICBtZW1vcnktcmVn aW9uLWRlc2NyaXB0b3IgPSA8JmZiX2Rlc2MgLi4uPjsKPiA+ID4gPiB9Owo+ID4gPiA+Cj4gPiA+ ID4gSXQgY291bGQgYmUgYSBtb3JlIGZsZXhpYmxlL2V4dGVuZGlibGUgdmFyaWFudC4KPiA+ID4K PiA+ID4gVGhpcyBwcm9ibGVtIHJlY2VudGx5IGNhbWUgdXAgb24gI2RyaS1kZXZlbCBhZ2Fpbi4g QWRkaW5nIEFseXNzYSBhbmQKPiA+ID4gU3ZlbiB3aG8gYXJlIGZhY2luZyBhIHNpbWlsYXIgY2hh bGxlbmdlIG9uIHRoZWlyIHdvcmsgb24gQXBwbGUgTTEgKGlmIEkKPiA+ID4gdW5kZXJzdG9vZCBj b3JyZWN0bHkpLiBBbHNvIGFkZGluZyBkcmktZGV2ZWwgZm9yIHZpc2liaWxpdHkgc2luY2UgdGhp cwo+ID4gPiBpcyBhIHZlcnkgY29tbW9uIHByb2JsZW0gZm9yIGRpc3BsYXkgaW4gcGFydGljdWxh ci4KPiA+ID4KPiA+ID4gT24gTTEgdGhlIHNpdHVhdGlvbiBpcyBzbGlnaHRseSBtb3JlIGNvbXBs aWNhdGVkOiB0aGUgZmlybXdhcmUgd2lsbAo+ID4gPiBhbGxvY2F0ZSBhIGNvdXBsZSBvZiBidWZm ZXJzIChpbmNsdWRpbmcgdGhlIGZyYW1lYnVmZmVyKSBpbiBoaWdoIG1lbW9yeQo+ID4gPiAoPiA0 IEdpQikgYW5kIHVzZSB0aGUgSU9NTVUgdG8gbWFwIHRoYXQgaW50byBhbiBJT1ZBIHJlZ2lvbiBi ZWxvdyA0IEdpQgo+ID4gPiBzbyB0aGF0IHRoZSBkaXNwbGF5IGhhcmR3YXJlIGNhbiBhY2Nlc3Mg aXQuIFRoaXMgbWFrZXMgaXQgaW1wb3NzaWJsZSB0bwo+ID4gPiBieXBhc3MgdGhlIElPTU1VIGxp a2Ugd2UgZG8gb24gb3RoZXIgY2hpcHMgKGluIHBhcnRpY3VsYXIgdG8gd29yayBhcm91bmQKPiA+ ID4gdGhlIGZhdWx0LWJ5LWRlZmF1bHQgcG9saWN5IG9mIHRoZSBBUk0gU01NVSBkcml2ZXIpLiBJ dCBhbHNvIG1lYW5zIHRoYXQKPiA+ID4gaW4gYWRkaXRpb24gdG8gdGhlIHNpbXBsZSByZXNlcnZl ZCByZWdpb25zIEkgbWVudGlvbmVkIHdlIG5lZWQgZm9yIGF1ZGlvCj4gPiA+IHVzZS1jYXNlcyBh bmQgaWRlbnRpdHkgbWFwcGluZyB1c2UtY2FzZXMgd2UgbmVlZCBmb3IgZGlzcGxheSBvbiBUZWdy YSwKPiA+ID4gd2Ugbm93IGFsc28gbmVlZCB0byBiZSBhYmxlIHRvIGNvbnZleSBwaHlzaWNhbCB0 byBJT1ZBIG1hcHBpbmdzLgo+ID4gPgo+ID4gPiBGaXR0aW5nIHRoZSBsYXR0ZXIgaW50byB0aGUg b3JpZ2luYWwgcHJvcG9zYWwgc291bmRzIGRpZmZpY3VsdC4gQSBxdWljawo+ID4gPiBmaXggd291 bGQndmUgYmVlbiB0byBnZW5lcmF0ZSBhIG1hcHBpbmcgdGFibGUgaW4gbWVtb3J5IGFuZCBwYXNz IHRoYXQgdG8KPiA+ID4gdGhlIGtlcm5lbCB1c2luZyBhIHJlc2VydmVkLW1lbW9yeSBub2RlIChz aW1pbGFyIHRvIHdoYXQncyBkb25lIGZvcgo+ID4gPiBleGFtcGxlIG9uIFRlZ3JhIGZvciB0aGUg RU1DIGZyZXF1ZW5jeSB0YWJsZSBvbiBUZWdyYTIxMCkgYW5kIG1hcmsgaXQgYXMKPiA+ID4gc3Vj aCB1c2luZyBhIHNwZWNpYWwgZmxhZy4gQnV0IHRoYXQgdGhlbiBpbnZvbHZlcyB0d28gbGF5ZXJz IG9mIHBhcnNpbmcsCj4gPiA+IHdoaWNoIHNlZW1zIGEgYml0IHN1Ym9wdGltYWwuIEFub3RoZXIg d2F5IHRvIHNob2Vob3JuIHRoYXQgaW50byB0aGUKPiA+ID4gb3JpZ2luYWwgcHJvcG9zYWwgd291 bGQndmUgYmVlbiB0byBhZGQgZmxhZ3MgZm9yIHBoeXNpY2FsIGFuZCB2aXJ0dWFsCj4gPiA+IGFk ZHJlc3MgcmVnaW9ucyBhbmQgdXNlIHBhaXJzIHRvIHBhc3MgdGhlbSB1c2luZyBzcGVjaWFsIGZs YWdzLiBBZ2FpbiwKPiA+ID4gdGhpcyBpcyBhIGJpdCB3b25reSBiZWNhdXNlIGl0IG5lZWRzIHRo ZXNlIHRvIGJlIGNhcmVmdWxseSBwYXJzZWQgYW5kCj4gPiA+IG1hdGNoZWQgdXAuCj4gPiA+Cj4g PiA+IEFub3RoZXIgZG93bnNpZGUgaXMgdGhhdCB3ZSBub3cgaGF2ZSBhIHNpdHVhdGlvbiB3aGVy ZSBzb21lIG9mIHRoZXNlCj4gPiA+IHJlZ2lvbnMgYXJlIG5vIGxvbmdlciAicmVzZXJ2ZWQtbWVt b3J5IHJlZ2lvbnMiIGluIHRoZSB0cmFkaXRpb25hbAo+ID4gPiBzZW5zZS4gVGhpcyB3b3VsZCBy ZXF1aXJlIGFuIGFkZGl0aW9uYWwgZmxhZyBpbiB0aGUgcmVzZXJ2ZWQtbWVtb3J5Cj4gPiA+IHJl Z2lvbiBub2RlcyB0byBwcmV2ZW50IHRoZSBJT1ZBIHJlZ2lvbnMgZnJvbSBiZWluZyByZXNlcnZl ZC4gQnkgdGhlCj4gPiA+IHdheSwgdGhpcyBpcyBzb21ldGhpbmcgdGhhdCB3b3VsZCBhbHNvIGJl IG5lZWRlZCBmb3IgdGhlIGF1ZGlvIHVzZS1jYXNlCj4gPiA+IEkgbWVudGlvbmVkIGJlZm9yZSwg YmVjYXVzZSB0aGUgcGh5c2ljYWwgbWVtb3J5IGF0IHRoYXQgYWRkcmVzcyBjYW4KPiA+ID4gc3Rp bGwgYmUgdXNlZCBieSBhbiBvcGVyYXRpbmcgc3lzdGVtLgo+ID4gPgo+ID4gPiBBIG1vcmUgZ2Vu ZXJhbCBzb2x1dGlvbiB3b3VsZCBiZSB0byBkcmF3IGEgYml0IGZyb20gRG1pdHJ5J3MgcHJvcG9z YWwKPiA+ID4gYW5kIGludHJvZHVjZSBhIG5ldyB0b3AtbGV2ZWwgImlvdi1yZXNlcnZlZC1tZW1v cnkiIG5vZGUuIFRoaXMgY291bGQgYmUKPiA+ID4gbW9kZWxsZWQgb24gdGhlIGV4aXN0aW5nIHJl c2VydmVkLW1lbW9yeSBub2RlLCBleGNlcHQgdGhhdCB0aGUgcGh5c2ljYWwKPiA+ID4gbWVtb3J5 IHBhZ2VzIGZvciByZWdpb25zIHJlcHJlc2VudGVkIGJ5IGNoaWxkIG5vZGVzIHdvdWxkIG5vdCBi ZSBtYXJrZWQKPiA+ID4gYXMgcmVzZXJ2ZWQuIE9ubHkgdGhlIElPVkEgcmFuZ2UgZGVzY3JpYmVk IGJ5IHRoZSByZWdpb24gd291bGQgYmUKPiA+ID4gcmVzZXJ2ZWQgc3Vic2VxdWVudGx5IGJ5IHRo ZSBJT01NVSBmcmFtZXdvcmsgYW5kL29yIElPTU1VIGRyaXZlci4KPiA+ID4KPiA+ID4gVGhlIHNp bXBsZXN0IGNhc2Ugd2hlcmUgd2UganVzdCB3YW50IHRvIHJlc2VydmUgc29tZSBJT1ZBIHJlZ2lv biBjb3VsZAo+ID4gPiB0aGVuIGJlIGRvbmUgbGlrZSB0aGlzOgo+ID4gPgo+ID4gPiAgICAgICAg IGlvdi1yZXNlcnZlZC1tZW1vcnkgewo+ID4gPiAgICAgICAgICAgICAgICAgLyoKPiA+ID4gICAg ICAgICAgICAgICAgICAqIFByb2JhYmx5IHNhZmVzdCB0byBkZWZhdWx0IHRvIDwyPiwgPDI+IGhl cmUgZ2l2ZW4KPiA+ID4gICAgICAgICAgICAgICAgICAqIHRoYXQgbW9zdCBJT01NVXMgc3VwcG9y dCBlaXRoZXIgPiAzMiBiaXRzIG9mIElBUwo+ID4gPiAgICAgICAgICAgICAgICAgICogb3IgT0FT Lgo+ID4gPiAgICAgICAgICAgICAgICAgICovCj4gPiA+ICAgICAgICAgICAgICAgICAjYWRkcmVz cy1jZWxscyA9IDwyPjsKPiA+ID4gICAgICAgICAgICAgICAgICNzaXplLWNlbGxzID0gPDI+Owo+ ID4gPgo+ID4gPiAgICAgICAgICAgICAgICAgZmlybXdhcmU6IGZpcm13YXJlQDgwMDAwMDAwIHsK PiA+ID4gICAgICAgICAgICAgICAgICAgICAgICAgcmVnID0gPDAgMHg4MDAwMDAwMCAwIDB4MDEw MDAwMDA+Owo+ID4gPiAgICAgICAgICAgICAgICAgfTsKPiA+ID4gICAgICAgICB9Owo+ID4gPgo+ ID4gPiAgICAgICAgIGF1ZGlvQDMwMDAwMDAwIHsKPiA+ID4gICAgICAgICAgICAgICAgIC4uLgo+ ID4gPiAgICAgICAgICAgICAgICAgaW92LW1lbW9yeS1yZWdpb25zID0gPCZmaXJtd2FyZT47Cj4g PiA+ICAgICAgICAgICAgICAgICAuLi4KPiA+ID4gICAgICAgICB9Owo+ID4gPgo+ID4gPiBNYXBw aW5ncyBjb3VsZCBiZSByZXByZXNlbnRlZCBieSBhbiBJT1YgcmVzZXJ2ZWQgcmVnaW9uIHRha2lu ZyBhCj4gPiA+IHJlZmVyZW5jZSB0byB0aGUgcmVzZXJ2ZWQtcmVnaW9uIHRoYXQgdGhleSBtYXA6 Cj4gPiA+Cj4gPiA+ICAgICAgICAgcmVzZXJ2ZWQtbWVtb3J5IHsKPiA+ID4gICAgICAgICAgICAg ICAgICNhZGRyZXNzLWNlbGxzID0gPDI+Owo+ID4gPiAgICAgICAgICAgICAgICAgI3NpemUtY2Vs bHMgPSA8Mj47Cj4gPiA+Cj4gPiA+ICAgICAgICAgICAgICAgICAvKiAxNiBNaUIgb2YgZnJhbWVi dWZmZXIgYXQgdG9wLW9mLW1lbW9yeSAqLwo+ID4gPiAgICAgICAgICAgICAgICAgZnJhbWVidWZm ZXI6IGZyYW1lYnVmZmVyQDEsZmYwMDAwMDAgewo+ID4gPiAgICAgICAgICAgICAgICAgICAgICAg ICByZWcgPSA8MHgxIDB4ZmYwMDAwMDAgMCAweDAxMDAwMDAwPjsKPiA+ID4gICAgICAgICAgICAg ICAgICAgICAgICAgbm8tbWFwOwo+ID4gPiAgICAgICAgICAgICAgICAgfTsKPiA+ID4gICAgICAg ICB9Owo+ID4gPgo+ID4gPiAgICAgICAgIGlvdi1yZXNlcnZlZC1tZW1vcnkgewo+ID4gPiAgICAg ICAgICAgICAgICAgLyogSU9NTVUgc3VwcG9ydHMgb25seSAzMi1iaXQgb3V0cHV0IGFkZHJlc3Mg c3BhY2UgKi8KPiA+ID4gICAgICAgICAgICAgICAgICNhZGRyZXNzLWNlbGxzID0gPDE+Owo+ID4g PiAgICAgICAgICAgICAgICAgI3NpemUtY2VsbHMgPSA8MT47Cj4gPiA+Cj4gPiA+ICAgICAgICAg ICAgICAgICAvKiAxNiBNaUIgb2YgZnJhbWVidWZmZXIgbWFwcGVkIHRvIHRvcCBvZiBJT1ZBICov Cj4gPiA+ICAgICAgICAgICAgICAgICBmYjogZmJAZmYwMDAwMDAgewo+ID4gPiAgICAgICAgICAg ICAgICAgICAgICAgICByZWcgPSA8MCAweGZmMDAwMDAwIDAgMHgwMTAwMDAwMD47Cj4gPiA+ICAg ICAgICAgICAgICAgICAgICAgICAgIG1lbW9yeS1yZWdpb24gPSA8JmZyYW1lYnVmZmVyPjsKPiA+ ID4gICAgICAgICAgICAgICAgIH07Cj4gPiA+ICAgICAgICAgfTsKPiA+ID4KPiA+ID4gICAgICAg ICBkaXNwbGF5QDQwMDAwMDAwIHsKPiA+ID4gICAgICAgICAgICAgICAgIC4uLgo+ID4gPiAgICAg ICAgICAgICAgICAgLyogb3B0aW9uYWw/ICovCj4gPiA+ICAgICAgICAgICAgICAgICBtZW1vcnkt cmVnaW9uID0gPCZmcmFtZWJ1ZmZlcj47Cj4gPiA+ICAgICAgICAgICAgICAgICBpb3YtbWVtb3J5 LXJlZ2lvbnMgPSA8JmZiPjsKPiA+ID4gICAgICAgICAgICAgICAgIC4uLgo+ID4gPiAgICAgICAg IH07Cj4gPiA+Cj4gPiA+IEl0J3MgaW50ZXJlc3RpbmcgaG93IGlkZW50aXR5IG1hcHBlZCByZWdp b25zIG5vdyBiZWNvbWUgYSB0cml2aWFsCj4gPiA+IHNwZWNpYWwgY2FzZSBvZiBtYXBwaW5ncy4g QWxsIHRoYXQgaXMgbmVlZGVkIGlzIHRvIG1ha2UgdGhlIHJlZyBwcm9wZXJ0eQo+ID4gPiBvZiB0 aGUgSU9WIHJlc2VydmVkIHJlZ2lvbiBjb3JyZXNwb25kIHRvIHRoZSByZWcgcHJvcGVydHkgb2Yg dGhlIG5vcm1hbAo+ID4gPiByZXNlcnZlZCByZWdpb24uIEFsdGVybmF0aXZlbHksIGFzIGEgc21h bGwgb3B0aW1pemF0aW9uIGZvciBsYXp5IHBlb3BsZQo+ID4gPiBsaWtlIG1lLCB3ZSBjb3VsZCBq dXN0IGFsbG93IHRoZXNlIGNhc2VzIHRvIG9taXQgdGhlIHJlZyBwcm9wZXJ0eSBhbmQKPiA+ID4g aW5zdGVhZCBpbmhlcml0IGl0IGZyb20gdGhlIHJlZmVyZW5jZWQgcmVzZXJ2ZWQgcmVnaW9uLgo+ ID4gPgo+ID4gPiBBcyB0aGUgc2Vjb25kIGV4YW1wbGUgc2hvd3MgaXQgbWlnaHQgYmUgY29udmVu aWVudCBpZiBtZW1vcnktcmVnaW9uCj4gPiA+IGNvdWxkIGJlIGRlcml2ZWQgZnJvbSBpb3YtbWVt b3J5LXJlZ2lvbnMuIFRoaXMgY291bGQgYmUgdXNlZnVsIGZvciBjYXNlcwo+ID4gPiB3aGVyZSB0 aGUgZHJpdmVyIHdhbnRzIHRvIGRvIHNvbWV0aGluZyB3aXRoIHRoZSBwaHlzaWNhbCBwYWdlcyBv ZiB0aGUKPiA+ID4gcmVzZXJ2ZWQgcmVnaW9uIChzdWNoIGFzIG1hcHBpbmcgdGhlbSBhbmQgY29w eWluZyBvdXQgdGhlIGZyYW1lYnVmZmVyCj4gPiA+IGRhdGEgdG8gYW5vdGhlciBidWZmZXIgc28g dGhhdCB0aGUgcmVzZXJ2ZWQgbWVtb3J5IGNhbiBiZSByZWN5Y2xlZCkuIElmCj4gPiA+IHdlIGhh dmUgdGhlIElPViByZXNlcnZlZCByZWdpb24sIHdlIGNvdWxkIHByb3ZpZGUgYW4gQVBJIHRvIGV4 dHJhY3QgdGhlCj4gPiA+IHBoeXNpY2FsIHJlc2VydmVkIHJlZ2lvbiAoaWYgaXQgZXhpc3RzKS4g VGhhdCB3YXkgd2UgY291bGQgYXZvaWQKPiA+ID4gcmVmZXJlbmNpbmcgaXQgdHdpY2UgaW4gRFQu IFRoZW4gYWdhaW4sIHRoZXJlJ3Mgc29tZXRoaW5nIGVsZWdhbnQgYWJvdXQKPiA+ID4gdGhlIGV4 cGxpY2l0IHNlY29uZCByZWZlcmVuY2UgdG8uIEl0IGluZGljYXRlcyB0aGUgaW50ZW50IHRoYXQg d2UgbWF5Cj4gPiA+IHdhbnQgdG8gdXNlIHRoZSByZWdpb24gZm9yIHNvbWV0aGluZyBvdGhlciB0 aGFuIGp1c3QgdGhlIElPViBtYXBwaW5nLgo+ID4gPgo+ID4gPiBBbnl3YXksIHRoaXMgaGFzIGJl ZW4gbG9uZyBlbm91Z2guIExldCBtZSBrbm93IHdoYXQgeW91IHRoaW5rLiBBbHlzc2EsCj4gPiA+ IFN2ZW4sIGl0J2QgYmUgaW50ZXJlc3RpbmcgdG8gaGVhciBpZiB5b3UgdGhpbmsgdGhpcyBjb3Vs ZCB3b3JrIGFzIGEKPiA+ID4gc29sdXRpb24gdG8gdGhlIHByb2JsZW0gb24gTTEuCj4gPiA+Cj4g PiA+IFJvYiwgSSB0aGluayB5b3UgbWlnaHQgbGlrZSB0aGlzIGFsdGVybmF0aXZlIGJlY2F1c2Ug aXQgYmFzaWNhbGx5IGdldHMKPiA+ID4gcmlkIG9mIGFsbCB0aGUgcG9pbnRzIGluIHRoZSBvcmln aW5hbCBwcm9wb3NhbCB0aGF0IHlvdSB3ZXJlIGNvbmNlcm5lZAo+ID4gPiBhYm91dC4gTGV0IG1l IGtub3cgd2hhdCB5b3UgdGhpbmsuCj4gPgo+ID4gQ291bGRuJ3Qgd2Uga2VlcCB0aGlzIGFsbCBp biAvcmVzZXJ2ZWQtbWVtb3J5PyBKdXN0IGFkZCBhbiBpb3ZhCj4gPiB2ZXJzaW9uIG9mIHJlZy4g UGVyaGFwcyBhYnVzZSAnYXNzaWduZWQtYWRkcmVzcycgZm9yIHRoaXMgcHVycG9zZS4gVGhlCj4g PiBpc3N1ZSBJIHNlZSB3b3VsZCBiZSBoYW5kbGluZyByZXNlcnZlZCBpb3ZhIGFyZWFzIHdpdGhv dXQgYSBwaHlzaWNhbAo+ID4gYXJlYS4gVGhhdCBjYW4gYmUgaGFuZGxlZCB3aXRoIGp1c3QgYSBp b3ZhIGFuZCBubyByZWcuIFdlIGFscmVhZHkgaGF2ZQo+ID4gYSBubyByZWcgY2FzZS4KPgo+IEkg aGFkIHRob3VnaHQgYWJvdXQgdGhhdCBpbml0aWFsbHkuIE9uZSB0aGluZyBJJ20gd29ycmllZCBh Ym91dCBpcyB0aGF0Cj4gZXZlcnkgY2hpbGQgbm9kZSBpbiAvcmVzZXJ2ZWQtbWVtb3J5IHdpbGwg ZWZmZWN0aXZlbHkgY2F1c2UgdGhlIG1lbW9yeQo+IHRoYXQgaXQgZGVzY3JpYmVkIHRvIGJlIHJl c2VydmVkLiBCdXQgd2UgZG9uJ3Qgd2FudCB0aGF0IGZvciByZWdpb25zCj4gdGhhdCBhcmUgInZp cnR1YWwgb25seSIgKGkuZS4gSU9NTVUgcmVzZXJ2YXRpb25zKS4KCkJ5IHZpcnR1YWwgb25seSwg eW91IG1lYW4gbm8gcGh5c2ljYWwgbWFwcGluZywganVzdCBhIHJlZ2lvbiBvZgp2aXJ0dWFsIHNw YWNlLCByaWdodD8gRm9yIHRoYXQgd2UnZCBoYXZlIG5vICdyZWcnIGFuZCB0aGVyZWZvcmUgbm8K KHBoeXNpY2FsKSByZXNlcnZhdGlvbiBieSB0aGUgT1MuIEl0J3Mgc2ltaWxhciB0byBub24tc3Rh dGljIHJlZ2lvbnMuCllvdSBuZWVkIGEgc3BlY2lmaWMgaGFuZGxlciBmb3IgdGhlbS4gV2UnZCBw cm9iYWJseSB3YW50IGEgY29tcGF0aWJsZQphcyB3ZWxsIGZvciB0aGVzZSB2aXJ0dWFsIHJlc2Vy dmF0aW9ucy4KCkFyZSB0aGVzZSBiZWluZyBnbG9iYWwgaW4gRFQgZ29pbmcgdG8gYmUgYSBwcm9i bGVtPyBQcmVzdW1hYmx5IHdlIGhhdmUKYSB2aXJ0dWFsIHNwYWNlIHBlciBJT01NVS4gV2UnZCBr bm93IHdoaWNoIElPTU1VIGJhc2VkIG9uIGEgZGV2aWNlJ3MKJ2lvbW11cycgYW5kICdtZW1vcnkt cmVnaW9uJyBwcm9wZXJ0aWVzLCBidXQgd2l0aGluIC9yZXNlcnZlZC1tZW1vcnkKd2Ugd291bGRu J3QgYmUgYWJsZSB0byBkaXN0aW5ndWlzaCBvdmVybGFwcGluZyBhZGRyZXNzZXMgZnJvbSBzZXBh cmF0ZQphZGRyZXNzIHNwYWNlcy4gT3Igd2UgY291bGQgaGF2ZSAyIGRpZmZlcmVudCBJT1ZBcyBm b3IgMSBwaHlzaWNhbApzcGFjZS4gVGhhdCBjb3VsZCBiZSBzb2x2ZWQgd2l0aCBzb21ldGhpbmcg bGlrZSB0aGlzOgoKaW9tbXUtYWRkcmVzc2VzID0gPCZpb21tdTEgPGFkZHJlc3MgY2VsbHM+IDxz aXplIGNlbGxzPj47CgpPciB0aGUgb3RoZXIgd2F5IHRvIGRvIHRoaXMgaXMgcmV1c2UgJ2lvbW11 cycgcHJvcGVydHkgdG8gZGVmaW5lIHRoZQptYXBwaW5nIG9mIGVhY2ggYWRkcmVzcyBlbnRyeSB0 byBpb21tdS4KCj4gT2J2aW91c2x5IHdlIGNhbiBmaXggdGhhdCBpbiBMaW51eCwgYnV0IHdoYXQg YWJvdXQgb3RoZXIgb3BlcmF0aW5nCj4gc3lzdGVtcz8gQ3VycmVudGx5ICJyZWciIGlzIGEgcmVx dWlyZWQgcHJvcGVydHkgZm9yIHN0YXRpY2FsbHkgYWxsb2NhdGVkCj4gcmVnaW9ucyAod2hpY2gg YWxsIG9mIHRoZXNlIHdvdWxkIGJlKS4gRG8geW91IGhhdmUgYW4gaWRlYSBvZiBob3cgd2lkZWx5 Cj4gdGhhdCdzIHVzZWQ/IFdoYXQgYWJvdXQgb3RoZXIgT1Nlcywgb3IgYm9vdGxvYWRlcnMsIHdo YXQgaWYgdGhleQo+IGVuY291bnRlciB0aGVzZSBub2RlcyB0aGF0IGRvbid0IGhhdmUgYSAicmVn IiBwcm9wZXJ0eT8KCldpdGhvdXQgJ3JlZycsIHRoZXJlIG11c3QgYmUgYSBjb21wYXRpYmxlIHRo YXQgdGhlIGNsaWVudCB1bmRlcnN0YW5kcwpvciB0aGUgbm9kZSBzaG91bGQgYmUgaWdub3JlZC4K Ck15IHN1c3BpY2lvbiBpcyB0aGF0IC9yZXNlcnZlZC1tZW1vcnkgaXMgYWJ1c2VkIGZvciBhbGwg c29ydHMgb2YKdGhpbmdzIGRvd25zdHJlYW0sIGJ1dCB0aGF0J3Mgbm90IHJlYWxseSByZWxldmFu dCBoZXJlLgoKUm9iCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f X19fCmlvbW11IG1haWxpbmcgbGlzdAppb21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpo dHRwczovL2xpc3RzLmxpbnV4Zm91bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQ== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.4 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C60E1C433FE for ; Fri, 3 Sep 2021 14:36:50 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 905F6610E6 for ; Fri, 3 Sep 2021 14:36:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 905F6610E6 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DF9006E89F; Fri, 3 Sep 2021 14:36:49 +0000 (UTC) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by gabe.freedesktop.org (Postfix) with ESMTPS id D5BF56E89F for ; Fri, 3 Sep 2021 14:36:48 +0000 (UTC) Received: by mail.kernel.org (Postfix) with ESMTPSA id 23773610E6 for ; Fri, 3 Sep 2021 14:36:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1630679808; bh=2B45XJbOwgTgbzqy60XQ3eiBE1S6v8PBW/gObbwsud8=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=NUibM1cBrIb7xdP+a9IlAobPtiDnSeLPkUNNE/lMoNDHxvqJXuhHyX0+YqqSAyqt1 T/3SNDXBa+RLpbi5W9qRBqErcWUpZNsja3UBQRrcZL3hSRqOpJVBC9DQ50e3Lk6MXu QmM7H63IU+ca7ViSkifp6z4FkxGxMs3FrVMofPJtis+mvUAXr+PVmsKuW0MGk52KSg hzm12Sdu6fCSB1clN/hJBmStmXHHmqxQGw8urgUwoy1TRNHHdyz6pY1aFP+kGpTXft wz9ydocqRJdhizSw8Fg9CWdkT/rtpk4zCqXBwy9k3p+fqY094SHO5sRuOjg+ISg2T9 6EkxA81Sb4O5w== Received: by mail-ed1-f43.google.com with SMTP id q3so8258434edt.5 for ; Fri, 03 Sep 2021 07:36:48 -0700 (PDT) X-Gm-Message-State: AOAM533JVoWU2rJbbQ0+TwUeBSToxdzinrIW7yNL3Ly6hXAH1reCvrX0 +JfcfopNxWaG0qXMUooRhesd1ETglXI0wN87qQ== X-Google-Smtp-Source: ABdhPJwT59/l+PqQGnGKvEzHeMnfgEAWnMmugUDNzDEHJ13EnpYQ3GZ2MWlaaOqiAHgzarYc+upjNQe9xqFSFdjQb5c= X-Received: by 2002:a50:ed09:: with SMTP id j9mr4323997eds.164.1630679806621; Fri, 03 Sep 2021 07:36:46 -0700 (PDT) MIME-Version: 1.0 References: <20210423163234.3651547-1-thierry.reding@gmail.com> <20210423163234.3651547-2-thierry.reding@gmail.com> <20210520220306.GA1976116@robh.at.kernel.org> <7995b0ed-a277-ced1-b3d0-e0e7e02817a6@gmail.com> In-Reply-To: From: Rob Herring Date: Fri, 3 Sep 2021 09:36:33 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH v2 1/5] dt-bindings: reserved-memory: Document memory region specifier To: Thierry Reding Cc: Alyssa Rosenzweig , Sven Peter , Dmitry Osipenko , Joerg Roedel , Will Deacon , Robin Murphy , Nicolin Chen , Krishna Reddy , devicetree@vger.kernel.org, Linux IOMMU , linux-tegra , dri-devel Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On Fri, Sep 3, 2021 at 8:52 AM Thierry Reding wr= ote: > > On Fri, Sep 03, 2021 at 08:20:55AM -0500, Rob Herring wrote: > > On Wed, Sep 1, 2021 at 9:13 AM Thierry Reding wrote: > > > > > > On Fri, Jul 02, 2021 at 05:16:25PM +0300, Dmitry Osipenko wrote: > > > > 01.07.2021 21:14, Thierry Reding =D0=BF=D0=B8=D1=88=D0=B5=D1=82: > > > > > On Tue, Jun 08, 2021 at 06:51:40PM +0200, Thierry Reding wrote: > > > > >> On Fri, May 28, 2021 at 06:54:55PM +0200, Thierry Reding wrote: > > > > >>> On Thu, May 20, 2021 at 05:03:06PM -0500, Rob Herring wrote: > > > > >>>> On Fri, Apr 23, 2021 at 06:32:30PM +0200, Thierry Reding wrote= : > > > > >>>>> From: Thierry Reding > > > > >>>>> > > > > >>>>> Reserved memory region phandle references can be accompanied = by a > > > > >>>>> specifier that provides additional information about how that= specific > > > > >>>>> reference should be treated. > > > > >>>>> > > > > >>>>> One use-case is to mark a memory region as needing an identit= y mapping > > > > >>>>> in the system's IOMMU for the device that references the regi= on. This is > > > > >>>>> needed for example when the bootloader has set up hardware (s= uch as a > > > > >>>>> display controller) to actively access a memory region (e.g. = a boot > > > > >>>>> splash screen framebuffer) during boot. The operating system = can use the > > > > >>>>> identity mapping flag from the specifier to make sure an IOMM= U identity > > > > >>>>> mapping is set up for the framebuffer before IOMMU translatio= ns are > > > > >>>>> enabled for the display controller. > > > > >>>>> > > > > >>>>> Signed-off-by: Thierry Reding > > > > >>>>> --- > > > > >>>>> .../reserved-memory/reserved-memory.txt | 21 +++++++++= ++++++++++ > > > > >>>>> include/dt-bindings/reserved-memory.h | 8 +++++++ > > > > >>>>> 2 files changed, 29 insertions(+) > > > > >>>>> create mode 100644 include/dt-bindings/reserved-memory.h > > > > >>>> > > > > >>>> Sorry for being slow on this. I have 2 concerns. > > > > >>>> > > > > >>>> First, this creates an ABI issue. A DT with cells in 'memory-r= egion' > > > > >>>> will not be understood by an existing OS. I'm less concerned a= bout this > > > > >>>> if we address that with a stable fix. (Though I'm pretty sure = we've > > > > >>>> naively added #?-cells in the past ignoring this issue.) > > > > >>> > > > > >>> A while ago I had proposed adding memory-region*s* as an altern= ative > > > > >>> name for memory-region to make the naming more consistent with = other > > > > >>> types of properties (think clocks, resets, gpios, ...). If we a= dded > > > > >>> that, we could easily differentiate between the "legacy" cases = where > > > > >>> no #memory-region-cells was allowed and the new cases where it = was. > > > > >>> > > > > >>>> Second, it could be the bootloader setting up the reserved reg= ion. If a > > > > >>>> node already has 'memory-region', then adding more regions is = more > > > > >>>> complicated compared to adding new properties. And defining wh= at each > > > > >>>> memory-region entry is or how many in schemas is impossible. > > > > >>> > > > > >>> It's true that updating the property gets a bit complicated, bu= t it's > > > > >>> not exactly rocket science. We really just need to splice the a= rray. I > > > > >>> have a working implemention for this in U-Boot. > > > > >>> > > > > >>> For what it's worth, we could run into the same issue with any = new > > > > >>> property that we add. Even if we renamed this to iommu-memory-r= egion, > > > > >>> it's still possible that a bootloader may have to update this p= roperty > > > > >>> if it already exists (it could be hard-coded in DT, or it could= have > > > > >>> been added by some earlier bootloader or firmware). > > > > >>> > > > > >>>> Both could be addressed with a new property. Perhaps something= like > > > > >>>> 'iommu-memory-region =3D <&phandle>;'. I think the 'iommu' pre= fix is > > > > >>>> appropriate given this is entirely because of the IOMMU being = in the > > > > >>>> mix. I might feel differently if we had other uses for cells, = but I > > > > >>>> don't really see it in this case. > > > > >>> > > > > >>> I'm afraid that down the road we'll end up with other cases and= then we > > > > >>> might proliferate a number of *-memory-region properties with v= arying > > > > >>> prefixes. > > > > >>> > > > > >>> I am aware of one other case where we might need something like= this: on > > > > >>> some Tegra SoCs we have audio processors that will access memor= y buffers > > > > >>> using a DMA engine. These processors are booted from early firm= ware > > > > >>> using firmware from system memory. In order to avoid trashing t= he > > > > >>> firmware, we need to reserve memory. We can do this using reser= ved > > > > >>> memory nodes. However, the audio DMA engine also uses the SMMU,= so we > > > > >>> need to make sure that the firmware memory is marked as reserve= d within > > > > >>> the SMMU. This is similar to the identity mapping case, but not= exactly > > > > >>> the same. Instead of creating a 1:1 mapping, we just want that = IOVA > > > > >>> region to be reserved (i.e. IOMMU_RESV_RESERVED instead of > > > > >>> IOMMU_RESV_DIRECT{,_RELAXABLE}). > > > > >>> > > > > >>> That would also fall into the IOMMU domain, but we can't reuse = the > > > > >>> iommu-memory-region property for that because then we don't hav= e enough > > > > >>> information to decide which type of reservation we need. > > > > >>> > > > > >>> We could obviously make iommu-memory-region take a specifier, b= ut we > > > > >>> could just as well use memory-regions in that case since we hav= e > > > > >>> something more generic anyway. > > > > >>> > > > > >>> With the #memory-region-cells proposal, we can easily extend th= e cell in > > > > >>> the specifier with an additional MEMORY_REGION_IOMMU_RESERVE fl= ag to > > > > >>> take that other use case into account. If we than also change t= o the new > > > > >>> memory-regions property name, we avoid the ABI issue (and we ga= in a bit > > > > >>> of consistency while at it). > > > > >> > > > > >> Ping? Rob, do you want me to add this second use-case to the pat= ch > > > > >> series to make it more obvious that this isn't just a one-off th= ing? Or > > > > >> how do we proceed? > > > > > > > > > > Rob, given that additional use-case, do you want me to run with t= his > > > > > proposal and send out an updated series? > > > > > > > > > > > > What about variant with a "descriptor" properties that will describ= e > > > > each region: > > > > > > > > fb_desc: display-framebuffer-memory-descriptor { > > > > needs-identity-mapping; > > > > } > > > > > > > > display@52400000 { > > > > memory-region =3D <&fb ...>; > > > > memory-region-descriptor =3D <&fb_desc ...>; > > > > }; > > > > > > > > It could be a more flexible/extendible variant. > > > > > > This problem recently came up on #dri-devel again. Adding Alyssa and > > > Sven who are facing a similar challenge on their work on Apple M1 (if= I > > > understood correctly). Also adding dri-devel for visibility since thi= s > > > is a very common problem for display in particular. > > > > > > On M1 the situation is slightly more complicated: the firmware will > > > allocate a couple of buffers (including the framebuffer) in high memo= ry > > > (> 4 GiB) and use the IOMMU to map that into an IOVA region below 4 G= iB > > > so that the display hardware can access it. This makes it impossible = to > > > bypass the IOMMU like we do on other chips (in particular to work aro= und > > > the fault-by-default policy of the ARM SMMU driver). It also means th= at > > > in addition to the simple reserved regions I mentioned we need for au= dio > > > use-cases and identity mapping use-cases we need for display on Tegra= , > > > we now also need to be able to convey physical to IOVA mappings. > > > > > > Fitting the latter into the original proposal sounds difficult. A qui= ck > > > fix would've been to generate a mapping table in memory and pass that= to > > > the kernel using a reserved-memory node (similar to what's done for > > > example on Tegra for the EMC frequency table on Tegra210) and mark it= as > > > such using a special flag. But that then involves two layers of parsi= ng, > > > which seems a bit suboptimal. Another way to shoehorn that into the > > > original proposal would've been to add flags for physical and virtual > > > address regions and use pairs to pass them using special flags. Again= , > > > this is a bit wonky because it needs these to be carefully parsed and > > > matched up. > > > > > > Another downside is that we now have a situation where some of these > > > regions are no longer "reserved-memory regions" in the traditional > > > sense. This would require an additional flag in the reserved-memory > > > region nodes to prevent the IOVA regions from being reserved. By the > > > way, this is something that would also be needed for the audio use-ca= se > > > I mentioned before, because the physical memory at that address can > > > still be used by an operating system. > > > > > > A more general solution would be to draw a bit from Dmitry's proposal > > > and introduce a new top-level "iov-reserved-memory" node. This could = be > > > modelled on the existing reserved-memory node, except that the physic= al > > > memory pages for regions represented by child nodes would not be mark= ed > > > as reserved. Only the IOVA range described by the region would be > > > reserved subsequently by the IOMMU framework and/or IOMMU driver. > > > > > > The simplest case where we just want to reserve some IOVA region coul= d > > > then be done like this: > > > > > > iov-reserved-memory { > > > /* > > > * Probably safest to default to <2>, <2> here given > > > * that most IOMMUs support either > 32 bits of IAS > > > * or OAS. > > > */ > > > #address-cells =3D <2>; > > > #size-cells =3D <2>; > > > > > > firmware: firmware@80000000 { > > > reg =3D <0 0x80000000 0 0x01000000>; > > > }; > > > }; > > > > > > audio@30000000 { > > > ... > > > iov-memory-regions =3D <&firmware>; > > > ... > > > }; > > > > > > Mappings could be represented by an IOV reserved region taking a > > > reference to the reserved-region that they map: > > > > > > reserved-memory { > > > #address-cells =3D <2>; > > > #size-cells =3D <2>; > > > > > > /* 16 MiB of framebuffer at top-of-memory */ > > > framebuffer: framebuffer@1,ff000000 { > > > reg =3D <0x1 0xff000000 0 0x01000000>; > > > no-map; > > > }; > > > }; > > > > > > iov-reserved-memory { > > > /* IOMMU supports only 32-bit output address space */ > > > #address-cells =3D <1>; > > > #size-cells =3D <1>; > > > > > > /* 16 MiB of framebuffer mapped to top of IOVA */ > > > fb: fb@ff000000 { > > > reg =3D <0 0xff000000 0 0x01000000>; > > > memory-region =3D <&framebuffer>; > > > }; > > > }; > > > > > > display@40000000 { > > > ... > > > /* optional? */ > > > memory-region =3D <&framebuffer>; > > > iov-memory-regions =3D <&fb>; > > > ... > > > }; > > > > > > It's interesting how identity mapped regions now become a trivial > > > special case of mappings. All that is needed is to make the reg prope= rty > > > of the IOV reserved region correspond to the reg property of the norm= al > > > reserved region. Alternatively, as a small optimization for lazy peop= le > > > like me, we could just allow these cases to omit the reg property and > > > instead inherit it from the referenced reserved region. > > > > > > As the second example shows it might be convenient if memory-region > > > could be derived from iov-memory-regions. This could be useful for ca= ses > > > where the driver wants to do something with the physical pages of the > > > reserved region (such as mapping them and copying out the framebuffer > > > data to another buffer so that the reserved memory can be recycled). = If > > > we have the IOV reserved region, we could provide an API to extract t= he > > > physical reserved region (if it exists). That way we could avoid > > > referencing it twice in DT. Then again, there's something elegant abo= ut > > > the explicit second reference to. It indicates the intent that we may > > > want to use the region for something other than just the IOV mapping. > > > > > > Anyway, this has been long enough. Let me know what you think. Alyssa= , > > > Sven, it'd be interesting to hear if you think this could work as a > > > solution to the problem on M1. > > > > > > Rob, I think you might like this alternative because it basically get= s > > > rid of all the points in the original proposal that you were concerne= d > > > about. Let me know what you think. > > > > Couldn't we keep this all in /reserved-memory? Just add an iova > > version of reg. Perhaps abuse 'assigned-address' for this purpose. The > > issue I see would be handling reserved iova areas without a physical > > area. That can be handled with just a iova and no reg. We already have > > a no reg case. > > I had thought about that initially. One thing I'm worried about is that > every child node in /reserved-memory will effectively cause the memory > that it described to be reserved. But we don't want that for regions > that are "virtual only" (i.e. IOMMU reservations). By virtual only, you mean no physical mapping, just a region of virtual space, right? For that we'd have no 'reg' and therefore no (physical) reservation by the OS. It's similar to non-static regions. You need a specific handler for them. We'd probably want a compatible as well for these virtual reservations. Are these being global in DT going to be a problem? Presumably we have a virtual space per IOMMU. We'd know which IOMMU based on a device's 'iommus' and 'memory-region' properties, but within /reserved-memory we wouldn't be able to distinguish overlapping addresses from separate address spaces. Or we could have 2 different IOVAs for 1 physical space. That could be solved with something like this: iommu-addresses =3D <&iommu1
>; Or the other way to do this is reuse 'iommus' property to define the mapping of each address entry to iommu. > Obviously we can fix that in Linux, but what about other operating > systems? Currently "reg" is a required property for statically allocated > regions (which all of these would be). Do you have an idea of how widely > that's used? What about other OSes, or bootloaders, what if they > encounter these nodes that don't have a "reg" property? Without 'reg', there must be a compatible that the client understands or the node should be ignored. My suspicion is that /reserved-memory is abused for all sorts of things downstream, but that's not really relevant here. Rob