From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 49889C433E9 for ; Fri, 12 Mar 2021 19:48:13 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 1338064F8D for ; Fri, 12 Mar 2021 19:48:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S234486AbhCLTrn (ORCPT ); Fri, 12 Mar 2021 14:47:43 -0500 Received: from foss.arm.com ([217.140.110.172]:59856 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234524AbhCLTrS (ORCPT ); Fri, 12 Mar 2021 14:47:18 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 20D92ED1; Fri, 12 Mar 2021 11:47:18 -0800 (PST) Received: from [10.57.52.136] (unknown [10.57.52.136]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CA35E3F793; Fri, 12 Mar 2021 11:47:13 -0800 (PST) Subject: Re: [RFC PATCH v2 08/11] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg To: Logan Gunthorpe , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Minturn Dave B , John Hubbard , Dave Hansen , Ira Weiny , Matthew Wilcox , =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Jason Ekstrand , Daniel Vetter , Dan Williams , Stephen Bates , Jakowski Andrzej , Christoph Hellwig , Xiong Jianxin References: <20210311233142.7900-1-logang@deltatee.com> <20210311233142.7900-9-logang@deltatee.com> <45701356-ee41-1ad2-0e06-ca74af87b05a@deltatee.com> From: Robin Murphy Message-ID: <76cc1c82-3cf4-92d3-992f-5c876ed30523@arm.com> Date: Fri, 12 Mar 2021 19:47:08 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <45701356-ee41-1ad2-0e06-ca74af87b05a@deltatee.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 2021-03-12 17:03, Logan Gunthorpe wrote: > > > On 2021-03-12 8:52 a.m., Robin Murphy wrote: >> On 2021-03-11 23:31, Logan Gunthorpe wrote: >>> When a PCI P2PDMA page is seen, set the IOVA length of the segment >>> to zero so that it is not mapped into the IOVA. Then, in finalise_sg(), >>> apply the appropriate bus address to the segment. The IOVA is not >>> created if the scatterlist only consists of P2PDMA pages. >> >> This misled me at first, but I see the implementation does actually >> appear to accomodate the case of working ACS where P2P *would* still >> need to be mapped at the IOMMU. > > Yes, that's correct. >>>   static int __finalise_sg(struct device *dev, struct scatterlist *sg, >>> int nents, >>> -        dma_addr_t dma_addr) >>> +        dma_addr_t dma_addr, unsigned long attrs) >>>   { >>>       struct scatterlist *s, *cur = sg; >>>       unsigned long seg_mask = dma_get_seg_boundary(dev); >>> @@ -864,6 +865,20 @@ static int __finalise_sg(struct device *dev, >>> struct scatterlist *sg, int nents, >>>           sg_dma_address(s) = DMA_MAPPING_ERROR; >>>           sg_dma_len(s) = 0; >>>   +        if (is_pci_p2pdma_page(sg_page(s)) && !s_iova_len) { >>> +            if (i > 0) >>> +                cur = sg_next(cur); >>> + >>> +            sg_dma_address(cur) = sg_phys(s) + s->offset - >> >> Are you sure about that? ;) > > Do you see a bug? I don't follow you... sg_phys() already accounts for the offset, so you're adding it twice. >>> +                pci_p2pdma_bus_offset(sg_page(s)); >> >> Can the bus offset make P2P addresses overlap with regions of mem space >> that we might use for regular IOVA allocation? That would be very bad... > > No. IOMMU drivers already disallow all PCI addresses from being used as > IOVA addresses. See, for example, dmar_init_reserved_ranges(). It would > be a huge problem for a whole lot of other reasons if it didn't. I know we reserve the outbound windows (largely *because* some host bridges will consider those addresses as attempts at unsupported P2P and prevent them working), I just wanted to confirm that this bus offset is always something small that stays within the relevant window, rather than something that might make a BAR appear in a completely different place for P2P purposes. If so, that's good. >>> @@ -960,11 +975,12 @@ static int iommu_dma_map_sg(struct device *dev, >>> struct scatterlist *sg, >>>       struct iommu_dma_cookie *cookie = domain->iova_cookie; >>>       struct iova_domain *iovad = &cookie->iovad; >>>       struct scatterlist *s, *prev = NULL; >>> +    struct dev_pagemap *pgmap = NULL; >>>       int prot = dma_info_to_prot(dir, dev_is_dma_coherent(dev), attrs); >>>       dma_addr_t iova; >>>       size_t iova_len = 0; >>>       unsigned long mask = dma_get_seg_boundary(dev); >>> -    int i; >>> +    int i, map = -1, ret = 0; >>>         if (static_branch_unlikely(&iommu_deferred_attach_enabled) && >>>           iommu_deferred_attach(dev, domain)) >>> @@ -993,6 +1009,23 @@ static int iommu_dma_map_sg(struct device *dev, >>> struct scatterlist *sg, >>>           s_length = iova_align(iovad, s_length + s_iova_off); >>>           s->length = s_length; >>>   +        if (is_pci_p2pdma_page(sg_page(s))) { >>> +            if (sg_page(s)->pgmap != pgmap) { >>> +                pgmap = sg_page(s)->pgmap; >>> +                map = pci_p2pdma_dma_map_type(dev, pgmap); >>> +            } >>> + >>> +            if (map < 0) { >> >> It rather feels like it should be the job of whoever creates the list in >> the first place not to put unusable pages in it, especially since the >> p2pdma_map_type looks to be a fairly coarse-grained and static thing. >> The DMA API isn't responsible for validating normal memory pages, so >> what makes P2P special? > > Yes, that would be ideal, but there's some difficulties there. For the > driver to check the pages, it would need to loop through the entire SG > one more time on every transaction, regardless of whether there are > P2PDMA pages, or not. So that will have a performance impact even when > the feature isn't being used. I don't think that'll be acceptable for > many drivers. > > The other possibility is for GUP to do it when it gets the pages from > userspace. But GUP doesn't have all the information to do this at the > moment. We'd have to pass the struct device that will eventually map the > pages through all the nested functions in the GUP to do that test at > that time. This might not be a bad option (that I half looked into), but > I'm not sure how acceptable it would be to the GUP developers. Urgh, yes, if a page may or may not be valid for p2p depending on which device is trying to map it, then it probably is most reasonable to figure that out at this point. It's a little unfortunate having to cope with failure so late, but oh well. > But even if we do verify the pages ahead of time, we still need the same > infrastructure in dma_map_sg(); it could only now be a BUG if the driver > sent invalid pages instead of an error return. The hope was that we could save doing even that - e.g. if you pass a dodgy page into dma_map_page(), maybe page_to_phys() will crash, maybe you'll just end up with a DMA address that won't work, but either way it doesn't care in its own right - but it seems that's moot. >>> +                ret = -EREMOTEIO; >>> +                goto out_restore_sg; >>> +            } >>> + >>> +            if (map) { >>> +                s->length = 0; >> >> I'm not really thrilled about the idea of passing zero-length segments >> to iommu_map_sg(). Yes, it happens to trick the concatenation logic in >> the current implementation into doing what you want, but it feels fragile. > > We're not passing zero length segments to iommu_map_sg() (or any > function). This loop is just scanning to calculate the length of the > required IOVA. __finalise_sg() (which is intimately tied to this loop) > then needs a way to determine which segments were P2P segments. The > existing code already overwrites s->length with an aligned length and > stores the original length in sg_dma_len. So we're not relying on > tricking any logic here. Yes, we temporarily shuffle in page-aligned quantities to satisfy the needs of the iommu_map_sg() call, before unpacking things again in __finalise_sg(). It's some disgusting trickery that I'm particularly proud of. My point is that if you have a mix of both p2p and normal segments - which seems to be a case you want to support - then the length of 0 that you set to flag p2p segments here will be seen by iommu_map_sg() (as it walks the list to map the other segments) before you then use it as a key to override the DMA address in the final step. It's not a concern if you have a p2p-only list and short-circuit straight to that step (in which case all the shuffling was wasted effort anyway), but since it's not entirely clear what a segment with zero length would mean in general, it seems like a good idea to avoid passing the list across a public boundary in that state, if possible. Robin. >>>   } >>>     static void iommu_dma_unmap_sg(struct device *dev, struct >>> scatterlist *sg, >>>           int nents, enum dma_data_direction dir, unsigned long attrs) >>>   { >>> -    dma_addr_t start, end; >>> +    dma_addr_t end, start = DMA_MAPPING_ERROR; >>>       struct scatterlist *tmp; >>>       int i; >>>   @@ -1054,14 +1090,20 @@ static void iommu_dma_unmap_sg(struct device >>> *dev, struct scatterlist *sg, >>>        * The scatterlist segments are mapped into a single >>>        * contiguous IOVA allocation, so this is incredibly easy. >>>        */ >>> -    start = sg_dma_address(sg); >>> -    for_each_sg(sg_next(sg), tmp, nents - 1, i) { >>> +    for_each_sg(sg, tmp, nents, i) { >>> +        if (sg_is_pci_p2pdma(tmp)) >> >> Since the flag is associated with the DMA address which will no longer >> be valid, shouldn't it be cleared? The circumstances in which leaving it >> around could cause a problem are tenuous, but definitely possible. > > Yes, that's a good idea. > > Thanks for the review! > > Logan > From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F4207C43333 for ; Fri, 12 Mar 2021 19:47:42 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3BE1364F84 for ; Fri, 12 Mar 2021 19:47:42 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3BE1364F84 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:Date:Message-ID:From: References:Cc:To:Subject:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=uP3Z+eWcIQ9LZ1ZeQxqf6zb7TiP2QL4Art2ehpSYkI4=; b=WrzU+sGe4/VwreymvOKn+oAf1 LLJZH2fsgwtRloTQ8TRqkJzPsPYvSMN7aI67pis1wuTMal5rsl3F7VlDE660vwHOGh1T9fGAkG0uP mz8hFZjL9X8wZ+YlNNxDJSL+76iZCSqON6X5ZGUdFXeIG5p3CO0qzsH+ohazV9tfyvwTA4kfTzABm y8x7nXF6JHY5/EW1slfg5BkT316/8RiDQyfrQgBCvv4+pZ+zmKU43F8MrQnBxIA62Tp9ZssVHmR/g dTShS5qwaptCLkbUoBCoKB0Sn+uyFXC+cZbR2nADp2zgNSTgYSUEeRpDk0NqMEzYCGkX9HMZGOhnm QEmoD4Cfw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKnkx-00CITx-Rr; Fri, 12 Mar 2021 19:47:31 +0000 Received: from foss.arm.com ([217.140.110.172]) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKnks-00CISs-7H for linux-nvme@lists.infradead.org; Fri, 12 Mar 2021 19:47:30 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 20D92ED1; Fri, 12 Mar 2021 11:47:18 -0800 (PST) Received: from [10.57.52.136] (unknown [10.57.52.136]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CA35E3F793; Fri, 12 Mar 2021 11:47:13 -0800 (PST) Subject: Re: [RFC PATCH v2 08/11] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg To: Logan Gunthorpe , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Minturn Dave B , John Hubbard , Dave Hansen , Ira Weiny , Matthew Wilcox , =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Jason Ekstrand , Daniel Vetter , Dan Williams , Stephen Bates , Jakowski Andrzej , Christoph Hellwig , Xiong Jianxin References: <20210311233142.7900-1-logang@deltatee.com> <20210311233142.7900-9-logang@deltatee.com> <45701356-ee41-1ad2-0e06-ca74af87b05a@deltatee.com> From: Robin Murphy Message-ID: <76cc1c82-3cf4-92d3-992f-5c876ed30523@arm.com> Date: Fri, 12 Mar 2021 19:47:08 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <45701356-ee41-1ad2-0e06-ca74af87b05a@deltatee.com> Content-Language: en-GB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_194728_280315_0FC31BCB X-CRM114-Status: GOOD ( 53.88 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org T24gMjAyMS0wMy0xMiAxNzowMywgTG9nYW4gR3VudGhvcnBlIHdyb3RlOgo+IAo+IAo+IE9uIDIw MjEtMDMtMTIgODo1MiBhLm0uLCBSb2JpbiBNdXJwaHkgd3JvdGU6Cj4+IE9uIDIwMjEtMDMtMTEg MjM6MzEsIExvZ2FuIEd1bnRob3JwZSB3cm90ZToKPj4+IFdoZW4gYSBQQ0kgUDJQRE1BIHBhZ2Ug aXMgc2Vlbiwgc2V0IHRoZSBJT1ZBIGxlbmd0aCBvZiB0aGUgc2VnbWVudAo+Pj4gdG8gemVybyBz byB0aGF0IGl0IGlzIG5vdCBtYXBwZWQgaW50byB0aGUgSU9WQS4gVGhlbiwgaW4gZmluYWxpc2Vf c2coKSwKPj4+IGFwcGx5IHRoZSBhcHByb3ByaWF0ZSBidXMgYWRkcmVzcyB0byB0aGUgc2VnbWVu dC4gVGhlIElPVkEgaXMgbm90Cj4+PiBjcmVhdGVkIGlmIHRoZSBzY2F0dGVybGlzdCBvbmx5IGNv bnNpc3RzIG9mIFAyUERNQSBwYWdlcy4KPj4KPj4gVGhpcyBtaXNsZWQgbWUgYXQgZmlyc3QsIGJ1 dCBJIHNlZSB0aGUgaW1wbGVtZW50YXRpb24gZG9lcyBhY3R1YWxseQo+PiBhcHBlYXIgdG8gYWNj b21vZGF0ZSB0aGUgY2FzZSBvZiB3b3JraW5nIEFDUyB3aGVyZSBQMlAgKndvdWxkKiBzdGlsbAo+ PiBuZWVkIHRvIGJlIG1hcHBlZCBhdCB0aGUgSU9NTVUuCj4gCj4gWWVzLCB0aGF0J3MgY29ycmVj dC4KPj4+ICDCoCBzdGF0aWMgaW50IF9fZmluYWxpc2Vfc2coc3RydWN0IGRldmljZSAqZGV2LCBz dHJ1Y3Qgc2NhdHRlcmxpc3QgKnNnLAo+Pj4gaW50IG5lbnRzLAo+Pj4gLcKgwqDCoMKgwqDCoMKg IGRtYV9hZGRyX3QgZG1hX2FkZHIpCj4+PiArwqDCoMKgwqDCoMKgwqAgZG1hX2FkZHJfdCBkbWFf YWRkciwgdW5zaWduZWQgbG9uZyBhdHRycykKPj4+ICDCoCB7Cj4+PiAgwqDCoMKgwqDCoCBzdHJ1 Y3Qgc2NhdHRlcmxpc3QgKnMsICpjdXIgPSBzZzsKPj4+ICDCoMKgwqDCoMKgIHVuc2lnbmVkIGxv bmcgc2VnX21hc2sgPSBkbWFfZ2V0X3NlZ19ib3VuZGFyeShkZXYpOwo+Pj4gQEAgLTg2NCw2ICs4 NjUsMjAgQEAgc3RhdGljIGludCBfX2ZpbmFsaXNlX3NnKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4+ IHN0cnVjdCBzY2F0dGVybGlzdCAqc2csIGludCBuZW50cywKPj4+ICDCoMKgwqDCoMKgwqDCoMKg wqAgc2dfZG1hX2FkZHJlc3MocykgPSBETUFfTUFQUElOR19FUlJPUjsKPj4+ICDCoMKgwqDCoMKg wqDCoMKgwqAgc2dfZG1hX2xlbihzKSA9IDA7Cj4+PiAgwqAgK8KgwqDCoMKgwqDCoMKgIGlmIChp c19wY2lfcDJwZG1hX3BhZ2Uoc2dfcGFnZShzKSkgJiYgIXNfaW92YV9sZW4pIHsKPj4+ICvCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIGlmIChpID4gMCkKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgY3VyID0gc2dfbmV4dChjdXIpOwo+Pj4gKwo+Pj4gK8KgwqDCoMKgwqDCoMKgwqDC oMKgwqAgc2dfZG1hX2FkZHJlc3MoY3VyKSA9IHNnX3BoeXMocykgKyBzLT5vZmZzZXQgLQo+Pgo+ PiBBcmUgeW91IHN1cmUgYWJvdXQgdGhhdD8gOykKPiAKPiBEbyB5b3Ugc2VlIGEgYnVnPyBJIGRv bid0IGZvbGxvdyB5b3UuLi4KCnNnX3BoeXMoKSBhbHJlYWR5IGFjY291bnRzIGZvciB0aGUgb2Zm c2V0LCBzbyB5b3UncmUgYWRkaW5nIGl0IHR3aWNlLgoKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgcGNpX3AycGRtYV9idXNfb2Zmc2V0KHNnX3BhZ2UocykpOwo+Pgo+PiBDYW4g dGhlIGJ1cyBvZmZzZXQgbWFrZSBQMlAgYWRkcmVzc2VzIG92ZXJsYXAgd2l0aCByZWdpb25zIG9m IG1lbSBzcGFjZQo+PiB0aGF0IHdlIG1pZ2h0IHVzZSBmb3IgcmVndWxhciBJT1ZBIGFsbG9jYXRp b24/IFRoYXQgd291bGQgYmUgdmVyeSBiYWQuLi4KPiAKPiBOby4gSU9NTVUgZHJpdmVycyBhbHJl YWR5IGRpc2FsbG93IGFsbCBQQ0kgYWRkcmVzc2VzIGZyb20gYmVpbmcgdXNlZCBhcwo+IElPVkEg YWRkcmVzc2VzLiBTZWUsIGZvciBleGFtcGxlLCAgZG1hcl9pbml0X3Jlc2VydmVkX3Jhbmdlcygp LiBJdCB3b3VsZAo+IGJlIGEgaHVnZSBwcm9ibGVtIGZvciBhIHdob2xlIGxvdCBvZiBvdGhlciBy ZWFzb25zIGlmIGl0IGRpZG4ndC4KCkkga25vdyB3ZSByZXNlcnZlIHRoZSBvdXRib3VuZCB3aW5k b3dzIChsYXJnZWx5ICpiZWNhdXNlKiBzb21lIGhvc3QgCmJyaWRnZXMgd2lsbCBjb25zaWRlciB0 aG9zZSBhZGRyZXNzZXMgYXMgYXR0ZW1wdHMgYXQgdW5zdXBwb3J0ZWQgUDJQIGFuZCAKcHJldmVu dCB0aGVtIHdvcmtpbmcpLCBJIGp1c3Qgd2FudGVkIHRvIGNvbmZpcm0gdGhhdCB0aGlzIGJ1cyBv ZmZzZXQgaXMgCmFsd2F5cyBzb21ldGhpbmcgc21hbGwgdGhhdCBzdGF5cyB3aXRoaW4gdGhlIHJl bGV2YW50IHdpbmRvdywgcmF0aGVyIAp0aGFuIHNvbWV0aGluZyB0aGF0IG1pZ2h0IG1ha2UgYSBC QVIgYXBwZWFyIGluIGEgY29tcGxldGVseSBkaWZmZXJlbnQgCnBsYWNlIGZvciBQMlAgcHVycG9z ZXMuIElmIHNvLCB0aGF0J3MgZ29vZC4KCj4+PiBAQCAtOTYwLDExICs5NzUsMTIgQEAgc3RhdGlj IGludCBpb21tdV9kbWFfbWFwX3NnKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4+IHN0cnVjdCBzY2F0 dGVybGlzdCAqc2csCj4+PiAgwqDCoMKgwqDCoCBzdHJ1Y3QgaW9tbXVfZG1hX2Nvb2tpZSAqY29v a2llID0gZG9tYWluLT5pb3ZhX2Nvb2tpZTsKPj4+ICDCoMKgwqDCoMKgIHN0cnVjdCBpb3ZhX2Rv bWFpbiAqaW92YWQgPSAmY29va2llLT5pb3ZhZDsKPj4+ICDCoMKgwqDCoMKgIHN0cnVjdCBzY2F0 dGVybGlzdCAqcywgKnByZXYgPSBOVUxMOwo+Pj4gK8KgwqDCoCBzdHJ1Y3QgZGV2X3BhZ2VtYXAg KnBnbWFwID0gTlVMTDsKPj4+ICDCoMKgwqDCoMKgIGludCBwcm90ID0gZG1hX2luZm9fdG9fcHJv dChkaXIsIGRldl9pc19kbWFfY29oZXJlbnQoZGV2KSwgYXR0cnMpOwo+Pj4gIMKgwqDCoMKgwqAg ZG1hX2FkZHJfdCBpb3ZhOwo+Pj4gIMKgwqDCoMKgwqAgc2l6ZV90IGlvdmFfbGVuID0gMDsKPj4+ ICDCoMKgwqDCoMKgIHVuc2lnbmVkIGxvbmcgbWFzayA9IGRtYV9nZXRfc2VnX2JvdW5kYXJ5KGRl dik7Cj4+PiAtwqDCoMKgIGludCBpOwo+Pj4gK8KgwqDCoCBpbnQgaSwgbWFwID0gLTEsIHJldCA9 IDA7Cj4+PiAgwqAgwqDCoMKgwqDCoCBpZiAoc3RhdGljX2JyYW5jaF91bmxpa2VseSgmaW9tbXVf ZGVmZXJyZWRfYXR0YWNoX2VuYWJsZWQpICYmCj4+PiAgwqDCoMKgwqDCoMKgwqDCoMKgIGlvbW11 X2RlZmVycmVkX2F0dGFjaChkZXYsIGRvbWFpbikpCj4+PiBAQCAtOTkzLDYgKzEwMDksMjMgQEAg c3RhdGljIGludCBpb21tdV9kbWFfbWFwX3NnKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4+IHN0cnVj dCBzY2F0dGVybGlzdCAqc2csCj4+PiAgwqDCoMKgwqDCoMKgwqDCoMKgIHNfbGVuZ3RoID0gaW92 YV9hbGlnbihpb3ZhZCwgc19sZW5ndGggKyBzX2lvdmFfb2ZmKTsKPj4+ICDCoMKgwqDCoMKgwqDC oMKgwqAgcy0+bGVuZ3RoID0gc19sZW5ndGg7Cj4+PiAgwqAgK8KgwqDCoMKgwqDCoMKgIGlmIChp c19wY2lfcDJwZG1hX3BhZ2Uoc2dfcGFnZShzKSkpIHsKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIGlmIChzZ19wYWdlKHMpLT5wZ21hcCAhPSBwZ21hcCkgewo+Pj4gK8KgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCBwZ21hcCA9IHNnX3BhZ2UocyktPnBnbWFwOwo+Pj4gK8KgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBtYXAgPSBwY2lfcDJwZG1hX2RtYV9tYXBfdHlwZShkZXYs IHBnbWFwKTsKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIH0KPj4+ICsKPj4+ICvCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIGlmIChtYXAgPCAwKSB7Cj4+Cj4+IEl0IHJhdGhlciBmZWVscyBsaWtl IGl0IHNob3VsZCBiZSB0aGUgam9iIG9mIHdob2V2ZXIgY3JlYXRlcyB0aGUgbGlzdCBpbgo+PiB0 aGUgZmlyc3QgcGxhY2Ugbm90IHRvIHB1dCB1bnVzYWJsZSBwYWdlcyBpbiBpdCwgZXNwZWNpYWxs eSBzaW5jZSB0aGUKPj4gcDJwZG1hX21hcF90eXBlIGxvb2tzIHRvIGJlIGEgZmFpcmx5IGNvYXJz ZS1ncmFpbmVkIGFuZCBzdGF0aWMgdGhpbmcuCj4+IFRoZSBETUEgQVBJIGlzbid0IHJlc3BvbnNp YmxlIGZvciB2YWxpZGF0aW5nIG5vcm1hbCBtZW1vcnkgcGFnZXMsIHNvCj4+IHdoYXQgbWFrZXMg UDJQIHNwZWNpYWw/Cj4gCj4gWWVzLCB0aGF0IHdvdWxkIGJlIGlkZWFsLCBidXQgdGhlcmUncyBz b21lIGRpZmZpY3VsdGllcyB0aGVyZS4gRm9yIHRoZQo+IGRyaXZlciB0byBjaGVjayB0aGUgcGFn ZXMsIGl0IHdvdWxkIG5lZWQgdG8gbG9vcCB0aHJvdWdoIHRoZSBlbnRpcmUgU0cKPiBvbmUgbW9y ZSB0aW1lIG9uIGV2ZXJ5IHRyYW5zYWN0aW9uLCByZWdhcmRsZXNzIG9mIHdoZXRoZXIgdGhlcmUg YXJlCj4gUDJQRE1BIHBhZ2VzLCBvciBub3QuIFNvIHRoYXQgd2lsbCBoYXZlIGEgcGVyZm9ybWFu Y2UgaW1wYWN0IGV2ZW4gd2hlbgo+IHRoZSBmZWF0dXJlIGlzbid0IGJlaW5nIHVzZWQuIEkgZG9u J3QgdGhpbmsgdGhhdCdsbCBiZSBhY2NlcHRhYmxlIGZvcgo+IG1hbnkgZHJpdmVycy4KPiAKPiBU aGUgb3RoZXIgcG9zc2liaWxpdHkgaXMgZm9yIEdVUCB0byBkbyBpdCB3aGVuIGl0IGdldHMgdGhl IHBhZ2VzIGZyb20KPiB1c2Vyc3BhY2UuIEJ1dCBHVVAgZG9lc24ndCBoYXZlIGFsbCB0aGUgaW5m b3JtYXRpb24gdG8gZG8gdGhpcyBhdCB0aGUKPiBtb21lbnQuIFdlJ2QgaGF2ZSB0byBwYXNzIHRo ZSBzdHJ1Y3QgZGV2aWNlIHRoYXQgd2lsbCBldmVudHVhbGx5IG1hcCB0aGUKPiBwYWdlcyB0aHJv dWdoIGFsbCB0aGUgbmVzdGVkIGZ1bmN0aW9ucyBpbiB0aGUgR1VQIHRvIGRvIHRoYXQgdGVzdCBh dAo+IHRoYXQgdGltZS4gVGhpcyBtaWdodCBub3QgYmUgYSBiYWQgb3B0aW9uICh0aGF0IEkgaGFs ZiBsb29rZWQgaW50byksIGJ1dAo+IEknbSBub3Qgc3VyZSBob3cgYWNjZXB0YWJsZSBpdCB3b3Vs ZCBiZSB0byB0aGUgR1VQIGRldmVsb3BlcnMuCgpVcmdoLCB5ZXMsIGlmIGEgcGFnZSBtYXkgb3Ig bWF5IG5vdCBiZSB2YWxpZCBmb3IgcDJwIGRlcGVuZGluZyBvbiB3aGljaCAKZGV2aWNlIGlzIHRy eWluZyB0byBtYXAgaXQsIHRoZW4gaXQgcHJvYmFibHkgaXMgbW9zdCByZWFzb25hYmxlIHRvIApm aWd1cmUgdGhhdCBvdXQgYXQgdGhpcyBwb2ludC4gSXQncyBhIGxpdHRsZSB1bmZvcnR1bmF0ZSBo YXZpbmcgdG8gY29wZSAKd2l0aCBmYWlsdXJlIHNvIGxhdGUsIGJ1dCBvaCB3ZWxsLgoKPiBCdXQg ZXZlbiBpZiB3ZSBkbyB2ZXJpZnkgdGhlIHBhZ2VzIGFoZWFkIG9mIHRpbWUsIHdlIHN0aWxsIG5l ZWQgdGhlIHNhbWUKPiBpbmZyYXN0cnVjdHVyZSBpbiBkbWFfbWFwX3NnKCk7IGl0IGNvdWxkIG9u bHkgbm93IGJlIGEgQlVHIGlmIHRoZSBkcml2ZXIKPiBzZW50IGludmFsaWQgcGFnZXMgaW5zdGVh ZCBvZiBhbiBlcnJvciByZXR1cm4uCgpUaGUgaG9wZSB3YXMgdGhhdCB3ZSBjb3VsZCBzYXZlIGRv aW5nIGV2ZW4gdGhhdCAtIGUuZy4gaWYgeW91IHBhc3MgYSAKZG9kZ3kgcGFnZSBpbnRvIGRtYV9t YXBfcGFnZSgpLCBtYXliZSBwYWdlX3RvX3BoeXMoKSB3aWxsIGNyYXNoLCBtYXliZSAKeW91J2xs IGp1c3QgZW5kIHVwIHdpdGggYSBETUEgYWRkcmVzcyB0aGF0IHdvbid0IHdvcmssIGJ1dCBlaXRo ZXIgd2F5IGl0IApkb2Vzbid0IGNhcmUgaW4gaXRzIG93biByaWdodCAtIGJ1dCBpdCBzZWVtcyB0 aGF0J3MgbW9vdC4KCj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldCA9IC1F UkVNT1RFSU87Cj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGdvdG8gb3V0X3Jl c3RvcmVfc2c7Cj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB9Cj4+PiArCj4+PiArwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCBpZiAobWFwKSB7Cj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIHMtPmxlbmd0aCA9IDA7Cj4+Cj4+IEknbSBub3QgcmVhbGx5IHRocmlsbGVkIGFib3V0 IHRoZSBpZGVhIG9mIHBhc3NpbmcgemVyby1sZW5ndGggc2VnbWVudHMKPj4gdG8gaW9tbXVfbWFw X3NnKCkuIFllcywgaXQgaGFwcGVucyB0byB0cmljayB0aGUgY29uY2F0ZW5hdGlvbiBsb2dpYyBp bgo+PiB0aGUgY3VycmVudCBpbXBsZW1lbnRhdGlvbiBpbnRvIGRvaW5nIHdoYXQgeW91IHdhbnQs IGJ1dCBpdCBmZWVscyBmcmFnaWxlLgo+IAo+IFdlJ3JlIG5vdCBwYXNzaW5nIHplcm8gbGVuZ3Ro IHNlZ21lbnRzIHRvIGlvbW11X21hcF9zZygpIChvciBhbnkKPiBmdW5jdGlvbikuIFRoaXMgbG9v cCBpcyBqdXN0IHNjYW5uaW5nIHRvIGNhbGN1bGF0ZSB0aGUgbGVuZ3RoIG9mIHRoZQo+IHJlcXVp cmVkIElPVkEuIF9fZmluYWxpc2Vfc2coKSAod2hpY2ggaXMgaW50aW1hdGVseSB0aWVkIHRvIHRo aXMgbG9vcCkKPiB0aGVuIG5lZWRzIGEgd2F5IHRvIGRldGVybWluZSB3aGljaCBzZWdtZW50cyB3 ZXJlIFAyUCBzZWdtZW50cy4gVGhlCj4gZXhpc3RpbmcgY29kZSBhbHJlYWR5IG92ZXJ3cml0ZXMg cy0+bGVuZ3RoIHdpdGggYW4gYWxpZ25lZCBsZW5ndGggYW5kCj4gc3RvcmVzIHRoZSBvcmlnaW5h bCBsZW5ndGggaW4gc2dfZG1hX2xlbi4gU28gd2UncmUgbm90IHJlbHlpbmcgb24KPiB0cmlja2lu ZyBhbnkgbG9naWMgaGVyZS4KClllcywgd2UgdGVtcG9yYXJpbHkgc2h1ZmZsZSBpbiBwYWdlLWFs aWduZWQgcXVhbnRpdGllcyB0byBzYXRpc2Z5IHRoZSAKbmVlZHMgb2YgdGhlIGlvbW11X21hcF9z ZygpIGNhbGwsIGJlZm9yZSB1bnBhY2tpbmcgdGhpbmdzIGFnYWluIGluIApfX2ZpbmFsaXNlX3Nn KCkuIEl0J3Mgc29tZSBkaXNndXN0aW5nIHRyaWNrZXJ5IHRoYXQgSSdtIHBhcnRpY3VsYXJseSAK cHJvdWQgb2YuIE15IHBvaW50IGlzIHRoYXQgaWYgeW91IGhhdmUgYSBtaXggb2YgYm90aCBwMnAg YW5kIG5vcm1hbCAKc2VnbWVudHMgLSB3aGljaCBzZWVtcyB0byBiZSBhIGNhc2UgeW91IHdhbnQg dG8gc3VwcG9ydCAtIHRoZW4gdGhlIApsZW5ndGggb2YgMCB0aGF0IHlvdSBzZXQgdG8gZmxhZyBw MnAgc2VnbWVudHMgaGVyZSB3aWxsIGJlIHNlZW4gYnkgCmlvbW11X21hcF9zZygpIChhcyBpdCB3 YWxrcyB0aGUgbGlzdCB0byBtYXAgdGhlIG90aGVyIHNlZ21lbnRzKSBiZWZvcmUgCnlvdSB0aGVu IHVzZSBpdCBhcyBhIGtleSB0byBvdmVycmlkZSB0aGUgRE1BIGFkZHJlc3MgaW4gdGhlIGZpbmFs IHN0ZXAuIApJdCdzIG5vdCBhIGNvbmNlcm4gaWYgeW91IGhhdmUgYSBwMnAtb25seSBsaXN0IGFu ZCBzaG9ydC1jaXJjdWl0IApzdHJhaWdodCB0byB0aGF0IHN0ZXAgKGluIHdoaWNoIGNhc2UgYWxs IHRoZSBzaHVmZmxpbmcgd2FzIHdhc3RlZCBlZmZvcnQgCmFueXdheSksIGJ1dCBzaW5jZSBpdCdz IG5vdCBlbnRpcmVseSBjbGVhciB3aGF0IGEgc2VnbWVudCB3aXRoIHplcm8gCmxlbmd0aCB3b3Vs ZCBtZWFuIGluIGdlbmVyYWwsIGl0IHNlZW1zIGxpa2UgYSBnb29kIGlkZWEgdG8gYXZvaWQgcGFz c2luZyAKdGhlIGxpc3QgYWNyb3NzIGEgcHVibGljIGJvdW5kYXJ5IGluIHRoYXQgc3RhdGUsIGlm IHBvc3NpYmxlLgoKUm9iaW4uCgo+Pj4gIMKgIH0KPj4+ICDCoCDCoCBzdGF0aWMgdm9pZCBpb21t dV9kbWFfdW5tYXBfc2coc3RydWN0IGRldmljZSAqZGV2LCBzdHJ1Y3QKPj4+IHNjYXR0ZXJsaXN0 ICpzZywKPj4+ICDCoMKgwqDCoMKgwqDCoMKgwqAgaW50IG5lbnRzLCBlbnVtIGRtYV9kYXRhX2Rp cmVjdGlvbiBkaXIsIHVuc2lnbmVkIGxvbmcgYXR0cnMpCj4+PiAgwqAgewo+Pj4gLcKgwqDCoCBk bWFfYWRkcl90IHN0YXJ0LCBlbmQ7Cj4+PiArwqDCoMKgIGRtYV9hZGRyX3QgZW5kLCBzdGFydCA9 IERNQV9NQVBQSU5HX0VSUk9SOwo+Pj4gIMKgwqDCoMKgwqAgc3RydWN0IHNjYXR0ZXJsaXN0ICp0 bXA7Cj4+PiAgwqDCoMKgwqDCoCBpbnQgaTsKPj4+ICDCoCBAQCAtMTA1NCwxNCArMTA5MCwyMCBA QCBzdGF0aWMgdm9pZCBpb21tdV9kbWFfdW5tYXBfc2coc3RydWN0IGRldmljZQo+Pj4gKmRldiwg c3RydWN0IHNjYXR0ZXJsaXN0ICpzZywKPj4+ICDCoMKgwqDCoMKgwqAgKiBUaGUgc2NhdHRlcmxp c3Qgc2VnbWVudHMgYXJlIG1hcHBlZCBpbnRvIGEgc2luZ2xlCj4+PiAgwqDCoMKgwqDCoMKgICog Y29udGlndW91cyBJT1ZBIGFsbG9jYXRpb24sIHNvIHRoaXMgaXMgaW5jcmVkaWJseSBlYXN5Lgo+ Pj4gIMKgwqDCoMKgwqDCoCAqLwo+Pj4gLcKgwqDCoCBzdGFydCA9IHNnX2RtYV9hZGRyZXNzKHNn KTsKPj4+IC3CoMKgwqAgZm9yX2VhY2hfc2coc2dfbmV4dChzZyksIHRtcCwgbmVudHMgLSAxLCBp KSB7Cj4+PiArwqDCoMKgIGZvcl9lYWNoX3NnKHNnLCB0bXAsIG5lbnRzLCBpKSB7Cj4+PiArwqDC oMKgwqDCoMKgwqAgaWYgKHNnX2lzX3BjaV9wMnBkbWEodG1wKSkKPj4KPj4gU2luY2UgdGhlIGZs YWcgaXMgYXNzb2NpYXRlZCB3aXRoIHRoZSBETUEgYWRkcmVzcyB3aGljaCB3aWxsIG5vIGxvbmdl cgo+PiBiZSB2YWxpZCwgc2hvdWxkbid0IGl0IGJlIGNsZWFyZWQ/IFRoZSBjaXJjdW1zdGFuY2Vz IGluIHdoaWNoIGxlYXZpbmcgaXQKPj4gYXJvdW5kIGNvdWxkIGNhdXNlIGEgcHJvYmxlbSBhcmUg dGVudW91cywgYnV0IGRlZmluaXRlbHkgcG9zc2libGUuCj4gCj4gWWVzLCB0aGF0J3MgYSBnb29k IGlkZWEuCj4gCj4gVGhhbmtzIGZvciB0aGUgcmV2aWV3IQo+IAo+IExvZ2FuCj4gCgpfX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXwpMaW51eC1udm1lIG1haWxp bmcgbGlzdApMaW51eC1udm1lQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xpc3RzLmluZnJh ZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9saW51eC1udm1lCg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.3 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI,NICE_REPLY_A, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B013EC433E0 for ; Fri, 12 Mar 2021 19:47:25 +0000 (UTC) Received: from smtp4.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 21BD264F83 for ; Fri, 12 Mar 2021 19:47:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 21BD264F83 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=arm.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp4.osuosl.org (Postfix) with ESMTP id A77AD4ED81; Fri, 12 Mar 2021 19:47:24 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp4.osuosl.org ([127.0.0.1]) by localhost (smtp4.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 7YHFz3L_75Uk; Fri, 12 Mar 2021 19:47:23 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp4.osuosl.org (Postfix) with ESMTP id 12EB94EBC5; Fri, 12 Mar 2021 19:47:23 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id D64ADC000B; Fri, 12 Mar 2021 19:47:22 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [IPv6:2605:bc80:3010::138]) by lists.linuxfoundation.org (Postfix) with ESMTP id C3027C0001 for ; Fri, 12 Mar 2021 19:47:20 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id A97898443B for ; Fri, 12 Mar 2021 19:47:20 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gqztMOTakb6i for ; Fri, 12 Mar 2021 19:47:19 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp1.osuosl.org (Postfix) with ESMTP id 5D46D8442C for ; Fri, 12 Mar 2021 19:47:19 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 20D92ED1; Fri, 12 Mar 2021 11:47:18 -0800 (PST) Received: from [10.57.52.136] (unknown [10.57.52.136]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id CA35E3F793; Fri, 12 Mar 2021 11:47:13 -0800 (PST) Subject: Re: [RFC PATCH v2 08/11] iommu/dma: Support PCI P2PDMA pages in dma-iommu map_sg To: Logan Gunthorpe , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org References: <20210311233142.7900-1-logang@deltatee.com> <20210311233142.7900-9-logang@deltatee.com> <45701356-ee41-1ad2-0e06-ca74af87b05a@deltatee.com> From: Robin Murphy Message-ID: <76cc1c82-3cf4-92d3-992f-5c876ed30523@arm.com> Date: Fri, 12 Mar 2021 19:47:08 +0000 User-Agent: Mozilla/5.0 (Windows NT 10.0; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <45701356-ee41-1ad2-0e06-ca74af87b05a@deltatee.com> Content-Language: en-GB Cc: Minturn Dave B , John Hubbard , Dave Hansen , Ira Weiny , Christoph Hellwig , Matthew Wilcox , Stephen Bates , Jason Gunthorpe , Jason Ekstrand , Daniel Vetter , Dan Williams , Jakowski Andrzej , =?UTF-8?Q?Christian_K=c3=b6nig?= , Xiong Jianxin X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: base64 Content-Type: text/plain; charset="utf-8"; Format="flowed" Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" T24gMjAyMS0wMy0xMiAxNzowMywgTG9nYW4gR3VudGhvcnBlIHdyb3RlOgo+IAo+IAo+IE9uIDIw MjEtMDMtMTIgODo1MiBhLm0uLCBSb2JpbiBNdXJwaHkgd3JvdGU6Cj4+IE9uIDIwMjEtMDMtMTEg MjM6MzEsIExvZ2FuIEd1bnRob3JwZSB3cm90ZToKPj4+IFdoZW4gYSBQQ0kgUDJQRE1BIHBhZ2Ug aXMgc2Vlbiwgc2V0IHRoZSBJT1ZBIGxlbmd0aCBvZiB0aGUgc2VnbWVudAo+Pj4gdG8gemVybyBz byB0aGF0IGl0IGlzIG5vdCBtYXBwZWQgaW50byB0aGUgSU9WQS4gVGhlbiwgaW4gZmluYWxpc2Vf c2coKSwKPj4+IGFwcGx5IHRoZSBhcHByb3ByaWF0ZSBidXMgYWRkcmVzcyB0byB0aGUgc2VnbWVu dC4gVGhlIElPVkEgaXMgbm90Cj4+PiBjcmVhdGVkIGlmIHRoZSBzY2F0dGVybGlzdCBvbmx5IGNv bnNpc3RzIG9mIFAyUERNQSBwYWdlcy4KPj4KPj4gVGhpcyBtaXNsZWQgbWUgYXQgZmlyc3QsIGJ1 dCBJIHNlZSB0aGUgaW1wbGVtZW50YXRpb24gZG9lcyBhY3R1YWxseQo+PiBhcHBlYXIgdG8gYWNj b21vZGF0ZSB0aGUgY2FzZSBvZiB3b3JraW5nIEFDUyB3aGVyZSBQMlAgKndvdWxkKiBzdGlsbAo+ PiBuZWVkIHRvIGJlIG1hcHBlZCBhdCB0aGUgSU9NTVUuCj4gCj4gWWVzLCB0aGF0J3MgY29ycmVj dC4KPj4+ICDCoCBzdGF0aWMgaW50IF9fZmluYWxpc2Vfc2coc3RydWN0IGRldmljZSAqZGV2LCBz dHJ1Y3Qgc2NhdHRlcmxpc3QgKnNnLAo+Pj4gaW50IG5lbnRzLAo+Pj4gLcKgwqDCoMKgwqDCoMKg IGRtYV9hZGRyX3QgZG1hX2FkZHIpCj4+PiArwqDCoMKgwqDCoMKgwqAgZG1hX2FkZHJfdCBkbWFf YWRkciwgdW5zaWduZWQgbG9uZyBhdHRycykKPj4+ICDCoCB7Cj4+PiAgwqDCoMKgwqDCoCBzdHJ1 Y3Qgc2NhdHRlcmxpc3QgKnMsICpjdXIgPSBzZzsKPj4+ICDCoMKgwqDCoMKgIHVuc2lnbmVkIGxv bmcgc2VnX21hc2sgPSBkbWFfZ2V0X3NlZ19ib3VuZGFyeShkZXYpOwo+Pj4gQEAgLTg2NCw2ICs4 NjUsMjAgQEAgc3RhdGljIGludCBfX2ZpbmFsaXNlX3NnKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4+ IHN0cnVjdCBzY2F0dGVybGlzdCAqc2csIGludCBuZW50cywKPj4+ICDCoMKgwqDCoMKgwqDCoMKg wqAgc2dfZG1hX2FkZHJlc3MocykgPSBETUFfTUFQUElOR19FUlJPUjsKPj4+ICDCoMKgwqDCoMKg wqDCoMKgwqAgc2dfZG1hX2xlbihzKSA9IDA7Cj4+PiAgwqAgK8KgwqDCoMKgwqDCoMKgIGlmIChp c19wY2lfcDJwZG1hX3BhZ2Uoc2dfcGFnZShzKSkgJiYgIXNfaW92YV9sZW4pIHsKPj4+ICvCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIGlmIChpID4gMCkKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqAgY3VyID0gc2dfbmV4dChjdXIpOwo+Pj4gKwo+Pj4gK8KgwqDCoMKgwqDCoMKgwqDC oMKgwqAgc2dfZG1hX2FkZHJlc3MoY3VyKSA9IHNnX3BoeXMocykgKyBzLT5vZmZzZXQgLQo+Pgo+ PiBBcmUgeW91IHN1cmUgYWJvdXQgdGhhdD8gOykKPiAKPiBEbyB5b3Ugc2VlIGEgYnVnPyBJIGRv bid0IGZvbGxvdyB5b3UuLi4KCnNnX3BoeXMoKSBhbHJlYWR5IGFjY291bnRzIGZvciB0aGUgb2Zm c2V0LCBzbyB5b3UncmUgYWRkaW5nIGl0IHR3aWNlLgoKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqAgcGNpX3AycGRtYV9idXNfb2Zmc2V0KHNnX3BhZ2UocykpOwo+Pgo+PiBDYW4g dGhlIGJ1cyBvZmZzZXQgbWFrZSBQMlAgYWRkcmVzc2VzIG92ZXJsYXAgd2l0aCByZWdpb25zIG9m IG1lbSBzcGFjZQo+PiB0aGF0IHdlIG1pZ2h0IHVzZSBmb3IgcmVndWxhciBJT1ZBIGFsbG9jYXRp b24/IFRoYXQgd291bGQgYmUgdmVyeSBiYWQuLi4KPiAKPiBOby4gSU9NTVUgZHJpdmVycyBhbHJl YWR5IGRpc2FsbG93IGFsbCBQQ0kgYWRkcmVzc2VzIGZyb20gYmVpbmcgdXNlZCBhcwo+IElPVkEg YWRkcmVzc2VzLiBTZWUsIGZvciBleGFtcGxlLCAgZG1hcl9pbml0X3Jlc2VydmVkX3Jhbmdlcygp LiBJdCB3b3VsZAo+IGJlIGEgaHVnZSBwcm9ibGVtIGZvciBhIHdob2xlIGxvdCBvZiBvdGhlciBy ZWFzb25zIGlmIGl0IGRpZG4ndC4KCkkga25vdyB3ZSByZXNlcnZlIHRoZSBvdXRib3VuZCB3aW5k b3dzIChsYXJnZWx5ICpiZWNhdXNlKiBzb21lIGhvc3QgCmJyaWRnZXMgd2lsbCBjb25zaWRlciB0 aG9zZSBhZGRyZXNzZXMgYXMgYXR0ZW1wdHMgYXQgdW5zdXBwb3J0ZWQgUDJQIGFuZCAKcHJldmVu dCB0aGVtIHdvcmtpbmcpLCBJIGp1c3Qgd2FudGVkIHRvIGNvbmZpcm0gdGhhdCB0aGlzIGJ1cyBv ZmZzZXQgaXMgCmFsd2F5cyBzb21ldGhpbmcgc21hbGwgdGhhdCBzdGF5cyB3aXRoaW4gdGhlIHJl bGV2YW50IHdpbmRvdywgcmF0aGVyIAp0aGFuIHNvbWV0aGluZyB0aGF0IG1pZ2h0IG1ha2UgYSBC QVIgYXBwZWFyIGluIGEgY29tcGxldGVseSBkaWZmZXJlbnQgCnBsYWNlIGZvciBQMlAgcHVycG9z ZXMuIElmIHNvLCB0aGF0J3MgZ29vZC4KCj4+PiBAQCAtOTYwLDExICs5NzUsMTIgQEAgc3RhdGlj IGludCBpb21tdV9kbWFfbWFwX3NnKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4+IHN0cnVjdCBzY2F0 dGVybGlzdCAqc2csCj4+PiAgwqDCoMKgwqDCoCBzdHJ1Y3QgaW9tbXVfZG1hX2Nvb2tpZSAqY29v a2llID0gZG9tYWluLT5pb3ZhX2Nvb2tpZTsKPj4+ICDCoMKgwqDCoMKgIHN0cnVjdCBpb3ZhX2Rv bWFpbiAqaW92YWQgPSAmY29va2llLT5pb3ZhZDsKPj4+ICDCoMKgwqDCoMKgIHN0cnVjdCBzY2F0 dGVybGlzdCAqcywgKnByZXYgPSBOVUxMOwo+Pj4gK8KgwqDCoCBzdHJ1Y3QgZGV2X3BhZ2VtYXAg KnBnbWFwID0gTlVMTDsKPj4+ICDCoMKgwqDCoMKgIGludCBwcm90ID0gZG1hX2luZm9fdG9fcHJv dChkaXIsIGRldl9pc19kbWFfY29oZXJlbnQoZGV2KSwgYXR0cnMpOwo+Pj4gIMKgwqDCoMKgwqAg ZG1hX2FkZHJfdCBpb3ZhOwo+Pj4gIMKgwqDCoMKgwqAgc2l6ZV90IGlvdmFfbGVuID0gMDsKPj4+ ICDCoMKgwqDCoMKgIHVuc2lnbmVkIGxvbmcgbWFzayA9IGRtYV9nZXRfc2VnX2JvdW5kYXJ5KGRl dik7Cj4+PiAtwqDCoMKgIGludCBpOwo+Pj4gK8KgwqDCoCBpbnQgaSwgbWFwID0gLTEsIHJldCA9 IDA7Cj4+PiAgwqAgwqDCoMKgwqDCoCBpZiAoc3RhdGljX2JyYW5jaF91bmxpa2VseSgmaW9tbXVf ZGVmZXJyZWRfYXR0YWNoX2VuYWJsZWQpICYmCj4+PiAgwqDCoMKgwqDCoMKgwqDCoMKgIGlvbW11 X2RlZmVycmVkX2F0dGFjaChkZXYsIGRvbWFpbikpCj4+PiBAQCAtOTkzLDYgKzEwMDksMjMgQEAg c3RhdGljIGludCBpb21tdV9kbWFfbWFwX3NnKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4+IHN0cnVj dCBzY2F0dGVybGlzdCAqc2csCj4+PiAgwqDCoMKgwqDCoMKgwqDCoMKgIHNfbGVuZ3RoID0gaW92 YV9hbGlnbihpb3ZhZCwgc19sZW5ndGggKyBzX2lvdmFfb2ZmKTsKPj4+ICDCoMKgwqDCoMKgwqDC oMKgwqAgcy0+bGVuZ3RoID0gc19sZW5ndGg7Cj4+PiAgwqAgK8KgwqDCoMKgwqDCoMKgIGlmIChp c19wY2lfcDJwZG1hX3BhZ2Uoc2dfcGFnZShzKSkpIHsKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDC oMKgIGlmIChzZ19wYWdlKHMpLT5wZ21hcCAhPSBwZ21hcCkgewo+Pj4gK8KgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCBwZ21hcCA9IHNnX3BhZ2UocyktPnBnbWFwOwo+Pj4gK8KgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCBtYXAgPSBwY2lfcDJwZG1hX2RtYV9tYXBfdHlwZShkZXYs IHBnbWFwKTsKPj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIH0KPj4+ICsKPj4+ICvCoMKgwqDC oMKgwqDCoMKgwqDCoMKgIGlmIChtYXAgPCAwKSB7Cj4+Cj4+IEl0IHJhdGhlciBmZWVscyBsaWtl IGl0IHNob3VsZCBiZSB0aGUgam9iIG9mIHdob2V2ZXIgY3JlYXRlcyB0aGUgbGlzdCBpbgo+PiB0 aGUgZmlyc3QgcGxhY2Ugbm90IHRvIHB1dCB1bnVzYWJsZSBwYWdlcyBpbiBpdCwgZXNwZWNpYWxs eSBzaW5jZSB0aGUKPj4gcDJwZG1hX21hcF90eXBlIGxvb2tzIHRvIGJlIGEgZmFpcmx5IGNvYXJz ZS1ncmFpbmVkIGFuZCBzdGF0aWMgdGhpbmcuCj4+IFRoZSBETUEgQVBJIGlzbid0IHJlc3BvbnNp YmxlIGZvciB2YWxpZGF0aW5nIG5vcm1hbCBtZW1vcnkgcGFnZXMsIHNvCj4+IHdoYXQgbWFrZXMg UDJQIHNwZWNpYWw/Cj4gCj4gWWVzLCB0aGF0IHdvdWxkIGJlIGlkZWFsLCBidXQgdGhlcmUncyBz b21lIGRpZmZpY3VsdGllcyB0aGVyZS4gRm9yIHRoZQo+IGRyaXZlciB0byBjaGVjayB0aGUgcGFn ZXMsIGl0IHdvdWxkIG5lZWQgdG8gbG9vcCB0aHJvdWdoIHRoZSBlbnRpcmUgU0cKPiBvbmUgbW9y ZSB0aW1lIG9uIGV2ZXJ5IHRyYW5zYWN0aW9uLCByZWdhcmRsZXNzIG9mIHdoZXRoZXIgdGhlcmUg YXJlCj4gUDJQRE1BIHBhZ2VzLCBvciBub3QuIFNvIHRoYXQgd2lsbCBoYXZlIGEgcGVyZm9ybWFu Y2UgaW1wYWN0IGV2ZW4gd2hlbgo+IHRoZSBmZWF0dXJlIGlzbid0IGJlaW5nIHVzZWQuIEkgZG9u J3QgdGhpbmsgdGhhdCdsbCBiZSBhY2NlcHRhYmxlIGZvcgo+IG1hbnkgZHJpdmVycy4KPiAKPiBU aGUgb3RoZXIgcG9zc2liaWxpdHkgaXMgZm9yIEdVUCB0byBkbyBpdCB3aGVuIGl0IGdldHMgdGhl IHBhZ2VzIGZyb20KPiB1c2Vyc3BhY2UuIEJ1dCBHVVAgZG9lc24ndCBoYXZlIGFsbCB0aGUgaW5m b3JtYXRpb24gdG8gZG8gdGhpcyBhdCB0aGUKPiBtb21lbnQuIFdlJ2QgaGF2ZSB0byBwYXNzIHRo ZSBzdHJ1Y3QgZGV2aWNlIHRoYXQgd2lsbCBldmVudHVhbGx5IG1hcCB0aGUKPiBwYWdlcyB0aHJv dWdoIGFsbCB0aGUgbmVzdGVkIGZ1bmN0aW9ucyBpbiB0aGUgR1VQIHRvIGRvIHRoYXQgdGVzdCBh dAo+IHRoYXQgdGltZS4gVGhpcyBtaWdodCBub3QgYmUgYSBiYWQgb3B0aW9uICh0aGF0IEkgaGFs ZiBsb29rZWQgaW50byksIGJ1dAo+IEknbSBub3Qgc3VyZSBob3cgYWNjZXB0YWJsZSBpdCB3b3Vs ZCBiZSB0byB0aGUgR1VQIGRldmVsb3BlcnMuCgpVcmdoLCB5ZXMsIGlmIGEgcGFnZSBtYXkgb3Ig bWF5IG5vdCBiZSB2YWxpZCBmb3IgcDJwIGRlcGVuZGluZyBvbiB3aGljaCAKZGV2aWNlIGlzIHRy eWluZyB0byBtYXAgaXQsIHRoZW4gaXQgcHJvYmFibHkgaXMgbW9zdCByZWFzb25hYmxlIHRvIApm aWd1cmUgdGhhdCBvdXQgYXQgdGhpcyBwb2ludC4gSXQncyBhIGxpdHRsZSB1bmZvcnR1bmF0ZSBo YXZpbmcgdG8gY29wZSAKd2l0aCBmYWlsdXJlIHNvIGxhdGUsIGJ1dCBvaCB3ZWxsLgoKPiBCdXQg ZXZlbiBpZiB3ZSBkbyB2ZXJpZnkgdGhlIHBhZ2VzIGFoZWFkIG9mIHRpbWUsIHdlIHN0aWxsIG5l ZWQgdGhlIHNhbWUKPiBpbmZyYXN0cnVjdHVyZSBpbiBkbWFfbWFwX3NnKCk7IGl0IGNvdWxkIG9u bHkgbm93IGJlIGEgQlVHIGlmIHRoZSBkcml2ZXIKPiBzZW50IGludmFsaWQgcGFnZXMgaW5zdGVh ZCBvZiBhbiBlcnJvciByZXR1cm4uCgpUaGUgaG9wZSB3YXMgdGhhdCB3ZSBjb3VsZCBzYXZlIGRv aW5nIGV2ZW4gdGhhdCAtIGUuZy4gaWYgeW91IHBhc3MgYSAKZG9kZ3kgcGFnZSBpbnRvIGRtYV9t YXBfcGFnZSgpLCBtYXliZSBwYWdlX3RvX3BoeXMoKSB3aWxsIGNyYXNoLCBtYXliZSAKeW91J2xs IGp1c3QgZW5kIHVwIHdpdGggYSBETUEgYWRkcmVzcyB0aGF0IHdvbid0IHdvcmssIGJ1dCBlaXRo ZXIgd2F5IGl0IApkb2Vzbid0IGNhcmUgaW4gaXRzIG93biByaWdodCAtIGJ1dCBpdCBzZWVtcyB0 aGF0J3MgbW9vdC4KCj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIHJldCA9IC1F UkVNT1RFSU87Cj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIGdvdG8gb3V0X3Jl c3RvcmVfc2c7Cj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCB9Cj4+PiArCj4+PiArwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCBpZiAobWFwKSB7Cj4+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgIHMtPmxlbmd0aCA9IDA7Cj4+Cj4+IEknbSBub3QgcmVhbGx5IHRocmlsbGVkIGFib3V0 IHRoZSBpZGVhIG9mIHBhc3NpbmcgemVyby1sZW5ndGggc2VnbWVudHMKPj4gdG8gaW9tbXVfbWFw X3NnKCkuIFllcywgaXQgaGFwcGVucyB0byB0cmljayB0aGUgY29uY2F0ZW5hdGlvbiBsb2dpYyBp bgo+PiB0aGUgY3VycmVudCBpbXBsZW1lbnRhdGlvbiBpbnRvIGRvaW5nIHdoYXQgeW91IHdhbnQs IGJ1dCBpdCBmZWVscyBmcmFnaWxlLgo+IAo+IFdlJ3JlIG5vdCBwYXNzaW5nIHplcm8gbGVuZ3Ro IHNlZ21lbnRzIHRvIGlvbW11X21hcF9zZygpIChvciBhbnkKPiBmdW5jdGlvbikuIFRoaXMgbG9v cCBpcyBqdXN0IHNjYW5uaW5nIHRvIGNhbGN1bGF0ZSB0aGUgbGVuZ3RoIG9mIHRoZQo+IHJlcXVp cmVkIElPVkEuIF9fZmluYWxpc2Vfc2coKSAod2hpY2ggaXMgaW50aW1hdGVseSB0aWVkIHRvIHRo aXMgbG9vcCkKPiB0aGVuIG5lZWRzIGEgd2F5IHRvIGRldGVybWluZSB3aGljaCBzZWdtZW50cyB3 ZXJlIFAyUCBzZWdtZW50cy4gVGhlCj4gZXhpc3RpbmcgY29kZSBhbHJlYWR5IG92ZXJ3cml0ZXMg cy0+bGVuZ3RoIHdpdGggYW4gYWxpZ25lZCBsZW5ndGggYW5kCj4gc3RvcmVzIHRoZSBvcmlnaW5h bCBsZW5ndGggaW4gc2dfZG1hX2xlbi4gU28gd2UncmUgbm90IHJlbHlpbmcgb24KPiB0cmlja2lu ZyBhbnkgbG9naWMgaGVyZS4KClllcywgd2UgdGVtcG9yYXJpbHkgc2h1ZmZsZSBpbiBwYWdlLWFs aWduZWQgcXVhbnRpdGllcyB0byBzYXRpc2Z5IHRoZSAKbmVlZHMgb2YgdGhlIGlvbW11X21hcF9z ZygpIGNhbGwsIGJlZm9yZSB1bnBhY2tpbmcgdGhpbmdzIGFnYWluIGluIApfX2ZpbmFsaXNlX3Nn KCkuIEl0J3Mgc29tZSBkaXNndXN0aW5nIHRyaWNrZXJ5IHRoYXQgSSdtIHBhcnRpY3VsYXJseSAK cHJvdWQgb2YuIE15IHBvaW50IGlzIHRoYXQgaWYgeW91IGhhdmUgYSBtaXggb2YgYm90aCBwMnAg YW5kIG5vcm1hbCAKc2VnbWVudHMgLSB3aGljaCBzZWVtcyB0byBiZSBhIGNhc2UgeW91IHdhbnQg dG8gc3VwcG9ydCAtIHRoZW4gdGhlIApsZW5ndGggb2YgMCB0aGF0IHlvdSBzZXQgdG8gZmxhZyBw MnAgc2VnbWVudHMgaGVyZSB3aWxsIGJlIHNlZW4gYnkgCmlvbW11X21hcF9zZygpIChhcyBpdCB3 YWxrcyB0aGUgbGlzdCB0byBtYXAgdGhlIG90aGVyIHNlZ21lbnRzKSBiZWZvcmUgCnlvdSB0aGVu IHVzZSBpdCBhcyBhIGtleSB0byBvdmVycmlkZSB0aGUgRE1BIGFkZHJlc3MgaW4gdGhlIGZpbmFs IHN0ZXAuIApJdCdzIG5vdCBhIGNvbmNlcm4gaWYgeW91IGhhdmUgYSBwMnAtb25seSBsaXN0IGFu ZCBzaG9ydC1jaXJjdWl0IApzdHJhaWdodCB0byB0aGF0IHN0ZXAgKGluIHdoaWNoIGNhc2UgYWxs IHRoZSBzaHVmZmxpbmcgd2FzIHdhc3RlZCBlZmZvcnQgCmFueXdheSksIGJ1dCBzaW5jZSBpdCdz IG5vdCBlbnRpcmVseSBjbGVhciB3aGF0IGEgc2VnbWVudCB3aXRoIHplcm8gCmxlbmd0aCB3b3Vs ZCBtZWFuIGluIGdlbmVyYWwsIGl0IHNlZW1zIGxpa2UgYSBnb29kIGlkZWEgdG8gYXZvaWQgcGFz c2luZyAKdGhlIGxpc3QgYWNyb3NzIGEgcHVibGljIGJvdW5kYXJ5IGluIHRoYXQgc3RhdGUsIGlm IHBvc3NpYmxlLgoKUm9iaW4uCgo+Pj4gIMKgIH0KPj4+ICDCoCDCoCBzdGF0aWMgdm9pZCBpb21t dV9kbWFfdW5tYXBfc2coc3RydWN0IGRldmljZSAqZGV2LCBzdHJ1Y3QKPj4+IHNjYXR0ZXJsaXN0 ICpzZywKPj4+ICDCoMKgwqDCoMKgwqDCoMKgwqAgaW50IG5lbnRzLCBlbnVtIGRtYV9kYXRhX2Rp cmVjdGlvbiBkaXIsIHVuc2lnbmVkIGxvbmcgYXR0cnMpCj4+PiAgwqAgewo+Pj4gLcKgwqDCoCBk bWFfYWRkcl90IHN0YXJ0LCBlbmQ7Cj4+PiArwqDCoMKgIGRtYV9hZGRyX3QgZW5kLCBzdGFydCA9 IERNQV9NQVBQSU5HX0VSUk9SOwo+Pj4gIMKgwqDCoMKgwqAgc3RydWN0IHNjYXR0ZXJsaXN0ICp0 bXA7Cj4+PiAgwqDCoMKgwqDCoCBpbnQgaTsKPj4+ICDCoCBAQCAtMTA1NCwxNCArMTA5MCwyMCBA QCBzdGF0aWMgdm9pZCBpb21tdV9kbWFfdW5tYXBfc2coc3RydWN0IGRldmljZQo+Pj4gKmRldiwg c3RydWN0IHNjYXR0ZXJsaXN0ICpzZywKPj4+ICDCoMKgwqDCoMKgwqAgKiBUaGUgc2NhdHRlcmxp c3Qgc2VnbWVudHMgYXJlIG1hcHBlZCBpbnRvIGEgc2luZ2xlCj4+PiAgwqDCoMKgwqDCoMKgICog Y29udGlndW91cyBJT1ZBIGFsbG9jYXRpb24sIHNvIHRoaXMgaXMgaW5jcmVkaWJseSBlYXN5Lgo+ Pj4gIMKgwqDCoMKgwqDCoCAqLwo+Pj4gLcKgwqDCoCBzdGFydCA9IHNnX2RtYV9hZGRyZXNzKHNn KTsKPj4+IC3CoMKgwqAgZm9yX2VhY2hfc2coc2dfbmV4dChzZyksIHRtcCwgbmVudHMgLSAxLCBp KSB7Cj4+PiArwqDCoMKgIGZvcl9lYWNoX3NnKHNnLCB0bXAsIG5lbnRzLCBpKSB7Cj4+PiArwqDC oMKgwqDCoMKgwqAgaWYgKHNnX2lzX3BjaV9wMnBkbWEodG1wKSkKPj4KPj4gU2luY2UgdGhlIGZs YWcgaXMgYXNzb2NpYXRlZCB3aXRoIHRoZSBETUEgYWRkcmVzcyB3aGljaCB3aWxsIG5vIGxvbmdl cgo+PiBiZSB2YWxpZCwgc2hvdWxkbid0IGl0IGJlIGNsZWFyZWQ/IFRoZSBjaXJjdW1zdGFuY2Vz IGluIHdoaWNoIGxlYXZpbmcgaXQKPj4gYXJvdW5kIGNvdWxkIGNhdXNlIGEgcHJvYmxlbSBhcmUg dGVudW91cywgYnV0IGRlZmluaXRlbHkgcG9zc2libGUuCj4gCj4gWWVzLCB0aGF0J3MgYSBnb29k IGlkZWEuCj4gCj4gVGhhbmtzIGZvciB0aGUgcmV2aWV3IQo+IAo+IExvZ2FuCj4gCl9fX19fX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCmlvbW11IG1haWxpbmcgbGlz dAppb21tdUBsaXN0cy5saW51eC1mb3VuZGF0aW9uLm9yZwpodHRwczovL2xpc3RzLmxpbnV4Zm91 bmRhdGlvbi5vcmcvbWFpbG1hbi9saXN0aW5mby9pb21tdQ==