From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.3 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 56AD0C43381 for ; Fri, 12 Mar 2021 16:26:06 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2126C6500B for ; Fri, 12 Mar 2021 16:26:06 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231636AbhCLQZd (ORCPT ); Fri, 12 Mar 2021 11:25:33 -0500 Received: from ale.deltatee.com ([204.191.154.188]:46686 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232550AbhCLQZI (ORCPT ); Fri, 12 Mar 2021 11:25:08 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:content-disposition; bh=z71Dy4S8XtDaYoM2XRkuE2wAHe2/+OE53EIe7Kw+TNU=; b=HqXLStXu7JHag3wGbK3mOgF/M8 gsYjKlyGMCNs6Kp2trWSjJWEdoTTL7l0upjMGEke4ssz2qv6SN3QkQZ8mcg+a143IC5xIOM3SURoN lw4HuqO4LhNWzh1WAKj6/9SxZhr2mvmmvBMwjW/oqcvFMPefmD+CQ/sZD2cfS3DQ9XdARnulhX379 XEmUxtLccrt+N+K+iBiVDqkj/U2P4KyQUA3p6kqQsHQwLLUNipLb7OSQ8YfLEh8+nzalEco/na8NL WnrW6BHAXcHt9X8Oyt7VfthZmCHT2QggtvHtaFRpzf4jqY5LDY0Xy1Vgj3qD6rJy+XfYFYP5DTIA3 2H01sjhg==; Received: from s01060023bee90a7d.cg.shawcable.net ([24.64.145.4] helo=[192.168.0.10]) by ale.deltatee.com with esmtpsa (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lKkar-0007E2-BR; Fri, 12 Mar 2021 09:24:54 -0700 To: Robin Murphy , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Minturn Dave B , John Hubbard , Dave Hansen , Ira Weiny , Matthew Wilcox , =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Jason Ekstrand , Daniel Vetter , Dan Williams , Stephen Bates , Jakowski Andrzej , Christoph Hellwig , Xiong Jianxin References: <20210311233142.7900-1-logang@deltatee.com> <20210311233142.7900-7-logang@deltatee.com> <215e1472-5294-d20a-a43a-ff6dfe8cd66e@arm.com> From: Logan Gunthorpe Message-ID: Date: Fri, 12 Mar 2021 09:24:51 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <215e1472-5294-d20a-a43a-ff6dfe8cd66e@arm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-CA Content-Transfer-Encoding: 8bit X-SA-Exim-Connect-IP: 24.64.145.4 X-SA-Exim-Rcpt-To: jianxin.xiong@intel.com, hch@lst.de, andrzej.jakowski@intel.com, sbates@raithlin.com, dan.j.williams@intel.com, daniel.vetter@ffwll.ch, jason@jlekstrand.net, jgg@ziepe.ca, christian.koenig@amd.com, willy@infradead.org, iweiny@intel.com, dave.hansen@linux.intel.com, jhubbard@nvidia.com, dave.b.minturn@intel.com, iommu@lists.linux-foundation.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, robin.murphy@arm.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org On 2021-03-12 8:52 a.m., Robin Murphy wrote: >> + >>           sg->dma_address = dma_direct_map_page(dev, sg_page(sg), >>                   sg->offset, sg->length, dir, attrs); >>           if (sg->dma_address == DMA_MAPPING_ERROR) >> @@ -411,7 +440,7 @@ int dma_direct_map_sg(struct device *dev, struct >> scatterlist *sgl, int nents, >>     out_unmap: >>       dma_direct_unmap_sg(dev, sgl, i, dir, attrs | >> DMA_ATTR_SKIP_CPU_SYNC); >> -    return 0; >> +    return ret; >>   } >>     dma_addr_t dma_direct_map_resource(struct device *dev, phys_addr_t >> paddr, >> diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c >> index b6a633679933..adc1a83950be 100644 >> --- a/kernel/dma/mapping.c >> +++ b/kernel/dma/mapping.c >> @@ -178,8 +178,15 @@ void dma_unmap_page_attrs(struct device *dev, >> dma_addr_t addr, size_t size, >>   EXPORT_SYMBOL(dma_unmap_page_attrs); >>     /* >> - * dma_maps_sg_attrs returns 0 on error and > 0 on success. >> - * It should never return a value < 0. >> + * dma_maps_sg_attrs returns 0 on any resource error and > 0 on success. >> + * >> + * If 0 is returned, the mapping can be retried and will succeed once >> + * sufficient resources are available. > > That's not a guarantee we can uphold. Retrying forever in the vain hope > that a device might evolve some extra address bits, or a bounce buffer > might magically grow big enough for a gigantic mapping, isn't > necessarily the best idea. Perhaps this is just poorly worded. Returning 0 is the normal case and nothing has changed there. The block layer, for example, will retry if zero is returned as this only happens if it failed to allocate resources for the mapping. The reason we have to return -1 is to tell the block layer not to retry these requests as they will never succeed in the future. >> + * >> + * If there are P2PDMA pages in the scatterlist then this function may >> + * return -EREMOTEIO to indicate that the pages are not mappable by the >> + * device. In this case, an error should be returned for the IO as it >> + * will never be successfully retried. >>    */ >>   int dma_map_sg_attrs(struct device *dev, struct scatterlist *sg, int >> nents, >>           enum dma_data_direction dir, unsigned long attrs) >> @@ -197,7 +204,7 @@ int dma_map_sg_attrs(struct device *dev, struct >> scatterlist *sg, int nents, >>           ents = dma_direct_map_sg(dev, sg, nents, dir, attrs); >>       else >>           ents = ops->map_sg(dev, sg, nents, dir, attrs); >> -    BUG_ON(ents < 0); >> + > > This scares me - I hesitate to imagine the amount of driver/subsystem > code out there that will see nonzero and merrily set off iterating a > negative number of segments, if we open the floodgates of allowing > implementations to return error codes here. Yes, but it will never happen on existing drivers/subsystems. The only way it can return a negative number is if the driver passes in P2PDMA pages which can't happen without changes in the driver. We are careful about where P2PDMA pages can get into so we don't have to worry about all the existing driver code out there. Logan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_SANE_1 autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 99169C433DB for ; Fri, 12 Mar 2021 16:27:24 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 338A660235 for ; Fri, 12 Mar 2021 16:27:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 338A660235 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=deltatee.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Subject:In-Reply-To:MIME-Version:Date:Message-ID: From:References:Cc:To:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=XGWgy2Aw3BSO5wcRzvrG+hQvVKYNZnGwRpN0yuGSxPQ=; b=iKYgPSovIJjgvv0ixOwZJZUDa p0APcpePGEis1YsgIIU+CGs77JfLl/FEsOyha3P09oqrICFqC3TJOxKCixuFjLv5G8qJuqwbQOSi+ wiYjdlshYE94CfOktouOI8XIatkYFdcNtwKi8mpSvGMkYyr2WKECsIXbHW/fk0nFOIGtO6C9laiP7 Bl/4bj6uAgmQWv8d/xBQjl0QfI3oeZt6J1SHtecBm1EIGC6wvNKoDoYUDyVufNbzyogj3+IExZ7cA 8MXfc5ocXeFns8CxDFhdx6KJVC4xFR6IMsMFq4AGu8P9pzlPK+5Zftb2eoRNse7P7ROJ9qX9rvX91 RzuJ9tY+A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lKkd1-00BxSh-Dk; Fri, 12 Mar 2021 16:27:07 +0000 Received: from ale.deltatee.com ([204.191.154.188]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lKkb2-00BweR-2S for linux-nvme@lists.infradead.org; Fri, 12 Mar 2021 16:25:07 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:content-disposition; bh=z71Dy4S8XtDaYoM2XRkuE2wAHe2/+OE53EIe7Kw+TNU=; b=HqXLStXu7JHag3wGbK3mOgF/M8 gsYjKlyGMCNs6Kp2trWSjJWEdoTTL7l0upjMGEke4ssz2qv6SN3QkQZ8mcg+a143IC5xIOM3SURoN lw4HuqO4LhNWzh1WAKj6/9SxZhr2mvmmvBMwjW/oqcvFMPefmD+CQ/sZD2cfS3DQ9XdARnulhX379 XEmUxtLccrt+N+K+iBiVDqkj/U2P4KyQUA3p6kqQsHQwLLUNipLb7OSQ8YfLEh8+nzalEco/na8NL WnrW6BHAXcHt9X8Oyt7VfthZmCHT2QggtvHtaFRpzf4jqY5LDY0Xy1Vgj3qD6rJy+XfYFYP5DTIA3 2H01sjhg==; Received: from s01060023bee90a7d.cg.shawcable.net ([24.64.145.4] helo=[192.168.0.10]) by ale.deltatee.com with esmtpsa (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lKkar-0007E2-BR; Fri, 12 Mar 2021 09:24:54 -0700 To: Robin Murphy , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org Cc: Minturn Dave B , John Hubbard , Dave Hansen , Ira Weiny , Matthew Wilcox , =?UTF-8?Q?Christian_K=c3=b6nig?= , Jason Gunthorpe , Jason Ekstrand , Daniel Vetter , Dan Williams , Stephen Bates , Jakowski Andrzej , Christoph Hellwig , Xiong Jianxin References: <20210311233142.7900-1-logang@deltatee.com> <20210311233142.7900-7-logang@deltatee.com> <215e1472-5294-d20a-a43a-ff6dfe8cd66e@arm.com> From: Logan Gunthorpe Message-ID: Date: Fri, 12 Mar 2021 09:24:51 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <215e1472-5294-d20a-a43a-ff6dfe8cd66e@arm.com> Content-Language: en-CA X-SA-Exim-Connect-IP: 24.64.145.4 X-SA-Exim-Rcpt-To: jianxin.xiong@intel.com, hch@lst.de, andrzej.jakowski@intel.com, sbates@raithlin.com, dan.j.williams@intel.com, daniel.vetter@ffwll.ch, jason@jlekstrand.net, jgg@ziepe.ca, christian.koenig@amd.com, willy@infradead.org, iweiny@intel.com, dave.hansen@linux.intel.com, jhubbard@nvidia.com, dave.b.minturn@intel.com, iommu@lists.linux-foundation.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, robin.murphy@arm.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210312_162505_358752_D23EA3C3 X-CRM114-Status: GOOD ( 23.18 ) X-BeenThere: linux-nvme@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "Linux-nvme" Errors-To: linux-nvme-bounces+linux-nvme=archiver.kernel.org@lists.infradead.org CgpPbiAyMDIxLTAzLTEyIDg6NTIgYS5tLiwgUm9iaW4gTXVycGh5IHdyb3RlOgo+PiArCj4+IMKg wqDCoMKgwqDCoMKgwqDCoCBzZy0+ZG1hX2FkZHJlc3MgPSBkbWFfZGlyZWN0X21hcF9wYWdlKGRl diwgc2dfcGFnZShzZyksCj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2ct Pm9mZnNldCwgc2ctPmxlbmd0aCwgZGlyLCBhdHRycyk7Cj4+IMKgwqDCoMKgwqDCoMKgwqDCoCBp ZiAoc2ctPmRtYV9hZGRyZXNzID09IERNQV9NQVBQSU5HX0VSUk9SKQo+PiBAQCAtNDExLDcgKzQ0 MCw3IEBAIGludCBkbWFfZGlyZWN0X21hcF9zZyhzdHJ1Y3QgZGV2aWNlICpkZXYsIHN0cnVjdAo+ PiBzY2F0dGVybGlzdCAqc2dsLCBpbnQgbmVudHMsCj4+IMKgIMKgIG91dF91bm1hcDoKPj4gwqDC oMKgwqDCoCBkbWFfZGlyZWN0X3VubWFwX3NnKGRldiwgc2dsLCBpLCBkaXIsIGF0dHJzIHwKPj4g RE1BX0FUVFJfU0tJUF9DUFVfU1lOQyk7Cj4+IC3CoMKgwqAgcmV0dXJuIDA7Cj4+ICvCoMKgwqAg cmV0dXJuIHJldDsKPj4gwqAgfQo+PiDCoCDCoCBkbWFfYWRkcl90IGRtYV9kaXJlY3RfbWFwX3Jl c291cmNlKHN0cnVjdCBkZXZpY2UgKmRldiwgcGh5c19hZGRyX3QKPj4gcGFkZHIsCj4+IGRpZmYg LS1naXQgYS9rZXJuZWwvZG1hL21hcHBpbmcuYyBiL2tlcm5lbC9kbWEvbWFwcGluZy5jCj4+IGlu ZGV4IGI2YTYzMzY3OTkzMy4uYWRjMWE4Mzk1MGJlIDEwMDY0NAo+PiAtLS0gYS9rZXJuZWwvZG1h L21hcHBpbmcuYwo+PiArKysgYi9rZXJuZWwvZG1hL21hcHBpbmcuYwo+PiBAQCAtMTc4LDggKzE3 OCwxNSBAQCB2b2lkIGRtYV91bm1hcF9wYWdlX2F0dHJzKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4g ZG1hX2FkZHJfdCBhZGRyLCBzaXplX3Qgc2l6ZSwKPj4gwqAgRVhQT1JUX1NZTUJPTChkbWFfdW5t YXBfcGFnZV9hdHRycyk7Cj4+IMKgIMKgIC8qCj4+IC0gKiBkbWFfbWFwc19zZ19hdHRycyByZXR1 cm5zIDAgb24gZXJyb3IgYW5kID4gMCBvbiBzdWNjZXNzLgo+PiAtICogSXQgc2hvdWxkIG5ldmVy IHJldHVybiBhIHZhbHVlIDwgMC4KPj4gKyAqIGRtYV9tYXBzX3NnX2F0dHJzIHJldHVybnMgMCBv biBhbnkgcmVzb3VyY2UgZXJyb3IgYW5kID4gMCBvbiBzdWNjZXNzLgo+PiArICoKPj4gKyAqIElm IDAgaXMgcmV0dXJuZWQsIHRoZSBtYXBwaW5nIGNhbiBiZSByZXRyaWVkIGFuZCB3aWxsIHN1Y2Nl ZWQgb25jZQo+PiArICogc3VmZmljaWVudCByZXNvdXJjZXMgYXJlIGF2YWlsYWJsZS4KPiAKPiBU aGF0J3Mgbm90IGEgZ3VhcmFudGVlIHdlIGNhbiB1cGhvbGQuIFJldHJ5aW5nIGZvcmV2ZXIgaW4g dGhlIHZhaW4gaG9wZQo+IHRoYXQgYSBkZXZpY2UgbWlnaHQgZXZvbHZlIHNvbWUgZXh0cmEgYWRk cmVzcyBiaXRzLCBvciBhIGJvdW5jZSBidWZmZXIKPiBtaWdodCBtYWdpY2FsbHkgZ3JvdyBiaWcg ZW5vdWdoIGZvciBhIGdpZ2FudGljIG1hcHBpbmcsIGlzbid0Cj4gbmVjZXNzYXJpbHkgdGhlIGJl c3QgaWRlYS4KClBlcmhhcHMgdGhpcyBpcyBqdXN0IHBvb3JseSB3b3JkZWQuIFJldHVybmluZyAw IGlzIHRoZSBub3JtYWwgY2FzZSBhbmQKbm90aGluZyBoYXMgY2hhbmdlZCB0aGVyZS4gVGhlIGJs b2NrIGxheWVyLCBmb3IgZXhhbXBsZSwgd2lsbCByZXRyeSBpZgp6ZXJvIGlzIHJldHVybmVkIGFz IHRoaXMgb25seSBoYXBwZW5zIGlmIGl0IGZhaWxlZCB0byBhbGxvY2F0ZSByZXNvdXJjZXMKZm9y IHRoZSBtYXBwaW5nLiBUaGUgcmVhc29uIHdlIGhhdmUgdG8gcmV0dXJuIC0xIGlzIHRvIHRlbGwg dGhlIGJsb2NrCmxheWVyIG5vdCB0byByZXRyeSB0aGVzZSByZXF1ZXN0cyBhcyB0aGV5IHdpbGwg bmV2ZXIgc3VjY2VlZCBpbiB0aGUgZnV0dXJlLgoKPj4gKyAqCj4+ICsgKiBJZiB0aGVyZSBhcmUg UDJQRE1BIHBhZ2VzIGluIHRoZSBzY2F0dGVybGlzdCB0aGVuIHRoaXMgZnVuY3Rpb24gbWF5Cj4+ ICsgKiByZXR1cm4gLUVSRU1PVEVJTyB0byBpbmRpY2F0ZSB0aGF0IHRoZSBwYWdlcyBhcmUgbm90 IG1hcHBhYmxlIGJ5IHRoZQo+PiArICogZGV2aWNlLiBJbiB0aGlzIGNhc2UsIGFuIGVycm9yIHNo b3VsZCBiZSByZXR1cm5lZCBmb3IgdGhlIElPIGFzIGl0Cj4+ICsgKiB3aWxsIG5ldmVyIGJlIHN1 Y2Nlc3NmdWxseSByZXRyaWVkLgo+PiDCoMKgICovCj4+IMKgIGludCBkbWFfbWFwX3NnX2F0dHJz KHN0cnVjdCBkZXZpY2UgKmRldiwgc3RydWN0IHNjYXR0ZXJsaXN0ICpzZywgaW50Cj4+IG5lbnRz LAo+PiDCoMKgwqDCoMKgwqDCoMKgwqAgZW51bSBkbWFfZGF0YV9kaXJlY3Rpb24gZGlyLCB1bnNp Z25lZCBsb25nIGF0dHJzKQo+PiBAQCAtMTk3LDcgKzIwNCw3IEBAIGludCBkbWFfbWFwX3NnX2F0 dHJzKHN0cnVjdCBkZXZpY2UgKmRldiwgc3RydWN0Cj4+IHNjYXR0ZXJsaXN0ICpzZywgaW50IG5l bnRzLAo+PiDCoMKgwqDCoMKgwqDCoMKgwqAgZW50cyA9IGRtYV9kaXJlY3RfbWFwX3NnKGRldiwg c2csIG5lbnRzLCBkaXIsIGF0dHJzKTsKPj4gwqDCoMKgwqDCoCBlbHNlCj4+IMKgwqDCoMKgwqDC oMKgwqDCoCBlbnRzID0gb3BzLT5tYXBfc2coZGV2LCBzZywgbmVudHMsIGRpciwgYXR0cnMpOwo+ PiAtwqDCoMKgIEJVR19PTihlbnRzIDwgMCk7Cj4+ICsKPiAKPiBUaGlzIHNjYXJlcyBtZSAtIEkg aGVzaXRhdGUgdG8gaW1hZ2luZSB0aGUgYW1vdW50IG9mIGRyaXZlci9zdWJzeXN0ZW0KPiBjb2Rl IG91dCB0aGVyZSB0aGF0IHdpbGwgc2VlIG5vbnplcm8gYW5kIG1lcnJpbHkgc2V0IG9mZiBpdGVy YXRpbmcgYQo+IG5lZ2F0aXZlIG51bWJlciBvZiBzZWdtZW50cywgaWYgd2Ugb3BlbiB0aGUgZmxv b2RnYXRlcyBvZiBhbGxvd2luZwo+IGltcGxlbWVudGF0aW9ucyB0byByZXR1cm4gZXJyb3IgY29k ZXMgaGVyZS4KClllcywgYnV0IGl0IHdpbGwgbmV2ZXIgaGFwcGVuIG9uIGV4aXN0aW5nIGRyaXZl cnMvc3Vic3lzdGVtcy4gVGhlIG9ubHkKd2F5IGl0IGNhbiByZXR1cm4gYSBuZWdhdGl2ZSBudW1i ZXIgaXMgaWYgdGhlIGRyaXZlciBwYXNzZXMgaW4gUDJQRE1BCnBhZ2VzIHdoaWNoIGNhbid0IGhh cHBlbiB3aXRob3V0IGNoYW5nZXMgaW4gdGhlIGRyaXZlci4gV2UgYXJlIGNhcmVmdWwKYWJvdXQg d2hlcmUgUDJQRE1BIHBhZ2VzIGNhbiBnZXQgaW50byBzbyB3ZSBkb24ndCBoYXZlIHRvIHdvcnJ5 IGFib3V0CmFsbCB0aGUgZXhpc3RpbmcgZHJpdmVyIGNvZGUgb3V0IHRoZXJlLgoKTG9nYW4KCl9f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fCkxpbnV4LW52bWUg bWFpbGluZyBsaXN0CkxpbnV4LW52bWVAbGlzdHMuaW5mcmFkZWFkLm9yZwpodHRwOi8vbGlzdHMu aW5mcmFkZWFkLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2xpbnV4LW52bWUK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.1 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_SANE_1 autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5676FC433E0 for ; Fri, 12 Mar 2021 16:25:07 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E8E6A65006 for ; Fri, 12 Mar 2021 16:25:06 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E8E6A65006 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=deltatee.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=iommu-bounces@lists.linux-foundation.org Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id ABA6E839B4; Fri, 12 Mar 2021 16:25:06 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id YvyDEuBP8vMt; Fri, 12 Mar 2021 16:25:05 +0000 (UTC) Received: from lists.linuxfoundation.org (lf-lists.osuosl.org [IPv6:2605:bc80:3010:104::8cd3:938]) by smtp1.osuosl.org (Postfix) with ESMTP id 88E5D834F7; Fri, 12 Mar 2021 16:25:05 +0000 (UTC) Received: from lf-lists.osuosl.org (localhost [127.0.0.1]) by lists.linuxfoundation.org (Postfix) with ESMTP id 5FD84C000A; Fri, 12 Mar 2021 16:25:05 +0000 (UTC) Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id A0C7FC0001 for ; Fri, 12 Mar 2021 16:25:03 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id 946D283A8E for ; Fri, 12 Mar 2021 16:25:03 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id hSPFsJ1tmLaN for ; Fri, 12 Mar 2021 16:25:02 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.8.0 Received: from ale.deltatee.com (ale.deltatee.com [204.191.154.188]) by smtp1.osuosl.org (Postfix) with ESMTPS id CC5B4834F7 for ; Fri, 12 Mar 2021 16:25:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:content-disposition; bh=z71Dy4S8XtDaYoM2XRkuE2wAHe2/+OE53EIe7Kw+TNU=; b=HqXLStXu7JHag3wGbK3mOgF/M8 gsYjKlyGMCNs6Kp2trWSjJWEdoTTL7l0upjMGEke4ssz2qv6SN3QkQZ8mcg+a143IC5xIOM3SURoN lw4HuqO4LhNWzh1WAKj6/9SxZhr2mvmmvBMwjW/oqcvFMPefmD+CQ/sZD2cfS3DQ9XdARnulhX379 XEmUxtLccrt+N+K+iBiVDqkj/U2P4KyQUA3p6kqQsHQwLLUNipLb7OSQ8YfLEh8+nzalEco/na8NL WnrW6BHAXcHt9X8Oyt7VfthZmCHT2QggtvHtaFRpzf4jqY5LDY0Xy1Vgj3qD6rJy+XfYFYP5DTIA3 2H01sjhg==; Received: from s01060023bee90a7d.cg.shawcable.net ([24.64.145.4] helo=[192.168.0.10]) by ale.deltatee.com with esmtpsa (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.92) (envelope-from ) id 1lKkar-0007E2-BR; Fri, 12 Mar 2021 09:24:54 -0700 To: Robin Murphy , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, linux-pci@vger.kernel.org, linux-mm@kvack.org, iommu@lists.linux-foundation.org References: <20210311233142.7900-1-logang@deltatee.com> <20210311233142.7900-7-logang@deltatee.com> <215e1472-5294-d20a-a43a-ff6dfe8cd66e@arm.com> From: Logan Gunthorpe Message-ID: Date: Fri, 12 Mar 2021 09:24:51 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.8.0 MIME-Version: 1.0 In-Reply-To: <215e1472-5294-d20a-a43a-ff6dfe8cd66e@arm.com> Content-Language: en-CA X-SA-Exim-Connect-IP: 24.64.145.4 X-SA-Exim-Rcpt-To: jianxin.xiong@intel.com, hch@lst.de, andrzej.jakowski@intel.com, sbates@raithlin.com, dan.j.williams@intel.com, daniel.vetter@ffwll.ch, jason@jlekstrand.net, jgg@ziepe.ca, christian.koenig@amd.com, willy@infradead.org, iweiny@intel.com, dave.hansen@linux.intel.com, jhubbard@nvidia.com, dave.b.minturn@intel.com, iommu@lists.linux-foundation.org, linux-mm@kvack.org, linux-pci@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, robin.murphy@arm.com X-SA-Exim-Mail-From: logang@deltatee.com Subject: Re: [RFC PATCH v2 06/11] dma-direct: Support PCI P2PDMA pages in dma-direct map_sg X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Cc: Minturn Dave B , John Hubbard , Dave Hansen , Ira Weiny , Christoph Hellwig , Matthew Wilcox , Stephen Bates , Jason Gunthorpe , Jason Ekstrand , Daniel Vetter , Dan Williams , Jakowski Andrzej , =?UTF-8?Q?Christian_K=c3=b6nig?= , Xiong Jianxin X-BeenThere: iommu@lists.linux-foundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Development issues for Linux IOMMU support List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Errors-To: iommu-bounces@lists.linux-foundation.org Sender: "iommu" CgpPbiAyMDIxLTAzLTEyIDg6NTIgYS5tLiwgUm9iaW4gTXVycGh5IHdyb3RlOgo+PiArCj4+IMKg wqDCoMKgwqDCoMKgwqDCoCBzZy0+ZG1hX2FkZHJlc3MgPSBkbWFfZGlyZWN0X21hcF9wYWdlKGRl diwgc2dfcGFnZShzZyksCj4+IMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgc2ct Pm9mZnNldCwgc2ctPmxlbmd0aCwgZGlyLCBhdHRycyk7Cj4+IMKgwqDCoMKgwqDCoMKgwqDCoCBp ZiAoc2ctPmRtYV9hZGRyZXNzID09IERNQV9NQVBQSU5HX0VSUk9SKQo+PiBAQCAtNDExLDcgKzQ0 MCw3IEBAIGludCBkbWFfZGlyZWN0X21hcF9zZyhzdHJ1Y3QgZGV2aWNlICpkZXYsIHN0cnVjdAo+ PiBzY2F0dGVybGlzdCAqc2dsLCBpbnQgbmVudHMsCj4+IMKgIMKgIG91dF91bm1hcDoKPj4gwqDC oMKgwqDCoCBkbWFfZGlyZWN0X3VubWFwX3NnKGRldiwgc2dsLCBpLCBkaXIsIGF0dHJzIHwKPj4g RE1BX0FUVFJfU0tJUF9DUFVfU1lOQyk7Cj4+IC3CoMKgwqAgcmV0dXJuIDA7Cj4+ICvCoMKgwqAg cmV0dXJuIHJldDsKPj4gwqAgfQo+PiDCoCDCoCBkbWFfYWRkcl90IGRtYV9kaXJlY3RfbWFwX3Jl c291cmNlKHN0cnVjdCBkZXZpY2UgKmRldiwgcGh5c19hZGRyX3QKPj4gcGFkZHIsCj4+IGRpZmYg LS1naXQgYS9rZXJuZWwvZG1hL21hcHBpbmcuYyBiL2tlcm5lbC9kbWEvbWFwcGluZy5jCj4+IGlu ZGV4IGI2YTYzMzY3OTkzMy4uYWRjMWE4Mzk1MGJlIDEwMDY0NAo+PiAtLS0gYS9rZXJuZWwvZG1h L21hcHBpbmcuYwo+PiArKysgYi9rZXJuZWwvZG1hL21hcHBpbmcuYwo+PiBAQCAtMTc4LDggKzE3 OCwxNSBAQCB2b2lkIGRtYV91bm1hcF9wYWdlX2F0dHJzKHN0cnVjdCBkZXZpY2UgKmRldiwKPj4g ZG1hX2FkZHJfdCBhZGRyLCBzaXplX3Qgc2l6ZSwKPj4gwqAgRVhQT1JUX1NZTUJPTChkbWFfdW5t YXBfcGFnZV9hdHRycyk7Cj4+IMKgIMKgIC8qCj4+IC0gKiBkbWFfbWFwc19zZ19hdHRycyByZXR1 cm5zIDAgb24gZXJyb3IgYW5kID4gMCBvbiBzdWNjZXNzLgo+PiAtICogSXQgc2hvdWxkIG5ldmVy IHJldHVybiBhIHZhbHVlIDwgMC4KPj4gKyAqIGRtYV9tYXBzX3NnX2F0dHJzIHJldHVybnMgMCBv biBhbnkgcmVzb3VyY2UgZXJyb3IgYW5kID4gMCBvbiBzdWNjZXNzLgo+PiArICoKPj4gKyAqIElm IDAgaXMgcmV0dXJuZWQsIHRoZSBtYXBwaW5nIGNhbiBiZSByZXRyaWVkIGFuZCB3aWxsIHN1Y2Nl ZWQgb25jZQo+PiArICogc3VmZmljaWVudCByZXNvdXJjZXMgYXJlIGF2YWlsYWJsZS4KPiAKPiBU aGF0J3Mgbm90IGEgZ3VhcmFudGVlIHdlIGNhbiB1cGhvbGQuIFJldHJ5aW5nIGZvcmV2ZXIgaW4g dGhlIHZhaW4gaG9wZQo+IHRoYXQgYSBkZXZpY2UgbWlnaHQgZXZvbHZlIHNvbWUgZXh0cmEgYWRk cmVzcyBiaXRzLCBvciBhIGJvdW5jZSBidWZmZXIKPiBtaWdodCBtYWdpY2FsbHkgZ3JvdyBiaWcg ZW5vdWdoIGZvciBhIGdpZ2FudGljIG1hcHBpbmcsIGlzbid0Cj4gbmVjZXNzYXJpbHkgdGhlIGJl c3QgaWRlYS4KClBlcmhhcHMgdGhpcyBpcyBqdXN0IHBvb3JseSB3b3JkZWQuIFJldHVybmluZyAw IGlzIHRoZSBub3JtYWwgY2FzZSBhbmQKbm90aGluZyBoYXMgY2hhbmdlZCB0aGVyZS4gVGhlIGJs b2NrIGxheWVyLCBmb3IgZXhhbXBsZSwgd2lsbCByZXRyeSBpZgp6ZXJvIGlzIHJldHVybmVkIGFz IHRoaXMgb25seSBoYXBwZW5zIGlmIGl0IGZhaWxlZCB0byBhbGxvY2F0ZSByZXNvdXJjZXMKZm9y IHRoZSBtYXBwaW5nLiBUaGUgcmVhc29uIHdlIGhhdmUgdG8gcmV0dXJuIC0xIGlzIHRvIHRlbGwg dGhlIGJsb2NrCmxheWVyIG5vdCB0byByZXRyeSB0aGVzZSByZXF1ZXN0cyBhcyB0aGV5IHdpbGwg bmV2ZXIgc3VjY2VlZCBpbiB0aGUgZnV0dXJlLgoKPj4gKyAqCj4+ICsgKiBJZiB0aGVyZSBhcmUg UDJQRE1BIHBhZ2VzIGluIHRoZSBzY2F0dGVybGlzdCB0aGVuIHRoaXMgZnVuY3Rpb24gbWF5Cj4+ ICsgKiByZXR1cm4gLUVSRU1PVEVJTyB0byBpbmRpY2F0ZSB0aGF0IHRoZSBwYWdlcyBhcmUgbm90 IG1hcHBhYmxlIGJ5IHRoZQo+PiArICogZGV2aWNlLiBJbiB0aGlzIGNhc2UsIGFuIGVycm9yIHNo b3VsZCBiZSByZXR1cm5lZCBmb3IgdGhlIElPIGFzIGl0Cj4+ICsgKiB3aWxsIG5ldmVyIGJlIHN1 Y2Nlc3NmdWxseSByZXRyaWVkLgo+PiDCoMKgICovCj4+IMKgIGludCBkbWFfbWFwX3NnX2F0dHJz KHN0cnVjdCBkZXZpY2UgKmRldiwgc3RydWN0IHNjYXR0ZXJsaXN0ICpzZywgaW50Cj4+IG5lbnRz LAo+PiDCoMKgwqDCoMKgwqDCoMKgwqAgZW51bSBkbWFfZGF0YV9kaXJlY3Rpb24gZGlyLCB1bnNp Z25lZCBsb25nIGF0dHJzKQo+PiBAQCAtMTk3LDcgKzIwNCw3IEBAIGludCBkbWFfbWFwX3NnX2F0 dHJzKHN0cnVjdCBkZXZpY2UgKmRldiwgc3RydWN0Cj4+IHNjYXR0ZXJsaXN0ICpzZywgaW50IG5l bnRzLAo+PiDCoMKgwqDCoMKgwqDCoMKgwqAgZW50cyA9IGRtYV9kaXJlY3RfbWFwX3NnKGRldiwg c2csIG5lbnRzLCBkaXIsIGF0dHJzKTsKPj4gwqDCoMKgwqDCoCBlbHNlCj4+IMKgwqDCoMKgwqDC oMKgwqDCoCBlbnRzID0gb3BzLT5tYXBfc2coZGV2LCBzZywgbmVudHMsIGRpciwgYXR0cnMpOwo+ PiAtwqDCoMKgIEJVR19PTihlbnRzIDwgMCk7Cj4+ICsKPiAKPiBUaGlzIHNjYXJlcyBtZSAtIEkg aGVzaXRhdGUgdG8gaW1hZ2luZSB0aGUgYW1vdW50IG9mIGRyaXZlci9zdWJzeXN0ZW0KPiBjb2Rl IG91dCB0aGVyZSB0aGF0IHdpbGwgc2VlIG5vbnplcm8gYW5kIG1lcnJpbHkgc2V0IG9mZiBpdGVy YXRpbmcgYQo+IG5lZ2F0aXZlIG51bWJlciBvZiBzZWdtZW50cywgaWYgd2Ugb3BlbiB0aGUgZmxv b2RnYXRlcyBvZiBhbGxvd2luZwo+IGltcGxlbWVudGF0aW9ucyB0byByZXR1cm4gZXJyb3IgY29k ZXMgaGVyZS4KClllcywgYnV0IGl0IHdpbGwgbmV2ZXIgaGFwcGVuIG9uIGV4aXN0aW5nIGRyaXZl cnMvc3Vic3lzdGVtcy4gVGhlIG9ubHkKd2F5IGl0IGNhbiByZXR1cm4gYSBuZWdhdGl2ZSBudW1i ZXIgaXMgaWYgdGhlIGRyaXZlciBwYXNzZXMgaW4gUDJQRE1BCnBhZ2VzIHdoaWNoIGNhbid0IGhh cHBlbiB3aXRob3V0IGNoYW5nZXMgaW4gdGhlIGRyaXZlci4gV2UgYXJlIGNhcmVmdWwKYWJvdXQg d2hlcmUgUDJQRE1BIHBhZ2VzIGNhbiBnZXQgaW50byBzbyB3ZSBkb24ndCBoYXZlIHRvIHdvcnJ5 IGFib3V0CmFsbCB0aGUgZXhpc3RpbmcgZHJpdmVyIGNvZGUgb3V0IHRoZXJlLgoKTG9nYW4KX19f X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KaW9tbXUgbWFpbGlu ZyBsaXN0CmlvbW11QGxpc3RzLmxpbnV4LWZvdW5kYXRpb24ub3JnCmh0dHBzOi8vbGlzdHMubGlu dXhmb3VuZGF0aW9uLm9yZy9tYWlsbWFuL2xpc3RpbmZvL2lvbW11