From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-x243.google.com (mail-io0-x243.google.com [IPv6:2607:f8b0:4001:c06::243]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id A7A8E21F3C181 for ; Wed, 11 Oct 2017 08:57:57 -0700 (PDT) Received: by mail-io0-x243.google.com with SMTP id h70so1446331ioi.4 for ; Wed, 11 Oct 2017 09:01:27 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20171011115410.GF30803@8bytes.org> References: <150764693502.16882.15848797003793552156.stgit@dwillia2-desk3.amr.corp.intel.com> <150764701194.16882.9682569707416653741.stgit@dwillia2-desk3.amr.corp.intel.com> <20171011115410.GF30803@8bytes.org> From: Dan Williams Date: Wed, 11 Oct 2017 09:01:25 -0700 Message-ID: Subject: Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" To: Joerg Roedel Cc: "J. Bruce Fields" , Doug Ledford , Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma@vger.kernel.org, Linux API , "linux-nvdimm@lists.01.org" , Dave Chinner , iommu@lists.linux-foundation.org, Hal Rosenstock , linux-xfs@vger.kernel.org, linux-mm , Jeff Layton , linux-fsdevel , Sean Hefty , David Woodhouse , Christoph Hellwig List-ID: On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel wrote: > On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote: >> +static void ib_umem_lease_break(void *__umem) >> +{ >> + struct ib_umem *umem = umem; >> + struct ib_device *idev = umem->context->device; >> + struct device *dev = idev->dma_device; >> + struct scatterlist *sgl = umem->sg_head.sgl; >> + >> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK, >> + iommu_sg_num_pages(dev, sgl, umem->npages)); >> +} > > This looks like an invitation to break your code by random iommu-driver > changes. There is no guarantee that an iommu-backed dma-api > implemenation will map exactly iommu_sg_num_pages() pages for a given > sg-list. In other words, you are mixing the use of the IOMMU-API and the > DMA-API in an incompatible way that only works because you know the > internals of the iommu-drivers. > > I've seen in another patch that your changes strictly require an IOMMU, > so you what you should do instead is to switch from the DMA-API to the > IOMMU-API and do the address-space management yourself. > Ok, I'll switch over completely to the iommu api for this. It will also address Robin's concern. _______________________________________________ Linux-nvdimm mailing list Linux-nvdimm@lists.01.org https://lists.01.org/mailman/listinfo/linux-nvdimm From mboxrd@z Thu Jan 1 00:00:00 1970 From: Dan Williams Subject: Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings Date: Wed, 11 Oct 2017 09:01:25 -0700 Message-ID: References: <150764693502.16882.15848797003793552156.stgit@dwillia2-desk3.amr.corp.intel.com> <150764701194.16882.9682569707416653741.stgit@dwillia2-desk3.amr.corp.intel.com> <20171011115410.GF30803@8bytes.org> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <20171011115410.GF30803-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org Errors-To: iommu-bounces-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org To: Joerg Roedel Cc: "J. Bruce Fields" , Doug Ledford , Jan Kara , "Darrick J. Wong" , linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Linux API , "linux-nvdimm-hn68Rpc1hR1g9hUCZPvPmw@public.gmane.org" , Dave Chinner , iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org, Hal Rosenstock , linux-xfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-mm , Jeff Moyer , Jeff Layton , Ross Zwisler , linux-fsdevel , Sean Hefty , David Woodhouse , Christoph Hellwig List-Id: linux-rdma@vger.kernel.org On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel wrote: > On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote: >> +static void ib_umem_lease_break(void *__umem) >> +{ >> + struct ib_umem *umem = umem; >> + struct ib_device *idev = umem->context->device; >> + struct device *dev = idev->dma_device; >> + struct scatterlist *sgl = umem->sg_head.sgl; >> + >> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK, >> + iommu_sg_num_pages(dev, sgl, umem->npages)); >> +} > > This looks like an invitation to break your code by random iommu-driver > changes. There is no guarantee that an iommu-backed dma-api > implemenation will map exactly iommu_sg_num_pages() pages for a given > sg-list. In other words, you are mixing the use of the IOMMU-API and the > DMA-API in an incompatible way that only works because you know the > internals of the iommu-drivers. > > I've seen in another patch that your changes strictly require an IOMMU, > so you what you should do instead is to switch from the DMA-API to the > IOMMU-API and do the address-space management yourself. > Ok, I'll switch over completely to the iommu api for this. It will also address Robin's concern. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: MIME-Version: 1.0 In-Reply-To: <20171011115410.GF30803@8bytes.org> References: <150764693502.16882.15848797003793552156.stgit@dwillia2-desk3.amr.corp.intel.com> <150764701194.16882.9682569707416653741.stgit@dwillia2-desk3.amr.corp.intel.com> <20171011115410.GF30803@8bytes.org> From: Dan Williams Date: Wed, 11 Oct 2017 09:01:25 -0700 Message-ID: Subject: Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings To: Joerg Roedel Cc: "linux-nvdimm@lists.01.org" , Sean Hefty , linux-xfs@vger.kernel.org, Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma@vger.kernel.org, Linux API , Dave Chinner , Jeff Moyer , iommu@lists.linux-foundation.org, Christoph Hellwig , "J. Bruce Fields" , linux-mm , Doug Ledford , Ross Zwisler , linux-fsdevel , Jeff Layton , David Woodhouse , Hal Rosenstock Content-Type: text/plain; charset="UTF-8" Sender: owner-linux-mm@kvack.org List-ID: On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel wrote: > On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote: >> +static void ib_umem_lease_break(void *__umem) >> +{ >> + struct ib_umem *umem = umem; >> + struct ib_device *idev = umem->context->device; >> + struct device *dev = idev->dma_device; >> + struct scatterlist *sgl = umem->sg_head.sgl; >> + >> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK, >> + iommu_sg_num_pages(dev, sgl, umem->npages)); >> +} > > This looks like an invitation to break your code by random iommu-driver > changes. There is no guarantee that an iommu-backed dma-api > implemenation will map exactly iommu_sg_num_pages() pages for a given > sg-list. In other words, you are mixing the use of the IOMMU-API and the > DMA-API in an incompatible way that only works because you know the > internals of the iommu-drivers. > > I've seen in another patch that your changes strictly require an IOMMU, > so you what you should do instead is to switch from the DMA-API to the > IOMMU-API and do the address-space management yourself. > Ok, I'll switch over completely to the iommu api for this. It will also address Robin's concern. -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-io0-f195.google.com ([209.85.223.195]:35868 "EHLO mail-io0-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751147AbdJKQB1 (ORCPT ); Wed, 11 Oct 2017 12:01:27 -0400 MIME-Version: 1.0 In-Reply-To: <20171011115410.GF30803@8bytes.org> References: <150764693502.16882.15848797003793552156.stgit@dwillia2-desk3.amr.corp.intel.com> <150764701194.16882.9682569707416653741.stgit@dwillia2-desk3.amr.corp.intel.com> <20171011115410.GF30803@8bytes.org> From: Dan Williams Date: Wed, 11 Oct 2017 09:01:25 -0700 Message-ID: Subject: Re: [PATCH v8 13/14] IB/core: use MAP_DIRECT to fix / enable RDMA to DAX mappings Content-Type: text/plain; charset="UTF-8" Sender: linux-xfs-owner@vger.kernel.org List-ID: List-Id: xfs To: Joerg Roedel Cc: "linux-nvdimm@lists.01.org" , Sean Hefty , linux-xfs@vger.kernel.org, Jan Kara , Ashok Raj , "Darrick J. Wong" , linux-rdma@vger.kernel.org, Linux API , Dave Chinner , Jeff Moyer , iommu@lists.linux-foundation.org, Christoph Hellwig , "J. Bruce Fields" , linux-mm , Doug Ledford , Ross Zwisler , linux-fsdevel , Jeff Layton , David Woodhouse , Hal Rosenstock On Wed, Oct 11, 2017 at 4:54 AM, Joerg Roedel wrote: > On Tue, Oct 10, 2017 at 07:50:12AM -0700, Dan Williams wrote: >> +static void ib_umem_lease_break(void *__umem) >> +{ >> + struct ib_umem *umem = umem; >> + struct ib_device *idev = umem->context->device; >> + struct device *dev = idev->dma_device; >> + struct scatterlist *sgl = umem->sg_head.sgl; >> + >> + iommu_unmap(umem->iommu, sg_dma_address(sgl) & PAGE_MASK, >> + iommu_sg_num_pages(dev, sgl, umem->npages)); >> +} > > This looks like an invitation to break your code by random iommu-driver > changes. There is no guarantee that an iommu-backed dma-api > implemenation will map exactly iommu_sg_num_pages() pages for a given > sg-list. In other words, you are mixing the use of the IOMMU-API and the > DMA-API in an incompatible way that only works because you know the > internals of the iommu-drivers. > > I've seen in another patch that your changes strictly require an IOMMU, > so you what you should do instead is to switch from the DMA-API to the > IOMMU-API and do the address-space management yourself. > Ok, I'll switch over completely to the iommu api for this. It will also address Robin's concern.