From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7738BC4363A for ; Fri, 23 Oct 2020 16:26:00 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 2D09B2192A for ; Fri, 23 Oct 2020 16:26:00 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750466AbgJWQZ7 (ORCPT ); Fri, 23 Oct 2020 12:25:59 -0400 Received: from mga07.intel.com ([134.134.136.100]:55036 "EHLO mga07.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750451AbgJWQZ7 (ORCPT ); Fri, 23 Oct 2020 12:25:59 -0400 IronPort-SDR: 0SpYtW6BE9hYiopanq7e9NB1UC0S9V1+sgQjmo8qG0VNWk3D+Yg3H5hVOSa7dXQa7ecEwGAhjO uL0t1TjV+4zw== X-IronPort-AV: E=McAfee;i="6000,8403,9782"; a="231890332" X-IronPort-AV: E=Sophos;i="5.77,409,1596524400"; d="scan'208";a="231890332" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Oct 2020 09:25:56 -0700 IronPort-SDR: r4R4V1dbGXOBYxncuCr6l/qY0eYvr7ohFtBiPZ9GpHKskxlbwUPEBDM+yyXGnqHWkIpMeZ1l9X OcjRumdg3j1w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,409,1596524400"; d="scan'208";a="349257122" Received: from cst-dev.jf.intel.com ([10.23.221.69]) by fmsmga004.fm.intel.com with ESMTP; 23 Oct 2020 09:25:56 -0700 From: Jianxin Xiong To: linux-rdma@vger.kernel.org, dri-devel@lists.freedesktop.org Cc: Jianxin Xiong , Doug Ledford , Jason Gunthorpe , Leon Romanovsky , Sumit Semwal , Christian Koenig , Daniel Vetter Subject: [PATCH v6 0/4] RDMA: Add dma-buf support Date: Fri, 23 Oct 2020 09:39:57 -0700 Message-Id: <1603471201-32588-1-git-send-email-jianxin.xiong@intel.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org This is the sixth version of the patch set. Changelog: v6: * Move the dma-buf invalidation callback from the core to the device driver * Move mapping update from work queue to pagefault handler * Add dma-buf based MRs to the xarray of mmkeys so that the pagefault handler can be reached * Update the new driver method and uverbs command signature by changing the paramter 'addr' to 'offset' * Modify the sg list returned from dma_buf_map_attachment() based on the parameters 'offset' and 'length' * Don't import dma-buf if 'dma_virt_ops' is used by the dma device * The patch that clarifies dma-buf sg lists alignment has landed at https://cgit.freedesktop.org/drm/drm-misc/commit/?id=ac80cd17a615 and thus is no longer included with this set v5: https://www.spinics.net/lists/linux-rdma/msg96786.html * Fix a few warnings reported by kernel test robot: - no previous prototype for function 'ib_umem_dmabuf_release' - no previous prototype for function 'ib_umem_dmabuf_map_pages' - comparison of distinct pointer types in 'check_add_overflow' * Add comment for the wait between getting the dma-buf sg tagle and updating the NIC page table v4: https://www.spinics.net/lists/linux-rdma/msg96767.html * Add a new ib_device method reg_user_mr_dmabuf() instead of expanding the existing method reg_user_mr() * Use a separate code flow for dma-buf instead of adding special cases to the ODP memory region code path * In invalidation callback, new mapping is updated as whole using work queue instead of being updated in page granularity in the page fault handler * Use dma_resv_get_excl() and dma_fence_wait() to ensure the content of the pages have been moved to the new location before the new mapping is programmed into the NIC * Add code to the ODP page fault handler to check the mapping status * The new access flag added in v3 is removed. * The checking for on-demand paging support in the new uverbs command is removed because it is implied by implementing the new ib_device method * Clarify that dma-buf sg lists are page aligned v3: https://www.spinics.net/lists/linux-rdma/msg96330.html * Use dma_buf_dynamic_attach() instead of dma_buf_attach() * Use on-demand paging mechanism to avoid pinning the GPU memory * Instead of adding a new parameter to the device method for memory registration, pass all the attributes including the file descriptor as a structure * Define a new access flag for dma-buf based memory region * Check for on-demand paging support in the new uverbs command v2: https://www.spinics.net/lists/linux-rdma/msg93643.html * The Kconfig option is removed. There is no dependence issue since dma-buf driver is always enabled. * The declaration of new data structure and functions is reorganized to minimize the visibility of the changes. * The new uverbs command now goes through ioctl() instead of write(). * The rereg functionality is removed. * Instead of adding new device method for dma-buf specific registration, existing method is extended to accept an extra parameter. * The correct function is now used for address range checking. v1: https://www.spinics.net/lists/linux-rdma/msg90720.html * The initial patch set * Implement core functions for importing and mapping dma-buf * Use dma-buf static attach interface * Add two ib_device methods reg_user_mr_fd() and rereg_user_mr_fd() * Add two uverbs commands via the write() interface * Add Kconfig option * Add dma-buf support to mlx5 device When enabled, an RDMA capable NIC can perform peer-to-peer transactions over PCIe to access the local memory located on another device. This can often lead to better performance than using a system memory buffer for RDMA and copying data between the buffer and device memory. Current kernel RDMA stack uses get_user_pages() to pin the physical pages backing the user buffer and uses dma_map_sg_attrs() to get the dma addresses for memory access. This usually doesn't work for peer device memory due to the lack of associated page structures. Several mechanisms exist today to facilitate device memory access. ZONE_DEVICE is a new zone for device memory in the memory management subsystem. It allows pages from device memory being described with specialized page structures, but what can be done with these page structures may be different from system memory. ZONE_DEVICE is further specialized into multiple memory types, such as one type for PCI p2pmem/p2pdma and one type for HMM. PCI p2pmem/p2pdma uses ZONE_DEVICE to represent device memory residing in a PCI BAR and provides a set of calls to publish, discover, allocate, and map such memory for peer-to-peer transactions. One feature of the API is that the buffer is allocated by the side that does the DMA transfer. This works well with the storage usage case, but is awkward with GPU-NIC communication, where typically the buffer is allocated by the GPU driver rather than the NIC driver. Heterogeneous Memory Management (HMM) utilizes mmu_interval_notifier and ZONE_DEVICE to support shared virtual address space and page migration between system memory and device memory. HMM doesn't support pinning device memory because pages located on device must be able to migrate to system memory when accessed by CPU. Peer-to-peer access is currently not supported by HMM. Dma-buf is a standard mechanism for sharing buffers among different device drivers. The buffer to be shared is exported by the owning driver and imported by the driver that wants to use it. The exporter provides a set of ops that the importer can call to pin and map the buffer. In addition, a file descriptor can be associated with a dma- buf object as the handle that can be passed to user space. This patch series adds dma-buf importer role to the RDMA driver in attempt to support RDMA using device memory such as GPU VRAM. Dma-buf is chosen for a few reasons: first, the API is relatively simple and allows a lot of flexibility in implementing the buffer manipulation ops. Second, it doesn't require page structure. Third, dma-buf is already supported in many GPU drivers. However, we are aware that existing GPU drivers don't allow pinning device memory via the dma-buf interface. Pinning would simply cause the backing storage to migrate to system RAM. True peer-to-peer access is only possible using dynamic attach, which requires on-demand paging support from the NIC to work. For this reason, this series only works with ODP capable NICs. This series consists of five patches. The first patch adds the common code for importing dma-buf from a file descriptor and mapping the dma-buf pages. Patch 2 add the new driver method reg_user_mr_dmabuf(). Patch 3 adds a new uverbs command for registering dma-buf based memory region. Patch 4 adds dma-buf support to the mlx5 driver. Patch 5 adds clarification to the dma-buf API documentation that dma-buf sg lists are page aligned. Related user space RDMA library changes will be provided as a separate patch series. Jianxin Xiong (4): RDMA/umem: Support importing dma-buf as user memory region RDMA/core: Add device method for registering dma-buf base memory region RDMA/uverbs: Add uverbs command for dma-buf based MR registration RDMA/mlx5: Support dma-buf based userspace memory region drivers/infiniband/core/Makefile | 2 +- drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/umem.c | 4 + drivers/infiniband/core/umem_dmabuf.c | 197 ++++++++++++++++++++++++++ drivers/infiniband/core/umem_dmabuf.h | 11 ++ drivers/infiniband/core/uverbs_std_types_mr.c | 112 +++++++++++++++ drivers/infiniband/hw/mlx5/main.c | 2 + drivers/infiniband/hw/mlx5/mlx5_ib.h | 11 ++ drivers/infiniband/hw/mlx5/mr.c | 114 ++++++++++++++- drivers/infiniband/hw/mlx5/odp.c | 70 ++++++++- include/rdma/ib_umem.h | 35 ++++- include/rdma/ib_verbs.h | 6 +- include/uapi/rdma/ib_user_ioctl_cmds.h | 14 ++ 13 files changed, 570 insertions(+), 9 deletions(-) create mode 100644 drivers/infiniband/core/umem_dmabuf.c create mode 100644 drivers/infiniband/core/umem_dmabuf.h -- 1.8.3.1 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-11.7 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING, SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED,USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B8BCC55179 for ; Fri, 23 Oct 2020 16:26:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0213B2192A for ; Fri, 23 Oct 2020 16:26:24 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0213B2192A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BBF6C6E2B4; Fri, 23 Oct 2020 16:26:23 +0000 (UTC) Received: from mga06.intel.com (mga06.intel.com [134.134.136.31]) by gabe.freedesktop.org (Postfix) with ESMTPS id A52D86E25B for ; Fri, 23 Oct 2020 16:26:17 +0000 (UTC) IronPort-SDR: bNgsXIv/rXEjZvgbOvbXM8YkRt4tcI1wVjizad8rREVvUEzlLGbY2jQlA37+ug3ZirzApB+AOM 80SWz4EA6VjQ== X-IronPort-AV: E=McAfee;i="6000,8403,9782"; a="229333773" X-IronPort-AV: E=Sophos;i="5.77,409,1596524400"; d="scan'208";a="229333773" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Oct 2020 09:25:56 -0700 IronPort-SDR: r4R4V1dbGXOBYxncuCr6l/qY0eYvr7ohFtBiPZ9GpHKskxlbwUPEBDM+yyXGnqHWkIpMeZ1l9X OcjRumdg3j1w== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.77,409,1596524400"; d="scan'208";a="349257122" Received: from cst-dev.jf.intel.com ([10.23.221.69]) by fmsmga004.fm.intel.com with ESMTP; 23 Oct 2020 09:25:56 -0700 From: Jianxin Xiong To: linux-rdma@vger.kernel.org, dri-devel@lists.freedesktop.org Subject: [PATCH v6 0/4] RDMA: Add dma-buf support Date: Fri, 23 Oct 2020 09:39:57 -0700 Message-Id: <1603471201-32588-1-git-send-email-jianxin.xiong@intel.com> X-Mailer: git-send-email 1.8.3.1 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Leon Romanovsky , Jason Gunthorpe , Doug Ledford , Daniel Vetter , Christian Koenig , Jianxin Xiong MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is the sixth version of the patch set. Changelog: v6: * Move the dma-buf invalidation callback from the core to the device driver * Move mapping update from work queue to pagefault handler * Add dma-buf based MRs to the xarray of mmkeys so that the pagefault handler can be reached * Update the new driver method and uverbs command signature by changing the paramter 'addr' to 'offset' * Modify the sg list returned from dma_buf_map_attachment() based on the parameters 'offset' and 'length' * Don't import dma-buf if 'dma_virt_ops' is used by the dma device * The patch that clarifies dma-buf sg lists alignment has landed at https://cgit.freedesktop.org/drm/drm-misc/commit/?id=ac80cd17a615 and thus is no longer included with this set v5: https://www.spinics.net/lists/linux-rdma/msg96786.html * Fix a few warnings reported by kernel test robot: - no previous prototype for function 'ib_umem_dmabuf_release' - no previous prototype for function 'ib_umem_dmabuf_map_pages' - comparison of distinct pointer types in 'check_add_overflow' * Add comment for the wait between getting the dma-buf sg tagle and updating the NIC page table v4: https://www.spinics.net/lists/linux-rdma/msg96767.html * Add a new ib_device method reg_user_mr_dmabuf() instead of expanding the existing method reg_user_mr() * Use a separate code flow for dma-buf instead of adding special cases to the ODP memory region code path * In invalidation callback, new mapping is updated as whole using work queue instead of being updated in page granularity in the page fault handler * Use dma_resv_get_excl() and dma_fence_wait() to ensure the content of the pages have been moved to the new location before the new mapping is programmed into the NIC * Add code to the ODP page fault handler to check the mapping status * The new access flag added in v3 is removed. * The checking for on-demand paging support in the new uverbs command is removed because it is implied by implementing the new ib_device method * Clarify that dma-buf sg lists are page aligned v3: https://www.spinics.net/lists/linux-rdma/msg96330.html * Use dma_buf_dynamic_attach() instead of dma_buf_attach() * Use on-demand paging mechanism to avoid pinning the GPU memory * Instead of adding a new parameter to the device method for memory registration, pass all the attributes including the file descriptor as a structure * Define a new access flag for dma-buf based memory region * Check for on-demand paging support in the new uverbs command v2: https://www.spinics.net/lists/linux-rdma/msg93643.html * The Kconfig option is removed. There is no dependence issue since dma-buf driver is always enabled. * The declaration of new data structure and functions is reorganized to minimize the visibility of the changes. * The new uverbs command now goes through ioctl() instead of write(). * The rereg functionality is removed. * Instead of adding new device method for dma-buf specific registration, existing method is extended to accept an extra parameter. * The correct function is now used for address range checking. v1: https://www.spinics.net/lists/linux-rdma/msg90720.html * The initial patch set * Implement core functions for importing and mapping dma-buf * Use dma-buf static attach interface * Add two ib_device methods reg_user_mr_fd() and rereg_user_mr_fd() * Add two uverbs commands via the write() interface * Add Kconfig option * Add dma-buf support to mlx5 device When enabled, an RDMA capable NIC can perform peer-to-peer transactions over PCIe to access the local memory located on another device. This can often lead to better performance than using a system memory buffer for RDMA and copying data between the buffer and device memory. Current kernel RDMA stack uses get_user_pages() to pin the physical pages backing the user buffer and uses dma_map_sg_attrs() to get the dma addresses for memory access. This usually doesn't work for peer device memory due to the lack of associated page structures. Several mechanisms exist today to facilitate device memory access. ZONE_DEVICE is a new zone for device memory in the memory management subsystem. It allows pages from device memory being described with specialized page structures, but what can be done with these page structures may be different from system memory. ZONE_DEVICE is further specialized into multiple memory types, such as one type for PCI p2pmem/p2pdma and one type for HMM. PCI p2pmem/p2pdma uses ZONE_DEVICE to represent device memory residing in a PCI BAR and provides a set of calls to publish, discover, allocate, and map such memory for peer-to-peer transactions. One feature of the API is that the buffer is allocated by the side that does the DMA transfer. This works well with the storage usage case, but is awkward with GPU-NIC communication, where typically the buffer is allocated by the GPU driver rather than the NIC driver. Heterogeneous Memory Management (HMM) utilizes mmu_interval_notifier and ZONE_DEVICE to support shared virtual address space and page migration between system memory and device memory. HMM doesn't support pinning device memory because pages located on device must be able to migrate to system memory when accessed by CPU. Peer-to-peer access is currently not supported by HMM. Dma-buf is a standard mechanism for sharing buffers among different device drivers. The buffer to be shared is exported by the owning driver and imported by the driver that wants to use it. The exporter provides a set of ops that the importer can call to pin and map the buffer. In addition, a file descriptor can be associated with a dma- buf object as the handle that can be passed to user space. This patch series adds dma-buf importer role to the RDMA driver in attempt to support RDMA using device memory such as GPU VRAM. Dma-buf is chosen for a few reasons: first, the API is relatively simple and allows a lot of flexibility in implementing the buffer manipulation ops. Second, it doesn't require page structure. Third, dma-buf is already supported in many GPU drivers. However, we are aware that existing GPU drivers don't allow pinning device memory via the dma-buf interface. Pinning would simply cause the backing storage to migrate to system RAM. True peer-to-peer access is only possible using dynamic attach, which requires on-demand paging support from the NIC to work. For this reason, this series only works with ODP capable NICs. This series consists of five patches. The first patch adds the common code for importing dma-buf from a file descriptor and mapping the dma-buf pages. Patch 2 add the new driver method reg_user_mr_dmabuf(). Patch 3 adds a new uverbs command for registering dma-buf based memory region. Patch 4 adds dma-buf support to the mlx5 driver. Patch 5 adds clarification to the dma-buf API documentation that dma-buf sg lists are page aligned. Related user space RDMA library changes will be provided as a separate patch series. Jianxin Xiong (4): RDMA/umem: Support importing dma-buf as user memory region RDMA/core: Add device method for registering dma-buf base memory region RDMA/uverbs: Add uverbs command for dma-buf based MR registration RDMA/mlx5: Support dma-buf based userspace memory region drivers/infiniband/core/Makefile | 2 +- drivers/infiniband/core/device.c | 1 + drivers/infiniband/core/umem.c | 4 + drivers/infiniband/core/umem_dmabuf.c | 197 ++++++++++++++++++++++++++ drivers/infiniband/core/umem_dmabuf.h | 11 ++ drivers/infiniband/core/uverbs_std_types_mr.c | 112 +++++++++++++++ drivers/infiniband/hw/mlx5/main.c | 2 + drivers/infiniband/hw/mlx5/mlx5_ib.h | 11 ++ drivers/infiniband/hw/mlx5/mr.c | 114 ++++++++++++++- drivers/infiniband/hw/mlx5/odp.c | 70 ++++++++- include/rdma/ib_umem.h | 35 ++++- include/rdma/ib_verbs.h | 6 +- include/uapi/rdma/ib_user_ioctl_cmds.h | 14 ++ 13 files changed, 570 insertions(+), 9 deletions(-) create mode 100644 drivers/infiniband/core/umem_dmabuf.c create mode 100644 drivers/infiniband/core/umem_dmabuf.h -- 1.8.3.1 _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel