From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753899AbcKNPlW (ORCPT ); Mon, 14 Nov 2016 10:41:22 -0500 Received: from hqemgate14.nvidia.com ([216.228.121.143]:14753 "EHLO hqemgate14.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753292AbcKNPlU (ORCPT ); Mon, 14 Nov 2016 10:41:20 -0500 X-PGP-Universal: processed; by hqnvupgp07.nvidia.com on Sun, 13 Nov 2016 19:40:24 -0800 From: Kirti Wankhede To: , , , CC: , , , , , , Kirti Wankhede Subject: [PATCH v12 00/22] Add Mediated device support Date: Mon, 14 Nov 2016 21:12:14 +0530 Message-ID: <1479138156-28905-1-git-send-email-kwankhede@nvidia.com> X-Mailer: git-send-email 2.7.0 X-NVConfidentiality: public MIME-Version: 1.0 Content-Type: text/plain Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This series adds Mediated device support to Linux host kernel. Purpose of this series is to provide a common interface for mediated device management that can be used by different devices. This series introduces Mdev core module that creates and manages mediated devices, VFIO based driver for mediated devices that are created by mdev core module and update VFIO type1 IOMMU module to support pinning & unpinning for mediated devices. What changed v11-> v12? vfio_iommu_type1: - Pinned pages are tracked in vfio_dma structure for that range, dma->iova to dma->iova + dma->size. - Key for pfn_list rb-tree is iova, instead of pfn. - Removing address space structure. vfio_dma keeps task structure, which is used to get mm structure (using get_task_mm(task) and mmput(mm)) for pin/unpin and page accounting. - vfio_unpin_pages() has array of user_pfns as input argument, instead of array of pfns. - On vfio_pin_pages(), pinning happens once. On later call to vfio_pin_pages() with same user_pfn, if iova is found in pfn_list, only ref_count is incremented. - In vfio_unpin_pages(), ref_count is decremented and page is unpinned when ref_count is 0. - For vfio_pin_pages() and vfio_unpin_pages() input array, number of elements in array should be less that PAGE_SIZE. If vendor driver wants to use for more pages, array should be split it in chunks of PAGE_SIZE. - Updating page accounting logic with above changes. Tested by assigning below combinations of devices to a single VM: - GPU pass through only - vGPU device only - One GPU pass through and one vGPU device - Linux VM hot plug and unplug vGPU device while GPU pass through device exist - Linux VM hot plug and unplug GPU pass through device while vGPU device exist Tested with Linux 4.9-rc5 Kirti Wankhede (22): vfio: Mediated device Core driver vfio: VFIO based driver for Mediated devices vfio: Rearrange functions to get vfio_group from dev vfio: Common function to increment container_users vfio iommu: Added pin and unpin callback functions to vfio_iommu_driver_ops vfio iommu type1: Update arguments of vfio_lock_acct vfio iommu type1: Update argument of vaddr_get_pfn() vfio iommu type1: Add find_iommu_group() function vfio iommu type1: Add task structure to vfio_dma vfio iommu type1: Add support for mediated devices vfio iommu: Add blocking notifier to notify DMA_UNMAP vfio: Add notifier callback to parent's ops structure of mdev vfio: Introduce common function to add capabilities vfio_pci: Update vfio_pci to use vfio_info_add_capability() vfio: Introduce vfio_set_irqs_validate_and_prepare() vfio_pci: Updated to use vfio_set_irqs_validate_and_prepare() vfio_platform: Updated to use vfio_set_irqs_validate_and_prepare() vfio: Define device_api strings docs: Add Documentation for Mediated devices docs: Sysfs ABI for mediated device framework docs: Sample driver to demonstrate how to use Mediated device framework. MAINTAINERS: Add entry VFIO based Mediated device drivers Documentation/ABI/testing/sysfs-bus-vfio-mdev | 111 ++ Documentation/vfio-mediated-device.txt | 399 +++++++ MAINTAINERS | 9 + drivers/vfio/Kconfig | 1 + drivers/vfio/Makefile | 1 + drivers/vfio/mdev/Kconfig | 17 + drivers/vfio/mdev/Makefile | 5 + drivers/vfio/mdev/mdev_core.c | 388 +++++++ drivers/vfio/mdev/mdev_driver.c | 120 ++ drivers/vfio/mdev/mdev_private.h | 41 + drivers/vfio/mdev/mdev_sysfs.c | 286 +++++ drivers/vfio/mdev/vfio_mdev.c | 167 +++ drivers/vfio/pci/vfio_pci.c | 83 +- drivers/vfio/platform/vfio_platform_common.c | 31 +- drivers/vfio/vfio.c | 341 +++++- drivers/vfio/vfio_iommu_type1.c | 831 +++++++++++--- include/linux/mdev.h | 177 +++ include/linux/vfio.h | 30 +- include/uapi/linux/vfio.h | 10 + samples/vfio-mdev/Makefile | 13 + samples/vfio-mdev/mtty.c | 1503 +++++++++++++++++++++++++ 21 files changed, 4328 insertions(+), 236 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-bus-vfio-mdev create mode 100644 Documentation/vfio-mediated-device.txt create mode 100644 drivers/vfio/mdev/Kconfig create mode 100644 drivers/vfio/mdev/Makefile create mode 100644 drivers/vfio/mdev/mdev_core.c create mode 100644 drivers/vfio/mdev/mdev_driver.c create mode 100644 drivers/vfio/mdev/mdev_private.h create mode 100644 drivers/vfio/mdev/mdev_sysfs.c create mode 100644 drivers/vfio/mdev/vfio_mdev.c create mode 100644 include/linux/mdev.h create mode 100644 samples/vfio-mdev/Makefile create mode 100644 samples/vfio-mdev/mtty.c -- 2.7.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34612) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1c6JNs-0002hR-Vb for qemu-devel@nongnu.org; Mon, 14 Nov 2016 10:41:26 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1c6JNp-0007Tj-S8 for qemu-devel@nongnu.org; Mon, 14 Nov 2016 10:41:25 -0500 Received: from hqemgate14.nvidia.com ([216.228.121.143]:14754) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1c6JNp-0007T8-Kf for qemu-devel@nongnu.org; Mon, 14 Nov 2016 10:41:21 -0500 From: Kirti Wankhede Date: Mon, 14 Nov 2016 21:12:14 +0530 Message-ID: <1479138156-28905-1-git-send-email-kwankhede@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain Subject: [Qemu-devel] [PATCH v12 00/22] Add Mediated device support List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: alex.williamson@redhat.com, pbonzini@redhat.com, kraxel@redhat.com, cjia@nvidia.com Cc: qemu-devel@nongnu.org, kvm@vger.kernel.org, kevin.tian@intel.com, jike.song@intel.com, bjsdjshi@linux.vnet.ibm.com, linux-kernel@vger.kernel.org, Kirti Wankhede This series adds Mediated device support to Linux host kernel. Purpose of this series is to provide a common interface for mediated device management that can be used by different devices. This series introduces Mdev core module that creates and manages mediated devices, VFIO based driver for mediated devices that are created by mdev core module and update VFIO type1 IOMMU module to support pinning & unpinning for mediated devices. What changed v11-> v12? vfio_iommu_type1: - Pinned pages are tracked in vfio_dma structure for that range, dma->iova to dma->iova + dma->size. - Key for pfn_list rb-tree is iova, instead of pfn. - Removing address space structure. vfio_dma keeps task structure, which is used to get mm structure (using get_task_mm(task) and mmput(mm)) for pin/unpin and page accounting. - vfio_unpin_pages() has array of user_pfns as input argument, instead of array of pfns. - On vfio_pin_pages(), pinning happens once. On later call to vfio_pin_pages() with same user_pfn, if iova is found in pfn_list, only ref_count is incremented. - In vfio_unpin_pages(), ref_count is decremented and page is unpinned when ref_count is 0. - For vfio_pin_pages() and vfio_unpin_pages() input array, number of elements in array should be less that PAGE_SIZE. If vendor driver wants to use for more pages, array should be split it in chunks of PAGE_SIZE. - Updating page accounting logic with above changes. Tested by assigning below combinations of devices to a single VM: - GPU pass through only - vGPU device only - One GPU pass through and one vGPU device - Linux VM hot plug and unplug vGPU device while GPU pass through device exist - Linux VM hot plug and unplug GPU pass through device while vGPU device exist Tested with Linux 4.9-rc5 Kirti Wankhede (22): vfio: Mediated device Core driver vfio: VFIO based driver for Mediated devices vfio: Rearrange functions to get vfio_group from dev vfio: Common function to increment container_users vfio iommu: Added pin and unpin callback functions to vfio_iommu_driver_ops vfio iommu type1: Update arguments of vfio_lock_acct vfio iommu type1: Update argument of vaddr_get_pfn() vfio iommu type1: Add find_iommu_group() function vfio iommu type1: Add task structure to vfio_dma vfio iommu type1: Add support for mediated devices vfio iommu: Add blocking notifier to notify DMA_UNMAP vfio: Add notifier callback to parent's ops structure of mdev vfio: Introduce common function to add capabilities vfio_pci: Update vfio_pci to use vfio_info_add_capability() vfio: Introduce vfio_set_irqs_validate_and_prepare() vfio_pci: Updated to use vfio_set_irqs_validate_and_prepare() vfio_platform: Updated to use vfio_set_irqs_validate_and_prepare() vfio: Define device_api strings docs: Add Documentation for Mediated devices docs: Sysfs ABI for mediated device framework docs: Sample driver to demonstrate how to use Mediated device framework. MAINTAINERS: Add entry VFIO based Mediated device drivers Documentation/ABI/testing/sysfs-bus-vfio-mdev | 111 ++ Documentation/vfio-mediated-device.txt | 399 +++++++ MAINTAINERS | 9 + drivers/vfio/Kconfig | 1 + drivers/vfio/Makefile | 1 + drivers/vfio/mdev/Kconfig | 17 + drivers/vfio/mdev/Makefile | 5 + drivers/vfio/mdev/mdev_core.c | 388 +++++++ drivers/vfio/mdev/mdev_driver.c | 120 ++ drivers/vfio/mdev/mdev_private.h | 41 + drivers/vfio/mdev/mdev_sysfs.c | 286 +++++ drivers/vfio/mdev/vfio_mdev.c | 167 +++ drivers/vfio/pci/vfio_pci.c | 83 +- drivers/vfio/platform/vfio_platform_common.c | 31 +- drivers/vfio/vfio.c | 341 +++++- drivers/vfio/vfio_iommu_type1.c | 831 +++++++++++--- include/linux/mdev.h | 177 +++ include/linux/vfio.h | 30 +- include/uapi/linux/vfio.h | 10 + samples/vfio-mdev/Makefile | 13 + samples/vfio-mdev/mtty.c | 1503 +++++++++++++++++++++++++ 21 files changed, 4328 insertions(+), 236 deletions(-) create mode 100644 Documentation/ABI/testing/sysfs-bus-vfio-mdev create mode 100644 Documentation/vfio-mediated-device.txt create mode 100644 drivers/vfio/mdev/Kconfig create mode 100644 drivers/vfio/mdev/Makefile create mode 100644 drivers/vfio/mdev/mdev_core.c create mode 100644 drivers/vfio/mdev/mdev_driver.c create mode 100644 drivers/vfio/mdev/mdev_private.h create mode 100644 drivers/vfio/mdev/mdev_sysfs.c create mode 100644 drivers/vfio/mdev/vfio_mdev.c create mode 100644 include/linux/mdev.h create mode 100644 samples/vfio-mdev/Makefile create mode 100644 samples/vfio-mdev/mtty.c -- 2.7.0