From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755456AbcKOGdQ (ORCPT ); Tue, 15 Nov 2016 01:33:16 -0500 Received: from hqemgate16.nvidia.com ([216.228.121.65]:17352 "EHLO hqemgate16.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755204AbcKOGdO (ORCPT ); Tue, 15 Nov 2016 01:33:14 -0500 X-PGP-Universal: processed; by hqnvupgp07.nvidia.com on Mon, 14 Nov 2016 10:32:16 -0800 Subject: Re: [PATCH v11 10/22] vfio iommu type1: Add support for mediated devices To: Alexey Kardashevskiy , , , , References: <1478293856-8191-1-git-send-email-kwankhede@nvidia.com> <1478293856-8191-11-git-send-email-kwankhede@nvidia.com> <93c0ace9-bf1f-3d94-56ab-3513f7183a1b@ozlabs.ru> CC: , , , , , X-Nvconfidentiality: public From: Kirti Wankhede Message-ID: <5219635a-011b-ed6a-9535-867085b072b0@nvidia.com> Date: Tue, 15 Nov 2016 12:03:00 +0530 MIME-Version: 1.0 In-Reply-To: X-Originating-IP: [10.24.216.210] X-ClientProxiedBy: DRBGMAIL101.nvidia.com (10.18.16.20) To bgmail102.nvidia.com (10.25.59.11) Content-Type: text/plain; charset="koi8-r" Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 11/15/2016 10:47 AM, Alexey Kardashevskiy wrote: > On 08/11/16 17:52, Alexey Kardashevskiy wrote: >> On 05/11/16 08:10, Kirti Wankhede wrote: >>> VFIO IOMMU drivers are designed for the devices which are IOMMU capable. >>> Mediated device only uses IOMMU APIs, the underlying hardware can be >>> managed by an IOMMU domain. >>> >>> Aim of this change is: >>> - To use most of the code of TYPE1 IOMMU driver for mediated devices >>> - To support direct assigned device and mediated device in single module >>> >>> This change adds pin and unpin support for mediated device to TYPE1 IOMMU >>> backend module. More details: >>> - vfio_pin_pages() callback here uses task and address space of vfio_dma, >>> that is, of the process who mapped that iova range. >>> - Added pfn_list tracking logic to address space structure. All pages >>> pinned through this interface are trached in its address space. >>> - Pinned pages list is used to verify unpinning request and to unpin >>> remaining pages while detaching the group for that device. >>> - Page accounting is updated to account in its address space where the >>> pages are pinned/unpinned. >>> - Accouting for mdev device is only done if there is no iommu capable >>> domain in the container. When there is a direct device assigned to the >>> container and that domain is iommu capable, all pages are already pinned >>> during DMA_MAP. >>> - Page accouting is updated on hot plug and unplug mdev device and pass >>> through device. >>> >>> Tested by assigning below combinations of devices to a single VM: >>> - GPU pass through only >> >> This does not require this patchset, right? >> Sorry I missed this earlier. This testing is required for this patch, because this patch touches code that is used for direct device assignment. Also for page accounting, all cases are considered i.e. when there is only pass through device in a container, when there is pass through device + vGPU device in a container. Also have to test that pages are pinned properly when device is hotplugged. In that case vfio_iommu_replay() is called to take necessary action. >>> - vGPU device only >> >> Out of curiosity - how exactly did you test this? The exact GPU, how to >> create vGPU, what was the QEMU command line and the guest does with this >> passed device? Thanks. > > ping? > I'm testing this code with M60, with custom changes in our driver. Steps how to create mediated device are listed in Documentation/vfio-mediated-device.txt for sample mtty driver. Same steps I'm following for GPU. Quoting those steps here for you: 2. Create a mediated device by using the dummy device that you created in the previous step. # echo "83b8f4f2-509f-382f-3c1e-e6bfe0fa1001" > \ /sys/devices/virtual/mtty/mtty/mdev_supported_types/mtty-2/create 3. Add parameters to qemu-kvm. -device vfio-pci,\ sysfsdev=/sys/bus/mdev/devices/83b8f4f2-509f-382f-3c1e-e6bfe0fa1001 Thanks, Kirti