From mboxrd@z Thu Jan 1 00:00:00 1970 From: aafabbri Subject: Re: kvm PCI assignment & VFIO ramblings Date: Mon, 22 Aug 2011 17:52:18 -0700 Message-ID: References: <1314049785.7662.44.camel@pasglop> Mime-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Cc: Alex Williamson , Avi Kivity , Alexey Kardashevskiy , , Paul Mackerras , qemu-devel , chrisw , iommu , Anthony Liguori , "linux-pci@vger.kernel.org" , linuxppc-dev , To: Benjamin Herrenschmidt Return-path: In-Reply-To: <1314049785.7662.44.camel@pasglop> Sender: linux-pci-owner@vger.kernel.org List-Id: kvm.vger.kernel.org On 8/22/11 2:49 PM, "Benjamin Herrenschmidt" wrote: > >>> I wouldn't use uiommu for that. >> >> Any particular reason besides saving a file descriptor? >> >> We use it today, and it seems like a cleaner API than what you propose >> changing it to. > > Well for one, we are back to square one vs. grouping constraints. I'm not following you. You have to enforce group/iommu domain assignment whether you have the existing uiommu API, or if you change it to your proposed ioctl(inherit_iommu) API. The only change needed to VFIO here should be to make uiommu fd assignment happen on the groups instead of on device fds. That operation fails or succeeds according to the group semantics (all-or-none assignment/same uiommu). I think the question is: do we force 1:1 iommu/group mapping, or do we allow arbitrary mapping (satisfying group constraints) as we do today. I'm saying I'm an existing user who wants the arbitrary iommu/group mapping ability and definitely think the uiommu approach is cleaner than the ioctl(inherit_iommu) approach. We considered that approach before but it seemed less clean so we went with the explicit uiommu context. > .../... > >> If we in singleton-group land were building our own "groups" which were sets >> of devices sharing the IOMMU domains we wanted, I suppose we could do away >> with uiommu fds, but it sounds like the current proposal would create 20 >> singleton groups (x86 iommu w/o PCI bridges => all devices are partitionable >> endpoints). Asking me to ioctl(inherit) them together into a blob sounds >> worse than the current explicit uiommu API. > > I'd rather have an API to create super-groups (groups of groups) > statically and then you can use such groups as normal groups using the > same interface. That create/management process could be done via a > simple command line utility or via sysfs banging, whatever... From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from rcdn-iport-7.cisco.com (rcdn-iport-7.cisco.com [173.37.86.78]) (using TLSv1 with cipher RC4-SHA (128/128 bits)) (Client CN "rcdn-iport-7.cisco.com", Issuer "Cisco SSCA2" (verified OK)) by ozlabs.org (Postfix) with ESMTPS id 55D79B6F86 for ; Tue, 23 Aug 2011 10:52:23 +1000 (EST) Date: Mon, 22 Aug 2011 17:52:18 -0700 Subject: Re: kvm PCI assignment & VFIO ramblings From: aafabbri To: Benjamin Herrenschmidt Message-ID: In-Reply-To: <1314049785.7662.44.camel@pasglop> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Cc: Alexey Kardashevskiy , kvm@vger.kernel.org, Paul Mackerras , "linux-pci@vger.kernel.org" , qemu-devel , iommu , chrisw , Alex Williamson , Avi Kivity , Anthony Liguori , linuxppc-dev , benve@cisco.com List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On 8/22/11 2:49 PM, "Benjamin Herrenschmidt" wrote: > >>> I wouldn't use uiommu for that. >> >> Any particular reason besides saving a file descriptor? >> >> We use it today, and it seems like a cleaner API than what you propose >> changing it to. > > Well for one, we are back to square one vs. grouping constraints. I'm not following you. You have to enforce group/iommu domain assignment whether you have the existing uiommu API, or if you change it to your proposed ioctl(inherit_iommu) API. The only change needed to VFIO here should be to make uiommu fd assignment happen on the groups instead of on device fds. That operation fails or succeeds according to the group semantics (all-or-none assignment/same uiommu). I think the question is: do we force 1:1 iommu/group mapping, or do we allow arbitrary mapping (satisfying group constraints) as we do today. I'm saying I'm an existing user who wants the arbitrary iommu/group mapping ability and definitely think the uiommu approach is cleaner than the ioctl(inherit_iommu) approach. We considered that approach before but it seemed less clean so we went with the explicit uiommu context. > .../... > >> If we in singleton-group land were building our own "groups" which were sets >> of devices sharing the IOMMU domains we wanted, I suppose we could do away >> with uiommu fds, but it sounds like the current proposal would create 20 >> singleton groups (x86 iommu w/o PCI bridges => all devices are partitionable >> endpoints). Asking me to ioctl(inherit) them together into a blob sounds >> worse than the current explicit uiommu API. > > I'd rather have an API to create super-groups (groups of groups) > statically and then you can use such groups as normal groups using the > same interface. That create/management process could be done via a > simple command line utility or via sysfs banging, whatever... From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:52589) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QvfE9-0004kA-Fo for qemu-devel@nongnu.org; Mon, 22 Aug 2011 20:52:26 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1QvfE7-0006db-F8 for qemu-devel@nongnu.org; Mon, 22 Aug 2011 20:52:25 -0400 Received: from rcdn-iport-7.cisco.com ([173.37.86.78]:61382) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1QvfE7-0006dR-22 for qemu-devel@nongnu.org; Mon, 22 Aug 2011 20:52:23 -0400 Date: Mon, 22 Aug 2011 17:52:18 -0700 From: aafabbri Message-ID: In-Reply-To: <1314049785.7662.44.camel@pasglop> Mime-version: 1.0 Content-type: text/plain; charset="US-ASCII" Content-transfer-encoding: 7bit Subject: Re: [Qemu-devel] kvm PCI assignment & VFIO ramblings List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Benjamin Herrenschmidt Cc: Alexey Kardashevskiy , kvm@vger.kernel.org, Paul Mackerras , "linux-pci@vger.kernel.org" , qemu-devel , iommu , chrisw , Alex Williamson , Avi Kivity , linuxppc-dev , benve@cisco.com On 8/22/11 2:49 PM, "Benjamin Herrenschmidt" wrote: > >>> I wouldn't use uiommu for that. >> >> Any particular reason besides saving a file descriptor? >> >> We use it today, and it seems like a cleaner API than what you propose >> changing it to. > > Well for one, we are back to square one vs. grouping constraints. I'm not following you. You have to enforce group/iommu domain assignment whether you have the existing uiommu API, or if you change it to your proposed ioctl(inherit_iommu) API. The only change needed to VFIO here should be to make uiommu fd assignment happen on the groups instead of on device fds. That operation fails or succeeds according to the group semantics (all-or-none assignment/same uiommu). I think the question is: do we force 1:1 iommu/group mapping, or do we allow arbitrary mapping (satisfying group constraints) as we do today. I'm saying I'm an existing user who wants the arbitrary iommu/group mapping ability and definitely think the uiommu approach is cleaner than the ioctl(inherit_iommu) approach. We considered that approach before but it seemed less clean so we went with the explicit uiommu context. > .../... > >> If we in singleton-group land were building our own "groups" which were sets >> of devices sharing the IOMMU domains we wanted, I suppose we could do away >> with uiommu fds, but it sounds like the current proposal would create 20 >> singleton groups (x86 iommu w/o PCI bridges => all devices are partitionable >> endpoints). Asking me to ioctl(inherit) them together into a blob sounds >> worse than the current explicit uiommu API. > > I'd rather have an API to create super-groups (groups of groups) > statically and then you can use such groups as normal groups using the > same interface. That create/management process could be done via a > simple command line utility or via sysfs banging, whatever...