From mboxrd@z Thu Jan 1 00:00:00 1970 From: Benjamin Herrenschmidt Subject: Re: kvm PCI assignment & VFIO ramblings Date: Wed, 24 Aug 2011 09:41:32 +1000 Message-ID: <1314142892.30478.64.camel@pasglop> References: <20110802082848.GD29719@yookeroo.fritz.box> <1312308847.2653.467.camel@bling.home> <1312310121.2653.470.camel@bling.home> <20110803020422.GF29719@yookeroo.fritz.box> <4E3F9E33.5000706@redhat.com> <1312932258.4524.55.camel@bling.home> <1312944513.29273.28.camel@pasglop> <1313859105.6866.192.camel@x201.home> <20110822055509.GI30097@yookeroo.fritz.box> <1314027950.6866.242.camel@x201.home> <20110823023822.GO30097@yookeroo.fritz.box> <1314116600.2859.28.camel@bling.home> Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 7bit Cc: chrisw , Alexey Kardashevskiy , kvm@vger.kernel.org, Paul Mackerras , "linux-pci@vger.kernel.org" , qemu-devel , David Gibson , aafabbri , iommu , Avi Kivity , linuxppc-dev , benve@cisco.com To: Alex Williamson Return-path: In-Reply-To: <1314116600.2859.28.camel@bling.home> List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org Sender: qemu-devel-bounces+gceq-qemu-devel=gmane.org@nongnu.org List-Id: kvm.vger.kernel.org On Tue, 2011-08-23 at 10:23 -0600, Alex Williamson wrote: > > Yeah. Joerg's idea of binding groups internally (pass the fd of one > group to another via ioctl) is one option. The tricky part will be > implementing it to support hot unplug of any group from the > supergroup. > I believe Ben had a suggestion that supergroups could be created in > sysfs, but I don't know what the mechanism to do that looks like. It > would also be an extra management step to dynamically bind and unbind > groups to the supergroup around hotplug. Thanks, I don't really care that much what the method for creating them is, to be honest, I just prefer this concept of "meta groups" or "super groups" or "synthetic groups" (whatever you want to name them) to having a separate uiommu file descriptor. The one reason I have a slight preference for creating them "statically" using some kind of separate interface (again, I don't care whether it's sysfs, netlink, etc...) is that it means things like qemu don't have to care about them. In general, apps that want to use vfio can just get passed the path to such a group or the /dev/ path or the group number (whatever we chose as the way to identify a group), and don't need to know anything about "super groups", how to manipulate them, create them, possible constraints etc... Now, libvirt might want to know about that other API in order to provide control on the creation of these things, but that's a different issue. By "static" I mean they persist, they aren't tied to the lifetime of an fd. Now that's purely a preference on my side because I believe it will make life easier for actual programs wanting to use vfio to not have to care about those super-groups, but as I said earlier, I don't actually care that much :-) Cheers, Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate.crashing.org (gate.crashing.org [63.228.1.57]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by ozlabs.org (Postfix) with ESMTPS id C5886B6F18 for ; Wed, 24 Aug 2011 09:42:21 +1000 (EST) Subject: Re: kvm PCI assignment & VFIO ramblings From: Benjamin Herrenschmidt To: Alex Williamson In-Reply-To: <1314116600.2859.28.camel@bling.home> References: <20110802082848.GD29719@yookeroo.fritz.box> <1312308847.2653.467.camel@bling.home> <1312310121.2653.470.camel@bling.home> <20110803020422.GF29719@yookeroo.fritz.box> <4E3F9E33.5000706@redhat.com> <1312932258.4524.55.camel@bling.home> <1312944513.29273.28.camel@pasglop> <1313859105.6866.192.camel@x201.home> <20110822055509.GI30097@yookeroo.fritz.box> <1314027950.6866.242.camel@x201.home> <20110823023822.GO30097@yookeroo.fritz.box> <1314116600.2859.28.camel@bling.home> Content-Type: text/plain; charset="UTF-8" Date: Wed, 24 Aug 2011 09:41:32 +1000 Message-ID: <1314142892.30478.64.camel@pasglop> Mime-Version: 1.0 Cc: chrisw , Alexey Kardashevskiy , kvm@vger.kernel.org, Paul Mackerras , "linux-pci@vger.kernel.org" , qemu-devel , David Gibson , aafabbri , iommu , Avi Kivity , Anthony Liguori , linuxppc-dev , benve@cisco.com List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , On Tue, 2011-08-23 at 10:23 -0600, Alex Williamson wrote: > > Yeah. Joerg's idea of binding groups internally (pass the fd of one > group to another via ioctl) is one option. The tricky part will be > implementing it to support hot unplug of any group from the > supergroup. > I believe Ben had a suggestion that supergroups could be created in > sysfs, but I don't know what the mechanism to do that looks like. It > would also be an extra management step to dynamically bind and unbind > groups to the supergroup around hotplug. Thanks, I don't really care that much what the method for creating them is, to be honest, I just prefer this concept of "meta groups" or "super groups" or "synthetic groups" (whatever you want to name them) to having a separate uiommu file descriptor. The one reason I have a slight preference for creating them "statically" using some kind of separate interface (again, I don't care whether it's sysfs, netlink, etc...) is that it means things like qemu don't have to care about them. In general, apps that want to use vfio can just get passed the path to such a group or the /dev/ path or the group number (whatever we chose as the way to identify a group), and don't need to know anything about "super groups", how to manipulate them, create them, possible constraints etc... Now, libvirt might want to know about that other API in order to provide control on the creation of these things, but that's a different issue. By "static" I mean they persist, they aren't tied to the lifetime of an fd. Now that's purely a preference on my side because I believe it will make life easier for actual programs wanting to use vfio to not have to care about those super-groups, but as I said earlier, I don't actually care that much :-) Cheers, Ben. From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([140.186.70.92]:47141) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qw0bx-0006p7-SF for qemu-devel@nongnu.org; Tue, 23 Aug 2011 19:42:27 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Qw0bw-0007NI-KR for qemu-devel@nongnu.org; Tue, 23 Aug 2011 19:42:25 -0400 Received: from gate.crashing.org ([63.228.1.57]:60675) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Qw0bw-0007N9-AM for qemu-devel@nongnu.org; Tue, 23 Aug 2011 19:42:24 -0400 From: Benjamin Herrenschmidt In-Reply-To: <1314116600.2859.28.camel@bling.home> References: <20110802082848.GD29719@yookeroo.fritz.box> <1312308847.2653.467.camel@bling.home> <1312310121.2653.470.camel@bling.home> <20110803020422.GF29719@yookeroo.fritz.box> <4E3F9E33.5000706@redhat.com> <1312932258.4524.55.camel@bling.home> <1312944513.29273.28.camel@pasglop> <1313859105.6866.192.camel@x201.home> <20110822055509.GI30097@yookeroo.fritz.box> <1314027950.6866.242.camel@x201.home> <20110823023822.GO30097@yookeroo.fritz.box> <1314116600.2859.28.camel@bling.home> Content-Type: text/plain; charset="UTF-8" Date: Wed, 24 Aug 2011 09:41:32 +1000 Message-ID: <1314142892.30478.64.camel@pasglop> Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: Re: [Qemu-devel] kvm PCI assignment & VFIO ramblings List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: Alex Williamson Cc: chrisw , Alexey Kardashevskiy , kvm@vger.kernel.org, Paul Mackerras , "linux-pci@vger.kernel.org" , qemu-devel , David Gibson , aafabbri , iommu , Avi Kivity , linuxppc-dev , benve@cisco.com On Tue, 2011-08-23 at 10:23 -0600, Alex Williamson wrote: > > Yeah. Joerg's idea of binding groups internally (pass the fd of one > group to another via ioctl) is one option. The tricky part will be > implementing it to support hot unplug of any group from the > supergroup. > I believe Ben had a suggestion that supergroups could be created in > sysfs, but I don't know what the mechanism to do that looks like. It > would also be an extra management step to dynamically bind and unbind > groups to the supergroup around hotplug. Thanks, I don't really care that much what the method for creating them is, to be honest, I just prefer this concept of "meta groups" or "super groups" or "synthetic groups" (whatever you want to name them) to having a separate uiommu file descriptor. The one reason I have a slight preference for creating them "statically" using some kind of separate interface (again, I don't care whether it's sysfs, netlink, etc...) is that it means things like qemu don't have to care about them. In general, apps that want to use vfio can just get passed the path to such a group or the /dev/ path or the group number (whatever we chose as the way to identify a group), and don't need to know anything about "super groups", how to manipulate them, create them, possible constraints etc... Now, libvirt might want to know about that other API in order to provide control on the creation of these things, but that's a different issue. By "static" I mean they persist, they aren't tied to the lifetime of an fd. Now that's purely a preference on my side because I believe it will make life easier for actual programs wanting to use vfio to not have to care about those super-groups, but as I said earlier, I don't actually care that much :-) Cheers, Ben.