* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-03-06 22:31 Konrad Rzeszutek Wilk 0 siblings, 0 replies; 23+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-03-06 22:31 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Michal Hocko, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Toshi Kani, Greg KH, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, On Mar 6, 2014 5:25 PM, Stuart Yoder <stuart.yoder@freescale.com> wrote: > > > > > -----Original Message----- > > From: Greg KH [mailto:gregkh@linuxfoundation.org] > > Sent: Thursday, February 20, 2014 4:44 PM > > To: Yoder Stuart-B08248 > > Cc: Antonios Motakis; alex.williamson@redhat.com; > > kvmarm@lists.cs.columbia.edu; iommu@lists.linux-foundation.org; linux- > > kernel@vger.kernel.org; tech@virtualopensystems.com; > > a.rigo@virtualopensystems.com; kim.phillips@linaro.org; > > jan.kiszka@siemens.com; kvm@vger.kernel.org; Bhushan Bharat-R65777; Wood > > Scott-B07421; christoffer.dall@linaro.org; agraf@suse.de; Sethi Varun- > > B16395; will.deacon@arm.com; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Thu, Feb 20, 2014 at 10:34:35PM +0000, Stuart Yoder wrote: > > > > > > > > > > -----Original Message----- > > > > From: Yoder Stuart-B08248 > > > > Sent: Saturday, February 15, 2014 12:19 PM > > > > To: 'Greg KH' > > > > Cc: Antonios Motakis; alex.williamson@redhat.com; > > > > kvmarm@lists.cs.columbia.edu; iommu@lists.linux-foundation.org; > > linux- > > > > kernel@vger.kernel.org; tech@virtualopensystems.com; > > > > a.rigo@virtualopensystems.com; kim.phillips@linaro.org; > > > > jan.kiszka@siemens.com; kvm@vger.kernel.org; Bhushan Bharat-R65777; > > Wood > > > > Scott-B07421; christoffer.dall@linaro.org; agraf@suse.de; Sethi > > Varun- > > > > B16395; will.deacon@arm.com; Tejun Heo; Rafael J. Wysocki; Guenter > > Roeck; > > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > > > Subject: RE: [RFC PATCH v4 01/10] driver core: export > > > > driver_probe_device() > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Greg KH [mailto:gregkh@linuxfoundation.org] > > > > > Sent: Saturday, February 15, 2014 11:34 AM > > > > > To: Yoder Stuart-B08248 > > > > > Cc: Antonios Motakis; alex.williamson@redhat.com; > > > > > kvmarm@lists.cs.columbia.edu; iommu@lists.linux-foundation.org; > > linux- > > > > > kernel@vger.kernel.org; tech@virtualopensystems.com; > > > > > a.rigo@virtualopensystems.com; kim.phillips@linaro.org; > > > > > jan.kiszka@siemens.com; kvm@vger.kernel.org; Bhushan Bharat-R65777; > > > > Wood > > > > > Scott-B07421; christoffer.dall@linaro.org; agraf@suse.de; Sethi > > Varun- > > > > > B16395; will.deacon@arm.com; Tejun Heo; Rafael J. Wysocki; Guenter > > > > Roeck; > > > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn > > Helgaas > > > > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > > > > driver_probe_device() > > > > > > > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > > > > Why? driver_probe_device() allows a driver to explicitly > > bind > > > > > > > > to a specific device. What is conceptually wrong with > > allowing > > > > > > > > that? > > > > > > > > > > > > > > Because that's not how a bus should work, and the fact that no > > > > other > > > > > > > subsystem in the kernel does that might be a hint you are > > trying to > > > > > do > > > > > > > something a bit "wrong" here. > > > > > > > > > > > > Let me try to succinctly as I can describe the problem we are > > trying > > > > to > > > > > > solve here... > > > > > > > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices > > to be > > > > > > exposed user space (via file descriptors), enabling user space > > > > > > drivers. So, for example to export an e1000 card to user space, > > I do > > > > > > this: > > > > > > > > > > > > echo 0001:03:00.0 > > > > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > > > > > > > What's wrong with using the "bind" file instead? That picks a > > specific > > > > > device and binds it to a specific driver. Or have we been down > > this > > > > > path before? :) > > > > > > > > Yes we have :) The "bind" file does not bypass device ID checks, so > > > > it wouldn't work without new_id or a wildcard match of some kind. > > > > > > > > > And that is for a PCI "driver" not a totally separate bus, which it > > > > > looks like you are wanting to do here. > > > > > > > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > > > > > > > The first step unbinds the target device (0001:03:00.0) from the > > > > normal > > > > > > e1000 driver. > > > > > > > > > > > > The second step causes the vfio-pci driver to bind to device > > > > > 0001:03:00.0. > > > > > > This second step tells vfio-pci that it now handles e1000 device > > IDs, > > > > > > and the vfio-pci drivers registers with the PCI bus to handle > > '8086 > > > > > 10d3'. > > > > > > > > > > > > That works, but it is ugly. We now have 2 active drivers > > handling > > > > > > the same device type...which introduces various possible race > > > > > conditions. > > > > > > > > > > > > We never want vfio-pci to auto-bind to any new device that shows > > up > > > > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > > > > action by an administrator. > > > > > > > > > > Then use the "bind" file. > > > > > > > > See above. > > > > > > > > > > You mentioned previously that user space can sort out the problem > > > > > > of multiple drivers registered for handling the same device type. > > > > > > That is true, but doesn't help here. We don't want vfio-pci > > > > > > to handle _all_ e1000 cards, just explicitly selected e1000 > > cards. > > > > > > > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > > > > devices that may be hot-plugged. > > > > > > > > > > I want a pony too... > > > > > > > > It's not that difficult...this patch accomplishes it by > > > > simply allowing drivers to call driver_probe_device(). > > > > > > > > > > There are 2 proposed mechanisms that have been put forth, both of > > > > > > which you have now rejected: > > > > > > > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > > > > driver (like vfio-pci) to only bind by explicit request > > > > through > > > > > > the sysfs 'bind' file. > > > > > > > > > > Why did I reject this? What did the patch look like? > > > > > > > > https://lkml.org/lkml/2013/12/3/253 > > > > > > > > > > > > > > 2. Have the vfio driver call driver_probe_device() to > > explicitly > > > > > bind > > > > > > a particular device instance to the driver. Only change > > we > > > > need > > > > > > here is the EXPORT_SYMBOL. > > > > > > > > > > How are you going to prevent the driver from being bound to the > > device > > > > > in the core with this change? How are you going to call this > > function? > > > > > When? On what action of the user? > > > > > > > > The vfio-pci driver would create a sysfs object "vfio_bind". > > > > > > > > User would do: > > > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > > > > > > > vfio-pci would call driver_probe_device() which binds > > > > the specific device to the vfio-pci driver...and there is > > > > no ambiguity. > > > > > > > > > > Are you in principle opposed to any mechanism that would allow 2 > > > > > drivers > > > > > > to be resident/active and allow a sysadmin to explicitly bind a > > > > > > particular device instance to the driver of their choice? > > > > > > > > > > No, that works today with the bind/unbind/new_id files, it's just > > that > > > > > you don't like it :) > > > > > > > > We don't like it because of the ambiguities/race-conditions with > > > > the current situation. > > > > > > > > A vfio driver, like vfio-pci, certainly is a bit different than a > > normal > > > > driver, in that it really is not device ID aware. It simply passes > > > > through device resources (mappable regions, IRQs) to user space > > without > > > > interpreting or understanding them. It is kind of a "meta" driver, > > but > > > > it is not a bus. Every bus type would need its own vfio driver to > > > > do this type of device pass through. > > > > > > Hi Greg, > > > > > > Any further thoughts on this? > > > > Sorry, been swamped with other patches and stable stuff and not had a > > time to look at it. Give me a few days... > > Hi Greg, wanted to ping you on this again... > > I know some days have gone by, so let me summarize the issue-- vfio > drivers in the kernel (regardless of bus type) need to bind to > devices of any type. There seem to be 3 approaches: > > 1. new_id -- (current approach) the user explicitly registers > each new device type with the vfio driver using the new_id > mechanism. > > Problem: multiple drivers will be resident that handle the > same device type...and there is nothing user space hotplug > infrastructure can do to help. > > 2. "any id" -- the vfio driver could specify a wildcard match of > some kind so that it can bind to any possible device id. However, > we don't want vfio grabbing all devices...just the ones we > explicitly want to pass to user space. > > Proposed patch to support this was to create a new flag > "sysfs_bind_only" in struct device_driver. When this flag > is set, the driver can only bind to devices via the sysfs > bind file. This would allow the wildcard match to work. > > Patch is here: > https://lkml.org/lkml/2013/12/3/253 > > 3. Driver initiated explicit bind -- with this approach the > vfio driver would create a private 'bind' sysfs object > and the user would echo the requested device into it: > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > In order to make that work, the driver would need to call > driver_probe_device() and thus we need this patch: > https://lkml.org/lkml/2014/2/8/175 There is a forth way. You can to the devices using BDF. And if you want to do it at startup you can have an 'hide' parameter that will assign to itself is he devices and the be the owner of said devices. > > > Thanks, > Stuart > > > > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 23+ messages in thread
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-03-06 22:31 Konrad Rzeszutek Wilk 0 siblings, 0 replies; 23+ messages in thread From: Konrad Rzeszutek Wilk @ 2014-03-06 22:31 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Michal Hocko, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Toshi Kani, Greg KH, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, On Mar 6, 2014 5:25 PM, Stuart Yoder <stuart.yoder@freescale.com> wrote: > > > > > -----Original Message----- > > From: Greg KH [mailto:gregkh@linuxfoundation.org] > > Sent: Thursday, February 20, 2014 4:44 PM > > To: Yoder Stuart-B08248 > > Cc: Antonios Motakis; alex.williamson@redhat.com; > > kvmarm@lists.cs.columbia.edu; iommu@lists.linux-foundation.org; linux- > > kernel@vger.kernel.org; tech@virtualopensystems.com; > > a.rigo@virtualopensystems.com; kim.phillips@linaro.org; > > jan.kiszka@siemens.com; kvm@vger.kernel.org; Bhushan Bharat-R65777; Wood > > Scott-B07421; christoffer.dall@linaro.org; agraf@suse.de; Sethi Varun- > > B16395; will.deacon@arm.com; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Thu, Feb 20, 2014 at 10:34:35PM +0000, Stuart Yoder wrote: > > > > > > > > > > -----Original Message----- > > > > From: Yoder Stuart-B08248 > > > > Sent: Saturday, February 15, 2014 12:19 PM > > > > To: 'Greg KH' > > > > Cc: Antonios Motakis; alex.williamson@redhat.com; > > > > kvmarm@lists.cs.columbia.edu; iommu@lists.linux-foundation.org; > > linux- > > > > kernel@vger.kernel.org; tech@virtualopensystems.com; > > > > a.rigo@virtualopensystems.com; kim.phillips@linaro.org; > > > > jan.kiszka@siemens.com; kvm@vger.kernel.org; Bhushan Bharat-R65777; > > Wood > > > > Scott-B07421; christoffer.dall@linaro.org; agraf@suse.de; Sethi > > Varun- > > > > B16395; will.deacon@arm.com; Tejun Heo; Rafael J. Wysocki; Guenter > > Roeck; > > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > > > Subject: RE: [RFC PATCH v4 01/10] driver core: export > > > > driver_probe_device() > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Greg KH [mailto:gregkh@linuxfoundation.org] > > > > > Sent: Saturday, February 15, 2014 11:34 AM > > > > > To: Yoder Stuart-B08248 > > > > > Cc: Antonios Motakis; alex.williamson@redhat.com; > > > > > kvmarm@lists.cs.columbia.edu; iommu@lists.linux-foundation.org; > > linux- > > > > > kernel@vger.kernel.org; tech@virtualopensystems.com; > > > > > a.rigo@virtualopensystems.com; kim.phillips@linaro.org; > > > > > jan.kiszka@siemens.com; kvm@vger.kernel.org; Bhushan Bharat-R65777; > > > > Wood > > > > > Scott-B07421; christoffer.dall@linaro.org; agraf@suse.de; Sethi > > Varun- > > > > > B16395; will.deacon@arm.com; Tejun Heo; Rafael J. Wysocki; Guenter > > > > Roeck; > > > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn > > Helgaas > > > > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > > > > driver_probe_device() > > > > > > > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > > > > Why? driver_probe_device() allows a driver to explicitly > > bind > > > > > > > > to a specific device. What is conceptually wrong with > > allowing > > > > > > > > that? > > > > > > > > > > > > > > Because that's not how a bus should work, and the fact that no > > > > other > > > > > > > subsystem in the kernel does that might be a hint you are > > trying to > > > > > do > > > > > > > something a bit "wrong" here. > > > > > > > > > > > > Let me try to succinctly as I can describe the problem we are > > trying > > > > to > > > > > > solve here... > > > > > > > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices > > to be > > > > > > exposed user space (via file descriptors), enabling user space > > > > > > drivers. So, for example to export an e1000 card to user space, > > I do > > > > > > this: > > > > > > > > > > > > echo 0001:03:00.0 > > > > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > > > > > > > What's wrong with using the "bind" file instead? That picks a > > specific > > > > > device and binds it to a specific driver. Or have we been down > > this > > > > > path before? :) > > > > > > > > Yes we have :) The "bind" file does not bypass device ID checks, so > > > > it wouldn't work without new_id or a wildcard match of some kind. > > > > > > > > > And that is for a PCI "driver" not a totally separate bus, which it > > > > > looks like you are wanting to do here. > > > > > > > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > > > > > > > The first step unbinds the target device (0001:03:00.0) from the > > > > normal > > > > > > e1000 driver. > > > > > > > > > > > > The second step causes the vfio-pci driver to bind to device > > > > > 0001:03:00.0. > > > > > > This second step tells vfio-pci that it now handles e1000 device > > IDs, > > > > > > and the vfio-pci drivers registers with the PCI bus to handle > > '8086 > > > > > 10d3'. > > > > > > > > > > > > That works, but it is ugly. We now have 2 active drivers > > handling > > > > > > the same device type...which introduces various possible race > > > > > conditions. > > > > > > > > > > > > We never want vfio-pci to auto-bind to any new device that shows > > up > > > > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > > > > action by an administrator. > > > > > > > > > > Then use the "bind" file. > > > > > > > > See above. > > > > > > > > > > You mentioned previously that user space can sort out the problem > > > > > > of multiple drivers registered for handling the same device type. > > > > > > That is true, but doesn't help here. We don't want vfio-pci > > > > > > to handle _all_ e1000 cards, just explicitly selected e1000 > > cards. > > > > > > > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > > > > devices that may be hot-plugged. > > > > > > > > > > I want a pony too... > > > > > > > > It's not that difficult...this patch accomplishes it by > > > > simply allowing drivers to call driver_probe_device(). > > > > > > > > > > There are 2 proposed mechanisms that have been put forth, both of > > > > > > which you have now rejected: > > > > > > > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > > > > driver (like vfio-pci) to only bind by explicit request > > > > through > > > > > > the sysfs 'bind' file. > > > > > > > > > > Why did I reject this? What did the patch look like? > > > > > > > > https://lkml.org/lkml/2013/12/3/253 > > > > > > > > > > > > > > 2. Have the vfio driver call driver_probe_device() to > > explicitly > > > > > bind > > > > > > a particular device instance to the driver. Only change > > we > > > > need > > > > > > here is the EXPORT_SYMBOL. > > > > > > > > > > How are you going to prevent the driver from being bound to the > > device > > > > > in the core with this change? How are you going to call this > > function? > > > > > When? On what action of the user? > > > > > > > > The vfio-pci driver would create a sysfs object "vfio_bind". > > > > > > > > User would do: > > > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > > > > > > > vfio-pci would call driver_probe_device() which binds > > > > the specific device to the vfio-pci driver...and there is > > > > no ambiguity. > > > > > > > > > > Are you in principle opposed to any mechanism that would allow 2 > > > > > drivers > > > > > > to be resident/active and allow a sysadmin to explicitly bind a > > > > > > particular device instance to the driver of their choice? > > > > > > > > > > No, that works today with the bind/unbind/new_id files, it's just > > that > > > > > you don't like it :) > > > > > > > > We don't like it because of the ambiguities/race-conditions with > > > > the current situation. > > > > > > > > A vfio driver, like vfio-pci, certainly is a bit different than a > > normal > > > > driver, in that it really is not device ID aware. It simply passes > > > > through device resources (mappable regions, IRQs) to user space > > without > > > > interpreting or understanding them. It is kind of a "meta" driver, > > but > > > > it is not a bus. Every bus type would need its own vfio driver to > > > > do this type of device pass through. > > > > > > Hi Greg, > > > > > > Any further thoughts on this? > > > > Sorry, been swamped with other patches and stable stuff and not had a > > time to look at it. Give me a few days... > > Hi Greg, wanted to ping you on this again... > > I know some days have gone by, so let me summarize the issue-- vfio > drivers in the kernel (regardless of bus type) need to bind to > devices of any type. There seem to be 3 approaches: > > 1. new_id -- (current approach) the user explicitly registers > each new device type with the vfio driver using the new_id > mechanism. > > Problem: multiple drivers will be resident that handle the > same device type...and there is nothing user space hotplug > infrastructure can do to help. > > 2. "any id" -- the vfio driver could specify a wildcard match of > some kind so that it can bind to any possible device id. However, > we don't want vfio grabbing all devices...just the ones we > explicitly want to pass to user space. > > Proposed patch to support this was to create a new flag > "sysfs_bind_only" in struct device_driver. When this flag > is set, the driver can only bind to devices via the sysfs > bind file. This would allow the wildcard match to work. > > Patch is here: > https://lkml.org/lkml/2013/12/3/253 > > 3. Driver initiated explicit bind -- with this approach the > vfio driver would create a private 'bind' sysfs object > and the user would echo the requested device into it: > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > In order to make that work, the driver would need to call > driver_probe_device() and thus we need this patch: > https://lkml.org/lkml/2014/2/8/175 There is a forth way. You can to the devices using BDF. And if you want to do it at startup you can have an 'hide' parameter that will assign to itself is he devices and the be the owner of said devices. > > > Thanks, > Stuart > > > > > _______________________________________________ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ^ permalink raw reply [flat|nested] 23+ messages in thread
* [RFC PATCH v4 00/10] VFIO support for platform devices @ 2014-02-08 17:29 Antonios Motakis 2014-02-08 17:29 ` Antonios Motakis 0 siblings, 1 reply; 23+ messages in thread From: Antonios Motakis @ 2014-02-08 17:29 UTC (permalink / raw) To: alex.williamson, kvmarm, iommu, linux-kernel, gregkh Cc: tech, a.rigo, B08248, kim.phillips, jan.kiszka, kvm, R65777, B07421, christoffer.dall, agraf, B16395, will.deacon, Antonios Motakis v4 of this series is functionally identical to v3 for VFIO_PLATFORM. The only change is the inclusion of Kim Phillips' patch to expose driver_probe_device() and the implementation of a binding mechanism for arbitrary devices via a file in sysfs. The latter has been folded in the skeleton patch (04/10). This patch series aims to implement VFIO support for platform devices that reside behind an IOMMU. Examples of such devices are devices behind an ARM SMMU, or behind a Samsung Exynos System MMU. The API used is based on the existing VFIO API that is also used with PCI devices. Only devices that include a basic set of IRQs and memory regions are targeted; devices with complex relationships with other devices on a device tree are not taken into account at this stage. A copy with all the dependencies applied can be cloned from branch vfio-platform-v4 at git@github.com:virtualopensystems/linux-kvm-arm.git This code can also be tested on ARM FastModels using the following test cases: - A user space implementation via VFIO for the PL330 on the FastModels: git@github.com:virtualopensystems/pl330-vfio-test.git - A QEMU prototype, also based on the PL330: git@github.com:virtualopensystems/qemu.git pl330-vfio-dev We have written detailed instructions on how to build and run these tests: http://www.virtualopensystems.com/en/solutions/guides/vfio-on-arm/ The following IOCTLs have been found to be working on FastModels with an ARM SMMU (MMU400). Testing was based on the ARM PL330 DMA Controller featured on those models. - VFIO_GET_API_VERSION - VFIO_CHECK_EXTENSION The TYPE1 fix proposed here enables the following IOCTLs: - VFIO_GROUP_GET_STATUS - VFIO_GROUP_SET_CONTAINER - VFIO_SET_IOMMU - VFIO_IOMMU_GET_INFO - VFIO_IOMMU_MAP_DMA For this ioctl specifically, a new flag has been added: VFIO_DMA_MAP_FLAG_EXEC. This flag is taken into account on systems with an ARM SMMU. The VFIO platform driver proposed here implements the following: - VFIO_GROUP_GET_DEVICE_FD - VFIO_DEVICE_GET_INFO - VFIO_DEVICE_GET_REGION_INFO - VFIO_DEVICE_GET_IRQ_INFO - VFIO_DEVICE_SET_IRQS IRQs are implemented partially using this ioctl. Handling incoming interrupts with an eventfd is supported, as is masking and unmasking. Level sensitive interrupts are automasked. What is not implemented is masking/unmasking via eventfd. In addition, the VFIO platform driver implements the following through the VFIO device file descriptor: - MMAPing memory regions to the virtual address space of the VFIO user. - Read / write of memory regions directly through the file descriptor. What still needs to be done, includes: - Eventfd for masking/unmasking - Extend the driver and API for device tree metadata - QEMU / KVM integration - Device specific functionality (e.g. VFIO_DEVICE_RESET) - Improve VFIO_IOMMU_TYPE1 driver to support multiple buses at the same time - Bind to ARM AMBA devices - IOMMUs with nested page tables (Stage 1 & 2 translation on ARM SMMUs) Changes since v3: - Use Kim Phillips' driver_probe_device() Changes since v2: - Fixed Read/Write and MMAP on device regions - Removed dependency on Device Tree - Interrupts support - Interrupt masking/unmasking - Automask level sensitive interrupts - Introduced VFIO_DMA_MAP_FLAG_EXEC - Code clean ups Antonios Motakis (9): VFIO_IOMMU_TYPE1: Introduce the VFIO_DMA_MAP_FLAG_EXEC flag VFIO_IOMMU_TYPE1: workaround to build for platform devices VFIO_PLATFORM: Initial skeleton of VFIO support for platform devices VFIO_PLATFORM: Return info for device and its memory mapped IO regions VFIO_PLATFORM: Read and write support for the device fd VFIO_PLATFORM: Support MMAP of MMIO regions VFIO_PLATFORM: Return IRQ info VFIO_PLATFORM: Initial interrupts support VFIO_PLATFORM: Support for maskable and automasked interrupts Kim Phillips (1): driver core: export driver_probe_device() drivers/base/base.h | 1 - drivers/base/dd.c | 1 + drivers/vfio/Kconfig | 3 +- drivers/vfio/Makefile | 1 + drivers/vfio/platform/Kconfig | 9 + drivers/vfio/platform/Makefile | 4 + drivers/vfio/platform/vfio_platform.c | 493 ++++++++++++++++++++++++++ drivers/vfio/platform/vfio_platform_irq.c | 289 +++++++++++++++ drivers/vfio/platform/vfio_platform_private.h | 50 +++ drivers/vfio/vfio_iommu_type1.c | 27 +- include/linux/device.h | 1 + include/uapi/linux/vfio.h | 2 + 12 files changed, 874 insertions(+), 7 deletions(-) create mode 100644 drivers/vfio/platform/Kconfig create mode 100644 drivers/vfio/platform/Makefile create mode 100644 drivers/vfio/platform/vfio_platform.c create mode 100644 drivers/vfio/platform/vfio_platform_irq.c create mode 100644 drivers/vfio/platform/vfio_platform_private.h -- 1.8.3.2 ^ permalink raw reply [flat|nested] 23+ messages in thread
* [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-08 17:29 ` Antonios Motakis 0 siblings, 0 replies; 23+ messages in thread From: Antonios Motakis @ 2014-02-08 17:29 UTC (permalink / raw) To: alex.williamson, kvmarm, iommu, linux-kernel, gregkh Cc: tech, a.rigo, B08248, kim.phillips, jan.kiszka, kvm, R65777, B07421, christoffer.dall, agraf, B16395, will.deacon, Tejun Heo, Rafael J. Wysocki, Guenter Roeck, Toshi Kani, Joe Perches, Dmitry Kasatkin, Michal Hocko, Bjorn Helgaas From: Kim Phillips <kim.phillips@linaro.org> Needed by drivers, such as the vfio platform driver [1], seeking to bypass bind_store()'s driver_match_device(), and bind to any device via a private sysfs bind file. [1] https://lkml.org/lkml/2013/12/11/522 note: the EXPORT_SYMBOL is needed because vfio-platform can be built as a module. Signed-off-by: Kim Phillips <kim.phillips@linaro.org> --- drivers/base/base.h | 1 - drivers/base/dd.c | 1 + include/linux/device.h | 1 + 3 files changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/base/base.h b/drivers/base/base.h index 24f4242..fe25ad87 100644 --- a/drivers/base/base.h +++ b/drivers/base/base.h @@ -112,7 +112,6 @@ extern int bus_add_driver(struct device_driver *drv); extern void bus_remove_driver(struct device_driver *drv); extern void driver_detach(struct device_driver *drv); -extern int driver_probe_device(struct device_driver *drv, struct device *dev); extern void driver_deferred_probe_del(struct device *dev); static inline int driver_match_device(struct device_driver *drv, struct device *dev) diff --git a/drivers/base/dd.c b/drivers/base/dd.c index 0605176..44f6184 100644 --- a/drivers/base/dd.c +++ b/drivers/base/dd.c @@ -384,6 +384,7 @@ int driver_probe_device(struct device_driver *drv, struct device *dev) return ret; } +EXPORT_SYMBOL_GPL(driver_probe_device); static int __device_attach(struct device_driver *drv, void *data) { diff --git a/include/linux/device.h b/include/linux/device.h index 952b010..ad80dd2 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -257,6 +257,7 @@ extern struct device_driver *driver_find(const char *name, struct bus_type *bus); extern int driver_probe_done(void); extern void wait_for_device_probe(void); +extern int driver_probe_device(struct device_driver *drv, struct device *dev); /* sysfs interface for exporting driver attributes */ -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-08 17:29 ` Antonios Motakis 0 siblings, 0 replies; 23+ messages in thread From: Antonios Motakis @ 2014-02-08 17:29 UTC (permalink / raw) To: alex.williamson-H+wXaHxf7aLQT0dZR+AlfA, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, linux-kernel-u79uwXL29TY76Z2rM5mHXA, gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r Cc: B07421-KZfg59tc24xl57MIdRCFDg, Dmitry Kasatkin, Toshi Kani, kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, Rafael J. Wysocki, will.deacon-5wv7dgnIgG8, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, agraf-l3A5Bk7waGM, Joe Perches, B08248-KZfg59tc24xl57MIdRCFDg, Guenter Roeck, R65777-KZfg59tc24xl57MIdRCFDg, Tejun Heo, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Bjorn Helgaas, Michal Hocko, christoffer.dall-QSEj5FYQhm4dnm+yROfE0A From: Kim Phillips <kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> Needed by drivers, such as the vfio platform driver [1], seeking to bypass bind_store()'s driver_match_device(), and bind to any device via a private sysfs bind file. [1] https://lkml.org/lkml/2013/12/11/522 note: the EXPORT_SYMBOL is needed because vfio-platform can be built as a module. Signed-off-by: Kim Phillips <kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> --- drivers/base/base.h | 1 - drivers/base/dd.c | 1 + include/linux/device.h | 1 + 3 files changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/base/base.h b/drivers/base/base.h index 24f4242..fe25ad87 100644 --- a/drivers/base/base.h +++ b/drivers/base/base.h @@ -112,7 +112,6 @@ extern int bus_add_driver(struct device_driver *drv); extern void bus_remove_driver(struct device_driver *drv); extern void driver_detach(struct device_driver *drv); -extern int driver_probe_device(struct device_driver *drv, struct device *dev); extern void driver_deferred_probe_del(struct device *dev); static inline int driver_match_device(struct device_driver *drv, struct device *dev) diff --git a/drivers/base/dd.c b/drivers/base/dd.c index 0605176..44f6184 100644 --- a/drivers/base/dd.c +++ b/drivers/base/dd.c @@ -384,6 +384,7 @@ int driver_probe_device(struct device_driver *drv, struct device *dev) return ret; } +EXPORT_SYMBOL_GPL(driver_probe_device); static int __device_attach(struct device_driver *drv, void *data) { diff --git a/include/linux/device.h b/include/linux/device.h index 952b010..ad80dd2 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -257,6 +257,7 @@ extern struct device_driver *driver_find(const char *name, struct bus_type *bus); extern int driver_probe_done(void); extern void wait_for_device_probe(void); +extern int driver_probe_device(struct device_driver *drv, struct device *dev); /* sysfs interface for exporting driver attributes */ -- 1.8.3.2 ^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-14 22:27 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-14 22:27 UTC (permalink / raw) To: Antonios Motakis Cc: alex.williamson, kvmarm, iommu, linux-kernel, tech, a.rigo, B08248, kim.phillips, jan.kiszka, kvm, R65777, B07421, christoffer.dall, agraf, B16395, will.deacon, Tejun Heo, Rafael J. Wysocki, Guenter Roeck, Toshi Kani, Joe Perches, Dmitry Kasatkin, Michal Hocko, Bjorn Helgaas On Sat, Feb 08, 2014 at 06:29:31PM +0100, Antonios Motakis wrote: > From: Kim Phillips <kim.phillips@linaro.org> > > Needed by drivers, such as the vfio platform driver [1], seeking to > bypass bind_store()'s driver_match_device(), and bind to any device > via a private sysfs bind file. > > [1] https://lkml.org/lkml/2013/12/11/522 > > note: the EXPORT_SYMBOL is needed because vfio-platform can be built > as a module. No code outside of drivers/base/ should be calling this function, you are doing something wrong in your bus if you want to do this, please fix your bus code. sorry, I can't accept this at all. greg k-h ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-14 22:27 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-14 22:27 UTC (permalink / raw) To: Antonios Motakis Cc: B07421-KZfg59tc24xl57MIdRCFDg, kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Michal Hocko, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, R65777-KZfg59tc24xl57MIdRCFDg, Guenter Roeck, Dmitry Kasatkin, B08248-KZfg59tc24xl57MIdRCFDg, Bjorn Helgaas, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, Joe Perches, Tejun Heo, christoffer.dall-QSEj5FYQhm4dnm+yROfE0A On Sat, Feb 08, 2014 at 06:29:31PM +0100, Antonios Motakis wrote: > From: Kim Phillips <kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> > > Needed by drivers, such as the vfio platform driver [1], seeking to > bypass bind_store()'s driver_match_device(), and bind to any device > via a private sysfs bind file. > > [1] https://lkml.org/lkml/2013/12/11/522 > > note: the EXPORT_SYMBOL is needed because vfio-platform can be built > as a module. No code outside of drivers/base/ should be calling this function, you are doing something wrong in your bus if you want to do this, please fix your bus code. sorry, I can't accept this at all. greg k-h ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <20140214222716.GA11838-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org>]
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <20140214222716.GA11838-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> @ 2014-02-14 23:00 ` Stuart Yoder [not found] ` <ba7597fd8c9f4d91bbccfb42e31a165e-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> 0 siblings, 1 reply; 23+ messages in thread From: Stuart Yoder @ 2014-02-14 23:00 UTC (permalink / raw) To: Greg KH, Antonios Motakis Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Michal Hocko, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, Joe Perches, > -----Original Message----- > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > Sent: Friday, February 14, 2014 4:27 PM > To: Antonios Motakis > Cc: alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; > iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; > tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; Yoder Stuart- > B08248; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; > kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood Scott-B07421; > christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun-B16395; > will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; Toshi > Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: Re: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > On Sat, Feb 08, 2014 at 06:29:31PM +0100, Antonios Motakis wrote: > > From: Kim Phillips <kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> > > > > Needed by drivers, such as the vfio platform driver [1], seeking to > > bypass bind_store()'s driver_match_device(), and bind to any device > > via a private sysfs bind file. > > > > [1] https://lkml.org/lkml/2013/12/11/522 > > > > note: the EXPORT_SYMBOL is needed because vfio-platform can be built > > as a module. > > No code outside of drivers/base/ should be calling this function Why? driver_probe_device() allows a driver to explicitly bind to a specific device. What is conceptually wrong with allowing that? Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <ba7597fd8c9f4d91bbccfb42e31a165e-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org>]
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <ba7597fd8c9f4d91bbccfb42e31a165e-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> @ 2014-02-15 2:47 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-15 2:47 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA On Fri, Feb 14, 2014 at 11:00:31PM +0000, Stuart Yoder wrote: > > > > -----Original Message----- > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > Sent: Friday, February 14, 2014 4:27 PM > > To: Antonios Motakis > > Cc: alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; > > iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; > > tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; Yoder Stuart- > > B08248; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; > > kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood Scott-B07421; > > christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun-B16395; > > will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; Toshi > > Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Sat, Feb 08, 2014 at 06:29:31PM +0100, Antonios Motakis wrote: > > > From: Kim Phillips <kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> > > > > > > Needed by drivers, such as the vfio platform driver [1], seeking to > > > bypass bind_store()'s driver_match_device(), and bind to any device > > > via a private sysfs bind file. > > > > > > [1] https://lkml.org/lkml/2013/12/11/522 > > > > > > note: the EXPORT_SYMBOL is needed because vfio-platform can be built > > > as a module. > > > > No code outside of drivers/base/ should be calling this function > > Why? driver_probe_device() allows a driver to explicitly bind > to a specific device. What is conceptually wrong with allowing > that? Because that's not how a bus should work, and the fact that no other subsystem in the kernel does that might be a hint you are trying to do something a bit "wrong" here. greg k-h ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-15 2:47 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-15 2:47 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA On Fri, Feb 14, 2014 at 11:00:31PM +0000, Stuart Yoder wrote: > > > > -----Original Message----- > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > Sent: Friday, February 14, 2014 4:27 PM > > To: Antonios Motakis > > Cc: alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; > > iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; > > tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; Yoder Stuart- > > B08248; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; > > kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood Scott-B07421; > > christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun-B16395; > > will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; Toshi > > Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Sat, Feb 08, 2014 at 06:29:31PM +0100, Antonios Motakis wrote: > > > From: Kim Phillips <kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org> > > > > > > Needed by drivers, such as the vfio platform driver [1], seeking to > > > bypass bind_store()'s driver_match_device(), and bind to any device > > > via a private sysfs bind file. > > > > > > [1] https://lkml.org/lkml/2013/12/11/522 > > > > > > note: the EXPORT_SYMBOL is needed because vfio-platform can be built > > > as a module. > > > > No code outside of drivers/base/ should be calling this function > > Why? driver_probe_device() allows a driver to explicitly bind > to a specific device. What is conceptually wrong with allowing > that? Because that's not how a bus should work, and the fact that no other subsystem in the kernel does that might be a hint you are trying to do something a bit "wrong" here. greg k-h ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <20140215024725.GA2542-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org>]
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <20140215024725.GA2542-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> @ 2014-02-15 16:33 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-02-15 16:33 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > > Why? driver_probe_device() allows a driver to explicitly bind > > to a specific device. What is conceptually wrong with allowing > > that? > > Because that's not how a bus should work, and the fact that no other > subsystem in the kernel does that might be a hint you are trying to do > something a bit "wrong" here. Let me try to succinctly as I can describe the problem we are trying to solve here... The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be exposed user space (via file descriptors), enabling user space drivers. So, for example to export an e1000 card to user space, I do this: echo 0001:03:00.0 > /sys/bus/pci/devices/0001:03:00.0/driver/unbind echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id The first step unbinds the target device (0001:03:00.0) from the normal e1000 driver. The second step causes the vfio-pci driver to bind to device 0001:03:00.0. This second step tells vfio-pci that it now handles e1000 device IDs, and the vfio-pci drivers registers with the PCI bus to handle '8086 10d3'. That works, but it is ugly. We now have 2 active drivers handling the same device type...which introduces various possible race conditions. We never want vfio-pci to auto-bind to any new device that shows up on the PCI bus. Binding a device to vfio-pci must be an explicit action by an administrator. You mentioned previously that user space can sort out the problem of multiple drivers registered for handling the same device type. That is true, but doesn't help here. We don't want vfio-pci to handle _all_ e1000 cards, just explicitly selected e1000 cards. We want the normal e1000 driver to be loaded and to bind to new devices that may be hot-plugged. There are 2 proposed mechanisms that have been put forth, both of which you have now rejected: 1. sysfs_bind_only flag was proposed which would allow a vfio driver (like vfio-pci) to only bind by explicit request through the sysfs 'bind' file. 2. Have the vfio driver call driver_probe_device() to explicitly bind a particular device instance to the driver. Only change we need here is the EXPORT_SYMBOL. Are you in principle opposed to any mechanism that would allow 2 drivers to be resident/active and allow a sysadmin to explicitly bind a particular device instance to the driver of their choice? Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-15 16:33 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-02-15 16:33 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > > Why? driver_probe_device() allows a driver to explicitly bind > > to a specific device. What is conceptually wrong with allowing > > that? > > Because that's not how a bus should work, and the fact that no other > subsystem in the kernel does that might be a hint you are trying to do > something a bit "wrong" here. Let me try to succinctly as I can describe the problem we are trying to solve here... The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be exposed user space (via file descriptors), enabling user space drivers. So, for example to export an e1000 card to user space, I do this: echo 0001:03:00.0 > /sys/bus/pci/devices/0001:03:00.0/driver/unbind echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id The first step unbinds the target device (0001:03:00.0) from the normal e1000 driver. The second step causes the vfio-pci driver to bind to device 0001:03:00.0. This second step tells vfio-pci that it now handles e1000 device IDs, and the vfio-pci drivers registers with the PCI bus to handle '8086 10d3'. That works, but it is ugly. We now have 2 active drivers handling the same device type...which introduces various possible race conditions. We never want vfio-pci to auto-bind to any new device that shows up on the PCI bus. Binding a device to vfio-pci must be an explicit action by an administrator. You mentioned previously that user space can sort out the problem of multiple drivers registered for handling the same device type. That is true, but doesn't help here. We don't want vfio-pci to handle _all_ e1000 cards, just explicitly selected e1000 cards. We want the normal e1000 driver to be loaded and to bind to new devices that may be hot-plugged. There are 2 proposed mechanisms that have been put forth, both of which you have now rejected: 1. sysfs_bind_only flag was proposed which would allow a vfio driver (like vfio-pci) to only bind by explicit request through the sysfs 'bind' file. 2. Have the vfio driver call driver_probe_device() to explicitly bind a particular device instance to the driver. Only change we need here is the EXPORT_SYMBOL. Are you in principle opposed to any mechanism that would allow 2 drivers to be resident/active and allow a sysadmin to explicitly bind a particular device instance to the driver of their choice? Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <7043e1edd9974de590dcb392cd8aff14-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org>]
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <7043e1edd9974de590dcb392cd8aff14-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> @ 2014-02-15 17:33 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-15 17:33 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > Why? driver_probe_device() allows a driver to explicitly bind > > > to a specific device. What is conceptually wrong with allowing > > > that? > > > > Because that's not how a bus should work, and the fact that no other > > subsystem in the kernel does that might be a hint you are trying to do > > something a bit "wrong" here. > > Let me try to succinctly as I can describe the problem we are trying to > solve here... > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > exposed user space (via file descriptors), enabling user space > drivers. So, for example to export an e1000 card to user space, I do > this: > > echo 0001:03:00.0 > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id What's wrong with using the "bind" file instead? That picks a specific device and binds it to a specific driver. Or have we been down this path before? :) And that is for a PCI "driver" not a totally separate bus, which it looks like you are wanting to do here. > The first step unbinds the target device (0001:03:00.0) from the normal > e1000 driver. > > The second step causes the vfio-pci driver to bind to device 0001:03:00.0. > This second step tells vfio-pci that it now handles e1000 device IDs, > and the vfio-pci drivers registers with the PCI bus to handle '8086 10d3'. > > That works, but it is ugly. We now have 2 active drivers handling > the same device type...which introduces various possible race conditions. > > We never want vfio-pci to auto-bind to any new device that shows up > on the PCI bus. Binding a device to vfio-pci must be an explicit > action by an administrator. Then use the "bind" file. > You mentioned previously that user space can sort out the problem > of multiple drivers registered for handling the same device type. > That is true, but doesn't help here. We don't want vfio-pci > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > We want the normal e1000 driver to be loaded and to bind to new > devices that may be hot-plugged. I want a pony too... > There are 2 proposed mechanisms that have been put forth, both of > which you have now rejected: > > 1. sysfs_bind_only flag was proposed which would allow a vfio > driver (like vfio-pci) to only bind by explicit request through > the sysfs 'bind' file. Why did I reject this? What did the patch look like? > 2. Have the vfio driver call driver_probe_device() to explicitly bind > a particular device instance to the driver. Only change we need > here is the EXPORT_SYMBOL. How are you going to prevent the driver from being bound to the device in the core with this change? How are you going to call this function? When? On what action of the user? > Are you in principle opposed to any mechanism that would allow 2 drivers > to be resident/active and allow a sysadmin to explicitly bind a > particular device instance to the driver of their choice? No, that works today with the bind/unbind/new_id files, it's just that you don't like it :) thanks, greg k-h ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-15 17:33 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-15 17:33 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > Why? driver_probe_device() allows a driver to explicitly bind > > > to a specific device. What is conceptually wrong with allowing > > > that? > > > > Because that's not how a bus should work, and the fact that no other > > subsystem in the kernel does that might be a hint you are trying to do > > something a bit "wrong" here. > > Let me try to succinctly as I can describe the problem we are trying to > solve here... > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > exposed user space (via file descriptors), enabling user space > drivers. So, for example to export an e1000 card to user space, I do > this: > > echo 0001:03:00.0 > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id What's wrong with using the "bind" file instead? That picks a specific device and binds it to a specific driver. Or have we been down this path before? :) And that is for a PCI "driver" not a totally separate bus, which it looks like you are wanting to do here. > The first step unbinds the target device (0001:03:00.0) from the normal > e1000 driver. > > The second step causes the vfio-pci driver to bind to device 0001:03:00.0. > This second step tells vfio-pci that it now handles e1000 device IDs, > and the vfio-pci drivers registers with the PCI bus to handle '8086 10d3'. > > That works, but it is ugly. We now have 2 active drivers handling > the same device type...which introduces various possible race conditions. > > We never want vfio-pci to auto-bind to any new device that shows up > on the PCI bus. Binding a device to vfio-pci must be an explicit > action by an administrator. Then use the "bind" file. > You mentioned previously that user space can sort out the problem > of multiple drivers registered for handling the same device type. > That is true, but doesn't help here. We don't want vfio-pci > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > We want the normal e1000 driver to be loaded and to bind to new > devices that may be hot-plugged. I want a pony too... > There are 2 proposed mechanisms that have been put forth, both of > which you have now rejected: > > 1. sysfs_bind_only flag was proposed which would allow a vfio > driver (like vfio-pci) to only bind by explicit request through > the sysfs 'bind' file. Why did I reject this? What did the patch look like? > 2. Have the vfio driver call driver_probe_device() to explicitly bind > a particular device instance to the driver. Only change we need > here is the EXPORT_SYMBOL. How are you going to prevent the driver from being bound to the device in the core with this change? How are you going to call this function? When? On what action of the user? > Are you in principle opposed to any mechanism that would allow 2 drivers > to be resident/active and allow a sysadmin to explicitly bind a > particular device instance to the driver of their choice? No, that works today with the bind/unbind/new_id files, it's just that you don't like it :) thanks, greg k-h ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <20140215173348.GA8056-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org>]
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <20140215173348.GA8056-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> @ 2014-02-15 18:19 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-02-15 18:19 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > -----Original Message----- > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > Sent: Saturday, February 15, 2014 11:34 AM > To: Yoder Stuart-B08248 > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: Re: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > Why? driver_probe_device() allows a driver to explicitly bind > > > > to a specific device. What is conceptually wrong with allowing > > > > that? > > > > > > Because that's not how a bus should work, and the fact that no other > > > subsystem in the kernel does that might be a hint you are trying to > do > > > something a bit "wrong" here. > > > > Let me try to succinctly as I can describe the problem we are trying to > > solve here... > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > > exposed user space (via file descriptors), enabling user space > > drivers. So, for example to export an e1000 card to user space, I do > > this: > > > > echo 0001:03:00.0 > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > What's wrong with using the "bind" file instead? That picks a specific > device and binds it to a specific driver. Or have we been down this > path before? :) Yes we have :) The "bind" file does not bypass device ID checks, so it wouldn't work without new_id or a wildcard match of some kind. > And that is for a PCI "driver" not a totally separate bus, which it > looks like you are wanting to do here. vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > The first step unbinds the target device (0001:03:00.0) from the normal > > e1000 driver. > > > > The second step causes the vfio-pci driver to bind to device > 0001:03:00.0. > > This second step tells vfio-pci that it now handles e1000 device IDs, > > and the vfio-pci drivers registers with the PCI bus to handle '8086 > 10d3'. > > > > That works, but it is ugly. We now have 2 active drivers handling > > the same device type...which introduces various possible race > conditions. > > > > We never want vfio-pci to auto-bind to any new device that shows up > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > action by an administrator. > > Then use the "bind" file. See above. > > You mentioned previously that user space can sort out the problem > > of multiple drivers registered for handling the same device type. > > That is true, but doesn't help here. We don't want vfio-pci > > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > > > We want the normal e1000 driver to be loaded and to bind to new > > devices that may be hot-plugged. > > I want a pony too... It's not that difficult...this patch accomplishes it by simply allowing drivers to call driver_probe_device(). > > There are 2 proposed mechanisms that have been put forth, both of > > which you have now rejected: > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > driver (like vfio-pci) to only bind by explicit request through > > the sysfs 'bind' file. > > Why did I reject this? What did the patch look like? https://lkml.org/lkml/2013/12/3/253 > > 2. Have the vfio driver call driver_probe_device() to explicitly > bind > > a particular device instance to the driver. Only change we need > > here is the EXPORT_SYMBOL. > > How are you going to prevent the driver from being bound to the device > in the core with this change? How are you going to call this function? > When? On what action of the user? The vfio-pci driver would create a sysfs object "vfio_bind". User would do: echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind vfio-pci would call driver_probe_device() which binds the specific device to the vfio-pci driver...and there is no ambiguity. > > Are you in principle opposed to any mechanism that would allow 2 > drivers > > to be resident/active and allow a sysadmin to explicitly bind a > > particular device instance to the driver of their choice? > > No, that works today with the bind/unbind/new_id files, it's just that > you don't like it :) We don't like it because of the ambiguities/race-conditions with the current situation. A vfio driver, like vfio-pci, certainly is a bit different than a normal driver, in that it really is not device ID aware. It simply passes through device resources (mappable regions, IRQs) to user space without interpreting or understanding them. It is kind of a "meta" driver, but it is not a bus. Every bus type would need its own vfio driver to do this type of device pass through. Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-15 18:19 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-02-15 18:19 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > -----Original Message----- > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > Sent: Saturday, February 15, 2014 11:34 AM > To: Yoder Stuart-B08248 > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: Re: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > Why? driver_probe_device() allows a driver to explicitly bind > > > > to a specific device. What is conceptually wrong with allowing > > > > that? > > > > > > Because that's not how a bus should work, and the fact that no other > > > subsystem in the kernel does that might be a hint you are trying to > do > > > something a bit "wrong" here. > > > > Let me try to succinctly as I can describe the problem we are trying to > > solve here... > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > > exposed user space (via file descriptors), enabling user space > > drivers. So, for example to export an e1000 card to user space, I do > > this: > > > > echo 0001:03:00.0 > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > What's wrong with using the "bind" file instead? That picks a specific > device and binds it to a specific driver. Or have we been down this > path before? :) Yes we have :) The "bind" file does not bypass device ID checks, so it wouldn't work without new_id or a wildcard match of some kind. > And that is for a PCI "driver" not a totally separate bus, which it > looks like you are wanting to do here. vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > The first step unbinds the target device (0001:03:00.0) from the normal > > e1000 driver. > > > > The second step causes the vfio-pci driver to bind to device > 0001:03:00.0. > > This second step tells vfio-pci that it now handles e1000 device IDs, > > and the vfio-pci drivers registers with the PCI bus to handle '8086 > 10d3'. > > > > That works, but it is ugly. We now have 2 active drivers handling > > the same device type...which introduces various possible race > conditions. > > > > We never want vfio-pci to auto-bind to any new device that shows up > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > action by an administrator. > > Then use the "bind" file. See above. > > You mentioned previously that user space can sort out the problem > > of multiple drivers registered for handling the same device type. > > That is true, but doesn't help here. We don't want vfio-pci > > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > > > We want the normal e1000 driver to be loaded and to bind to new > > devices that may be hot-plugged. > > I want a pony too... It's not that difficult...this patch accomplishes it by simply allowing drivers to call driver_probe_device(). > > There are 2 proposed mechanisms that have been put forth, both of > > which you have now rejected: > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > driver (like vfio-pci) to only bind by explicit request through > > the sysfs 'bind' file. > > Why did I reject this? What did the patch look like? https://lkml.org/lkml/2013/12/3/253 > > 2. Have the vfio driver call driver_probe_device() to explicitly > bind > > a particular device instance to the driver. Only change we need > > here is the EXPORT_SYMBOL. > > How are you going to prevent the driver from being bound to the device > in the core with this change? How are you going to call this function? > When? On what action of the user? The vfio-pci driver would create a sysfs object "vfio_bind". User would do: echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind vfio-pci would call driver_probe_device() which binds the specific device to the vfio-pci driver...and there is no ambiguity. > > Are you in principle opposed to any mechanism that would allow 2 > drivers > > to be resident/active and allow a sysadmin to explicitly bind a > > particular device instance to the driver of their choice? > > No, that works today with the bind/unbind/new_id files, it's just that > you don't like it :) We don't like it because of the ambiguities/race-conditions with the current situation. A vfio driver, like vfio-pci, certainly is a bit different than a normal driver, in that it really is not device ID aware. It simply passes through device resources (mappable regions, IRQs) to user space without interpreting or understanding them. It is kind of a "meta" driver, but it is not a bus. Every bus type would need its own vfio driver to do this type of device pass through. Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <38f0473542954fe8b312a1f7b61a3d21-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org>]
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <38f0473542954fe8b312a1f7b61a3d21-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> @ 2014-02-18 0:38 ` Scott Wood 0 siblings, 0 replies; 23+ messages in thread From: Scott Wood @ 2014-02-18 0:38 UTC (permalink / raw) To: Yoder Stuart-B08248 Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Sethi Varun-B16395, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Bjorn Helgaas, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, Greg KH, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org On Sat, 2014-02-15 at 12:19 -0600, Yoder Stuart-B08248 wrote: > > > -----Original Message----- > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > Sent: Saturday, February 15, 2014 11:34 AM > > To: Yoder Stuart-B08248 > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > Are you in principle opposed to any mechanism that would allow 2 > > drivers > > > to be resident/active and allow a sysadmin to explicitly bind a > > > particular device instance to the driver of their choice? > > > > No, that works today with the bind/unbind/new_id files, it's just that > > you don't like it :) > > We don't like it because of the ambiguities/race-conditions with > the current situation. Plus, it's semantically weird (a.k.a. a hack). The user isn't trying to bind an entire type of device to the vfio driver, but rather a specific device. Races and similar ugliness is often what you get when you try to pile things on top of the wrong abstraction. That you can hack around the races with a userspace loop (and hope that no damage was done by the wrong driver in the meantime -- packets sent, filesystems automounted, other inappropriate I/O performed, driver unbind bugs/unwillingness encountered, etc) is not a particularly satisfying answer. At best the race fixup will end up being a poorly tested code path (if the person scripting userspace thinks of doing it at all). It also doesn't "work today" because there is no new_id for platform devices, and the matching situation for platform devices is more complicated than on PCI, so it would be more awkward to implement and more awkward to use. We can apply enough grease and pound the square peg through the round hole if we must, but we'd like to first exhaust our options for doing it in a simple, straightforward, robust, and semantically sensible manner -- especially since once we start supporting the new_id approach for vfio binding on platform devices it'll be ABI that we're stuck with. -Scott ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-18 0:38 ` Scott Wood 0 siblings, 0 replies; 23+ messages in thread From: Scott Wood @ 2014-02-18 0:38 UTC (permalink / raw) To: Yoder Stuart-B08248 Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Sethi Varun-B16395, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Bjorn Helgaas, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, Greg KH, linux-kernel-u79uwXL29TY76Z2rM5mHXA, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org On Sat, 2014-02-15 at 12:19 -0600, Yoder Stuart-B08248 wrote: > > > -----Original Message----- > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > Sent: Saturday, February 15, 2014 11:34 AM > > To: Yoder Stuart-B08248 > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > Are you in principle opposed to any mechanism that would allow 2 > > drivers > > > to be resident/active and allow a sysadmin to explicitly bind a > > > particular device instance to the driver of their choice? > > > > No, that works today with the bind/unbind/new_id files, it's just that > > you don't like it :) > > We don't like it because of the ambiguities/race-conditions with > the current situation. Plus, it's semantically weird (a.k.a. a hack). The user isn't trying to bind an entire type of device to the vfio driver, but rather a specific device. Races and similar ugliness is often what you get when you try to pile things on top of the wrong abstraction. That you can hack around the races with a userspace loop (and hope that no damage was done by the wrong driver in the meantime -- packets sent, filesystems automounted, other inappropriate I/O performed, driver unbind bugs/unwillingness encountered, etc) is not a particularly satisfying answer. At best the race fixup will end up being a poorly tested code path (if the person scripting userspace thinks of doing it at all). It also doesn't "work today" because there is no new_id for platform devices, and the matching situation for platform devices is more complicated than on PCI, so it would be more awkward to implement and more awkward to use. We can apply enough grease and pound the square peg through the round hole if we must, but we'd like to first exhaust our options for doing it in a simple, straightforward, robust, and semantically sensible manner -- especially since once we start supporting the new_id approach for vfio binding on platform devices it'll be ABI that we're stuck with. -Scott ^ permalink raw reply [flat|nested] 23+ messages in thread
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() 2014-02-15 17:33 ` Greg KH @ 2014-02-20 22:34 ` Stuart Yoder -1 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-02-20 22:34 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > -----Original Message----- > From: Yoder Stuart-B08248 > Sent: Saturday, February 15, 2014 12:19 PM > To: 'Greg KH' > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: RE: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > > > > -----Original Message----- > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > Sent: Saturday, February 15, 2014 11:34 AM > > To: Yoder Stuart-B08248 > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > Wood > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > Why? driver_probe_device() allows a driver to explicitly bind > > > > > to a specific device. What is conceptually wrong with allowing > > > > > that? > > > > > > > > Because that's not how a bus should work, and the fact that no > other > > > > subsystem in the kernel does that might be a hint you are trying to > > do > > > > something a bit "wrong" here. > > > > > > Let me try to succinctly as I can describe the problem we are trying > to > > > solve here... > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > > > exposed user space (via file descriptors), enabling user space > > > drivers. So, for example to export an e1000 card to user space, I do > > > this: > > > > > > echo 0001:03:00.0 > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > What's wrong with using the "bind" file instead? That picks a specific > > device and binds it to a specific driver. Or have we been down this > > path before? :) > > Yes we have :) The "bind" file does not bypass device ID checks, so > it wouldn't work without new_id or a wildcard match of some kind. > > > And that is for a PCI "driver" not a totally separate bus, which it > > looks like you are wanting to do here. > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > The first step unbinds the target device (0001:03:00.0) from the > normal > > > e1000 driver. > > > > > > The second step causes the vfio-pci driver to bind to device > > 0001:03:00.0. > > > This second step tells vfio-pci that it now handles e1000 device IDs, > > > and the vfio-pci drivers registers with the PCI bus to handle '8086 > > 10d3'. > > > > > > That works, but it is ugly. We now have 2 active drivers handling > > > the same device type...which introduces various possible race > > conditions. > > > > > > We never want vfio-pci to auto-bind to any new device that shows up > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > action by an administrator. > > > > Then use the "bind" file. > > See above. > > > > You mentioned previously that user space can sort out the problem > > > of multiple drivers registered for handling the same device type. > > > That is true, but doesn't help here. We don't want vfio-pci > > > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > devices that may be hot-plugged. > > > > I want a pony too... > > It's not that difficult...this patch accomplishes it by > simply allowing drivers to call driver_probe_device(). > > > > There are 2 proposed mechanisms that have been put forth, both of > > > which you have now rejected: > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > driver (like vfio-pci) to only bind by explicit request > through > > > the sysfs 'bind' file. > > > > Why did I reject this? What did the patch look like? > > https://lkml.org/lkml/2013/12/3/253 > > > > > 2. Have the vfio driver call driver_probe_device() to explicitly > > bind > > > a particular device instance to the driver. Only change we > need > > > here is the EXPORT_SYMBOL. > > > > How are you going to prevent the driver from being bound to the device > > in the core with this change? How are you going to call this function? > > When? On what action of the user? > > The vfio-pci driver would create a sysfs object "vfio_bind". > > User would do: > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > vfio-pci would call driver_probe_device() which binds > the specific device to the vfio-pci driver...and there is > no ambiguity. > > > > Are you in principle opposed to any mechanism that would allow 2 > > drivers > > > to be resident/active and allow a sysadmin to explicitly bind a > > > particular device instance to the driver of their choice? > > > > No, that works today with the bind/unbind/new_id files, it's just that > > you don't like it :) > > We don't like it because of the ambiguities/race-conditions with > the current situation. > > A vfio driver, like vfio-pci, certainly is a bit different than a normal > driver, in that it really is not device ID aware. It simply passes > through device resources (mappable regions, IRQs) to user space without > interpreting or understanding them. It is kind of a "meta" driver, but > it is not a bus. Every bus type would need its own vfio driver to > do this type of device pass through. Hi Greg, Any further thoughts on this? Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-20 22:34 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-02-20 22:34 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > -----Original Message----- > From: Yoder Stuart-B08248 > Sent: Saturday, February 15, 2014 12:19 PM > To: 'Greg KH' > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: RE: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > > > > -----Original Message----- > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > Sent: Saturday, February 15, 2014 11:34 AM > > To: Yoder Stuart-B08248 > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > Wood > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > Why? driver_probe_device() allows a driver to explicitly bind > > > > > to a specific device. What is conceptually wrong with allowing > > > > > that? > > > > > > > > Because that's not how a bus should work, and the fact that no > other > > > > subsystem in the kernel does that might be a hint you are trying to > > do > > > > something a bit "wrong" here. > > > > > > Let me try to succinctly as I can describe the problem we are trying > to > > > solve here... > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > > > exposed user space (via file descriptors), enabling user space > > > drivers. So, for example to export an e1000 card to user space, I do > > > this: > > > > > > echo 0001:03:00.0 > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > What's wrong with using the "bind" file instead? That picks a specific > > device and binds it to a specific driver. Or have we been down this > > path before? :) > > Yes we have :) The "bind" file does not bypass device ID checks, so > it wouldn't work without new_id or a wildcard match of some kind. > > > And that is for a PCI "driver" not a totally separate bus, which it > > looks like you are wanting to do here. > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > The first step unbinds the target device (0001:03:00.0) from the > normal > > > e1000 driver. > > > > > > The second step causes the vfio-pci driver to bind to device > > 0001:03:00.0. > > > This second step tells vfio-pci that it now handles e1000 device IDs, > > > and the vfio-pci drivers registers with the PCI bus to handle '8086 > > 10d3'. > > > > > > That works, but it is ugly. We now have 2 active drivers handling > > > the same device type...which introduces various possible race > > conditions. > > > > > > We never want vfio-pci to auto-bind to any new device that shows up > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > action by an administrator. > > > > Then use the "bind" file. > > See above. > > > > You mentioned previously that user space can sort out the problem > > > of multiple drivers registered for handling the same device type. > > > That is true, but doesn't help here. We don't want vfio-pci > > > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > devices that may be hot-plugged. > > > > I want a pony too... > > It's not that difficult...this patch accomplishes it by > simply allowing drivers to call driver_probe_device(). > > > > There are 2 proposed mechanisms that have been put forth, both of > > > which you have now rejected: > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > driver (like vfio-pci) to only bind by explicit request > through > > > the sysfs 'bind' file. > > > > Why did I reject this? What did the patch look like? > > https://lkml.org/lkml/2013/12/3/253 > > > > > 2. Have the vfio driver call driver_probe_device() to explicitly > > bind > > > a particular device instance to the driver. Only change we > need > > > here is the EXPORT_SYMBOL. > > > > How are you going to prevent the driver from being bound to the device > > in the core with this change? How are you going to call this function? > > When? On what action of the user? > > The vfio-pci driver would create a sysfs object "vfio_bind". > > User would do: > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > vfio-pci would call driver_probe_device() which binds > the specific device to the vfio-pci driver...and there is > no ambiguity. > > > > Are you in principle opposed to any mechanism that would allow 2 > > drivers > > > to be resident/active and allow a sysadmin to explicitly bind a > > > particular device instance to the driver of their choice? > > > > No, that works today with the bind/unbind/new_id files, it's just that > > you don't like it :) > > We don't like it because of the ambiguities/race-conditions with > the current situation. > > A vfio driver, like vfio-pci, certainly is a bit different than a normal > driver, in that it really is not device ID aware. It simply passes > through device resources (mappable regions, IRQs) to user space without > interpreting or understanding them. It is kind of a "meta" driver, but > it is not a bus. Every bus type would need its own vfio driver to > do this type of device pass through. Hi Greg, Any further thoughts on this? Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <b6374a0f30194969ba4622ff2f58ae65-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org>]
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <b6374a0f30194969ba4622ff2f58ae65-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> @ 2014-02-20 22:43 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-20 22:43 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA On Thu, Feb 20, 2014 at 10:34:35PM +0000, Stuart Yoder wrote: > > > > -----Original Message----- > > From: Yoder Stuart-B08248 > > Sent: Saturday, February 15, 2014 12:19 PM > > To: 'Greg KH' > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: RE: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > > > > > > -----Original Message----- > > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > > Sent: Saturday, February 15, 2014 11:34 AM > > > To: Yoder Stuart-B08248 > > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > > Wood > > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > > Roeck; > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > > driver_probe_device() > > > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > > Why? driver_probe_device() allows a driver to explicitly bind > > > > > > to a specific device. What is conceptually wrong with allowing > > > > > > that? > > > > > > > > > > Because that's not how a bus should work, and the fact that no > > other > > > > > subsystem in the kernel does that might be a hint you are trying to > > > do > > > > > something a bit "wrong" here. > > > > > > > > Let me try to succinctly as I can describe the problem we are trying > > to > > > > solve here... > > > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > > > > exposed user space (via file descriptors), enabling user space > > > > drivers. So, for example to export an e1000 card to user space, I do > > > > this: > > > > > > > > echo 0001:03:00.0 > > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > > > What's wrong with using the "bind" file instead? That picks a specific > > > device and binds it to a specific driver. Or have we been down this > > > path before? :) > > > > Yes we have :) The "bind" file does not bypass device ID checks, so > > it wouldn't work without new_id or a wildcard match of some kind. > > > > > And that is for a PCI "driver" not a totally separate bus, which it > > > looks like you are wanting to do here. > > > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > > > The first step unbinds the target device (0001:03:00.0) from the > > normal > > > > e1000 driver. > > > > > > > > The second step causes the vfio-pci driver to bind to device > > > 0001:03:00.0. > > > > This second step tells vfio-pci that it now handles e1000 device IDs, > > > > and the vfio-pci drivers registers with the PCI bus to handle '8086 > > > 10d3'. > > > > > > > > That works, but it is ugly. We now have 2 active drivers handling > > > > the same device type...which introduces various possible race > > > conditions. > > > > > > > > We never want vfio-pci to auto-bind to any new device that shows up > > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > > action by an administrator. > > > > > > Then use the "bind" file. > > > > See above. > > > > > > You mentioned previously that user space can sort out the problem > > > > of multiple drivers registered for handling the same device type. > > > > That is true, but doesn't help here. We don't want vfio-pci > > > > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > > devices that may be hot-plugged. > > > > > > I want a pony too... > > > > It's not that difficult...this patch accomplishes it by > > simply allowing drivers to call driver_probe_device(). > > > > > > There are 2 proposed mechanisms that have been put forth, both of > > > > which you have now rejected: > > > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > > driver (like vfio-pci) to only bind by explicit request > > through > > > > the sysfs 'bind' file. > > > > > > Why did I reject this? What did the patch look like? > > > > https://lkml.org/lkml/2013/12/3/253 > > > > > > > > 2. Have the vfio driver call driver_probe_device() to explicitly > > > bind > > > > a particular device instance to the driver. Only change we > > need > > > > here is the EXPORT_SYMBOL. > > > > > > How are you going to prevent the driver from being bound to the device > > > in the core with this change? How are you going to call this function? > > > When? On what action of the user? > > > > The vfio-pci driver would create a sysfs object "vfio_bind". > > > > User would do: > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > > > vfio-pci would call driver_probe_device() which binds > > the specific device to the vfio-pci driver...and there is > > no ambiguity. > > > > > > Are you in principle opposed to any mechanism that would allow 2 > > > drivers > > > > to be resident/active and allow a sysadmin to explicitly bind a > > > > particular device instance to the driver of their choice? > > > > > > No, that works today with the bind/unbind/new_id files, it's just that > > > you don't like it :) > > > > We don't like it because of the ambiguities/race-conditions with > > the current situation. > > > > A vfio driver, like vfio-pci, certainly is a bit different than a normal > > driver, in that it really is not device ID aware. It simply passes > > through device resources (mappable regions, IRQs) to user space without > > interpreting or understanding them. It is kind of a "meta" driver, but > > it is not a bus. Every bus type would need its own vfio driver to > > do this type of device pass through. > > Hi Greg, > > Any further thoughts on this? Sorry, been swamped with other patches and stable stuff and not had a time to look at it. Give me a few days... ^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-02-20 22:43 ` Greg KH 0 siblings, 0 replies; 23+ messages in thread From: Greg KH @ 2014-02-20 22:43 UTC (permalink / raw) To: Stuart Yoder Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA On Thu, Feb 20, 2014 at 10:34:35PM +0000, Stuart Yoder wrote: > > > > -----Original Message----- > > From: Yoder Stuart-B08248 > > Sent: Saturday, February 15, 2014 12:19 PM > > To: 'Greg KH' > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > Subject: RE: [RFC PATCH v4 01/10] driver core: export > > driver_probe_device() > > > > > > > > > -----Original Message----- > > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > > Sent: Saturday, February 15, 2014 11:34 AM > > > To: Yoder Stuart-B08248 > > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > > Wood > > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > > Roeck; > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > > driver_probe_device() > > > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > > Why? driver_probe_device() allows a driver to explicitly bind > > > > > > to a specific device. What is conceptually wrong with allowing > > > > > > that? > > > > > > > > > > Because that's not how a bus should work, and the fact that no > > other > > > > > subsystem in the kernel does that might be a hint you are trying to > > > do > > > > > something a bit "wrong" here. > > > > > > > > Let me try to succinctly as I can describe the problem we are trying > > to > > > > solve here... > > > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices to be > > > > exposed user space (via file descriptors), enabling user space > > > > drivers. So, for example to export an e1000 card to user space, I do > > > > this: > > > > > > > > echo 0001:03:00.0 > > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > > > What's wrong with using the "bind" file instead? That picks a specific > > > device and binds it to a specific driver. Or have we been down this > > > path before? :) > > > > Yes we have :) The "bind" file does not bypass device ID checks, so > > it wouldn't work without new_id or a wildcard match of some kind. > > > > > And that is for a PCI "driver" not a totally separate bus, which it > > > looks like you are wanting to do here. > > > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > > > The first step unbinds the target device (0001:03:00.0) from the > > normal > > > > e1000 driver. > > > > > > > > The second step causes the vfio-pci driver to bind to device > > > 0001:03:00.0. > > > > This second step tells vfio-pci that it now handles e1000 device IDs, > > > > and the vfio-pci drivers registers with the PCI bus to handle '8086 > > > 10d3'. > > > > > > > > That works, but it is ugly. We now have 2 active drivers handling > > > > the same device type...which introduces various possible race > > > conditions. > > > > > > > > We never want vfio-pci to auto-bind to any new device that shows up > > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > > action by an administrator. > > > > > > Then use the "bind" file. > > > > See above. > > > > > > You mentioned previously that user space can sort out the problem > > > > of multiple drivers registered for handling the same device type. > > > > That is true, but doesn't help here. We don't want vfio-pci > > > > to handle _all_ e1000 cards, just explicitly selected e1000 cards. > > > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > > devices that may be hot-plugged. > > > > > > I want a pony too... > > > > It's not that difficult...this patch accomplishes it by > > simply allowing drivers to call driver_probe_device(). > > > > > > There are 2 proposed mechanisms that have been put forth, both of > > > > which you have now rejected: > > > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > > driver (like vfio-pci) to only bind by explicit request > > through > > > > the sysfs 'bind' file. > > > > > > Why did I reject this? What did the patch look like? > > > > https://lkml.org/lkml/2013/12/3/253 > > > > > > > > 2. Have the vfio driver call driver_probe_device() to explicitly > > > bind > > > > a particular device instance to the driver. Only change we > > need > > > > here is the EXPORT_SYMBOL. > > > > > > How are you going to prevent the driver from being bound to the device > > > in the core with this change? How are you going to call this function? > > > When? On what action of the user? > > > > The vfio-pci driver would create a sysfs object "vfio_bind". > > > > User would do: > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > > > vfio-pci would call driver_probe_device() which binds > > the specific device to the vfio-pci driver...and there is > > no ambiguity. > > > > > > Are you in principle opposed to any mechanism that would allow 2 > > > drivers > > > > to be resident/active and allow a sysadmin to explicitly bind a > > > > particular device instance to the driver of their choice? > > > > > > No, that works today with the bind/unbind/new_id files, it's just that > > > you don't like it :) > > > > We don't like it because of the ambiguities/race-conditions with > > the current situation. > > > > A vfio driver, like vfio-pci, certainly is a bit different than a normal > > driver, in that it really is not device ID aware. It simply passes > > through device resources (mappable regions, IRQs) to user space without > > interpreting or understanding them. It is kind of a "meta" driver, but > > it is not a bus. Every bus type would need its own vfio driver to > > do this type of device pass through. > > Hi Greg, > > Any further thoughts on this? Sorry, been swamped with other patches and stable stuff and not had a time to look at it. Give me a few days... ^ permalink raw reply [flat|nested] 23+ messages in thread
[parent not found: <20140220224337.GA20097-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org>]
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() [not found] ` <20140220224337.GA20097-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> @ 2014-03-06 22:25 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-03-06 22:25 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > -----Original Message----- > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > Sent: Thursday, February 20, 2014 4:44 PM > To: Yoder Stuart-B08248 > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: Re: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > On Thu, Feb 20, 2014 at 10:34:35PM +0000, Stuart Yoder wrote: > > > > > > > -----Original Message----- > > > From: Yoder Stuart-B08248 > > > Sent: Saturday, February 15, 2014 12:19 PM > > > To: 'Greg KH' > > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; > linux- > > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > Wood > > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi > Varun- > > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > Roeck; > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > > Subject: RE: [RFC PATCH v4 01/10] driver core: export > > > driver_probe_device() > > > > > > > > > > > > > -----Original Message----- > > > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > > > Sent: Saturday, February 15, 2014 11:34 AM > > > > To: Yoder Stuart-B08248 > > > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; > linux- > > > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > > > Wood > > > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi > Varun- > > > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > > > Roeck; > > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn > Helgaas > > > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > > > driver_probe_device() > > > > > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > > > Why? driver_probe_device() allows a driver to explicitly > bind > > > > > > > to a specific device. What is conceptually wrong with > allowing > > > > > > > that? > > > > > > > > > > > > Because that's not how a bus should work, and the fact that no > > > other > > > > > > subsystem in the kernel does that might be a hint you are > trying to > > > > do > > > > > > something a bit "wrong" here. > > > > > > > > > > Let me try to succinctly as I can describe the problem we are > trying > > > to > > > > > solve here... > > > > > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices > to be > > > > > exposed user space (via file descriptors), enabling user space > > > > > drivers. So, for example to export an e1000 card to user space, > I do > > > > > this: > > > > > > > > > > echo 0001:03:00.0 > > > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > > > > > What's wrong with using the "bind" file instead? That picks a > specific > > > > device and binds it to a specific driver. Or have we been down > this > > > > path before? :) > > > > > > Yes we have :) The "bind" file does not bypass device ID checks, so > > > it wouldn't work without new_id or a wildcard match of some kind. > > > > > > > And that is for a PCI "driver" not a totally separate bus, which it > > > > looks like you are wanting to do here. > > > > > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > > > > > The first step unbinds the target device (0001:03:00.0) from the > > > normal > > > > > e1000 driver. > > > > > > > > > > The second step causes the vfio-pci driver to bind to device > > > > 0001:03:00.0. > > > > > This second step tells vfio-pci that it now handles e1000 device > IDs, > > > > > and the vfio-pci drivers registers with the PCI bus to handle > '8086 > > > > 10d3'. > > > > > > > > > > That works, but it is ugly. We now have 2 active drivers > handling > > > > > the same device type...which introduces various possible race > > > > conditions. > > > > > > > > > > We never want vfio-pci to auto-bind to any new device that shows > up > > > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > > > action by an administrator. > > > > > > > > Then use the "bind" file. > > > > > > See above. > > > > > > > > You mentioned previously that user space can sort out the problem > > > > > of multiple drivers registered for handling the same device type. > > > > > That is true, but doesn't help here. We don't want vfio-pci > > > > > to handle _all_ e1000 cards, just explicitly selected e1000 > cards. > > > > > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > > > devices that may be hot-plugged. > > > > > > > > I want a pony too... > > > > > > It's not that difficult...this patch accomplishes it by > > > simply allowing drivers to call driver_probe_device(). > > > > > > > > There are 2 proposed mechanisms that have been put forth, both of > > > > > which you have now rejected: > > > > > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > > > driver (like vfio-pci) to only bind by explicit request > > > through > > > > > the sysfs 'bind' file. > > > > > > > > Why did I reject this? What did the patch look like? > > > > > > https://lkml.org/lkml/2013/12/3/253 > > > > > > > > > > > 2. Have the vfio driver call driver_probe_device() to > explicitly > > > > bind > > > > > a particular device instance to the driver. Only change > we > > > need > > > > > here is the EXPORT_SYMBOL. > > > > > > > > How are you going to prevent the driver from being bound to the > device > > > > in the core with this change? How are you going to call this > function? > > > > When? On what action of the user? > > > > > > The vfio-pci driver would create a sysfs object "vfio_bind". > > > > > > User would do: > > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > > > > > vfio-pci would call driver_probe_device() which binds > > > the specific device to the vfio-pci driver...and there is > > > no ambiguity. > > > > > > > > Are you in principle opposed to any mechanism that would allow 2 > > > > drivers > > > > > to be resident/active and allow a sysadmin to explicitly bind a > > > > > particular device instance to the driver of their choice? > > > > > > > > No, that works today with the bind/unbind/new_id files, it's just > that > > > > you don't like it :) > > > > > > We don't like it because of the ambiguities/race-conditions with > > > the current situation. > > > > > > A vfio driver, like vfio-pci, certainly is a bit different than a > normal > > > driver, in that it really is not device ID aware. It simply passes > > > through device resources (mappable regions, IRQs) to user space > without > > > interpreting or understanding them. It is kind of a "meta" driver, > but > > > it is not a bus. Every bus type would need its own vfio driver to > > > do this type of device pass through. > > > > Hi Greg, > > > > Any further thoughts on this? > > Sorry, been swamped with other patches and stable stuff and not had a > time to look at it. Give me a few days... Hi Greg, wanted to ping you on this again... I know some days have gone by, so let me summarize the issue-- vfio drivers in the kernel (regardless of bus type) need to bind to devices of any type. There seem to be 3 approaches: 1. new_id -- (current approach) the user explicitly registers each new device type with the vfio driver using the new_id mechanism. Problem: multiple drivers will be resident that handle the same device type...and there is nothing user space hotplug infrastructure can do to help. 2. "any id" -- the vfio driver could specify a wildcard match of some kind so that it can bind to any possible device id. However, we don't want vfio grabbing all devices...just the ones we explicitly want to pass to user space. Proposed patch to support this was to create a new flag "sysfs_bind_only" in struct device_driver. When this flag is set, the driver can only bind to devices via the sysfs bind file. This would allow the wildcard match to work. Patch is here: https://lkml.org/lkml/2013/12/3/253 3. Driver initiated explicit bind -- with this approach the vfio driver would create a private 'bind' sysfs object and the user would echo the requested device into it: echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind In order to make that work, the driver would need to call driver_probe_device() and thus we need this patch: https://lkml.org/lkml/2014/2/8/175 Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
* RE: [RFC PATCH v4 01/10] driver core: export driver_probe_device() @ 2014-03-06 22:25 ` Stuart Yoder 0 siblings, 0 replies; 23+ messages in thread From: Stuart Yoder @ 2014-03-06 22:25 UTC (permalink / raw) To: Greg KH Cc: kvm-u79uwXL29TY76Z2rM5mHXA, jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ, will.deacon-5wv7dgnIgG8, linux-kernel-u79uwXL29TY76Z2rM5mHXA, Bjorn Helgaas, Varun Sethi, kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg, Rafael J. Wysocki, agraf-l3A5Bk7waGM, Guenter Roeck, Dmitry Kasatkin, Tejun Heo, Scott Wood, Antonios Motakis, tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, Michal Hocko, Toshi Kani, a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J, iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA > -----Original Message----- > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > Sent: Thursday, February 20, 2014 4:44 PM > To: Yoder Stuart-B08248 > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; linux- > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; Wood > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi Varun- > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter Roeck; > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > Subject: Re: [RFC PATCH v4 01/10] driver core: export > driver_probe_device() > > On Thu, Feb 20, 2014 at 10:34:35PM +0000, Stuart Yoder wrote: > > > > > > > -----Original Message----- > > > From: Yoder Stuart-B08248 > > > Sent: Saturday, February 15, 2014 12:19 PM > > > To: 'Greg KH' > > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; > linux- > > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > Wood > > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi > Varun- > > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > Roeck; > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn Helgaas > > > Subject: RE: [RFC PATCH v4 01/10] driver core: export > > > driver_probe_device() > > > > > > > > > > > > > -----Original Message----- > > > > From: Greg KH [mailto:gregkh-hQyY1W1yCW8ekmWlsbkhG0B+6BGkLq7r@public.gmane.org] > > > > Sent: Saturday, February 15, 2014 11:34 AM > > > > To: Yoder Stuart-B08248 > > > > Cc: Antonios Motakis; alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org; > > > > kvmarm-FPEHb7Xf0XXUo1n7N8X6UoWGPAHP3yOg@public.gmane.org; iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org; > linux- > > > > kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; tech-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; > > > > a.rigo-lrHrjnjw1UfHK3s98zE1ajGjJy/sRE9J@public.gmane.org; kim.phillips-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; > > > > jan.kiszka-kv7WeFo6aLtBDgjK7y7TUQ@public.gmane.org; kvm-u79uwXL29TY76Z2rM5mHXA@public.gmane.org; Bhushan Bharat-R65777; > > > Wood > > > > Scott-B07421; christoffer.dall-QSEj5FYQhm4dnm+yROfE0A@public.gmane.org; agraf-l3A5Bk7waGM@public.gmane.org; Sethi > Varun- > > > > B16395; will.deacon-5wv7dgnIgG8@public.gmane.org; Tejun Heo; Rafael J. Wysocki; Guenter > > > Roeck; > > > > Toshi Kani; Joe Perches; Dmitry Kasatkin; Michal Hocko; Bjorn > Helgaas > > > > Subject: Re: [RFC PATCH v4 01/10] driver core: export > > > > driver_probe_device() > > > > > > > > On Sat, Feb 15, 2014 at 04:33:44PM +0000, Stuart Yoder wrote: > > > > > > > Why? driver_probe_device() allows a driver to explicitly > bind > > > > > > > to a specific device. What is conceptually wrong with > allowing > > > > > > > that? > > > > > > > > > > > > Because that's not how a bus should work, and the fact that no > > > other > > > > > > subsystem in the kernel does that might be a hint you are > trying to > > > > do > > > > > > something a bit "wrong" here. > > > > > > > > > > Let me try to succinctly as I can describe the problem we are > trying > > > to > > > > > solve here... > > > > > > > > > > The vfio mechanism in the kernel (e.g. vfio-pci) allows devices > to be > > > > > exposed user space (via file descriptors), enabling user space > > > > > drivers. So, for example to export an e1000 card to user space, > I do > > > > > this: > > > > > > > > > > echo 0001:03:00.0 > > > > /sys/bus/pci/devices/0001:03:00.0/driver/unbind > > > > > echo 8086 10d3 > /sys/bus/pci/drivers/vfio-pci/new_id > > > > > > > > What's wrong with using the "bind" file instead? That picks a > specific > > > > device and binds it to a specific driver. Or have we been down > this > > > > path before? :) > > > > > > Yes we have :) The "bind" file does not bypass device ID checks, so > > > it wouldn't work without new_id or a wildcard match of some kind. > > > > > > > And that is for a PCI "driver" not a totally separate bus, which it > > > > looks like you are wanting to do here. > > > > > > vfio-pci is a PCI driver, not a bus (drivers/vfio/pci/vfio_pci.c). > > > > > > > > The first step unbinds the target device (0001:03:00.0) from the > > > normal > > > > > e1000 driver. > > > > > > > > > > The second step causes the vfio-pci driver to bind to device > > > > 0001:03:00.0. > > > > > This second step tells vfio-pci that it now handles e1000 device > IDs, > > > > > and the vfio-pci drivers registers with the PCI bus to handle > '8086 > > > > 10d3'. > > > > > > > > > > That works, but it is ugly. We now have 2 active drivers > handling > > > > > the same device type...which introduces various possible race > > > > conditions. > > > > > > > > > > We never want vfio-pci to auto-bind to any new device that shows > up > > > > > on the PCI bus. Binding a device to vfio-pci must be an explicit > > > > > action by an administrator. > > > > > > > > Then use the "bind" file. > > > > > > See above. > > > > > > > > You mentioned previously that user space can sort out the problem > > > > > of multiple drivers registered for handling the same device type. > > > > > That is true, but doesn't help here. We don't want vfio-pci > > > > > to handle _all_ e1000 cards, just explicitly selected e1000 > cards. > > > > > > > > > > We want the normal e1000 driver to be loaded and to bind to new > > > > > devices that may be hot-plugged. > > > > > > > > I want a pony too... > > > > > > It's not that difficult...this patch accomplishes it by > > > simply allowing drivers to call driver_probe_device(). > > > > > > > > There are 2 proposed mechanisms that have been put forth, both of > > > > > which you have now rejected: > > > > > > > > > > 1. sysfs_bind_only flag was proposed which would allow a vfio > > > > > driver (like vfio-pci) to only bind by explicit request > > > through > > > > > the sysfs 'bind' file. > > > > > > > > Why did I reject this? What did the patch look like? > > > > > > https://lkml.org/lkml/2013/12/3/253 > > > > > > > > > > > 2. Have the vfio driver call driver_probe_device() to > explicitly > > > > bind > > > > > a particular device instance to the driver. Only change > we > > > need > > > > > here is the EXPORT_SYMBOL. > > > > > > > > How are you going to prevent the driver from being bound to the > device > > > > in the core with this change? How are you going to call this > function? > > > > When? On what action of the user? > > > > > > The vfio-pci driver would create a sysfs object "vfio_bind". > > > > > > User would do: > > > echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind > > > > > > vfio-pci would call driver_probe_device() which binds > > > the specific device to the vfio-pci driver...and there is > > > no ambiguity. > > > > > > > > Are you in principle opposed to any mechanism that would allow 2 > > > > drivers > > > > > to be resident/active and allow a sysadmin to explicitly bind a > > > > > particular device instance to the driver of their choice? > > > > > > > > No, that works today with the bind/unbind/new_id files, it's just > that > > > > you don't like it :) > > > > > > We don't like it because of the ambiguities/race-conditions with > > > the current situation. > > > > > > A vfio driver, like vfio-pci, certainly is a bit different than a > normal > > > driver, in that it really is not device ID aware. It simply passes > > > through device resources (mappable regions, IRQs) to user space > without > > > interpreting or understanding them. It is kind of a "meta" driver, > but > > > it is not a bus. Every bus type would need its own vfio driver to > > > do this type of device pass through. > > > > Hi Greg, > > > > Any further thoughts on this? > > Sorry, been swamped with other patches and stable stuff and not had a > time to look at it. Give me a few days... Hi Greg, wanted to ping you on this again... I know some days have gone by, so let me summarize the issue-- vfio drivers in the kernel (regardless of bus type) need to bind to devices of any type. There seem to be 3 approaches: 1. new_id -- (current approach) the user explicitly registers each new device type with the vfio driver using the new_id mechanism. Problem: multiple drivers will be resident that handle the same device type...and there is nothing user space hotplug infrastructure can do to help. 2. "any id" -- the vfio driver could specify a wildcard match of some kind so that it can bind to any possible device id. However, we don't want vfio grabbing all devices...just the ones we explicitly want to pass to user space. Proposed patch to support this was to create a new flag "sysfs_bind_only" in struct device_driver. When this flag is set, the driver can only bind to devices via the sysfs bind file. This would allow the wildcard match to work. Patch is here: https://lkml.org/lkml/2013/12/3/253 3. Driver initiated explicit bind -- with this approach the vfio driver would create a private 'bind' sysfs object and the user would echo the requested device into it: echo 0001:03:00.0 > /sys/bus/pci/drivers/vfio-pci/vfio_bind In order to make that work, the driver would need to call driver_probe_device() and thus we need this patch: https://lkml.org/lkml/2014/2/8/175 Thanks, Stuart ^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2014-03-06 22:31 UTC | newest] Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2014-03-06 22:31 [RFC PATCH v4 01/10] driver core: export driver_probe_device() Konrad Rzeszutek Wilk -- strict thread matches above, loose matches on Subject: below -- 2014-03-06 22:31 Konrad Rzeszutek Wilk 2014-02-08 17:29 [RFC PATCH v4 00/10] VFIO support for platform devices Antonios Motakis 2014-02-08 17:29 ` [RFC PATCH v4 01/10] driver core: export driver_probe_device() Antonios Motakis 2014-02-08 17:29 ` Antonios Motakis 2014-02-14 22:27 ` Greg KH 2014-02-14 22:27 ` Greg KH [not found] ` <20140214222716.GA11838-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> 2014-02-14 23:00 ` Stuart Yoder [not found] ` <ba7597fd8c9f4d91bbccfb42e31a165e-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> 2014-02-15 2:47 ` Greg KH 2014-02-15 2:47 ` Greg KH [not found] ` <20140215024725.GA2542-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> 2014-02-15 16:33 ` Stuart Yoder 2014-02-15 16:33 ` Stuart Yoder [not found] ` <7043e1edd9974de590dcb392cd8aff14-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> 2014-02-15 17:33 ` Greg KH 2014-02-15 17:33 ` Greg KH [not found] ` <20140215173348.GA8056-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> 2014-02-15 18:19 ` Stuart Yoder 2014-02-15 18:19 ` Stuart Yoder [not found] ` <38f0473542954fe8b312a1f7b61a3d21-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> 2014-02-18 0:38 ` Scott Wood 2014-02-18 0:38 ` Scott Wood 2014-02-20 22:34 ` Stuart Yoder 2014-02-20 22:34 ` Stuart Yoder [not found] ` <b6374a0f30194969ba4622ff2f58ae65-ufbTtyGzTTT8GZusEWM6WuO6mTEJWrR4XA4E9RH9d+qIuWR1G4zioA@public.gmane.org> 2014-02-20 22:43 ` Greg KH 2014-02-20 22:43 ` Greg KH [not found] ` <20140220224337.GA20097-U8xfFu+wG4EAvxtiuMwx3w@public.gmane.org> 2014-03-06 22:25 ` Stuart Yoder 2014-03-06 22:25 ` Stuart Yoder
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.