All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] don't allow vfio drivers to bind on driver_attach
@ 2016-01-12 21:33 Jacob Keller
  2016-01-12 21:33 ` [PATCH] driver: add manual_bind_only option for virtual stub drivers Jacob Keller
  2016-01-12 22:06 ` [PATCH] don't allow vfio drivers to bind on driver_attach Alex Williamson
  0 siblings, 2 replies; 7+ messages in thread
From: Jacob Keller @ 2016-01-12 21:33 UTC (permalink / raw)
  To: linux-pci
  Cc: Jacob Keller, Alex Williamson, Kay Sievers, Rafael J . Wysocki,
	Alan Stern, Arjan van de Ven

This patch is an attempt to resolve an issue which can occur when using
vfio-pci (and vfio-platform) with IOMMU direct assignment.

When attempting to directly assign a device to a virtual machine, the
normal flow is something like the following:

echo <pci address> > /sys/bus/pci/drivers/<old driver>/unbind
echo <device id> > /sys/bus/pci/drivers/vfio-pci/new_id
echo <pci address> > /sys/bus/pci/drivers/vfio-pci/bind

This process results in the device being bound to the vfio-pci stub
driver, and then this device can be safely given to a virtual machine.

The issue can occur, that if <device id> doesn't currently have a driver
loaded, (like say the module was removed), the process of adding
<new_id> can result in *all* devices which match that id being bound to
vfio-pci. This may not seem like a problem, but it can be confusing when
you have a dual-port device, such as a networking card. In some use
cases, you only want to assign a single port to a VM and not all the
ports. If you happen to not have the driver loaded already, the result
is that even loading the driver later will not gain control of the 2nd
port, because the device is already bound to vfio-pci. The solution (if
you know this is the case) would be to unbind that particular device
from vfio and bind it to the real driver.

This patch fixes the issue by instead requiring vfio-pci and
vfio-platform to only bind to devices given via the sysfs bind route,
and prevents bind to any devices in other scenarios. The result is that
vfio-pci will never bind a device unless explicitly requested. Given the
nature of what vfio-pci does, I think this is a much better solution.

I've tried to Cc several people who've made changes to this area of th
kernel for more feedback.

Jacob Keller (1):
  driver: add manual_bind_only option for virtual stub drivers

 drivers/base/dd.c                     | 6 ++++++
 drivers/vfio/pci/vfio_pci.c           | 3 +++
 drivers/vfio/platform/vfio_platform.c | 1 +
 include/linux/device.h                | 1 +
 4 files changed, 11 insertions(+)

-- 
2.6.3.505.g5cc1fd1


^ permalink raw reply	[flat|nested] 7+ messages in thread

* [PATCH] driver: add manual_bind_only option for virtual stub drivers
  2016-01-12 21:33 [PATCH] don't allow vfio drivers to bind on driver_attach Jacob Keller
@ 2016-01-12 21:33 ` Jacob Keller
  2016-01-12 21:53   ` kbuild test robot
  2016-01-13  0:02   ` kbuild test robot
  2016-01-12 22:06 ` [PATCH] don't allow vfio drivers to bind on driver_attach Alex Williamson
  1 sibling, 2 replies; 7+ messages in thread
From: Jacob Keller @ 2016-01-12 21:33 UTC (permalink / raw)
  To: linux-pci; +Cc: Jacob Keller

The vfio-pci and vfio-platform drivers are intended to be used as stub
drivers which can bind to any pci or platform device and are used to
enable direct assignment of a host device to a guest virtual machine
using IOMMU. However, today, these drivers will bind on device or driver
attach, once given a dynamic id to match against. This can cause
problems as the drivers may attach to devices that match an id but
haven't specifically been requested using the sysfs bind interface.

Add a boolean "manual_bind_only" which can be set by drivers which
should only bind to devices if they have been explicitly added using the
sysfs bind flow. This prevents these drivers from taking control of
devices unintentionally.

Signed-off-by: Jacob Keller <jacob.e.keller@intel.com>
---
 drivers/base/dd.c                     | 6 ++++++
 drivers/vfio/pci/vfio_pci.c           | 3 +++
 drivers/vfio/platform/vfio_platform.c | 1 +
 include/linux/device.h                | 1 +
 4 files changed, 11 insertions(+)

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index a641cf3ccad6..e21bf1d67168 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -491,6 +491,9 @@ static int __device_attach_driver(struct device_driver *drv, void *_data)
 	struct device *dev = data->dev;
 	bool async_allowed;
 
+	if (drv->manual_bind_only)
+		return 0;
+
 	/*
 	 * Check if device has already been claimed. This may
 	 * happen with driver loading, device discovery/registration,
@@ -632,6 +635,9 @@ static int __driver_attach(struct device *dev, void *data)
 	 * is an error.
 	 */
 
+	if (drv->manual_bind_only)
+		return 0;
+
 	if (!driver_match_device(drv, dev))
 		return 0;
 
diff --git a/drivers/vfio/pci/vfio_pci.c b/drivers/vfio/pci/vfio_pci.c
index 56bf6dbb93db..82f139854b56 100644
--- a/drivers/vfio/pci/vfio_pci.c
+++ b/drivers/vfio/pci/vfio_pci.c
@@ -1045,6 +1045,9 @@ static struct pci_driver vfio_pci_driver = {
 	.probe		= vfio_pci_probe,
 	.remove		= vfio_pci_remove,
 	.err_handler	= &vfio_err_handlers,
+	.driver = {
+		.manual_bind_only = true;
+	},
 };
 
 struct vfio_devices {
diff --git a/drivers/vfio/platform/vfio_platform.c b/drivers/vfio/platform/vfio_platform.c
index b1cc3a768784..91138ac6d1a8 100644
--- a/drivers/vfio/platform/vfio_platform.c
+++ b/drivers/vfio/platform/vfio_platform.c
@@ -92,6 +92,7 @@ static struct platform_driver vfio_platform_driver = {
 	.remove		= vfio_platform_remove,
 	.driver	= {
 		.name	= "vfio-platform",
+		.manual_bind_only = true;
 	},
 };
 
diff --git a/include/linux/device.h b/include/linux/device.h
index b8f411b57dcb..de755bb64994 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -264,6 +264,7 @@ struct device_driver {
 	const char		*mod_name;	/* used for built-in modules */
 
 	bool suppress_bind_attrs;	/* disables bind/unbind via sysfs */
+	bool manual_bind_only;		/* prevent bind on driver_attach */
 	enum probe_type probe_type;
 
 	const struct of_device_id	*of_match_table;
-- 
2.6.3.505.g5cc1fd1


^ permalink raw reply related	[flat|nested] 7+ messages in thread

* Re: [PATCH] driver: add manual_bind_only option for virtual stub drivers
  2016-01-12 21:33 ` [PATCH] driver: add manual_bind_only option for virtual stub drivers Jacob Keller
@ 2016-01-12 21:53   ` kbuild test robot
  2016-01-13  0:02   ` kbuild test robot
  1 sibling, 0 replies; 7+ messages in thread
From: kbuild test robot @ 2016-01-12 21:53 UTC (permalink / raw)
  To: Jacob Keller; +Cc: kbuild-all, linux-pci, Jacob Keller

[-- Attachment #1: Type: text/plain, Size: 1202 bytes --]

Hi Jacob,

[auto build test ERROR on vfio/next]
[also build test ERROR on v4.4 next-20160112]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Jacob-Keller/driver-add-manual_bind_only-option-for-virtual-stub-drivers/20160113-053451
base:   https://github.com/awilliam/linux-vfio.git next
config: x86_64-randconfig-x012-01110856 (attached as .config)
reproduce:
        # save the attached .config to linux build tree
        make ARCH=x86_64 

All errors (new ones prefixed by >>):

>> drivers/vfio/pci/vfio_pci.c:1049:27: error: expected '}' before ';' token
      .manual_bind_only = true;
                              ^

vim +1049 drivers/vfio/pci/vfio_pci.c

  1043		.name		= "vfio-pci",
  1044		.id_table	= NULL, /* only dynamic ids */
  1045		.probe		= vfio_pci_probe,
  1046		.remove		= vfio_pci_remove,
  1047		.err_handler	= &vfio_err_handlers,
  1048		.driver = {
> 1049			.manual_bind_only = true;
  1050		},
  1051	};
  1052	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 25767 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] don't allow vfio drivers to bind on driver_attach
  2016-01-12 21:33 [PATCH] don't allow vfio drivers to bind on driver_attach Jacob Keller
  2016-01-12 21:33 ` [PATCH] driver: add manual_bind_only option for virtual stub drivers Jacob Keller
@ 2016-01-12 22:06 ` Alex Williamson
  2016-01-12 22:25   ` Keller, Jacob E
  1 sibling, 1 reply; 7+ messages in thread
From: Alex Williamson @ 2016-01-12 22:06 UTC (permalink / raw)
  To: Jacob Keller, linux-pci
  Cc: Kay Sievers, Rafael J . Wysocki, Alan Stern, Arjan van de Ven

On Tue, 2016-01-12 at 13:33 -0800, Jacob Keller wrote:
> This patch is an attempt to resolve an issue which can occur when using
> vfio-pci (and vfio-platform) with IOMMU direct assignment.
> 
> When attempting to directly assign a device to a virtual machine, the
> normal flow is something like the following:
> 
> echo  > /sys/bus/pci/drivers//unbind
> echo  > /sys/bus/pci/drivers/vfio-pci/new_id
> echo  > /sys/bus/pci/drivers/vfio-pci/bind
> 
> This process results in the device being bound to the vfio-pci stub
> driver, and then this device can be safely given to a virtual machine.
> 
> The issue can occur, that if  doesn't currently have a driver
> loaded, (like say the module was removed), the process of adding
>  can result in *all* devices which match that id being bound to
> vfio-pci. This may not seem like a problem, but it can be confusing when
> you have a dual-port device, such as a networking card. In some use
> cases, you only want to assign a single port to a VM and not all the
> ports. If you happen to not have the driver loaded already, the result
> is that even loading the driver later will not gain control of the 2nd
> port, because the device is already bound to vfio-pci. The solution (if
> you know this is the case) would be to unbind that particular device
> from vfio and bind it to the real driver.
> 
> This patch fixes the issue by instead requiring vfio-pci and
> vfio-platform to only bind to devices given via the sysfs bind route,
> and prevents bind to any devices in other scenarios. The result is that
> vfio-pci will never bind a device unless explicitly requested. Given the
> nature of what vfio-pci does, I think this is a much better solution.
> 
> I've tried to Cc several people who've made changes to this area of th
> kernel for more feedback.

We already have a solution in the kernel for this, it's the
driver_override interface.  Rather than your above example of adding a
new ID to the dynamic list for the driver, such that it will
automatically probe the device, we reverse the process to allow the
device to match the driver.  The sequence becomes:

echo vfio-pci > /sys/bus/pci/devices/<pci address>/driver_override
echo <pci address> > /sys/bus/pci/drivers/<old driver>/unbind
echo <pci address> /sys/bus/pci/drivers_probe

The old interface is kept around because there are quite a few users of
it, some of which would be broken by the proposed manual_bind_only code
change, including the example of using driver_override above.  This
also avoids even the need for pci-stub (which has the same issue),
since we can simply put a dummy name in driver_override to avoid a
device from matching any driver.  Thanks,

Alex

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] don't allow vfio drivers to bind on driver_attach
  2016-01-12 22:06 ` [PATCH] don't allow vfio drivers to bind on driver_attach Alex Williamson
@ 2016-01-12 22:25   ` Keller, Jacob E
  2016-01-12 22:43     ` Alex Williamson
  0 siblings, 1 reply; 7+ messages in thread
From: Keller, Jacob E @ 2016-01-12 22:25 UTC (permalink / raw)
  To: linux-pci, alex.williamson; +Cc: arjan, rjw, kay.sievers, stern

T24gVHVlLCAyMDE2LTAxLTEyIGF0IDE1OjA2IC0wNzAwLCBBbGV4IFdpbGxpYW1zb24gd3JvdGU6
DQo+IFdlIGFscmVhZHkgaGF2ZSBhIHNvbHV0aW9uIGluIHRoZSBrZXJuZWwgZm9yIHRoaXMsIGl0
J3MgdGhlDQo+IGRyaXZlcl9vdmVycmlkZSBpbnRlcmZhY2UuIMKgUmF0aGVyIHRoYW4geW91ciBh
Ym92ZSBleGFtcGxlIG9mIGFkZGluZw0KPiBhDQo+IG5ldyBJRCB0byB0aGUgZHluYW1pYyBsaXN0
IGZvciB0aGUgZHJpdmVyLCBzdWNoIHRoYXQgaXQgd2lsbA0KPiBhdXRvbWF0aWNhbGx5IHByb2Jl
IHRoZSBkZXZpY2UsIHdlIHJldmVyc2UgdGhlIHByb2Nlc3MgdG8gYWxsb3cgdGhlDQo+IGRldmlj
ZSB0byBtYXRjaCB0aGUgZHJpdmVyLiDCoFRoZSBzZXF1ZW5jZSBiZWNvbWVzOg0KPiANCj4gZWNo
byB2ZmlvLXBjaSA+IC9zeXMvYnVzL3BjaS9kZXZpY2VzLzxwY2kgYWRkcmVzcz4vZHJpdmVyX292
ZXJyaWRlDQo+IGVjaG8gPHBjaSBhZGRyZXNzPiA+IC9zeXMvYnVzL3BjaS9kcml2ZXJzLzxvbGQg
ZHJpdmVyPi91bmJpbmQNCj4gZWNobyA8cGNpIGFkZHJlc3M+IC9zeXMvYnVzL3BjaS9kcml2ZXJz
X3Byb2JlDQo+IA0KPiBUaGUgb2xkIGludGVyZmFjZSBpcyBrZXB0IGFyb3VuZCBiZWNhdXNlIHRo
ZXJlIGFyZSBxdWl0ZSBhIGZldyB1c2Vycw0KPiBvZg0KPiBpdCwgc29tZSBvZiB3aGljaCB3b3Vs
ZCBiZSBicm9rZW4gYnkgdGhlIHByb3Bvc2VkwqBtYW51YWxfYmluZF9vbmx5DQo+IGNvZGUNCj4g
Y2hhbmdlLCBpbmNsdWRpbmcgdGhlIGV4YW1wbGUgb2YgdXNpbmcgZHJpdmVyX292ZXJyaWRlIGFi
b3ZlLiDCoFRoaXMNCj4gYWxzbyBhdm9pZHMgZXZlbiB0aGUgbmVlZCBmb3IgcGNpLXN0dWIgKHdo
aWNoIGhhcyB0aGUgc2FtZSBpc3N1ZSksDQo+IHNpbmNlIHdlIGNhbiBzaW1wbHkgcHV0IGEgZHVt
bXkgbmFtZSBpbiBkcml2ZXJfb3ZlcnJpZGUgdG8gYXZvaWQgYQ0KPiBkZXZpY2UgZnJvbSBtYXRj
aGluZyBhbnkgZHJpdmVyLiDCoFRoYW5rcywNCj4gDQo+IEFsZXgNCg0KDQpIbW0sIHRoYXQgbWFr
ZXMgc2Vuc2UuIFNvIHJlYWxseSBJIHNob3VsZCBzZWUgYWJvdXQgZ2V0dGluZyB0aGUgdG9vbHMN
CndoaWNoIGN1cnJlbnRseSB1c2UgdGhlIG9sZCBpbnRlcmZhY2UgdG8gdXNlIGEgbmV3IG9uZT8N
Cg0KKERvbid0IHdlIHN0aWxsIHdhbnQvbmVlZCB2ZmlvLXBjaSBmb3Igb3RoZXIgcmVhc29ucyBi
ZXNpZGVzIGp1c3QgYmVpbmcNCmEgc3R1YiBkcml2ZXI/KQ0KDQpSZWdhcmRzLA0KSmFrZQ==

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] don't allow vfio drivers to bind on driver_attach
  2016-01-12 22:25   ` Keller, Jacob E
@ 2016-01-12 22:43     ` Alex Williamson
  0 siblings, 0 replies; 7+ messages in thread
From: Alex Williamson @ 2016-01-12 22:43 UTC (permalink / raw)
  To: Keller, Jacob E, linux-pci; +Cc: arjan, rjw, kay.sievers, stern

On Tue, 2016-01-12 at 22:25 +0000, Keller, Jacob E wrote:
> On Tue, 2016-01-12 at 15:06 -0700, Alex Williamson wrote:
> > We already have a solution in the kernel for this, it's the
> > driver_override interface.  Rather than your above example of
> > adding
> > a
> > new ID to the dynamic list for the driver, such that it will
> > automatically probe the device, we reverse the process to allow the
> > device to match the driver.  The sequence becomes:
> > 
> > echo vfio-pci > /sys/bus/pci/devices/<pci address>/driver_override
> > echo <pci address> > /sys/bus/pci/drivers/<old driver>/unbind
> > echo <pci address> /sys/bus/pci/drivers_probe
> > 
> > The old interface is kept around because there are quite a few
> > users
> > of
> > it, some of which would be broken by the proposed manual_bind_only
> > code
> > change, including the example of using driver_override above.  This
> > also avoids even the need for pci-stub (which has the same issue),
> > since we can simply put a dummy name in driver_override to avoid a
> > device from matching any driver.  Thanks,
> > 
> > Alex
> 
> 
> Hmm, that makes sense. So really I should see about getting the tools
> which currently use the old interface to use a new one?
> 
> (Don't we still want/need vfio-pci for other reasons besides just being
> a stub driver?)

Of course, vfio-pci is much more than a stub driver, I was only noting
pci-stub as another meta driver that only uses dynamic adds.  It's more
of a side comment that the functionality that pci-stub provides is mostly unnecessary with driver_override.  Thanks,
Alex

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [PATCH] driver: add manual_bind_only option for virtual stub drivers
  2016-01-12 21:33 ` [PATCH] driver: add manual_bind_only option for virtual stub drivers Jacob Keller
  2016-01-12 21:53   ` kbuild test robot
@ 2016-01-13  0:02   ` kbuild test robot
  1 sibling, 0 replies; 7+ messages in thread
From: kbuild test robot @ 2016-01-13  0:02 UTC (permalink / raw)
  To: Jacob Keller; +Cc: kbuild-all, linux-pci, Jacob Keller

[-- Attachment #1: Type: text/plain, Size: 3570 bytes --]

Hi Jacob,

[auto build test WARNING on vfio/next]
[also build test WARNING on v4.4 next-20160112]
[if your patch is applied to the wrong git tree, please drop us a note to help improving the system]

url:    https://github.com/0day-ci/linux/commits/Jacob-Keller/driver-add-manual_bind_only-option-for-virtual-stub-drivers/20160113-053451
base:   https://github.com/awilliam/linux-vfio.git next
reproduce: make htmldocs

All warnings (new ones prefixed by >>):

   include/linux/init.h:1: warning: no structured comments found
   kernel/sys.c:1: warning: no structured comments found
>> include/linux/device.h:283: warning: No description found for parameter 'manual_bind_only'
   drivers/dma-buf/seqno-fence.c:1: warning: no structured comments found
   drivers/dma-buf/reservation.c:1: warning: no structured comments found
   include/linux/reservation.h:1: warning: no structured comments found
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef member 'e_handler' description in 'hsi_client'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef member 'pclaimed' description in 'hsi_client'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef member 'nb' description in 'hsi_client'

vim +/manual_bind_only +283 include/linux/device.h

f98ec3ac Jacob Keller       2016-01-12  267  	bool manual_bind_only;		/* prevent bind on driver_attach */
765230b5 Dmitry Torokhov    2015-03-30  268  	enum probe_type probe_type;
1a6f2a75 Dmitry Torokhov    2009-10-12  269  
597b9d1e Grant Likely       2010-04-13  270  	const struct of_device_id	*of_match_table;
06f64c8f Mika Westerberg    2012-10-31  271  	const struct acpi_device_id	*acpi_match_table;
597b9d1e Grant Likely       2010-04-13  272  
^1da177e Linus Torvalds     2005-04-16  273  	int (*probe) (struct device *dev);
^1da177e Linus Torvalds     2005-04-16  274  	int (*remove) (struct device *dev);
^1da177e Linus Torvalds     2005-04-16  275  	void (*shutdown) (struct device *dev);
9480e307 Russell King       2005-10-28  276  	int (*suspend) (struct device *dev, pm_message_t state);
9480e307 Russell King       2005-10-28  277  	int (*resume) (struct device *dev);
a4dbd674 David Brownell     2009-06-24  278  	const struct attribute_group **groups;
e5dd1278 Greg Kroah-Hartman 2007-11-28  279  
8150f32b Dmitry Torokhov    2009-07-24  280  	const struct dev_pm_ops *pm;
1eede070 Rafael J. Wysocki  2008-05-20  281  
e5dd1278 Greg Kroah-Hartman 2007-11-28  282  	struct driver_private *p;
^1da177e Linus Torvalds     2005-04-16 @283  };
^1da177e Linus Torvalds     2005-04-16  284  
^1da177e Linus Torvalds     2005-04-16  285  
4a7fb636 Andrew Morton      2006-08-14  286  extern int __must_check driver_register(struct device_driver *drv);
^1da177e Linus Torvalds     2005-04-16  287  extern void driver_unregister(struct device_driver *drv);
^1da177e Linus Torvalds     2005-04-16  288  
d462943a Greg Kroah-Hartman 2008-01-24  289  extern struct device_driver *driver_find(const char *name,
d462943a Greg Kroah-Hartman 2008-01-24  290  					 struct bus_type *bus);
d779249e Greg Kroah-Hartman 2006-07-18  291  extern int driver_probe_done(void);

:::::: The code at line 283 was first introduced by commit
:::::: 1da177e4c3f41524e886b7f1b8a0c1fc7321cac2 Linux-2.6.12-rc2

:::::: TO: Linus Torvalds <torvalds@ppc970.osdl.org>
:::::: CC: Linus Torvalds <torvalds@ppc970.osdl.org>

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation

[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 6124 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2016-01-13  0:03 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-01-12 21:33 [PATCH] don't allow vfio drivers to bind on driver_attach Jacob Keller
2016-01-12 21:33 ` [PATCH] driver: add manual_bind_only option for virtual stub drivers Jacob Keller
2016-01-12 21:53   ` kbuild test robot
2016-01-13  0:02   ` kbuild test robot
2016-01-12 22:06 ` [PATCH] don't allow vfio drivers to bind on driver_attach Alex Williamson
2016-01-12 22:25   ` Keller, Jacob E
2016-01-12 22:43     ` Alex Williamson

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.