All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH V8 0/7] xen pvusb toolstack work
@ 2015-10-21  9:08 Chunyan Liu
  2015-10-21  9:08 ` [PATCH V8 1/7] libxl: export some functions for pvusb use Chunyan Liu
                   ` (6 more replies)
  0 siblings, 7 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig

This patch series is to add pvusb toolstack work, supporting hot add|remove
USB device to|from guest and specify USB device in domain configuration file.

Changes to V7:
* lots of changes in libxl pvusb API (patch 3/7)
* update 2/7 to follow internal function rules
* update parse_usb(usbctrl)_config to follow parse_nic_config
* update user interface about USB attach, using hostbus=x hostaddr=
  instead of hostbus.hostaddr
* address other comments to V6

V7:
http://lists.xen.org/archives/html/xen-devel/2015-09/msg03115.html

V6:
http://lists.xen.org/archives/html/xen-devel/2015-08/msg00750.html

V5:
http://lists.xen.org/archives/html/xen-devel/2015-06/msg04052.html

V4:
http://lists.xenproject.org/archives/html/xen-devel/2015-06/msg01327.html

Related Discussion Threads:
http://www.redhat.com/archives/libvir-list/2014-June/msg00038.html
http://lists.xen.org/archives/html/xen-devel/2014-06/msg00086.html

              <<< pvusb work introduction >>>

1. Overview

There are two general methods for passing through individual host
devices to a guest. The first is via an emulated USB device
controller; the second is PVUSB.

Additionally, there are two ways to add USB devices to a guest: via
the config file at domain creation time, and via hot-plug while the VM
is running.

* Emulated USB

In emulated USB, the device model (qemu) presents an emulated USB
controller to the guest. The device model process then grabs control
of the device from domain 0 and and passes the USB commands between
the guest OS and the host USB device.

This method is only available to HVM domains, and is not available for
domains running with device model stubdomains.

* PVUSB

PVUSB uses a paravirtialized front-end/back-end interface, similar to
the traditional Xen PV network and disk protocols. In order to use
PVUSB, you need usbfront in your guest OS, and usbback in dom0 (or
your USB driver domain).

2. Specifying a host USB device

QEMU qmp commands allows USB devices to be specified either by their
bus address (in the form bus.device) or their device tag (in the form
vendorid:deviceid).

Each way of specifying has its advantages:

    Specifying by device tag will always get the same device,
regardless of where the device ends up in the USB bus topology.
However, if there are two identical devices, it will not allow you to
specify which one.

    Specifying by bus address will always allow you to choose a
specific device, even if you have duplicates. However, the bus address
may change depending on which port you plugged the device into, and
possibly also after a reboot.

To avoid duplication of vendorid:deviceid, we'll use bus address to
specify host USB device in xl toolstack.

You can use lsusb to list the USB devices on the system:

Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0
Hub
Bus 003 Device 002: ID f617:0905
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 004: ID 0424:2640 Standard Microsystems Corp. USB 2.0
Hub
Bus 001 Device 005: ID 0424:4060 Standard Microsystems Corp. Ultra
Fast Media Reader
Bus 001 Device 006: ID 046d:c016 Logitech, Inc. Optical Wheel Mouse

To pass through the Logitec mouse, for instance, you could specify
1.6 (remove leading zeroes).

Note: USB hubs can not be assigned to guest.

3. PVUSB toolstack

* Specify USB device in xl config file

You can just specify usb devices, like:
usbdev=['1.6']

Then it will create a USB controller automatically and attach the USB
device to the first available USB controller:port.

or, you can explicitly specify usb controllers and usb devices, like:
usbctrl=['verison=1, ports=4', 'version=2, ports=8', ]
usbdev=['1.6, controller=0, port=1']

Then it will create two USB controllers as you specified.
And if controller and port are specified in usb config, then it will
attach the USB device to that controller:port. About the controller
and port value:
Each USB controller has a index (or called devid) based on 0. The 1st
controller has index 0, the 2nd controller has index 1, ...
Under controller, each port has a port number based on 1. In above
configuration, the 1st controller will have port 1,2,3,4.

* Hot-Plug USB device

To attach a USB device, you should first create a USB controller.
e.g.
xl usb-ctrl-attach domain [version=1|2] [ports=value]
By default, it will create a USB2.0 controller with 8 ports.

Then you could attach a USB device.
e.g.
xl usb-attach domain 1.6 [controller=index port=number]
By default, it will find the 1st available controller:port to attach
the USB device.

You could view USB device status of the domain by usb-list.
e.g.
xl usb-list domain
It will list USB controllers and USB devices under each controller.

You could detach a USB device with usb-detach command.
e.g.
xl usb-detach domain 1.6

You can also remove the whole USB controller by usb-ctrl-detach
command.
e.g.
xl usb-ctrl-detach domain 0
It will remove the USB controller with index 0 and all USB devices
under it.

4. PVUSB Libxl implementation

* usb-ctrl-attach
To create a usb controller, we need:
1) generate usb controler related information
2) write usb controller frontend/backend info to xenstore
PVUSB frontend and backend driver will probe xenstore paths and build
connection between frontend and backend.

* usb-ctrl-detach
To remove a usb controller, we need:
1) check if the usb controller exists or not
2) remove all usb devices under controller
3) remove usb controller info from xenstore

* usb-attach
To attach a usb device, we need:
1) check if the usb device type is assignable
2) check if the usb device is already assigned to a domain
3) add 'busid' of the usb device to xenstore contoller/port/.
   PVUSB driver watches the xenstore changes and detects that,
   and needs to use 'busid' to do following work.
4) unbind usb device from original driver and bind to usbback.
   If usb device has many interfaces, then:
   - unbind each interface from its original driver and bind to usbback.
   - store the original driver to xenstore for later rebinding when
     detaching the device.

* usb-detach
To detach a usb device, we need:
1) check if the usb device is assigned to the domain
2) remove the usb device from xenstore controller/port.
3) unbind usb device from usbback and rebind to its original driver.
   If usb device has many interfaces, do it to each interface.

* usb-list
List all USB controllers and USB devices under each controller.

5. PVUSB xenstore information

PVUSB xenstore information includes three parts: frontend, backend
and /libxl part.

A USB controller is corresponding to a "vusb" device in xenstore.
Adding a USB controller will add a new "vusb" device, removing a
USB controller will delete the related "vusb" device.

Following is an example xenstore values of a USB controller.
Backend:
   backend = ""
    vusb = ""
     1 = ""
      0 = ""
       frontend = "/local/domain/1/device/vusb/0"
       frontend-id = "1"
       online = "1"
       state = "4"
       type = "pv"
       usb-ver = "1"
       num-ports = "4"
       port = ""
        1 = ""
        2 = ""
        3 = ""
        4 = ""

Frontend:
   device = ""
    vusb = ""
     0 = ""
      backend = "/local/domain/0/backend/vusb/1/0"
      backend-id = "0"
      state = "4"
      urb-ring-ref = "348"
      conn-ring-ref = "346"
      event-channel = "20"

Adding a USB device won't create a new "vusb" device, but only write
the USB device busid to one port of USB controller.
For example, attaching a USB device (busid is 2-1.6) to above USB
controller port 1, it only need write 2-1.6 to port 1 of this USB
controller:
Backend:
   backend = ""
    vusb = ""
     1 = ""
      0 = ""
       frontend = "/local/domain/1/device/vusb/0"
       frontend-id = "1"
       online = "1"
       state = "4"
       type = "pv"
       usb-ver = "1"
       num-ports = "4"
       port = ""
        1 = "2-1.6"
        2 = ""
        3 = ""
        4 = ""
Frontend doesn't change.

Since assign a host USB device to guest, we'll unbind USB interfaces
from their original drivers and bind them to usbback. After detaching
this USB device from guest, one would hope the USB interfaces could
be rebind to their original drivers, so there should some place to
get the original driver info. To support that, when attaching a USB
device to guest, we'll save the original driver info in xenstore too,
the place is /libxl/usbback, for example:
libxl = ""
 1 = ""
  dm-version = "qemu_xen"
 usbback = ""
  3-11 = ""
   3-11-1_0 = ""
    driver_path = "/sys/bus/usb/drivers/btusb"

In this example, USB device (busid is 3-11, /sys/bus/usb/devices/3-11).
It has interface 3-11:1.0, whose original dirver is btusb.
Since xenstore doesn't allow ':' and '.' in a key, so we encode the
interface by changing ':' to '-' and changing '.' to '_'.

When detaching the USB device from guest, we can rebind 3-11:1.0 to
btusb driver.

Chunyan Liu (7):
  libxl: export some functions for pvusb use
  libxl_read_file_contents: add new entry to read sysfs file
  libxl: add pvusb API
  libxl: add libxl_device_usb_assignable_list API
  xl: add pvusb commands
  xl: add usb-assignable-list command
  domcreate: support pvusb in configuration file

 docs/man/xl.cfg.pod.5                |   81 ++
 docs/man/xl.pod.1                    |   40 +
 tools/libxl/Makefile                 |    2 +-
 tools/libxl/libxl.c                  |   57 +-
 tools/libxl/libxl.h                  |   77 ++
 tools/libxl/libxl_create.c           |   73 +-
 tools/libxl/libxl_device.c           |    9 +-
 tools/libxl/libxl_internal.h         |   33 +
 tools/libxl/libxl_osdeps.h           |   13 +
 tools/libxl/libxl_pvusb.c            | 1505 ++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_types.idl          |   57 ++
 tools/libxl/libxl_types_internal.idl |    1 +
 tools/libxl/libxl_utils.c            |   77 +-
 tools/libxl/libxl_utils.h            |    5 +
 tools/libxl/xl.h                     |    6 +
 tools/libxl/xl_cmdimpl.c             |  331 +++++++-
 tools/libxl/xl_cmdtable.c            |   29 +
 17 files changed, 2374 insertions(+), 22 deletions(-)
 create mode 100644 tools/libxl/libxl_pvusb.c

-- 
2.1.4

^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH V8 1/7] libxl: export some functions for pvusb use
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:08   ` Juergen Gross
  2015-10-21  9:08 ` [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file Chunyan Liu
                   ` (5 subsequent siblings)
  6 siblings, 1 reply; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig, Simon Cao

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
Reviewed-by: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxl/libxl.c          | 4 ++--
 tools/libxl/libxl_internal.h | 3 +++
 2 files changed, 5 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 22bbc29..dacfaae 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -2035,7 +2035,7 @@ out:
 }
 
 /* common function to get next device id */
-static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
+int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
 {
     char *dompath, **l;
     unsigned int nb;
@@ -2054,7 +2054,7 @@ static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
     return nextid;
 }
 
-static int libxl__resolve_domid(libxl__gc *gc, const char *name,
+int libxl__resolve_domid(libxl__gc *gc, const char *name,
                                 uint32_t *domid)
 {
     if (!name)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index b00add0..ab981d2 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1165,6 +1165,9 @@ _hidden int libxl__init_console_from_channel(libxl__gc *gc,
                                              libxl__device_console *console,
                                              int dev_num,
                                              libxl_device_channel *channel);
+_hidden int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device);
+_hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,
+                                 uint32_t *domid);
 
 /*
  * For each aggregate type which can be used as an input we provide:
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
  2015-10-21  9:08 ` [PATCH V8 1/7] libxl: export some functions for pvusb use Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:31   ` Juergen Gross
  2015-11-16 14:03   ` Ian Campbell
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
                   ` (4 subsequent siblings)
  6 siblings, 2 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig

Sysfs file has size=4096 but actual file content is less than that.
Current libxl_read_file_contents will treat it as error when file size
and actual file content differs, so reading sysfs file content with
this function always fails.

Add a new entry libxl_read_sysfs_file_contents to handle sysfs file
specially. It would be used in later pvusb work.

Signed-off-by: Chunyan Liu <cyliu@suse.com>

---
Changes:
  - Update libxl_read_sysfs_file_contents to follow internal
    function definition rules, and adjust related implementation.

 tools/libxl/libxl_internal.h |  4 +++
 tools/libxl/libxl_utils.c    | 61 ++++++++++++++++++++++++++++++++++----------
 2 files changed, 51 insertions(+), 14 deletions(-)

diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index ab981d2..a3a5cc1 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -4022,6 +4022,10 @@ void libxl__bitmap_copy_best_effort(libxl__gc *gc, libxl_bitmap *dptr,
 
 int libxl__count_physical_sockets(libxl__gc *gc, int *sockets);
 #endif
+_hidden int libxl__read_sysfs_file_contents(libxl__gc *gc,
+                                            const char *filename,
+                                            void **data_r,
+                                            int *datalen_r);
 
 /*
  * Local variables:
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index e42422a..9c5c4d0 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -322,9 +322,10 @@ out:
     return rc;
 }
 
-int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
-                             void **data_r, int *datalen_r) {
-    GC_INIT(ctx);
+static int read_file_contents_core(libxl__gc *gc, const char *filename,
+                                   void **data_r, int *datalen_r,
+                                   bool tolerate_shrinking_file)
+{
     FILE *f = 0;
     uint8_t *data = 0;
     int datalen = 0;
@@ -359,20 +360,35 @@ int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
     datalen = stab.st_size;
 
     if (stab.st_size && data_r) {
-        data = malloc(datalen);
+        data = malloc(datalen + 1);
         if (!data) goto xe;
 
-        rs = fread(data, 1, datalen, f);
-        if (rs != datalen) {
-            if (ferror(f))
+        rs = fread(data, 1, datalen + 1, f);
+        if (rs > datalen) {
+            LOG(ERROR, "%s increased size while we were reading it",
+                filename);
+            goto xe;
+        }
+
+        if (rs < datalen) {
+            if (ferror(f)) {
                 LOGE(ERROR, "failed to read %s", filename);
-            else if (feof(f))
-                LOG(ERROR, "%s changed size while we were reading it",
-		    filename);
-            else
+                goto xe;
+            } else if (feof(f)) {
+                if (tolerate_shrinking_file) {
+                    datalen = rs;
+                } else {
+                    LOG(ERROR, "%s shrunk size while we were reading it",
+                        filename);
+                    goto xe;
+                }
+            } else {
                 abort();
-            goto xe;
+            }
         }
+
+        data = realloc(data, datalen);
+        if (!data) goto xe;
     }
 
     if (fclose(f)) {
@@ -384,11 +400,9 @@ int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
     if (data_r) *data_r = data;
     if (datalen_r) *datalen_r = datalen;
 
-    GC_FREE;
     return 0;
 
  xe:
-    GC_FREE;
     e = errno;
     assert(e != ENOENT);
     if (f) fclose(f);
@@ -396,6 +410,25 @@ int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
     return e;
 }
 
+int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
+                             void **data_r, int *datalen_r)
+{
+    GC_INIT(ctx);
+    int rc;
+
+    rc = read_file_contents_core(gc, filename, data_r, datalen_r, 0);
+
+    GC_FREE;
+    return rc;
+}
+
+int libxl__read_sysfs_file_contents(libxl__gc *gc, const char *filename,
+                                   void **data_r, int *datalen_r)
+{
+    return read_file_contents_core(gc, filename, data_r, datalen_r, 1);
+}
+
+
 #define READ_WRITE_EXACTLY(rw, zero_is_eof, constdata)                    \
                                                                           \
   int libxl_##rw##_exactly(libxl_ctx *ctx, int fd,                 \
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
  2015-10-21  9:08 ` [PATCH V8 1/7] libxl: export some functions for pvusb use Chunyan Liu
  2015-10-21  9:08 ` [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:31   ` Juergen Gross
                     ` (5 more replies)
  2015-10-21  9:08 ` [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API Chunyan Liu
                   ` (3 subsequent siblings)
  6 siblings, 6 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig, Simon Cao

Add pvusb APIs, including:
 - attach/detach (create/destroy) virtual usb controller.
 - attach/detach usb device
 - list usb controller and usb devices
 - some other helper functions

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>

---
changes:
  - update COMPARE_USB to compare ctrl and port
  - add check in usb_add/remove to disable non-Dom0 backend so that
    not worring about codes which are effective on Dom0 but not
    compatible on non-Dom0 backend.
  - define READ_SUBPATH macro within functions
  - do not initialize rc but give it value in each return case
  - libxl__strdup gc or NOGC update, internal function using gc,
    external using NOGC.
  - address other comments from George and Ian J.

 tools/libxl/Makefile                 |    2 +-
 tools/libxl/libxl.c                  |   53 ++
 tools/libxl/libxl.h                  |   74 ++
 tools/libxl/libxl_device.c           |    5 +-
 tools/libxl/libxl_internal.h         |   18 +
 tools/libxl/libxl_osdeps.h           |   13 +
 tools/libxl/libxl_pvusb.c            | 1451 ++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_types.idl          |   57 ++
 tools/libxl/libxl_types_internal.idl |    1 +
 tools/libxl/libxl_utils.c            |   16 +
 tools/libxl/libxl_utils.h            |    5 +
 11 files changed, 1693 insertions(+), 2 deletions(-)
 create mode 100644 tools/libxl/libxl_pvusb.c

diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index c5ecec1..ef9ccd3 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -103,7 +103,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
 			libxl_stream_read.o libxl_stream_write.o \
 			libxl_save_callout.o _libxl_save_msgs_callout.o \
 			libxl_qmp.o libxl_event.o libxl_fork.o \
-			libxl_dom_suspend.o $(LIBXL_OBJS-y)
+			libxl_dom_suspend.o libxl_pvusb.o $(LIBXL_OBJS-y)
 LIBXL_OBJS += libxl_genid.o
 LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
 
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index dacfaae..a050e8b 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -4218,11 +4218,54 @@ DEFINE_DEVICE_REMOVE(vtpm, destroy, 1)
 
 /******************************************************************************/
 
+/* Macro for defining device remove/destroy functions for usbctrl */
+/* Following functions are defined:
+ * libxl_device_usbctrl_remove
+ * libxl_device_usbctrl_destroy
+ */
+
+#define DEFINE_DEVICE_REMOVE_EXT(type, removedestroy, f)                \
+    int libxl_device_##type##_##removedestroy(libxl_ctx *ctx,           \
+        uint32_t domid, libxl_device_##type *type,                      \
+        const libxl_asyncop_how *ao_how)                                \
+    {                                                                   \
+        AO_CREATE(ctx, domid, ao_how);                                  \
+        libxl__device *device;                                          \
+        libxl__ao_device *aodev;                                        \
+        int rc;                                                         \
+                                                                        \
+        GCNEW(device);                                                  \
+        rc = libxl__device_from_##type(gc, domid, type, device);        \
+        if (rc != 0) goto out;                                          \
+                                                                        \
+        GCNEW(aodev);                                                   \
+        libxl__prepare_ao_device(ao, aodev);                            \
+        aodev->action = LIBXL__DEVICE_ACTION_REMOVE;                    \
+        aodev->dev = device;                                            \
+        aodev->callback = device_addrm_aocomplete;                      \
+        aodev->force = f;                                               \
+        libxl__initiate_device_##type##_remove(egc, aodev);             \
+                                                                        \
+    out:                                                                \
+        if (rc) return AO_CREATE_FAIL(rc);                              \
+        return AO_INPROGRESS;                                           \
+    }
+
+
+DEFINE_DEVICE_REMOVE_EXT(usbctrl, remove, 0)
+DEFINE_DEVICE_REMOVE_EXT(usbctrl, destroy, 1)
+
+#undef DEFINE_DEVICE_REMOVE_EXT
+
+/******************************************************************************/
+
 /* Macro for defining device addition functions in a compact way */
 /* The following functions are defined:
  * libxl_device_disk_add
  * libxl_device_nic_add
  * libxl_device_vtpm_add
+ * libxl_device_usbctrl_add
+ * libxl_device_usb_add
  */
 
 #define DEFINE_DEVICE_ADD(type)                                         \
@@ -4254,6 +4297,12 @@ DEFINE_DEVICE_ADD(nic)
 /* vtpm */
 DEFINE_DEVICE_ADD(vtpm)
 
+/* usbctrl */
+DEFINE_DEVICE_ADD(usbctrl)
+
+/* usb */
+DEFINE_DEVICE_ADD(usb)
+
 #undef DEFINE_DEVICE_ADD
 
 /******************************************************************************/
@@ -6795,6 +6844,10 @@ int libxl_retrieve_domain_configuration(libxl_ctx *ctx, uint32_t domid,
 
     MERGE(pci, pcidevs, COMPARE_PCI, {});
 
+    MERGE(usbctrl, usbctrls, COMPARE_USBCTRL, {});
+
+    MERGE(usb, usbs, COMPARE_USB, {});
+
     /* Take care of removable device. We maintain invariant in the
      * insert / remove operation so that:
      * 1. if xenstore is "empty" while JSON is not, the result
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 615b1de..594e23d 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -123,6 +123,12 @@
 #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
 
 /*
+ * LIBXL_HAVE_PVUSB indicates functions for plugging in USB devices
+ * through pvusb -- both hotplug and at domain creation time..
+ */
+#define LIBXL_HAVE_PVUSB 1
+
+/*
  * LIBXL_HAVE_BUILDINFO_HVM_VENDOR_DEVICE indicates that the
  * libxl_vendor_device field is present in the hvm sections of
  * libxl_domain_build_info. This field tells libxl which
@@ -1411,6 +1417,74 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
                        const libxl_asyncop_how *ao_how)
                        LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/*
+ * USB
+ *
+ * For each device removed or added, one of these protocols is available:
+ * - PV (i.e., PVUSB)
+ * - DEVICEMODEL (i.e, qemu)
+ *
+ * PV is available for either PV or HVM domains.  DEVICEMODEL is only
+ * available for HVM domains.  The caller can additionally specify
+ * "AUTO", in which case the library will try to determine the best
+ * protocol automatically.
+ *
+ * At the moment, the only protocol implemented is PV.
+ *
+ * One can add/remove USB controllers to/from guest, and attach/detach USB
+ * devices to/from USB controllers.
+ *
+ * To add USB controllers and USB devices, one can adding USB controllers
+ * first and then attaching USB devices to some USB controller, or adding
+ * USB devices to guest directly, it will automatically create a USB
+ * controller for USB devices to attach.
+ *
+ * To remove USB controllers or USB devices, one can remove USB devices
+ * under USB controller one by one and then remove USB controller, or
+ * remove USB controller directly, it will remove all USB devices under
+ * it automatically.
+ *
+ */
+/* USB Controllers*/
+int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,
+                             libxl_device_usbctrl *usbctrl,
+                             const libxl_asyncop_how *ao_how)
+                             LIBXL_EXTERNAL_CALLERS_ONLY;
+
+int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid,
+                                libxl_device_usbctrl *usbctrl,
+                                const libxl_asyncop_how *ao_how)
+                                LIBXL_EXTERNAL_CALLERS_ONLY;
+
+int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid,
+                                 libxl_device_usbctrl *usbctrl,
+                                 const libxl_asyncop_how *ao_how)
+                                 LIBXL_EXTERNAL_CALLERS_ONLY;
+
+libxl_device_usbctrl *libxl_device_usbctrl_list(libxl_ctx *ctx,
+                                                uint32_t domid, int *num);
+
+void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr);
+
+
+int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
+                                 libxl_device_usbctrl *usbctrl,
+                                 libxl_usbctrlinfo *usbctrlinfo);
+
+/* USB Devices */
+int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
+                         const libxl_asyncop_how *ao_how)
+                         LIBXL_EXTERNAL_CALLERS_ONLY;
+
+int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
+                            const libxl_asyncop_how *ao_how)
+                            LIBXL_EXTERNAL_CALLERS_ONLY;
+
+libxl_device_usb *
+libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, int *num);
+
+void libxl_device_usb_list_free(libxl_device_usb *list, int nr);
+
 /* Network Interfaces */
 int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic *nic,
                          const libxl_asyncop_how *ao_how)
diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
index 8bb5e93..6715c16 100644
--- a/tools/libxl/libxl_device.c
+++ b/tools/libxl/libxl_device.c
@@ -676,7 +676,10 @@ void libxl__devices_destroy(libxl__egc *egc, libxl__devices_remove_state *drs)
                 aodev->action = LIBXL__DEVICE_ACTION_REMOVE;
                 aodev->dev = dev;
                 aodev->force = drs->force;
-                libxl__initiate_device_remove(egc, aodev);
+                if (dev->backend_kind == LIBXL__DEVICE_KIND_VUSB)
+                    libxl__initiate_device_usbctrl_remove(egc, aodev);
+                else
+                    libxl__initiate_device_remove(egc, aodev);
             }
         }
     }
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index a3a5cc1..1aa0aaf 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -2569,6 +2569,14 @@ _hidden void libxl__device_vtpm_add(libxl__egc *egc, uint32_t domid,
                                    libxl_device_vtpm *vtpm,
                                    libxl__ao_device *aodev);
 
+_hidden void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,
+                                       libxl_device_usbctrl *usbctrl,
+                                       libxl__ao_device *aodev);
+
+_hidden void libxl__device_usb_add(libxl__egc *egc, uint32_t domid,
+                                   libxl_device_usb *usb,
+                                   libxl__ao_device *aodev);
+
 /* Internal function to connect a vkb device */
 _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid,
                                   libxl_device_vkb *vkb);
@@ -2601,6 +2609,13 @@ _hidden void libxl__wait_device_connection(libxl__egc*,
 _hidden void libxl__initiate_device_remove(libxl__egc *egc,
                                            libxl__ao_device *aodev);
 
+_hidden int libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid,
+                               libxl_device_usbctrl *usbctrl,
+                               libxl__device *device);
+
+_hidden void libxl__initiate_device_usbctrl_remove(libxl__egc *egc,
+                                                   libxl__ao_device *aodev);
+
 /*
  * libxl__get_hotplug_script_info returns the args and env that should
  * be passed to the hotplug script for the requested device.
@@ -3958,6 +3973,9 @@ static inline void libxl__update_config_vtpm(libxl__gc *gc,
 #define COMPARE_PCI(a, b) ((a)->func == (b)->func &&    \
                            (a)->bus == (b)->bus &&      \
                            (a)->dev == (b)->dev)
+#define COMPARE_USB(a, b) ((a)->ctrl == (b)->ctrl && \
+                           (a)->port == (b)->port)
+#define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid)
 
 /* DEVICE_ADD
  *
diff --git a/tools/libxl/libxl_osdeps.h b/tools/libxl/libxl_osdeps.h
index d9661c9..802c762 100644
--- a/tools/libxl/libxl_osdeps.h
+++ b/tools/libxl/libxl_osdeps.h
@@ -24,6 +24,8 @@
 #define _GNU_SOURCE
 
 #if defined(__NetBSD__)
+#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
+#define SYSFS_USBBACK_DRIVER   "/kern/xen/usb"
 #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
 #define SYSFS_PCIBACK_DRIVER   "/kern/xen/pci"
 #define NETBACK_NIC_NAME       "xvif%ui%d"
@@ -31,6 +33,8 @@
 #elif defined(__OpenBSD__)
 #include <util.h>
 #elif defined(__linux__)
+#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
+#define SYSFS_USBBACK_DRIVER   "/sys/bus/usb/drivers/usbback"
 #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
 #define SYSFS_PCIBACK_DRIVER   "/sys/bus/pci/drivers/pciback"
 #define NETBACK_NIC_NAME       "vif%u.%d"
@@ -38,6 +42,8 @@
 #elif defined(__sun__)
 #include <stropts.h>
 #elif defined(__FreeBSD__)
+#define SYSFS_USB_DEV          "/dev/null"
+#define SYSFS_USBBACK_DRIVER   "/dev/null"
 #define SYSFS_PCI_DEV          "/dev/null"
 #define SYSFS_PCIBACK_DRIVER   "/dev/null"
 #define NETBACK_NIC_NAME       "xnb%u.%d"
@@ -45,6 +51,13 @@
 #include <sys/endian.h>
 #endif
 
+#ifndef SYSFS_USBBACK_DRIVER
+#error define SYSFS_USBBACK_DRIVER for your platform
+#endif
+#ifndef SYSFS_USB_DEV
+#error define SYSFS_USB_DEV for your platform
+#endif
+
 #ifndef SYSFS_PCIBACK_DRIVER
 #error define SYSFS_PCIBACK_DRIVER for your platform
 #endif
diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
new file mode 100644
index 0000000..aa1a653
--- /dev/null
+++ b/tools/libxl/libxl_pvusb.c
@@ -0,0 +1,1451 @@
+/*
+ * Copyright (C) 2015 SUSE LINUX Products GmbH, Nuernberg, Germany.
+ * Author Chunyan Liu <cyliu@suse.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+
+#include "libxl_internal.h"
+#include <inttypes.h>
+
+#define USBBACK_INFO_PATH "/libxl/usbback"
+
+#define USBHUB_CLASS_CODE 9
+
+static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
+                                            libxl_device_usbctrl *usbctrl)
+{
+    int rc;
+    libxl_domain_type domtype = libxl__domain_type(gc, domid);
+
+    if (!usbctrl->version)
+        usbctrl->version = 2;
+
+    if (!usbctrl->ports)
+        usbctrl->ports = 8;
+
+    if (usbctrl->type == LIBXL_USBCTRL_TYPE_AUTO) {
+        if (domtype == LIBXL_DOMAIN_TYPE_PV) {
+            usbctrl->type = LIBXL_USBCTRL_TYPE_PV;
+        } else if (domtype == LIBXL_DOMAIN_TYPE_HVM) {
+            /* FIXME: See if we can detect PV frontend */
+            usbctrl->type = LIBXL_USBCTRL_TYPE_DEVICEMODEL;
+        }
+    }
+
+    rc = libxl__resolve_domid(gc, usbctrl->backend_domname,
+                              &usbctrl->backend_domid);
+    return rc;
+}
+
+int libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid,
+                               libxl_device_usbctrl *usbctrl,
+                               libxl__device *device)
+{
+    device->backend_devid   = usbctrl->devid;
+    device->backend_domid   = usbctrl->backend_domid;
+    device->backend_kind    = LIBXL__DEVICE_KIND_VUSB;
+    device->devid           = usbctrl->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_VUSB;
+
+    return 0;
+}
+
+/* Add usbctrl information to xenstore.
+ *
+ * Adding a usb controller will add a new 'vusb' device in xenstore, and
+ * add corresponding frontend, backend information to it. According to
+ * "update_json", decide wether to update json config file.
+ */
+static int libxl__device_usbctrl_add_xenstore(libxl__gc *gc, uint32_t domid,
+                                              libxl_device_usbctrl *usbctrl,
+                                              bool update_json)
+{
+    libxl__device *device;
+    flexarray_t *front;
+    flexarray_t *back;
+    xs_transaction_t t = XBT_NULL;
+    int i, rc;
+    libxl_domain_config d_config;
+    libxl_device_usbctrl usbctrl_saved;
+    libxl__domain_userdata_lock *lock = NULL;
+
+    libxl_domain_config_init(&d_config);
+    libxl_device_usbctrl_init(&usbctrl_saved);
+    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl);
+
+    GCNEW(device);
+    rc = libxl__device_from_usbctrl(gc, domid, usbctrl, device);
+    if (rc) goto out;
+
+    front = flexarray_make(gc, 4, 1);
+    back = flexarray_make(gc, 12, 1);
+
+    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid));
+    flexarray_append_pair(back, "online", "1");
+    flexarray_append_pair(back, "state", "1");
+    flexarray_append_pair(back, "type",
+                    (char *)libxl_usbctrl_type_to_string(usbctrl->type));
+    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d", usbctrl->version));
+    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d", usbctrl->ports));
+    flexarray_append_pair(back, "port", "");
+    for (i = 0; i < usbctrl->ports; i++)
+        flexarray_append_pair(back, GCSPRINTF("port/%d", i + 1), "");
+
+    flexarray_append_pair(front, "backend-id",
+                          GCSPRINTF("%d", usbctrl->backend_domid));
+    flexarray_append_pair(front, "state", "1");
+
+    if (update_json) {
+        lock = libxl__lock_domain_userdata(gc, domid);
+        if (!lock) {
+            rc = ERROR_LOCK_FAIL;
+            goto out;
+        }
+
+        rc = libxl__get_domain_configuration(gc, domid, &d_config);
+        if (rc) goto out;
+
+        DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved,
+                   COMPARE_USBCTRL, &d_config);
+    }
+
+    for (;;) {
+        rc = libxl__xs_transaction_start(gc, &t);
+        if (rc) goto out;
+
+        rc = libxl__device_exists(gc, t, device);
+        if (rc < 0) goto out;
+        if (rc == 1) {
+            /* already exists in xenstore */
+            LOG(ERROR, "device already exists in xenstore");
+            rc = ERROR_DEVICE_EXISTS;
+            goto out;
+        }
+
+        if (update_json) {
+            rc = libxl__set_domain_configuration(gc, domid, &d_config);
+            if (rc) goto out;
+        }
+
+        libxl__device_generic_add(gc, t, device,
+                          libxl__xs_kvs_of_flexarray(gc, back, back->count),
+                          libxl__xs_kvs_of_flexarray(gc, front, front->count),
+                          NULL);
+
+        rc = libxl__xs_transaction_commit(gc, &t);
+        if (!rc) break;
+        if (rc < 0) goto out;
+    }
+
+out:
+    libxl__xs_transaction_abort(gc, &t);
+    if (lock) libxl__unlock_domain_userdata(lock);
+    libxl_device_usbctrl_dispose(&usbctrl_saved);
+    libxl_domain_config_dispose(&d_config);
+    return rc;
+}
+
+/* AO operation to add a usb controller.
+ *
+ * Generally, it does:
+ * 1) fill in necessary usb controler information with default value
+ * 2) write usb controller frontend/backend info to xenstore, update json
+ *    config file if necessary.
+ * 3) wait for device connection. PVUSB frontend and backend driver will
+ *    probe xenstore paths and build connection between frontend and backend.
+ *
+ * Before calling this function, aodev should be properly filled:
+ * aodev->ao, aodev->callback, aodev->update_json, ...
+ */
+void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,
+                               libxl_device_usbctrl *usbctrl,
+                               libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    libxl__device *device;
+    int rc;
+
+    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
+    if (rc < 0) goto out;
+
+    if (usbctrl->devid == -1) {
+        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");
+        if (usbctrl->devid < 0) {
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    }
+
+    if (usbctrl->type != LIBXL_USBCTRL_TYPE_PV) {
+        LOG(ERROR, "Unsupported USB controller type");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = libxl__device_usbctrl_add_xenstore(gc, domid, usbctrl,
+                                            aodev->update_json);
+    if (rc) goto out;
+
+    GCNEW(device);
+    rc = libxl__device_from_usbctrl(gc, domid, usbctrl, device);
+    if (rc) goto out;
+
+    aodev->dev = device;
+    aodev->action = LIBXL__DEVICE_ACTION_ADD;
+    libxl__wait_device_connection(egc, aodev);
+    return;
+
+out:
+    aodev->rc = rc;
+    if (rc) aodev->callback(egc, aodev);
+    return;
+}
+
+static int
+libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
+                                   libxl_devid usbctrl,
+                                   libxl_device_usb **usbs, int *num);
+
+static int
+libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb);
+
+/* AO function to remove a usb controller.
+ *
+ * Generally, it does:
+ * 1) check if the usb controller exists or not
+ * 2) remove all usb devices under controller
+ * 3) remove usb controller information from xenstore
+ *
+ * Before calling this function, aodev should be properly filled:
+ * aodev->ao, aodev->dev, aodev->callback, ...
+ */
+void libxl__initiate_device_usbctrl_remove(libxl__egc *egc,
+                                           libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    libxl_ctx *ctx = CTX;
+    libxl_device_usb *usbs = NULL;
+    int numusb = 0;
+    int i, rc;
+    uint32_t domid = ao->domid;
+    int usbctrl_devid = aodev->dev->devid;
+    libxl_device_usbctrl usbctrl;
+    libxl_usbctrlinfo usbctrlinfo;
+
+    libxl_device_usbctrl_init(&usbctrl);
+    libxl_usbctrlinfo_init(&usbctrlinfo);
+    usbctrl.devid = usbctrl_devid;
+
+    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo);
+    if (rc) goto out;
+
+    if (usbctrlinfo.type != LIBXL_USBCTRL_TYPE_PV) {
+        LOG(ERROR, "Unsupported USB controller type");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /* Remove usb devices first */
+    rc  = libxl__device_usb_list_for_usbctrl(gc, domid, usbctrl_devid,
+                                             &usbs, &numusb);
+    if (rc) goto out;
+
+    for (i = 0; i < numusb; i++) {
+        if (libxl__device_usb_remove(gc, domid, &usbs[i])) {
+            LOG(ERROR, "libxl__device_usb_remove failed");
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    }
+
+    libxl_device_usbctrl_dispose(&usbctrl);
+    libxl_usbctrlinfo_dispose(&usbctrlinfo);
+
+    /* Remove usbctrl */
+    return libxl__initiate_device_remove(egc, aodev);
+
+out:
+    libxl_device_usbctrl_dispose(&usbctrl);
+    libxl_usbctrlinfo_dispose(&usbctrlinfo);
+    aodev->rc = rc;
+    if (rc) aodev->callback(egc, aodev);
+    return;
+}
+
+libxl_device_usbctrl *
+libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num)
+{
+    GC_INIT(ctx);
+    libxl_device_usbctrl *usbctrls = NULL;
+    char *path = NULL;
+    char **entry = NULL;
+    unsigned int nentries = 0;
+
+    *num = 0;
+
+    path = GCSPRINTF("%s/device/vusb",
+                     libxl__xs_get_dompath(gc, domid));
+    entry = libxl__xs_directory(gc, XBT_NULL, path, &nentries);
+
+    if (entry && nentries) {
+        usbctrls = libxl__zalloc(NOGC, sizeof(*usbctrls) * nentries);
+        libxl_device_usbctrl *usbctrl;
+        libxl_device_usbctrl *end = usbctrls + nentries;
+        for (usbctrl = usbctrls; usbctrl < end;
+             usbctrl++, entry++, (*num)++) {
+            const char *tmp, *be_path;
+            int rc;
+            const char *fe_path = GCSPRINTF("%s/%s", path, *entry);
+
+            libxl_device_usbctrl_init(usbctrl);
+            usbctrl->devid = atoi(*entry);
+
+#define READ_SUBPATH(path, subpath) ({                                  \
+        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \
+                                    GCSPRINTF("%s/" subpath, path),     \
+                                    &tmp);                              \
+        if (rc) goto outerr;                                            \
+        (char *)tmp;                                                    \
+    })
+            be_path = READ_SUBPATH(fe_path, "backend");
+            usbctrl->backend_domid = atoi(READ_SUBPATH(fe_path, "backend-id"));
+            usbctrl->version = atoi(READ_SUBPATH(be_path, "usb-ver"));
+            usbctrl->ports = atoi(READ_SUBPATH(be_path, "num-ports"));
+            libxl_usbctrl_type_from_string(READ_SUBPATH(be_path, "type"),
+                                           &usbctrl->type);
+#undef READ_SUBPATH
+       }
+    }
+
+    goto out;
+
+outerr:
+    LOG(ERROR, "Unable to list USB Controllers");
+    libxl_device_usbctrl_list_free(usbctrls, *num);
+    *num = 0;
+    usbctrls = NULL;
+
+out:
+    GC_FREE;
+    return usbctrls;
+}
+
+int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
+                                libxl_device_usbctrl *usbctrl,
+                                libxl_usbctrlinfo *usbctrlinfo)
+{
+    GC_INIT(ctx);
+    const char *dompath, *fe_path, *be_path, *tmp;
+    int rc;
+
+    usbctrlinfo->devid = usbctrl->devid;
+
+#define READ_SUBPATH(path, subpath) ({                                  \
+        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \
+                                    GCSPRINTF("%s/" subpath, path),     \
+                                    &tmp);                              \
+        if (rc) goto out;                                               \
+        (char *)tmp;                                                    \
+    })
+
+    dompath = libxl__xs_get_dompath(gc, domid);
+    fe_path = GCSPRINTF("%s/device/vusb/%d", dompath, usbctrl->devid);
+    be_path = READ_SUBPATH(fe_path, "backend");
+
+    usbctrlinfo->backend = libxl__strdup(NOGC, be_path);
+
+    tmp = READ_SUBPATH(fe_path, "backend-id");
+    usbctrlinfo->backend_id = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(fe_path, "state");
+    usbctrlinfo->state = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(fe_path, "event-channel");
+    usbctrlinfo->evtch = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(fe_path, "urb-ring-ref");
+    usbctrlinfo->ref_urb = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(fe_path, "conn-ring-ref");
+    usbctrlinfo->ref_conn = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(be_path, "frontend");
+    usbctrlinfo->frontend = libxl__strdup(NOGC, tmp);
+    tmp = READ_SUBPATH(be_path, "frontend-id");
+    usbctrlinfo->frontend_id = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(be_path, "num-ports");
+    usbctrlinfo->ports = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(be_path, "usb-ver");
+    usbctrlinfo->version = tmp ? strtoul(tmp, NULL, 10) : -1;
+    tmp = READ_SUBPATH(be_path, "type");
+    libxl_usbctrl_type_from_string(tmp, &usbctrlinfo->type);
+
+#undef READ_SUBPATH
+
+    rc = 0;
+
+out:
+    GC_FREE;
+    return rc;
+}
+
+int libxl_devid_to_device_usbctrl(libxl_ctx *ctx,
+                                  uint32_t domid,
+                                  int devid,
+                                  libxl_device_usbctrl *usbctrl)
+{
+    libxl_device_usbctrl *usbctrls;
+    int nb = 0;
+    int i, rc;
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &nb);
+    if (!usbctrls)
+        return ERROR_FAIL;
+
+    rc = ERROR_FAIL;
+    for (i = 0; i < nb; i++) {
+        if (devid == usbctrls[i].devid) {
+            libxl_device_usbctrl_copy(ctx, usbctrl, &usbctrls[i]);
+            rc = 0;
+            break;
+        }
+    }
+
+    libxl_device_usbctrl_list_free(usbctrls, nb);
+    return rc;
+}
+
+static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
+{
+    struct dirent *de;
+    DIR *dir;
+    char *busid = NULL;
+
+    /* invalid hostbus or hostaddr */
+    if (bus < 1 || addr < 1)
+        return NULL;
+
+    if (!(dir = opendir(SYSFS_USB_DEV)))
+        return NULL;
+
+    while ((de = readdir(dir))) {
+        char *filename;
+        void *buf;
+        int busnum = -1;
+        int devnum = -1;
+
+        if (!de->d_name)
+            continue;
+
+        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
+        if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
+            devnum = atoi((char *)buf);
+
+        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
+        if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
+            busnum = atoi((char *)buf);
+
+        if (bus == busnum && addr == devnum) {
+            busid = libxl__strdup(gc, de->d_name);
+            break;
+        }
+    }
+
+    closedir(dir);
+    return busid;
+}
+
+static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid,
+                                  uint8_t *bus, uint8_t *addr)
+{
+    char *filename;
+    void *buf;
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
+    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
+        *bus = atoi((char *)buf);
+    else
+        return ERROR_FAIL;
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", busid);
+    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
+        *addr = atoi((char *)buf);
+    else
+        return ERROR_FAIL;
+
+    return 0;
+}
+
+static int
+get_assigned_devices(libxl__gc *gc,
+                     libxl_device_usb **list, int *num)
+{
+    char **domlist;
+    unsigned int nd = 0, i, j, k;
+    int rc;
+
+    *list = NULL;
+    *num = 0;
+
+    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd);
+    for (i = 0; i < nd; i++) {
+        char *path, **ctrl_list;
+        unsigned int nc = 0;
+
+        path = GCSPRINTF("/local/domain/%s/device/vusb", domlist[i]);
+        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path, &nc);
+
+        for (j = 0; j < nc; j++) {
+            const char *be_path, *num_ports;
+            int nport;
+
+            rc = libxl__xs_read_checked(gc, XBT_NULL,
+                          GCSPRINTF("%s/%s/backend", path, ctrl_list[j]),
+                          &be_path);
+            if (rc) goto out;
+
+            rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                        GCSPRINTF("%s/num-ports", be_path),
+                                        &num_ports);
+            if (rc) goto out;
+
+            nport = atoi(num_ports);
+            for (k = 0; k < nport; k++) {
+                libxl_device_usb *usb;
+                const char *portpath, *busid;
+
+                portpath = GCSPRINTF("%s/port/%d", be_path, k + 1);
+                busid = libxl__xs_read(gc, XBT_NULL, portpath);
+                /* If there is USB device attached, add it to list */
+                if (busid && strcmp(busid, "")) {
+                    GCREALLOC_ARRAY(*list, *num + 1);
+                    usb = *list + *num;
+                    (*num)++;
+                    libxl_device_usb_init(usb);
+                    usb->ctrl = atoi(ctrl_list[j]);
+                    usb->port = k + 1;
+                    rc = usb_busaddr_from_busid(gc, busid,
+                                                &usb->u.hostdev.hostbus,
+                                                &usb->u.hostdev.hostaddr);
+                    if (rc) goto out;
+                }
+            }
+        }
+    }
+
+    rc = 0;
+
+out:
+    if (rc) {
+        *list = NULL;
+        *num = 0;
+    }
+    return rc;
+}
+
+static bool is_usbdev_in_array(libxl_device_usb *usbs, int num,
+                               libxl_device_usb *usb)
+{
+    int i;
+
+    for (i = 0; i < num; i++) {
+        if (usbs[i].u.hostdev.hostbus == usb->u.hostdev.hostbus &&
+            usbs[i].u.hostdev.hostaddr == usb->u.hostdev.hostaddr)
+            return true;
+    }
+
+    return false;
+}
+
+/* check if USB device is already assigned to a domain */
+/* check if USB device type is assignable */
+static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
+{
+    int classcode;
+    char *filename;
+    void *buf = NULL;
+    char *busid = NULL;
+
+    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
+                                 usb->u.hostdev.hostaddr);
+    if (!busid) return false;
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid);
+    if (libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
+        return false;
+
+    classcode = atoi(buf);
+    return classcode != USBHUB_CLASS_CODE;
+}
+
+/* get usb devices under certain usb controller */
+static int
+libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
+                                   libxl_devid usbctrl,
+                                   libxl_device_usb **usbs, int *num)
+{
+    const char *fe_path, *be_path, *num_devs;
+    int n, i, rc;
+
+    *usbs = NULL;
+    *num = 0;
+
+    fe_path = GCSPRINTF("%s/device/vusb/%d",
+                        libxl__xs_get_dompath(gc, domid), usbctrl);
+
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                GCSPRINTF("%s/backend", fe_path),
+                                &be_path);
+    if (rc) return rc;
+
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                GCSPRINTF("%s/num-ports", be_path),
+                                &num_devs);
+    if (rc) return rc;
+
+    n = atoi(num_devs);
+
+    for (i = 0; i < n; i++) {
+        char *busid;
+        libxl_device_usb *usb;
+
+        busid = libxl__xs_read(gc, XBT_NULL,
+                               GCSPRINTF("%s/port/%d", be_path, i + 1));
+        if (busid && strcmp(busid, "")) {
+            GCREALLOC_ARRAY(*usbs, *num + 1);
+            usb = *usbs + *num;
+            (*num)++;
+            libxl_device_usb_init(usb);
+            usb->ctrl = usbctrl;
+            usb->port = i + 1;
+            rc = usb_busaddr_from_busid(gc, busid,
+                                        &usb->u.hostdev.hostbus,
+                                        &usb->u.hostdev.hostaddr);
+            if (rc) goto out;
+        }
+    }
+
+    rc = 0;
+
+out:
+    if (rc) {
+        *usbs = NULL;
+        *num = 0;
+    }
+    return rc;
+}
+
+/* get all usb devices of the domain */
+libxl_device_usb *
+libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, int *num)
+{
+    GC_INIT(ctx);
+    char *path;
+    char **usbctrls;
+    unsigned int nc = 0;
+    int i, j;
+    libxl_device_usb *usbs = NULL;
+
+    *num = 0;
+
+    path = GCSPRINTF("%s/device/vusb",
+                        libxl__xs_get_dompath(gc, domid));
+    usbctrls = libxl__xs_directory(gc, XBT_NULL, path, &nc);
+
+    for (i = 0; i < nc; i++) {
+        int nd = 0;
+        libxl_device_usb *tmp = NULL;
+        libxl__device_usb_list_for_usbctrl(gc, domid,
+                                           atoi(usbctrls[i]), &tmp, &nd);
+        if (!nd) continue;
+
+        usbs = libxl__realloc(NOGC, usbs, sizeof(*usbs) * (*num + nd));
+        for (j = 0; j < nd; j++) {
+            libxl_device_usb_copy(ctx, usbs + *num, tmp + j);
+            (*num)++;
+        }
+    }
+
+    GC_FREE;
+    return usbs;
+}
+
+/* find first unused controller:port and give that to usb device */
+static int
+libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
+                                      libxl_device_usb *usb)
+{
+    libxl_ctx *ctx = CTX;
+    libxl_device_usbctrl *usbctrls = NULL;
+    int numctrl = 0;
+    int i, j, rc;
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
+    if (!numctrl) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = ERROR_FAIL;
+    for (i = 0; i < numctrl; i++) {
+        for (j = 0; j < usbctrls[i].ports; j++) {
+            char *path, *tmp;
+
+            path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                             libxl__xs_get_dompath(gc, LIBXL_TOOLSTACK_DOMID),
+                             domid, usbctrls[i].devid, j + 1);
+            tmp = libxl__xs_read(gc, XBT_NULL, path);
+            if (tmp && !strcmp(tmp, "")) {
+                usb->ctrl = usbctrls[i].devid;
+                usb->port = j + 1;
+                rc = 0;
+                goto out;
+            }
+        }
+    }
+
+out:
+    libxl_device_usbctrl_list_free(usbctrls, numctrl);
+    return rc;
+}
+
+/* Fill in usb information with default value.
+ *
+ * Generally, it does:
+ * 1) if "controller" is not specified:
+ *    - if "port" is not specified, try to find an available controller:port,
+ *      if found, use that; otherwise, create a new controller, use this
+ *      controller and its first port
+ *    - if "port" is specified, report error.
+ * 2) if "controller" is specified, but port is not specified:
+ *    try to find an available port under this controller, if found, use
+ *    that, otherwise, report error.
+ * 3) if both "controller" and "port" are specified:
+ *    check the controller:port is available, if not, report error.
+ */
+static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
+                                        libxl_device_usb *usb,
+                                        bool update_json)
+{
+    int rc;
+
+    if (!usb->devtype)
+        usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV;
+
+    if (usb->ctrl == -1) {
+        if (usb->port) {
+            LOG(ERROR, "USB controller must be specified if you specify port");
+            return ERROR_INVAL;
+        }
+
+        rc = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
+        /* If no existing controller to host this usb device, add a new one */
+        if (rc) {
+            libxl_device_usbctrl *usbctrl;
+
+            GCNEW(usbctrl);
+            libxl_device_usbctrl_init(usbctrl);
+            rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
+            if (rc < 0) goto out;
+
+            if (usbctrl->devid == -1) {
+                usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");
+                if (usbctrl->devid < 0) {
+                    rc = ERROR_FAIL;
+                    goto out;
+                }
+            }
+
+            rc = libxl__device_usbctrl_add_xenstore(gc, domid, usbctrl,
+                                                    update_json);
+            if (rc) goto out;
+
+            usb->ctrl = usbctrl->devid;
+            usb->port = 1;
+        }
+    } else if (!usb->port) {
+        /* Valid port starts from 1. Choose port for us. */
+        int i, ports;
+        const char *fe_path, *be_path, *tmp;
+
+        fe_path = GCSPRINTF("%s/device/vusb/%d",
+                         libxl__xs_get_dompath(gc, domid), usb->ctrl);
+
+        rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                    GCSPRINTF("%s/backend", fe_path), &be_path);
+        if (rc) goto out;
+
+        rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                    GCSPRINTF("%s/num-ports", be_path), &tmp);
+        if (rc) goto out;
+
+        ports = atoi(tmp);
+
+        for (i = 0; i < ports; i++) {
+            tmp = libxl__xs_read(gc, XBT_NULL,
+                                 GCSPRINTF("%s/port/%d", be_path, i + 1));
+            if (tmp && !strcmp(tmp, "")) {
+                usb->port = i + 1;
+                break;
+            }
+        }
+
+        if (!usb->port) {
+            LOG(ERROR, "No available port under specified controller");
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    } else {
+        const char *fe_path, *be_path, *tmp;
+
+        fe_path = GCSPRINTF("%s/device/vusb/%d",
+                         libxl__xs_get_dompath(gc, domid), usb->ctrl);
+
+        rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                    GCSPRINTF("%s/backend", fe_path), &be_path);
+        if (rc) goto out;
+
+        rc = libxl__xs_read_checked(gc, XBT_NULL,
+                                    GCSPRINTF("%s/port/%d", be_path, usb->port),
+                                    &tmp);
+        if (rc) goto out;
+
+        if (strcmp(tmp, "")) {
+            LOG(ERROR, "The controller port isn't available");
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    }
+
+    rc = 0;
+
+out:
+    return rc;
+}
+
+/* Add usb information to xenstore
+ *
+ * Adding a usb device won't create new 'vusb' device, but only write
+ * the device busid to the controller:port in xenstore.
+ */
+static int libxl__device_usb_add_xenstore(libxl__gc *gc, uint32_t domid,
+                                          libxl_device_usb *usb,
+                                          bool update_json)
+{
+    char *be_path;
+    char *busid;
+    int rc;
+    xs_transaction_t t = XBT_NULL;
+    libxl_domain_config d_config;
+    libxl_device_usb usb_saved;
+    libxl__domain_userdata_lock *lock = NULL;
+
+    libxl_domain_config_init(&d_config);
+    libxl_device_usb_init(&usb_saved);
+    libxl_device_usb_copy(CTX, &usb_saved, usb);
+
+    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
+                                 usb->u.hostdev.hostaddr);
+    if (!busid) {
+        LOG(DEBUG, "Fail to get busid of usb device");
+        goto out;
+    }
+
+    if (update_json) {
+        lock = libxl__lock_domain_userdata(gc, domid);
+        if (!lock) {
+            rc = ERROR_LOCK_FAIL;
+            goto out;
+        }
+
+        rc = libxl__get_domain_configuration(gc, domid, &d_config);
+        if (rc) goto out;
+
+        DEVICE_ADD(usb, usbs, domid, &usb_saved, COMPARE_USB, &d_config);
+    }
+
+    for (;;) {
+        rc = libxl__xs_transaction_start(gc, &t);
+        if (rc) goto out;
+
+        if (update_json) {
+            rc = libxl__set_domain_configuration(gc, domid, &d_config);
+            if (rc) goto out;
+        }
+
+        be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                            libxl__xs_get_dompath(gc, LIBXL_TOOLSTACK_DOMID),
+                            domid, usb->ctrl, usb->port);
+
+        LOG(DEBUG, "Adding new usb device to xenstore");
+        if (libxl__xs_write_checked(gc, t, be_path, busid))
+            goto out;
+
+        rc = libxl__xs_transaction_commit(gc, &t);
+        if (!rc) break;
+        if (rc < 0) goto out;
+    }
+
+    rc = 0;
+
+out:
+    if (lock) libxl__unlock_domain_userdata(lock);
+    libxl_device_usb_dispose(&usb_saved);
+    libxl_domain_config_dispose(&d_config);
+    return rc;
+}
+
+static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t domid,
+                                             libxl_device_usb *usb)
+{
+    char *be_path;
+
+    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                        libxl__xs_get_dompath(gc, LIBXL_TOOLSTACK_DOMID),
+                        domid, usb->ctrl, usb->port);
+    LOG(DEBUG, "Removing USB device from xenstore");
+    if (libxl__xs_write_checked(gc, XBT_NULL, be_path, ""))
+        return ERROR_FAIL;
+
+    return 0;
+}
+
+/* bind/unbind usb device interface */
+static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)
+{
+    char *path, *spath, *dp = NULL;
+    int fd = -1;
+    int rc;
+    struct stat st;
+
+    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf);
+    if (!lstat(spath, &st)) {
+        /* Find the canonical path to the driver. */
+        dp = libxl__zalloc(gc, PATH_MAX);
+        dp = realpath(spath, dp);
+
+        path = GCSPRINTF("%s/unbind", spath);
+        fd = open(path, O_WRONLY);
+        if (fd < 0)
+            return ERROR_FAIL;
+        rc = write(fd, intf, strlen(intf));
+        close(fd);
+        if (rc < 0)
+            return ERROR_FAIL;
+    }
+
+    if (drvpath)
+        *drvpath = dp;
+
+    return 0;
+}
+
+static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath)
+{
+    char *path;
+    struct stat st;
+    int fd, rc;
+
+    path = GCSPRINTF("%s/%s", drvpath, intf);
+    rc = lstat(path, &st);
+    /* already bind, return */
+    if (!rc)
+        return 0;
+
+    path = GCSPRINTF("%s/bind", drvpath);
+    fd = open(path, O_WRONLY);
+    if (fd < 0)
+        return ERROR_FAIL;
+
+    rc = write(fd, intf, strlen(intf));
+    close(fd);
+    if (rc < 0)
+        return ERROR_FAIL;
+
+    return 0;
+}
+
+/* Is usb interface bound to usbback? */
+static int usb_intf_is_assigned(libxl__gc *gc, char *intf)
+{
+    char *spath;
+    int rc;
+    struct stat st;
+
+    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf);
+    rc = lstat(spath, &st);
+
+    if (rc == 0)
+        return 1;
+    if (rc < 0 && errno == ENOENT)
+        return 0;
+    LOGE(ERROR, "Accessing %s", spath);
+    return -1;
+}
+
+static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb,
+                                  char ***intfs, int *num)
+{
+    DIR *dir;
+    struct dirent *entry;
+    char *buf;
+    char *busid;
+    int rc;
+
+    *intfs = NULL;
+    *num = 0;
+
+    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
+                                 usb->u.hostdev.hostaddr);
+    if (!busid) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    buf = GCSPRINTF("%s:", busid);
+
+    if (!(dir = opendir(SYSFS_USB_DEV))) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    while ((entry = readdir(dir)) != NULL) {
+        if (!strncmp(entry->d_name, buf, strlen(buf))) {
+            GCREALLOC_ARRAY(*intfs, *num + 1);
+            if (*intfs == NULL) {
+                rc = ERROR_FAIL;
+                goto out;
+            }
+            (*intfs)[*num] = libxl__strdup(gc, entry->d_name);
+            (*num)++;
+        }
+    }
+
+    closedir(dir);
+    rc = 0;
+
+out:
+    return rc;
+}
+
+/* Encode usb interface so that it could be written to xenstore as a key.
+ *
+ * Since xenstore key cannot include '.' or ':', we'll change '.' to '_',
+ * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1.
+ * This will be used to save original driver of USB device to xenstore.
+ */
+static char *usb_interface_xenstore_encode(char *busid)
+{
+    char *str = strdup(busid);
+    int i, len = strlen(str);
+
+    for (i = 0; i < len; i++) {
+        if (str[i] == '.')
+            str[i] = '_';
+         if (str[i] == ':')
+            str[i] = '-';
+    }
+    return str;
+}
+
+/* Unbind USB device from "usbback" driver.
+ *
+ * If there are many interfaces under USB device, check each interface,
+ * unbind from "usbback" driver and rebind to its original driver.
+ */
+static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
+{
+    char **intfs = NULL;
+    char *path;
+    int num = 0, i;
+    int rc;
+    char *busid;
+    char *usb_encode = NULL;
+
+    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
+        return ERROR_FAIL;
+
+    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
+                                 usb->u.hostdev.hostaddr);
+    usb_encode = usb_interface_xenstore_encode(busid);
+
+    for (i = 0; i < num; i++) {
+        char *intf = intfs[i];
+        char *drvpath = NULL;
+
+        /* check if the USB interface is already bound to "usbbcak" */
+        if (usb_intf_is_assigned(gc, intf) > 0) {
+            /* unbind interface from usbback driver */
+            if (unbind_usb_intf(gc, intf, NULL) < 0) {
+                rc = ERROR_FAIL;
+                goto out;
+            }
+        }
+
+        /* bind interface to its originial driver */
+        drvpath = libxl__xs_read(gc, XBT_NULL,
+                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
+                  usb_encode, usb_interface_xenstore_encode(intf)));
+        if (drvpath && bind_usb_intf(gc, intf, drvpath))
+            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath);
+    }
+
+    /* finally, remove xs driver path */
+    path = GCSPRINTF(USBBACK_INFO_PATH"/%s", usb_encode);
+    libxl__xs_rm_checked(gc, XBT_NULL, path);
+    rc = 0;
+
+out:
+    free(usb_encode);
+    return rc;
+}
+
+/* Bind USB device to "usbback" driver.
+ *
+ * If there are many interfaces under USB device, check each interface,
+ * unbind from original driver and bind to "usbback" driver.
+ */
+static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
+{
+    char **intfs = NULL;
+    int num = 0, i;
+    int rc;
+    char *busid;
+    char *usb_encode = NULL;
+
+    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
+        return ERROR_FAIL;
+
+    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
+                                 usb->u.hostdev.hostaddr);
+    usb_encode = usb_interface_xenstore_encode(busid);
+
+    for (i = 0; i < num; i++) {
+        char *intf = intfs[i];
+        char *path = NULL;
+        char *drvpath = NULL;
+
+        /* already assigned to usbback */
+        if (usb_intf_is_assigned(gc, intf) > 0)
+            continue;
+
+        /* unbind interface from original driver */
+        if (unbind_usb_intf(gc, intf, &drvpath) < 0) {
+            rc = ERROR_FAIL;
+            goto out_rebind;
+        }
+
+        if (drvpath) {
+            /* write driver path to xenstore for later rebinding */
+            path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
+                             usb_encode, usb_interface_xenstore_encode(intf));
+            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
+                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
+            }
+        }
+
+        /* bind interface to usbback */
+        if (bind_usb_intf(gc, intf, SYSFS_USBBACK_DRIVER) < 0) {
+            LOGE(ERROR, "Couldn't bind %s to %s", intf, SYSFS_USBBACK_DRIVER);
+            rc = ERROR_FAIL;
+            goto out_rebind;
+        }
+    }
+
+    rc = 0;
+    goto out;
+
+out_rebind:
+    /* some interfaces might be bound to usbback, unbind it then and
+     * rebind to its original driver
+     */
+    usbback_dev_unassign(gc, usb);
+out:
+    free(usb_encode);
+    return rc;
+}
+
+/*
+ * USB add
+ */
+static int do_usb_add(libxl__gc *gc, uint32_t domid,
+                      libxl_device_usb *usbdev,
+                      libxl__ao_device *aodev)
+{
+    libxl_ctx *ctx = CTX;
+    libxl_usbctrlinfo usbctrlinfo;
+    libxl_device_usbctrl usbctrl;
+    int rc;
+
+    libxl_device_usbctrl_init(&usbctrl);
+    libxl_usbctrlinfo_init(&usbctrlinfo);
+    usbctrl.devid = usbdev->ctrl;
+    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo);
+    if (rc)
+        goto out;
+
+    switch (usbctrlinfo.type) {
+    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:
+        LOG(ERROR, "Not supported");
+        break;
+    case LIBXL_USBCTRL_TYPE_PV:
+        rc = libxl__device_usb_add_xenstore(gc, domid, usbdev,
+                                            aodev->update_json);
+        if (rc) goto out;
+
+        rc = usbback_dev_assign(gc, usbdev);
+        if (rc) {
+            libxl__device_usb_remove_xenstore(gc, domid, usbdev);
+            goto out;
+        }
+        break;
+    default:
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+out:
+    libxl_device_usbctrl_dispose(&usbctrl);
+    libxl_usbctrlinfo_dispose(&usbctrlinfo);
+    return rc;
+}
+
+/* AO operation to add a usb device.
+ *
+ * Generally, it does:
+ * 1) check if the usb device type is assignable
+ * 2) check if the usb device is already assigned to a domain
+ * 3) add 'busid' of the usb device to xenstore contoller/port/.
+ *    (PVUSB driver watches the xenstore changes and will detect that.)
+ * 4) unbind usb device from original driver and bind to usbback.
+ *    If usb device has many interfaces, then:
+ *    - unbind each interface from its original driver and bind to usbback.
+ *    - store the original driver to xenstore for later rebinding when
+ *      detaching the device.
+ *
+ * Before calling this function, aodev should be properly filled:
+ * aodev->ao, aodev->callback, aodev->update_json, ...
+ */
+void libxl__device_usb_add(libxl__egc *egc, uint32_t domid,
+                           libxl_device_usb *usb,
+                           libxl__ao_device *aodev)
+{
+    STATE_AO_GC(aodev->ao);
+    int rc;
+    char *busid = NULL;
+    libxl_device_usb *assigned;
+    int num_assigned;
+    libxl_device_usbctrl usbctrl;
+    libxl_usbctrlinfo usbctrlinfo;
+
+    libxl_device_usbctrl_init(&usbctrl);
+    libxl_usbctrlinfo_init(&usbctrlinfo);
+
+    /* Currently only support adding USB device from Dom0 backend.
+     * So, if USB controller is specified, check backend domain, if
+     * it is not Dom0, report error as not supported.
+     *
+     * -1 is the default value of libxl_device_usbctrl.
+     */
+    if (usb->ctrl != -1) {
+        usbctrl.devid = usb->ctrl;
+        rc = libxl_device_usbctrl_getinfo(CTX, domid, &usbctrl, &usbctrlinfo);
+        if (rc) goto out;
+
+        if (usbctrlinfo.backend_id != 0) {
+            LOG(ERROR, "Don't support adding USB device from non-Dom0 backend");
+            rc = ERROR_INVAL;
+            goto out;
+        }
+    }
+
+    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
+                                 usb->u.hostdev.hostaddr);
+    if (!busid) {
+        LOG(ERROR, "USB device doesn't exist in sysfs");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    if (!is_usb_assignable(gc, usb)) {
+        LOG(ERROR, "USB device is not assignable.");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /* check usb device is already assigned */
+    rc = get_assigned_devices(gc, &assigned, &num_assigned);
+    if (rc) {
+        LOG(ERROR, "cannot determine if device is assigned,"
+                   " refusing to continue");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    if (is_usbdev_in_array(assigned, num_assigned, usb)) {
+        LOG(ERROR, "USB device already attached to a domain");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = libxl__device_usb_setdefault(gc, domid, usb, aodev->update_json);
+    if (rc) goto out;
+
+    /* Do the add */
+    if (do_usb_add(gc, domid, usb, aodev)) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    libxl__ao_complete(egc, ao, 0);
+    rc = 0;
+
+out:
+    libxl_device_usbctrl_dispose(&usbctrl);
+    libxl_usbctrlinfo_dispose(&usbctrlinfo);
+    aodev->rc = rc;
+    if (rc) aodev->callback(egc, aodev);
+    return;
+}
+
+static int do_usb_remove(libxl__gc *gc, uint32_t domid,
+                         libxl_device_usb *usbdev)
+{
+    libxl_ctx *ctx = CTX;
+    libxl_usbctrlinfo usbctrlinfo;
+    libxl_device_usbctrl usbctrl;
+    int rc;
+
+    libxl_device_usbctrl_init(&usbctrl);
+    libxl_usbctrlinfo_init(&usbctrlinfo);
+    usbctrl.devid = usbdev->ctrl;
+
+    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo);
+    if (rc) goto out;
+
+    if (usbctrlinfo.backend_id != 0) {
+        LOG(ERROR, "Don't support removing USB device from non-Dom0 backend");
+        rc = ERROR_INVAL;
+        goto out;
+    }
+
+    switch (usbctrlinfo.type) {
+    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:
+        LOG(ERROR, "Not supported");
+        break;
+    case LIBXL_USBCTRL_TYPE_PV:
+        rc = libxl__device_usb_remove_xenstore(gc, domid, usbdev);
+        if (rc) goto out;
+
+        usbback_dev_unassign(gc, usbdev);
+        break;
+    default:
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = 0;
+
+out:
+    libxl_device_usbctrl_dispose(&usbctrl);
+    libxl_usbctrlinfo_dispose(&usbctrlinfo);
+    return rc;
+}
+
+/* Operation to remove usb device.
+ *
+ * Generally, it does:
+ * 1) check if the usb device is assigned to the domain
+ * 2) remove the usb device from xenstore controller/port.
+ * 3) unbind usb device from usbback and rebind to its original driver.
+ *    If usb device has many interfaces, do it to each interface.
+ */
+static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
+                                    libxl_device_usb *usb)
+{
+    int rc;
+
+    if (usb->ctrl < 0 || usb->port < 1) {
+        LOG(ERROR, "Invalid USB device");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV &&
+        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) {
+        LOG(ERROR, "Invalid USB device of hostdev");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /* Do the remove */
+    rc = do_usb_remove(gc, domid, usb);
+
+out:
+    return rc;
+}
+
+int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
+                            libxl_device_usb *usb,
+                            const libxl_asyncop_how *ao_how)
+
+{
+    AO_CREATE(ctx, domid, ao_how);
+    int rc;
+
+    rc = libxl__device_usb_remove(gc, domid, usb);
+
+    libxl__ao_complete(egc, ao, rc);
+    return AO_INPROGRESS;
+}
+
+int libxl_ctrlport_to_device_usb(libxl_ctx *ctx,
+                                 uint32_t domid,
+                                 int ctrl,
+                                 int port,
+                                 libxl_device_usb *usb)
+{
+    GC_INIT(ctx);
+    const char *dompath, *be_path, *busid;
+    int rc;
+
+    dompath = libxl__xs_get_dompath(gc, domid);
+
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                  GCSPRINTF("%s/device/vusb/%d/backend", dompath, ctrl),
+                  &be_path);
+    if (rc) goto out;
+
+    rc = libxl__xs_read_checked(gc, XBT_NULL,
+                           GCSPRINTF("%s/port/%d", be_path, port),
+                           &busid);
+    if (rc) goto out;
+
+    if (!strcmp(busid, "")) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    usb->ctrl = ctrl;
+    usb->port = port;
+    rc = usb_busaddr_from_busid(gc, busid, &usb->u.hostdev.hostbus,
+                                &usb->u.hostdev.hostaddr);
+
+out:
+    GC_FREE;
+    return rc;
+}
+/*
+ * Local variables:
+ * mode: C
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index d6ef9a2..dccb317 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -603,6 +603,35 @@ libxl_device_rdm = Struct("device_rdm", [
     ("policy", libxl_rdm_reserve_policy),
     ])
 
+libxl_usbctrl_type = Enumeration("usbctrl_type", [
+    (0, "AUTO"),
+    (1, "PV"),
+    (2, "DEVICEMODEL"),
+    ])
+
+libxl_usbdev_type = Enumeration("usbdev_type", [
+    (1, "hostdev"),
+    ])
+
+libxl_device_usbctrl = Struct("device_usbctrl", [
+    ("type", libxl_usbctrl_type),
+    ("devid", libxl_devid),
+    ("version", integer),
+    ("ports", integer),
+    ("backend_domid", libxl_domid),
+    ("backend_domname", string),
+   ])
+
+libxl_device_usb = Struct("device_usb", [
+    ("ctrl", libxl_devid),
+    ("port", integer),
+    ("u", KeyedUnion(None, libxl_usbdev_type, "devtype",
+           [("hostdev", Struct(None, [
+                 ("hostbus",   uint8),
+                 ("hostaddr",  uint8)])),
+           ])),
+    ])
+
 libxl_device_dtdev = Struct("device_dtdev", [
     ("path", string),
     ])
@@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [
     ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
     ("rdms", Array(libxl_device_rdm, "num_rdms")),
     ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
+    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")),
+    ("usbs", Array(libxl_device_usb, "num_usbs")),
     ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
     ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
     ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
@@ -684,6 +715,32 @@ libxl_vtpminfo = Struct("vtpminfo", [
     ("uuid", libxl_uuid),
     ], dir=DIR_OUT)
 
+libxl_usbctrlinfo = Struct("usbctrlinfo", [
+    ("type", libxl_usbctrl_type),
+    ("devid", libxl_devid),
+    ("version", integer),
+    ("ports", integer),
+    ("backend", string),
+    ("backend_id", uint32),
+    ("frontend", string),
+    ("frontend_id", uint32),
+    ("state", integer),
+    ("evtch", integer),
+    ("ref_urb", integer),
+    ("ref_conn", integer),
+    ], dir=DIR_OUT)
+
+libxl_usbinfo = Struct("usbinfo", [
+    ("ctrl", libxl_devid),
+    ("port", integer),
+    ("busnum", uint8),
+    ("devnum", uint8),
+    ("idVendor", uint16),
+    ("idProduct", uint16),
+    ("prod", string),
+    ("manuf", string),
+    ], dir=DIR_OUT)
+
 libxl_vcpuinfo = Struct("vcpuinfo", [
     ("vcpuid", uint32),
     ("cpu", uint32),
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index 5e55685..696f5f8 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -22,6 +22,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (6, "VKBD"),
     (7, "CONSOLE"),
     (8, "VTPM"),
+    (9, "VUSB"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 9c5c4d0..706a0c1 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -1270,6 +1270,22 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf, size_t len)
     return ret;
 }
 
+void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr)
+{
+   int i;
+   for (i = 0; i < nr; i++)
+       libxl_device_usbctrl_dispose(&list[i]);
+   free(list);
+}
+
+void libxl_device_usb_list_free(libxl_device_usb *list, int nr)
+{
+   int i;
+   for (i = 0; i < nr; i++)
+       libxl_device_usb_dispose(&list[i]);
+   free(list);
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h
index 339ebdf..60487e3 100644
--- a/tools/libxl/libxl_utils.h
+++ b/tools/libxl/libxl_utils.h
@@ -76,6 +76,11 @@ int libxl_uuid_to_device_vtpm(libxl_ctx *ctx, uint32_t domid,
                                libxl_uuid *uuid, libxl_device_vtpm *vtpm);
 int libxl_devid_to_device_vtpm(libxl_ctx *ctx, uint32_t domid,
                                int devid, libxl_device_vtpm *vtpm);
+int libxl_devid_to_device_usbctrl(libxl_ctx *ctx, uint32_t domid,
+                                  int devid, libxl_device_usbctrl *usbctrl);
+int libxl_ctrlport_to_device_usb(libxl_ctx *ctx, uint32_t domid,
+                                 int ctrl, int port,
+                                 libxl_device_usb *usb);
 
 int libxl_bitmap_alloc(libxl_ctx *ctx, libxl_bitmap *bitmap, int n_bits);
     /* Allocated bimap is from malloc, libxl_bitmap_dispose() to be
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
                   ` (2 preceding siblings ...)
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:32   ` Juergen Gross
  2015-11-11 16:07   ` George Dunlap
  2015-10-21  9:08 ` [PATCH V8 5/7] xl: add pvusb commands Chunyan Liu
                   ` (2 subsequent siblings)
  6 siblings, 2 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig

Add API for listing assignable USB devices info.
Assignable USB device means the USB device type is assignable and
it's not assigned to any guest yet.

Signed-off-by: Chunyan Liu <cyliu@suse.com>

---
This could be squashed with previous patch. Split because there is
some dispute on this. If this is acceptable, could be squashed,
otherwise could be removed.

 tools/libxl/libxl.h       |  3 +++
 tools/libxl/libxl_pvusb.c | 54 +++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 57 insertions(+)

diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 594e23d..7548b6a 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -1472,6 +1472,9 @@ int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
                                  libxl_usbctrlinfo *usbctrlinfo);
 
 /* USB Devices */
+libxl_device_usb *
+libxl_device_usb_assignable_list(libxl_ctx *ctx, int *num);
+
 int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
                          const libxl_asyncop_how *ao_how)
                          LIBXL_EXTERNAL_CALLERS_ONLY;
diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
index aa1a653..ab57371 100644
--- a/tools/libxl/libxl_pvusb.c
+++ b/tools/libxl/libxl_pvusb.c
@@ -585,6 +585,60 @@ static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
     return classcode != USBHUB_CLASS_CODE;
 }
 
+libxl_device_usb *
+libxl_device_usb_assignable_list(libxl_ctx *ctx, int *num)
+{
+    GC_INIT(ctx);
+    libxl_device_usb *usbs = NULL;
+    libxl_device_usb *assigned;
+    int num_assigned;
+    struct dirent *de;
+    DIR *dir;
+
+    *num = 0;
+
+    if (get_assigned_devices(gc, &assigned, &num_assigned) < 0) {
+        LOG(ERROR, "cannot determine if device is assigned");
+        goto out;
+    }
+
+    if (!(dir = opendir(SYSFS_USB_DEV)))
+        goto out;
+
+    while ((de = readdir(dir))) {
+        libxl_device_usb *usb;
+        uint8_t bus, addr;
+
+        if (!de->d_name)
+            continue;
+
+        if (usb_busaddr_from_busid(gc, de->d_name, &bus, &addr))
+            continue;
+
+        GCNEW(usb);
+        usb->u.hostdev.hostbus = bus;
+        usb->u.hostdev.hostaddr = addr;
+
+        if (!is_usb_assignable(gc, usb))
+            continue;
+
+        if (is_usbdev_in_array(assigned, num_assigned, usb))
+            continue;
+
+        usbs = libxl__realloc(NOGC, usbs, sizeof(*usbs) * (*num + 1));
+        libxl_device_usb_init(usbs + *num);
+        usbs[*num].u.hostdev.hostbus = bus;
+        usbs[*num].u.hostdev.hostaddr = addr;
+        (*num)++;
+    }
+
+    closedir(dir);
+
+out:
+    GC_FREE;
+    return usbs;
+}
+
 /* get usb devices under certain usb controller */
 static int
 libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V8 5/7] xl: add pvusb commands
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
                   ` (3 preceding siblings ...)
  2015-10-21  9:08 ` [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:37   ` Juergen Gross
                     ` (2 more replies)
  2015-10-21  9:08 ` [PATCH V8 6/7] xl: add usb-assignable-list command Chunyan Liu
  2015-10-21  9:08 ` [PATCH V8 7/7] domcreate: support pvusb in configuration file Chunyan Liu
  6 siblings, 3 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig, Simon Cao

Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list,
usb-attach and usb-detach.

To attach a usb device to guest through pvusb, one could follow
following example:

 #xl usbctrl-attach test_vm version=1 ports=8

 #xl usb-list test_vm
 will show the usb controllers and port usage under the domain.

 #xl usbattach test_vm hostbus=1 hostaddr=2
 will find the first usable controller:port, and attach usb
 device whose busnum is 1 and devnum is 6.
 One could also specify which <controller> and which <port>.

 #xl usb-detach test_vm 0 1
 will detach USB device under controller 0 port 1.

 #xl usbctrl-detach test_vm dev_id
 will destroy the controller with specified dev_id. Dev_id
 can be traced in usb-list info.

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>

---
Changes:
  - change usb-attach parameter from hostbus.hostaddr to
    hostbus=xx hostaddr=
  - since we get rid of libxl_device_usb_getinfo, so adjust usb-list
    information a little bit.
  - parse_usb_config and parse_usbctrl_config following parse_nic_config
    way, put in this patch, and shared domcreate routine.

 docs/man/xl.pod.1         |  40 ++++++++
 tools/libxl/xl.h          |   5 +
 tools/libxl/xl_cmdimpl.c  | 250 ++++++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/xl_cmdtable.c |  25 +++++
 4 files changed, 320 insertions(+)

diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index d0cd612..f09a872 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -1345,6 +1345,46 @@ List pass-through pci devices for a domain.
 
 =back
 
+=head1 USB PASS-THROUGH
+
+=over 4
+
+=item B<usbctrl-attach> I<domain-id> I[<type=val>] [I<version=val>] [I<ports=number>]
+
+Create a new USB controller for the specified domain.
+B<type=val> is the usb controller type, currently only support 'pv'.
+B<version=val> is the usb controller version, could be 1 (USB1.1) or 2 (USB2.0).
+B<ports=number> is the total ports of the usb controller.
+By default, it will create a USB2.0 controller with 8 ports.
+
+=item B<usbctrl-detach> I<domain-id> I<devid>
+
+Destroy a USB controller from the specified domain.
+B<devid> is devid of the USB controller.
+
+If B<-f> is specified, B<xl> is going to forcefully remove the device even
+without guest's collaboration.
+
+=item B<usb-attach> I<domain-id> I<hostbus=busnum> I<hostaddr=devnum> [I<controller=devid> [I<port=number>]]
+
+Hot-plug a new pass-through USB device to the specified domain.
+B<bus.addr> is the busnum.devnum of the physical USB device to pass-through.
+B<controller=devid> B<port=number> is the USB controller:port to hotplug the
+USB device to. By default, it will find the first available controller:port
+and use it; if there is no controller, it will create one.
+
+=item B<usb-detach> I<domain-id> I<controller=devid> I<port=number>
+
+Hot-unplug a previously assigned USB device from a domain.
+B<controller=devid> and B<port=number> is USB controller:port in guest where the
+USB device is attached to.
+
+=item B<usb-list> I<domain-id>
+
+List pass-through usb devices for a domain.
+
+=back
+
 =head1 TMEM
 
 =over 4
diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
index 0021112..ddd9690 100644
--- a/tools/libxl/xl.h
+++ b/tools/libxl/xl.h
@@ -85,6 +85,11 @@ int main_blockdetach(int argc, char **argv);
 int main_vtpmattach(int argc, char **argv);
 int main_vtpmlist(int argc, char **argv);
 int main_vtpmdetach(int argc, char **argv);
+int main_usbctrl_attach(int argc, char **argv);
+int main_usbctrl_detach(int argc, char **argv);
+int main_usbattach(int argc, char **argv);
+int main_usbdetach(int argc, char **argv);
+int main_usblist(int argc, char **argv);
 int main_uptime(int argc, char **argv);
 int main_claims(int argc, char **argv);
 int main_tmem_list(int argc, char **argv);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 365798b..e6ff6f4 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -1255,6 +1255,64 @@ static void parse_vnuma_config(const XLU_Config *config,
     free(vcpu_parsed);
 }
 
+/* Parses usbctrl data and adds info into usbctrl
+ * Returns 1 if the input token does not match one of the keys
+ * or parsed values are not correct. Successful parse returns 0 */
+static int parse_usbctrl_config(libxl_device_usbctrl *usbctrl, char *token)
+{
+    char *oparg;
+
+    if (MATCH_OPTION("type", token, oparg)) {
+        if (!strcmp("pv", oparg)) {
+            usbctrl->type = LIBXL_USBCTRL_TYPE_PV;
+        } else {
+            fprintf(stderr,
+                    "Unsupported USB controller type '%s'\n", oparg);
+            return 1;
+        }
+    } else if (MATCH_OPTION("version", token, oparg)) {
+        usbctrl->version = atoi(oparg);
+    } else if (MATCH_OPTION("ports", token, oparg)) {
+        usbctrl->ports = atoi(oparg);
+    } else {
+        fprintf(stderr, "Unknown string `%s' in usb spec\n", token);
+        return 1;
+    }
+
+    return 0;
+}
+
+/* Parses usb data and adds info into usb
+ * Returns 1 if the input token does not match one of the keys
+ * or parsed values are not correct. Successful parse returns 0 */
+static int parse_usb_config(libxl_device_usb *usb, char *token)
+{
+    char *oparg;
+
+    if (MATCH_OPTION("devtype", token, oparg)) {
+        if (!strcmp("hostdev", oparg)) {
+            usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV;
+        } else {
+            fprintf(stderr,
+                    "Unsupported USB device type '%s'\n", oparg);
+            return 1;
+        }
+    } else if (MATCH_OPTION("hostbus", token, oparg)) {
+        usb->u.hostdev.hostbus = strtoul(oparg, NULL, 0);
+    } else if (MATCH_OPTION("hostaddr", token, oparg)) {
+        usb->u.hostdev.hostaddr = strtoul(oparg, NULL, 0);
+    } else if (MATCH_OPTION("controller", token, oparg)) {
+        usb->ctrl = atoi(oparg);
+    } else if (MATCH_OPTION("port", token, oparg)) {
+        usb->port = atoi(oparg);
+    } else {
+        fprintf(stderr, "Unknown string `%s' in usb spec\n", token);
+        return 1;
+    }
+
+    return 0;
+}
+
 static void parse_config_data(const char *config_source,
                               const char *config_data,
                               int config_len,
@@ -3396,6 +3454,198 @@ int main_cd_insert(int argc, char **argv)
     return 0;
 }
 
+int main_usbctrl_attach(int argc, char **argv)
+{
+    uint32_t domid;
+    int opt, rc = 0;
+    libxl_device_usbctrl usbctrl;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usbctrl-attach", 1) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind++]);
+
+    libxl_device_usbctrl_init(&usbctrl);
+
+    for (argv += optind, argc -= optind; argc > 0; ++argv, --argc) {
+        if (parse_usbctrl_config(&usbctrl, *argv))
+            return 1;
+    }
+
+    rc = libxl_device_usbctrl_add(ctx, domid, &usbctrl, 0);
+    if (rc) {
+        fprintf(stderr, "libxl_device_usbctrl_add failed.\n");
+        rc = 1;
+    }
+
+    libxl_device_usbctrl_dispose(&usbctrl);
+    return rc;
+}
+
+int main_usbctrl_detach(int argc, char **argv)
+{
+    uint32_t domid;
+    int opt, devid, rc;
+    libxl_device_usbctrl usbctrl;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usbctrl-detach", 2) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind]);
+    devid = atoi(argv[optind+1]);
+
+    libxl_device_usbctrl_init(&usbctrl);
+    if (libxl_devid_to_device_usbctrl(ctx, domid, devid, &usbctrl)) {
+        fprintf(stderr, "Unknown device %s.\n", argv[optind+1]);
+        return 1;
+    }
+
+    rc = libxl_device_usbctrl_remove(ctx, domid, &usbctrl, 0);
+    if (rc) {
+        fprintf(stderr, "libxl_device_usbctrl_remove failed.\n");
+        rc = 1;
+    }
+
+    libxl_device_usbctrl_dispose(&usbctrl);
+    return rc;
+
+}
+
+int main_usbattach(int argc, char **argv)
+{
+    uint32_t domid;
+    int opt, rc;
+    libxl_device_usb usb;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-attach", 2) {
+        /* No options */
+    }
+
+    libxl_device_usb_init(&usb);
+
+    domid = find_domain(argv[optind++]);
+
+    for (argv += optind, argc -= optind; argc > 0; ++argv, --argc) {
+        if (parse_usb_config(&usb, *argv))
+            return 1;
+    }
+
+    rc = libxl_device_usb_add(ctx, domid, &usb, 0);
+    if (rc) {
+        fprintf(stderr, "libxl_device_usb_add failed.\n");
+        rc = 1;
+    }
+
+    libxl_device_usb_dispose(&usb);
+    return rc;
+}
+
+int main_usbdetach(int argc, char **argv)
+{
+    uint32_t domid;
+    int ctrl, port;
+    int opt, rc = 1;
+    libxl_device_usb usb;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-detach", 3) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind]);
+    ctrl = atoi(argv[optind+1]);
+    port = atoi(argv[optind+2]);
+
+    if (argc - optind > 3) {
+        fprintf(stderr, "Invalid arguments.\n");
+        return 1;
+    }
+
+    libxl_device_usb_init(&usb);
+    if (libxl_ctrlport_to_device_usb(ctx, domid, ctrl, port, &usb)) {
+        fprintf(stderr, "Unknown device at controller %d port %d.\n",
+                ctrl, port);
+        return 1;
+    }
+
+    rc = libxl_device_usb_remove(ctx, domid, &usb, 0);
+    if (rc) {
+        fprintf(stderr, "libxl_device_usb_remove failed.\n");
+        rc = 1;
+    }
+
+    libxl_device_usb_dispose(&usb);
+    return rc;
+}
+
+int main_usblist(int argc, char **argv)
+{
+    uint32_t domid;
+    libxl_device_usbctrl *usbctrls;
+    libxl_usbctrlinfo usbctrlinfo;
+    int numctrl, i, j, opt;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-list", 1) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind++]);
+
+    if (argc > optind) {
+        fprintf(stderr, "Invalid arguments.\n");
+        exit(-1);
+    }
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
+    if (!usbctrls) {
+        return 0;
+    }
+
+    for (i = 0; i < numctrl; ++i) {
+        printf("%-6s %-6s %-3s %-5s %-7s %-5s %-30s\n",
+                "Devid", "Type", "BE", "state", "usb-ver", "ports", "BE-path");
+
+        libxl_usbctrlinfo_init(&usbctrlinfo);
+
+        if (!libxl_device_usbctrl_getinfo(ctx, domid,
+                                &usbctrls[i], &usbctrlinfo)) {
+            printf("%-6d %-6s %-3d %-5d %-7d %-5d %-30s\n",
+                    usbctrlinfo.devid,
+                    libxl_usbctrl_type_to_string(usbctrlinfo.type),
+                    usbctrlinfo.backend_id, usbctrlinfo.state,
+                    usbctrlinfo.version, usbctrlinfo.ports,
+                    usbctrlinfo.backend);
+
+            for (j = 1; j <= usbctrlinfo.ports; j++) {
+                libxl_device_usb usb;
+                libxl_usbinfo usbinfo;
+
+                libxl_device_usb_init(&usb);
+                libxl_usbinfo_init(&usbinfo);
+
+                printf("  Port %d:", j);
+
+                if (!libxl_ctrlport_to_device_usb(ctx, domid,
+                                                  usbctrlinfo.devid, j, &usb)) {
+                    printf(" Bus %03x Device %03x\n",
+                           usbinfo.busnum, usbinfo.devnum);
+                } else {
+                    printf("\n");
+                }
+
+                libxl_usbinfo_dispose(&usbinfo);
+                libxl_device_usb_dispose(&usb);
+            }
+        }
+
+        libxl_usbctrlinfo_dispose(&usbctrlinfo);
+    }
+
+    libxl_device_usbctrl_list_free(usbctrls, numctrl);
+    return 0;
+}
+
 int main_console(int argc, char **argv)
 {
     uint32_t domid;
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 0071f12..c7231ae 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -551,6 +551,31 @@ struct cmd_spec cmd_table[] = {
     },
 
 #endif
+    { "usbctrl-attach",
+      &main_usbctrl_attach, 0, 1,
+      "Create a virtual USB controller for a domain",
+      "<Domain> [type=pv] [version=<version>] [ports=<number>]",
+    },
+    { "usbctrl-detach",
+      &main_usbctrl_detach, 0, 1,
+      "Remove the virtual USB controller specified by <DevId> for a domain",
+      "<Domain> <DevId>",
+    },
+    { "usb-attach",
+      &main_usbattach, 0, 1,
+      "Attach a USB device to a domain",
+      "<Domain> hostbus=<busnum> hostaddr=<devnum> [controller=<DevId> [port=<port>]]",
+    },
+    { "usb-detach",
+      &main_usbdetach, 0, 1,
+      "Detach a USB device from a domain",
+      "<Domain> <controller> <port>",
+    },
+    { "usb-list",
+      &main_usblist, 0, 0,
+      "List information about USB devices for a domain",
+      "<Domain>",
+    },
 };
 
 int cmdtable_len = sizeof(cmd_table)/sizeof(struct cmd_spec);
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V8 6/7] xl: add usb-assignable-list command
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
                   ` (4 preceding siblings ...)
  2015-10-21  9:08 ` [PATCH V8 5/7] xl: add pvusb commands Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:38   ` Juergen Gross
  2015-11-12 11:44   ` George Dunlap
  2015-10-21  9:08 ` [PATCH V8 7/7] domcreate: support pvusb in configuration file Chunyan Liu
  6 siblings, 2 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig

Add xl usb-assignable-list command to list assignable USB devices.
Assignable USB device means the USB device type is assignable and
it's not assigned to any guest yet.

Signed-off-by: Chunyan Liu <cyliu@suse.com>

---
  Same as "libxl: add libxl_device_usb_assignable_list API" patch,
  this patch could be sqaushed to previous one. Split because of
  some dispute. Could be squashed if acceptable, otherwise could
  be removed.
---
 tools/libxl/xl.h          |  1 +
 tools/libxl/xl_cmdimpl.c  | 27 +++++++++++++++++++++++++++
 tools/libxl/xl_cmdtable.c |  4 ++++
 3 files changed, 32 insertions(+)

diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
index ddd9690..2c6f370 100644
--- a/tools/libxl/xl.h
+++ b/tools/libxl/xl.h
@@ -85,6 +85,7 @@ int main_blockdetach(int argc, char **argv);
 int main_vtpmattach(int argc, char **argv);
 int main_vtpmlist(int argc, char **argv);
 int main_vtpmdetach(int argc, char **argv);
+int main_usbassignable_list(int argc, char **argv);
 int main_usbctrl_attach(int argc, char **argv);
 int main_usbctrl_detach(int argc, char **argv);
 int main_usbattach(int argc, char **argv);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index e6ff6f4..33df66b 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -3454,6 +3454,33 @@ int main_cd_insert(int argc, char **argv)
     return 0;
 }
 
+static void usb_assignable_list(void)
+{
+    libxl_device_usb *usbs;
+    int num, i;
+
+    usbs = libxl_device_usb_assignable_list(ctx, &num);
+
+    for (i = 0; i < num; i++) {
+        printf("%d.%d\n", usbs[i].u.hostdev.hostbus,
+               usbs[i].u.hostdev.hostaddr);
+    }
+
+    libxl_device_usb_list_free(usbs, num);
+}
+
+int main_usbassignable_list(int argc, char **argv)
+{
+    int opt;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-assignable-list", 0) {
+        /* No options */
+    }
+
+    usb_assignable_list();
+    return 0;
+}
+
 int main_usbctrl_attach(int argc, char **argv)
 {
     uint32_t domid;
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index c7231ae..4e9c0c8 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -576,6 +576,10 @@ struct cmd_spec cmd_table[] = {
       "List information about USB devices for a domain",
       "<Domain>",
     },
+    { "usb-assignable-list",
+      &main_usbassignable_list, 0, 0,
+      "List all assignable USB devices",
+    },
 };
 
 int cmdtable_len = sizeof(cmd_table)/sizeof(struct cmd_spec);
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V8 7/7] domcreate: support pvusb in configuration file
  2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
                   ` (5 preceding siblings ...)
  2015-10-21  9:08 ` [PATCH V8 6/7] xl: add usb-assignable-list command Chunyan Liu
@ 2015-10-21  9:08 ` Chunyan Liu
  2015-10-27 11:41   ` Juergen Gross
  2015-11-12 16:10   ` George Dunlap
  6 siblings, 2 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-10-21  9:08 UTC (permalink / raw)
  To: xen-devel
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	Chunyan Liu, jfehlig, Simon Cao

Add code to support pvusb in domain config file. One could specify
usbctrl and usb in domain's configuration file and create domain,
then usb controllers will be created and usb device would be attached
to guest automatically.

One could specify usb controllers and usb devices in config file
like this:
usbctrl=['version=2,ports=4', 'version=1, ports=4', ]
usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>

---
changes:
  - change parse_usb_config and parse_usbctrl_config, following
    parse_nic_config way, and move to previous patch
  - update user interface to specify hostbus, hostaddr instead of
    hostbus.hostaddr for future extension

 docs/man/xl.cfg.pod.5        | 81 ++++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_create.c   | 73 +++++++++++++++++++++++++++++++++++++--
 tools/libxl/libxl_device.c   |  4 +++
 tools/libxl/libxl_internal.h |  8 +++++
 tools/libxl/xl_cmdimpl.c     | 54 ++++++++++++++++++++++++++++-
 5 files changed, 216 insertions(+), 4 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index b63846a..f24c008 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -722,6 +722,87 @@ Note this may be overridden by rdm_policy option in PCI device configuration.
 
 =back
 
+=item B<usbctrl=[ "USBCTRL_SPEC_STRING", "USBCTRL_SPEC_STRING", ... ]>
+
+Specifies the USB controllers created for this guest. Each
+B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where:
+
+=over 4
+
+=item B<KEY=VALUE>
+
+Possible B<KEY>s are:
+
+=over 4
+
+=item B<type=TYPE>
+
+Specifies the protocol to implement USB controller, could be "pv" (indicates
+PVUSB) or "qemu" (indicates QEMU emulated). Currently only "pv" is supported.
+
+=item B<version=VERSION>
+
+Specifies version of the USB controller, could be 1 (USB1.1) or 2 (USB2.0).
+Default is 2 (USB2.0).
+
+=item B<ports=PORTS>
+
+Specifies port number of the USB controller. Default is 8.
+
+Each USB controller will have an index starting from 0. On the same
+controller, each port will have an index starting from 1.
+
+E.g.
+usbctrl=["version=1,ports=4", "version=2,ports=8",]
+The first controller has:
+controller index = 0, and port 1,2,3,4.
+The second controller has:
+controller index = 1, and port 1,2,3,4,5,6,7,8.
+
+=back
+
+=back
+
+=item B<usbdev=[ "USB_SPEC_STRING", "USB_SPEC_STRING", ... ]>
+
+Specifies the host USB devices to passthrough to this guest. Each
+B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where:
+
+=over 4
+
+=item B<KEY=VALUE>
+
+Possible B<KEY>s are:
+
+=over 4
+
+=item B<devtype=hostdev>
+
+Specifies USB device type. Currently only support 'hostdev'.
+
+=item B<hostbus=busnum>
+
+Specifies busnum of the USB device from the host perspective.
+
+=item B<hostaddr=devnum>
+
+Specifies devnum of the USB device from the host perspective.
+
+=item B<controller=CONTROLLER>
+
+Specifies USB controller index, to which controller the USB device is attached.
+
+=item B<port=PORT>
+
+Specifies USB port index, to which port the USB device is attached. B<port=PORT>
+is valid only when B<controller=CONTROLLER> is specified. Without
+B<controller=CONTROLLER>, it will find the first available USB controller:port
+and use it. If there is no controller at all, it will create one.
+
+=back
+
+=back
+
 =item B<pci=[ "PCI_SPEC_STRING", "PCI_SPEC_STRING", ... ]>
 
 Specifies the host PCI devices to passthrough to this guest. Each B<PCI_SPEC_STRING>
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index f0fee00..f817187 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -727,6 +727,10 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *aodevs,
 
 static void domcreate_attach_vtpms(libxl__egc *egc, libxl__multidev *multidev,
                                    int ret);
+static void domcreate_attach_usbctrls(libxl__egc *egc,
+                                      libxl__multidev *multidev, int ret);
+static void domcreate_attach_usbs(libxl__egc *egc, libxl__multidev *multidev,
+                                   int ret);
 static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *aodevs,
                                  int ret);
 static void domcreate_attach_dtdev(libxl__egc *egc,
@@ -1385,13 +1389,13 @@ static void domcreate_attach_vtpms(libxl__egc *egc,
    if (d_config->num_vtpms > 0) {
        /* Attach vtpms */
        libxl__multidev_begin(ao, &dcs->multidev);
-       dcs->multidev.callback = domcreate_attach_pci;
+       dcs->multidev.callback = domcreate_attach_usbctrls;
        libxl__add_vtpms(egc, ao, domid, d_config, &dcs->multidev);
        libxl__multidev_prepared(egc, &dcs->multidev, 0);
        return;
    }
 
-   domcreate_attach_pci(egc, multidev, 0);
+   domcreate_attach_usbctrls(egc, multidev, 0);
    return;
 
 error_out:
@@ -1399,6 +1403,69 @@ error_out:
    domcreate_complete(egc, dcs, ret);
 }
 
+static void domcreate_attach_usbctrls(libxl__egc *egc,
+                                      libxl__multidev *multidev, int ret)
+{
+    libxl__domain_create_state *dcs = CONTAINER_OF(multidev, *dcs, multidev);
+    STATE_AO_GC(dcs->ao);
+    int domid = dcs->guest_domid;
+
+    libxl_domain_config *const d_config = dcs->guest_config;
+
+    if (ret) {
+        LOG(ERROR, "unable to add vtpm devices");
+        goto error_out;
+    }
+
+    if (d_config->num_usbctrls > 0) {
+        /* Attach usbctrls */
+        libxl__multidev_begin(ao, &dcs->multidev);
+        dcs->multidev.callback = domcreate_attach_usbs;
+        libxl__add_usbctrls(egc, ao, domid, d_config, &dcs->multidev);
+        libxl__multidev_prepared(egc, &dcs->multidev, 0);
+        return;
+    }
+
+    domcreate_attach_usbs(egc, multidev, 0);
+    return;
+
+error_out:
+    assert(ret);
+    domcreate_complete(egc, dcs, ret);
+}
+
+
+static void domcreate_attach_usbs(libxl__egc *egc, libxl__multidev *multidev,
+                                int ret)
+{
+    libxl__domain_create_state *dcs = CONTAINER_OF(multidev, *dcs, multidev);
+    STATE_AO_GC(dcs->ao);
+    int domid = dcs->guest_domid;
+
+    libxl_domain_config *const d_config = dcs->guest_config;
+
+    if (ret) {
+        LOG(ERROR, "unable to add usbctrl devices");
+        goto error_out;
+    }
+
+    if (d_config->num_usbs > 0) {
+        /* Attach usbctrls */
+        libxl__multidev_begin(ao, &dcs->multidev);
+        dcs->multidev.callback = domcreate_attach_pci;
+        libxl__add_usbs(egc, ao, domid, d_config, &dcs->multidev);
+        libxl__multidev_prepared(egc, &dcs->multidev, 0);
+        return;
+    }
+
+    domcreate_attach_pci(egc, multidev, 0);
+    return;
+
+error_out:
+    assert(ret);
+    domcreate_complete(egc, dcs, ret);
+}
+
 static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
                                  int ret)
 {
@@ -1411,7 +1478,7 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
     libxl_domain_config *const d_config = dcs->guest_config;
 
     if (ret) {
-        LOG(ERROR, "unable to add vtpm devices");
+        LOG(ERROR, "unable to add usb devices");
         goto error_out;
     }
 
diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
index 6715c16..0ede029 100644
--- a/tools/libxl/libxl_device.c
+++ b/tools/libxl/libxl_device.c
@@ -544,6 +544,8 @@ void libxl__multidev_prepared(libxl__egc *egc,
  * libxl__add_disks
  * libxl__add_nics
  * libxl__add_vtpms
+ * libxl__add_usbctrls
+ * libxl__add_usbs
  */
 
 #define DEFINE_DEVICES_ADD(type)                                        \
@@ -563,6 +565,8 @@ void libxl__multidev_prepared(libxl__egc *egc,
 DEFINE_DEVICES_ADD(disk)
 DEFINE_DEVICES_ADD(nic)
 DEFINE_DEVICES_ADD(vtpm)
+DEFINE_DEVICES_ADD(usbctrl)
+DEFINE_DEVICES_ADD(usb)
 
 #undef DEFINE_DEVICES_ADD
 
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 1aa0aaf..4d18cbc 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -3302,6 +3302,14 @@ _hidden void libxl__add_vtpms(libxl__egc *egc, libxl__ao *ao, uint32_t domid,
                              libxl_domain_config *d_config,
                              libxl__multidev *multidev);
 
+_hidden void libxl__add_usbctrls(libxl__egc *egc, libxl__ao *ao,
+                                 uint32_t domid, libxl_domain_config *d_config,
+                                 libxl__multidev *multidev);
+
+_hidden void libxl__add_usbs(libxl__egc *egc, libxl__ao *ao,
+                             uint32_t domid, libxl_domain_config *d_config,
+                             libxl__multidev *multidev);
+
 /*----- device model creation -----*/
 
 /* First layer; wraps libxl__spawn_spawn. */
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 33df66b..88958d2 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -1321,7 +1321,8 @@ static void parse_config_data(const char *config_source,
     const char *buf;
     long l, vcpus = 0;
     XLU_Config *config;
-    XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms;
+    XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms,
+                   *usbctrls, *usbs;
     XLU_ConfigList *channels, *ioports, *irqs, *iomem, *viridian, *dtdevs;
     int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian;
     int pci_power_mgmt = 0;
@@ -2138,6 +2139,57 @@ skip_vfb:
         }
     }
 
+    if (!xlu_cfg_get_list(config, "usbctrl", &usbctrls, 0, 0)) {
+        d_config->num_usbctrls = 0;
+        d_config->usbctrls = NULL;
+        while ((buf = xlu_cfg_get_listitem(usbctrls, d_config->num_usbctrls))
+               != NULL) {
+            libxl_device_usbctrl *usbctrl;
+            char *buf2 = strdup(buf);
+            char *p;
+
+            usbctrl = ARRAY_EXTEND_INIT(d_config->usbctrls,
+                                        d_config->num_usbctrls,
+                                        libxl_device_usbctrl_init);
+            p = strtok(buf2, ",");
+            if (!p)
+                goto skip_usbctrl;
+            do {
+                while (*p == ' ')
+                    p++;
+                if (parse_usbctrl_config(usbctrl, p))
+                    exit(1);
+            } while ((p = strtok(NULL, ",")) != NULL);
+skip_usbctrl:
+            free(buf2);
+        }
+    }
+
+    if (!xlu_cfg_get_list(config, "usbdev", &usbs, 0, 0)) {
+        d_config->num_usbs = 0;
+        d_config->usbs = NULL;
+        while ((buf = xlu_cfg_get_listitem(usbs, d_config->num_usbs)) != NULL) {
+            libxl_device_usb *usb;
+            char *buf2 = strdup(buf);
+            char *p;
+
+            usb = ARRAY_EXTEND_INIT_NODEVID(d_config->usbs,
+                                            d_config->num_usbs,
+                                            libxl_device_usb_init);
+            p = strtok(buf2, ",");
+            if (!p)
+                goto skip_usb;
+            do {
+                while (*p == ' ')
+                    p++;
+                if (parse_usb_config(usb, p))
+                    exit(1);
+            } while ((p = strtok(NULL, ",")) != NULL);
+skip_usb:
+            free(buf2);
+        }
+    }
+
     switch (xlu_cfg_get_list(config, "cpuid", &cpuids, 0, 1)) {
     case 0:
         {
-- 
2.1.4

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 1/7] libxl: export some functions for pvusb use
  2015-10-21  9:08 ` [PATCH V8 1/7] libxl: export some functions for pvusb use Chunyan Liu
@ 2015-10-27 11:08   ` Juergen Gross
  0 siblings, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:08 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: wei.liu2, ian.campbell, george.dunlap, Ian.Jackson, jfehlig, Simon Cao

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
> Reviewed-by: Wei Liu <wei.liu2@citrix.com>
> ---
>   tools/libxl/libxl.c          | 4 ++--
>   tools/libxl/libxl_internal.h | 3 +++
>   2 files changed, 5 insertions(+), 2 deletions(-)
>
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 22bbc29..dacfaae 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -2035,7 +2035,7 @@ out:
>   }
>
>   /* common function to get next device id */
> -static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
> +int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
>   {
>       char *dompath, **l;
>       unsigned int nb;
> @@ -2054,7 +2054,7 @@ static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
>       return nextid;
>   }
>
> -static int libxl__resolve_domid(libxl__gc *gc, const char *name,
> +int libxl__resolve_domid(libxl__gc *gc, const char *name,
>                                   uint32_t *domid)

Unwrap the two lines or correct indentation.


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
@ 2015-10-27 11:31   ` Juergen Gross
  2015-11-04  6:31   ` Chun Yan Liu
                     ` (4 subsequent siblings)
  5 siblings, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:31 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: wei.liu2, ian.campbell, george.dunlap, Ian.Jackson, jfehlig, Simon Cao

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Add pvusb APIs, including:
>   - attach/detach (create/destroy) virtual usb controller.
>   - attach/detach usb device
>   - list usb controller and usb devices
>   - some other helper functions
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>
> ---
> changes:
>    - update COMPARE_USB to compare ctrl and port
>    - add check in usb_add/remove to disable non-Dom0 backend so that
>      not worring about codes which are effective on Dom0 but not
>      compatible on non-Dom0 backend.
>    - define READ_SUBPATH macro within functions
>    - do not initialize rc but give it value in each return case
>    - libxl__strdup gc or NOGC update, internal function using gc,
>      external using NOGC.
>    - address other comments from George and Ian J.
>
>   tools/libxl/Makefile                 |    2 +-
>   tools/libxl/libxl.c                  |   53 ++
>   tools/libxl/libxl.h                  |   74 ++
>   tools/libxl/libxl_device.c           |    5 +-
>   tools/libxl/libxl_internal.h         |   18 +
>   tools/libxl/libxl_osdeps.h           |   13 +
>   tools/libxl/libxl_pvusb.c            | 1451 ++++++++++++++++++++++++++++++++++
>   tools/libxl/libxl_types.idl          |   57 ++
>   tools/libxl/libxl_types_internal.idl |    1 +
>   tools/libxl/libxl_utils.c            |   16 +
>   tools/libxl/libxl_utils.h            |    5 +
>   11 files changed, 1693 insertions(+), 2 deletions(-)
>   create mode 100644 tools/libxl/libxl_pvusb.c

...

> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
> new file mode 100644
> index 0000000..aa1a653
> --- /dev/null
> +++ b/tools/libxl/libxl_pvusb.c
> @@ -0,0 +1,1451 @@
> +/*
> + * Copyright (C) 2015 SUSE LINUX Products GmbH, Nuernberg, Germany.

Hmm, shouldn't this be just "SUSE LINUX GmbH, ..."?

> + * Author Chunyan Liu <cyliu@suse.com>
> + *
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU Lesser General Public License for more details.
> + */

...

> +/* Encode usb interface so that it could be written to xenstore as a key.
> + *
> + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_',
> + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1.
> + * This will be used to save original driver of USB device to xenstore.
> + */
> +static char *usb_interface_xenstore_encode(char *busid)
> +{
> +    char *str = strdup(busid);
> +    int i, len = strlen(str);
> +
> +    for (i = 0; i < len; i++) {
> +        if (str[i] == '.')
> +            str[i] = '_';
> +         if (str[i] == ':')

Indentation.

> +            str[i] = '-';
> +    }
> +    return str;
> +}

...

> diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
> index 9c5c4d0..706a0c1 100644
> --- a/tools/libxl/libxl_utils.c
> +++ b/tools/libxl/libxl_utils.c
> @@ -1270,6 +1270,22 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf, size_t len)
>       return ret;
>   }
>
> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr)
> +{
> +   int i;

Blank line.

> +   for (i = 0; i < nr; i++)
> +       libxl_device_usbctrl_dispose(&list[i]);
> +   free(list);
> +}
> +
> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr)
> +{
> +   int i;

Blank line.

> +   for (i = 0; i < nr; i++)
> +       libxl_device_usb_dispose(&list[i]);
> +   free(list);
> +}


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file
  2015-10-21  9:08 ` [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file Chunyan Liu
@ 2015-10-27 11:31   ` Juergen Gross
  2015-11-16 14:03   ` Ian Campbell
  1 sibling, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:31 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: george.dunlap, wei.liu2, jfehlig, Ian.Jackson, ian.campbell

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Sysfs file has size=4096 but actual file content is less than that.
> Current libxl_read_file_contents will treat it as error when file size
> and actual file content differs, so reading sysfs file content with
> this function always fails.
>
> Add a new entry libxl_read_sysfs_file_contents to handle sysfs file
> specially. It would be used in later pvusb work.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>

Acked-by: Juergen Gross <jgross@suse.com>


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API
  2015-10-21  9:08 ` [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API Chunyan Liu
@ 2015-10-27 11:32   ` Juergen Gross
  2015-11-11 16:07   ` George Dunlap
  1 sibling, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:32 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: george.dunlap, wei.liu2, jfehlig, Ian.Jackson, ian.campbell

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Add API for listing assignable USB devices info.
> Assignable USB device means the USB device type is assignable and
> it's not assigned to any guest yet.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>

Acked-by: Juergen Gross <jgross@suse.com>


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-10-21  9:08 ` [PATCH V8 5/7] xl: add pvusb commands Chunyan Liu
@ 2015-10-27 11:37   ` Juergen Gross
  2015-11-12 11:38   ` George Dunlap
  2015-11-12 14:42   ` Olaf Hering
  2 siblings, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:37 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: wei.liu2, ian.campbell, george.dunlap, Ian.Jackson, jfehlig, Simon Cao

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list,
> usb-attach and usb-detach.
>
> To attach a usb device to guest through pvusb, one could follow
> following example:
>
>   #xl usbctrl-attach test_vm version=1 ports=8
>
>   #xl usb-list test_vm
>   will show the usb controllers and port usage under the domain.
>
>   #xl usbattach test_vm hostbus=1 hostaddr=2
>   will find the first usable controller:port, and attach usb
>   device whose busnum is 1 and devnum is 6.
>   One could also specify which <controller> and which <port>.
>
>   #xl usb-detach test_vm 0 1
>   will detach USB device under controller 0 port 1.
>
>   #xl usbctrl-detach test_vm dev_id
>   will destroy the controller with specified dev_id. Dev_id
>   can be traced in usb-list info.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>

Acked-by: Juergen Gross <jgross@suse.com>


juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 6/7] xl: add usb-assignable-list command
  2015-10-21  9:08 ` [PATCH V8 6/7] xl: add usb-assignable-list command Chunyan Liu
@ 2015-10-27 11:38   ` Juergen Gross
  2015-11-12 11:44   ` George Dunlap
  1 sibling, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:38 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: george.dunlap, wei.liu2, jfehlig, Ian.Jackson, ian.campbell

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Add xl usb-assignable-list command to list assignable USB devices.
> Assignable USB device means the USB device type is assignable and
> it's not assigned to any guest yet.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>

Acked-by: Juergen Gross <jgross@suse.com>


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 7/7] domcreate: support pvusb in configuration file
  2015-10-21  9:08 ` [PATCH V8 7/7] domcreate: support pvusb in configuration file Chunyan Liu
@ 2015-10-27 11:41   ` Juergen Gross
  2015-11-12 16:10   ` George Dunlap
  1 sibling, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-10-27 11:41 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: wei.liu2, ian.campbell, george.dunlap, Ian.Jackson, jfehlig, Simon Cao

On 10/21/2015 11:08 AM, Chunyan Liu wrote:
> Add code to support pvusb in domain config file. One could specify
> usbctrl and usb in domain's configuration file and create domain,
> then usb controllers will be created and usb device would be attached
> to guest automatically.
>
> One could specify usb controllers and usb devices in config file
> like this:
> usbctrl=['version=2,ports=4', 'version=1, ports=4', ]
> usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>

Acked-by: Juergen Gross <jgross@ssue.com>


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
  2015-10-27 11:31   ` Juergen Gross
@ 2015-11-04  6:31   ` Chun Yan Liu
  2015-11-05 15:54     ` George Dunlap
  2015-11-09 18:11   ` Ian Jackson
                     ` (3 subsequent siblings)
  5 siblings, 1 reply; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-04  6:31 UTC (permalink / raw)
  To: xen-devel, Chun Yan Liu
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap,
	Ian.Jackson, Jim Fehlig, Simon Cao

Ian & George, any comments?

>>> On 10/21/2015 at 05:08 PM, in message
<1445418510-19614-4-git-send-email-cyliu@suse.com>, Chunyan Liu
<cyliu@suse.com> wrote: 
> Add pvusb APIs, including: 
>  - attach/detach (create/destroy) virtual usb controller. 
>  - attach/detach usb device 
>  - list usb controller and usb devices 
>  - some other helper functions 
>  
> Signed-off-by: Chunyan Liu <cyliu@suse.com> 
> Signed-off-by: Simon Cao <caobosimon@gmail.com> 
>  
> --- 
> changes: 
>   - update COMPARE_USB to compare ctrl and port 
>   - add check in usb_add/remove to disable non-Dom0 backend so that 
>     not worring about codes which are effective on Dom0 but not 
>     compatible on non-Dom0 backend. 
>   - define READ_SUBPATH macro within functions 
>   - do not initialize rc but give it value in each return case 
>   - libxl__strdup gc or NOGC update, internal function using gc, 
>     external using NOGC. 
>   - address other comments from George and Ian J. 
>  
>  tools/libxl/Makefile                 |    2 +- 
>  tools/libxl/libxl.c                  |   53 ++ 
>  tools/libxl/libxl.h                  |   74 ++ 
>  tools/libxl/libxl_device.c           |    5 +- 
>  tools/libxl/libxl_internal.h         |   18 + 
>  tools/libxl/libxl_osdeps.h           |   13 + 
>  tools/libxl/libxl_pvusb.c            | 1451  
> ++++++++++++++++++++++++++++++++++ 
>  tools/libxl/libxl_types.idl          |   57 ++ 
>  tools/libxl/libxl_types_internal.idl |    1 + 
>  tools/libxl/libxl_utils.c            |   16 + 
>  tools/libxl/libxl_utils.h            |    5 + 
>  11 files changed, 1693 insertions(+), 2 deletions(-) 
>  create mode 100644 tools/libxl/libxl_pvusb.c 
>  
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile 
> index c5ecec1..ef9ccd3 100644 
> --- a/tools/libxl/Makefile 
> +++ b/tools/libxl/Makefile 
> @@ -103,7 +103,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o  
> libxl_dm.o libxl_pci.o \ 
>  			libxl_stream_read.o libxl_stream_write.o \ 
>  			libxl_save_callout.o _libxl_save_msgs_callout.o \ 
>  			libxl_qmp.o libxl_event.o libxl_fork.o \ 
> -			libxl_dom_suspend.o $(LIBXL_OBJS-y) 
> +			libxl_dom_suspend.o libxl_pvusb.o $(LIBXL_OBJS-y) 
>  LIBXL_OBJS += libxl_genid.o 
>  LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o 
>   
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c 
> index dacfaae..a050e8b 100644 
> --- a/tools/libxl/libxl.c 
> +++ b/tools/libxl/libxl.c 
> @@ -4218,11 +4218,54 @@ DEFINE_DEVICE_REMOVE(vtpm, destroy, 1) 
>   
>   
> /**************************************************************************** 
> **/ 
>   
> +/* Macro for defining device remove/destroy functions for usbctrl */ 
> +/* Follo:wing functions are defined: 
> + * libxl_device_usbctrl_remove 
> + * libxl_device_usbctrl_destroy 
> + */ 
> + 
> +#define DEFINE_DEVICE_REMOVE_EXT(type, removedestroy, f)                \ 
> +    int libxl_device_##type##_##removedestroy(libxl_ctx *ctx,           \ 
> +        uint32_t domid, libxl_device_##type *type,                      \ 
> +        const libxl_asyncop_how *ao_how)                                \ 
> +    {                                                                   \ 
> +        AO_CREATE(ctx, domid, ao_how);                                  \ 
> +        libxl__device *device;                                          \ 
> +        libxl__ao_device *aodev;                                        \ 
> +        int rc;                                                         \ 
> +                                                                        \ 
> +        GCNEW(device);                                                  \ 
> +        rc = libxl__device_from_##type(gc, domid, type, device);        \ 
> +        if (rc != 0) goto out;                                          \ 
> +                                                                        \ 
> +        GCNEW(aodev);                                                   \ 
> +        libxl__prepare_ao_device(ao, aodev);                            \ 
> +        aodev->action = LIBXL__DEVICE_ACTION_REMOVE;                    \ 
> +        aodev->dev = device;                                            \ 
> +        aodev->callback = device_addrm_aocomplete;                      \ 
> +        aodev->force = f;                                               \ 
> +        libxl__initiate_device_##type##_remove(egc, aodev);             \ 
> +                                                                        \ 
> +    out:                                                                \ 
> +        if (rc) return AO_CREATE_FAIL(rc);                              \ 
> +        return AO_INPROGRESS;                                           \ 
> +    } 
> + 
> + 
> +DEFINE_DEVICE_REMOVE_EXT(usbctrl, remove, 0) 
> +DEFINE_DEVICE_REMOVE_EXT(usbctrl, destroy, 1) 
> + 
> +#undef DEFINE_DEVICE_REMOVE_EXT 
> + 
> +/************************************************************************** 
> ****/ 
> + 
>  /* Macro for defining device addition functions in a compact way */ 
>  /* The following functions are defined: 
>   * libxl_device_disk_add 
>   * libxl_device_nic_add 
>   * libxl_device_vtpm_add 
> + * libxl_device_usbctrl_add 
> + * libxl_device_usb_add 
>   */ 
>   
>  #define DEFINE_DEVICE_ADD(type)                                         \ 
> @@ -4254,6 +4297,12 @@ DEFINE_DEVICE_ADD(nic) 
>  /* vtpm */ 
>  DEFINE_DEVICE_ADD(vtpm) 
>   
> +/* usbctrl */ 
> +DEFINE_DEVICE_ADD(usbctrl) 
> + 
> +/* usb */ 
> +DEFINE_DEVICE_ADD(usb) 
> + 
>  #undef DEFINE_DEVICE_ADD 
>   
>   
> /**************************************************************************** 
> **/ 
> @@ -6795,6 +6844,10 @@ int libxl_retrieve_domain_configuration(libxl_ctx  
> *ctx, uint32_t domid, 
>   
>      MERGE(pci, pcidevs, COMPARE_PCI, {}); 
>   
> +    MERGE(usbctrl, usbctrls, COMPARE_USBCTRL, {}); 
> + 
> +    MERGE(usb, usbs, COMPARE_USB, {}); 
> + 
>      /* Take care of removable device. We maintain invariant in the 
>       * insert / remove operation so that: 
>       * 1. if xenstore is "empty" while JSON is not, the result 
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h 
> index 615b1de..594e23d 100644 
> --- a/tools/libxl/libxl.h 
> +++ b/tools/libxl/libxl.h 
> @@ -123,6 +123,12 @@ 
>  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1 
>   
>  /* 
> + * LIBXL_HAVE_PVUSB indicates functions for plugging in USB devices 
> + * through pvusb -- both hotplug and at domain creation time.. 
> + */ 
> +#define LIBXL_HAVE_PVUSB 1 
> + 
> +/* 
>   * LIBXL_HAVE_BUILDINFO_HVM_VENDOR_DEVICE indicates that the 
>   * libxl_vendor_device field is present in the hvm sections of 
>   * libxl_domain_build_info. This field tells libxl which 
> @@ -1411,6 +1417,74 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid,  
> libxl_device_disk *disk, 
>                         const libxl_asyncop_how *ao_how) 
>                         LIBXL_EXTERNAL_CALLERS_ONLY; 
>   
> +/* 
> + * USB 
> + * 
> + * For each device removed or added, one of these protocols is available: 
> + * - PV (i.e., PVUSB) 
> + * - DEVICEMODEL (i.e, qemu) 
> + * 
> + * PV is available for either PV or HVM domains.  DEVICEMODEL is only 
> + * available for HVM domains.  The caller can additionally specify 
> + * "AUTO", in which case the library will try to determine the best 
> + * protocol automatically. 
> + * 
> + * At the moment, the only protocol implemented is PV. 
> + * 
> + * One can add/remove USB controllers to/from guest, and attach/detach USB 
> + * devices to/from USB controllers. 
> + * 
> + * To add USB controllers and USB devices, one can adding USB controllers 
> + * first and then attaching USB devices to some USB controller, or adding 
> + * USB devices to guest directly, it will automatically create a USB 
> + * controller for USB devices to attach. 
> + * 
> + * To remove USB controllers or USB devices, one can remove USB devices 
> + * under USB controller one by one and then remove USB controller, or 
> + * remove USB controller directly, it will remove all USB devices under 
> + * it automatically. 
> + * 
> + */ 
> +/* USB Controllers*/ 
> +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid, 
> +                             libxl_device_usbctrl *usbctrl, 
> +                             const libxl_asyncop_how *ao_how) 
> +                             LIBXL_EXTERNAL_CALLERS_ONLY; 
> + 
> +int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid, 
> +                                libxl_device_usbctrl *usbctrl, 
> +                                const libxl_asyncop_how *ao_how) 
> +                                LIBXL_EXTERNAL_CALLERS_ONLY; 
> + 
> +int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid, 
> +                                 libxl_device_usbctrl *usbctrl, 
> +                                 const libxl_asyncop_how *ao_how) 
> +                                 LIBXL_EXTERNAL_CALLERS_ONLY; 
> + 
> +libxl_device_usbctrl *libxl_device_usbctrl_list(libxl_ctx *ctx, 
> +                                                uint32_t domid, int *num); 
> + 
> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr); 
> + 
> + 
> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid, 
> +                                 libxl_device_usbctrl *usbctrl, 
> +                                 libxl_usbctrlinfo *usbctrlinfo); 
> + 
> +/* USB Devices */ 
> +int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb  
> *usb, 
> +                         const libxl_asyncop_how *ao_how) 
> +                         LIBXL_EXTERNAL_CALLERS_ONLY; 
> + 
> +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,  
> libxl_device_usb *usb, 
> +                            const libxl_asyncop_how *ao_how) 
> +                            LIBXL_EXTERNAL_CALLERS_ONLY; 
> + 
> +libxl_device_usb * 
> +libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, int *num); 
> + 
> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr); 
> + 
>  /* Network Interfaces */ 
>  int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic  
> *nic, 
>                           const libxl_asyncop_how *ao_how) 
> diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c 
> index 8bb5e93..6715c16 100644 
> --- a/tools/libxl/libxl_device.c 
> +++ b/tools/libxl/libxl_device.c 
> @@ -676,7 +676,10 @@ void libxl__devices_destroy(libxl__egc *egc,  
> libxl__devices_remove_state *drs) 
>                  aodev->action = LIBXL__DEVICE_ACTION_REMOVE; 
>                  aodev->dev = dev; 
>                  aodev->force = drs->force; 
> -                libxl__initiate_device_remove(egc, aodev); 
> +                if (dev->backend_kind == LIBXL__DEVICE_KIND_VUSB) 
> +                    libxl__initiate_device_usbctrl_remove(egc, aodev); 
> +                else 
> +                    libxl__initiate_device_remove(egc, aodev); 
>              } 
>          } 
>      } 
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h 
> index a3a5cc1..1aa0aaf 100644 
> --- a/tools/libxl/libxl_internal.h 
> +++ b/tools/libxl/libxl_internal.h 
> @@ -2569,6 +2569,14 @@ _hidden void libxl__device_vtpm_add(libxl__egc *egc,  
> uint32_t domid, 
>                                     libxl_device_vtpm *vtpm, 
>                                     libxl__ao_device *aodev); 
>   
> +_hidden void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid, 
> +                                       libxl_device_usbctrl *usbctrl, 
> +                                       libxl__ao_device *aodev); 
> + 
> +_hidden void libxl__device_usb_add(libxl__egc *egc, uint32_t domid, 
> +                                   libxl_device_usb *usb, 
> +                                   libxl__ao_device *aodev); 
> + 
>  /* Internal function to connect a vkb device */ 
>  _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid, 
>                                    libxl_device_vkb *vkb); 
> @@ -2601,6 +2609,13 @@ _hidden void  
> libxl__wait_device_connection(libxl__egc*, 
>  _hidden void libxl__initiate_device_remove(libxl__egc *egc, 
>                                             libxl__ao_device *aodev); 
>   
> +_hidden int libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid, 
> +                               libxl_device_usbctrl *usbctrl, 
> +                               libxl__device *device); 
> + 
> +_hidden void libxl__initiate_device_usbctrl_remove(libxl__egc *egc, 
> +                                                   libxl__ao_device  
> *aodev); 
> + 
>  /* 
>   * libxl__get_hotplug_script_info returns the args and env that should 
>   * be passed to the hotplug script for the requested device. 
> @@ -3958,6 +3973,9 @@ static inline void libxl__update_config_vtpm(libxl__gc  
> *gc, 
>  #define COMPARE_PCI(a, b) ((a)->func == (b)->func &&    \ 
>                             (a)->bus == (b)->bus &&      \ 
>                             (a)->dev == (b)->dev) 
> +#define COMPARE_USB(a, b) ((a)->ctrl == (b)->ctrl && \ 
> +                           (a)->port == (b)->port) 
> +#define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid) 
>   
>  /* DEVICE_ADD 
>   * 
> diff --git a/tools/libxl/libxl_osdeps.h b/tools/libxl/libxl_osdeps.h 
> index d9661c9..802c762 100644 
> --- a/tools/libxl/libxl_osdeps.h 
> +++ b/tools/libxl/libxl_osdeps.h 
> @@ -24,6 +24,8 @@ 
>  #define _GNU_SOURCE 
>   
>  #if defined(__NetBSD__) 
> +#define SYSFS_USB_DEV          "/sys/bus/usb/devices" 
> +#define SYSFS_USBBACK_DRIVER   "/kern/xen/usb" 
>  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices" 
>  #define SYSFS_PCIBACK_DRIVER   "/kern/xen/pci" 
>  #define NETBACK_NIC_NAME       "xvif%ui%d" 
> @@ -31,6 +33,8 @@ 
>  #elif defined(__OpenBSD__) 
>  #include <util.h> 
>  #elif defined(__linux__) 
> +#define SYSFS_USB_DEV          "/sys/bus/usb/devices" 
> +#define SYSFS_USBBACK_DRIVER   "/sys/bus/usb/drivers/usbback" 
>  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices" 
>  #define SYSFS_PCIBACK_DRIVER   "/sys/bus/pci/drivers/pciback" 
>  #define NETBACK_NIC_NAME       "vif%u.%d" 
> @@ -38,6 +42,8 @@ 
>  #elif defined(__sun__) 
>  #include <stropts.h> 
>  #elif defined(__FreeBSD__) 
> +#define SYSFS_USB_DEV          "/dev/null" 
> +#define SYSFS_USBBACK_DRIVER   "/dev/null" 
>  #define SYSFS_PCI_DEV          "/dev/null" 
>  #define SYSFS_PCIBACK_DRIVER   "/dev/null" 
>  #define NETBACK_NIC_NAME       "xnb%u.%d" 
> @@ -45,6 +51,13 @@ 
>  #include <sys/endian.h> 
>  #endif 
>   
> +#ifndef SYSFS_USBBACK_DRIVER 
> +#error define SYSFS_USBBACK_DRIVER for your platform 
> +#endif 
> +#ifndef SYSFS_USB_DEV 
> +#error define SYSFS_USB_DEV for your platform 
> +#endif 
> + 
>  #ifndef SYSFS_PCIBACK_DRIVER 
>  #error define SYSFS_PCIBACK_DRIVER for your platform 
>  #endif 
> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c 
> new file mode 100644 
> index 0000000..aa1a653 
> --- /dev/null 
> +++ b/tools/libxl/libxl_pvusb.c 
> @@ -0,0 +1,1451 @@ 
> +/* 
> + * Copyright (C) 2015 SUSE LINUX Products GmbH, Nuernberg, Germany. 
> + * Author Chunyan Liu <cyliu@suse.com> 
> + * 
> + * This program is free software; you can redistribute it and/or modify 
> + * it under the terms of the GNU Lesser General Public License as published 
> + * by the Free Software Foundation; version 2.1 only. with the special 
> + * exception on linking described in file LICENSE. 
> + * 
> + * This program is distributed in the hope that it will be useful, 
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of 
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the 
> + * GNU Lesser General Public License for more details. 
> + */ 
> + 
> +#include "libxl_osdeps.h" /* must come before any other headers */ 
> + 
> +#include "libxl_internal.h" 
> +#include <inttypes.h> 
> + 
> +#define USBBACK_INFO_PATH "/libxl/usbback" 
> + 
> +#define USBHUB_CLASS_CODE 9 
> + 
> +static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid, 
> +                                            libxl_device_usbctrl *usbctrl) 
> +{ 
> +    int rc; 
> +    libxl_domain_type domtype = libxl__domain_type(gc, domid); 
> + 
> +    if (!usbctrl->version) 
> +        usbctrl->version = 2; 
> + 
> +    if (!usbctrl->ports) 
> +        usbctrl->ports = 8; 
> + 
> +    if (usbctrl->type == LIBXL_USBCTRL_TYPE_AUTO) { 
> +        if (domtype == LIBXL_DOMAIN_TYPE_PV) { 
> +            usbctrl->type = LIBXL_USBCTRL_TYPE_PV; 
> +        } else if (domtype == LIBXL_DOMAIN_TYPE_HVM) { 
> +            /* FIXME: See if we can detect PV frontend */ 
> +            usbctrl->type = LIBXL_USBCTRL_TYPE_DEVICEMODEL; 
> +        } 
> +    } 
> + 
> +    rc = libxl__resolve_domid(gc, usbctrl->backend_domname, 
> +                              &usbctrl->backend_domid); 
> +    return rc; 
> +} 
> + 
> +int libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid, 
> +                               libxl_device_usbctrl *usbctrl, 
> +                               libxl__device *device) 
> +{ 
> +    device->backend_devid   = usbctrl->devid; 
> +    device->backend_domid   = usbctrl->backend_domid; 
> +    device->backend_kind    = LIBXL__DEVICE_KIND_VUSB; 
> +    device->devid           = usbctrl->devid; 
> +    device->domid           = domid; 
> +    device->kind            = LIBXL__DEVICE_KIND_VUSB; 
> + 
> +    return 0; 
> +} 
> + 
> +/* Add usbctrl information to xenstore. 
> + * 
> + * Adding a usb controller will add a new 'vusb' device in xenstore, and 
> + * add corresponding frontend, backend information to it. According to 
> + * "update_json", decide wether to update json config file. 
> + */ 
> +static int libxl__device_usbctrl_add_xenstore(libxl__gc *gc, uint32_t  
> domid, 
> +                                              libxl_device_usbctrl  
> *usbctrl, 
> +                                              bool update_json) 
> +{ 
> +    libxl__device *device; 
> +    flexarray_t *front; 
> +    flexarray_t *back; 
> +    xs_transaction_t t = XBT_NULL; 
> +    int i, rc; 
> +    libxl_domain_config d_config; 
> +    libxl_device_usbctrl usbctrl_saved; 
> +    libxl__domain_userdata_lock *lock = NULL; 
> + 
> +    libxl_domain_config_init(&d_config); 
> +    libxl_device_usbctrl_init(&usbctrl_saved); 
> +    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl); 
> + 
> +    GCNEW(device); 
> +    rc = libxl__device_from_usbctrl(gc, domid, usbctrl, device); 
> +    if (rc) goto out; 
> + 
> +    front = flexarray_make(gc, 4, 1); 
> +    back = flexarray_make(gc, 12, 1); 
> + 
> +    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid)); 
> +    flexarray_append_pair(back, "online", "1"); 
> +    flexarray_append_pair(back, "state", "1"); 
> +    flexarray_append_pair(back, "type", 
> +                    (char *)libxl_usbctrl_type_to_string(usbctrl->type)); 
> +    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d", usbctrl->version)); 
> +    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d", usbctrl->ports)); 
> +    flexarray_append_pair(back, "port", ""); 
> +    for (i = 0; i < usbctrl->ports; i++) 
> +        flexarray_append_pair(back, GCSPRINTF("port/%d", i + 1), ""); 
> + 
> +    flexarray_append_pair(front, "backend-id", 
> +                          GCSPRINTF("%d", usbctrl->backend_domid)); 
> +    flexarray_append_pair(front, "state", "1"); 
> + 
> +    if (update_json) { 
> +        lock = libxl__lock_domain_userdata(gc, domid); 
> +        if (!lock) { 
> +            rc = ERROR_LOCK_FAIL; 
> +            goto out; 
> +        } 
> + 
> +        rc = libxl__get_domain_configuration(gc, domid, &d_config); 
> +        if (rc) goto out; 
> + 
> +        DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved, 
> +                   COMPARE_USBCTRL, &d_config); 
> +    } 
> + 
> +    for (;;) { 
> +        rc = libxl__xs_transaction_start(gc, &t); 
> +        if (rc) goto out; 
> + 
> +        rc = libxl__device_exists(gc, t, device); 
> +        if (rc < 0) goto out; 
> +        if (rc == 1) { 
> +            /* already exists in xenstore */ 
> +            LOG(ERROR, "device already exists in xenstore"); 
> +            rc = ERROR_DEVICE_EXISTS; 
> +            goto out; 
> +        } 
> + 
> +        if (update_json) { 
> +            rc = libxl__set_domain_configuration(gc, domid, &d_config); 
> +            if (rc) goto out; 
> +        } 
> + 
> +        libxl__device_generic_add(gc, t, device, 
> +                          libxl__xs_kvs_of_flexarray(gc, back, back->count), 
> +                          libxl__xs_kvs_of_flexarray(gc, front,  
> front->count), 
> +                          NULL); 
> + 
> +        rc = libxl__xs_transaction_commit(gc, &t); 
> +        if (!rc) break; 
> +        if (rc < 0) goto out; 
> +    } 
> + 
> +out: 
> +    libxl__xs_transaction_abort(gc, &t); 
> +    if (lock) libxl__unlock_domain_userdata(lock); 
> +    libxl_device_usbctrl_dispose(&usbctrl_saved); 
> +    libxl_domain_config_dispose(&d_config); 
> +    return rc; 
> +} 
> + 
> +/* AO operation to add a usb controller. 
> + * 
> + * Generally, it does: 
> + * 1) fill in necessary usb controler information with default value 
> + * 2) write usb controller frontend/backend info to xenstore, update json 
> + *    config file if necessary. 
> + * 3) wait for device connection. PVUSB frontend and backend driver will 
> + *    probe xenstore paths and build connection between frontend and  
> backend. 
> + * 
> + * Before calling this function, aodev should be properly filled: 
> + * aodev->ao, aodev->callback, aodev->update_json, ... 
> + */ 
> +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid, 
> +                               libxl_device_usbctrl *usbctrl, 
> +                               libxl__ao_device *aodev) 
> +{ 
> +    STATE_AO_GC(aodev->ao); 
> +    libxl__device *device; 
> +    int rc; 
> + 
> +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl); 
> +    if (rc < 0) goto out; 
> + 
> +    if (usbctrl->devid == -1) { 
> +        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb"); 
> +        if (usbctrl->devid < 0) { 
> +            rc = ERROR_FAIL; 
> +            goto out; 
> +        } 
> +    } 
> + 
> +    if (usbctrl->type != LIBXL_USBCTRL_TYPE_PV) { 
> +        LOG(ERROR, "Unsupported USB controller type"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    rc = libxl__device_usbctrl_add_xenstore(gc, domid, usbctrl, 
> +                                            aodev->update_json); 
> +    if (rc) goto out; 
> + 
> +    GCNEW(device); 
> +    rc = libxl__device_from_usbctrl(gc, domid, usbctrl, device); 
> +    if (rc) goto out; 
> + 
> +    aodev->dev = device; 
> +    aodev->action = LIBXL__DEVICE_ACTION_ADD; 
> +    libxl__wait_device_connection(egc, aodev); 
> +    return; 
> + 
> +out: 
> +    aodev->rc = rc; 
> +    if (rc) aodev->callback(egc, aodev); 
> +    return; 
> +} 
> + 
> +static int 
> +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid, 
> +                                   libxl_devid usbctrl, 
> +                                   libxl_device_usb **usbs, int *num); 
> + 
> +static int 
> +libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb  
> *usb); 
> + 
> +/* AO function to remove a usb controller. 
> + * 
> + * Generally, it does: 
> + * 1) check if the usb controller exists or not 
> + * 2) remove all usb devices under controller 
> + * 3) remove usb controller information from xenstore 
> + * 
> + * Before calling this function, aodev should be properly filled: 
> + * aodev->ao, aodev->dev, aodev->callback, ... 
> + */ 
> +void libxl__initiate_device_usbctrl_remove(libxl__egc *egc, 
> +                                           libxl__ao_device *aodev) 
> +{ 
> +    STATE_AO_GC(aodev->ao); 
> +    libxl_ctx *ctx = CTX; 
> +    libxl_device_usb *usbs = NULL; 
> +    int numusb = 0; 
> +    int i, rc; 
> +    uint32_t domid = ao->domid; 
> +    int usbctrl_devid = aodev->dev->devid; 
> +    libxl_device_usbctrl usbctrl; 
> +    libxl_usbctrlinfo usbctrlinfo; 
> + 
> +    libxl_device_usbctrl_init(&usbctrl); 
> +    libxl_usbctrlinfo_init(&usbctrlinfo); 
> +    usbctrl.devid = usbctrl_devid; 
> + 
> +    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo); 
> +    if (rc) goto out; 
> + 
> +    if (usbctrlinfo.type != LIBXL_USBCTRL_TYPE_PV) { 
> +        LOG(ERROR, "Unsupported USB controller type"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    /* Remove usb devices first */ 
> +    rc  = libxl__device_usb_list_for_usbctrl(gc, domid, usbctrl_devid, 
> +                                             &usbs, &numusb); 
> +    if (rc) goto out; 
> + 
> +    for (i = 0; i < numusb; i++) { 
> +        if (libxl__device_usb_remove(gc, domid, &usbs[i])) { 
> +            LOG(ERROR, "libxl__device_usb_remove failed"); 
> +            rc = ERROR_FAIL; 
> +            goto out; 
> +        } 
> +    } 
> + 
> +    libxl_device_usbctrl_dispose(&usbctrl); 
> +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> + 
> +    /* Remove usbctrl */ 
> +    return libxl__initiate_device_remove(egc, aodev); 
> + 
> +out: 
> +    libxl_device_usbctrl_dispose(&usbctrl); 
> +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> +    aodev->rc = rc; 
> +    if (rc) aodev->callback(egc, aodev); 
> +    return; 
> +} 
> + 
> +libxl_device_usbctrl * 
> +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num) 
> +{ 
> +    GC_INIT(ctx); 
> +    libxl_device_usbctrl *usbctrls = NULL; 
> +    char *path = NULL; 
> +    char **entry = NULL; 
> +    unsigned int nentries = 0; 
> + 
> +    *num = 0; 
> + 
> +    path = GCSPRINTF("%s/device/vusb", 
> +                     libxl__xs_get_dompath(gc, domid)); 
> +    entry = libxl__xs_directory(gc, XBT_NULL, path, &nentries); 
> + 
> +    if (entry && nentries) { 
> +        usbctrls = libxl__zalloc(NOGC, sizeof(*usbctrls) * nentries); 
> +        libxl_device_usbctrl *usbctrl; 
> +        libxl_device_usbctrl *end = usbctrls + nentries; 
> +        for (usbctrl = usbctrls; usbctrl < end; 
> +             usbctrl++, entry++, (*num)++) { 
> +            const char *tmp, *be_path; 
> +            int rc; 
> +            const char *fe_path = GCSPRINTF("%s/%s", path, *entry); 
> + 
> +            libxl_device_usbctrl_init(usbctrl); 
> +            usbctrl->devid = atoi(*entry); 
> + 
> +#define READ_SUBPATH(path, subpath) ({                                  \ 
> +        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \ 
> +                                    GCSPRINTF("%s/" subpath, path),     \ 
> +                                    &tmp);                              \ 
> +        if (rc) goto outerr;                                            \ 
> +        (char *)tmp;                                                    \ 
> +    }) 
> +            be_path = READ_SUBPATH(fe_path, "backend"); 
> +            usbctrl->backend_domid = atoi(READ_SUBPATH(fe_path,  
> "backend-id")); 
> +            usbctrl->version = atoi(READ_SUBPATH(be_path, "usb-ver")); 
> +            usbctrl->ports = atoi(READ_SUBPATH(be_path, "num-ports")); 
> +            libxl_usbctrl_type_from_string(READ_SUBPATH(be_path, "type"), 
> +                                           &usbctrl->type); 
> +#undef READ_SUBPATH 
> +       } 
> +    } 
> + 
> +    goto out; 
> + 
> +outerr: 
> +    LOG(ERROR, "Unable to list USB Controllers"); 
> +    libxl_device_usbctrl_list_free(usbctrls, *num); 
> +    *num = 0; 
> +    usbctrls = NULL; 
> + 
> +out: 
> +    GC_FREE; 
> +    return usbctrls; 
> +} 
> + 
> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid, 
> +                                libxl_device_usbctrl *usbctrl, 
> +                                libxl_usbctrlinfo *usbctrlinfo) 
> +{ 
> +    GC_INIT(ctx); 
> +    const char *dompath, *fe_path, *be_path, *tmp; 
> +    int rc; 
> + 
> +    usbctrlinfo->devid = usbctrl->devid; 
> + 
> +#define READ_SUBPATH(path, subpath) ({                                  \ 
> +        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \ 
> +                                    GCSPRINTF("%s/" subpath, path),     \ 
> +                                    &tmp);                              \ 
> +        if (rc) goto out;                                               \ 
> +        (char *)tmp;                                                    \ 
> +    }) 
> + 
> +    dompath = libxl__xs_get_dompath(gc, domid); 
> +    fe_path = GCSPRINTF("%s/device/vusb/%d", dompath, usbctrl->devid); 
> +    be_path = READ_SUBPATH(fe_path, "backend"); 
> + 
> +    usbctrlinfo->backend = libxl__strdup(NOGC, be_path); 
> + 
> +    tmp = READ_SUBPATH(fe_path, "backend-id"); 
> +    usbctrlinfo->backend_id = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(fe_path, "state"); 
> +    usbctrlinfo->state = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(fe_path, "event-channel"); 
> +    usbctrlinfo->evtch = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(fe_path, "urb-ring-ref"); 
> +    usbctrlinfo->ref_urb = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(fe_path, "conn-ring-ref"); 
> +    usbctrlinfo->ref_conn = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(be_path, "frontend"); 
> +    usbctrlinfo->frontend = libxl__strdup(NOGC, tmp); 
> +    tmp = READ_SUBPATH(be_path, "frontend-id"); 
> +    usbctrlinfo->frontend_id = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(be_path, "num-ports"); 
> +    usbctrlinfo->ports = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(be_path, "usb-ver"); 
> +    usbctrlinfo->version = tmp ? strtoul(tmp, NULL, 10) : -1; 
> +    tmp = READ_SUBPATH(be_path, "type"); 
> +    libxl_usbctrl_type_from_string(tmp, &usbctrlinfo->type); 
> + 
> +#undef READ_SUBPATH 
> + 
> +    rc = 0; 
> + 
> +out: 
> +    GC_FREE; 
> +    return rc; 
> +} 
> + 
> +int libxl_devid_to_device_usbctrl(libxl_ctx *ctx, 
> +                                  uint32_t domid, 
> +                                  int devid, 
> +                                  libxl_device_usbctrl *usbctrl) 
> +{ 
> +    libxl_device_usbctrl *usbctrls; 
> +    int nb = 0; 
> +    int i, rc; 
> + 
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &nb); 
> +    if (!usbctrls) 
> +        return ERROR_FAIL; 
> + 
> +    rc = ERROR_FAIL; 
> +    for (i = 0; i < nb; i++) { 
> +        if (devid == usbctrls[i].devid) { 
> +            libxl_device_usbctrl_copy(ctx, usbctrl, &usbctrls[i]); 
> +            rc = 0; 
> +            break; 
> +        } 
> +    } 
> + 
> +    libxl_device_usbctrl_list_free(usbctrls, nb); 
> +    return rc; 
> +} 
> + 
> +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr) 
> +{ 
> +    struct dirent *de; 
> +    DIR *dir; 
> +    char *busid = NULL; 
> + 
> +    /* invalid hostbus or hostaddr */ 
> +    if (bus < 1 || addr < 1) 
> +        return NULL; 
> + 
> +    if (!(dir = opendir(SYSFS_USB_DEV))) 
> +        return NULL; 
> + 
> +    while ((de = readdir(dir))) { 
> +        char *filename; 
> +        void *buf; 
> +        int busnum = -1; 
> +        int devnum = -1; 
> + 
> +        if (!de->d_name) 
> +            continue; 
> + 
> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name); 
> +        if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> +            devnum = atoi((char *)buf); 
> + 
> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name); 
> +        if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> +            busnum = atoi((char *)buf); 
> + 
> +        if (bus == busnum && addr == devnum) { 
> +            busid = libxl__strdup(gc, de->d_name); 
> +            break; 
> +        } 
> +    } 
> + 
> +    closedir(dir); 
> +    return busid; 
> +} 
> + 
> +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid, 
> +                                  uint8_t *bus, uint8_t *addr) 
> +{ 
> +    char *filename; 
> +    void *buf; 
> + 
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid); 
> +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> +        *bus = atoi((char *)buf); 
> +    else 
> +        return ERROR_FAIL; 
> + 
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", busid); 
> +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> +        *addr = atoi((char *)buf); 
> +    else 
> +        return ERROR_FAIL; 
> + 
> +    return 0; 
> +} 
> + 
> +static int 
> +get_assigned_devices(libxl__gc *gc, 
> +                     libxl_device_usb **list, int *num) 
> +{ 
> +    char **domlist; 
> +    unsigned int nd = 0, i, j, k; 
> +    int rc; 
> + 
> +    *list = NULL; 
> +    *num = 0; 
> + 
> +    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd); 
> +    for (i = 0; i < nd; i++) { 
> +        char *path, **ctrl_list; 
> +        unsigned int nc = 0; 
> + 
> +        path = GCSPRINTF("/local/domain/%s/device/vusb", domlist[i]); 
> +        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path, &nc); 
> + 
> +        for (j = 0; j < nc; j++) { 
> +            const char *be_path, *num_ports; 
> +            int nport; 
> + 
> +            rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                          GCSPRINTF("%s/%s/backend", path, ctrl_list[j]), 
> +                          &be_path); 
> +            if (rc) goto out; 
> + 
> +            rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                        GCSPRINTF("%s/num-ports", be_path), 
> +                                        &num_ports); 
> +            if (rc) goto out; 
> + 
> +            nport = atoi(num_ports); 
> +            for (k = 0; k < nport; k++) { 
> +                libxl_device_usb *usb; 
> +                const char *portpath, *busid; 
> + 
> +                portpath = GCSPRINTF("%s/port/%d", be_path, k + 1); 
> +                busid = libxl__xs_read(gc, XBT_NULL, portpath); 
> +                /* If there is USB device attached, add it to list */ 
> +                if (busid && strcmp(busid, "")) { 
> +                    GCREALLOC_ARRAY(*list, *num + 1); 
> +                    usb = *list + *num; 
> +                    (*num)++; 
> +                    libxl_device_usb_init(usb); 
> +                    usb->ctrl = atoi(ctrl_list[j]); 
> +                    usb->port = k + 1; 
> +                    rc = usb_busaddr_from_busid(gc, busid, 
> +                                                &usb->u.hostdev.hostbus, 
> +                                                &usb->u.hostdev.hostaddr); 
> +                    if (rc) goto out; 
> +                } 
> +            } 
> +        } 
> +    } 
> + 
> +    rc = 0; 
> + 
> +out: 
> +    if (rc) { 
> +        *list = NULL; 
> +        *num = 0; 
> +    } 
> +    return rc; 
> +} 
> + 
> +static bool is_usbdev_in_array(libxl_device_usb *usbs, int num, 
> +                               libxl_device_usb *usb) 
> +{ 
> +    int i; 
> + 
> +    for (i = 0; i < num; i++) { 
> +        if (usbs[i].u.hostdev.hostbus == usb->u.hostdev.hostbus && 
> +            usbs[i].u.hostdev.hostaddr == usb->u.hostdev.hostaddr) 
> +            return true; 
> +    } 
> + 
> +    return false; 
> +} 
> + 
> +/* check if USB device is already assigned to a domain */ 
> +/* check if USB device type is assignable */ 
> +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb) 
> +{ 
> +    int classcode; 
> +    char *filename; 
> +    void *buf = NULL; 
> +    char *busid = NULL; 
> + 
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> +                                 usb->u.hostdev.hostaddr); 
> +    if (!busid) return false; 
> + 
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid); 
> +    if (libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> +        return false; 
> + 
> +    classcode = atoi(buf); 
> +    return classcode != USBHUB_CLASS_CODE; 
> +} 
> + 
> +/* get usb devices under certain usb controller */ 
> +static int 
> +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid, 
> +                                   libxl_devid usbctrl, 
> +                                   libxl_device_usb **usbs, int *num) 
> +{ 
> +    const char *fe_path, *be_path, *num_devs; 
> +    int n, i, rc; 
> + 
> +    *usbs = NULL; 
> +    *num = 0; 
> + 
> +    fe_path = GCSPRINTF("%s/device/vusb/%d", 
> +                        libxl__xs_get_dompath(gc, domid), usbctrl); 
> + 
> +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                GCSPRINTF("%s/backend", fe_path), 
> +                                &be_path); 
> +    if (rc) return rc; 
> + 
> +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                GCSPRINTF("%s/num-ports", be_path), 
> +                                &num_devs); 
> +    if (rc) return rc; 
> + 
> +    n = atoi(num_devs); 
> + 
> +    for (i = 0; i < n; i++) { 
> +        char *busid; 
> +        libxl_device_usb *usb; 
> + 
> +        busid = libxl__xs_read(gc, XBT_NULL, 
> +                               GCSPRINTF("%s/port/%d", be_path, i + 1)); 
> +        if (busid && strcmp(busid, "")) { 
> +            GCREALLOC_ARRAY(*usbs, *num + 1); 
> +            usb = *usbs + *num; 
> +            (*num)++; 
> +            libxl_device_usb_init(usb); 
> +            usb->ctrl = usbctrl; 
> +            usb->port = i + 1; 
> +            rc = usb_busaddr_from_busid(gc, busid, 
> +                                        &usb->u.hostdev.hostbus, 
> +                                        &usb->u.hostdev.hostaddr); 
> +            if (rc) goto out; 
> +        } 
> +    } 
> + 
> +    rc = 0; 
> + 
> +out: 
> +    if (rc) { 
> +        *usbs = NULL; 
> +        *num = 0; 
> +    } 
> +    return rc; 
> +} 
> + 
> +/* get all usb devices of the domain */ 
> +libxl_device_usb * 
> +libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, int *num) 
> +{ 
> +    GC_INIT(ctx); 
> +    char *path; 
> +    char **usbctrls; 
> +    unsigned int nc = 0; 
> +    int i, j; 
> +    libxl_device_usb *usbs = NULL; 
> + 
> +    *num = 0; 
> + 
> +    path = GCSPRINTF("%s/device/vusb", 
> +                        libxl__xs_get_dompath(gc, domid)); 
> +    usbctrls = libxl__xs_directory(gc, XBT_NULL, path, &nc); 
> + 
> +    for (i = 0; i < nc; i++) { 
> +        int nd = 0; 
> +        libxl_device_usb *tmp = NULL; 
> +        libxl__device_usb_list_for_usbctrl(gc, domid, 
> +                                           atoi(usbctrls[i]), &tmp, &nd); 
> +        if (!nd) continue; 
> + 
> +        usbs = libxl__realloc(NOGC, usbs, sizeof(*usbs) * (*num + nd)); 
> +        for (j = 0; j < nd; j++) { 
> +            libxl_device_usb_copy(ctx, usbs + *num, tmp + j); 
> +            (*num)++; 
> +        } 
> +    } 
> + 
> +    GC_FREE; 
> +    return usbs; 
> +} 
> + 
> +/* find first unused controller:port and give that to usb device */ 
> +static int 
> +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid, 
> +                                      libxl_device_usb *usb) 
> +{ 
> +    libxl_ctx *ctx = CTX; 
> +    libxl_device_usbctrl *usbctrls = NULL; 
> +    int numctrl = 0; 
> +    int i, j, rc; 
> + 
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl); 
> +    if (!numctrl) { 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    rc = ERROR_FAIL; 
> +    for (i = 0; i < numctrl; i++) { 
> +        for (j = 0; j < usbctrls[i].ports; j++) { 
> +            char *path, *tmp; 
> + 
> +            path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> +                             libxl__xs_get_dompath(gc,  
> LIBXL_TOOLSTACK_DOMID), 
> +                             domid, usbctrls[i].devid, j + 1); 
> +            tmp = libxl__xs_read(gc, XBT_NULL, path); 
> +            if (tmp && !strcmp(tmp, "")) { 
> +                usb->ctrl = usbctrls[i].devid; 
> +                usb->port = j + 1; 
> +                rc = 0; 
> +                goto out; 
> +            } 
> +        } 
> +    } 
> + 
> +out: 
> +    libxl_device_usbctrl_list_free(usbctrls, numctrl); 
> +    return rc; 
> +} 
> + 
> +/* Fill in usb information with default value. 
> + * 
> + * Generally, it does: 
> + * 1) if "controller" is not specified: 
> + *    - if "port" is not specified, try to find an available  
> controller:port, 
> + *      if found, use that; otherwise, create a new controller, use this 
> + *      controller and its first port 
> + *    - if "port" is specified, report error. 
> + * 2) if "controller" is specified, but port is not specified: 
> + *    try to find an available port under this controller, if found, use 
> + *    that, otherwise, report error. 
> + * 3) if both "controller" and "port" are specified: 
> + *    check the controller:port is available, if not, report error. 
> + */ 
> +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid, 
> +                                        libxl_device_usb *usb, 
> +                                        bool update_json) 
> +{ 
> +    int rc; 
> + 
> +    if (!usb->devtype) 
> +        usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV; 
> + 
> +    if (usb->ctrl == -1) { 
> +        if (usb->port) { 
> +            LOG(ERROR, "USB controller must be specified if you specify  
> port"); 
> +            return ERROR_INVAL; 
> +        } 
> + 
> +        rc = libxl__device_usb_set_default_usbctrl(gc, domid, usb); 
> +        /* If no existing controller to host this usb device, add a new one  
> */ 
> +        if (rc) { 
> +            libxl_device_usbctrl *usbctrl; 
> + 
> +            GCNEW(usbctrl); 
> +            libxl_device_usbctrl_init(usbctrl); 
> +            rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl); 
> +            if (rc < 0) goto out; 
> + 
> +            if (usbctrl->devid == -1) { 
> +                usbctrl->devid = libxl__device_nextid(gc, domid, "vusb"); 
> +                if (usbctrl->devid < 0) { 
> +                    rc = ERROR_FAIL; 
> +                    goto out; 
> +                } 
> +            } 
> + 
> +            rc = libxl__device_usbctrl_add_xenstore(gc, domid, usbctrl, 
> +                                                    update_json); 
> +            if (rc) goto out; 
> + 
> +            usb->ctrl = usbctrl->devid; 
> +            usb->port = 1; 
> +        } 
> +    } else if (!usb->port) { 
> +        /* Valid port starts from 1. Choose port for us. */ 
> +        int i, ports; 
> +        const char *fe_path, *be_path, *tmp; 
> + 
> +        fe_path = GCSPRINTF("%s/device/vusb/%d", 
> +                         libxl__xs_get_dompath(gc, domid), usb->ctrl); 
> + 
> +        rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                    GCSPRINTF("%s/backend", fe_path),  
> &be_path); 
> +        if (rc) goto out; 
> + 
> +        rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                    GCSPRINTF("%s/num-ports", be_path),  
> &tmp); 
> +        if (rc) goto out; 
> + 
> +        ports = atoi(tmp); 
> + 
> +        for (i = 0; i < ports; i++) { 
> +            tmp = libxl__xs_read(gc, XBT_NULL, 
> +                                 GCSPRINTF("%s/port/%d", be_path, i + 1)); 
> +            if (tmp && !strcmp(tmp, "")) { 
> +                usb->port = i + 1; 
> +                break; 
> +            } 
> +        } 
> + 
> +        if (!usb->port) { 
> +            LOG(ERROR, "No available port under specified controller"); 
> +            rc = ERROR_FAIL; 
> +            goto out; 
> +        } 
> +    } else { 
> +        const char *fe_path, *be_path, *tmp; 
> + 
> +        fe_path = GCSPRINTF("%s/device/vusb/%d", 
> +                         libxl__xs_get_dompath(gc, domid), usb->ctrl); 
> + 
> +        rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                    GCSPRINTF("%s/backend", fe_path),  
> &be_path); 
> +        if (rc) goto out; 
> + 
> +        rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                                    GCSPRINTF("%s/port/%d", be_path,  
> usb->port), 
> +                                    &tmp); 
> +        if (rc) goto out; 
> + 
> +        if (strcmp(tmp, "")) { 
> +            LOG(ERROR, "The controller port isn't available"); 
> +            rc = ERROR_FAIL; 
> +            goto out; 
> +        } 
> +    } 
> + 
> +    rc = 0; 
> + 
> +out: 
> +    return rc; 
> +} 
> + 
> +/* Add usb information to xenstore 
> + * 
> + * Adding a usb device won't create new 'vusb' device, but only write 
> + * the device busid to the controller:port in xenstore. 
> + */ 
> +static int libxl__device_usb_add_xenstore(libxl__gc *gc, uint32_t domid, 
> +                                          libxl_device_usb *usb, 
> +                                          bool update_json) 
> +{ 
> +    char *be_path; 
> +    char *busid; 
> +    int rc; 
> +    xs_transaction_t t = XBT_NULL; 
> +    libxl_domain_config d_config; 
> +    libxl_device_usb usb_saved; 
> +    libxl__domain_userdata_lock *lock = NULL; 
> + 
> +    libxl_domain_config_init(&d_config); 
> +    libxl_device_usb_init(&usb_saved); 
> +    libxl_device_usb_copy(CTX, &usb_saved, usb); 
> + 
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> +                                 usb->u.hostdev.hostaddr); 
> +    if (!busid) { 
> +        LOG(DEBUG, "Fail to get busid of usb device"); 
> +        goto out; 
> +    } 
> + 
> +    if (update_json) { 
> +        lock = libxl__lock_domain_userdata(gc, domid); 
> +        if (!lock) { 
> +            rc = ERROR_LOCK_FAIL; 
> +            goto out; 
> +        } 
> + 
> +        rc = libxl__get_domain_configuration(gc, domid, &d_config); 
> +        if (rc) goto out; 
> + 
> +        DEVICE_ADD(usb, usbs, domid, &usb_saved, COMPARE_USB, &d_config); 
> +    } 
> + 
> +    for (;;) { 
> +        rc = libxl__xs_transaction_start(gc, &t); 
> +        if (rc) goto out; 
> + 
> +        if (update_json) { 
> +            rc = libxl__set_domain_configuration(gc, domid, &d_config); 
> +            if (rc) goto out; 
> +        } 
> + 
> +        be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> +                            libxl__xs_get_dompath(gc,  
> LIBXL_TOOLSTACK_DOMID), 
> +                            domid, usb->ctrl, usb->port); 
> + 
> +        LOG(DEBUG, "Adding new usb device to xenstore"); 
> +        if (libxl__xs_write_checked(gc, t, be_path, busid)) 
> +            goto out; 
> + 
> +        rc = libxl__xs_transaction_commit(gc, &t); 
> +        if (!rc) break; 
> +        if (rc < 0) goto out; 
> +    } 
> + 
> +    rc = 0; 
> + 
> +out: 
> +    if (lock) libxl__unlock_domain_userdata(lock); 
> +    libxl_device_usb_dispose(&usb_saved); 
> +    libxl_domain_config_dispose(&d_config); 
> +    return rc; 
> +} 
> + 
> +static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t domid, 
> +                                             libxl_device_usb *usb) 
> +{ 
> +    char *be_path; 
> + 
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> +                        libxl__xs_get_dompath(gc, LIBXL_TOOLSTACK_DOMID), 
> +                        domid, usb->ctrl, usb->port); 
> +    LOG(DEBUG, "Removing USB device from xenstore"); 
> +    if (libxl__xs_write_checked(gc, XBT_NULL, be_path, "")) 
> +        return ERROR_FAIL; 
> + 
> +    return 0; 
> +} 
> + 
> +/* bind/unbind usb device interface */ 
> +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath) 
> +{ 
> +    char *path, *spath, *dp = NULL; 
> +    int fd = -1; 
> +    int rc; 
> +    struct stat st; 
> + 
> +    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf); 
> +    if (!lstat(spath, &st)) { 
> +        /* Find the canonical path to the driver. */ 
> +        dp = libxl__zalloc(gc, PATH_MAX); 
> +        dp = realpath(spath, dp); 
> + 
> +        path = GCSPRINTF("%s/unbind", spath); 
> +        fd = open(path, O_WRONLY); 
> +        if (fd < 0) 
> +            return ERROR_FAIL; 
> +        rc = write(fd, intf, strlen(intf)); 
> +        close(fd); 
> +        if (rc < 0) 
> +            return ERROR_FAIL; 
> +    } 
> + 
> +    if (drvpath) 
> +        *drvpath = dp; 
> + 
> +    return 0; 
> +} 
> + 
> +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath) 
> +{ 
> +    char *path; 
> +    struct stat st; 
> +    int fd, rc; 
> + 
> +    path = GCSPRINTF("%s/%s", drvpath, intf); 
> +    rc = lstat(path, &st); 
> +    /* already bind, return */ 
> +    if (!rc) 
> +        return 0; 
> + 
> +    path = GCSPRINTF("%s/bind", drvpath); 
> +    fd = open(path, O_WRONLY); 
> +    if (fd < 0) 
> +        return ERROR_FAIL; 
> + 
> +    rc = write(fd, intf, strlen(intf)); 
> +    close(fd); 
> +    if (rc < 0) 
> +        return ERROR_FAIL; 
> + 
> +    return 0; 
> +} 
> + 
> +/* Is usb interface bound to usbback? */ 
> +static int usb_intf_is_assigned(libxl__gc *gc, char *intf) 
> +{ 
> +    char *spath; 
> +    int rc; 
> +    struct stat st; 
> + 
> +    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf); 
> +    rc = lstat(spath, &st); 
> + 
> +    if (rc == 0) 
> +        return 1; 
> +    if (rc < 0 && errno == ENOENT) 
> +        return 0; 
> +    LOGE(ERROR, "Accessing %s", spath); 
> +    return -1; 
> +} 
> + 
> +static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb, 
> +                                  char ***intfs, int *num) 
> +{ 
> +    DIR *dir; 
> +    struct dirent *entry; 
> +    char *buf; 
> +    char *busid; 
> +    int rc; 
> + 
> +    *intfs = NULL; 
> +    *num = 0; 
> + 
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> +                                 usb->u.hostdev.hostaddr); 
> +    if (!busid) { 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    buf = GCSPRINTF("%s:", busid); 
> + 
> +    if (!(dir = opendir(SYSFS_USB_DEV))) { 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    while ((entry = readdir(dir)) != NULL) { 
> +        if (!strncmp(entry->d_name, buf, strlen(buf))) { 
> +            GCREALLOC_ARRAY(*intfs, *num + 1); 
> +            if (*intfs == NULL) { 
> +                rc = ERROR_FAIL; 
> +                goto out; 
> +            } 
> +            (*intfs)[*num] = libxl__strdup(gc, entry->d_name); 
> +            (*num)++; 
> +        } 
> +    } 
> + 
> +    closedir(dir); 
> +    rc = 0; 
> + 
> +out: 
> +    return rc; 
> +} 
> + 
> +/* Encode usb interface so that it could be written to xenstore as a key. 
> + * 
> + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_', 
> + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1. 
> + * This will be used to save original driver of USB device to xenstore. 
> + */ 
> +static char *usb_interface_xenstore_encode(char *busid) 
> +{ 
> +    char *str = strdup(busid); 
> +    int i, len = strlen(str); 
> + 
> +    for (i = 0; i < len; i++) { 
> +        if (str[i] == '.') 
> +            str[i] = '_'; 
> +         if (str[i] == ':') 
> +            str[i] = '-'; 
> +    } 
> +    return str; 
> +} 
> + 
> +/* Unbind USB device from "usbback" driver. 
> + * 
> + * If there are many interfaces under USB device, check each interface, 
> + * unbind from "usbback" driver and rebind to its original driver. 
> + */ 
> +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb) 
> +{ 
> +    char **intfs = NULL; 
> +    char *path; 
> +    int num = 0, i; 
> +    int rc; 
> +    char *busid; 
> +    char *usb_encode = NULL; 
> + 
> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0) 
> +        return ERROR_FAIL; 
> + 
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> +                                 usb->u.hostdev.hostaddr); 
> +    usb_encode = usb_interface_xenstore_encode(busid); 
> + 
> +    for (i = 0; i < num; i++) { 
> +        char *intf = intfs[i]; 
> +        char *drvpath = NULL; 
> + 
> +        /* check if the USB interface is already bound to "usbbcak" */ 
> +        if (usb_intf_is_assigned(gc, intf) > 0) { 
> +            /* unbind interface from usbback driver */ 
> +            if (unbind_usb_intf(gc, intf, NULL) < 0) { 
> +                rc = ERROR_FAIL; 
> +                goto out; 
> +            } 
> +        } 
> + 
> +        /* bind interface to its originial driver */ 
> +        drvpath = libxl__xs_read(gc, XBT_NULL, 
> +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path", 
> +                  usb_encode, usb_interface_xenstore_encode(intf))); 
> +        if (drvpath && bind_usb_intf(gc, intf, drvpath)) 
> +            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath); 
> +    } 
> + 
> +    /* finally, remove xs driver path */ 
> +    path = GCSPRINTF(USBBACK_INFO_PATH"/%s", usb_encode); 
> +    libxl__xs_rm_checked(gc, XBT_NULL, path); 
> +    rc = 0; 
> + 
> +out: 
> +    free(usb_encode); 
> +    return rc; 
> +} 
> + 
> +/* Bind USB device to "usbback" driver. 
> + * 
> + * If there are many interfaces under USB device, check each interface, 
> + * unbind from original driver and bind to "usbback" driver. 
> + */ 
> +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb) 
> +{ 
> +    char **intfs = NULL; 
> +    int num = 0, i; 
> +    int rc; 
> +    char *busid; 
> +    char *usb_encode = NULL; 
> + 
> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0) 
> +        return ERROR_FAIL; 
> + 
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> +                                 usb->u.hostdev.hostaddr); 
> +    usb_encode = usb_interface_xenstore_encode(busid); 
> + 
> +    for (i = 0; i < num; i++) { 
> +        char *intf = intfs[i]; 
> +        char *path = NULL; 
> +        char *drvpath = NULL; 
> + 
> +        /* already assigned to usbback */ 
> +        if (usb_intf_is_assigned(gc, intf) > 0) 
> +            continue; 
> + 
> +        /* unbind interface from original driver */ 
> +        if (unbind_usb_intf(gc, intf, &drvpath) < 0) { 
> +            rc = ERROR_FAIL; 
> +            goto out_rebind; 
> +        } 
> + 
> +        if (drvpath) { 
> +            /* write driver path to xenstore for later rebinding */ 
> +            path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path", 
> +                             usb_encode,  
> usb_interface_xenstore_encode(intf)); 
> +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) { 
> +                LOG(WARN, "Write of %s to node %s failed", drvpath, path); 
> +            } 
> +        } 
> + 
> +        /* bind interface to usbback */ 
> +        if (bind_usb_intf(gc, intf, SYSFS_USBBACK_DRIVER) < 0) { 
> +            LOGE(ERROR, "Couldn't bind %s to %s", intf,  
> SYSFS_USBBACK_DRIVER); 
> +            rc = ERROR_FAIL; 
> +            goto out_rebind; 
> +        } 
> +    } 
> + 
> +    rc = 0; 
> +    goto out; 
> + 
> +out_rebind: 
> +    /* some interfaces might be bound to usbback, unbind it then and 
> +     * rebind to its original driver 
> +     */ 
> +    usbback_dev_unassign(gc, usb); 
> +out: 
> +    free(usb_encode); 
> +    return rc; 
> +} 
> + 
> +/* 
> + * USB add 
> + */ 
> +static int do_usb_add(libxl__gc *gc, uint32_t domid, 
> +                      libxl_device_usb *usbdev, 
> +                      libxl__ao_device *aodev) 
> +{ 
> +    libxl_ctx *ctx = CTX; 
> +    libxl_usbctrlinfo usbctrlinfo; 
> +    libxl_device_usbctrl usbctrl; 
> +    int rc; 
> + 
> +    libxl_device_usbctrl_init(&usbctrl); 
> +    libxl_usbctrlinfo_init(&usbctrlinfo); 
> +    usbctrl.devid = usbdev->ctrl; 
> +    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo); 
> +    if (rc) 
> +        goto out; 
> + 
> +    switch (usbctrlinfo.type) { 
> +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL: 
> +        LOG(ERROR, "Not supported"); 
> +        break; 
> +    case LIBXL_USBCTRL_TYPE_PV: 
> +        rc = libxl__device_usb_add_xenstore(gc, domid, usbdev, 
> +                                            aodev->update_json); 
> +        if (rc) goto out; 
> + 
> +        rc = usbback_dev_assign(gc, usbdev); 
> +        if (rc) { 
> +            libxl__device_usb_remove_xenstore(gc, domid, usbdev); 
> +            goto out; 
> +        } 
> +        break; 
> +    default: 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +out: 
> +    libxl_device_usbctrl_dispose(&usbctrl); 
> +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> +    return rc; 
> +} 
> + 
> +/* AO operation to add a usb device. 
> + * 
> + * Generally, it does: 
> + * 1) check if the usb device type is assignable 
> + * 2) check if the usb device is already assigned to a domain 
> + * 3) add 'busid' of the usb device to xenstore contoller/port/. 
> + *    (PVUSB driver watches the xenstore changes and will detect that.) 
> + * 4) unbind usb device from original driver and bind to usbback. 
> + *    If usb device has many interfaces, then: 
> + *    - unbind each interface from its original driver and bind to usbback. 
> + *    - store the original driver to xenstore for later rebinding when 
> + *      detaching the device. 
> + * 
> + * Before calling this function, aodev should be properly filled: 
> + * aodev->ao, aodev->callback, aodev->update_json, ... 
> + */ 
> +void libxl__device_usb_add(libxl__egc *egc, uint32_t domid, 
> +                           libxl_device_usb *usb, 
> +                           libxl__ao_device *aodev) 
> +{ 
> +    STATE_AO_GC(aodev->ao); 
> +    int rc; 
> +    char *busid = NULL; 
> +    libxl_device_usb *assigned; 
> +    int num_assigned; 
> +    libxl_device_usbctrl usbctrl; 
> +    libxl_usbctrlinfo usbctrlinfo; 
> + 
> +    libxl_device_usbctrl_init(&usbctrl); 
> +    libxl_usbctrlinfo_init(&usbctrlinfo); 
> + 
> +    /* Currently only support adding USB device from Dom0 backend. 
> +     * So, if USB controller is specified, check backend domain, if 
> +     * it is not Dom0, report error as not supported. 
> +     * 
> +     * -1 is the default value of libxl_device_usbctrl. 
> +     */ 
> +    if (usb->ctrl != -1) { 
> +        usbctrl.devid = usb->ctrl; 
> +        rc = libxl_device_usbctrl_getinfo(CTX, domid, &usbctrl,  
> &usbctrlinfo); 
> +        if (rc) goto out; 
> + 
> +        if (usbctrlinfo.backend_id != 0) { 
> +            LOG(ERROR, "Don't support adding USB device from non-Dom0  
> backend"); 
> +            rc = ERROR_INVAL; 
> +            goto out; 
> +        } 
> +    } 
> + 
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> +                                 usb->u.hostdev.hostaddr); 
> +    if (!busid) { 
> +        LOG(ERROR, "USB device doesn't exist in sysfs"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    if (!is_usb_assignable(gc, usb)) { 
> +        LOG(ERROR, "USB device is not assignable."); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    /* check usb device is already assigned */ 
> +    rc = get_assigned_devices(gc, &assigned, &num_assigned); 
> +    if (rc) { 
> +        LOG(ERROR, "cannot determine if device is assigned," 
> +                   " refusing to continue"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    if (is_usbdev_in_array(assigned, num_assigned, usb)) { 
> +        LOG(ERROR, "USB device already attached to a domain"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    rc = libxl__device_usb_setdefault(gc, domid, usb, aodev->update_json); 
> +    if (rc) goto out; 
> + 
> +    /* Do the add */ 
> +    if (do_usb_add(gc, domid, usb, aodev)) { 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    libxl__ao_complete(egc, ao, 0); 
> +    rc = 0; 
> + 
> +out: 
> +    libxl_device_usbctrl_dispose(&usbctrl); 
> +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> +    aodev->rc = rc; 
> +    if (rc) aodev->callback(egc, aodev); 
> +    return; 
> +} 
> + 
> +static int do_usb_remove(libxl__gc *gc, uint32_t domid, 
> +                         libxl_device_usb *usbdev) 
> +{ 
> +    libxl_ctx *ctx = CTX; 
> +    libxl_usbctrlinfo usbctrlinfo; 
> +    libxl_device_usbctrl usbctrl; 
> +    int rc; 
> + 
> +    libxl_device_usbctrl_init(&usbctrl); 
> +    libxl_usbctrlinfo_init(&usbctrlinfo); 
> +    usbctrl.devid = usbdev->ctrl; 
> + 
> +    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo); 
> +    if (rc) goto out; 
> + 
> +    if (usbctrlinfo.backend_id != 0) { 
> +        LOG(ERROR, "Don't support removing USB device from non-Dom0  
> backend"); 
> +        rc = ERROR_INVAL; 
> +        goto out; 
> +    } 
> + 
> +    switch (usbctrlinfo.type) { 
> +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL: 
> +        LOG(ERROR, "Not supported"); 
> +        break; 
> +    case LIBXL_USBCTRL_TYPE_PV: 
> +        rc = libxl__device_usb_remove_xenstore(gc, domid, usbdev); 
> +        if (rc) goto out; 
> + 
> +        usbback_dev_unassign(gc, usbdev); 
> +        break; 
> +    default: 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    rc = 0; 
> + 
> +out: 
> +    libxl_device_usbctrl_dispose(&usbctrl); 
> +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> +    return rc; 
> +} 
> + 
> +/* Operation to remove usb device. 
> + * 
> + * Generally, it does: 
> + * 1) check if the usb device is assigned to the domain 
> + * 2) remove the usb device from xenstore controller/port. 
> + * 3) unbind usb device from usbback and rebind to its original driver. 
> + *    If usb device has many interfaces, do it to each interface. 
> + */ 
> +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, 
> +                                    libxl_device_usb *usb) 
> +{ 
> +    int rc; 
> + 
> +    if (usb->ctrl < 0 || usb->port < 1) { 
> +        LOG(ERROR, "Invalid USB device"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV && 
> +        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) { 
> +        LOG(ERROR, "Invalid USB device of hostdev"); 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    /* Do the remove */ 
> +    rc = do_usb_remove(gc, domid, usb); 
> + 
> +out: 
> +    return rc; 
> +} 
> + 
> +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid, 
> +                            libxl_device_usb *usb, 
> +                            const libxl_asyncop_how *ao_how) 
> + 
> +{ 
> +    AO_CREATE(ctx, domid, ao_how); 
> +    int rc; 
> + 
> +    rc = libxl__device_usb_remove(gc, domid, usb); 
> + 
> +    libxl__ao_complete(egc, ao, rc); 
> +    return AO_INPROGRESS; 
> +} 
> + 
> +int libxl_ctrlport_to_device_usb(libxl_ctx *ctx, 
> +                                 uint32_t domid, 
> +                                 int ctrl, 
> +                                 int port, 
> +                                 libxl_device_usb *usb) 
> +{ 
> +    GC_INIT(ctx); 
> +    const char *dompath, *be_path, *busid; 
> +    int rc; 
> + 
> +    dompath = libxl__xs_get_dompath(gc, domid); 
> + 
> +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                  GCSPRINTF("%s/device/vusb/%d/backend", dompath, ctrl), 
> +                  &be_path); 
> +    if (rc) goto out; 
> + 
> +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> +                           GCSPRINTF("%s/port/%d", be_path, port), 
> +                           &busid); 
> +    if (rc) goto out; 
> + 
> +    if (!strcmp(busid, "")) { 
> +        rc = ERROR_FAIL; 
> +        goto out; 
> +    } 
> + 
> +    usb->ctrl = ctrl; 
> +    usb->port = port; 
> +    rc = usb_busaddr_from_busid(gc, busid, &usb->u.hostdev.hostbus, 
> +                                &usb->u.hostdev.hostaddr); 
> + 
> +out: 
> +    GC_FREE; 
> +    return rc; 
> +} 
> +/* 
> + * Local variables: 
> + * mode: C 
> + * c-basic-offset: 4 
> + * indent-tabs-mode: nil 
> + * End: 
> + */ 
> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl 
> index d6ef9a2..dccb317 100644 
> --- a/tools/libxl/libxl_types.idl 
> +++ b/tools/libxl/libxl_types.idl 
> @@ -603,6 +603,35 @@ libxl_device_rdm = Struct("device_rdm", [ 
>      ("policy", libxl_rdm_reserve_policy), 
>      ]) 
>   
> +libxl_usbctrl_type = Enumeration("usbctrl_type", [ 
> +    (0, "AUTO"), 
> +    (1, "PV"), 
> +    (2, "DEVICEMODEL"), 
> +    ]) 
> + 
> +libxl_usbdev_type = Enumeration("usbdev_type", [ 
> +    (1, "hostdev"), 
> +    ]) 
> + 
> +libxl_device_usbctrl = Struct("device_usbctrl", [ 
> +    ("type", libxl_usbctrl_type), 
> +    ("devid", libxl_devid), 
> +    ("version", integer), 
> +    ("ports", integer), 
> +    ("backend_domid", libxl_domid), 
> +    ("backend_domname", string), 
> +   ]) 
> + 
> +libxl_device_usb = Struct("device_usb", [ 
> +    ("ctrl", libxl_devid), 
> +    ("port", integer), 
> +    ("u", KeyedUnion(None, libxl_usbdev_type, "devtype", 
> +           [("hostdev", Struct(None, [ 
> +                 ("hostbus",   uint8), 
> +                 ("hostaddr",  uint8)])), 
> +           ])), 
> +    ]) 
> + 
>  libxl_device_dtdev = Struct("device_dtdev", [ 
>      ("path", string), 
>      ]) 
> @@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [ 
>      ("pcidevs", Array(libxl_device_pci, "num_pcidevs")), 
>      ("rdms", Array(libxl_device_rdm, "num_rdms")), 
>      ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")), 
> +    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")), 
> +    ("usbs", Array(libxl_device_usb, "num_usbs")), 
>      ("vfbs", Array(libxl_device_vfb, "num_vfbs")), 
>      ("vkbs", Array(libxl_device_vkb, "num_vkbs")), 
>      ("vtpms", Array(libxl_device_vtpm, "num_vtpms")), 
> @@ -684,6 +715,32 @@ libxl_vtpminfo = Struct("vtpminfo", [ 
>      ("uuid", libxl_uuid), 
>      ], dir=DIR_OUT) 
>   
> +libxl_usbctrlinfo = Struct("usbctrlinfo", [ 
> +    ("type", libxl_usbctrl_type), 
> +    ("devid", libxl_devid), 
> +    ("version", integer), 
> +    ("ports", integer), 
> +    ("backend", string), 
> +    ("backend_id", uint32), 
> +    ("frontend", string), 
> +    ("frontend_id", uint32), 
> +    ("state", integer), 
> +    ("evtch", integer), 
> +    ("ref_urb", integer), 
> +    ("ref_conn", integer), 
> +    ], dir=DIR_OUT) 
> + 
> +libxl_usbinfo = Struct("usbinfo", [ 
> +    ("ctrl", libxl_devid), 
> +    ("port", integer), 
> +    ("busnum", uint8), 
> +    ("devnum", uint8), 
> +    ("idVendor", uint16), 
> +    ("idProduct", uint16), 
> +    ("prod", string), 
> +    ("manuf", string), 
> +    ], dir=DIR_OUT) 
> + 
>  libxl_vcpuinfo = Struct("vcpuinfo", [ 
>      ("vcpuid", uint32), 
>      ("cpu", uint32), 
> diff --git a/tools/libxl/libxl_types_internal.idl  
> b/tools/libxl/libxl_types_internal.idl 
> index 5e55685..696f5f8 100644 
> --- a/tools/libxl/libxl_types_internal.idl 
> +++ b/tools/libxl/libxl_types_internal.idl 
> @@ -22,6 +22,7 @@ libxl__device_kind = Enumeration("device_kind", [ 
>      (6, "VKBD"), 
>      (7, "CONSOLE"), 
>      (8, "VTPM"), 
> +    (9, "VUSB"), 
>      ]) 
>   
>  libxl__console_backend = Enumeration("console_backend", [ 
> diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c 
> index 9c5c4d0..706a0c1 100644 
> --- a/tools/libxl/libxl_utils.c 
> +++ b/tools/libxl/libxl_utils.c 
> @@ -1270,6 +1270,22 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf,  
> size_t len) 
>      return ret; 
>  } 
>   
> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr) 
> +{ 
> +   int i; 
> +   for (i = 0; i < nr; i++) 
> +       libxl_device_usbctrl_dispose(&list[i]); 
> +   free(list); 
> +} 
> + 
> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr) 
> +{ 
> +   int i; 
> +   for (i = 0; i < nr; i++) 
> +       libxl_device_usb_dispose(&list[i]); 
> +   free(list); 
> +} 
> + 
>  /* 
>   * Local variables: 
>   * mode: C 
> diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h 
> index 339ebdf..60487e3 100644 
> --- a/tools/libxl/libxl_utils.h 
> +++ b/tools/libxl/libxl_utils.h 
> @@ -76,6 +76,11 @@ int libxl_uuid_to_device_vtpm(libxl_ctx *ctx, uint32_t  
> domid, 
>                                 libxl_uuid *uuid, libxl_device_vtpm *vtpm); 
>  int libxl_devid_to_device_vtpm(libxl_ctx *ctx, uint32_t domid, 
>                                 int devid, libxl_device_vtpm *vtpm); 
> +int libxl_devid_to_device_usbctrl(libxl_ctx *ctx, uint32_t domid, 
> +                                  int devid, libxl_device_usbctrl  
> *usbctrl); 
> +int libxl_ctrlport_to_device_usb(libxl_ctx *ctx, uint32_t domid, 
> +                                 int ctrl, int port, 
> +                                 libxl_device_usb *usb); 
>   
>  int libxl_bitmap_alloc(libxl_ctx *ctx, libxl_bitmap *bitmap, int n_bits); 
>      /* Allocated bimap is from malloc, libxl_bitmap_dispose() to be 

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-04  6:31   ` Chun Yan Liu
@ 2015-11-05 15:54     ` George Dunlap
  0 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-05 15:54 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao

On Wed, Nov 4, 2015 at 6:31 AM, Chun Yan Liu <cyliu@suse.com> wrote:
> Ian & George, any comments?

Hey Chunyan,

I did actually spend a chunk of time looking at this last week.
Looking at the diff-of-diffs, it looks like you've addressed
everything I asked you to address.  I still want to take a longer look
at it before giving it a reviewed-by.  Unfortunately this will have to
wait until next week.

One thing that came up though in an offline discussion between IanJ
and I was that we would like you to actually address the
DEFINE_DEVICE_REMOVE_EXT code duplication issue before this is checked
in.  Let me know if you understand the request clearly; I'd be willing
to send you a patch you can fold in if that would be helpful.

IanJ said he has some more comments on the AO stuff as well.

 -George

>
>>>> On 10/21/2015 at 05:08 PM, in message
> <1445418510-19614-4-git-send-email-cyliu@suse.com>, Chunyan Liu
> <cyliu@suse.com> wrote:
>> Add pvusb APIs, including:
>>  - attach/detach (create/destroy) virtual usb controller.
>>  - attach/detach usb device
>>  - list usb controller and usb devices
>>  - some other helper functions
>>
>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>
>> ---
>> changes:
>>   - update COMPARE_USB to compare ctrl and port
>>   - add check in usb_add/remove to disable non-Dom0 backend so that
>>     not worring about codes which are effective on Dom0 but not
>>     compatible on non-Dom0 backend.
>>   - define READ_SUBPATH macro within functions
>>   - do not initialize rc but give it value in each return case
>>   - libxl__strdup gc or NOGC update, internal function using gc,
>>     external using NOGC.
>>   - address other comments from George and Ian J.
>>
>>  tools/libxl/Makefile                 |    2 +-
>>  tools/libxl/libxl.c                  |   53 ++
>>  tools/libxl/libxl.h                  |   74 ++
>>  tools/libxl/libxl_device.c           |    5 +-
>>  tools/libxl/libxl_internal.h         |   18 +
>>  tools/libxl/libxl_osdeps.h           |   13 +
>>  tools/libxl/libxl_pvusb.c            | 1451
>> ++++++++++++++++++++++++++++++++++
>>  tools/libxl/libxl_types.idl          |   57 ++
>>  tools/libxl/libxl_types_internal.idl |    1 +
>>  tools/libxl/libxl_utils.c            |   16 +
>>  tools/libxl/libxl_utils.h            |    5 +
>>  11 files changed, 1693 insertions(+), 2 deletions(-)
>>  create mode 100644 tools/libxl/libxl_pvusb.c
>>
>> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
>> index c5ecec1..ef9ccd3 100644
>> --- a/tools/libxl/Makefile
>> +++ b/tools/libxl/Makefile
>> @@ -103,7 +103,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o
>> libxl_dm.o libxl_pci.o \
>>                       libxl_stream_read.o libxl_stream_write.o \
>>                       libxl_save_callout.o _libxl_save_msgs_callout.o \
>>                       libxl_qmp.o libxl_event.o libxl_fork.o \
>> -                     libxl_dom_suspend.o $(LIBXL_OBJS-y)
>> +                     libxl_dom_suspend.o libxl_pvusb.o $(LIBXL_OBJS-y)
>>  LIBXL_OBJS += libxl_genid.o
>>  LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
>>
>> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
>> index dacfaae..a050e8b 100644
>> --- a/tools/libxl/libxl.c
>> +++ b/tools/libxl/libxl.c
>> @@ -4218,11 +4218,54 @@ DEFINE_DEVICE_REMOVE(vtpm, destroy, 1)
>>
>>
>> /****************************************************************************
>> **/
>>
>> +/* Macro for defining device remove/destroy functions for usbctrl */
>> +/* Follo:wing functions are defined:
>> + * libxl_device_usbctrl_remove
>> + * libxl_device_usbctrl_destroy
>> + */
>> +
>> +#define DEFINE_DEVICE_REMOVE_EXT(type, removedestroy, f)                \
>> +    int libxl_device_##type##_##removedestroy(libxl_ctx *ctx,           \
>> +        uint32_t domid, libxl_device_##type *type,                      \
>> +        const libxl_asyncop_how *ao_how)                                \
>> +    {                                                                   \
>> +        AO_CREATE(ctx, domid, ao_how);                                  \
>> +        libxl__device *device;                                          \
>> +        libxl__ao_device *aodev;                                        \
>> +        int rc;                                                         \
>> +                                                                        \
>> +        GCNEW(device);                                                  \
>> +        rc = libxl__device_from_##type(gc, domid, type, device);        \
>> +        if (rc != 0) goto out;                                          \
>> +                                                                        \
>> +        GCNEW(aodev);                                                   \
>> +        libxl__prepare_ao_device(ao, aodev);                            \
>> +        aodev->action = LIBXL__DEVICE_ACTION_REMOVE;                    \
>> +        aodev->dev = device;                                            \
>> +        aodev->callback = device_addrm_aocomplete;                      \
>> +        aodev->force = f;                                               \
>> +        libxl__initiate_device_##type##_remove(egc, aodev);             \
>> +                                                                        \
>> +    out:                                                                \
>> +        if (rc) return AO_CREATE_FAIL(rc);                              \
>> +        return AO_INPROGRESS;                                           \
>> +    }
>> +
>> +
>> +DEFINE_DEVICE_REMOVE_EXT(usbctrl, remove, 0)
>> +DEFINE_DEVICE_REMOVE_EXT(usbctrl, destroy, 1)
>> +
>> +#undef DEFINE_DEVICE_REMOVE_EXT
>> +
>> +/**************************************************************************
>> ****/
>> +
>>  /* Macro for defining device addition functions in a compact way */
>>  /* The following functions are defined:
>>   * libxl_device_disk_add
>>   * libxl_device_nic_add
>>   * libxl_device_vtpm_add
>> + * libxl_device_usbctrl_add
>> + * libxl_device_usb_add
>>   */
>>
>>  #define DEFINE_DEVICE_ADD(type)                                         \
>> @@ -4254,6 +4297,12 @@ DEFINE_DEVICE_ADD(nic)
>>  /* vtpm */
>>  DEFINE_DEVICE_ADD(vtpm)
>>
>> +/* usbctrl */
>> +DEFINE_DEVICE_ADD(usbctrl)
>> +
>> +/* usb */
>> +DEFINE_DEVICE_ADD(usb)
>> +
>>  #undef DEFINE_DEVICE_ADD
>>
>>
>> /****************************************************************************
>> **/
>> @@ -6795,6 +6844,10 @@ int libxl_retrieve_domain_configuration(libxl_ctx
>> *ctx, uint32_t domid,
>>
>>      MERGE(pci, pcidevs, COMPARE_PCI, {});
>>
>> +    MERGE(usbctrl, usbctrls, COMPARE_USBCTRL, {});
>> +
>> +    MERGE(usb, usbs, COMPARE_USB, {});
>> +
>>      /* Take care of removable device. We maintain invariant in the
>>       * insert / remove operation so that:
>>       * 1. if xenstore is "empty" while JSON is not, the result
>> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
>> index 615b1de..594e23d 100644
>> --- a/tools/libxl/libxl.h
>> +++ b/tools/libxl/libxl.h
>> @@ -123,6 +123,12 @@
>>  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
>>
>>  /*
>> + * LIBXL_HAVE_PVUSB indicates functions for plugging in USB devices
>> + * through pvusb -- both hotplug and at domain creation time..
>> + */
>> +#define LIBXL_HAVE_PVUSB 1
>> +
>> +/*
>>   * LIBXL_HAVE_BUILDINFO_HVM_VENDOR_DEVICE indicates that the
>>   * libxl_vendor_device field is present in the hvm sections of
>>   * libxl_domain_build_info. This field tells libxl which
>> @@ -1411,6 +1417,74 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid,
>> libxl_device_disk *disk,
>>                         const libxl_asyncop_how *ao_how)
>>                         LIBXL_EXTERNAL_CALLERS_ONLY;
>>
>> +/*
>> + * USB
>> + *
>> + * For each device removed or added, one of these protocols is available:
>> + * - PV (i.e., PVUSB)
>> + * - DEVICEMODEL (i.e, qemu)
>> + *
>> + * PV is available for either PV or HVM domains.  DEVICEMODEL is only
>> + * available for HVM domains.  The caller can additionally specify
>> + * "AUTO", in which case the library will try to determine the best
>> + * protocol automatically.
>> + *
>> + * At the moment, the only protocol implemented is PV.
>> + *
>> + * One can add/remove USB controllers to/from guest, and attach/detach USB
>> + * devices to/from USB controllers.
>> + *
>> + * To add USB controllers and USB devices, one can adding USB controllers
>> + * first and then attaching USB devices to some USB controller, or adding
>> + * USB devices to guest directly, it will automatically create a USB
>> + * controller for USB devices to attach.
>> + *
>> + * To remove USB controllers or USB devices, one can remove USB devices
>> + * under USB controller one by one and then remove USB controller, or
>> + * remove USB controller directly, it will remove all USB devices under
>> + * it automatically.
>> + *
>> + */
>> +/* USB Controllers*/
>> +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,
>> +                             libxl_device_usbctrl *usbctrl,
>> +                             const libxl_asyncop_how *ao_how)
>> +                             LIBXL_EXTERNAL_CALLERS_ONLY;
>> +
>> +int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid,
>> +                                libxl_device_usbctrl *usbctrl,
>> +                                const libxl_asyncop_how *ao_how)
>> +                                LIBXL_EXTERNAL_CALLERS_ONLY;
>> +
>> +int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid,
>> +                                 libxl_device_usbctrl *usbctrl,
>> +                                 const libxl_asyncop_how *ao_how)
>> +                                 LIBXL_EXTERNAL_CALLERS_ONLY;
>> +
>> +libxl_device_usbctrl *libxl_device_usbctrl_list(libxl_ctx *ctx,
>> +                                                uint32_t domid, int *num);
>> +
>> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr);
>> +
>> +
>> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
>> +                                 libxl_device_usbctrl *usbctrl,
>> +                                 libxl_usbctrlinfo *usbctrlinfo);
>> +
>> +/* USB Devices */
>> +int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb
>> *usb,
>> +                         const libxl_asyncop_how *ao_how)
>> +                         LIBXL_EXTERNAL_CALLERS_ONLY;
>> +
>> +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
>> libxl_device_usb *usb,
>> +                            const libxl_asyncop_how *ao_how)
>> +                            LIBXL_EXTERNAL_CALLERS_ONLY;
>> +
>> +libxl_device_usb *
>> +libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, int *num);
>> +
>> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr);
>> +
>>  /* Network Interfaces */
>>  int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic
>> *nic,
>>                           const libxl_asyncop_how *ao_how)
>> diff --git a/tools/libxl/libxl_device.c b/tools/libxl/libxl_device.c
>> index 8bb5e93..6715c16 100644
>> --- a/tools/libxl/libxl_device.c
>> +++ b/tools/libxl/libxl_device.c
>> @@ -676,7 +676,10 @@ void libxl__devices_destroy(libxl__egc *egc,
>> libxl__devices_remove_state *drs)
>>                  aodev->action = LIBXL__DEVICE_ACTION_REMOVE;
>>                  aodev->dev = dev;
>>                  aodev->force = drs->force;
>> -                libxl__initiate_device_remove(egc, aodev);
>> +                if (dev->backend_kind == LIBXL__DEVICE_KIND_VUSB)
>> +                    libxl__initiate_device_usbctrl_remove(egc, aodev);
>> +                else
>> +                    libxl__initiate_device_remove(egc, aodev);
>>              }
>>          }
>>      }
>> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
>> index a3a5cc1..1aa0aaf 100644
>> --- a/tools/libxl/libxl_internal.h
>> +++ b/tools/libxl/libxl_internal.h
>> @@ -2569,6 +2569,14 @@ _hidden void libxl__device_vtpm_add(libxl__egc *egc,
>> uint32_t domid,
>>                                     libxl_device_vtpm *vtpm,
>>                                     libxl__ao_device *aodev);
>>
>> +_hidden void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,
>> +                                       libxl_device_usbctrl *usbctrl,
>> +                                       libxl__ao_device *aodev);
>> +
>> +_hidden void libxl__device_usb_add(libxl__egc *egc, uint32_t domid,
>> +                                   libxl_device_usb *usb,
>> +                                   libxl__ao_device *aodev);
>> +
>>  /* Internal function to connect a vkb device */
>>  _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid,
>>                                    libxl_device_vkb *vkb);
>> @@ -2601,6 +2609,13 @@ _hidden void
>> libxl__wait_device_connection(libxl__egc*,
>>  _hidden void libxl__initiate_device_remove(libxl__egc *egc,
>>                                             libxl__ao_device *aodev);
>>
>> +_hidden int libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid,
>> +                               libxl_device_usbctrl *usbctrl,
>> +                               libxl__device *device);
>> +
>> +_hidden void libxl__initiate_device_usbctrl_remove(libxl__egc *egc,
>> +                                                   libxl__ao_device
>> *aodev);
>> +
>>  /*
>>   * libxl__get_hotplug_script_info returns the args and env that should
>>   * be passed to the hotplug script for the requested device.
>> @@ -3958,6 +3973,9 @@ static inline void libxl__update_config_vtpm(libxl__gc
>> *gc,
>>  #define COMPARE_PCI(a, b) ((a)->func == (b)->func &&    \
>>                             (a)->bus == (b)->bus &&      \
>>                             (a)->dev == (b)->dev)
>> +#define COMPARE_USB(a, b) ((a)->ctrl == (b)->ctrl && \
>> +                           (a)->port == (b)->port)
>> +#define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid)
>>
>>  /* DEVICE_ADD
>>   *
>> diff --git a/tools/libxl/libxl_osdeps.h b/tools/libxl/libxl_osdeps.h
>> index d9661c9..802c762 100644
>> --- a/tools/libxl/libxl_osdeps.h
>> +++ b/tools/libxl/libxl_osdeps.h
>> @@ -24,6 +24,8 @@
>>  #define _GNU_SOURCE
>>
>>  #if defined(__NetBSD__)
>> +#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
>> +#define SYSFS_USBBACK_DRIVER   "/kern/xen/usb"
>>  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
>>  #define SYSFS_PCIBACK_DRIVER   "/kern/xen/pci"
>>  #define NETBACK_NIC_NAME       "xvif%ui%d"
>> @@ -31,6 +33,8 @@
>>  #elif defined(__OpenBSD__)
>>  #include <util.h>
>>  #elif defined(__linux__)
>> +#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
>> +#define SYSFS_USBBACK_DRIVER   "/sys/bus/usb/drivers/usbback"
>>  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
>>  #define SYSFS_PCIBACK_DRIVER   "/sys/bus/pci/drivers/pciback"
>>  #define NETBACK_NIC_NAME       "vif%u.%d"
>> @@ -38,6 +42,8 @@
>>  #elif defined(__sun__)
>>  #include <stropts.h>
>>  #elif defined(__FreeBSD__)
>> +#define SYSFS_USB_DEV          "/dev/null"
>> +#define SYSFS_USBBACK_DRIVER   "/dev/null"
>>  #define SYSFS_PCI_DEV          "/dev/null"
>>  #define SYSFS_PCIBACK_DRIVER   "/dev/null"
>>  #define NETBACK_NIC_NAME       "xnb%u.%d"
>> @@ -45,6 +51,13 @@
>>  #include <sys/endian.h>
>>  #endif
>>
>> +#ifndef SYSFS_USBBACK_DRIVER
>> +#error define SYSFS_USBBACK_DRIVER for your platform
>> +#endif
>> +#ifndef SYSFS_USB_DEV
>> +#error define SYSFS_USB_DEV for your platform
>> +#endif
>> +
>>  #ifndef SYSFS_PCIBACK_DRIVER
>>  #error define SYSFS_PCIBACK_DRIVER for your platform
>>  #endif
>> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
>> new file mode 100644
>> index 0000000..aa1a653
>> --- /dev/null
>> +++ b/tools/libxl/libxl_pvusb.c
>> @@ -0,0 +1,1451 @@
>> +/*
>> + * Copyright (C) 2015 SUSE LINUX Products GmbH, Nuernberg, Germany.
>> + * Author Chunyan Liu <cyliu@suse.com>
>> + *
>> + * This program is free software; you can redistribute it and/or modify
>> + * it under the terms of the GNU Lesser General Public License as published
>> + * by the Free Software Foundation; version 2.1 only. with the special
>> + * exception on linking described in file LICENSE.
>> + *
>> + * This program is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>> + * GNU Lesser General Public License for more details.
>> + */
>> +
>> +#include "libxl_osdeps.h" /* must come before any other headers */
>> +
>> +#include "libxl_internal.h"
>> +#include <inttypes.h>
>> +
>> +#define USBBACK_INFO_PATH "/libxl/usbback"
>> +
>> +#define USBHUB_CLASS_CODE 9
>> +
>> +static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
>> +                                            libxl_device_usbctrl *usbctrl)
>> +{
>> +    int rc;
>> +    libxl_domain_type domtype = libxl__domain_type(gc, domid);
>> +
>> +    if (!usbctrl->version)
>> +        usbctrl->version = 2;
>> +
>> +    if (!usbctrl->ports)
>> +        usbctrl->ports = 8;
>> +
>> +    if (usbctrl->type == LIBXL_USBCTRL_TYPE_AUTO) {
>> +        if (domtype == LIBXL_DOMAIN_TYPE_PV) {
>> +            usbctrl->type = LIBXL_USBCTRL_TYPE_PV;
>> +        } else if (domtype == LIBXL_DOMAIN_TYPE_HVM) {
>> +            /* FIXME: See if we can detect PV frontend */
>> +            usbctrl->type = LIBXL_USBCTRL_TYPE_DEVICEMODEL;
>> +        }
>> +    }
>> +
>> +    rc = libxl__resolve_domid(gc, usbctrl->backend_domname,
>> +                              &usbctrl->backend_domid);
>> +    return rc;
>> +}
>> +
>> +int libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid,
>> +                               libxl_device_usbctrl *usbctrl,
>> +                               libxl__device *device)
>> +{
>> +    device->backend_devid   = usbctrl->devid;
>> +    device->backend_domid   = usbctrl->backend_domid;
>> +    device->backend_kind    = LIBXL__DEVICE_KIND_VUSB;
>> +    device->devid           = usbctrl->devid;
>> +    device->domid           = domid;
>> +    device->kind            = LIBXL__DEVICE_KIND_VUSB;
>> +
>> +    return 0;
>> +}
>> +
>> +/* Add usbctrl information to xenstore.
>> + *
>> + * Adding a usb controller will add a new 'vusb' device in xenstore, and
>> + * add corresponding frontend, backend information to it. According to
>> + * "update_json", decide wether to update json config file.
>> + */
>> +static int libxl__device_usbctrl_add_xenstore(libxl__gc *gc, uint32_t
>> domid,
>> +                                              libxl_device_usbctrl
>> *usbctrl,
>> +                                              bool update_json)
>> +{
>> +    libxl__device *device;
>> +    flexarray_t *front;
>> +    flexarray_t *back;
>> +    xs_transaction_t t = XBT_NULL;
>> +    int i, rc;
>> +    libxl_domain_config d_config;
>> +    libxl_device_usbctrl usbctrl_saved;
>> +    libxl__domain_userdata_lock *lock = NULL;
>> +
>> +    libxl_domain_config_init(&d_config);
>> +    libxl_device_usbctrl_init(&usbctrl_saved);
>> +    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl);
>> +
>> +    GCNEW(device);
>> +    rc = libxl__device_from_usbctrl(gc, domid, usbctrl, device);
>> +    if (rc) goto out;
>> +
>> +    front = flexarray_make(gc, 4, 1);
>> +    back = flexarray_make(gc, 12, 1);
>> +
>> +    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid));
>> +    flexarray_append_pair(back, "online", "1");
>> +    flexarray_append_pair(back, "state", "1");
>> +    flexarray_append_pair(back, "type",
>> +                    (char *)libxl_usbctrl_type_to_string(usbctrl->type));
>> +    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d", usbctrl->version));
>> +    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d", usbctrl->ports));
>> +    flexarray_append_pair(back, "port", "");
>> +    for (i = 0; i < usbctrl->ports; i++)
>> +        flexarray_append_pair(back, GCSPRINTF("port/%d", i + 1), "");
>> +
>> +    flexarray_append_pair(front, "backend-id",
>> +                          GCSPRINTF("%d", usbctrl->backend_domid));
>> +    flexarray_append_pair(front, "state", "1");
>> +
>> +    if (update_json) {
>> +        lock = libxl__lock_domain_userdata(gc, domid);
>> +        if (!lock) {
>> +            rc = ERROR_LOCK_FAIL;
>> +            goto out;
>> +        }
>> +
>> +        rc = libxl__get_domain_configuration(gc, domid, &d_config);
>> +        if (rc) goto out;
>> +
>> +        DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved,
>> +                   COMPARE_USBCTRL, &d_config);
>> +    }
>> +
>> +    for (;;) {
>> +        rc = libxl__xs_transaction_start(gc, &t);
>> +        if (rc) goto out;
>> +
>> +        rc = libxl__device_exists(gc, t, device);
>> +        if (rc < 0) goto out;
>> +        if (rc == 1) {
>> +            /* already exists in xenstore */
>> +            LOG(ERROR, "device already exists in xenstore");
>> +            rc = ERROR_DEVICE_EXISTS;
>> +            goto out;
>> +        }
>> +
>> +        if (update_json) {
>> +            rc = libxl__set_domain_configuration(gc, domid, &d_config);
>> +            if (rc) goto out;
>> +        }
>> +
>> +        libxl__device_generic_add(gc, t, device,
>> +                          libxl__xs_kvs_of_flexarray(gc, back, back->count),
>> +                          libxl__xs_kvs_of_flexarray(gc, front,
>> front->count),
>> +                          NULL);
>> +
>> +        rc = libxl__xs_transaction_commit(gc, &t);
>> +        if (!rc) break;
>> +        if (rc < 0) goto out;
>> +    }
>> +
>> +out:
>> +    libxl__xs_transaction_abort(gc, &t);
>> +    if (lock) libxl__unlock_domain_userdata(lock);
>> +    libxl_device_usbctrl_dispose(&usbctrl_saved);
>> +    libxl_domain_config_dispose(&d_config);
>> +    return rc;
>> +}
>> +
>> +/* AO operation to add a usb controller.
>> + *
>> + * Generally, it does:
>> + * 1) fill in necessary usb controler information with default value
>> + * 2) write usb controller frontend/backend info to xenstore, update json
>> + *    config file if necessary.
>> + * 3) wait for device connection. PVUSB frontend and backend driver will
>> + *    probe xenstore paths and build connection between frontend and
>> backend.
>> + *
>> + * Before calling this function, aodev should be properly filled:
>> + * aodev->ao, aodev->callback, aodev->update_json, ...
>> + */
>> +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,
>> +                               libxl_device_usbctrl *usbctrl,
>> +                               libxl__ao_device *aodev)
>> +{
>> +    STATE_AO_GC(aodev->ao);
>> +    libxl__device *device;
>> +    int rc;
>> +
>> +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
>> +    if (rc < 0) goto out;
>> +
>> +    if (usbctrl->devid == -1) {
>> +        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");
>> +        if (usbctrl->devid < 0) {
>> +            rc = ERROR_FAIL;
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    if (usbctrl->type != LIBXL_USBCTRL_TYPE_PV) {
>> +        LOG(ERROR, "Unsupported USB controller type");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    rc = libxl__device_usbctrl_add_xenstore(gc, domid, usbctrl,
>> +                                            aodev->update_json);
>> +    if (rc) goto out;
>> +
>> +    GCNEW(device);
>> +    rc = libxl__device_from_usbctrl(gc, domid, usbctrl, device);
>> +    if (rc) goto out;
>> +
>> +    aodev->dev = device;
>> +    aodev->action = LIBXL__DEVICE_ACTION_ADD;
>> +    libxl__wait_device_connection(egc, aodev);
>> +    return;
>> +
>> +out:
>> +    aodev->rc = rc;
>> +    if (rc) aodev->callback(egc, aodev);
>> +    return;
>> +}
>> +
>> +static int
>> +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
>> +                                   libxl_devid usbctrl,
>> +                                   libxl_device_usb **usbs, int *num);
>> +
>> +static int
>> +libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb
>> *usb);
>> +
>> +/* AO function to remove a usb controller.
>> + *
>> + * Generally, it does:
>> + * 1) check if the usb controller exists or not
>> + * 2) remove all usb devices under controller
>> + * 3) remove usb controller information from xenstore
>> + *
>> + * Before calling this function, aodev should be properly filled:
>> + * aodev->ao, aodev->dev, aodev->callback, ...
>> + */
>> +void libxl__initiate_device_usbctrl_remove(libxl__egc *egc,
>> +                                           libxl__ao_device *aodev)
>> +{
>> +    STATE_AO_GC(aodev->ao);
>> +    libxl_ctx *ctx = CTX;
>> +    libxl_device_usb *usbs = NULL;
>> +    int numusb = 0;
>> +    int i, rc;
>> +    uint32_t domid = ao->domid;
>> +    int usbctrl_devid = aodev->dev->devid;
>> +    libxl_device_usbctrl usbctrl;
>> +    libxl_usbctrlinfo usbctrlinfo;
>> +
>> +    libxl_device_usbctrl_init(&usbctrl);
>> +    libxl_usbctrlinfo_init(&usbctrlinfo);
>> +    usbctrl.devid = usbctrl_devid;
>> +
>> +    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo);
>> +    if (rc) goto out;
>> +
>> +    if (usbctrlinfo.type != LIBXL_USBCTRL_TYPE_PV) {
>> +        LOG(ERROR, "Unsupported USB controller type");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    /* Remove usb devices first */
>> +    rc  = libxl__device_usb_list_for_usbctrl(gc, domid, usbctrl_devid,
>> +                                             &usbs, &numusb);
>> +    if (rc) goto out;
>> +
>> +    for (i = 0; i < numusb; i++) {
>> +        if (libxl__device_usb_remove(gc, domid, &usbs[i])) {
>> +            LOG(ERROR, "libxl__device_usb_remove failed");
>> +            rc = ERROR_FAIL;
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    libxl_device_usbctrl_dispose(&usbctrl);
>> +    libxl_usbctrlinfo_dispose(&usbctrlinfo);
>> +
>> +    /* Remove usbctrl */
>> +    return libxl__initiate_device_remove(egc, aodev);
>> +
>> +out:
>> +    libxl_device_usbctrl_dispose(&usbctrl);
>> +    libxl_usbctrlinfo_dispose(&usbctrlinfo);
>> +    aodev->rc = rc;
>> +    if (rc) aodev->callback(egc, aodev);
>> +    return;
>> +}
>> +
>> +libxl_device_usbctrl *
>> +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num)
>> +{
>> +    GC_INIT(ctx);
>> +    libxl_device_usbctrl *usbctrls = NULL;
>> +    char *path = NULL;
>> +    char **entry = NULL;
>> +    unsigned int nentries = 0;
>> +
>> +    *num = 0;
>> +
>> +    path = GCSPRINTF("%s/device/vusb",
>> +                     libxl__xs_get_dompath(gc, domid));
>> +    entry = libxl__xs_directory(gc, XBT_NULL, path, &nentries);
>> +
>> +    if (entry && nentries) {
>> +        usbctrls = libxl__zalloc(NOGC, sizeof(*usbctrls) * nentries);
>> +        libxl_device_usbctrl *usbctrl;
>> +        libxl_device_usbctrl *end = usbctrls + nentries;
>> +        for (usbctrl = usbctrls; usbctrl < end;
>> +             usbctrl++, entry++, (*num)++) {
>> +            const char *tmp, *be_path;
>> +            int rc;
>> +            const char *fe_path = GCSPRINTF("%s/%s", path, *entry);
>> +
>> +            libxl_device_usbctrl_init(usbctrl);
>> +            usbctrl->devid = atoi(*entry);
>> +
>> +#define READ_SUBPATH(path, subpath) ({                                  \
>> +        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \
>> +                                    GCSPRINTF("%s/" subpath, path),     \
>> +                                    &tmp);                              \
>> +        if (rc) goto outerr;                                            \
>> +        (char *)tmp;                                                    \
>> +    })
>> +            be_path = READ_SUBPATH(fe_path, "backend");
>> +            usbctrl->backend_domid = atoi(READ_SUBPATH(fe_path,
>> "backend-id"));
>> +            usbctrl->version = atoi(READ_SUBPATH(be_path, "usb-ver"));
>> +            usbctrl->ports = atoi(READ_SUBPATH(be_path, "num-ports"));
>> +            libxl_usbctrl_type_from_string(READ_SUBPATH(be_path, "type"),
>> +                                           &usbctrl->type);
>> +#undef READ_SUBPATH
>> +       }
>> +    }
>> +
>> +    goto out;
>> +
>> +outerr:
>> +    LOG(ERROR, "Unable to list USB Controllers");
>> +    libxl_device_usbctrl_list_free(usbctrls, *num);
>> +    *num = 0;
>> +    usbctrls = NULL;
>> +
>> +out:
>> +    GC_FREE;
>> +    return usbctrls;
>> +}
>> +
>> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
>> +                                libxl_device_usbctrl *usbctrl,
>> +                                libxl_usbctrlinfo *usbctrlinfo)
>> +{
>> +    GC_INIT(ctx);
>> +    const char *dompath, *fe_path, *be_path, *tmp;
>> +    int rc;
>> +
>> +    usbctrlinfo->devid = usbctrl->devid;
>> +
>> +#define READ_SUBPATH(path, subpath) ({                                  \
>> +        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \
>> +                                    GCSPRINTF("%s/" subpath, path),     \
>> +                                    &tmp);                              \
>> +        if (rc) goto out;                                               \
>> +        (char *)tmp;                                                    \
>> +    })
>> +
>> +    dompath = libxl__xs_get_dompath(gc, domid);
>> +    fe_path = GCSPRINTF("%s/device/vusb/%d", dompath, usbctrl->devid);
>> +    be_path = READ_SUBPATH(fe_path, "backend");
>> +
>> +    usbctrlinfo->backend = libxl__strdup(NOGC, be_path);
>> +
>> +    tmp = READ_SUBPATH(fe_path, "backend-id");
>> +    usbctrlinfo->backend_id = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(fe_path, "state");
>> +    usbctrlinfo->state = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(fe_path, "event-channel");
>> +    usbctrlinfo->evtch = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(fe_path, "urb-ring-ref");
>> +    usbctrlinfo->ref_urb = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(fe_path, "conn-ring-ref");
>> +    usbctrlinfo->ref_conn = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(be_path, "frontend");
>> +    usbctrlinfo->frontend = libxl__strdup(NOGC, tmp);
>> +    tmp = READ_SUBPATH(be_path, "frontend-id");
>> +    usbctrlinfo->frontend_id = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(be_path, "num-ports");
>> +    usbctrlinfo->ports = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(be_path, "usb-ver");
>> +    usbctrlinfo->version = tmp ? strtoul(tmp, NULL, 10) : -1;
>> +    tmp = READ_SUBPATH(be_path, "type");
>> +    libxl_usbctrl_type_from_string(tmp, &usbctrlinfo->type);
>> +
>> +#undef READ_SUBPATH
>> +
>> +    rc = 0;
>> +
>> +out:
>> +    GC_FREE;
>> +    return rc;
>> +}
>> +
>> +int libxl_devid_to_device_usbctrl(libxl_ctx *ctx,
>> +                                  uint32_t domid,
>> +                                  int devid,
>> +                                  libxl_device_usbctrl *usbctrl)
>> +{
>> +    libxl_device_usbctrl *usbctrls;
>> +    int nb = 0;
>> +    int i, rc;
>> +
>> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &nb);
>> +    if (!usbctrls)
>> +        return ERROR_FAIL;
>> +
>> +    rc = ERROR_FAIL;
>> +    for (i = 0; i < nb; i++) {
>> +        if (devid == usbctrls[i].devid) {
>> +            libxl_device_usbctrl_copy(ctx, usbctrl, &usbctrls[i]);
>> +            rc = 0;
>> +            break;
>> +        }
>> +    }
>> +
>> +    libxl_device_usbctrl_list_free(usbctrls, nb);
>> +    return rc;
>> +}
>> +
>> +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
>> +{
>> +    struct dirent *de;
>> +    DIR *dir;
>> +    char *busid = NULL;
>> +
>> +    /* invalid hostbus or hostaddr */
>> +    if (bus < 1 || addr < 1)
>> +        return NULL;
>> +
>> +    if (!(dir = opendir(SYSFS_USB_DEV)))
>> +        return NULL;
>> +
>> +    while ((de = readdir(dir))) {
>> +        char *filename;
>> +        void *buf;
>> +        int busnum = -1;
>> +        int devnum = -1;
>> +
>> +        if (!de->d_name)
>> +            continue;
>> +
>> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
>> +        if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
>> +            devnum = atoi((char *)buf);
>> +
>> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
>> +        if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
>> +            busnum = atoi((char *)buf);
>> +
>> +        if (bus == busnum && addr == devnum) {
>> +            busid = libxl__strdup(gc, de->d_name);
>> +            break;
>> +        }
>> +    }
>> +
>> +    closedir(dir);
>> +    return busid;
>> +}
>> +
>> +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid,
>> +                                  uint8_t *bus, uint8_t *addr)
>> +{
>> +    char *filename;
>> +    void *buf;
>> +
>> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
>> +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
>> +        *bus = atoi((char *)buf);
>> +    else
>> +        return ERROR_FAIL;
>> +
>> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", busid);
>> +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
>> +        *addr = atoi((char *)buf);
>> +    else
>> +        return ERROR_FAIL;
>> +
>> +    return 0;
>> +}
>> +
>> +static int
>> +get_assigned_devices(libxl__gc *gc,
>> +                     libxl_device_usb **list, int *num)
>> +{
>> +    char **domlist;
>> +    unsigned int nd = 0, i, j, k;
>> +    int rc;
>> +
>> +    *list = NULL;
>> +    *num = 0;
>> +
>> +    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd);
>> +    for (i = 0; i < nd; i++) {
>> +        char *path, **ctrl_list;
>> +        unsigned int nc = 0;
>> +
>> +        path = GCSPRINTF("/local/domain/%s/device/vusb", domlist[i]);
>> +        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path, &nc);
>> +
>> +        for (j = 0; j < nc; j++) {
>> +            const char *be_path, *num_ports;
>> +            int nport;
>> +
>> +            rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                          GCSPRINTF("%s/%s/backend", path, ctrl_list[j]),
>> +                          &be_path);
>> +            if (rc) goto out;
>> +
>> +            rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                        GCSPRINTF("%s/num-ports", be_path),
>> +                                        &num_ports);
>> +            if (rc) goto out;
>> +
>> +            nport = atoi(num_ports);
>> +            for (k = 0; k < nport; k++) {
>> +                libxl_device_usb *usb;
>> +                const char *portpath, *busid;
>> +
>> +                portpath = GCSPRINTF("%s/port/%d", be_path, k + 1);
>> +                busid = libxl__xs_read(gc, XBT_NULL, portpath);
>> +                /* If there is USB device attached, add it to list */
>> +                if (busid && strcmp(busid, "")) {
>> +                    GCREALLOC_ARRAY(*list, *num + 1);
>> +                    usb = *list + *num;
>> +                    (*num)++;
>> +                    libxl_device_usb_init(usb);
>> +                    usb->ctrl = atoi(ctrl_list[j]);
>> +                    usb->port = k + 1;
>> +                    rc = usb_busaddr_from_busid(gc, busid,
>> +                                                &usb->u.hostdev.hostbus,
>> +                                                &usb->u.hostdev.hostaddr);
>> +                    if (rc) goto out;
>> +                }
>> +            }
>> +        }
>> +    }
>> +
>> +    rc = 0;
>> +
>> +out:
>> +    if (rc) {
>> +        *list = NULL;
>> +        *num = 0;
>> +    }
>> +    return rc;
>> +}
>> +
>> +static bool is_usbdev_in_array(libxl_device_usb *usbs, int num,
>> +                               libxl_device_usb *usb)
>> +{
>> +    int i;
>> +
>> +    for (i = 0; i < num; i++) {
>> +        if (usbs[i].u.hostdev.hostbus == usb->u.hostdev.hostbus &&
>> +            usbs[i].u.hostdev.hostaddr == usb->u.hostdev.hostaddr)
>> +            return true;
>> +    }
>> +
>> +    return false;
>> +}
>> +
>> +/* check if USB device is already assigned to a domain */
>> +/* check if USB device type is assignable */
>> +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
>> +{
>> +    int classcode;
>> +    char *filename;
>> +    void *buf = NULL;
>> +    char *busid = NULL;
>> +
>> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
>> +                                 usb->u.hostdev.hostaddr);
>> +    if (!busid) return false;
>> +
>> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid);
>> +    if (libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
>> +        return false;
>> +
>> +    classcode = atoi(buf);
>> +    return classcode != USBHUB_CLASS_CODE;
>> +}
>> +
>> +/* get usb devices under certain usb controller */
>> +static int
>> +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
>> +                                   libxl_devid usbctrl,
>> +                                   libxl_device_usb **usbs, int *num)
>> +{
>> +    const char *fe_path, *be_path, *num_devs;
>> +    int n, i, rc;
>> +
>> +    *usbs = NULL;
>> +    *num = 0;
>> +
>> +    fe_path = GCSPRINTF("%s/device/vusb/%d",
>> +                        libxl__xs_get_dompath(gc, domid), usbctrl);
>> +
>> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                GCSPRINTF("%s/backend", fe_path),
>> +                                &be_path);
>> +    if (rc) return rc;
>> +
>> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                GCSPRINTF("%s/num-ports", be_path),
>> +                                &num_devs);
>> +    if (rc) return rc;
>> +
>> +    n = atoi(num_devs);
>> +
>> +    for (i = 0; i < n; i++) {
>> +        char *busid;
>> +        libxl_device_usb *usb;
>> +
>> +        busid = libxl__xs_read(gc, XBT_NULL,
>> +                               GCSPRINTF("%s/port/%d", be_path, i + 1));
>> +        if (busid && strcmp(busid, "")) {
>> +            GCREALLOC_ARRAY(*usbs, *num + 1);
>> +            usb = *usbs + *num;
>> +            (*num)++;
>> +            libxl_device_usb_init(usb);
>> +            usb->ctrl = usbctrl;
>> +            usb->port = i + 1;
>> +            rc = usb_busaddr_from_busid(gc, busid,
>> +                                        &usb->u.hostdev.hostbus,
>> +                                        &usb->u.hostdev.hostaddr);
>> +            if (rc) goto out;
>> +        }
>> +    }
>> +
>> +    rc = 0;
>> +
>> +out:
>> +    if (rc) {
>> +        *usbs = NULL;
>> +        *num = 0;
>> +    }
>> +    return rc;
>> +}
>> +
>> +/* get all usb devices of the domain */
>> +libxl_device_usb *
>> +libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, int *num)
>> +{
>> +    GC_INIT(ctx);
>> +    char *path;
>> +    char **usbctrls;
>> +    unsigned int nc = 0;
>> +    int i, j;
>> +    libxl_device_usb *usbs = NULL;
>> +
>> +    *num = 0;
>> +
>> +    path = GCSPRINTF("%s/device/vusb",
>> +                        libxl__xs_get_dompath(gc, domid));
>> +    usbctrls = libxl__xs_directory(gc, XBT_NULL, path, &nc);
>> +
>> +    for (i = 0; i < nc; i++) {
>> +        int nd = 0;
>> +        libxl_device_usb *tmp = NULL;
>> +        libxl__device_usb_list_for_usbctrl(gc, domid,
>> +                                           atoi(usbctrls[i]), &tmp, &nd);
>> +        if (!nd) continue;
>> +
>> +        usbs = libxl__realloc(NOGC, usbs, sizeof(*usbs) * (*num + nd));
>> +        for (j = 0; j < nd; j++) {
>> +            libxl_device_usb_copy(ctx, usbs + *num, tmp + j);
>> +            (*num)++;
>> +        }
>> +    }
>> +
>> +    GC_FREE;
>> +    return usbs;
>> +}
>> +
>> +/* find first unused controller:port and give that to usb device */
>> +static int
>> +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
>> +                                      libxl_device_usb *usb)
>> +{
>> +    libxl_ctx *ctx = CTX;
>> +    libxl_device_usbctrl *usbctrls = NULL;
>> +    int numctrl = 0;
>> +    int i, j, rc;
>> +
>> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
>> +    if (!numctrl) {
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    rc = ERROR_FAIL;
>> +    for (i = 0; i < numctrl; i++) {
>> +        for (j = 0; j < usbctrls[i].ports; j++) {
>> +            char *path, *tmp;
>> +
>> +            path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
>> +                             libxl__xs_get_dompath(gc,
>> LIBXL_TOOLSTACK_DOMID),
>> +                             domid, usbctrls[i].devid, j + 1);
>> +            tmp = libxl__xs_read(gc, XBT_NULL, path);
>> +            if (tmp && !strcmp(tmp, "")) {
>> +                usb->ctrl = usbctrls[i].devid;
>> +                usb->port = j + 1;
>> +                rc = 0;
>> +                goto out;
>> +            }
>> +        }
>> +    }
>> +
>> +out:
>> +    libxl_device_usbctrl_list_free(usbctrls, numctrl);
>> +    return rc;
>> +}
>> +
>> +/* Fill in usb information with default value.
>> + *
>> + * Generally, it does:
>> + * 1) if "controller" is not specified:
>> + *    - if "port" is not specified, try to find an available
>> controller:port,
>> + *      if found, use that; otherwise, create a new controller, use this
>> + *      controller and its first port
>> + *    - if "port" is specified, report error.
>> + * 2) if "controller" is specified, but port is not specified:
>> + *    try to find an available port under this controller, if found, use
>> + *    that, otherwise, report error.
>> + * 3) if both "controller" and "port" are specified:
>> + *    check the controller:port is available, if not, report error.
>> + */
>> +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
>> +                                        libxl_device_usb *usb,
>> +                                        bool update_json)
>> +{
>> +    int rc;
>> +
>> +    if (!usb->devtype)
>> +        usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV;
>> +
>> +    if (usb->ctrl == -1) {
>> +        if (usb->port) {
>> +            LOG(ERROR, "USB controller must be specified if you specify
>> port");
>> +            return ERROR_INVAL;
>> +        }
>> +
>> +        rc = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
>> +        /* If no existing controller to host this usb device, add a new one
>> */
>> +        if (rc) {
>> +            libxl_device_usbctrl *usbctrl;
>> +
>> +            GCNEW(usbctrl);
>> +            libxl_device_usbctrl_init(usbctrl);
>> +            rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
>> +            if (rc < 0) goto out;
>> +
>> +            if (usbctrl->devid == -1) {
>> +                usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");
>> +                if (usbctrl->devid < 0) {
>> +                    rc = ERROR_FAIL;
>> +                    goto out;
>> +                }
>> +            }
>> +
>> +            rc = libxl__device_usbctrl_add_xenstore(gc, domid, usbctrl,
>> +                                                    update_json);
>> +            if (rc) goto out;
>> +
>> +            usb->ctrl = usbctrl->devid;
>> +            usb->port = 1;
>> +        }
>> +    } else if (!usb->port) {
>> +        /* Valid port starts from 1. Choose port for us. */
>> +        int i, ports;
>> +        const char *fe_path, *be_path, *tmp;
>> +
>> +        fe_path = GCSPRINTF("%s/device/vusb/%d",
>> +                         libxl__xs_get_dompath(gc, domid), usb->ctrl);
>> +
>> +        rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                    GCSPRINTF("%s/backend", fe_path),
>> &be_path);
>> +        if (rc) goto out;
>> +
>> +        rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                    GCSPRINTF("%s/num-ports", be_path),
>> &tmp);
>> +        if (rc) goto out;
>> +
>> +        ports = atoi(tmp);
>> +
>> +        for (i = 0; i < ports; i++) {
>> +            tmp = libxl__xs_read(gc, XBT_NULL,
>> +                                 GCSPRINTF("%s/port/%d", be_path, i + 1));
>> +            if (tmp && !strcmp(tmp, "")) {
>> +                usb->port = i + 1;
>> +                break;
>> +            }
>> +        }
>> +
>> +        if (!usb->port) {
>> +            LOG(ERROR, "No available port under specified controller");
>> +            rc = ERROR_FAIL;
>> +            goto out;
>> +        }
>> +    } else {
>> +        const char *fe_path, *be_path, *tmp;
>> +
>> +        fe_path = GCSPRINTF("%s/device/vusb/%d",
>> +                         libxl__xs_get_dompath(gc, domid), usb->ctrl);
>> +
>> +        rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                    GCSPRINTF("%s/backend", fe_path),
>> &be_path);
>> +        if (rc) goto out;
>> +
>> +        rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                                    GCSPRINTF("%s/port/%d", be_path,
>> usb->port),
>> +                                    &tmp);
>> +        if (rc) goto out;
>> +
>> +        if (strcmp(tmp, "")) {
>> +            LOG(ERROR, "The controller port isn't available");
>> +            rc = ERROR_FAIL;
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    rc = 0;
>> +
>> +out:
>> +    return rc;
>> +}
>> +
>> +/* Add usb information to xenstore
>> + *
>> + * Adding a usb device won't create new 'vusb' device, but only write
>> + * the device busid to the controller:port in xenstore.
>> + */
>> +static int libxl__device_usb_add_xenstore(libxl__gc *gc, uint32_t domid,
>> +                                          libxl_device_usb *usb,
>> +                                          bool update_json)
>> +{
>> +    char *be_path;
>> +    char *busid;
>> +    int rc;
>> +    xs_transaction_t t = XBT_NULL;
>> +    libxl_domain_config d_config;
>> +    libxl_device_usb usb_saved;
>> +    libxl__domain_userdata_lock *lock = NULL;
>> +
>> +    libxl_domain_config_init(&d_config);
>> +    libxl_device_usb_init(&usb_saved);
>> +    libxl_device_usb_copy(CTX, &usb_saved, usb);
>> +
>> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
>> +                                 usb->u.hostdev.hostaddr);
>> +    if (!busid) {
>> +        LOG(DEBUG, "Fail to get busid of usb device");
>> +        goto out;
>> +    }
>> +
>> +    if (update_json) {
>> +        lock = libxl__lock_domain_userdata(gc, domid);
>> +        if (!lock) {
>> +            rc = ERROR_LOCK_FAIL;
>> +            goto out;
>> +        }
>> +
>> +        rc = libxl__get_domain_configuration(gc, domid, &d_config);
>> +        if (rc) goto out;
>> +
>> +        DEVICE_ADD(usb, usbs, domid, &usb_saved, COMPARE_USB, &d_config);
>> +    }
>> +
>> +    for (;;) {
>> +        rc = libxl__xs_transaction_start(gc, &t);
>> +        if (rc) goto out;
>> +
>> +        if (update_json) {
>> +            rc = libxl__set_domain_configuration(gc, domid, &d_config);
>> +            if (rc) goto out;
>> +        }
>> +
>> +        be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
>> +                            libxl__xs_get_dompath(gc,
>> LIBXL_TOOLSTACK_DOMID),
>> +                            domid, usb->ctrl, usb->port);
>> +
>> +        LOG(DEBUG, "Adding new usb device to xenstore");
>> +        if (libxl__xs_write_checked(gc, t, be_path, busid))
>> +            goto out;
>> +
>> +        rc = libxl__xs_transaction_commit(gc, &t);
>> +        if (!rc) break;
>> +        if (rc < 0) goto out;
>> +    }
>> +
>> +    rc = 0;
>> +
>> +out:
>> +    if (lock) libxl__unlock_domain_userdata(lock);
>> +    libxl_device_usb_dispose(&usb_saved);
>> +    libxl_domain_config_dispose(&d_config);
>> +    return rc;
>> +}
>> +
>> +static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t domid,
>> +                                             libxl_device_usb *usb)
>> +{
>> +    char *be_path;
>> +
>> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
>> +                        libxl__xs_get_dompath(gc, LIBXL_TOOLSTACK_DOMID),
>> +                        domid, usb->ctrl, usb->port);
>> +    LOG(DEBUG, "Removing USB device from xenstore");
>> +    if (libxl__xs_write_checked(gc, XBT_NULL, be_path, ""))
>> +        return ERROR_FAIL;
>> +
>> +    return 0;
>> +}
>> +
>> +/* bind/unbind usb device interface */
>> +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)
>> +{
>> +    char *path, *spath, *dp = NULL;
>> +    int fd = -1;
>> +    int rc;
>> +    struct stat st;
>> +
>> +    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf);
>> +    if (!lstat(spath, &st)) {
>> +        /* Find the canonical path to the driver. */
>> +        dp = libxl__zalloc(gc, PATH_MAX);
>> +        dp = realpath(spath, dp);
>> +
>> +        path = GCSPRINTF("%s/unbind", spath);
>> +        fd = open(path, O_WRONLY);
>> +        if (fd < 0)
>> +            return ERROR_FAIL;
>> +        rc = write(fd, intf, strlen(intf));
>> +        close(fd);
>> +        if (rc < 0)
>> +            return ERROR_FAIL;
>> +    }
>> +
>> +    if (drvpath)
>> +        *drvpath = dp;
>> +
>> +    return 0;
>> +}
>> +
>> +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath)
>> +{
>> +    char *path;
>> +    struct stat st;
>> +    int fd, rc;
>> +
>> +    path = GCSPRINTF("%s/%s", drvpath, intf);
>> +    rc = lstat(path, &st);
>> +    /* already bind, return */
>> +    if (!rc)
>> +        return 0;
>> +
>> +    path = GCSPRINTF("%s/bind", drvpath);
>> +    fd = open(path, O_WRONLY);
>> +    if (fd < 0)
>> +        return ERROR_FAIL;
>> +
>> +    rc = write(fd, intf, strlen(intf));
>> +    close(fd);
>> +    if (rc < 0)
>> +        return ERROR_FAIL;
>> +
>> +    return 0;
>> +}
>> +
>> +/* Is usb interface bound to usbback? */
>> +static int usb_intf_is_assigned(libxl__gc *gc, char *intf)
>> +{
>> +    char *spath;
>> +    int rc;
>> +    struct stat st;
>> +
>> +    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf);
>> +    rc = lstat(spath, &st);
>> +
>> +    if (rc == 0)
>> +        return 1;
>> +    if (rc < 0 && errno == ENOENT)
>> +        return 0;
>> +    LOGE(ERROR, "Accessing %s", spath);
>> +    return -1;
>> +}
>> +
>> +static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb,
>> +                                  char ***intfs, int *num)
>> +{
>> +    DIR *dir;
>> +    struct dirent *entry;
>> +    char *buf;
>> +    char *busid;
>> +    int rc;
>> +
>> +    *intfs = NULL;
>> +    *num = 0;
>> +
>> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
>> +                                 usb->u.hostdev.hostaddr);
>> +    if (!busid) {
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    buf = GCSPRINTF("%s:", busid);
>> +
>> +    if (!(dir = opendir(SYSFS_USB_DEV))) {
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    while ((entry = readdir(dir)) != NULL) {
>> +        if (!strncmp(entry->d_name, buf, strlen(buf))) {
>> +            GCREALLOC_ARRAY(*intfs, *num + 1);
>> +            if (*intfs == NULL) {
>> +                rc = ERROR_FAIL;
>> +                goto out;
>> +            }
>> +            (*intfs)[*num] = libxl__strdup(gc, entry->d_name);
>> +            (*num)++;
>> +        }
>> +    }
>> +
>> +    closedir(dir);
>> +    rc = 0;
>> +
>> +out:
>> +    return rc;
>> +}
>> +
>> +/* Encode usb interface so that it could be written to xenstore as a key.
>> + *
>> + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_',
>> + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1.
>> + * This will be used to save original driver of USB device to xenstore.
>> + */
>> +static char *usb_interface_xenstore_encode(char *busid)
>> +{
>> +    char *str = strdup(busid);
>> +    int i, len = strlen(str);
>> +
>> +    for (i = 0; i < len; i++) {
>> +        if (str[i] == '.')
>> +            str[i] = '_';
>> +         if (str[i] == ':')
>> +            str[i] = '-';
>> +    }
>> +    return str;
>> +}
>> +
>> +/* Unbind USB device from "usbback" driver.
>> + *
>> + * If there are many interfaces under USB device, check each interface,
>> + * unbind from "usbback" driver and rebind to its original driver.
>> + */
>> +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
>> +{
>> +    char **intfs = NULL;
>> +    char *path;
>> +    int num = 0, i;
>> +    int rc;
>> +    char *busid;
>> +    char *usb_encode = NULL;
>> +
>> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
>> +        return ERROR_FAIL;
>> +
>> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
>> +                                 usb->u.hostdev.hostaddr);
>> +    usb_encode = usb_interface_xenstore_encode(busid);
>> +
>> +    for (i = 0; i < num; i++) {
>> +        char *intf = intfs[i];
>> +        char *drvpath = NULL;
>> +
>> +        /* check if the USB interface is already bound to "usbbcak" */
>> +        if (usb_intf_is_assigned(gc, intf) > 0) {
>> +            /* unbind interface from usbback driver */
>> +            if (unbind_usb_intf(gc, intf, NULL) < 0) {
>> +                rc = ERROR_FAIL;
>> +                goto out;
>> +            }
>> +        }
>> +
>> +        /* bind interface to its originial driver */
>> +        drvpath = libxl__xs_read(gc, XBT_NULL,
>> +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
>> +                  usb_encode, usb_interface_xenstore_encode(intf)));
>> +        if (drvpath && bind_usb_intf(gc, intf, drvpath))
>> +            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath);
>> +    }
>> +
>> +    /* finally, remove xs driver path */
>> +    path = GCSPRINTF(USBBACK_INFO_PATH"/%s", usb_encode);
>> +    libxl__xs_rm_checked(gc, XBT_NULL, path);
>> +    rc = 0;
>> +
>> +out:
>> +    free(usb_encode);
>> +    return rc;
>> +}
>> +
>> +/* Bind USB device to "usbback" driver.
>> + *
>> + * If there are many interfaces under USB device, check each interface,
>> + * unbind from original driver and bind to "usbback" driver.
>> + */
>> +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
>> +{
>> +    char **intfs = NULL;
>> +    int num = 0, i;
>> +    int rc;
>> +    char *busid;
>> +    char *usb_encode = NULL;
>> +
>> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
>> +        return ERROR_FAIL;
>> +
>> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
>> +                                 usb->u.hostdev.hostaddr);
>> +    usb_encode = usb_interface_xenstore_encode(busid);
>> +
>> +    for (i = 0; i < num; i++) {
>> +        char *intf = intfs[i];
>> +        char *path = NULL;
>> +        char *drvpath = NULL;
>> +
>> +        /* already assigned to usbback */
>> +        if (usb_intf_is_assigned(gc, intf) > 0)
>> +            continue;
>> +
>> +        /* unbind interface from original driver */
>> +        if (unbind_usb_intf(gc, intf, &drvpath) < 0) {
>> +            rc = ERROR_FAIL;
>> +            goto out_rebind;
>> +        }
>> +
>> +        if (drvpath) {
>> +            /* write driver path to xenstore for later rebinding */
>> +            path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
>> +                             usb_encode,
>> usb_interface_xenstore_encode(intf));
>> +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
>> +                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
>> +            }
>> +        }
>> +
>> +        /* bind interface to usbback */
>> +        if (bind_usb_intf(gc, intf, SYSFS_USBBACK_DRIVER) < 0) {
>> +            LOGE(ERROR, "Couldn't bind %s to %s", intf,
>> SYSFS_USBBACK_DRIVER);
>> +            rc = ERROR_FAIL;
>> +            goto out_rebind;
>> +        }
>> +    }
>> +
>> +    rc = 0;
>> +    goto out;
>> +
>> +out_rebind:
>> +    /* some interfaces might be bound to usbback, unbind it then and
>> +     * rebind to its original driver
>> +     */
>> +    usbback_dev_unassign(gc, usb);
>> +out:
>> +    free(usb_encode);
>> +    return rc;
>> +}
>> +
>> +/*
>> + * USB add
>> + */
>> +static int do_usb_add(libxl__gc *gc, uint32_t domid,
>> +                      libxl_device_usb *usbdev,
>> +                      libxl__ao_device *aodev)
>> +{
>> +    libxl_ctx *ctx = CTX;
>> +    libxl_usbctrlinfo usbctrlinfo;
>> +    libxl_device_usbctrl usbctrl;
>> +    int rc;
>> +
>> +    libxl_device_usbctrl_init(&usbctrl);
>> +    libxl_usbctrlinfo_init(&usbctrlinfo);
>> +    usbctrl.devid = usbdev->ctrl;
>> +    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo);
>> +    if (rc)
>> +        goto out;
>> +
>> +    switch (usbctrlinfo.type) {
>> +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:
>> +        LOG(ERROR, "Not supported");
>> +        break;
>> +    case LIBXL_USBCTRL_TYPE_PV:
>> +        rc = libxl__device_usb_add_xenstore(gc, domid, usbdev,
>> +                                            aodev->update_json);
>> +        if (rc) goto out;
>> +
>> +        rc = usbback_dev_assign(gc, usbdev);
>> +        if (rc) {
>> +            libxl__device_usb_remove_xenstore(gc, domid, usbdev);
>> +            goto out;
>> +        }
>> +        break;
>> +    default:
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +out:
>> +    libxl_device_usbctrl_dispose(&usbctrl);
>> +    libxl_usbctrlinfo_dispose(&usbctrlinfo);
>> +    return rc;
>> +}
>> +
>> +/* AO operation to add a usb device.
>> + *
>> + * Generally, it does:
>> + * 1) check if the usb device type is assignable
>> + * 2) check if the usb device is already assigned to a domain
>> + * 3) add 'busid' of the usb device to xenstore contoller/port/.
>> + *    (PVUSB driver watches the xenstore changes and will detect that.)
>> + * 4) unbind usb device from original driver and bind to usbback.
>> + *    If usb device has many interfaces, then:
>> + *    - unbind each interface from its original driver and bind to usbback.
>> + *    - store the original driver to xenstore for later rebinding when
>> + *      detaching the device.
>> + *
>> + * Before calling this function, aodev should be properly filled:
>> + * aodev->ao, aodev->callback, aodev->update_json, ...
>> + */
>> +void libxl__device_usb_add(libxl__egc *egc, uint32_t domid,
>> +                           libxl_device_usb *usb,
>> +                           libxl__ao_device *aodev)
>> +{
>> +    STATE_AO_GC(aodev->ao);
>> +    int rc;
>> +    char *busid = NULL;
>> +    libxl_device_usb *assigned;
>> +    int num_assigned;
>> +    libxl_device_usbctrl usbctrl;
>> +    libxl_usbctrlinfo usbctrlinfo;
>> +
>> +    libxl_device_usbctrl_init(&usbctrl);
>> +    libxl_usbctrlinfo_init(&usbctrlinfo);
>> +
>> +    /* Currently only support adding USB device from Dom0 backend.
>> +     * So, if USB controller is specified, check backend domain, if
>> +     * it is not Dom0, report error as not supported.
>> +     *
>> +     * -1 is the default value of libxl_device_usbctrl.
>> +     */
>> +    if (usb->ctrl != -1) {
>> +        usbctrl.devid = usb->ctrl;
>> +        rc = libxl_device_usbctrl_getinfo(CTX, domid, &usbctrl,
>> &usbctrlinfo);
>> +        if (rc) goto out;
>> +
>> +        if (usbctrlinfo.backend_id != 0) {
>> +            LOG(ERROR, "Don't support adding USB device from non-Dom0
>> backend");
>> +            rc = ERROR_INVAL;
>> +            goto out;
>> +        }
>> +    }
>> +
>> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
>> +                                 usb->u.hostdev.hostaddr);
>> +    if (!busid) {
>> +        LOG(ERROR, "USB device doesn't exist in sysfs");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    if (!is_usb_assignable(gc, usb)) {
>> +        LOG(ERROR, "USB device is not assignable.");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    /* check usb device is already assigned */
>> +    rc = get_assigned_devices(gc, &assigned, &num_assigned);
>> +    if (rc) {
>> +        LOG(ERROR, "cannot determine if device is assigned,"
>> +                   " refusing to continue");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    if (is_usbdev_in_array(assigned, num_assigned, usb)) {
>> +        LOG(ERROR, "USB device already attached to a domain");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    rc = libxl__device_usb_setdefault(gc, domid, usb, aodev->update_json);
>> +    if (rc) goto out;
>> +
>> +    /* Do the add */
>> +    if (do_usb_add(gc, domid, usb, aodev)) {
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    libxl__ao_complete(egc, ao, 0);
>> +    rc = 0;
>> +
>> +out:
>> +    libxl_device_usbctrl_dispose(&usbctrl);
>> +    libxl_usbctrlinfo_dispose(&usbctrlinfo);
>> +    aodev->rc = rc;
>> +    if (rc) aodev->callback(egc, aodev);
>> +    return;
>> +}
>> +
>> +static int do_usb_remove(libxl__gc *gc, uint32_t domid,
>> +                         libxl_device_usb *usbdev)
>> +{
>> +    libxl_ctx *ctx = CTX;
>> +    libxl_usbctrlinfo usbctrlinfo;
>> +    libxl_device_usbctrl usbctrl;
>> +    int rc;
>> +
>> +    libxl_device_usbctrl_init(&usbctrl);
>> +    libxl_usbctrlinfo_init(&usbctrlinfo);
>> +    usbctrl.devid = usbdev->ctrl;
>> +
>> +    rc = libxl_device_usbctrl_getinfo(ctx, domid, &usbctrl, &usbctrlinfo);
>> +    if (rc) goto out;
>> +
>> +    if (usbctrlinfo.backend_id != 0) {
>> +        LOG(ERROR, "Don't support removing USB device from non-Dom0
>> backend");
>> +        rc = ERROR_INVAL;
>> +        goto out;
>> +    }
>> +
>> +    switch (usbctrlinfo.type) {
>> +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:
>> +        LOG(ERROR, "Not supported");
>> +        break;
>> +    case LIBXL_USBCTRL_TYPE_PV:
>> +        rc = libxl__device_usb_remove_xenstore(gc, domid, usbdev);
>> +        if (rc) goto out;
>> +
>> +        usbback_dev_unassign(gc, usbdev);
>> +        break;
>> +    default:
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    rc = 0;
>> +
>> +out:
>> +    libxl_device_usbctrl_dispose(&usbctrl);
>> +    libxl_usbctrlinfo_dispose(&usbctrlinfo);
>> +    return rc;
>> +}
>> +
>> +/* Operation to remove usb device.
>> + *
>> + * Generally, it does:
>> + * 1) check if the usb device is assigned to the domain
>> + * 2) remove the usb device from xenstore controller/port.
>> + * 3) unbind usb device from usbback and rebind to its original driver.
>> + *    If usb device has many interfaces, do it to each interface.
>> + */
>> +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
>> +                                    libxl_device_usb *usb)
>> +{
>> +    int rc;
>> +
>> +    if (usb->ctrl < 0 || usb->port < 1) {
>> +        LOG(ERROR, "Invalid USB device");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV &&
>> +        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) {
>> +        LOG(ERROR, "Invalid USB device of hostdev");
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    /* Do the remove */
>> +    rc = do_usb_remove(gc, domid, usb);
>> +
>> +out:
>> +    return rc;
>> +}
>> +
>> +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
>> +                            libxl_device_usb *usb,
>> +                            const libxl_asyncop_how *ao_how)
>> +
>> +{
>> +    AO_CREATE(ctx, domid, ao_how);
>> +    int rc;
>> +
>> +    rc = libxl__device_usb_remove(gc, domid, usb);
>> +
>> +    libxl__ao_complete(egc, ao, rc);
>> +    return AO_INPROGRESS;
>> +}
>> +
>> +int libxl_ctrlport_to_device_usb(libxl_ctx *ctx,
>> +                                 uint32_t domid,
>> +                                 int ctrl,
>> +                                 int port,
>> +                                 libxl_device_usb *usb)
>> +{
>> +    GC_INIT(ctx);
>> +    const char *dompath, *be_path, *busid;
>> +    int rc;
>> +
>> +    dompath = libxl__xs_get_dompath(gc, domid);
>> +
>> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                  GCSPRINTF("%s/device/vusb/%d/backend", dompath, ctrl),
>> +                  &be_path);
>> +    if (rc) goto out;
>> +
>> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
>> +                           GCSPRINTF("%s/port/%d", be_path, port),
>> +                           &busid);
>> +    if (rc) goto out;
>> +
>> +    if (!strcmp(busid, "")) {
>> +        rc = ERROR_FAIL;
>> +        goto out;
>> +    }
>> +
>> +    usb->ctrl = ctrl;
>> +    usb->port = port;
>> +    rc = usb_busaddr_from_busid(gc, busid, &usb->u.hostdev.hostbus,
>> +                                &usb->u.hostdev.hostaddr);
>> +
>> +out:
>> +    GC_FREE;
>> +    return rc;
>> +}
>> +/*
>> + * Local variables:
>> + * mode: C
>> + * c-basic-offset: 4
>> + * indent-tabs-mode: nil
>> + * End:
>> + */
>> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
>> index d6ef9a2..dccb317 100644
>> --- a/tools/libxl/libxl_types.idl
>> +++ b/tools/libxl/libxl_types.idl
>> @@ -603,6 +603,35 @@ libxl_device_rdm = Struct("device_rdm", [
>>      ("policy", libxl_rdm_reserve_policy),
>>      ])
>>
>> +libxl_usbctrl_type = Enumeration("usbctrl_type", [
>> +    (0, "AUTO"),
>> +    (1, "PV"),
>> +    (2, "DEVICEMODEL"),
>> +    ])
>> +
>> +libxl_usbdev_type = Enumeration("usbdev_type", [
>> +    (1, "hostdev"),
>> +    ])
>> +
>> +libxl_device_usbctrl = Struct("device_usbctrl", [
>> +    ("type", libxl_usbctrl_type),
>> +    ("devid", libxl_devid),
>> +    ("version", integer),
>> +    ("ports", integer),
>> +    ("backend_domid", libxl_domid),
>> +    ("backend_domname", string),
>> +   ])
>> +
>> +libxl_device_usb = Struct("device_usb", [
>> +    ("ctrl", libxl_devid),
>> +    ("port", integer),
>> +    ("u", KeyedUnion(None, libxl_usbdev_type, "devtype",
>> +           [("hostdev", Struct(None, [
>> +                 ("hostbus",   uint8),
>> +                 ("hostaddr",  uint8)])),
>> +           ])),
>> +    ])
>> +
>>  libxl_device_dtdev = Struct("device_dtdev", [
>>      ("path", string),
>>      ])
>> @@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [
>>      ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
>>      ("rdms", Array(libxl_device_rdm, "num_rdms")),
>>      ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
>> +    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")),
>> +    ("usbs", Array(libxl_device_usb, "num_usbs")),
>>      ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
>>      ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
>>      ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
>> @@ -684,6 +715,32 @@ libxl_vtpminfo = Struct("vtpminfo", [
>>      ("uuid", libxl_uuid),
>>      ], dir=DIR_OUT)
>>
>> +libxl_usbctrlinfo = Struct("usbctrlinfo", [
>> +    ("type", libxl_usbctrl_type),
>> +    ("devid", libxl_devid),
>> +    ("version", integer),
>> +    ("ports", integer),
>> +    ("backend", string),
>> +    ("backend_id", uint32),
>> +    ("frontend", string),
>> +    ("frontend_id", uint32),
>> +    ("state", integer),
>> +    ("evtch", integer),
>> +    ("ref_urb", integer),
>> +    ("ref_conn", integer),
>> +    ], dir=DIR_OUT)
>> +
>> +libxl_usbinfo = Struct("usbinfo", [
>> +    ("ctrl", libxl_devid),
>> +    ("port", integer),
>> +    ("busnum", uint8),
>> +    ("devnum", uint8),
>> +    ("idVendor", uint16),
>> +    ("idProduct", uint16),
>> +    ("prod", string),
>> +    ("manuf", string),
>> +    ], dir=DIR_OUT)
>> +
>>  libxl_vcpuinfo = Struct("vcpuinfo", [
>>      ("vcpuid", uint32),
>>      ("cpu", uint32),
>> diff --git a/tools/libxl/libxl_types_internal.idl
>> b/tools/libxl/libxl_types_internal.idl
>> index 5e55685..696f5f8 100644
>> --- a/tools/libxl/libxl_types_internal.idl
>> +++ b/tools/libxl/libxl_types_internal.idl
>> @@ -22,6 +22,7 @@ libxl__device_kind = Enumeration("device_kind", [
>>      (6, "VKBD"),
>>      (7, "CONSOLE"),
>>      (8, "VTPM"),
>> +    (9, "VUSB"),
>>      ])
>>
>>  libxl__console_backend = Enumeration("console_backend", [
>> diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
>> index 9c5c4d0..706a0c1 100644
>> --- a/tools/libxl/libxl_utils.c
>> +++ b/tools/libxl/libxl_utils.c
>> @@ -1270,6 +1270,22 @@ int libxl__random_bytes(libxl__gc *gc, uint8_t *buf,
>> size_t len)
>>      return ret;
>>  }
>>
>> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr)
>> +{
>> +   int i;
>> +   for (i = 0; i < nr; i++)
>> +       libxl_device_usbctrl_dispose(&list[i]);
>> +   free(list);
>> +}
>> +
>> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr)
>> +{
>> +   int i;
>> +   for (i = 0; i < nr; i++)
>> +       libxl_device_usb_dispose(&list[i]);
>> +   free(list);
>> +}
>> +
>>  /*
>>   * Local variables:
>>   * mode: C
>> diff --git a/tools/libxl/libxl_utils.h b/tools/libxl/libxl_utils.h
>> index 339ebdf..60487e3 100644
>> --- a/tools/libxl/libxl_utils.h
>> +++ b/tools/libxl/libxl_utils.h
>> @@ -76,6 +76,11 @@ int libxl_uuid_to_device_vtpm(libxl_ctx *ctx, uint32_t
>> domid,
>>                                 libxl_uuid *uuid, libxl_device_vtpm *vtpm);
>>  int libxl_devid_to_device_vtpm(libxl_ctx *ctx, uint32_t domid,
>>                                 int devid, libxl_device_vtpm *vtpm);
>> +int libxl_devid_to_device_usbctrl(libxl_ctx *ctx, uint32_t domid,
>> +                                  int devid, libxl_device_usbctrl
>> *usbctrl);
>> +int libxl_ctrlport_to_device_usb(libxl_ctx *ctx, uint32_t domid,
>> +                                 int ctrl, int port,
>> +                                 libxl_device_usb *usb);
>>
>>  int libxl_bitmap_alloc(libxl_ctx *ctx, libxl_bitmap *bitmap, int n_bits);
>>      /* Allocated bimap is from malloc, libxl_bitmap_dispose() to be
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
  2015-10-27 11:31   ` Juergen Gross
  2015-11-04  6:31   ` Chun Yan Liu
@ 2015-11-09 18:11   ` Ian Jackson
  2015-11-10  8:41     ` Chun Yan Liu
  2015-11-12 11:32   ` Olaf Hering
                     ` (2 subsequent siblings)
  5 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-11-09 18:11 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	jfehlig, Simon Cao

Chunyan Liu writes ("[PATCH V8 3/7] libxl: add pvusb API"):
> Add pvusb APIs, including:
>  - attach/detach (create/destroy) virtual usb controller.
>  - attach/detach usb device
>  - list usb controller and usb devices
>  - some other helper functions

Thanks for this.


I have reviewed it in detail (not just the ao handling aspects) and
(I'm afraid) produced a large number of comments, relating to style,
error handling, etc., as well as ao handling.

Please let me know if anything I've said is unclear.  I've been rather
brief in my comments, rather than writing a paragraph for each one.
Thanks.


> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index dacfaae..a050e8b 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -4218,11 +4218,54 @@ DEFINE_DEVICE_REMOVE(vtpm, destroy, 1)
...
> +/* Macro for defining device remove/destroy functions for usbctrl */
> +/* Following functions are defined:
> + * libxl_device_usbctrl_remove
> + * libxl_device_usbctrl_destroy
> + */
> +
> +#define DEFINE_DEVICE_REMOVE_EXT(type, removedestroy, f)                \

As George has said, I would prefer to avoid committing this
duplication in-tree, even with a promise to de-duplicated it later.


> +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,
> +                               libxl_device_usbctrl *usbctrl,
> +                               libxl__ao_device *aodev)
> +{

Thanks for adjusting the error-handling patterns in these functions.
The new way is good, except that:

> +out:
> +    aodev->rc = rc;
> +    if (rc) aodev->callback(egc, aodev);

Here, rc is always set, and indeed the code would be wrong if it were
not.  So can you remove the conditional please ?  Ie:

  +   aodev->callback(egc, aodev);

The same goes for:

> +static int
> +libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb);
...

> +    /* Remove usb devices first */
> +    rc  = libxl__device_usb_list_for_usbctrl(gc, domid, usbctrl_devid,
> +                                             &usbs, &numusb);
         ^^
While you're fixing other things, you could remove one of these space.s

> +libxl_device_usbctrl *
> +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num)
> +{
...
> +        for (usbctrl = usbctrls; usbctrl < end;
> +             usbctrl++, entry++, (*num)++) {

I think this would be clearer if there were a line break at the first
semicolon too.  Ie, I like
    for (A; B; C) {
or
    for (A;
         B;
         C) {

But I don't like very much
    for (A; B;
         C) {
or
    for (A;
         B; C) {

> +#define READ_SUBPATH(path, subpath) ({                                  \
> +        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \
> +                                    GCSPRINTF("%s/" subpath, path),     \
> +                                    &tmp);                              \
> +        if (rc) goto outerr;                                            \
> +        (char *)tmp;                                                    \
> +    })

Thanks, I'm pleased with how you have done this.

> +            be_path = READ_SUBPATH(fe_path, "backend");
> +            usbctrl->backend_domid = atoi(READ_SUBPATH(fe_path, "backend-id"));
> +            usbctrl->version = atoi(READ_SUBPATH(be_path, "usb-ver"));
> +            usbctrl->ports = atoi(READ_SUBPATH(be_path, "num-ports"));

However, I have a question:

Why do you use atoi here, but strtoul in libxl_device_usbctrl_getinfo ?

> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
> +                                libxl_device_usbctrl *usbctrl,
> +                                libxl_usbctrlinfo *usbctrlinfo)
> +{
...
> +    tmp = READ_SUBPATH(fe_path, "backend-id");
> +    usbctrlinfo->backend_id = tmp ? strtoul(tmp, NULL, 10) : -1;

There are ten copies of this pattern with tmp and strtoul.  I really
think this needs to be refactored somehow.  Can you please make a
macro which returns the return value from strtoul ?

> +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
> +{
...
> +    while ((de = readdir(dir))) {

You need to use readdir_r, I'm afraid.  Ordinary readdir is not
threadsafe.

> +    }
> +
> +    closedir(dir);

This assumes that all errors from readdir are EOF.  But that's not the
case.  Luckily readdir_r has more sensible error behaviour.  But you
do need to check separately for errors and EOF.

All in all this probably means that it would be clearer to write this
as:
     for (;;) {
         r = readdir_r etc.
         if (r) ....
         if (!de) break;

There is at least one more call to readdir in this patch which also
needs to be fixed.

> +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid,
> +                                  uint8_t *bus, uint8_t *addr)
> +{
> +    char *filename;
> +    void *buf;
> +
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
> +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
> +        *bus = atoi((char *)buf);

I don't think this cast (and the one for addr) are necessary ?

> +static int
> +get_assigned_devices(libxl__gc *gc,
> +                     libxl_device_usb **list, int *num)
> +{
...
> +out:
> +    if (rc) {
> +        *list = NULL;
> +        *num = 0;
> +    }

Is this necessary ?  Given that get_assigned_devices only has internal
callers which expect gc'd memory, there is no memory cleanup issue.
And a caller who gets a nonzero return value ought not to then look at
the results.

The only existing call site seems not to care.

However, it would probably be better style to print a log message
here, rather than in the caller.


> +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
> +{
...
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid);
                                         ^
Our usual coding style would have a space here.

> +static int
> +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
> +                                   libxl_devid usbctrl,
> +                                   libxl_device_usb **usbs, int *num)
...
> +    for (i = 0; i < n; i++) {
> +        char *busid;
> +        libxl_device_usb *usb;
> +
> +        busid = libxl__xs_read(gc, XBT_NULL,
> +                               GCSPRINTF("%s/port/%d", be_path, i + 1));

You should be using libxl__xs_read_checked here I think.

This is true throughout.  There are more calls to libxl__xs_read which
probably need to be replaced.

...
> +out:
> +    if (rc) {
> +        *usbs = NULL;
> +        *num = 0;
> +    }

As comment as before.  However, the one call site for this ignores
errors, which is a bug.


However.  I think that get_assigned_devices has too much code that
looks like the middle of libxl__device_usb_list_for_usbctrl and
libxl_device_usb_list.

I think you could write get_assigned_devices in terms of
libxl_device_usb_list or maybe libxl__device_usb_list_for_usbctrl.


> +        fe_path = GCSPRINTF("%s/device/vusb/%d",
> +                         libxl__xs_get_dompath(gc, domid), usb->ctrl);
> +
> +        rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                                    GCSPRINTF("%s/backend", fe_path), &be_path);
> +        if (rc) goto out;

After reading the backend path out of the frontend, you need to check
that it's plausible.  Otherwise there perhaps possible attacks where
the frontend writes a backend path pointing somewhere else in
xenstore.

> +        rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                                    GCSPRINTF("%s/port/%d", be_path, usb->port),
> +                                    &tmp);
> +        if (rc) goto out;
> +
> +        if (strcmp(tmp, "")) {

This is rather odd.  What happens if the path doesn't exist ?  AFAICT
this will segfault.  (Another case in libxl_ctrlport_to_device_usb, at
least.)

....
> +/* bind/unbind usb device interface */
> +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)
> +{
> +    char *path, *spath, *dp = NULL;
> +    int fd = -1;
> +    int rc;
> +    struct stat st;
> +
> +    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf);
> +    if (!lstat(spath, &st)) {

The code should not assume that all lstate failures are ENOENT.

> +        /* Find the canonical path to the driver. */
> +        dp = libxl__zalloc(gc, PATH_MAX);
> +        dp = realpath(spath, dp);

Why is this call to realpath needed ?

> +        path = GCSPRINTF("%s/unbind", spath);
> +        fd = open(path, O_WRONLY);
> +        if (fd < 0)
> +            return ERROR_FAIL;
> +        rc = write(fd, intf, strlen(intf));
> +        close(fd);
> +        if (rc < 0)
> +            return ERROR_FAIL;

All of these system calls should log errors (errno) appropriately.
The write should check that the expected amount was written.

rc should only be used for a libxl error code.  See CODING_STYLE.
(Multiple occurrences throughout this patch.)

Do we know that writes to sysfs don't ever return short or EINTR ?

> +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath)
> +{

This function has a remarkable resemblance to much of the unbind
function.  The common code should be factored out.

> +static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb,
> +                                  char ***intfs, int *num)
> +{
> +    DIR *dir;

This (and the earlier function which also uses opendir) uses the wrong
resource / error-handling pattern for dir.  See ERROR HANDLING in
CODING_SYTLE.

> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
> +                                 usb->u.hostdev.hostaddr);
> +    if (!busid) {
> +        rc = ERROR_FAIL;

I think these internal functions such as usb_busaddr_to_busid, _bind,
_is_assignable, etc., ought to return rc values, rather than all the
callers inventing ERROR_FAIL.


> +    if (!(dir = opendir(SYSFS_USB_DEV))) {
> +        rc = ERROR_FAIL;

Again, you need to log errno (and the path, too).

> +/* Encode usb interface so that it could be written to xenstore as a key.
> + *
> + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_',
> + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1.
> + * This will be used to save original driver of USB device to xenstore.
> + */

What is the syntax of the incoming busid ?  Could it contain _ or - ?
You should perhaps spot them and reject if it does.

> +static char *usb_interface_xenstore_encode(char *busid)
> +{
> +    char *str = strdup(busid);

This function should either update the value in place, or be
const-correct (ie, take a const char*).

Do not use bare strdup.  This means if it's going to make a copy, it
needs to take a gc.


> +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
> +{
...
> +        /* bind interface to its originial driver */
> +        drvpath = libxl__xs_read(gc, XBT_NULL,
> +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
> +                  usb_encode, usb_interface_xenstore_encode(intf)));
> +        if (drvpath && bind_usb_intf(gc, intf, drvpath))
> +            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath);

This error message could be clearer.

Also it would be worth a comment in the code to explain why this is a
warning (merely logged), rather than an error (logged and reported
with nonzero status to caller).

> +/* Bind USB device to "usbback" driver.
> + *
> + * If there are many interfaces under USB device, check each interface,
> + * unbind from original driver and bind to "usbback" driver.
> + */
> +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
> +{
...
> +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
> +                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
> +            }

One of the advantages of libxl__xs_write_checked is that it logs
errors.  So you can leave that log message out.

However, if the xs write fails, you should presumably stop, rather
than carrying on.

I think you probably do things in the wrong order here.  You should
write the old driver info to xenstore first, so that if the bind to
usbback fails, you have the old driver info.

> +    free(usb_encode);

This came from the gc, I think.


> +/*
> + * USB add
> + */

Many of these comments often don't add much.  OTOH I won't insist you
remove them.

> +static int do_usb_add(libxl__gc *gc, uint32_t domid,
> +                      libxl_device_usb *usbdev,
> +                      libxl__ao_device *aodev)
> +{
...
> +    switch (usbctrlinfo.type) {
> +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:
> +        LOG(ERROR, "Not supported");
> +        break;

Needs to set rc.

> +    case LIBXL_USBCTRL_TYPE_PV:
> +        rc = libxl__device_usb_add_xenstore(gc, domid, usbdev,
> +                                            aodev->update_json);
> +        if (rc) goto out;
> +
> +        rc = usbback_dev_assign(gc, usbdev);
> +        if (rc) {
> +            libxl__device_usb_remove_xenstore(gc, domid, usbdev);
> +            goto out;
> +        }
> +        break;
> +    default:
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +out:

Before out, rc should be explicitly set to 0, rather than relying on
the happenstance of the previous setting.

> +void libxl__device_usb_add(libxl__egc *egc, uint32_t domid,
> +                           libxl_device_usb *usb,
> +                           libxl__ao_device *aodev)
> +{
...
> +        if (usbctrlinfo.backend_id != 0) {

LIBXL_TOOLSTACK_DOMID please.  (And in remove, too.)

Why is this check done in libxl__device_usb_add and do_usb_remove,
rather than in a symmetrical way ?

> +    /* Do the add */
> +    if (do_usb_add(gc, domid, usb, aodev)) {
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    libxl__ao_complete(egc, ao, 0);
> +    rc = 0;
> +
> +out:
> +    libxl_device_usbctrl_dispose(&usbctrl);
> +    libxl_usbctrlinfo_dispose(&usbctrlinfo);
> +    aodev->rc = rc;
> +    if (rc) aodev->callback(egc, aodev);
> +    return;

Calling libxl__ao_complete here is quite wrong.  I think you should
call aodev->callback in both success and failure cases.  And it must
be the last thing you do.

> +static int do_usb_remove(libxl__gc *gc, uint32_t domid,
> +                         libxl_device_usb *usbdev)
> +{
> +    libxl_ctx *ctx = CTX;

What is this for ?  I think, instead, you can just refer to CTX where
needed.


...
> +    switch (usbctrlinfo.type) {
> +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:
> +        LOG(ERROR, "Not supported");
> +        break;
> +    case LIBXL_USBCTRL_TYPE_PV:
> +        rc = libxl__device_usb_remove_xenstore(gc, domid, usbdev);
> +        if (rc) goto out;
> +
> +        usbback_dev_unassign(gc, usbdev);
> +        break;
> +    default:
> +        rc = ERROR_FAIL;
> +        goto out;

This `default' case should do something more sensible than simply
returning ERROR_FAIL with no log message (for `add', too).

> +/* Operation to remove usb device.
> + *
> + * Generally, it does:
> + * 1) check if the usb device is assigned to the domain
> + * 2) remove the usb device from xenstore controller/port.
> + * 3) unbind usb device from usbback and rebind to its original driver.
> + *    If usb device has many interfaces, do it to each interface.
> + */
> +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
> +                                    libxl_device_usb *usb)
> +{
> +    int rc;
> +
> +    if (usb->ctrl < 0 || usb->port < 1) {
> +        LOG(ERROR, "Invalid USB device");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV &&
> +        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) {
> +        LOG(ERROR, "Invalid USB device of hostdev");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }

Why are these checks here rather than in do_usb_remove ?

For that matter, why is this function not the same as do_usb_remove ?

I might ask the same question about do_usb_add and
libxl__device_usb_add ?


> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr)
> +{
> +   int i;
> +   for (i = 0; i < nr; i++)
> +       libxl_device_usbctrl_dispose(&list[i]);
> +   free(list);
> +}
> +
> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr)
> +{
> +   int i;
> +   for (i = 0; i < nr; i++)
> +       libxl_device_usb_dispose(&list[i]);
> +   free(list);
> +}

It is a shame that these aren't autogenerated, but I won't ask you to
arrange that (as there are already too many instances of this
in-tree).


Regards,
Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-09 18:11   ` Ian Jackson
@ 2015-11-10  8:41     ` Chun Yan Liu
  2015-11-10 17:57       ` George Dunlap
  2015-11-12 17:00       ` Ian Jackson
  0 siblings, 2 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-10  8:41 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/10/2015 at 02:11 AM, in message
<22080.57829.461049.37192@mariner.uk.xensource.com>, Ian Jackson
<Ian.Jackson@eu.citrix.com> wrote: 
> Chunyan Liu writes ("[PATCH V8 3/7] libxl: add pvusb API"): 
> > Add pvusb APIs, including: 
> >  - attach/detach (create/destroy) virtual usb controller. 
> >  - attach/detach usb device 
> >  - list usb controller and usb devices 
> >  - some other helper functions 
>  
> Thanks for this. 
>  
>  
> I have reviewed it in detail (not just the ao handling aspects) and 
> (I'm afraid) produced a large number of comments, relating to style, 
> error handling, etc., as well as ao handling. 
>  
> Please let me know if anything I've said is unclear.  I've been rather 
> brief in my comments, rather than writing a paragraph for each one. 
> Thanks. 
>  
>  
> > diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c 
> > index dacfaae..a050e8b 100644 
> > --- a/tools/libxl/libxl.c 
> > +++ b/tools/libxl/libxl.c 
> > @@ -4218,11 +4218,54 @@ DEFINE_DEVICE_REMOVE(vtpm, destroy, 1) 
> ... 
> > +/* Macro for defining device remove/destroy functions for usbctrl */ 
> > +/* Following functions are defined: 
> > + * libxl_device_usbctrl_remove 
> > + * libxl_device_usbctrl_destroy 
> > + */ 
> > + 
> > +#define DEFINE_DEVICE_REMOVE_EXT(type, removedestroy, f)                \ 
>  
> As George has said, I would prefer to avoid committing this 
> duplication in-tree, even with a promise to de-duplicated it later. 
>  
>  
> > +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid, 
> > +                               libxl_device_usbctrl *usbctrl, 
> > +                               libxl__ao_device *aodev) 
> > +{ 
>  
> Thanks for adjusting the error-handling patterns in these functions. 
> The new way is good, except that: 
>  
> > +out: 
> > +    aodev->rc = rc; 
> > +    if (rc) aodev->callback(egc, aodev); 
>  
> Here, rc is always set, and indeed the code would be wrong if it were 
> not.  So can you remove the conditional please ?  Ie: 

Reading the codes, libxl__wait_device_connection will call aodev->callback
properly. So here, only if (rc != 0), that means error happens, then we need to
call aodev->callback to end the process. (Refer to current libxl__device_disk_add,
all current code does similar work.) So I think the code is not wrong (?)

>  
>   +   aodev->callback(egc, aodev); 
>  
> The same goes for: 
>  
> > +static int 
> > +libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb  
> *usb); 
> ... 
>  
> > +    /* Remove usb devices first */ 
> > +    rc  = libxl__device_usb_list_for_usbctrl(gc, domid, usbctrl_devid, 
> > +                                             &usbs, &numusb); 
>          ^^ 
> While you're fixing other things, you could remove one of these space.s 
>  
> > +libxl_device_usbctrl * 
> > +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num) 
> > +{ 
> ... 
> > +        for (usbctrl = usbctrls; usbctrl < end; 
> > +             usbctrl++, entry++, (*num)++) { 
>  
> I think this would be clearer if there were a line break at the first 
> semicolon too.  Ie, I like 
>     for (A; B; C) { 
> or 
>     for (A; 
>          B; 
>          C) { 
>  
> But I don't like very much 
>     for (A; B; 
>          C) { 
> or 
>     for (A; 
>          B; C) { 
>  
> > +#define READ_SUBPATH(path, subpath) ({                                  \ 
> > +        rc = libxl__xs_read_checked(gc, XBT_NULL,                       \ 
> > +                                    GCSPRINTF("%s/" subpath, path),     \ 
> > +                                    &tmp);                              \ 
> > +        if (rc) goto outerr;                                            \ 
> > +        (char *)tmp;                                                    \ 
> > +    }) 
>  
> Thanks, I'm pleased with how you have done this. 
>  
> > +            be_path = READ_SUBPATH(fe_path, "backend"); 
> > +            usbctrl->backend_domid = atoi(READ_SUBPATH(fe_path,  
> "backend-id")); 
> > +            usbctrl->version = atoi(READ_SUBPATH(be_path, "usb-ver")); 
> > +            usbctrl->ports = atoi(READ_SUBPATH(be_path, "num-ports")); 
>  
> However, I have a question: 
>  
> Why do you use atoi here, but strtoul in libxl_device_usbctrl_getinfo ? 

Didn't care much about that. atoi and strtoul both work. Previously use aoti,
in libxl_device_usbctrl_getinfo, to keep consistent with other functions (like
libxl_device_disk_getinfo), use strtoul.

Will update and use the same.

>  
> > +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid, 
> > +                                libxl_device_usbctrl *usbctrl, 
> > +                                libxl_usbctrlinfo *usbctrlinfo) 
> > +{ 
> ... 
> > +    tmp = READ_SUBPATH(fe_path, "backend-id"); 
> > +    usbctrlinfo->backend_id = tmp ? strtoul(tmp, NULL, 10) : -1; 
>  
> There are ten copies of this pattern with tmp and strtoul.  I really 
> think this needs to be refactored somehow.  Can you please make a 
> macro which returns the return value from strtoul ? 

Will update.

>  
> > +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr) 
> > +{ 
> ... 
> > +    while ((de = readdir(dir))) { 
>  
> You need to use readdir_r, I'm afraid.  Ordinary readdir is not 
> threadsafe. 

Yep. Thanks.

>  
> > +    } 
> > + 
> > +    closedir(dir); 
>  
> This assumes that all errors from readdir are EOF.  But that's not the 
> case.  Luckily readdir_r has more sensible error behaviour.  But you 
> do need to check separately for errors and EOF. 

Will update.

>  
> All in all this probably means that it would be clearer to write this 
> as: 
>      for (;;) { 
>          r = readdir_r etc. 
>          if (r) .... 
>          if (!de) break; 
>  
> There is at least one more call to readdir in this patch which also 
> needs to be fixed. 

Yep.

>  
> > +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid, 
> > +                                  uint8_t *bus, uint8_t *addr) 
> > +{ 
> > +    char *filename; 
> > +    void *buf; 
> > + 
> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid); 
> > +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> > +        *bus = atoi((char *)buf); 
>  
> I don't think this cast (and the one for addr) are necessary ? 

Which cast? Here, we want to get a uint_8 value, but buf is a string,
we need to do atoi.

>  
> > +static int 
> > +get_assigned_devices(libxl__gc *gc, 
> > +                     libxl_device_usb **list, int *num) 
> > +{ 
> ... 
> > +out: 
> > +    if (rc) { 
> > +        *list = NULL; 
> > +        *num = 0; 
> > +    } 
>  
> Is this necessary ?  Given that get_assigned_devices only has internal 
> callers which expect gc'd memory, there is no memory cleanup issue. 
> And a caller who gets a nonzero return value ought not to then look at 
> the results. 
>  
> The only existing call site seems not to care. 
>  
> However, it would probably be better style to print a log message 
> here, rather than in the caller.

Will update.
 
>  
>  
> > +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> ... 
> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid); 
>                                          ^ 
> Our usual coding style would have a space here. 
>  
> > +static int 
> > +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid, 
> > +                                   libxl_devid usbctrl, 
> > +                                   libxl_device_usb **usbs, int *num) 
> ... 
> > +    for (i = 0; i < n; i++) { 
> > +        char *busid; 
> > +        libxl_device_usb *usb; 
> > + 
> > +        busid = libxl__xs_read(gc, XBT_NULL, 
> > +                               GCSPRINTF("%s/port/%d", be_path, i + 1)); 
>  
> You should be using libxl__xs_read_checked here I think. 
>  
> This is true throughout.  There are more calls to libxl__xs_read which 
> probably need to be replaced.

Got it, will update all.
 
>  
> ... 
> > +out: 
> > +    if (rc) { 
> > +        *usbs = NULL; 
> > +        *num = 0; 
> > +    } 
>  
> As comment as before.  However, the one call site for this ignores 
> errors, which is a bug.

Will update.
 
>  
>  
> However.  I think that get_assigned_devices has too much code that 
> looks like the middle of libxl__device_usb_list_for_usbctrl and 
> libxl_device_usb_list. 
>  
> I think you could write get_assigned_devices in terms of 
> libxl_device_usb_list or maybe libxl__device_usb_list_for_usbctrl.

Will check and update.
 
>  
>  
> > +        fe_path = GCSPRINTF("%s/device/vusb/%d", 
> > +                         libxl__xs_get_dompath(gc, domid), usb->ctrl); 
> > + 
> > +        rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                                    GCSPRINTF("%s/backend", fe_path),  
> &be_path); 
> > +        if (rc) goto out; 
>  
> After reading the backend path out of the frontend, you need to check 
> that it's plausible.  Otherwise there perhaps possible attacks where 
> the frontend writes a backend path pointing somewhere else in 
> xenstore. 
>  
> > +        rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                                    GCSPRINTF("%s/port/%d", be_path,  
> usb->port), 
> > +                                    &tmp); 
> > +        if (rc) goto out; 
> > + 
> > +        if (strcmp(tmp, "")) { 
>  
> This is rather odd.  What happens if the path doesn't exist ?  AFAICT 
> this will segfault.  (Another case in libxl_ctrlport_to_device_usb, at 
> least.) 

Hmm, we need to check tmp first, if (tmp && strcmp(tmp, ""))

>  
> .... 
> > +/* bind/unbind usb device interface */ 
> > +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath) 
> > +{ 
> > +    char *path, *spath, *dp = NULL; 
> > +    int fd = -1; 
> > +    int rc; 
> > +    struct stat st; 
> > + 
> > +    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf); 
> > +    if (!lstat(spath, &st)) { 
>  
> The code should not assume that all lstate failures are ENOENT.

Will update.

> > +        dp = libxl__zalloc(gc, PATH_MAX); 
> > +        dp = realpath(spath, dp); 
>  
> Why is this call to realpath needed ?

In sysfs, /driver sometimes is a link, since we need to save the original
driver to xenstore, so need to get the realpath of the driver.
 
>  
> > +        path = GCSPRINTF("%s/unbind", spath); 
> > +        fd = open(path, O_WRONLY); 
> > +        if (fd < 0) 
> > +            return ERROR_FAIL; 
> > +        rc = write(fd, intf, strlen(intf)); 
> > +        close(fd); 
> > +        if (rc < 0) 
> > +            return ERROR_FAIL; 
>  
> All of these system calls should log errors (errno) appropriately. 
> The write should check that the expected amount was written. 
>  
> rc should only be used for a libxl error code.  See CODING_STYLE. 
> (Multiple occurrences throughout this patch.) 

Yep. Will update.

>  
> Do we know that writes to sysfs don't ever return short or EINTR ? 
>  
> > +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath) 
> > +{ 
>  
> This function has a remarkable resemblance to much of the unbind 
> function.  The common code should be factored out.

Will check and update.
 
>  
> > +static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb, 
> > +                                  char ***intfs, int *num) 
> > +{ 
> > +    DIR *dir; 
>  
> This (and the earlier function which also uses opendir) uses the wrong 
> resource / error-handling pattern for dir.  See ERROR HANDLING in 
> CODING_SYTLE.

Will check and update.
 
>  
> > +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, a
> > +    if (!busid) {
> > +        rc = ERROR_FAIL; 
>  
> I think these internal functions such as usb_busaddr_to_busid, _bind, 
> _is_assignable, etc., ought to return rc values, rather than all the 
> callers inventing ERROR_FAIL.

I'll check.
 
>  
>  
> > +    if (!(dir = opendir(SYSFS_USB_DEV))) { 
> > +        rc = ERROR_FAIL; 
>  
> Again, you need to log errno (and the path, too). 

OK. Will update.

>  
> > +/* Encode usb interface so that it could be written to xenstore as a key. 
> > + * 
> > + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_', 
> > + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1. 
> > + * This will be used to save original driver of USB device to xenstore. 
> > + */ 
>  
> What is the syntax of the incoming busid ?  Could it contain _ or - ? 
> You should perhaps spot them and reject if it does.

The busid is in syntax like 3-1:2.1. It does contain '-'. But since we only use
the single direction, never decode back, so it won't harm.
 
>  
> > +static char *usb_interface_xenstore_encode(char *busid) 
> > +{ 
> > +    char *str = strdup(busid); 
>  
> This function should either update the value in place, or be 
> const-correct (ie, take a const char*). 
>  
> Do not use bare strdup.  This means if it's going to make a copy, it 
> needs to take a gc. 

Will update.

>  
>  
> > +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> ... 
> > +        /* bind interface to its originial driver */ 
> > +        drvpath = libxl__xs_read(gc, XBT_NULL, 
> > +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path", 
> > +                  usb_encode, usb_interface_xenstore_encode(intf))); 
> > +        if (drvpath && bind_usb_intf(gc, intf, drvpath)) 
> > +            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath); 
>  
> This error message could be clearer. 
>  
> Also it would be worth a comment in the code to explain why this is a 
> warning (merely logged), rather than an error (logged and reported 
> with nonzero status to caller).

Will update. When doing USB remove, it's the ideal result we can rebound
the USB to its original driver, in case the USB interface failed to be
rebound to its original driver, we will export warning but won't stop the
removing process.

> 
> > +/* Bind USB device to "usbback" driver. 
> > + * 
> > + * If there are many interfaces under USB device, check each interface, 
> > + * unbind from original driver and bind to "usbback" driver. 
> > + */ 
> > +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> ... 
> > +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) { 
> > +                LOG(WARN, "Write of %s to node %s failed", drvpath, path); 
> > +            } 
>  
> One of the advantages of libxl__xs_write_checked is that it logs 
> errors.  So you can leave that log message out. 
>  
> However, if the xs write fails, you should presumably stop, rather 
> than carrying on. 
> 
> I think you probably do things in the wrong order here.  You should 
> write the old driver info to xenstore first, so that if the bind to 
> usbback fails, you have the old driver info. 

Perhaps not. Once we finished adding entries to xenstore, pvusb
frontend/backend drivers will detect the change and do probe work, if
USB device is still not bound to USBBACK, there might be error.

>  
> > +    free(usb_encode); 
>  
> This came from the gc, I think. 
>  
>  
> > +/* 
> > + * USB add 
> > + */ 
>  
> Many of these comments often don't add much.  OTOH I won't insist you 
> remove them. 
>  
> > +static int do_usb_add(libxl__gc *gc, uint32_t domid, 
> > +                      libxl_device_usb *usbdev, 
> > +                      libxl__ao_device *aodev) 
> > +{ 
> ... 
> > +    switch (usbctrlinfo.type) { 
> > +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL: 
> > +        LOG(ERROR, "Not supported"); 
> > +        break; 
>  
> Needs to set rc. 
>  
> > +    case LIBXL_USBCTRL_TYPE_PV: 
> > +        rc = libxl__device_usb_add_xenstore(gc, domid, usbdev, 
> > +                                            aodev->update_json); 
> > +        if (rc) goto out; 
> > + 
> > +        rc = usbback_dev_assign(gc, usbdev); 
> > +        if (rc) { 
> > +            libxl__device_usb_remove_xenstore(gc, domid, usbdev); 
> > +            goto out; 
> > +        } 
> > +        break; 
> > +    default: 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +out: 
>  
> Before out, rc should be explicitly set to 0, rather than relying on 
> the happenstance of the previous setting. 

Yep.

>  
> > +void libxl__device_usb_add(libxl__egc *egc, uint32_t domid, 
> > +                           libxl_device_usb *usb, 
> > +                           libxl__ao_device *aodev) 
> > +{ 
> ... 
> > +        if (usbctrlinfo.backend_id != 0) { 
>  
> LIBXL_TOOLSTACK_DOMID please.  (And in remove, too.) 
>  
> Why is this check done in libxl__device_usb_add and do_usb_remove, 
> rather than in a symmetrical way?
 
In libxl__device_usb_add, the set_default function implementation won't
work if backend domain is USB driver domain in future, so we must add
a check before set_default, so in libxl__device_usb_add. To check backend
domain, we need to get usbctrl info. And in do_usb_add, to check usbctrl
type, again we need to get usbctrl info. It's disgusting to do the same thing
twice.

In usb remove, nothing in libxl__device_usb_remove is Dom0 specific, so
I do all check in do_usb_remove, then only one time to get usbctrl info.

If we merge libxl__device_usb_add and do_usb_add, then it's cleaner.
 
>  
> > +    /* Do the add */ 
> > +    if (do_usb_add(gc, domid, usb, aodev)) { 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    libxl__ao_complete(egc, ao, 0); 
> > +    rc = 0; 
> > + 
> > +out: 
> > +    libxl_device_usbctrl_dispose(&usbctrl); 
> > +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> > +    aodev->rc = rc; 
> > +    if (rc) aodev->callback(egc, aodev); 
> > +    return; 
>  
> Calling libxl__ao_complete here is quite wrong.  I think you should 
> call aodev->callback in both success and failure cases.  And it must 
> be the last thing you do. 

As mentioned in libxl__device_usbctrl_add, libxl__ao_complete will
call callback, so in 'out' section, only if (rc !=0), which means error happens,
then we need to call aodev->callback explicitly to end the process.

Here, since we don't have actual async work needs to do, so call
libxl__ao_complete(egc, ao, 0) directly to end the process.

>  
> > +static int do_usb_remove(libxl__gc *gc, uint32_t domid, 
> > +                         libxl_device_usb *usbdev) 
> > +{ 
> > +    libxl_ctx *ctx = CTX; 
>  
> What is this for ?  I think, instead, you can just refer to CTX where 
> needed. 

Yep.

>  
>  
> ... 
> > +    switch (usbctrlinfo.type) { 
> > +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL: 
> > +        LOG(ERROR, "Not supported"); 
> > +        break; 
> > +    case LIBXL_USBCTRL_TYPE_PV: 
> > +        rc = libxl__device_usb_remove_xenstore(gc, domid, usbdev); 
> > +        if (rc) goto out; 
> > + 
> > +        usbback_dev_unassign(gc, usbdev); 
> > +        break; 
> > +    default: 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
>  
> This `default' case should do something more sensible than simply 
> returning ERROR_FAIL with no log message (for `add', too). 

Will update.

>  
> > +/* Operation to remove usb device. 
> > + * 
> > + * Generally, it does: 
> > + * 1) check if the usb device is assigned to the domain 
> > + * 2) remove the usb device from xenstore controller/port. 
> > + * 3) unbind usb device from usbback and rebind to its original driver. 
> > + *    If usb device has many interfaces, do it to each interface. 
> > + */ 
> > +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, 
> > +                                    libxl_device_usb *usb) 
> > +{ 
> > +    int rc; 
> > + 
> > +    if (usb->ctrl < 0 || usb->port < 1) { 
> > +        LOG(ERROR, "Invalid USB device"); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV && 
> > +        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) { 
> > +        LOG(ERROR, "Invalid USB device of hostdev"); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
>  
> Why are these checks here rather than in do_usb_remove ? 
>  
> For that matter, why is this function not the same as do_usb_remove ? 
>  
> I might ask the same question about do_usb_add and 
> libxl__device_usb_add ? 

Using libxl__device_usb_add and within it extracting do_usb_add is to:
* doing some initialization work and necessary check work in
  libxl__device_usb_add (this part are common to PVUSB and QEMU USB)
* doing actual adding usb operations in do_usb_add (this part will diverge
  for PVUSB and QEMU USB) .

Same to libxl__device_usb_remove and do_usb_remove.

Certainly, we could merge into one function if that is better. George may
have some opinions on this?

-Chunyan

>  
>  
> > +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr) 
> > +{ 
> > +   int i; 
> > +   for (i = 0; i < nr; i++) 
> > +       libxl_device_usbctrl_dispose(&list[i]); 
> > +   free(list); 
> > +} 
> > + 
> > +void libxl_device_usb_list_free(libxl_device_usb *list, int nr) 
> > +{ 
> > +   int i; 
> > +   for (i = 0; i < nr; i++) 
> > +       libxl_device_usb_dispose(&list[i]); 
> > +   free(list); 
> > +} 
>  
> It is a shame that these aren't autogenerated, but I won't ask you to 
> arrange that (as there are already too many instances of this 
> in-tree). 
>  
>  
> Regards, 
> Ian. 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-10  8:41     ` Chun Yan Liu
@ 2015-11-10 17:57       ` George Dunlap
  2015-11-10 18:11         ` Ian Jackson
  2015-11-11  2:37         ` Chun Yan Liu
  2015-11-12 17:00       ` Ian Jackson
  1 sibling, 2 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-10 17:57 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao

[-- Attachment #1: Type: text/plain, Size: 7307 bytes --]

On Tue, Nov 10, 2015 at 8:41 AM, Chun Yan Liu <cyliu@suse.com> wrote:
>> > +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,
>> > +                               libxl_device_usbctrl *usbctrl,
>> > +                               libxl__ao_device *aodev)
>> > +{
>>
>> Thanks for adjusting the error-handling patterns in these functions.
>> The new way is good, except that:
>>
>> > +out:
>> > +    aodev->rc = rc;
>> > +    if (rc) aodev->callback(egc, aodev);
>>
>> Here, rc is always set, and indeed the code would be wrong if it were
>> not.  So can you remove the conditional please ?  Ie:
>
> Reading the codes, libxl__wait_device_connection will call aodev->callback
> properly. So here, only if (rc != 0), that means error happens, then we need to
> call aodev->callback to end the process. (Refer to current libxl__device_disk_add,
> all current code does similar work.) So I think the code is not wrong (?)

>> > +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid,
>> > +                                  uint8_t *bus, uint8_t *addr)
>> > +{
>> > +    char *filename;
>> > +    void *buf;
>> > +
>> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
>> > +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
>> > +        *bus = atoi((char *)buf);
>>
>> I don't think this cast (and the one for addr) are necessary ?
>
> Which cast? Here, we want to get a uint_8 value, but buf is a string,
> we need to do atoi.

atoi() isn't a cast, it's a function call.  The cast is the (char *) bit.

I think probably you could get away with declaring buf as (char *).
&buf should be converted to void** automatically without any warnings,
I think.

>> > +/* Encode usb interface so that it could be written to xenstore as a key.
>> > + *
>> > + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_',
>> > + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1.
>> > + * This will be used to save original driver of USB device to xenstore.
>> > + */
>>
>> What is the syntax of the incoming busid ?  Could it contain _ or - ?
>> You should perhaps spot them and reject if it does.
>
> The busid is in syntax like 3-1:2.1. It does contain '-'. But since we only use
> the single direction, never decode back, so it won't harm.

So the only potential problem is that 3-1:2, 3-1-2, 3:1-2, and 3:1:2
all collapse down to 3-1-2.  Is there any risk of something like that
happening?  (Ian: NB these are all read from sysfs.)

Since this isn't really being read by anyone,  maybe it would be
better to replace ':' with another character, just to be safe.  It
could even be something like 'c' if no other punctuation is available.

>> > +/* Bind USB device to "usbback" driver.
>> > + *
>> > + * If there are many interfaces under USB device, check each interface,
>> > + * unbind from original driver and bind to "usbback" driver.
>> > + */
>> > +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
>> > +{
>> ...
>> > +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
>> > +                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
>> > +            }
>>
>> One of the advantages of libxl__xs_write_checked is that it logs
>> errors.  So you can leave that log message out.
>>
>> However, if the xs write fails, you should presumably stop, rather
>> than carrying on.
>>
>> I think you probably do things in the wrong order here.  You should
>> write the old driver info to xenstore first, so that if the bind to
>> usbback fails, you have the old driver info.

Ian, I don't understand what you're saying here.  The code I'm looking at does:
1. unbind + get old driver path in drvpath
2. if(drvpath) write to xenstore
3. bind to usbback

If the bind fails, then we do have drvpath in xenstore.  Or am I
missing something?

Bailing out if the above xenstore write fails seems sensible though.

>> > +/* Operation to remove usb device.
>> > + *
>> > + * Generally, it does:
>> > + * 1) check if the usb device is assigned to the domain
>> > + * 2) remove the usb device from xenstore controller/port.
>> > + * 3) unbind usb device from usbback and rebind to its original driver.
>> > + *    If usb device has many interfaces, do it to each interface.
>> > + */
>> > +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
>> > +                                    libxl_device_usb *usb)
>> > +{
>> > +    int rc;
>> > +
>> > +    if (usb->ctrl < 0 || usb->port < 1) {
>> > +        LOG(ERROR, "Invalid USB device");
>> > +        rc = ERROR_FAIL;
>> > +        goto out;
>> > +    }
>> > +
>> > +    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV &&
>> > +        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) {
>> > +        LOG(ERROR, "Invalid USB device of hostdev");
>> > +        rc = ERROR_FAIL;
>> > +        goto out;
>> > +    }
>>
>> Why are these checks here rather than in do_usb_remove ?
>>
>> For that matter, why is this function not the same as do_usb_remove ?
>>
>> I might ask the same question about do_usb_add and
>> libxl__device_usb_add ?
>
> Using libxl__device_usb_add and within it extracting do_usb_add is to:
> * doing some initialization work and necessary check work in
>   libxl__device_usb_add (this part are common to PVUSB and QEMU USB)
> * doing actual adding usb operations in do_usb_add (this part will diverge
>   for PVUSB and QEMU USB) .
>
> Same to libxl__device_usb_remove and do_usb_remove.
>
> Certainly, we could merge into one function if that is better. George may
> have some opinions on this?

I think it would be nice to have things broken down that way, but it's
likely I'll have to do some adjustments when I come to add in
devicemodel usb anyway.  So making it one big function wouldn't hurt.
(Nor would leaving it the way it is, as far as I'm concerned.)

The big thing though is with the removal interface here, where we
again have somewhat of a confusion between naming things based on the
*host* name (as we do in pci pass-through) vs the *guest* names (as we
do for disks and nics).

If you have the controller and virtual port, there's no need for the
caller to *also* go and look up the host bus and host address; or vice
versa -- if you have the hostbus and hostaddr, you shouldn't also need
to go look up the usbctrl and virtual port.

It looks like for libxl_device_{disk,nic,vtpm}_remove(), it gets a
libxl_device struct using libxl__device_from_{disk,nic,vtpm}; and in
all cases, the information used to construct the "device" is the
backend + either a devid (for nic and vtpm) or vdev, for disks.  It
looks like for those devices, any conversion from other labels (mac
address for nic, uuid for vtpm) is done in xl_cmdimpli.c.

I think we want to follow suit here -- libxl_device_usb_remove()
should take a usbctrl and port number only, and should look up
whatever information it needs to do the removal from that.

All we really need for the re-assignment is the busid, which is
already stored in a xenstore location we can find using domid, ctrl,
and port.  Something like the attached patch (compile-tested only).
What do you think?

(NB the patch doesn't fix the gc or aliasing issues in
usb_interface_xenstore_encode() -- those still need to be addressed.)

 -George

[-- Attachment #2: 0001-tools-Don-t-use-usb-hostbus-and-hostaddr-during-remo.patch --]
[-- Type: text/x-diff, Size: 6015 bytes --]

From 92a4725d9f669cbd63ec19d85424f47fa4eb96b6 Mon Sep 17 00:00:00 2001
From: George Dunlap <george.dunlap@eu.citrix.com>
Date: Tue, 10 Nov 2015 17:22:39 +0000
Subject: [PATCH] tools: Don't use usb hostbus and hostaddr during removal

The busid is already stored at a xenstore location we can find with
<ctrl,port>.  Read this value before removing the entry; then use it
to re-assign the host device back to the original drivers.

Change usb_get_all_interfaces() to take busid, rather than looking for
it itself.  In the remove case, use the busid value we got from the
backend; in the add() case, just reorder the calls so that
busaddr_to_busid happens first.  (This actually avoids a duplicate
busaddr_to_busid conversion in the add case.)

Signed-off-by: George Dunlap <george.dunlap@citrix.com>
---
 tools/libxl/libxl_pvusb.c | 57 +++++++++++++++++++++++------------------------
 1 file changed, 28 insertions(+), 29 deletions(-)

diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
index ab57371..ead3e50 100644
--- a/tools/libxl/libxl_pvusb.c
+++ b/tools/libxl/libxl_pvusb.c
@@ -971,6 +971,15 @@ static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t domid,
     return 0;
 }
 
+static char * usbback_busid_from_ctrlport(libxl__gc *gc, uint32_t domid,
+                                             libxl_device_usb *usb)
+{
+    return libxl__xs_read(gc, XBT_NULL,
+                          GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                                    libxl__xs_get_dompath(gc, LIBXL_TOOLSTACK_DOMID),
+                                    domid, usb->ctrl, usb->port));
+}
+
 /* bind/unbind usb device interface */
 static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)
 {
@@ -1044,25 +1053,17 @@ static int usb_intf_is_assigned(libxl__gc *gc, char *intf)
     return -1;
 }
 
-static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb,
+static int usb_get_all_interfaces(libxl__gc *gc, char *busid,
                                   char ***intfs, int *num)
 {
     DIR *dir;
     struct dirent *entry;
     char *buf;
-    char *busid;
     int rc;
 
     *intfs = NULL;
     *num = 0;
 
-    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
-                                 usb->u.hostdev.hostaddr);
-    if (!busid) {
-        rc = ERROR_FAIL;
-        goto out;
-    }
-
     buf = GCSPRINTF("%s:", busid);
 
     if (!(dir = opendir(SYSFS_USB_DEV))) {
@@ -1114,20 +1115,17 @@ static char *usb_interface_xenstore_encode(char *busid)
  * If there are many interfaces under USB device, check each interface,
  * unbind from "usbback" driver and rebind to its original driver.
  */
-static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
+static int usbback_dev_unassign(libxl__gc *gc, char *busid)
 {
     char **intfs = NULL;
     char *path;
     int num = 0, i;
     int rc;
-    char *busid;
     char *usb_encode = NULL;
 
-    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
+    if (usb_get_all_interfaces(gc, busid, &intfs, &num) < 0)
         return ERROR_FAIL;
 
-    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
-                                 usb->u.hostdev.hostaddr);
     usb_encode = usb_interface_xenstore_encode(busid);
 
     for (i = 0; i < num; i++) {
@@ -1174,11 +1172,15 @@ static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
     char *busid;
     char *usb_encode = NULL;
 
-    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
-        return ERROR_FAIL;
-
     busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
                                  usb->u.hostdev.hostaddr);
+
+    if (!busid)
+        return ERROR_FAIL;
+    
+    if (usb_get_all_interfaces(gc, busid, &intfs, &num) < 0)
+        return ERROR_FAIL;
+
     usb_encode = usb_interface_xenstore_encode(busid);
 
     for (i = 0; i < num; i++) {
@@ -1200,9 +1202,8 @@ static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
             /* write driver path to xenstore for later rebinding */
             path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
                              usb_encode, usb_interface_xenstore_encode(intf));
-            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
-                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
-            }
+            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0)
+                goto out_rebind;
         }
 
         /* bind interface to usbback */
@@ -1220,7 +1221,7 @@ out_rebind:
     /* some interfaces might be bound to usbback, unbind it then and
      * rebind to its original driver
      */
-    usbback_dev_unassign(gc, usb);
+    usbback_dev_unassign(gc, busid);
 out:
     free(usb_encode);
     return rc;
@@ -1376,6 +1377,7 @@ static int do_usb_remove(libxl__gc *gc, uint32_t domid,
     libxl_usbctrlinfo usbctrlinfo;
     libxl_device_usbctrl usbctrl;
     int rc;
+    char *busid;
 
     libxl_device_usbctrl_init(&usbctrl);
     libxl_usbctrlinfo_init(&usbctrlinfo);
@@ -1395,10 +1397,14 @@ static int do_usb_remove(libxl__gc *gc, uint32_t domid,
         LOG(ERROR, "Not supported");
         break;
     case LIBXL_USBCTRL_TYPE_PV:
+        busid = usbback_busid_from_ctrlport(gc, domid, usbdev);
+
+        if (!busid) goto out;
+        
         rc = libxl__device_usb_remove_xenstore(gc, domid, usbdev);
         if (rc) goto out;
 
-        usbback_dev_unassign(gc, usbdev);
+        usbback_dev_unassign(gc, busid);
         break;
     default:
         rc = ERROR_FAIL;
@@ -1432,13 +1438,6 @@ static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
         goto out;
     }
 
-    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV &&
-        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) {
-        LOG(ERROR, "Invalid USB device of hostdev");
-        rc = ERROR_FAIL;
-        goto out;
-    }
-
     /* Do the remove */
     rc = do_usb_remove(gc, domid, usb);
 
-- 
2.1.4


[-- Attachment #3: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-10 17:57       ` George Dunlap
@ 2015-11-10 18:11         ` Ian Jackson
  2015-11-11  7:21           ` Chun Yan Liu
  2015-11-11  2:37         ` Chun Yan Liu
  1 sibling, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-11-10 18:11 UTC (permalink / raw)
  To: George Dunlap
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Chun Yan Liu, xen-devel,
	Jim Fehlig, Simon Cao

George Dunlap writes ("Re: [Xen-devel] [PATCH V8 3/7] libxl: add pvusb API"):
> On Tue, Nov 10, 2015 at 8:41 AM, Chun Yan Liu <cyliu@suse.com> wrote:
> > Which cast? Here, we want to get a uint_8 value, but buf is a string,
> > we need to do atoi.
> 
> atoi() isn't a cast, it's a function call.  The cast is the (char *) bit.
> 
> I think probably you could get away with declaring buf as (char *).
> &buf should be converted to void** automatically without any warnings,
> I think.

No, char** won't be automatically converted to void**.  But void* will
be automatically converted to char*.

(Will read the rest of this thread later, probably tomorrow.)

Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-10 17:57       ` George Dunlap
  2015-11-10 18:11         ` Ian Jackson
@ 2015-11-11  2:37         ` Chun Yan Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-11  2:37 UTC (permalink / raw)
  To: George Dunlap
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/11/2015 at 01:57 AM, in message
<CAFLBxZZU9O7jHm6fLvKZuvd4Lnt1AEcYBSBS_eO__fr0Yw_bOg@mail.gmail.com>, George
Dunlap <George.Dunlap@eu.citrix.com> wrote: 
> On Tue, Nov 10, 2015 at 8:41 AM, Chun Yan Liu <cyliu@suse.com> wrote: 
> >> > +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid, 
> >> > +                               libxl_device_usbctrl *usbctrl, 
> >> > +                               libxl__ao_device *aodev) 
> >> > +{ 
> >> 
> >> Thanks for adjusting the error-handling patterns in these functions. 
> >> The new way is good, except that: 
> >> 
> >> > +out: 
> >> > +    aodev->rc = rc; 
> >> > +    if (rc) aodev->callback(egc, aodev); 
> >> 
> >> Here, rc is always set, and indeed the code would be wrong if it were 
> >> not.  So can you remove the conditional please ?  Ie: 
> > 
> > Reading the codes, libxl__wait_device_connection will call aodev->callback 
> > properly. So here, only if (rc != 0), that means error happens, then we  
> need to 
> > call aodev->callback to end the process. (Refer to current  
> libxl__device_disk_add, 
> > all current code does similar work.) So I think the code is not wrong (?) 
>  
> >> > +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid, 
> >> > +                                  uint8_t *bus, uint8_t *addr) 
> >> > +{ 
> >> > +    char *filename; 
> >> > +    void *buf; 
> >> > + 
> >> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid); 
> >> > +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> >> > +        *bus = atoi((char *)buf); 
> >> 
> >> I don't think this cast (and the one for addr) are necessary ? 
> > 
> > Which cast? Here, we want to get a uint_8 value, but buf is a string, 
> > we need to do atoi. 
>  
> atoi() isn't a cast, it's a function call.  The cast is the (char *) bit. 
>  
> I think probably you could get away with declaring buf as (char *). 
> &buf should be converted to void** automatically without any warnings, 
> I think. 

It will report warning if declaring char *buf.

>  
> >> > +/* Encode usb interface so that it could be written to xenstore as a key. 
> >> > + * 
> >> > + * Since xenstore key cannot include '.' or ':', we'll change '.' to '_', 
> >> > + * change ':' to '-'. For example, 3-1:2.1 will be encoded to 3-1-2_1. 
> >> > + * This will be used to save original driver of USB device to xenstore. 
> >> > + */ 
> >> 
> >> What is the syntax of the incoming busid ?  Could it contain _ or - ? 
> >> You should perhaps spot them and reject if it does. 
> > 
> > The busid is in syntax like 3-1:2.1. It does contain '-'. But since we only  
> use 
> > the single direction, never decode back, so it won't harm. 
>  
> So the only potential problem is that 3-1:2, 3-1-2, 3:1-2, and 3:1:2 
> all collapse down to 3-1-2.  Is there any risk of something like that 
> happening?  (Ian: NB these are all read from sysfs.) 
>  
> Since this isn't really being read by anyone,  maybe it would be 
> better to replace ':' with another character, just to be safe.  It 
> could even be something like 'c' if no other punctuation is available.

OK. Will update.
 
>  
> >> > +/* Bind USB device to "usbback" driver. 
> >> > + * 
> >> > + * If there are many interfaces under USB device, check each interface, 
> >> > + * unbind from original driver and bind to "usbback" driver. 
> >> > + */ 
> >> > +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb) 
> >> > +{ 
> >> ... 
> >> > +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0)  
> { 
> >> > +                LOG(WARN, "Write of %s to node %s failed", drvpath,  
> path); 
> >> > +            } 
> >> 
> >> One of the advantages of libxl__xs_write_checked is that it logs 
> >> errors.  So you can leave that log message out. 
> >> 
> >> However, if the xs write fails, you should presumably stop, rather 
> >> than carrying on. 
> >> 
> >> I think you probably do things in the wrong order here.  You should 
> >> write the old driver info to xenstore first, so that if the bind to 
> >> usbback fails, you have the old driver info. 
>  
> Ian, I don't understand what you're saying here.  The code I'm looking at  
> does: 
> 1. unbind + get old driver path in drvpath 
> 2. if(drvpath) write to xenstore 
> 3. bind to usbback 
>  
> If the bind fails, then we do have drvpath in xenstore.  Or am I 
> missing something? 

If bind to usbback fails, it will goto 'out_rebind', that is we'll try to
rebind it to its original driver and remove xenstore info. 

>  
> Bailing out if the above xenstore write fails seems sensible though. 
>  
> >> > +/* Operation to remove usb device. 
> >> > + * 
> >> > + * Generally, it does: 
> >> > + * 1) check if the usb device is assigned to the domain 
> >> > + * 2) remove the usb device from xenstore controller/port. 
> >> > + * 3) unbind usb device from usbback and rebind to its original driver. 
> >> > + *    If usb device has many interfaces, do it to each interface. 
> >> > + */ 
> >> > +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, 
> >> > +                                    libxl_device_usb *usb) 
> >> > +{ 
> >> > +    int rc; 
> >> > + 
> >> > +    if (usb->ctrl < 0 || usb->port < 1) { 
> >> > +        LOG(ERROR, "Invalid USB device"); 
> >> > +        rc = ERROR_FAIL; 
> >> > +        goto out; 
> >> > +    } 
> >> > + 
> >> > +    if (usb->devtype == LIBXL_USBDEV_TYPE_HOSTDEV && 
> >> > +        (usb->u.hostdev.hostbus < 1 || usb->u.hostdev.hostaddr < 1)) { 
> >> > +        LOG(ERROR, "Invalid USB device of hostdev"); 
> >> > +        rc = ERROR_FAIL; 
> >> > +        goto out; 
> >> > +    } 
> >> 
> >> Why are these checks here rather than in do_usb_remove ? 
> >> 
> >> For that matter, why is this function not the same as do_usb_remove ? 
> >> 
> >> I might ask the same question about do_usb_add and 
> >> libxl__device_usb_add ? 
> > 
> > Using libxl__device_usb_add and within it extracting do_usb_add is to: 
> > * doing some initialization work and necessary check work in 
> >   libxl__device_usb_add (this part are common to PVUSB and QEMU USB) 
> > * doing actual adding usb operations in do_usb_add (this part will diverge 
> >   for PVUSB and QEMU USB) . 
> > 
> > Same to libxl__device_usb_remove and do_usb_remove. 
> > 
> > Certainly, we could merge into one function if that is better. George may 
> > have some opinions on this? 
>  
> I think it would be nice to have things broken down that way, but it's 
> likely I'll have to do some adjustments when I come to add in 
> devicemodel usb anyway.  So making it one big function wouldn't hurt. 
> (Nor would leaving it the way it is, as far as I'm concerned.)

Got it. Will update.
 
>  
> The big thing though is with the removal interface here, where we 
> again have somewhat of a confusion between naming things based on the 
> *host* name (as we do in pci pass-through) vs the *guest* names (as we 
> do for disks and nics). 
>  
> If you have the controller and virtual port, there's no need for the 
> caller to *also* go and look up the host bus and host address; or vice 
> versa -- if you have the hostbus and hostaddr, you shouldn't also need 
> to go look up the usbctrl and virtual port. 
>  
> It looks like for libxl_device_{disk,nic,vtpm}_remove(), it gets a 
> libxl_device struct using libxl__device_from_{disk,nic,vtpm}; and in 
> all cases, the information used to construct the "device" is the 
> backend + either a devid (for nic and vtpm) or vdev, for disks.  It 
> looks like for those devices, any conversion from other labels (mac 
> address for nic, uuid for vtpm) is done in xl_cmdimpli.c. 
>  
> I think we want to follow suit here -- libxl_device_usb_remove() 
> should take a usbctrl and port number only, and should look up 
> whatever information it needs to do the removal from that. 
>  
> All we really need for the re-assignment is the busid, which is 
> already stored in a xenstore location we can find using domid, ctrl, 
> and port.  Something like the attached patch (compile-tested only). 
> What do you think? 

Thanks. Much better. Change interface to use 'busid' instead of
'struct usb' usbback_dev_assign and usbback_dev_unassign makes things
much cleaner.

>  
> (NB the patch doesn't fix the gc or aliasing issues in 
> usb_interface_xenstore_encode() -- those still need to be addressed.)

I'll take care of the left.

- Chunyan
 
>  
>  -George 
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-10 18:11         ` Ian Jackson
@ 2015-11-11  7:21           ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-11  7:21 UTC (permalink / raw)
  To: George Dunlap, Ian Jackson
  Cc: Juergen Gross, Wei Liu, Ian Campbell, xen-devel, Jim Fehlig, Simon Cao



>>> On 11/11/2015 at 02:11 AM, in message
<22082.13115.276320.572627@mariner.uk.xensource.com>, Ian Jackson
<Ian.Jackson@eu.citrix.com> wrote: 
> George Dunlap writes ("Re: [Xen-devel] [PATCH V8 3/7] libxl: add pvusb API"): 
> > On Tue, Nov 10, 2015 at 8:41 AM, Chun Yan Liu <cyliu@suse.com> wrote: 
> > > Which cast? Here, we want to get a uint_8 value, but buf is a string, 
> > > we need to do atoi. 
> >  
> > atoi() isn't a cast, it's a function call.  The cast is the (char *) bit. 
> >  
> > I think probably you could get away with declaring buf as (char *). 
> > &buf should be converted to void** automatically without any warnings, 
> > I think. 
>  
> No, char** won't be automatically converted to void**.  But void* will 
> be automatically converted to char*. 

Yes, that's right.

>  
> (Will read the rest of this thread later, probably tomorrow.) 
>  
> Ian. 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API
  2015-10-21  9:08 ` [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API Chunyan Liu
  2015-10-27 11:32   ` Juergen Gross
@ 2015-11-11 16:07   ` George Dunlap
  1 sibling, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-11 16:07 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, xen-devel, Jim Fehlig

On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add API for listing assignable USB devices info.
> Assignable USB device means the USB device type is assignable and
> it's not assigned to any guest yet.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>

So going back to what I wrote about general usefulness, I am more
firmly convinced that xl should use libusb to provide the useful user
interface features we've been discussing.  As such, I think this patch
is no longer necessary and should be dropped.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
                     ` (2 preceding siblings ...)
  2015-11-09 18:11   ` Ian Jackson
@ 2015-11-12 11:32   ` Olaf Hering
  2015-11-13  2:32     ` Chun Yan Liu
  2015-11-12 17:27   ` George Dunlap
  2015-11-13 11:19   ` Olaf Hering
  5 siblings, 1 reply; 53+ messages in thread
From: Olaf Hering @ 2015-11-12 11:32 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	xen-devel, jfehlig, Simon Cao

On Wed, Oct 21, Chunyan Liu wrote:

> Add pvusb APIs, including:

Some comments below.

After a quick look I miss the proposed ctrl/device separation for pvscsi
(what handles "state" changes?). But, I have to read all the other
dozen+ threads about that topic first.


> +    flexarray_append_pair(back, "state", "1");

4.6+ has macros for "state" values, like
flexarray_append_pair(back, "state", GCSPRINTF("%d", XenbusStateInitialising));


> +    flexarray_append_pair(front, "state", "1");

4.6+ has macros for "state" values.

> +        LOG(DEBUG, "Adding new usb device to xenstore");

Which one? Perhaps print also details.

> +    LOG(DEBUG, "Removing USB device from xenstore");

Which one? Perhaps print also details.

> +        /* check if the USB interface is already bound to "usbbcak" */

Typo.


Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-10-21  9:08 ` [PATCH V8 5/7] xl: add pvusb commands Chunyan Liu
  2015-10-27 11:37   ` Juergen Gross
@ 2015-11-12 11:38   ` George Dunlap
  2015-11-12 11:39     ` George Dunlap
  2015-11-13  2:43     ` Chun Yan Liu
  2015-11-12 14:42   ` Olaf Hering
  2 siblings, 2 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-12 11:38 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, xen-devel, Jim Fehlig,
	Simon Cao

On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list,
> usb-attach and usb-detach.
>
> To attach a usb device to guest through pvusb, one could follow
> following example:
>
>  #xl usbctrl-attach test_vm version=1 ports=8
>
>  #xl usb-list test_vm
>  will show the usb controllers and port usage under the domain.
>
>  #xl usbattach test_vm hostbus=1 hostaddr=2

Nit: usb-attach (missing a '-')

>  will find the first usable controller:port, and attach usb
>  device whose busnum is 1 and devnum is 6.
>  One could also specify which <controller> and which <port>.
>
>  #xl usb-detach test_vm 0 1
>  will detach USB device under controller 0 port 1.
>
>  #xl usbctrl-detach test_vm dev_id
>  will destroy the controller with specified dev_id. Dev_id
>  can be traced in usb-list info.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>
> ---
> Changes:
>   - change usb-attach parameter from hostbus.hostaddr to
>     hostbus=xx hostaddr=
>   - since we get rid of libxl_device_usb_getinfo, so adjust usb-list
>     information a little bit.
>   - parse_usb_config and parse_usbctrl_config following parse_nic_config
>     way, put in this patch, and shared domcreate routine.
>
>  docs/man/xl.pod.1         |  40 ++++++++
>  tools/libxl/xl.h          |   5 +
>  tools/libxl/xl_cmdimpl.c  | 250 ++++++++++++++++++++++++++++++++++++++++++++++
>  tools/libxl/xl_cmdtable.c |  25 +++++
>  4 files changed, 320 insertions(+)
>
> diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
> index d0cd612..f09a872 100644
> --- a/docs/man/xl.pod.1
> +++ b/docs/man/xl.pod.1
> @@ -1345,6 +1345,46 @@ List pass-through pci devices for a domain.
>
>  =back
>
> +=head1 USB PASS-THROUGH
> +
> +=over 4
> +
> +=item B<usbctrl-attach> I<domain-id> I[<type=val>] [I<version=val>] [I<ports=number>]
> +
> +Create a new USB controller for the specified domain.
> +B<type=val> is the usb controller type, currently only support 'pv'.

"B<type=val> is the usb controller type.  Currently only 'pv' and
'auto' are supported."

Note the period rather than the comma.

(See below on how to make this true.)

> +B<version=val> is the usb controller version, could be 1 (USB1.1) or 2 (USB2.0).

"<version=val> is the usb controller version.  Possible values include
1 (USB1.1) and 2 (USB2.0)."

(Same thing wrt the period.)

> +B<ports=number> is the total ports of the usb controller.

Is there a maximum number of ports?

> +By default, it will create a USB2.0 controller with 8 ports.
> +
> +=item B<usbctrl-detach> I<domain-id> I<devid>
> +
> +Destroy a USB controller from the specified domain.
> +B<devid> is devid of the USB controller.
> +
> +If B<-f> is specified, B<xl> is going to forcefully remove the device even
> +without guest's collaboration.

"If B<-f> is specified, B<xl> will forcefully remove removal (i.e.,
without the guest's cooperation)."

> +
> +=item B<usb-attach> I<domain-id> I<hostbus=busnum> I<hostaddr=devnum> [I<controller=devid> [I<port=number>]]
> +
> +Hot-plug a new pass-through USB device to the specified domain.
> +B<bus.addr> is the busnum.devnum of the physical USB device to pass-through.

"hostbus and hostaddr are the bus and device number of the physical
USB device to pass through."

> +B<controller=devid> B<port=number> is the USB controller:port to hotplug the
> +USB device to. By default, it will find the first available controller:port
> +and use it; if there is no controller, it will create one.
> +
> +=item B<usb-detach> I<domain-id> I<controller=devid> I<port=number>
> +
> +Hot-unplug a previously assigned USB device from a domain.
> +B<controller=devid> and B<port=number> is USB controller:port in guest where the
> +USB device is attached to.
> +
> +=item B<usb-list> I<domain-id>
> +
> +List pass-through usb devices for a domain.
> +
> +=back
> +
>  =head1 TMEM
>
>  =over 4
> diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
> index 0021112..ddd9690 100644
> --- a/tools/libxl/xl.h
> +++ b/tools/libxl/xl.h
> @@ -85,6 +85,11 @@ int main_blockdetach(int argc, char **argv);
>  int main_vtpmattach(int argc, char **argv);
>  int main_vtpmlist(int argc, char **argv);
>  int main_vtpmdetach(int argc, char **argv);
> +int main_usbctrl_attach(int argc, char **argv);
> +int main_usbctrl_detach(int argc, char **argv);
> +int main_usbattach(int argc, char **argv);
> +int main_usbdetach(int argc, char **argv);
> +int main_usblist(int argc, char **argv);
>  int main_uptime(int argc, char **argv);
>  int main_claims(int argc, char **argv);
>  int main_tmem_list(int argc, char **argv);
> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 365798b..e6ff6f4 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -1255,6 +1255,64 @@ static void parse_vnuma_config(const XLU_Config *config,
>      free(vcpu_parsed);
>  }
>
> +/* Parses usbctrl data and adds info into usbctrl
> + * Returns 1 if the input token does not match one of the keys
> + * or parsed values are not correct. Successful parse returns 0 */
> +static int parse_usbctrl_config(libxl_device_usbctrl *usbctrl, char *token)
> +{
> +    char *oparg;
> +
> +    if (MATCH_OPTION("type", token, oparg)) {
> +        if (!strcmp("pv", oparg)) {
> +            usbctrl->type = LIBXL_USBCTRL_TYPE_PV;
> +        } else {
> +            fprintf(stderr,
> +                    "Unsupported USB controller type '%s'\n", oparg);
> +            return 1;
> +        }

One of the reasons for making the enum types in the IDL is that the
IDL then automatically generates functions for converting to/from
strings.

So here I would do something like:

if(libxl_usbctrl_type_from_string(optarg, &usbctrl->type)) {
 fprintf(stderr, "Invalid usbctrl type: %s\n", optarg);
 exit(1);
}

(Just to emphasize -- you don't need to write the above function; it's
generated automatically.)


> +/* Parses usb data and adds info into usb
> + * Returns 1 if the input token does not match one of the keys
> + * or parsed values are not correct. Successful parse returns 0 */
> +static int parse_usb_config(libxl_device_usb *usb, char *token)
> +{
> +    char *oparg;
> +
> +    if (MATCH_OPTION("devtype", token, oparg)) {
> +        if (!strcmp("hostdev", oparg)) {
> +            usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV;

Same thing here, with libxl_usbdev_type_to_string().


> +int main_usbctrl_attach(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    int opt, rc = 0;
> +    libxl_device_usbctrl usbctrl;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usbctrl-attach", 1) {
> +        /* No options */
> +    }
> +
> +    domid = find_domain(argv[optind++]);
> +
> +    libxl_device_usbctrl_init(&usbctrl);
> +
> +    for (argv += optind, argc -= optind; argc > 0; ++argv, --argc) {
> +        if (parse_usbctrl_config(&usbctrl, *argv))
> +            return 1;
> +    }
> +
> +    rc = libxl_device_usbctrl_add(ctx, domid, &usbctrl, 0);
> +    if (rc) {
> +        fprintf(stderr, "libxl_device_usbctrl_add failed.\n");
> +        rc = 1;
> +    }
> +
> +    libxl_device_usbctrl_dispose(&usbctrl);
> +    return rc;
> +}
> +
> +int main_usbctrl_detach(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    int opt, devid, rc;
> +    libxl_device_usbctrl usbctrl;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usbctrl-detach", 2) {
> +        /* No options */
> +    }
> +
> +    domid = find_domain(argv[optind]);
> +    devid = atoi(argv[optind+1]);
> +
> +    libxl_device_usbctrl_init(&usbctrl);
> +    if (libxl_devid_to_device_usbctrl(ctx, domid, devid, &usbctrl)) {
> +        fprintf(stderr, "Unknown device %s.\n", argv[optind+1]);
> +        return 1;
> +    }
> +
> +    rc = libxl_device_usbctrl_remove(ctx, domid, &usbctrl, 0);
> +    if (rc) {
> +        fprintf(stderr, "libxl_device_usbctrl_remove failed.\n");
> +        rc = 1;
> +    }
> +
> +    libxl_device_usbctrl_dispose(&usbctrl);
> +    return rc;
> +
> +}
> +
> +int main_usbattach(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    int opt, rc;
> +    libxl_device_usb usb;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-attach", 2) {
> +        /* No options */
> +    }
> +
> +    libxl_device_usb_init(&usb);
> +
> +    domid = find_domain(argv[optind++]);
> +
> +    for (argv += optind, argc -= optind; argc > 0; ++argv, --argc) {
> +        if (parse_usb_config(&usb, *argv))
> +            return 1;
> +    }
> +
> +    rc = libxl_device_usb_add(ctx, domid, &usb, 0);
> +    if (rc) {
> +        fprintf(stderr, "libxl_device_usb_add failed.\n");
> +        rc = 1;
> +    }
> +
> +    libxl_device_usb_dispose(&usb);
> +    return rc;
> +}
> +
> +int main_usbdetach(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    int ctrl, port;
> +    int opt, rc = 1;
> +    libxl_device_usb usb;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-detach", 3) {
> +        /* No options */
> +    }
> +
> +    domid = find_domain(argv[optind]);
> +    ctrl = atoi(argv[optind+1]);
> +    port = atoi(argv[optind+2]);
> +
> +    if (argc - optind > 3) {
> +        fprintf(stderr, "Invalid arguments.\n");
> +        return 1;
> +    }
> +
> +    libxl_device_usb_init(&usb);
> +    if (libxl_ctrlport_to_device_usb(ctx, domid, ctrl, port, &usb)) {
> +        fprintf(stderr, "Unknown device at controller %d port %d.\n",
> +                ctrl, port);
> +        return 1;
> +    }
> +
> +    rc = libxl_device_usb_remove(ctx, domid, &usb, 0);
> +    if (rc) {
> +        fprintf(stderr, "libxl_device_usb_remove failed.\n");
> +        rc = 1;
> +    }
> +
> +    libxl_device_usb_dispose(&usb);
> +    return rc;
> +}
> +
> +int main_usblist(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    libxl_device_usbctrl *usbctrls;
> +    libxl_usbctrlinfo usbctrlinfo;
> +    int numctrl, i, j, opt;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-list", 1) {
> +        /* No options */
> +    }
> +
> +    domid = find_domain(argv[optind++]);
> +
> +    if (argc > optind) {
> +        fprintf(stderr, "Invalid arguments.\n");
> +        exit(-1);
> +    }
> +
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> +    if (!usbctrls) {
> +        return 0;
> +    }
> +
> +    for (i = 0; i < numctrl; ++i) {
> +        printf("%-6s %-6s %-3s %-5s %-7s %-5s %-30s\n",
> +                "Devid", "Type", "BE", "state", "usb-ver", "ports", "BE-path");
> +
> +        libxl_usbctrlinfo_init(&usbctrlinfo);
> +
> +        if (!libxl_device_usbctrl_getinfo(ctx, domid,
> +                                &usbctrls[i], &usbctrlinfo)) {
> +            printf("%-6d %-6s %-3d %-5d %-7d %-5d %-30s\n",
> +                    usbctrlinfo.devid,
> +                    libxl_usbctrl_type_to_string(usbctrlinfo.type),

Oh, I see you already know about the "type_to_string()" functions. :-)

> +                    usbctrlinfo.backend_id, usbctrlinfo.state,
> +                    usbctrlinfo.version, usbctrlinfo.ports,
> +                    usbctrlinfo.backend);
> +
> +            for (j = 1; j <= usbctrlinfo.ports; j++) {
> +                libxl_device_usb usb;
> +                libxl_usbinfo usbinfo;
> +
> +                libxl_device_usb_init(&usb);
> +                libxl_usbinfo_init(&usbinfo);
> +
> +                printf("  Port %d:", j);
> +
> +                if (!libxl_ctrlport_to_device_usb(ctx, domid,
> +                                                  usbctrlinfo.devid, j, &usb)) {
> +                    printf(" Bus %03x Device %03x\n",
> +                           usbinfo.busnum, usbinfo.devnum);
> +                } else {
> +                    printf("\n");
> +                }
> +
> +                libxl_usbinfo_dispose(&usbinfo);

Er, what's going on with the usbinfo?  Did you take the usbinfo stuff
out of libxl_pvusb.c, but forget to take it out of libxl_types.idl and
here?

I guess you really want usb.u.hostdev.* here.

Everything else looks good, thanks.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-11-12 11:38   ` George Dunlap
@ 2015-11-12 11:39     ` George Dunlap
  2015-11-13  2:43     ` Chun Yan Liu
  1 sibling, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-12 11:39 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, xen-devel, Jim Fehlig,
	Simon Cao

On Thu, Nov 12, 2015 at 11:38 AM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
>> +/* Parses usb data and adds info into usb
>> + * Returns 1 if the input token does not match one of the keys
>> + * or parsed values are not correct. Successful parse returns 0 */
>> +static int parse_usb_config(libxl_device_usb *usb, char *token)
>> +{
>> +    char *oparg;
>> +
>> +    if (MATCH_OPTION("devtype", token, oparg)) {
>> +        if (!strcmp("hostdev", oparg)) {
>> +            usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV;
>
> Same thing here, with libxl_usbdev_type_to_string().

Obviously I mean "type_from_string" here.

 -G

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 6/7] xl: add usb-assignable-list command
  2015-10-21  9:08 ` [PATCH V8 6/7] xl: add usb-assignable-list command Chunyan Liu
  2015-10-27 11:38   ` Juergen Gross
@ 2015-11-12 11:44   ` George Dunlap
  1 sibling, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-12 11:44 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, xen-devel, Jim Fehlig

On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add xl usb-assignable-list command to list assignable USB devices.
> Assignable USB device means the USB device type is assignable and
> it's not assigned to any guest yet.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>
> ---
>   Same as "libxl: add libxl_device_usb_assignable_list API" patch,
>   this patch could be sqaushed to previous one. Split because of
>   some dispute. Could be squashed if acceptable, otherwise could
>   be removed.

I think the functionality is useful, but it should use libusb instead.
As such I'd probably send this in a follow-up series.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-10-21  9:08 ` [PATCH V8 5/7] xl: add pvusb commands Chunyan Liu
  2015-10-27 11:37   ` Juergen Gross
  2015-11-12 11:38   ` George Dunlap
@ 2015-11-12 14:42   ` Olaf Hering
  2015-11-12 14:49     ` George Dunlap
  2015-11-13  2:49     ` Chun Yan Liu
  2 siblings, 2 replies; 53+ messages in thread
From: Olaf Hering @ 2015-11-12 14:42 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	xen-devel, jfehlig, Simon Cao

On Wed, Oct 21, Chunyan Liu wrote:

> Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list,
> usb-attach and usb-detach.

How is this supposed to be handled in libvirt? It looks like libvirt has
to copy what is done here. If thats true the logic should be in libxlu
so that both xl and libvirt can call the same functions.

Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-11-12 14:42   ` Olaf Hering
@ 2015-11-12 14:49     ` George Dunlap
  2015-11-13  2:49     ` Chun Yan Liu
  1 sibling, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-12 14:49 UTC (permalink / raw)
  To: Olaf Hering
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, Chunyan Liu, xen-devel,
	Jim Fehlig, Simon Cao

On Thu, Nov 12, 2015 at 2:42 PM, Olaf Hering <olaf@aepfle.de> wrote:
> On Wed, Oct 21, Chunyan Liu wrote:
>
>> Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list,
>> usb-attach and usb-detach.
>
> How is this supposed to be handled in libvirt? It looks like libvirt has
> to copy what is done here. If thats true the logic should be in libxlu
> so that both xl and libvirt can call the same functions.

What logic are you talking about?  The usb code in xl mirrors all the
other code (nic, disk, &c).

In any case, I think it would be the person writing the libvirt
bindings who would be responsible for abstracting things like that
(just as I will have to do a bit of re-work to get the HVM usb stuff
implemented).

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 7/7] domcreate: support pvusb in configuration file
  2015-10-21  9:08 ` [PATCH V8 7/7] domcreate: support pvusb in configuration file Chunyan Liu
  2015-10-27 11:41   ` Juergen Gross
@ 2015-11-12 16:10   ` George Dunlap
  2015-11-13  2:54     ` Chun Yan Liu
  1 sibling, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-11-12 16:10 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, xen-devel, Jim Fehlig,
	Simon Cao

On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add code to support pvusb in domain config file. One could specify
> usbctrl and usb in domain's configuration file and create domain,
> then usb controllers will be created and usb device would be attached
> to guest automatically.
>
> One could specify usb controllers and usb devices in config file
> like this:
> usbctrl=['version=2,ports=4', 'version=1, ports=4', ]
> usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ]
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>
> ---
> changes:
>   - change parse_usb_config and parse_usbctrl_config, following
>     parse_nic_config way, and move to previous patch
>   - update user interface to specify hostbus, hostaddr instead of
>     hostbus.hostaddr for future extension
>
>  docs/man/xl.cfg.pod.5        | 81 ++++++++++++++++++++++++++++++++++++++++++++
>  tools/libxl/libxl_create.c   | 73 +++++++++++++++++++++++++++++++++++++--
>  tools/libxl/libxl_device.c   |  4 +++
>  tools/libxl/libxl_internal.h |  8 +++++
>  tools/libxl/xl_cmdimpl.c     | 54 ++++++++++++++++++++++++++++-
>  5 files changed, 216 insertions(+), 4 deletions(-)
>
> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index b63846a..f24c008 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -722,6 +722,87 @@ Note this may be overridden by rdm_policy option in PCI device configuration.
>
>  =back
>
> +=item B<usbctrl=[ "USBCTRL_SPEC_STRING", "USBCTRL_SPEC_STRING", ... ]>
> +
> +Specifies the USB controllers created for this guest. Each
> +B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where:
> +
> +=over 4
> +
> +=item B<KEY=VALUE>
> +
> +Possible B<KEY>s are:
> +
> +=over 4
> +
> +=item B<type=TYPE>
> +
> +Specifies the protocol to implement USB controller, could be "pv" (indicates
> +PVUSB) or "qemu" (indicates QEMU emulated). Currently only "pv" is supported.

I think most of these should basically be copied from the descriptions
of the arguments in patch 5.

> +=item B<version=VERSION>
> +
> +Specifies version of the USB controller, could be 1 (USB1.1) or 2 (USB2.0).
> +Default is 2 (USB2.0).
> +
> +=item B<ports=PORTS>
> +
> +Specifies port number of the USB controller. Default is 8.
> +
> +Each USB controller will have an index starting from 0. On the same
> +controller, each port will have an index starting from 1.

I'd say:

"USB controler ids start from 0.  In line with the USB spec, however,
ports on a controller start from 1."

> +
> +E.g.
> +usbctrl=["version=1,ports=4", "version=2,ports=8",]
> +The first controller has:
> +controller index = 0, and port 1,2,3,4.
> +The second controller has:
> +controller index = 1, and port 1,2,3,4,5,6,7,8.

The example is good, but I'd use "controller id" rather than "controller index".

> +=back
> +
> +=back
> +
> +=item B<usbdev=[ "USB_SPEC_STRING", "USB_SPEC_STRING", ... ]>
> +
> +Specifies the host USB devices to passthrough to this guest. Each
> +B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where:

"Specifiec the USB devices to be attached to the guest at boot."
(i.e., don't assume all devices are hostdev.)

> +=over 4
> +
> +=item B<KEY=VALUE>
> +
> +Possible B<KEY>s are:
> +
> +=over 4
> +
> +=item B<devtype=hostdev>
> +
> +Specifies USB device type. Currently only support 'hostdev'.
> +
> +=item B<hostbus=busnum>
> +
> +Specifies busnum of the USB device from the host perspective.
> +
> +=item B<hostaddr=devnum>
> +
> +Specifies devnum of the USB device from the host perspective.
> +
> +=item B<controller=CONTROLLER>
> +
> +Specifies USB controller index, to which controller the USB device is attached.
> +
> +=item B<port=PORT>
> +
> +Specifies USB port index, to which port the USB device is attached. B<port=PORT>
> +is valid only when B<controller=CONTROLLER> is specified. Without
> +B<controller=CONTROLLER>, it will find the first available USB controller:port
> +and use it. If there is no controller at all, it will create one.

I think the last sentence should be a separate paragraph, probably
after the "=back".  And a more idiomatic way to write it might be:

"If no controller is specified, an available controller:port
combination will be used.  If there are no available controller:port
options, a new controller will be created."

Only one other minor comment from me:

> +    if (!xlu_cfg_get_list(config, "usbctrl", &usbctrls, 0, 0)) {
> +        d_config->num_usbctrls = 0;
> +        d_config->usbctrls = NULL;
> +        while ((buf = xlu_cfg_get_listitem(usbctrls, d_config->num_usbctrls))
> +               != NULL) {

I don't know what the other maintainers think, but particularly given
that this is spanning a line, I would personally take out the
comparison, and just make it

 while ((buf = ...)) {
 ...
}

Other than that, this one looks good to me.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-10  8:41     ` Chun Yan Liu
  2015-11-10 17:57       ` George Dunlap
@ 2015-11-12 17:00       ` Ian Jackson
  2015-11-13  2:30         ` Chun Yan Liu
  1 sibling, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-11-12 17:00 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	Jim Fehlig, Simon Cao

Chun Yan Liu writes ("Re: [PATCH V8 3/7] libxl: add pvusb API"):
> On 11/10/2015 at 02:11 AM, in message
> <22080.57829.461049.37192@mariner.uk.xensource.com>, Ian Jackson
> <Ian.Jackson@eu.citrix.com> wrote: 
> > Chunyan Liu writes ("[PATCH V8 3/7] libxl: add pvusb API"): 
> > > +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid, 
> > > +                               libxl_device_usbctrl *usbctrl, 
> > > +                               libxl__ao_device *aodev) 
> > > +{ 
> >  
> > Thanks for adjusting the error-handling patterns in these functions. 
> > The new way is good, except that: 
> >  
> > > +out: 
> > > +    aodev->rc = rc; 
> > > +    if (rc) aodev->callback(egc, aodev); 
> >  
> > Here, rc is always set, and indeed the code would be wrong if it were 
> > not.  So can you remove the conditional please ?  Ie: 
> 
> Reading the codes, libxl__wait_device_connection will call
> aodev->callback properly.

Indeed.  So you need to call aodev->callback() iff you don't call
libxl__wait_device_connection.  But libxl__wait_device_connection is
called only on the success exit path which ends in `return', not in
`goto out'.  So:

> So here, only if (rc != 0), that means
> error happens, then we need to call aodev->callback to end the
> process. (Refer to current libxl__device_disk_add, all current code
> does similar work.) So I think the code is not wrong (?)

In the `goto out' path, rc is always set.  Writing `if (rc)' implies
that it might not be (or is allowed not to be), which is misleading.


> > > +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid, 
> > > +                                  uint8_t *bus, uint8_t *addr) 
> > > +{ 
> > > +    char *filename; 
> > > +    void *buf; 
> > > + 
> > > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid); 
> > > +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> > > +        *bus = atoi((char *)buf); 
> >  
> > I don't think this cast (and the one for addr) are necessary ? 
> 
> Which cast? Here, we want to get a uint_8 value, but buf is a string,
> we need to do atoi.

I mean that I think

     +        *bus = atoi(buf); 

would compile and be correct.  buf would be automatically converted
from void* to const char*.  It's better to avoid casts where they are
not needed, because casts will suppress compiler warnings.

For example if someone wanted to have the buffer be an updated
parameter they might do this:

  static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid, 
+                                   void **buf,
                                    uint8_t *bus, uint8_t *addr)
-    void *buf; 

and hope or expect the compiler to notice places where they had failed
to update the usage of buf.  With void **buf,
  atoi((char*)buf)
would compile and do a wrong thing whereas
  atoi(buf)
would produce a fatal warning.

> > > +/* bind/unbind usb device interface */ 
> > > +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath) 
> > > +{ 
...
> > > +        dp = libxl__zalloc(gc, PATH_MAX); 
> > > +        dp = realpath(spath, dp); 
> >  
> > Why is this call to realpath needed ?
> 
> In sysfs, /driver sometimes is a link, since we need to save the original
> driver to xenstore, so need to get the realpath of the driver.

I mean, why is the path with all the symlinks in it not suitable for
storage and later use ?


> > > +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb) 
> > > +{ 
> > ... 
> > > +        /* bind interface to its originial driver */ 
> > > +        drvpath = libxl__xs_read(gc, XBT_NULL, 
> > > +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path", 
> > > +                  usb_encode, usb_interface_xenstore_encode(intf))); 
> > > +        if (drvpath && bind_usb_intf(gc, intf, drvpath)) 
> > > +            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath); 
> >  
> > This error message could be clearer. 
> >  
> > Also it would be worth a comment in the code to explain why this is a 
> > warning (merely logged), rather than an error (logged and reported 
> > with nonzero status to caller).
> 
> Will update. When doing USB remove, it's the ideal result we can rebound
> the USB to its original driver, in case the USB interface failed to be
> rebound to its original driver, we will export warning but won't stop the
> removing process.

Right.  Please put that in a comment :-).


> > > +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb) 
> > > +{ 
> > ... 
> > > +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) { 
> > > +                LOG(WARN, "Write of %s to node %s failed", drvpath, path); 
> > > +            } 
> >  
> > One of the advantages of libxl__xs_write_checked is that it logs 
> > errors.  So you can leave that log message out. 
> >  
> > However, if the xs write fails, you should presumably stop, rather 
> > than carrying on. 
> > 
> > I think you probably do things in the wrong order here.  You should 
> > write the old driver info to xenstore first, so that if the bind to 
> > usbback fails, you have the old driver info. 
> 
> Perhaps not. Once we finished adding entries to xenstore, pvusb
> frontend/backend drivers will detect the change and do probe work, if
> USB device is still not bound to USBBACK, there might be error.

What I mean is this:

Is it not possible to write the original path to xenstore before doing
the unbind ?  Otherwise it seems like there could be error paths where
the original path is not recorded, the xenstore write fails, and then
the information about how to rebind to the original driver has been
lost.

Does writing USBBACK_INFO_PATH"/%s/%s/driver_path" really trigger
usbback ?

> > > +    free(usb_encode); 
> >  
> > This came from the gc, I think. 
...
> > > +    switch (usbctrlinfo.type) { 
> > > +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL: 
> > > +        LOG(ERROR, "Not supported"); 
> > > +        break; 
> >  
> > Needs to set rc. 

(Did you spot these comments ?  Normally I wouldn't even expect you to
quote the comments that you are just going to fix, but since you have
replied to the others, I thought I would check.)


> > > +void libxl__device_usb_add(libxl__egc *egc, uint32_t domid, 
> > > +                           libxl_device_usb *usb, 
> > > +                           libxl__ao_device *aodev) 
> > > +{ 
> > ... 
> > > +        if (usbctrlinfo.backend_id != 0) { 
> >  
> > LIBXL_TOOLSTACK_DOMID please.  (And in remove, too.) 
> >  
> > Why is this check done in libxl__device_usb_add and do_usb_remove, 
> > rather than in a symmetrical way?
>  
> In libxl__device_usb_add, the set_default function implementation won't
> work if backend domain is USB driver domain in future, so we must add
> a check before set_default, so in libxl__device_usb_add. To check backend
> domain, we need to get usbctrl info. And in do_usb_add, to check usbctrl
> type, again we need to get usbctrl info. It's disgusting to do the same thing
> twice.
> 
> In usb remove, nothing in libxl__device_usb_remove is Dom0 specific, so
> I do all check in do_usb_remove, then only one time to get usbctrl info.

Right.

> If we merge libxl__device_usb_add and do_usb_add, then it's cleaner.

See my comment below.  You've explained the distinction to my
satisfaction.

But, to solve the duplication of the controller info acquisition,
perhaps you could have do_usb_add take the controller info as a
paramaeter ?


> > > +    /* Do the add */ 
> > > +    if (do_usb_add(gc, domid, usb, aodev)) { 
> > > +        rc = ERROR_FAIL; 
> > > +        goto out; 
> > > +    } 
> > > + 
> > > +    libxl__ao_complete(egc, ao, 0); 
> > > +    rc = 0; 
> > > + 
> > > +out: 
> > > +    libxl_device_usbctrl_dispose(&usbctrl); 
> > > +    libxl_usbctrlinfo_dispose(&usbctrlinfo); 
> > > +    aodev->rc = rc; 
> > > +    if (rc) aodev->callback(egc, aodev); 
> > > +    return; 
> >  
> > Calling libxl__ao_complete here is quite wrong.  I think you should 
> > call aodev->callback in both success and failure cases.  And it must 
> > be the last thing you do. 
> 
> As mentioned in libxl__device_usbctrl_add, libxl__ao_complete will
> call callback, so in 'out' section, only if (rc !=0), which means
> error happens, then we need to call aodev->callback explicitly to
> end the process.
> 
> Here, since we don't have actual async work needs to do, so call
> libxl__ao_complete(egc, ao, 0) directly to end the process.

I'm afraid this is definiitely wrong.  There are a number of things
wrong with it, or a number of ways of looking at it:

Most obviously, a single entrypoint should always complete its
execution the same way.  In this case the entrypoint takes an aodev
and in the error path calls aodev->callback().  So it should do the
same on the success path.

Secondly (and relatedly), it is generally wrong for anything inside a
particular piece of machinery to terminate an ao.

In this particular case I think it leads to a use-after-free bug:
completing the ao will (maybe) free the ao, and aodev was allocated
from the ao's gc.  So the write to aodev->rc is improper.


> > I might ask the same question about do_usb_add and 
> > libxl__device_usb_add ? 
> 
> Using libxl__device_usb_add and within it extracting do_usb_add is to:
> * doing some initialization work and necessary check work in
>   libxl__device_usb_add (this part are common to PVUSB and QEMU USB)
> * doing actual adding usb operations in do_usb_add (this part will diverge
>   for PVUSB and QEMU USB) .

This is a good reason for keeping them split.  Thanks for the
explanation.


Thanks for your other replies.

Regards,
Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
                     ` (3 preceding siblings ...)
  2015-11-12 11:32   ` Olaf Hering
@ 2015-11-12 17:27   ` George Dunlap
  2015-11-13  2:56     ` Chun Yan Liu
  2015-11-13 11:19   ` Olaf Hering
  5 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-11-12 17:27 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Jürgen Groß,
	Wei Liu, Ian Campbell, Ian Jackson, xen-devel, Jim Fehlig,
	Simon Cao

On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote:
> +static int
> +get_assigned_devices(libxl__gc *gc,
> +                     libxl_device_usb **list, int *num)
> +{
> +    char **domlist;
> +    unsigned int nd = 0, i, j, k;
> +    int rc;
> +
> +    *list = NULL;
> +    *num = 0;
> +
> +    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd);
> +    for (i = 0; i < nd; i++) {
> +        char *path, **ctrl_list;
> +        unsigned int nc = 0;
> +
> +        path = GCSPRINTF("/local/domain/%s/device/vusb", domlist[i]);
> +        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path, &nc);
> +
> +        for (j = 0; j < nc; j++) {
> +            const char *be_path, *num_ports;
> +            int nport;
> +
> +            rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                          GCSPRINTF("%s/%s/backend", path, ctrl_list[j]),
> +                          &be_path);
> +            if (rc) goto out;
> +
> +            rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                                        GCSPRINTF("%s/num-ports", be_path),
> +                                        &num_ports);
> +            if (rc) goto out;
> +
> +            nport = atoi(num_ports);
> +            for (k = 0; k < nport; k++) {
> +                libxl_device_usb *usb;
> +                const char *portpath, *busid;
> +
> +                portpath = GCSPRINTF("%s/port/%d", be_path, k + 1);
> +                busid = libxl__xs_read(gc, XBT_NULL, portpath);
> +                /* If there is USB device attached, add it to list */
> +                if (busid && strcmp(busid, "")) {
> +                    GCREALLOC_ARRAY(*list, *num + 1);
> +                    usb = *list + *num;
> +                    (*num)++;
> +                    libxl_device_usb_init(usb);
> +                    usb->ctrl = atoi(ctrl_list[j]);
> +                    usb->port = k + 1;
> +                    rc = usb_busaddr_from_busid(gc, busid,
> +                                                &usb->u.hostdev.hostbus,
> +                                                &usb->u.hostdev.hostaddr);

You should probably set usb->devtype to HOSTDEV here, even though this
function is internal.

> +                    if (rc) goto out;
> +                }
> +            }
> +        }
> +    }
> +
> +    rc = 0;
> +
> +out:
> +    if (rc) {
> +        *list = NULL;
> +        *num = 0;
> +    }
> +    return rc;
> +}
> +
> +static bool is_usbdev_in_array(libxl_device_usb *usbs, int num,
> +                               libxl_device_usb *usb)
> +{
> +    int i;
> +
> +    for (i = 0; i < num; i++) {
> +        if (usbs[i].u.hostdev.hostbus == usb->u.hostdev.hostbus &&
> +            usbs[i].u.hostdev.hostaddr == usb->u.hostdev.hostaddr)
> +            return true;
> +    }
> +
> +    return false;
> +}
> +
> +/* check if USB device is already assigned to a domain */
> +/* check if USB device type is assignable */
> +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    int classcode;
> +    char *filename;
> +    void *buf = NULL;
> +    char *busid = NULL;
> +
> +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus,
> +                                 usb->u.hostdev.hostaddr);
> +    if (!busid) return false;
> +
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid);
> +    if (libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))
> +        return false;
> +
> +    classcode = atoi(buf);
> +    return classcode != USBHUB_CLASS_CODE;
> +}
> +
> +/* get usb devices under certain usb controller */
> +static int
> +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid,
> +                                   libxl_devid usbctrl,
> +                                   libxl_device_usb **usbs, int *num)
> +{
> +    const char *fe_path, *be_path, *num_devs;
> +    int n, i, rc;
> +
> +    *usbs = NULL;
> +    *num = 0;
> +
> +    fe_path = GCSPRINTF("%s/device/vusb/%d",
> +                        libxl__xs_get_dompath(gc, domid), usbctrl);
> +
> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                                GCSPRINTF("%s/backend", fe_path),
> +                                &be_path);
> +    if (rc) return rc;
> +
> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                                GCSPRINTF("%s/num-ports", be_path),
> +                                &num_devs);
> +    if (rc) return rc;
> +
> +    n = atoi(num_devs);
> +
> +    for (i = 0; i < n; i++) {
> +        char *busid;
> +        libxl_device_usb *usb;
> +
> +        busid = libxl__xs_read(gc, XBT_NULL,
> +                               GCSPRINTF("%s/port/%d", be_path, i + 1));
> +        if (busid && strcmp(busid, "")) {
> +            GCREALLOC_ARRAY(*usbs, *num + 1);
> +            usb = *usbs + *num;
> +            (*num)++;
> +            libxl_device_usb_init(usb);
> +            usb->ctrl = usbctrl;
> +            usb->port = i + 1;
> +            rc = usb_busaddr_from_busid(gc, busid,
> +                                        &usb->u.hostdev.hostbus,
> +                                        &usb->u.hostdev.hostaddr);

Same thing re devtype.

> +int libxl_ctrlport_to_device_usb(libxl_ctx *ctx,
> +                                 uint32_t domid,
> +                                 int ctrl,
> +                                 int port,
> +                                 libxl_device_usb *usb)
> +{
> +    GC_INIT(ctx);
> +    const char *dompath, *be_path, *busid;
> +    int rc;
> +
> +    dompath = libxl__xs_get_dompath(gc, domid);
> +
> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                  GCSPRINTF("%s/device/vusb/%d/backend", dompath, ctrl),
> +                  &be_path);
> +    if (rc) goto out;
> +
> +    rc = libxl__xs_read_checked(gc, XBT_NULL,
> +                           GCSPRINTF("%s/port/%d", be_path, port),
> +                           &busid);
> +    if (rc) goto out;
> +
> +    if (!strcmp(busid, "")) {
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    usb->ctrl = ctrl;
> +    usb->port = port;
> +    rc = usb_busaddr_from_busid(gc, busid, &usb->u.hostdev.hostbus,
> +                                &usb->u.hostdev.hostaddr);

You definitely need to set usb->devtype here to HOSTDEV.

> +libxl_usbinfo = Struct("usbinfo", [
> +    ("ctrl", libxl_devid),
> +    ("port", integer),
> +    ("busnum", uint8),
> +    ("devnum", uint8),
> +    ("idVendor", uint16),
> +    ("idProduct", uint16),
> +    ("prod", string),
> +    ("manuf", string),
> +    ], dir=DIR_OUT)

As mentioned in the review of another patch, it looks like this is
vestigal and should be removed.

> +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr)
> +{
> +   int i;
> +   for (i = 0; i < nr; i++)
> +       libxl_device_usbctrl_dispose(&list[i]);
> +   free(list);
> +}
> +
> +void libxl_device_usb_list_free(libxl_device_usb *list, int nr)
> +{
> +   int i;
> +   for (i = 0; i < nr; i++)
> +       libxl_device_usb_dispose(&list[i]);
> +   free(list);
> +}

This is nitpicky, but as long as you're going to repost: the
first-level indents in these two functions are only 3 spaces instead
of 4.

Other than that (and my previous comments + Ian's comments) it looks good to me!

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-12 17:00       ` Ian Jackson
@ 2015-11-13  2:30         ` Chun Yan Liu
  2015-11-16 18:06           ` Ian Jackson
  0 siblings, 1 reply; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-13  2:30 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/13/2015 at 01:00 AM, in message
<22084.50631.506663.212889@mariner.uk.xensource.com>, Ian Jackson
<Ian.Jackson@eu.citrix.com> wrote: 
> Chun Yan Liu writes ("Re: [PATCH V8 3/7] libxl: add pvusb API"): 
> > On 11/10/2015 at 02:11 AM, in message 
> > <22080.57829.461049.37192@mariner.uk.xensource.com>, Ian Jackson 
> > <Ian.Jackson@eu.citrix.com> wrote:  
> > > Chunyan Liu writes ("[PATCH V8 3/7] libxl: add pvusb API"):  
> > > > +void libxl__device_usbctrl_add(libxl__egc *egc, uint32_t domid,  
> > > > +                               libxl_device_usbctrl *usbctrl,  
> > > > +                               libxl__ao_device *aodev)  
> > > > +{  
> > >   
> > > Thanks for adjusting the error-handling patterns in these functions.  
> > > The new way is good, except that:  
> > >   
> > > > +out:  
> > > > +    aodev->rc = rc;  
> > > > +    if (rc) aodev->callback(egc, aodev);  
> > >   
> > > Here, rc is always set, and indeed the code would be wrong if it were  
> > > not.  So can you remove the conditional please ?  Ie:  
> >  
> > Reading the codes, libxl__wait_device_connection will call 
> > aodev->callback properly. 
>  
> Indeed.  So you need to call aodev->callback() iff you don't call 
> libxl__wait_device_connection.  But libxl__wait_device_connection is 
> called only on the success exit path which ends in `return', not in 
> `goto out'.  So: 
>  
> > So here, only if (rc != 0), that means 
> > error happens, then we need to call aodev->callback to end the 
> > process. (Refer to current libxl__device_disk_add, all current code 
> > does similar work.) So I think the code is not wrong (?) 
>  
> In the `goto out' path, rc is always set.  Writing `if (rc)' implies 
> that it might not be (or is allowed not to be), which is misleading.

I'm convinced your suggestion is better :-). I'll adjust codes.
 
>  
>  
> > > > +static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid,  
> > > > +                                  uint8_t *bus, uint8_t *addr)  
> > > > +{  
> > > > +    char *filename;  
> > > > +    void *buf;  
> > > > +  
> > > > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);  
> > > > +    if (!libxl__read_sysfs_file_contents(gc, filename, &buf, NULL))  
> > > > +        *bus = atoi((char *)buf);  
> > >   
> > > I don't think this cast (and the one for addr) are necessary ?  
> >  
> > Which cast? Here, we want to get a uint_8 value, but buf is a string, 
> > we need to do atoi. 
>  
> I mean that I think 
>  
>      +        *bus = atoi(buf);  
>  
> would compile and be correct.  buf would be automatically converted 
> from void* to const char*.  It's better to avoid casts where they are 
> not needed, because casts will suppress compiler warnings. 

Yes, you're right. Thanks very much!

>  
> For example if someone wanted to have the buffer be an updated 
> parameter they might do this: 
>  
>   static int usb_busaddr_from_busid(libxl__gc *gc, const char *busid,  
> +                                   void **buf, 
>                                     uint8_t *bus, uint8_t *addr) 
> -    void *buf;  
>  
> and hope or expect the compiler to notice places where they had failed 
> to update the usage of buf.  With void **buf, 
>   atoi((char*)buf) 
> would compile and do a wrong thing whereas 
>   atoi(buf) 
> would produce a fatal warning. 
>  
> > > > +/* bind/unbind usb device interface */  
> > > > +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)  
> > > > +{  
> ... 
> > > > +        dp = libxl__zalloc(gc, PATH_MAX);  
> > > > +        dp = realpath(spath, dp);  
> > >   
> > > Why is this call to realpath needed ? 
> >  
> > In sysfs, /driver sometimes is a link, since we need to save the original 
> > driver to xenstore, so need to get the realpath of the driver. 
>  
> I mean, why is the path with all the symlinks in it not suitable for 
> storage and later use ? 

The symlink might be "../../../../../../bus/usb/drivers/btusb", we couldn't save
that to xenstore for later usage.

>  
>  
> > > > +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)  
> > > > +{  
> > > ...  
> > > > +        /* bind interface to its originial driver */  
> > > > +        drvpath = libxl__xs_read(gc, XBT_NULL,  
> > > > +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",  
> > > > +                  usb_encode, usb_interface_xenstore_encode(intf)));  
> > > > +        if (drvpath && bind_usb_intf(gc, intf, drvpath))  
> > > > +            LOGE(WARN, "Couldn't bind %s to %s", intf, drvpath);  
> > >   
> > > This error message could be clearer.  
> > >   
> > > Also it would be worth a comment in the code to explain why this is a  
> > > warning (merely logged), rather than an error (logged and reported  
> > > with nonzero status to caller). 
> >  
> > Will update. When doing USB remove, it's the ideal result we can rebound 
> > the USB to its original driver, in case the USB interface failed to be 
> > rebound to its original driver, we will export warning but won't stop the 
> > removing process. 
>  
> Right.  Please put that in a comment :-). 
>  
>  
> > > > +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)  
> > > > +{  
> > > ...  
> > > > +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0)  
> {  
> > > > +                LOG(WARN, "Write of %s to node %s failed", drvpath,  
> path);  
> > > > +            }  
> > >   
> > > One of the advantages of libxl__xs_write_checked is that it logs  
> > > errors.  So you can leave that log message out.  
> > >   
> > > However, if the xs write fails, you should presumably stop, rather  
> > > than carrying on.  
> > >  
> > > I think you probably do things in the wrong order here.  You should  
> > > write the old driver info to xenstore first, so that if the bind to  
> > > usbback fails, you have the old driver info.  
> >  
> > Perhaps not. Once we finished adding entries to xenstore, pvusb 
> > frontend/backend drivers will detect the change and do probe work, if 
> > USB device is still not bound to USBBACK, there might be error. 
>  
> What I mean is this: 
>  
> Is it not possible to write the original path to xenstore before doing 
> the unbind ?  Otherwise it seems like there could be error paths where 
> the original path is not recorded, the xenstore write fails, and then 
> the information about how to rebind to the original driver has been 
> lost. 

I see. 

>  
> Does writing USBBACK_INFO_PATH"/%s/%s/driver_path" really trigger 
> usbback ? 

No, writing driver_path info to xenstore won't trigger usbback. Writing
frontend/backend info will.

>  
> > > > +    free(usb_encode);  
> > >   
> > > This came from the gc, I think.  
> ... 
> > > > +    switch (usbctrlinfo.type) {  
> > > > +    case LIBXL_USBCTRL_TYPE_DEVICEMODEL:  
> > > > +        LOG(ERROR, "Not supported");  
> > > > +        break;  
> > >   
> > > Needs to set rc.  
>  
> (Did you spot these comments ?  Normally I wouldn't even expect you to 
> quote the comments that you are just going to fix, but since you have 
> replied to the others, I thought I would check.)

No, it's definitely right, so I'll fix surely.
 
>  
>  
> > > > +void libxl__device_usb_add(libxl__egc *egc, uint32_t domid,  
> > > > +                           libxl_device_usb *usb,  
> > > > +                           libxl__ao_device *aodev)  
> > > > +{  
> > > ...  
> > > > +        if (usbctrlinfo.backend_id != 0) {  
> > >   
> > > LIBXL_TOOLSTACK_DOMID please.  (And in remove, too.)  
> > >   
> > > Why is this check done in libxl__device_usb_add and do_usb_remove,  
> > > rather than in a symmetrical way? 
> >   
> > In libxl__device_usb_add, the set_default function implementation won't 
> > work if backend domain is USB driver domain in future, so we must add 
> > a check before set_default, so in libxl__device_usb_add. To check backend 
> > domain, we need to get usbctrl info. And in do_usb_add, to check usbctrl 
> > type, again we need to get usbctrl info. It's disgusting to do the same  
> thing 
> > twice. 
> >  
> > In usb remove, nothing in libxl__device_usb_remove is Dom0 specific, so 
> > I do all check in do_usb_remove, then only one time to get usbctrl info. 
>  
> Right. 
>  
> > If we merge libxl__device_usb_add and do_usb_add, then it's cleaner. 
>  
> See my comment below.  You've explained the distinction to my 
> satisfaction. 
>  
> But, to solve the duplication of the controller info acquisition, 
> perhaps you could have do_usb_add take the controller info as a 
> paramaeter ?

Yes, could be. Only do_usb_add and do_usb_remove parameters are not
symmetrical. 

>  
>  
> > > > +    /* Do the add */  
> > > > +    if (do_usb_add(gc, domid, usb, aodev)) {  
> > > > +        rc = ERROR_FAIL;  
> > > > +        goto out;  
> > > > +    }  
> > > > +  
> > > > +    libxl__ao_complete(egc, ao, 0);  
> > > > +    rc = 0;  
> > > > +  
> > > > +out:  
> > > > +    libxl_device_usbctrl_dispose(&usbctrl);  
> > > > +    libxl_usbctrlinfo_dispose(&usbctrlinfo);  
> > > > +    aodev->rc = rc;  
> > > > +    if (rc) aodev->callback(egc, aodev);  
> > > > +    return;  
> > >   
> > > Calling libxl__ao_complete here is quite wrong.  I think you should  
> > > call aodev->callback in both success and failure cases.  And it must  
> > > be the last thing you do.  
> >  
> > As mentioned in libxl__device_usbctrl_add, libxl__ao_complete will 
> > call callback, so in 'out' section, only if (rc !=0), which means 
> > error happens, then we need to call aodev->callback explicitly to 
> > end the process. 
> >  
> > Here, since we don't have actual async work needs to do, so call 
> > libxl__ao_complete(egc, ao, 0) directly to end the process. 
>  
> I'm afraid this is definiitely wrong.  There are a number of things 
> wrong with it, or a number of ways of looking at it: 
>  
> Most obviously, a single entrypoint should always complete its 
> execution the same way.  In this case the entrypoint takes an aodev 
> and in the error path calls aodev->callback().  So it should do the 
> same on the success path. 
>  
> Secondly (and relatedly), it is generally wrong for anything inside a 
> particular piece of machinery to terminate an ao. 
>  
> In this particular case I think it leads to a use-after-free bug: 
> completing the ao will (maybe) free the ao, and aodev was allocated 
> from the ao's gc.  So the write to aodev->rc is improper. 

Reasonable, I'll adjust codes. Thanks!

- Chunyan

>  
>  
> > > I might ask the same question about do_usb_add and  
> > > libxl__device_usb_add ?  
> >  
> > Using libxl__device_usb_add and within it extracting do_usb_add is to: 
> > * doing some initialization work and necessary check work in 
> >   libxl__device_usb_add (this part are common to PVUSB and QEMU USB) 
> > * doing actual adding usb operations in do_usb_add (this part will diverge 
> >   for PVUSB and QEMU USB) . 
>  
> This is a good reason for keeping them split.  Thanks for the 
> explanation. 
>  
>  
> Thanks for your other replies. 
>  
> Regards, 
> Ian. 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-12 11:32   ` Olaf Hering
@ 2015-11-13  2:32     ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-13  2:32 UTC (permalink / raw)
  To: Olaf Hering
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap,
	Ian.Jackson, xen-devel, Jim Fehlig, Simon Cao



>>> On 11/12/2015 at 07:32 PM, in message <20151112113200.GA302@aepfle.de>, Olaf
Hering <olaf@aepfle.de> wrote: 
> On Wed, Oct 21, Chunyan Liu wrote: 
>  
> > Add pvusb APIs, including: 
>  
> Some comments below. 
>  
> After a quick look I miss the proposed ctrl/device separation for pvscsi 
> (what handles "state" changes?). But, I have to read all the other 
> dozen+ threads about that topic first. 
>  
>  
> > +    flexarray_append_pair(back, "state", "1"); 
>  
> 4.6+ has macros for "state" values, like 
> flexarray_append_pair(back, "state", GCSPRINTF("%d",  
> XenbusStateInitialising)); 
>  
>  
> > +    flexarray_append_pair(front, "state", "1"); 
>  
> 4.6+ has macros for "state" values. 
>  
> > +        LOG(DEBUG, "Adding new usb device to xenstore"); 
>  
> Which one? Perhaps print also details. 
>  
> > +    LOG(DEBUG, "Removing USB device from xenstore"); 
>  
> Which one? Perhaps print also details. 
>  
> > +        /* check if the USB interface is already bound to "usbbcak" */ 
>  
> Typo. 

Take all. Thanks Olaf!

- Chunyan

>  
>  
> Olaf 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-11-12 11:38   ` George Dunlap
  2015-11-12 11:39     ` George Dunlap
@ 2015-11-13  2:43     ` Chun Yan Liu
  2015-11-16 10:05       ` George Dunlap
  1 sibling, 1 reply; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-13  2:43 UTC (permalink / raw)
  To: George Dunlap
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/12/2015 at 07:38 PM, in message
<CAFLBxZb3LeZY1V78GdswCBB0B1_069-DbY-tuNVHnQ2h=ywT4A@mail.gmail.com>, George
Dunlap <George.Dunlap@eu.citrix.com> wrote: 
> On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote: 
> > Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list, 
> > usb-attach and usb-detach. 
> > 
> > To attach a usb device to guest through pvusb, one could follow 
> > following example: 
> > 
> >  #xl usbctrl-attach test_vm version=1 ports=8 
> > 
> >  #xl usb-list test_vm 
> >  will show the usb controllers and port usage under the domain. 
> > 
> >  #xl usbattach test_vm hostbus=1 hostaddr=2 
>  
> Nit: usb-attach (missing a '-') 

Oh, yes.

>  
> >  will find the first usable controller:port, and attach usb 
> >  device whose busnum is 1 and devnum is 6. 
> >  One could also specify which <controller> and which <port>. 
> > 
> >  #xl usb-detach test_vm 0 1 
> >  will detach USB device under controller 0 port 1. 
> > 
> >  #xl usbctrl-detach test_vm dev_id 
> >  will destroy the controller with specified dev_id. Dev_id 
> >  can be traced in usb-list info. 
> > 
> > Signed-off-by: Chunyan Liu <cyliu@suse.com> 
> > Signed-off-by: Simon Cao <caobosimon@gmail.com> 
> > 
> > --- 
> > Changes: 
> >   - change usb-attach parameter from hostbus.hostaddr to 
> >     hostbus=xx hostaddr= 
> >   - since we get rid of libxl_device_usb_getinfo, so adjust usb-list 
> >     information a little bit. 
> >   - parse_usb_config and parse_usbctrl_config following parse_nic_config 
> >     way, put in this patch, and shared domcreate routine. 
> > 
> >  docs/man/xl.pod.1         |  40 ++++++++ 
> >  tools/libxl/xl.h          |   5 + 
> >  tools/libxl/xl_cmdimpl.c  | 250  
> ++++++++++++++++++++++++++++++++++++++++++++++ 
> >  tools/libxl/xl_cmdtable.c |  25 +++++ 
> >  4 files changed, 320 insertions(+) 
> > 
> > diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1 
> > index d0cd612..f09a872 100644 
> > --- a/docs/man/xl.pod.1 
> > +++ b/docs/man/xl.pod.1 
> > @@ -1345,6 +1345,46 @@ List pass-through pci devices for a domain. 
> > 
> >  =back 
> > 
> > +=head1 USB PASS-THROUGH 
> > + 
> > +=over 4 
> > + 
> > +=item B<usbctrl-attach> I<domain-id> I[<type=val>] [I<version=val>]  
> [I<ports=number>] 
> > + 
> > +Create a new USB controller for the specified domain. 
> > +B<type=val> is the usb controller type, currently only support 'pv'. 
>  
> "B<type=val> is the usb controller type.  Currently only 'pv' and 
> 'auto' are supported." 
>  
> Note the period rather than the comma. 
>  
> (See below on how to make this true.) 
>  
> > +B<version=val> is the usb controller version, could be 1 (USB1.1) or 2  
> (USB2.0). 
>  
> "<version=val> is the usb controller version.  Possible values include 
> 1 (USB1.1) and 2 (USB2.0)." 
>  
> (Same thing wrt the period.) 

Thanks. Will check all.

>  
> > +B<ports=number> is the total ports of the usb controller. 
>  
> Is there a maximum number of ports? 

Yes, according to Juergen, it should be no more than 31.

>  
> > +By default, it will create a USB2.0 controller with 8 ports. 
> > + 
> > +=item B<usbctrl-detach> I<domain-id> I<devid> 
> > + 
> > +Destroy a USB controller from the specified domain. 
> > +B<devid> is devid of the USB controller. 
> > + 
> > +If B<-f> is specified, B<xl> is going to forcefully remove the device even 
> > +without guest's collaboration. 
>  
> "If B<-f> is specified, B<xl> will forcefully remove removal (i.e., 
> without the guest's cooperation)." 

Thanks.

>  
> > + 
> > +=item B<usb-attach> I<domain-id> I<hostbus=busnum> I<hostaddr=devnum>  
> [I<controller=devid> [I<port=number>]] 
> > + 
> > +Hot-plug a new pass-through USB device to the specified domain. 
> > +B<bus.addr> is the busnum.devnum of the physical USB device to pass-through. 
>  
> "hostbus and hostaddr are the bus and device number of the physical 
> USB device to pass through." 

Thanks. Forgot to update here.

>  
> > +B<controller=devid> B<port=number> is the USB controller:port to hotplug the 
> > +USB device to. By default, it will find the first available  
> controller:port 
> > +and use it; if there is no controller, it will create one. 
> > + 
> > +=item B<usb-detach> I<domain-id> I<controller=devid> I<port=number> 
> > + 
> > +Hot-unplug a previously assigned USB device from a domain. 
> > +B<controller=devid> and B<port=number> is USB controller:port in guest where  
> the 
> > +USB device is attached to. 
> > + 
> > +=item B<usb-list> I<domain-id> 
> > + 
> > +List pass-through usb devices for a domain. 
> > + 
> > +=back 
> > + 
> >  =head1 TMEM 
> > 
> >  =over 4 
> > diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h 
> > index 0021112..ddd9690 100644 
> > --- a/tools/libxl/xl.h 
> > +++ b/tools/libxl/xl.h 
> > @@ -85,6 +85,11 @@ int main_blockdetach(int argc, char **argv); 
> >  int main_vtpmattach(int argc, char **argv); 
> >  int main_vtpmlist(int argc, char **argv); 
> >  int main_vtpmdetach(int argc, char **argv); 
> > +int main_usbctrl_attach(int argc, char **argv); 
> > +int main_usbctrl_detach(int argc, char **argv); 
> > +int main_usbattach(int argc, char **argv); 
> > +int main_usbdetach(int argc, char **argv); 
> > +int main_usblist(int argc, char **argv); 
> >  int main_uptime(int argc, char **argv); 
> >  int main_claims(int argc, char **argv); 
> >  int main_tmem_list(int argc, char **argv); 
> > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c 
> > index 365798b..e6ff6f4 100644 
> > --- a/tools/libxl/xl_cmdimpl.c 
> > +++ b/tools/libxl/xl_cmdimpl.c 
> > @@ -1255,6 +1255,64 @@ static void parse_vnuma_config(const XLU_Config  
> *config, 
> >      free(vcpu_parsed); 
> >  } 
> > 
> > +/* Parses usbctrl data and adds info into usbctrl 
> > + * Returns 1 if the input token does not match one of the keys 
> > + * or parsed values are not correct. Successful parse returns 0 */ 
> > +static int parse_usbctrl_config(libxl_device_usbctrl *usbctrl, char  
> *token) 
> > +{ 
> > +    char *oparg; 
> > + 
> > +    if (MATCH_OPTION("type", token, oparg)) { 
> > +        if (!strcmp("pv", oparg)) { 
> > +            usbctrl->type = LIBXL_USBCTRL_TYPE_PV; 
> > +        } else { 
> > +            fprintf(stderr, 
> > +                    "Unsupported USB controller type '%s'\n", oparg); 
> > +            return 1; 
> > +        } 
>  
> One of the reasons for making the enum types in the IDL is that the 
> IDL then automatically generates functions for converting to/from 
> strings. 
>  
> So here I would do something like: 
>  
> if(libxl_usbctrl_type_from_string(optarg, &usbctrl->type)) { 
>  fprintf(stderr, "Invalid usbctrl type: %s\n", optarg); 
>  exit(1); 
> } 
>  
> (Just to emphasize -- you don't need to write the above function; it's 
> generated automatically.) 

Yes, I know. I'll adjust codes.

>  
>  
> > +/* Parses usb data and adds info into usb 
> > + * Returns 1 if the input token does not match one of the keys 
> > + * or parsed values are not correct. Successful parse returns 0 */ 
> > +static int parse_usb_config(libxl_device_usb *usb, char *token) 
> > +{ 
> > +    char *oparg; 
> > + 
> > +    if (MATCH_OPTION("devtype", token, oparg)) { 
> > +        if (!strcmp("hostdev", oparg)) { 
> > +            usb->devtype = LIBXL_USBDEV_TYPE_HOSTDEV; 
>  
> Same thing here, with libxl_usbdev_type_to_string(). 

Got it.

>  
>  
> > +int main_usbctrl_attach(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    int opt, rc = 0; 
> > +    libxl_device_usbctrl usbctrl; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usbctrl-attach", 1) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    domid = find_domain(argv[optind++]); 
> > + 
> > +    libxl_device_usbctrl_init(&usbctrl); 
> > + 
> > +    for (argv += optind, argc -= optind; argc > 0; ++argv, --argc) { 
> > +        if (parse_usbctrl_config(&usbctrl, *argv)) 
> > +            return 1; 
> > +    } 
> > + 
> > +    rc = libxl_device_usbctrl_add(ctx, domid, &usbctrl, 0); 
> > +    if (rc) { 
> > +        fprintf(stderr, "libxl_device_usbctrl_add failed.\n"); 
> > +        rc = 1; 
> > +    } 
> > + 
> > +    libxl_device_usbctrl_dispose(&usbctrl); 
> > +    return rc; 
> > +} 
> > + 
> > +int main_usbctrl_detach(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    int opt, devid, rc; 
> > +    libxl_device_usbctrl usbctrl; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usbctrl-detach", 2) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    domid = find_domain(argv[optind]); 
> > +    devid = atoi(argv[optind+1]); 
> > + 
> > +    libxl_device_usbctrl_init(&usbctrl); 
> > +    if (libxl_devid_to_device_usbctrl(ctx, domid, devid, &usbctrl)) { 
> > +        fprintf(stderr, "Unknown device %s.\n", argv[optind+1]); 
> > +        return 1; 
> > +    } 
> > + 
> > +    rc = libxl_device_usbctrl_remove(ctx, domid, &usbctrl, 0); 
> > +    if (rc) { 
> > +        fprintf(stderr, "libxl_device_usbctrl_remove failed.\n"); 
> > +        rc = 1; 
> > +    } 
> > + 
> > +    libxl_device_usbctrl_dispose(&usbctrl); 
> > +    return rc; 
> > + 
> > +} 
> > + 
> > +int main_usbattach(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    int opt, rc; 
> > +    libxl_device_usb usb; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-attach", 2) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    libxl_device_usb_init(&usb); 
> > + 
> > +    domid = find_domain(argv[optind++]); 
> > + 
> > +    for (argv += optind, argc -= optind; argc > 0; ++argv, --argc) { 
> > +        if (parse_usb_config(&usb, *argv)) 
> > +            return 1; 
> > +    } 
> > + 
> > +    rc = libxl_device_usb_add(ctx, domid, &usb, 0); 
> > +    if (rc) { 
> > +        fprintf(stderr, "libxl_device_usb_add failed.\n"); 
> > +        rc = 1; 
> > +    } 
> > + 
> > +    libxl_device_usb_dispose(&usb); 
> > +    return rc; 
> > +} 
> > + 
> > +int main_usbdetach(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    int ctrl, port; 
> > +    int opt, rc = 1; 
> > +    libxl_device_usb usb; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-detach", 3) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    domid = find_domain(argv[optind]); 
> > +    ctrl = atoi(argv[optind+1]); 
> > +    port = atoi(argv[optind+2]); 
> > + 
> > +    if (argc - optind > 3) { 
> > +        fprintf(stderr, "Invalid arguments.\n"); 
> > +        return 1; 
> > +    } 
> > + 
> > +    libxl_device_usb_init(&usb); 
> > +    if (libxl_ctrlport_to_device_usb(ctx, domid, ctrl, port, &usb)) { 
> > +        fprintf(stderr, "Unknown device at controller %d port %d.\n", 
> > +                ctrl, port); 
> > +        return 1; 
> > +    } 
> > + 
> > +    rc = libxl_device_usb_remove(ctx, domid, &usb, 0); 
> > +    if (rc) { 
> > +        fprintf(stderr, "libxl_device_usb_remove failed.\n"); 
> > +        rc = 1; 
> > +    } 
> > + 
> > +    libxl_device_usb_dispose(&usb); 
> > +    return rc; 
> > +} 
> > + 
> > +int main_usblist(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    libxl_device_usbctrl *usbctrls; 
> > +    libxl_usbctrlinfo usbctrlinfo; 
> > +    int numctrl, i, j, opt; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-list", 1) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    domid = find_domain(argv[optind++]); 
> > + 
> > +    if (argc > optind) { 
> > +        fprintf(stderr, "Invalid arguments.\n"); 
> > +        exit(-1); 
> > +    } 
> > + 
> > +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl); 
> > +    if (!usbctrls) { 
> > +        return 0; 
> > +    } 
> > + 
> > +    for (i = 0; i < numctrl; ++i) { 
> > +        printf("%-6s %-6s %-3s %-5s %-7s %-5s %-30s\n", 
> > +                "Devid", "Type", "BE", "state", "usb-ver", "ports",  
> "BE-path"); 
> > + 
> > +        libxl_usbctrlinfo_init(&usbctrlinfo); 
> > + 
> > +        if (!libxl_device_usbctrl_getinfo(ctx, domid, 
> > +                                &usbctrls[i], &usbctrlinfo)) { 
> > +            printf("%-6d %-6s %-3d %-5d %-7d %-5d %-30s\n", 
> > +                    usbctrlinfo.devid, 
> > +                    libxl_usbctrl_type_to_string(usbctrlinfo.type), 
>  
> Oh, I see you already know about the "type_to_string()" functions. :-) 
>  
> > +                    usbctrlinfo.backend_id, usbctrlinfo.state, 
> > +                    usbctrlinfo.version, usbctrlinfo.ports, 
> > +                    usbctrlinfo.backend); 
> > + 
> > +            for (j = 1; j <= usbctrlinfo.ports; j++) { 
> > +                libxl_device_usb usb; 
> > +                libxl_usbinfo usbinfo; 
> > + 
> > +                libxl_device_usb_init(&usb); 
> > +                libxl_usbinfo_init(&usbinfo); 
> > + 
> > +                printf("  Port %d:", j); 
> > + 
> > +                if (!libxl_ctrlport_to_device_usb(ctx, domid, 
> > +                                                  usbctrlinfo.devid, j,  
> &usb)) { 
> > +                    printf(" Bus %03x Device %03x\n", 
> > +                           usbinfo.busnum, usbinfo.devnum); 
> > +                } else { 
> > +                    printf("\n"); 
> > +                } 
> > + 
> > +                libxl_usbinfo_dispose(&usbinfo); 
>  
> Er, what's going on with the usbinfo?  Did you take the usbinfo stuff 
> out of libxl_pvusb.c, but forget to take it out of libxl_types.idl and 
> here? 
>  
> I guess you really want usb.u.hostdev.* here. 

Yes, you're totally right. I'll adjust codes.

Thanks.

- Chunyan

>  
> Everything else looks good, thanks. 
>  
>  -George 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-11-12 14:42   ` Olaf Hering
  2015-11-12 14:49     ` George Dunlap
@ 2015-11-13  2:49     ` Chun Yan Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-13  2:49 UTC (permalink / raw)
  To: Olaf Hering
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap,
	Ian.Jackson, xen-devel, Jim Fehlig, Simon Cao



>>> On 11/12/2015 at 10:42 PM, in message <20151112144211.GA5773@aepfle.de>, Olaf
Hering <olaf@aepfle.de> wrote: 
> On Wed, Oct 21, Chunyan Liu wrote: 
>  
> > Add pvusb commands: usbctrl-attach, usbctrl-detach, usb-list, 
> > usb-attach and usb-detach. 
>  
> How is this supposed to be handled in libvirt? It looks like libvirt has 
> to copy what is done here. If thats true the logic should be in libxlu 
> so that both xl and libvirt can call the same functions.

For usbctrl-attach/detach, usb-attach/detach, libxl has API, like
libxl_device_usbctrl_add/remove, libxl_device_usb_add/remove. That's
could be used for libvirt usage. The user interface processing things
are always do-by-itslef for xl and libvirt, what's libvirt needs to do is
preparing the usbctrl/usb device structure and call libxl API to do the
work.

- Chunyan 
 
>  
> Olaf 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 7/7] domcreate: support pvusb in configuration file
  2015-11-12 16:10   ` George Dunlap
@ 2015-11-13  2:54     ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-13  2:54 UTC (permalink / raw)
  To: George Dunlap
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/13/2015 at 12:10 AM, in message
<CAFLBxZZ_KN0vMr77+dfRTgGEWW3LSbv8m__AWfvEeCH1WBRrYw@mail.gmail.com>, George
Dunlap <George.Dunlap@eu.citrix.com> wrote: 
> On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote: 
> > Add code to support pvusb in domain config file. One could specify 
> > usbctrl and usb in domain's configuration file and create domain, 
> > then usb controllers will be created and usb device would be attached 
> > to guest automatically. 
> > 
> > One could specify usb controllers and usb devices in config file 
> > like this: 
> > usbctrl=['version=2,ports=4', 'version=1, ports=4', ] 
> > usbdev=['hostbus=2, hostaddr=1, controller=0,port=1', ] 
> > 
> > Signed-off-by: Chunyan Liu <cyliu@suse.com> 
> > Signed-off-by: Simon Cao <caobosimon@gmail.com> 
> > 
> > --- 
> > changes: 
> >   - change parse_usb_config and parse_usbctrl_config, following 
> >     parse_nic_config way, and move to previous patch 
> >   - update user interface to specify hostbus, hostaddr instead of 
> >     hostbus.hostaddr for future extension 
> > 
> >  docs/man/xl.cfg.pod.5        | 81  
> ++++++++++++++++++++++++++++++++++++++++++++ 
> >  tools/libxl/libxl_create.c   | 73 +++++++++++++++++++++++++++++++++++++-- 
> >  tools/libxl/libxl_device.c   |  4 +++ 
> >  tools/libxl/libxl_internal.h |  8 +++++ 
> >  tools/libxl/xl_cmdimpl.c     | 54 ++++++++++++++++++++++++++++- 
> >  5 files changed, 216 insertions(+), 4 deletions(-) 
> > 
> > diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5 
> > index b63846a..f24c008 100644 
> > --- a/docs/man/xl.cfg.pod.5 
> > +++ b/docs/man/xl.cfg.pod.5 
> > @@ -722,6 +722,87 @@ Note this may be overridden by rdm_policy option in PCI  
> device configuration. 
> > 
> >  =back 
> > 
> > +=item B<usbctrl=[ "USBCTRL_SPEC_STRING", "USBCTRL_SPEC_STRING", ... ]> 
> > + 
> > +Specifies the USB controllers created for this guest. Each 
> > +B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where: 
> > + 
> > +=over 4 
> > + 
> > +=item B<KEY=VALUE> 
> > + 
> > +Possible B<KEY>s are: 
> > + 
> > +=over 4 
> > + 
> > +=item B<type=TYPE> 
> > + 
> > +Specifies the protocol to implement USB controller, could be "pv"  
> (indicates 
> > +PVUSB) or "qemu" (indicates QEMU emulated). Currently only "pv" is  
> supported. 
>  
> I think most of these should basically be copied from the descriptions 
> of the arguments in patch 5. 

I'll check.

>  
> > +=item B<version=VERSION> 
> > + 
> > +Specifies version of the USB controller, could be 1 (USB1.1) or 2  
> (USB2.0). 
> > +Default is 2 (USB2.0). 
> > + 
> > +=item B<ports=PORTS> 
> > + 
> > +Specifies port number of the USB controller. Default is 8. 
> > + 
> > +Each USB controller will have an index starting from 0. On the same 
> > +controller, each port will have an index starting from 1. 
>  
> I'd say: 
>  
> "USB controler ids start from 0.  In line with the USB spec, however, 
> ports on a controller start from 1." 

Much better. Thanks!

>  
> > + 
> > +E.g. 
> > +usbctrl=["version=1,ports=4", "version=2,ports=8",] 
> > +The first controller has: 
> > +controller index = 0, and port 1,2,3,4. 
> > +The second controller has: 
> > +controller index = 1, and port 1,2,3,4,5,6,7,8. 
>  
> The example is good, but I'd use "controller id" rather than "controller  
> index".

Take it.
 
>  
> > +=back 
> > + 
> > +=back 
> > + 
> > +=item B<usbdev=[ "USB_SPEC_STRING", "USB_SPEC_STRING", ... ]> 
> > + 
> > +Specifies the host USB devices to passthrough to this guest. Each 
> > +B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where: 
>  
> "Specifiec the USB devices to be attached to the guest at boot." 
> (i.e., don't assume all devices are hostdev.) 

Thanks, I'll update.

>  
> > +=over 4 
> > + 
> > +=item B<KEY=VALUE> 
> > + 
> > +Possible B<KEY>s are: 
> > + 
> > +=over 4 
> > + 
> > +=item B<devtype=hostdev> 
> > + 
> > +Specifies USB device type. Currently only support 'hostdev'. 
> > + 
> > +=item B<hostbus=busnum> 
> > + 
> > +Specifies busnum of the USB device from the host perspective. 
> > + 
> > +=item B<hostaddr=devnum> 
> > + 
> > +Specifies devnum of the USB device from the host perspective. 
> > + 
> > +=item B<controller=CONTROLLER> 
> > + 
> > +Specifies USB controller index, to which controller the USB device is  
> attached. 
> > + 
> > +=item B<port=PORT> 
> > + 
> > +Specifies USB port index, to which port the USB device is attached.  
> B<port=PORT> 
> > +is valid only when B<controller=CONTROLLER> is specified. Without 
> > +B<controller=CONTROLLER>, it will find the first available USB  
> controller:port 
> > +and use it. If there is no controller at all, it will create one. 
>  
> I think the last sentence should be a separate paragraph, probably 
> after the "=back".  And a more idiomatic way to write it might be: 
>  
> "If no controller is specified, an available controller:port 
> combination will be used.  If there are no available controller:port 
> options, a new controller will be created." 

Thanks very much! Much better.

>  
> Only one other minor comment from me: 
>  
> > +    if (!xlu_cfg_get_list(config, "usbctrl", &usbctrls, 0, 0)) { 
> > +        d_config->num_usbctrls = 0; 
> > +        d_config->usbctrls = NULL; 
> > +        while ((buf = xlu_cfg_get_listitem(usbctrls,  
> d_config->num_usbctrls)) 
> > +               != NULL) { 
>  
> I don't know what the other maintainers think, but particularly given 
> that this is spanning a line, I would personally take out the 
> comparison, and just make it 
>  
>  while ((buf = ...)) { 
>  ... 
> } 
>  
> Other than that, this one looks good to me. 
>  
>  -George 
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-12 17:27   ` George Dunlap
@ 2015-11-13  2:56     ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-13  2:56 UTC (permalink / raw)
  To: George Dunlap
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/13/2015 at 01:27 AM, in message
<CAFLBxZawnBaQ791ebbv1bmw3DX2+cWfk8QD1JdAVopk0-Ma6Zw@mail.gmail.com>, George
Dunlap <George.Dunlap@eu.citrix.com> wrote: 
> On Wed, Oct 21, 2015 at 10:08 AM, Chunyan Liu <cyliu@suse.com> wrote: 
> > +static int 
> > +get_assigned_devices(libxl__gc *gc, 
> > +                     libxl_device_usb **list, int *num) 
> > +{ 
> > +    char **domlist; 
> > +    unsigned int nd = 0, i, j, k; 
> > +    int rc; 
> > + 
> > +    *list = NULL; 
> > +    *num = 0; 
> > + 
> > +    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd); 
> > +    for (i = 0; i < nd; i++) { 
> > +        char *path, **ctrl_list; 
> > +        unsigned int nc = 0; 
> > + 
> > +        path = GCSPRINTF("/local/domain/%s/device/vusb", domlist[i]); 
> > +        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path, &nc); 
> > + 
> > +        for (j = 0; j < nc; j++) { 
> > +            const char *be_path, *num_ports; 
> > +            int nport; 
> > + 
> > +            rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                          GCSPRINTF("%s/%s/backend", path, ctrl_list[j]), 
> > +                          &be_path); 
> > +            if (rc) goto out; 
> > + 
> > +            rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                                        GCSPRINTF("%s/num-ports", be_path), 
> > +                                        &num_ports); 
> > +            if (rc) goto out; 
> > + 
> > +            nport = atoi(num_ports); 
> > +            for (k = 0; k < nport; k++) { 
> > +                libxl_device_usb *usb; 
> > +                const char *portpath, *busid; 
> > + 
> > +                portpath = GCSPRINTF("%s/port/%d", be_path, k + 1); 
> > +                busid = libxl__xs_read(gc, XBT_NULL, portpath); 
> > +                /* If there is USB device attached, add it to list */ 
> > +                if (busid && strcmp(busid, "")) { 
> > +                    GCREALLOC_ARRAY(*list, *num + 1); 
> > +                    usb = *list + *num; 
> > +                    (*num)++; 
> > +                    libxl_device_usb_init(usb); 
> > +                    usb->ctrl = atoi(ctrl_list[j]); 
> > +                    usb->port = k + 1; 
> > +                    rc = usb_busaddr_from_busid(gc, busid, 
> > +                                                &usb->u.hostdev.hostbus, 
> > +                                                &usb->u.hostdev.hostaddr); 
>  
> You should probably set usb->devtype to HOSTDEV here, even though this 
> function is internal. 
>  
> > +                    if (rc) goto out; 
> > +                } 
> > +            } 
> > +        } 
> > +    } 
> > + 
> > +    rc = 0; 
> > + 
> > +out: 
> > +    if (rc) { 
> > +        *list = NULL; 
> > +        *num = 0; 
> > +    } 
> > +    return rc; 
> > +} 
> > + 
> > +static bool is_usbdev_in_array(libxl_device_usb *usbs, int num, 
> > +                               libxl_device_usb *usb) 
> > +{ 
> > +    int i; 
> > + 
> > +    for (i = 0; i < num; i++) { 
> > +        if (usbs[i].u.hostdev.hostbus == usb->u.hostdev.hostbus && 
> > +            usbs[i].u.hostdev.hostaddr == usb->u.hostdev.hostaddr) 
> > +            return true; 
> > +    } 
> > + 
> > +    return false; 
> > +} 
> > + 
> > +/* check if USB device is already assigned to a domain */ 
> > +/* check if USB device type is assignable */ 
> > +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> > +    int classcode; 
> > +    char *filename; 
> > +    void *buf = NULL; 
> > +    char *busid = NULL; 
> > + 
> > +    busid = usb_busaddr_to_busid(gc, usb->u.hostdev.hostbus, 
> > +                                 usb->u.hostdev.hostaddr); 
> > +    if (!busid) return false; 
> > + 
> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", busid); 
> > +    if (libxl__read_sysfs_file_contents(gc, filename, &buf, NULL)) 
> > +        return false; 
> > + 
> > +    classcode = atoi(buf); 
> > +    return classcode != USBHUB_CLASS_CODE; 
> > +} 
> > + 
> > +/* get usb devices under certain usb controller */ 
> > +static int 
> > +libxl__device_usb_list_for_usbctrl(libxl__gc *gc, uint32_t domid, 
> > +                                   libxl_devid usbctrl, 
> > +                                   libxl_device_usb **usbs, int *num) 
> > +{ 
> > +    const char *fe_path, *be_path, *num_devs; 
> > +    int n, i, rc; 
> > + 
> > +    *usbs = NULL; 
> > +    *num = 0; 
> > + 
> > +    fe_path = GCSPRINTF("%s/device/vusb/%d", 
> > +                        libxl__xs_get_dompath(gc, domid), usbctrl); 
> > + 
> > +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                                GCSPRINTF("%s/backend", fe_path), 
> > +                                &be_path); 
> > +    if (rc) return rc; 
> > + 
> > +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                                GCSPRINTF("%s/num-ports", be_path), 
> > +                                &num_devs); 
> > +    if (rc) return rc; 
> > + 
> > +    n = atoi(num_devs); 
> > + 
> > +    for (i = 0; i < n; i++) { 
> > +        char *busid; 
> > +        libxl_device_usb *usb; 
> > + 
> > +        busid = libxl__xs_read(gc, XBT_NULL, 
> > +                               GCSPRINTF("%s/port/%d", be_path, i + 1)); 
> > +        if (busid && strcmp(busid, "")) { 
> > +            GCREALLOC_ARRAY(*usbs, *num + 1); 
> > +            usb = *usbs + *num; 
> > +            (*num)++; 
> > +            libxl_device_usb_init(usb); 
> > +            usb->ctrl = usbctrl; 
> > +            usb->port = i + 1; 
> > +            rc = usb_busaddr_from_busid(gc, busid, 
> > +                                        &usb->u.hostdev.hostbus, 
> > +                                        &usb->u.hostdev.hostaddr); 
>  
> Same thing re devtype. 
>  
> > +int libxl_ctrlport_to_device_usb(libxl_ctx *ctx, 
> > +                                 uint32_t domid, 
> > +                                 int ctrl, 
> > +                                 int port, 
> > +                                 libxl_device_usb *usb) 
> > +{ 
> > +    GC_INIT(ctx); 
> > +    const char *dompath, *be_path, *busid; 
> > +    int rc; 
> > + 
> > +    dompath = libxl__xs_get_dompath(gc, domid); 
> > + 
> > +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                  GCSPRINTF("%s/device/vusb/%d/backend", dompath, ctrl), 
> > +                  &be_path); 
> > +    if (rc) goto out; 
> > + 
> > +    rc = libxl__xs_read_checked(gc, XBT_NULL, 
> > +                           GCSPRINTF("%s/port/%d", be_path, port), 
> > +                           &busid); 
> > +    if (rc) goto out; 
> > + 
> > +    if (!strcmp(busid, "")) { 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    usb->ctrl = ctrl; 
> > +    usb->port = port; 
> > +    rc = usb_busaddr_from_busid(gc, busid, &usb->u.hostdev.hostbus, 
> > +                                &usb->u.hostdev.hostaddr); 
>  
> You definitely need to set usb->devtype here to HOSTDEV. 
>  
> > +libxl_usbinfo = Struct("usbinfo", [ 
> > +    ("ctrl", libxl_devid), 
> > +    ("port", integer), 
> > +    ("busnum", uint8), 
> > +    ("devnum", uint8), 
> > +    ("idVendor", uint16), 
> > +    ("idProduct", uint16), 
> > +    ("prod", string), 
> > +    ("manuf", string), 
> > +    ], dir=DIR_OUT) 
>  
> As mentioned in the review of another patch, it looks like this is 
> vestigal and should be removed. 
>  
> > +void libxl_device_usbctrl_list_free(libxl_device_usbctrl *list, int nr) 
> > +{ 
> > +   int i; 
> > +   for (i = 0; i < nr; i++) 
> > +       libxl_device_usbctrl_dispose(&list[i]); 
> > +   free(list); 
> > +} 
> > + 
> > +void libxl_device_usb_list_free(libxl_device_usb *list, int nr) 
> > +{ 
> > +   int i; 
> > +   for (i = 0; i < nr; i++) 
> > +       libxl_device_usb_dispose(&list[i]); 
> > +   free(list); 
> > +} 
>  
> This is nitpicky, but as long as you're going to repost: the 
> first-level indents in these two functions are only 3 spaces instead 
> of 4.

Take all. Thanks very much!

- Chunyan
 
>  
> Other than that (and my previous comments + Ian's comments) it looks good to  
> me! 
>  
>  -George 
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
                     ` (4 preceding siblings ...)
  2015-11-12 17:27   ` George Dunlap
@ 2015-11-13 11:19   ` Olaf Hering
  2015-11-16 10:01     ` George Dunlap
  5 siblings, 1 reply; 53+ messages in thread
From: Olaf Hering @ 2015-11-13 11:19 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	xen-devel, jfehlig, Simon Cao

On Wed, Oct 21, Chunyan Liu wrote:

> Add pvusb APIs, including:

> @@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [
>      ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
>      ("rdms", Array(libxl_device_rdm, "num_rdms")),
>      ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
> +    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")),
> +    ("usbs", Array(libxl_device_usb, "num_usbs")),

Should that perhaps be "usbctrls" + "usbdevs" if the latter really
represents a device below one of the "usbctrls"? I dont know if it does,
just wondering.


Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-13 11:19   ` Olaf Hering
@ 2015-11-16 10:01     ` George Dunlap
  2015-11-18  5:48       ` Chun Yan Liu
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-11-16 10:01 UTC (permalink / raw)
  To: Olaf Hering, Chunyan Liu
  Cc: jgross, wei.liu2, ian.campbell, george.dunlap, Ian.Jackson,
	xen-devel, jfehlig, Simon Cao

On 13/11/15 11:19, Olaf Hering wrote:
> On Wed, Oct 21, Chunyan Liu wrote:
> 
>> Add pvusb APIs, including:
> 
>> @@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [
>>      ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
>>      ("rdms", Array(libxl_device_rdm, "num_rdms")),
>>      ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")),
>> +    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")),
>> +    ("usbs", Array(libxl_device_usb, "num_usbs")),
> 
> Should that perhaps be "usbctrls" + "usbdevs" if the latter really
> represents a device below one of the "usbctrls"? I dont know if it does,
> just wondering.

That seems like a reasonable suggestion.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 5/7] xl: add pvusb commands
  2015-11-13  2:43     ` Chun Yan Liu
@ 2015-11-16 10:05       ` George Dunlap
  0 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-16 10:05 UTC (permalink / raw)
  To: Chun Yan Liu, George Dunlap
  Cc: Juergen Gross, Wei Liu, Ian Campbell, Ian Jackson, xen-devel,
	Jim Fehlig, Simon Cao

On 13/11/15 02:43, Chun Yan Liu wrote:
>>> +By default, it will create a USB2.0 controller with 8 ports. 
>>> + 
>>> +=item B<usbctrl-detach> I<domain-id> I<devid> 
>>> + 
>>> +Destroy a USB controller from the specified domain. 
>>> +B<devid> is devid of the USB controller. 
>>> + 
>>> +If B<-f> is specified, B<xl> is going to forcefully remove the device even 
>>> +without guest's collaboration. 
>>  
>> "If B<-f> is specified, B<xl> will forcefully remove removal (i.e., 
>> without the guest's cooperation)." 

Sorry, I'm looking again at what I wrote here and it's gibberish!  Let
me try again:

"If B<-f> is specified, B<xl> will force removal (i.e., without the
guest's cooperation)."

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file
  2015-10-21  9:08 ` [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file Chunyan Liu
  2015-10-27 11:31   ` Juergen Gross
@ 2015-11-16 14:03   ` Ian Campbell
  2015-11-16 18:15     ` Ian Jackson
  1 sibling, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-11-16 14:03 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: george.dunlap, jgross, jfehlig, Ian.Jackson, wei.liu2

On Wed, 2015-10-21 at 17:08 +0800, Chunyan Liu wrote:
> Sysfs file has size=4096 but actual file content is less than that.
> Current libxl_read_file_contents will treat it as error when file size
> and actual file content differs, so reading sysfs file content with
> this function always fails.

You changes to the existing code seem to do more than just cope with this
possibility.


[...]
@@ -359,20 +360,35 @@ int libxl_read_file_contents(libxl_ctx *ctx, const
> char *filename,
>      datalen = stab.st_size;
>  
>      if (stab.st_size && data_r) {
> -        data = malloc(datalen);
> +        data = malloc(datalen + 1);

This (and the related change) seem to be fixing an off by one bug in the
existing code? At a minimum this should be mentioned in the commit message
but ideally it would be split out into a precursor code, so as to allow it
to be backported (assuming it is a bug fix and not some other consequence
of reading from sysfs).

>          if (!data) goto xe;
>  
> -        rs = fread(data, 1, datalen, f);
> -        if (rs != datalen) {
> -            if (ferror(f))
> +        rs = fread(data, 1, datalen + 1, f);
> +        if (rs > datalen) {
> +            LOG(ERROR, "%s increased size while we were reading it",
> +                filename);
> +            goto xe;
> +        }
> +
> +        if (rs < datalen) {
> +            if (ferror(f)) {
>                  LOGE(ERROR, "failed to read %s", filename);
> -            else if (feof(f))
> -                LOG(ERROR, "%s changed size while we were reading it",
> -		    filename);
> -            else
> +                goto xe;
> +            } else if (feof(f)) {
> +                if (tolerate_shrinking_file) {
> +                    datalen = rs;
> +                } else {
> +                    LOG(ERROR, "%s shrunk size while we were reading
> it",
> +                        filename);
> +                    goto xe;
> +                }
> +            } else {
>                  abort();
> -            goto xe;
> +            }
>          }
> +
> +        data = realloc(data, datalen);
> +        if (!data) goto xe;

and this change I don't follow at all. Is it shrinking the buffer? I'm not
sure that is needed, it should be a smallish waste of memory.

If you feel strongly that the realloc is needed then you should call
libxl__realloc(NOGC, ...) to get the correct behaviour on error.

That NOGC brings me onto my last point:
> +int libxl__read_sysfs_file_contents(libxl__gc *gc, const char *filename,
> +                                   void **data_r, int *datalen_r)
> +{
> +    return read_file_contents_core(gc, filename, data_r, datalen_r, 1);

Since this is now an internal function it would normally be expected that
the returned buffer would be garbage collected.

Since you want to share the core code with an external function which
should return un-gc'd memory it seems like the easiest change would be to
call libxl__ptr_add here.

Ian.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-13  2:30         ` Chun Yan Liu
@ 2015-11-16 18:06           ` Ian Jackson
  2015-11-17  5:47             ` Chun Yan Liu
  0 siblings, 1 reply; 53+ messages in thread
From: Ian Jackson @ 2015-11-16 18:06 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	Jim Fehlig, Simon Cao

Thanks for your attention to my earlier mail.  I'll delete all the
comments where we agree :-).

> > > > > +/* bind/unbind usb device interface */  
> > > > > +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)  
> > > > > +{  
> > ... 
> > > > > +        dp = libxl__zalloc(gc, PATH_MAX);  
> > > > > +        dp = realpath(spath, dp);  
> > > >   
> > > > Why is this call to realpath needed ? 
> > >  
> > > In sysfs, /driver sometimes is a link, since we need to save the original 
> > > driver to xenstore, so need to get the realpath of the driver. 
> >  
> > I mean, why is the path with all the symlinks in it not suitable for 
> > storage and later use ? 
> 
> The symlink might be "../../../../../../bus/usb/drivers/btusb", we
> couldn't save that to xenstore for later usage.

So the real reason is simply that it is a relative path and we need an
absolute one, because a relative path is not useful ?

OK, thanks for the explanation.  (I'm not sure whether this is
unobvious enough to warrant a comment, perhaps
 /* sysfs can produce relative paths */
or something.)

> > > > I think you probably do things in the wrong order here.  You should  
> > > > write the old driver info to xenstore first, so that if the bind to  
> > > > usbback fails, you have the old driver info.  
> > >  
> > > Perhaps not. Once we finished adding entries to xenstore, pvusb 
> > > frontend/backend drivers will detect the change and do probe work, if 
> > > USB device is still not bound to USBBACK, there might be error. 
> >  
> > What I mean is this: 
> >  
> > Is it not possible to write the original path to xenstore before doing 
> > the unbind ?  Otherwise it seems like there could be error paths where 
> > the original path is not recorded, the xenstore write fails, and then 
> > the information about how to rebind to the original driver has been 
> > lost. 
> 
> I see. 

Right.  Do you think this warrants a change to the code ?

> > Does writing USBBACK_INFO_PATH"/%s/%s/driver_path" really trigger 
> > usbback ? 
> 
> No, writing driver_path info to xenstore won't trigger usbback. Writing
> frontend/backend info will.

Jolly good.

> > > If we merge libxl__device_usb_add and do_usb_add, then it's cleaner. 
> >  
> > See my comment below.  You've explained the distinction to my 
> > satisfaction. 
> >  
> > But, to solve the duplication of the controller info acquisition, 
> > perhaps you could have do_usb_add take the controller info as a 
> > paramaeter ?
> 
> Yes, could be. Only do_usb_add and do_usb_remove parameters are not
> symmetrical. 

I don't think that matters.  (If it did it would be better to add an
unused parameter to the remove function.)

Regards,
Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file
  2015-11-16 14:03   ` Ian Campbell
@ 2015-11-16 18:15     ` Ian Jackson
  0 siblings, 0 replies; 53+ messages in thread
From: Ian Jackson @ 2015-11-16 18:15 UTC (permalink / raw)
  To: Ian Campbell
  Cc: jgross, wei.liu2, george.dunlap, Chunyan Liu, xen-devel, jfehlig

Ian Campbell writes ("Re: [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file"):
> @@ -359,20 +360,35 @@ int libxl_read_file_contents(libxl_ctx *ctx, const
> > char *filename,
> >      datalen = stab.st_size;
> >  
> >      if (stab.st_size && data_r) {
> > -        data = malloc(datalen);
> > +        data = malloc(datalen + 1);
> 
> This (and the related change) seem to be fixing an off by one bug in the
> existing code? At a minimum this should be mentioned in the commit message
> but ideally it would be split out into a precursor code, so as to allow it
> to be backported (assuming it is a bug fix and not some other consequence
> of reading from sysfs).

There is no off-by-one error in the existing code.  I think this +1
arises because Chunyan wants libxl__read_sysfs_file_contents to
produce a nul-terminated string, even though libxl_read_file_contents
doesn't.

In June I wrote:

  If you want to make an API which is useable for [code which wants a
  trailing nul], and not intended for byte blocks, that's fine, but
  you must then always nul-terminate (not only if the data was less
  than 4k) and your new function probably doesn't want to return a
  length at all.

I am coming to the thought that it might be better for this to be two
functions with entirely separate implementations.  The resulting
unified function is already full of tangled conditional logic, and our
complaint here will involve yet another conditional.

Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-16 18:06           ` Ian Jackson
@ 2015-11-17  5:47             ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-17  5:47 UTC (permalink / raw)
  To: Ian Jackson
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/17/2015 at 02:06 AM, in message
<22090.6954.35639.703183@mariner.uk.xensource.com>, Ian Jackson
<Ian.Jackson@eu.citrix.com> wrote: 
> Thanks for your attention to my earlier mail.  I'll delete all the 
> comments where we agree :-). 
>  
> > > > > > +/* bind/unbind usb device interface */   
> > > > > > +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)   
>  
> > > > > > +{   
> > > ...  
> > > > > > +        dp = libxl__zalloc(gc, PATH_MAX);   
> > > > > > +        dp = realpath(spath, dp);   
> > > > >    
> > > > > Why is this call to realpath needed ?  
> > > >   
> > > > In sysfs, /driver sometimes is a link, since we need to save the original  
>  
> > > > driver to xenstore, so need to get the realpath of the driver.  
> > >   
> > > I mean, why is the path with all the symlinks in it not suitable for  
> > > storage and later use ?  
> >  
> > The symlink might be "../../../../../../bus/usb/drivers/btusb", we 
> > couldn't save that to xenstore for later usage. 
>  
> So the real reason is simply that it is a relative path and we need an 
> absolute one, because a relative path is not useful ? 
>  
> OK, thanks for the explanation.  (I'm not sure whether this is 
> unobvious enough to warrant a comment, perhaps 
>  /* sysfs can produce relative paths */ 
> or something.) 
>  
> > > > > I think you probably do things in the wrong order here.  You should   
> > > > > write the old driver info to xenstore first, so that if the bind to   
> > > > > usbback fails, you have the old driver info.   
> > > >   
> > > > Perhaps not. Once we finished adding entries to xenstore, pvusb  
> > > > frontend/backend drivers will detect the change and do probe work, if  
> > > > USB device is still not bound to USBBACK, there might be error.  
> > >   
> > > What I mean is this:  
> > >   
> > > Is it not possible to write the original path to xenstore before doing  
> > > the unbind ?  Otherwise it seems like there could be error paths where  
> > > the original path is not recorded, the xenstore write fails, and then  
> > > the information about how to rebind to the original driver has been  
> > > lost.  
> >  
> > I see.  
>  
> Right.  Do you think this warrants a change to the code ? 
>  
> > > Does writing USBBACK_INFO_PATH"/%s/%s/driver_path" really trigger  
> > > usbback ?  
> >  
> > No, writing driver_path info to xenstore won't trigger usbback. Writing 
> > frontend/backend info will. 
>  
> Jolly good. 
>  
> > > > If we merge libxl__device_usb_add and do_usb_add, then it's cleaner.  
> > >   
> > > See my comment below.  You've explained the distinction to my  
> > > satisfaction.  
> > >   
> > > But, to solve the duplication of the controller info acquisition,  
> > > perhaps you could have do_usb_add take the controller info as a  
> > > paramaeter ? 
> >  
> > Yes, could be. Only do_usb_add and do_usb_remove parameters are not 
> > symmetrical.  
>  
> I don't think that matters.  (If it did it would be better to add an 
> unused parameter to the remove function.) 

Take all. Will update codes.

>  
> Regards, 
> Ian. 
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-16 10:01     ` George Dunlap
@ 2015-11-18  5:48       ` Chun Yan Liu
  2015-11-18  9:44         ` Olaf Hering
  0 siblings, 1 reply; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-18  5:48 UTC (permalink / raw)
  To: Olaf Hering, George Dunlap
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap,
	Ian.Jackson, xen-devel, Jim Fehlig, Simon Cao



>>> On 11/16/2015 at 06:01 PM, in message <5649A988.7010609@citrix.com>, George
Dunlap <george.dunlap@citrix.com> wrote: 
> On 13/11/15 11:19, Olaf Hering wrote: 
> > On Wed, Oct 21, Chunyan Liu wrote: 
> >  
> >> Add pvusb APIs, including: 
> >  
> >> @@ -635,6 +664,8 @@ libxl_domain_config = Struct("domain_config", [ 
> >>      ("pcidevs", Array(libxl_device_pci, "num_pcidevs")), 
> >>      ("rdms", Array(libxl_device_rdm, "num_rdms")), 
> >>      ("dtdevs", Array(libxl_device_dtdev, "num_dtdevs")), 
> >> +    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")), 
> >> +    ("usbs", Array(libxl_device_usb, "num_usbs")), 
> >  
> > Should that perhaps be "usbctrls" + "usbdevs" if the latter really 
> > represents a device below one of the "usbctrls"? I dont know if it does, 
> > just wondering. 
>  
> That seems like a reasonable suggestion. 

OK. I'll update "usbs" to "usbdevs".
I think libxl_device_usb doesn't need to be changed into libxl_device_usbdev? 

>  
>  -George 
>  
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-18  5:48       ` Chun Yan Liu
@ 2015-11-18  9:44         ` Olaf Hering
  2015-11-18 10:03           ` Ian Campbell
  2015-11-19  1:33           ` Chun Yan Liu
  0 siblings, 2 replies; 53+ messages in thread
From: Olaf Hering @ 2015-11-18  9:44 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap,
	Ian.Jackson, George Dunlap, xen-devel, Jim Fehlig, Simon Cao

On Tue, Nov 17, Chun Yan Liu wrote:

> I think libxl_device_usb doesn't need to be changed into libxl_device_usbdev? 

In case of vscsi the struct and functions names are odd. It was not
obvious which one belongs to a ctrl and which one belongs to a device.
In the meantime I have changed everything related to a scsi host, it
contains now 'vscsictrl'. The names related to devices will follow.

I suggest to do apply the same also to usb and make it clear what
belongs to a ctrl and what to a device. Havent checked your patch what
places that actually would be.

Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-18  9:44         ` Olaf Hering
@ 2015-11-18 10:03           ` Ian Campbell
  2015-11-18 10:42             ` Olaf Hering
  2015-11-19  1:33           ` Chun Yan Liu
  1 sibling, 1 reply; 53+ messages in thread
From: Ian Campbell @ 2015-11-18 10:03 UTC (permalink / raw)
  To: Olaf Hering, Chun Yan Liu
  Cc: Juergen Gross, wei.liu2, george.dunlap, Ian.Jackson,
	George Dunlap, xen-devel, Jim Fehlig, Simon Cao

On Wed, 2015-11-18 at 10:44 +0100, Olaf Hering wrote:
> On Tue, Nov 17, Chun Yan Liu wrote:
> 
> > I think libxl_device_usb doesn't need to be changed into
> > libxl_device_usbdev? 
> 
> In case of vscsi the struct and functions names are odd. It was not
> obvious which one belongs to a ctrl and which one belongs to a device.
> In the meantime I have changed everything related to a scsi host, it
> contains now 'vscsictrl'. The names related to devices will follow.
> 
> I suggest to do apply the same also to usb and make it clear what
> belongs to a ctrl and what to a device. Havent checked your patch what
> places that actually would be.

Whatever you guys decide on please make sure the model used for these two
device types are consistent with each other (and potentially other future
device kinds using this model) and where it makes sense consistent with the
controller-less device types whose template set of operations is already
defined in libxl.h

I'd been hoping that someone involved ion this would generate a patch
adding a template for this controller+devices model to libxl.h, I've not
seen anything since George's original RFC[0] "libxl: Introduce a template
for devices with a controller".

Ian.

[0] <1432228052-15667-1-git-send-email-george.dunlap@eu.citrix.com>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-18 10:03           ` Ian Campbell
@ 2015-11-18 10:42             ` Olaf Hering
  0 siblings, 0 replies; 53+ messages in thread
From: Olaf Hering @ 2015-11-18 10:42 UTC (permalink / raw)
  To: Ian Campbell
  Cc: Juergen Gross, xen-devel, wei.liu2, george.dunlap, Ian.Jackson,
	Chun Yan Liu, George Dunlap, Jim Fehlig, Simon Cao

On Wed, Nov 18, Ian Campbell wrote:

> I'd been hoping that someone involved ion this would generate a patch
> adding a template for this controller+devices model to libxl.h, I've not
> seen anything since George's original RFC[0] "libxl: Introduce a template
> for devices with a controller".

Its still on my list. So far its not clear to me how it will look like
in the end, depends on what the existing backend drivers expect.

Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-18  9:44         ` Olaf Hering
  2015-11-18 10:03           ` Ian Campbell
@ 2015-11-19  1:33           ` Chun Yan Liu
  2015-11-19  6:24             ` Chun Yan Liu
  1 sibling, 1 reply; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-19  1:33 UTC (permalink / raw)
  To: Olaf Hering
  Cc: Juergen Gross, wei.liu2, ian.campbell, george.dunlap,
	Ian.Jackson, George Dunlap, xen-devel, Jim Fehlig, Simon Cao



>>> On 11/18/2015 at 05:44 PM, in message <20151118094410.GB21970@aepfle.de>, Olaf
Hering <olaf@aepfle.de> wrote: 
> On Tue, Nov 17, Chun Yan Liu wrote: 
>  
> > I think libxl_device_usb doesn't need to be changed into  
> libxl_device_usbdev?  
>  
> In case of vscsi the struct and functions names are odd. It was not 
> obvious which one belongs to a ctrl and which one belongs to a device. 
> In the meantime I have changed everything related to a scsi host, it 
> contains now 'vscsictrl'. The names related to devices will follow. 
>  
> I suggest to do apply the same also to usb and make it clear what 
> belongs to a ctrl and what to a device. Havent checked your patch what 
> places that actually would be. 

Currently in pvusb patches, all places using 'usbctrl' means usb controller,
'usb' means usb device.

- Chunyan

>  
> Olaf 
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-19  1:33           ` Chun Yan Liu
@ 2015-11-19  6:24             ` Chun Yan Liu
  2015-11-23 17:24               ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Chun Yan Liu @ 2015-11-19  6:24 UTC (permalink / raw)
  To: Olaf Hering, george.dunlap, Chun Yan Liu
  Cc: Juergen Gross, wei.liu2, ian.campbell, Ian.Jackson, xen-devel,
	Jim Fehlig, Simon Cao



>>> On 11/19/2015 at 09:33 AM, in message
<564D97610200006600085301@relay2.provo.novell.com>, "Chun Yan Liu"
<cyliu@suse.com> wrote: 

>  
>>>> On 11/18/2015 at 05:44 PM, in message <20151118094410.GB21970@aepfle.de>, Olaf 
> Hering <olaf@aepfle.de> wrote:  
> > On Tue, Nov 17, Chun Yan Liu wrote:  
> >   
> > > I think libxl_device_usb doesn't need to be changed into   
> > libxl_device_usbdev?   

George & Olaf,

About the naming, can we get to a decision?
e.g.
* usb controller and everything related, using "usbctrl"
* usb device and everything related, using "usbdev" (?)
Currently in pvusb, almost everywhere referring to a usb device, we use "usb".
Like: libxl_device_usb, libxl_device_usb_add/remove, etc.

If we decide, I can update all together.

- Chunyan

> >   
> > In case of vscsi the struct and functions names are odd. It was not  
> > obvious which one belongs to a ctrl and which one belongs to a device.  
> > In the meantime I have changed everything related to a scsi host, it  
> > contains now 'vscsictrl'. The names related to devices will follow.  
> >   
> > I suggest to do apply the same also to usb and make it clear what  
> > belongs to a ctrl and what to a device. Havent checked your patch what  
> > places that actually would be.  
>  
> Currently in pvusb patches, all places using 'usbctrl' means usb controller, 
> 'usb' means usb device. 
>  
> - Chunyan 
>  
> >   
> > Olaf  
> >   
> > _______________________________________________  
> > Xen-devel mailing list  
> > Xen-devel@lists.xen.org  
> > http://lists.xen.org/xen-devel  
> >   
> >   
>  
>  
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V8 3/7] libxl: add pvusb API
  2015-11-19  6:24             ` Chun Yan Liu
@ 2015-11-23 17:24               ` George Dunlap
  0 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-11-23 17:24 UTC (permalink / raw)
  To: Chun Yan Liu, Olaf Hering, george.dunlap
  Cc: Juergen Gross, wei.liu2, ian.campbell, Ian.Jackson, xen-devel,
	Jim Fehlig, Simon Cao

On 19/11/15 06:24, Chun Yan Liu wrote:
> 
> 
>>>> On 11/19/2015 at 09:33 AM, in message
> <564D97610200006600085301@relay2.provo.novell.com>, "Chun Yan Liu"
> <cyliu@suse.com> wrote: 
> 
>>  
>>>>> On 11/18/2015 at 05:44 PM, in message <20151118094410.GB21970@aepfle.de>, Olaf 
>> Hering <olaf@aepfle.de> wrote:  
>>> On Tue, Nov 17, Chun Yan Liu wrote:  
>>>   
>>>> I think libxl_device_usb doesn't need to be changed into   
>>> libxl_device_usbdev?   
> 
> George & Olaf,
> 
> About the naming, can we get to a decision?
> e.g.
> * usb controller and everything related, using "usbctrl"
> * usb device and everything related, using "usbdev" (?)
> Currently in pvusb, almost everywhere referring to a usb device, we use "usb".
> Like: libxl_device_usb, libxl_device_usb_add/remove, etc.
> 
> If we decide, I can update all together.

So I finally went back and spent some time mulling over the e-mail
thread we had before about designing the interface.

Just as a reminder, SCSI has a topology of host / bus / target / LUN;
USB has a topology controller / hub[bus] / [port]device / interface.

At the moment in USB we're not dealing with virtual hubs, so each
controller will have a single bus.  Additionally, we are only passing
through and/or creating devices; a physical device with more than one
interface will have all of its interfaces passed through transparently.
 So we're only effectively exposing two levels in our API.

In the previous discussion, for pvscsi, Juergen argued it makes sense to
be able to assign LUNs from the same physical target to different
virtual targets; and it also makes sense to be able to assign
newly-created physical LUNs to existing virtual targets.

(Please people correct me if I've misunderstood something anywhere.)

So for SCSI, there may be three levels at which people want to be able
to do things.

Given, that, I wonder if it would make sense to name the different
"levels" for these multi-level devices after the name for that level in
their respective specificaitons.

I.e., libxl_device_usbctrl, libxl_device_usbdev, libxl_device_scsihost,
libxl_device_scsitgt, libxl_device_scsilun or something like that.

In which case, "usbdev" would be indicated (since it's the "device"
we're talking about).

Thoughts?

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2015-11-23 17:24 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-10-21  9:08 [PATCH V8 0/7] xen pvusb toolstack work Chunyan Liu
2015-10-21  9:08 ` [PATCH V8 1/7] libxl: export some functions for pvusb use Chunyan Liu
2015-10-27 11:08   ` Juergen Gross
2015-10-21  9:08 ` [PATCH V8 2/7] libxl_read_file_contents: add new entry to read sysfs file Chunyan Liu
2015-10-27 11:31   ` Juergen Gross
2015-11-16 14:03   ` Ian Campbell
2015-11-16 18:15     ` Ian Jackson
2015-10-21  9:08 ` [PATCH V8 3/7] libxl: add pvusb API Chunyan Liu
2015-10-27 11:31   ` Juergen Gross
2015-11-04  6:31   ` Chun Yan Liu
2015-11-05 15:54     ` George Dunlap
2015-11-09 18:11   ` Ian Jackson
2015-11-10  8:41     ` Chun Yan Liu
2015-11-10 17:57       ` George Dunlap
2015-11-10 18:11         ` Ian Jackson
2015-11-11  7:21           ` Chun Yan Liu
2015-11-11  2:37         ` Chun Yan Liu
2015-11-12 17:00       ` Ian Jackson
2015-11-13  2:30         ` Chun Yan Liu
2015-11-16 18:06           ` Ian Jackson
2015-11-17  5:47             ` Chun Yan Liu
2015-11-12 11:32   ` Olaf Hering
2015-11-13  2:32     ` Chun Yan Liu
2015-11-12 17:27   ` George Dunlap
2015-11-13  2:56     ` Chun Yan Liu
2015-11-13 11:19   ` Olaf Hering
2015-11-16 10:01     ` George Dunlap
2015-11-18  5:48       ` Chun Yan Liu
2015-11-18  9:44         ` Olaf Hering
2015-11-18 10:03           ` Ian Campbell
2015-11-18 10:42             ` Olaf Hering
2015-11-19  1:33           ` Chun Yan Liu
2015-11-19  6:24             ` Chun Yan Liu
2015-11-23 17:24               ` George Dunlap
2015-10-21  9:08 ` [PATCH V8 4/7] libxl: add libxl_device_usb_assignable_list API Chunyan Liu
2015-10-27 11:32   ` Juergen Gross
2015-11-11 16:07   ` George Dunlap
2015-10-21  9:08 ` [PATCH V8 5/7] xl: add pvusb commands Chunyan Liu
2015-10-27 11:37   ` Juergen Gross
2015-11-12 11:38   ` George Dunlap
2015-11-12 11:39     ` George Dunlap
2015-11-13  2:43     ` Chun Yan Liu
2015-11-16 10:05       ` George Dunlap
2015-11-12 14:42   ` Olaf Hering
2015-11-12 14:49     ` George Dunlap
2015-11-13  2:49     ` Chun Yan Liu
2015-10-21  9:08 ` [PATCH V8 6/7] xl: add usb-assignable-list command Chunyan Liu
2015-10-27 11:38   ` Juergen Gross
2015-11-12 11:44   ` George Dunlap
2015-10-21  9:08 ` [PATCH V8 7/7] domcreate: support pvusb in configuration file Chunyan Liu
2015-10-27 11:41   ` Juergen Gross
2015-11-12 16:10   ` George Dunlap
2015-11-13  2:54     ` Chun Yan Liu

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.