All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH V3 0/6] libxl pvusb toolstack work
@ 2015-04-19  3:50 Chunyan Liu
  2015-04-19  3:50 ` [PATCH V3 1/6] libxl: export some functions for pvusb use Chunyan Liu
                   ` (5 more replies)
  0 siblings, 6 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

This patch series is to add pvusb toolstack work, supporting hot add|remove
USB device to|from guest and specify USB device in domain configuration file.

1/6~5/6 implement clean pvusb work. 6/6 is an independent patch to refactor
codes for merging QEMU emulated USB code later.

1/6 is to export some functions for later usage
2/6 fixes read_file_content limit when reading sysfs file, for later usage.
3/6 libxl pvusb API implementation
4/6 xl pvusb hotplug commands implementation
5/6 usb from config file implementation
6/6 changing codes to unify qemu and pvusb.

Changes to v2:
* change xl interfaces from specifing usb device by busid to specifing
  usb device by bus.addr, so that pvusb and qemu can use same interface
* remove usb-assignable-list interface
* call existing helper functions to read sysfs file content to get usb
  device info.
* reorganize patches, first implement clean pvusb work, then in last
  patch changing codes for pvusb and qemu merging.
* Many fixes to address coding style, build on different platform, etc.
  Codes adjustment to better view.

V2 is here:
http://lists.xen.org/archives/html/xen-devel/2015-01/threads.html#03636

Related Discussion Threads:
http://www.redhat.com/archives/libvir-list/2014-June/msg00038.html
http://lists.xen.org/archives/html/xen-devel/2014-06/msg00086.html

For a better understanding, a summary is posted in following.

1. Overview

There are two general methods for passing through individual host
devices to a guest. The first is via an emulated USB device
controller; the second is PVUSB.

Additionally, there are two ways to add USB devices to a guest: via
the config file at domain creation time, and via hot-plug while the VM
is running. 

* Emulated USB

In emulated USB, the device model (qemu) presents an emulated USB
controller to the guest. The device model process then grabs control
of the device from domain 0 and and passes the USB commands between
the guest OS and the host USB device.

This method is only available to HVM domains, and is not available for
domains running with device model stubdomains.

* PVUSB

PVUSB uses a paravirtialized front-end/back-end interface, similar to
the traditional Xen PV network and disk protocols. In order to use
PVUSB, you need usbfront in your guest OS, and usbback in dom0 (or
your USB driver domain).

Additionally, for easy use of PVUSB, you need support in the toolstack
to get the two sides to talk to each other.

2. Specifying a host USB device

QEMU hmp commands allows USB devices to be specified either by their
bus address (in the form bus.device) or their device tag (in the form
vendorid:deviceid).

Each way of specifying has its advantages:

    Specifying by device tag will always get the same device,
regardless of where the device ends up in the USB bus topology.
However, if there are two identical devices, it will not allow you to
specify which one. 

    Specifying by bus address will always allow you to choose a
specific device, even if you have duplicates. However, the bus address
may change depending on which port you plugged the device into, and
possibly also after a reboot. 

To avoid duplication of vendorid:deviceid, we'll use bus address to
specify host USB device in xl toolstack.

You can use lsusb to list the USB devices on the system:

Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0
Hub
Bus 003 Device 002: ID f617:0905  
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 004: ID 0424:2640 Standard Microsystems Corp. USB 2.0
Hub
Bus 001 Device 005: ID 0424:4060 Standard Microsystems Corp. Ultra
Fast Media Reader
Bus 001 Device 006: ID 046d:c016 Logitech, Inc. Optical Wheel Mouse

To pass through the Logitec mouse, for instance, you could specify
1.6 (remove leading zeroes).

Note: USB Hub could not be assigned to guest.

3. PVUSB toolstack

* Specify USB device in xl config file

You can just specify usb devices, like:
usb=['1.6']

Then it will create a USB controller automatically and attach the USB
device to the first available USB controller:port.

or, you can explicitly specify usb controllers and usb devices, like:
usbctrl=['verison=1, ports=4', 'version=2, ports=8', ]
usb=['1.6, controller=0, port=1']

Then it will create two USB controllers as you specified.
And if controller and port are specified in usb config, then it will
attach the USB device to that controller:port. About the controller
and port value:
Each USB controller has a index (or called devid) based on 0. The 1st
controller has index 0, the 2nd controller has index 1, ...
Under controller, each port has a port number based on 1. In above
configuration, the 1st controller will have port 1,2,3,4.

To be compatible with QEMU emulated USB, this interface could be
extended to:
usb=['1.6, type=pv|qemu',]
type=pv means attaching the USB device in pvusb way; type=qemu means
attaching USB device in QEMU emulated way.

* Hot-Plug USB device

To attach a USB device, you should first create a USB controller.
e.g.
xl usb-ctrl-attach domain [version=1|2] [ports=value]
By default, it will create a USB2.0 controller with 8 ports.

Then you could attach a USB device.
e.g.
xl usb-attach domain 1.6 [controller=index port=number]
By default, it will find the 1st available controller:port to attach
the USB device.

You could view USB device status of the domain by usb-list.
e.g.
xl usb-list domain
It will list USB controllers and USB devices under each controller.

You could detach a USB device with usb-detach command.
e.g.
xl usb-detach domain 1.6

You can also remove the whole USB controller by usb-ctrl-detach
command.
e.g.
xl usb-ctrl-detach domain 0
It will remove the USB controller with index 0 and all USB devices
under it.

To be compatible with QEMU emulated USB, these interfaces could be
extended to:
xl usb-attach domain 1.6 type=pv|qemu
xl usb-detach domain 1.6 type=pv|qemu
QEMU emulated USB won't handle USB controller create/delete through
xl, so usb-ctrl-attach|detach operations are currently pvusb only.

4. PVUSB Libxl implementation

* usb-ctrl-attach
To create a usb controller, we need:
1) generate usb controler related information
2) write usb controller frontend/backend info to xenstore
PVUSB frontend and backend driver will probe xenstore paths and build
connection between frontend and backend.

* usb-ctrl-detach
To remove a usb controller, we need:
1) check if the usb controller exists or not
2) remove all usb devices under controller
3) remove usb controller info from xenstore

* usb-attach
To attach a usb device, we need:
1) check if the usb device type is assignable
2) check if the usb device is already assigned to a domain
3) add 'busid' of the usb device to xenstore contoller/port/.
   PVUSB driver watches the xenstore changes and detects that,
   and needs to use 'busid' to do following work.
4) unbind usb device from original driver and bind to usbback.
   If usb device has many interfaces, then:
   - unbind each interface from its original driver and bind to usbback.
   - store the original driver to xenstore for later rebinding when
     detaching the device.

* usb-detach
To detach a usb device, we need:
1) check if the usb device is assigned to the domain
2) remove the usb device from xenstore controller/port.
3) unbind usb device from usbback and rebind to its original driver.
   If usb device has many interfaces, do it to each interface.

* usb-list
List all USB controllers and USB devices under each controller.


Chunyan Liu (6):
  libxl: export some functions for pvusb use
  libxl_read_file_contents: fix reading sysfs file
  libxl: add pvusb API
  xl: add pvusb commands
  domcreate: support pvusb in configuration file
  refactor codes to unify pvusb and qemu emulated usb

 docs/man/xl.cfg.pod.5                |   70 +++
 docs/man/xl.pod.1                    |   38 ++
 tools/libxl/Makefile                 |    2 +-
 tools/libxl/libxl.c                  |    8 +-
 tools/libxl/libxl.h                  |   45 ++
 tools/libxl/libxl_create.c           |   73 ++-
 tools/libxl/libxl_internal.h         |   20 +-
 tools/libxl/libxl_osdeps.h           |   13 +
 tools/libxl/libxl_pvusb.c            | 1042 ++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_types.idl          |   52 ++
 tools/libxl/libxl_types_internal.idl |    1 +
 tools/libxl/libxl_usb.c              |  224 ++++++++
 tools/libxl/libxl_utils.c            |    5 +-
 tools/libxl/xl.h                     |    5 +
 tools/libxl/xl_cmdimpl.c             |  364 +++++++++++-
 tools/libxl/xl_cmdtable.c            |   25 +
 16 files changed, 1974 insertions(+), 13 deletions(-)
 create mode 100644 tools/libxl/libxl_pvusb.c
 create mode 100644 tools/libxl/libxl_usb.c

-- 
1.8.5.2

^ permalink raw reply	[flat|nested] 53+ messages in thread

* [PATCH V3 1/6] libxl: export some functions for pvusb use
  2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
@ 2015-04-19  3:50 ` Chunyan Liu
  2015-04-20 16:25   ` Olaf Hering
  2015-05-18 14:05   ` Wei Liu
  2015-04-19  3:50 ` [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file Chunyan Liu
                   ` (4 subsequent siblings)
  5 siblings, 2 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
---
 tools/libxl/libxl.c          | 6 +++---
 tools/libxl/libxl_internal.h | 5 +++++
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index 511eef1..b05d18b 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1963,7 +1963,7 @@ out:
 /******************************************************************************/
 
 /* generic callback for devices that only need to set ao_complete */
-static void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev)
+void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev)
 {
     STATE_AO_GC(aodev->ao);
 
@@ -1986,7 +1986,7 @@ out:
 }
 
 /* common function to get next device id */
-static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
+int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
 {
     char *dompath, **l;
     unsigned int nb;
@@ -2005,7 +2005,7 @@ static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
     return nextid;
 }
 
-static int libxl__resolve_domid(libxl__gc *gc, const char *name,
+int libxl__resolve_domid(libxl__gc *gc, const char *name,
                                 uint32_t *domid)
 {
     if (!name)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 9c22309..42eb1b9 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -1080,6 +1080,9 @@ _hidden int libxl__init_console_from_channel(libxl__gc *gc,
                                              libxl__device_console *console,
                                              int dev_num,
                                              libxl_device_channel *channel);
+_hidden int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device);
+_hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,
+                                 uint32_t *domid);
 
 /*
  * For each aggregate type which can be used as an input we provide:
@@ -2208,6 +2211,8 @@ struct libxl__ao_device {
 
 /* Starts preparing to add/remove a bunch of devices. */
 _hidden void libxl__multidev_begin(libxl__ao *ao, libxl__multidev*);
+/* generic callback for devices that only need to set ao_complete */
+_hidden void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev);
 
 /* Prepares to add/remove one of many devices.
  * Calls libxl__prepare_ao_device on libxl__ao_device argument provided and
-- 
1.8.5.2

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file
  2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
  2015-04-19  3:50 ` [PATCH V3 1/6] libxl: export some functions for pvusb use Chunyan Liu
@ 2015-04-19  3:50 ` Chunyan Liu
  2015-05-18 14:23   ` Ian Jackson
  2015-05-18 14:25   ` Wei Liu
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
                   ` (3 subsequent siblings)
  5 siblings, 2 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

Sysfs file has size=4096 but actual file content is less than that.
Current libxl_read_file_contents will treat it as error when file size
and actual file content differs, so reading sysfs file content with
this function always fails. Fix it so that we can reuse this function
to get sysfs file content in later pvusb work.

Signed-off-by: Chunyan Liu <cyliu@suse.com>
---
 tools/libxl/libxl_utils.c | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
index 9053b27..18ad2b8 100644
--- a/tools/libxl/libxl_utils.c
+++ b/tools/libxl/libxl_utils.c
@@ -363,12 +363,9 @@ int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
         if (!data) goto xe;
 
         rs = fread(data, 1, datalen, f);
-        if (rs != datalen) {
+        if (rs != datalen && !feof(f)) {
             if (ferror(f))
                 LOGE(ERROR, "failed to read %s", filename);
-            else if (feof(f))
-                LOG(ERROR, "%s changed size while we were reading it",
-		    filename);
             else
                 abort();
             goto xe;
-- 
1.8.5.2

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
  2015-04-19  3:50 ` [PATCH V3 1/6] libxl: export some functions for pvusb use Chunyan Liu
  2015-04-19  3:50 ` [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file Chunyan Liu
@ 2015-04-19  3:50 ` Chunyan Liu
  2015-04-20  5:53   ` Juergen Gross
                     ` (5 more replies)
  2015-04-19  3:50 ` [PATCH V3 4/6] xl: add pvusb commands Chunyan Liu
                   ` (2 subsequent siblings)
  5 siblings, 6 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

Add pvusb APIs, including:
 - attach/detach (create/destroy) virtual usb controller.
 - attach/detach usb device
 - list usb controller and usb devices
 - some other helper functions

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
---
Changes to v2:
  * remove qemu emulated usb related definitions, keep the work pure pvusb.
    In last patch, will do all refactor work to unify pvusb and qemu emulated
    usb.
  * use bus.addr as user interface instead of busid to do usb-attach|detach
  * remove usb-assignable-list APIs and some other unnecessary APIs
  * reuse libxl_read_file_contents function instead of another new function
    to handle getting sysfs file content
  * fix build on different platforms as pci does
  * fix many coding style problems
  * address other comments in last version
  * adjust codes to let it look better

 tools/libxl/Makefile                 |    2 +-
 tools/libxl/libxl.c                  |    2 +
 tools/libxl/libxl.h                  |   45 ++
 tools/libxl/libxl_internal.h         |   11 +-
 tools/libxl/libxl_osdeps.h           |   13 +
 tools/libxl/libxl_pvusb.c            | 1201 ++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_types.idl          |   41 ++
 tools/libxl/libxl_types_internal.idl |    1 +
 8 files changed, 1314 insertions(+), 2 deletions(-)
 create mode 100644 tools/libxl/libxl_pvusb.c

diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index 1b16598..d52281f 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -95,7 +95,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
 			libxl_internal.o libxl_utils.o libxl_uuid.o \
 			libxl_json.o libxl_aoutils.o libxl_numa.o libxl_vnuma.o \
 			libxl_save_callout.o _libxl_save_msgs_callout.o \
-			libxl_qmp.o libxl_event.o libxl_fork.o $(LIBXL_OBJS-y)
+			libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o $(LIBXL_OBJS-y)
 LIBXL_OBJS += libxl_genid.o
 LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
 
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index b05d18b..a7c81d9 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1611,6 +1611,8 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
 
     if (libxl__device_pci_destroy_all(gc, domid) < 0)
         LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "pci shutdown failed for domid %d", domid);
+    if (libxl__device_usb_destroy_all(gc, domid) < 0)
+         LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "usb shutdown failed for domid %d", domid);
     rc = xc_domain_pause(ctx->xch, domid);
     if (rc < 0) {
         LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_pause failed for %d", domid);
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 6bc75c5..cbe3519 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -114,6 +114,12 @@
 #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
 
 /*
+ * LIBXL_HAVE_PVUSB indicates the functions for doing hot-plug of
+ * USB devices through pvusb.
+ */
+#define LIBXL_HAVE_PVUSB 1
+
+/*
  * LIBXL_HAVE_BUILDINFO_HVM_VENDOR_DEVICE indicates that the
  * libxl_vendor_device field is present in the hvm sections of
  * libxl_domain_build_info. This field tells libxl which
@@ -1224,6 +1230,45 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
                        const libxl_asyncop_how *ao_how)
                        LIBXL_EXTERNAL_CALLERS_ONLY;
 
+/* USB Controllers*/
+int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,
+                         libxl_device_usbctrl *usbctrl,
+                         const libxl_asyncop_how *ao_how)
+                         LIBXL_EXTERNAL_CALLERS_ONLY;
+
+int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid,
+                         libxl_device_usbctrl *usbctrl,
+                         const libxl_asyncop_how *ao_how)
+                         LIBXL_EXTERNAL_CALLERS_ONLY;
+
+int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid,
+                         libxl_device_usbctrl *usbctrl,
+                         const libxl_asyncop_how *ao_how)
+                         LIBXL_EXTERNAL_CALLERS_ONLY;
+
+libxl_device_usbctrl *libxl_device_usbctrl_list(libxl_ctx *ctx,
+                            uint32_t domid, int *num);
+
+
+int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
+                                libxl_device_usbctrl *usbctrl,
+                                libxl_usbctrlinfo *usbctrlinfo)
+                                LIBXL_EXTERNAL_CALLERS_ONLY;
+
+/* USB Devices */
+int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
+                         const libxl_asyncop_how *ao_how)
+                         LIBXL_EXTERNAL_CALLERS_ONLY;
+
+int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
+                            const libxl_asyncop_how *ao_how)
+                            LIBXL_EXTERNAL_CALLERS_ONLY;
+
+libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid,
+                                        int usbctrl, int *num);
+
+int libxl_device_usb_getinfo(libxl_ctx *ctx, char *intf, libxl_usbinfo *usbinfo)
+                             LIBXL_EXTERNAL_CALLERS_ONLY;
 /* Network Interfaces */
 int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic *nic,
                          const libxl_asyncop_how *ao_how)
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index 42eb1b9..f426ed8 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -2422,6 +2422,13 @@ _hidden void libxl__device_vtpm_add(libxl__egc *egc, uint32_t domid,
                                    libxl_device_vtpm *vtpm,
                                    libxl__ao_device *aodev);
 
+/* from libxl_usb */
+_hidden int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
+                                      libxl_device_usbctrl *usbctrl);
+_hidden int libxl__device_usb_add(libxl__gc *gc, uint32_t domid,
+                            libxl_device_usb *usb);
+_hidden int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid);
+
 /* Internal function to connect a vkb device */
 _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid,
                                   libxl_device_vkb *vkb);
@@ -3628,7 +3635,9 @@ static inline void libxl__update_config_vtpm(libxl__gc *gc,
 #define COMPARE_PCI(a, b) ((a)->func == (b)->func &&    \
                            (a)->bus == (b)->bus &&      \
                            (a)->dev == (b)->dev)
-
+#define COMPARE_USB(a, b) (!strcmp((a)->busid, (b)->busid))
+#define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid)
+ 
 /* DEVICE_ADD
  *
  * Add a device in libxl_domain_config structure
diff --git a/tools/libxl/libxl_osdeps.h b/tools/libxl/libxl_osdeps.h
index 08eaf0c..55caf71 100644
--- a/tools/libxl/libxl_osdeps.h
+++ b/tools/libxl/libxl_osdeps.h
@@ -24,6 +24,8 @@
 #define _GNU_SOURCE
 
 #if defined(__NetBSD__)
+#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
+#define SYSFS_USBBACK_DRIVER   "/kern/xen/usb"
 #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
 #define SYSFS_PCIBACK_DRIVER   "/kern/xen/pci"
 #define NETBACK_NIC_NAME       "xvif%ui%d"
@@ -31,6 +33,8 @@
 #elif defined(__OpenBSD__)
 #include <util.h>
 #elif defined(__linux__)
+#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
+#define SYSFS_USBBACK_DRIVER   "/sys/bus/usb/drivers/usbback"
 #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
 #define SYSFS_PCIBACK_DRIVER   "/sys/bus/pci/drivers/pciback"
 #define NETBACK_NIC_NAME       "vif%u.%d"
@@ -38,12 +42,21 @@
 #elif defined(__sun__)
 #include <stropts.h>
 #elif defined(__FreeBSD__)
+#define SYSFS_USB_DEV          "/dev/null"
+#define SYSFS_USBBACK_DRIVER   "/dev/null"
 #define SYSFS_PCI_DEV          "/dev/null"
 #define SYSFS_PCIBACK_DRIVER   "/dev/null"
 #define NETBACK_NIC_NAME       "xnb%u.%d"
 #include <libutil.h>
 #endif
 
+#ifndef SYSFS_USBBACK_DRIVER
+#error define SYSFS_USBBACK_DRIVER for your platform
+#endif
+#ifndef SYSFS_USB_DEV
+#error define SYSFS_USB_DEV for your platform
+#endif
+
 #ifndef SYSFS_PCIBACK_DRIVER
 #error define SYSFS_PCIBACK_DRIVER for your platform
 #endif
diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
new file mode 100644
index 0000000..4e4975a
--- /dev/null
+++ b/tools/libxl/libxl_pvusb.c
@@ -0,0 +1,1201 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+
+#include "libxl_internal.h"
+
+#define USBBACK_INFO_PATH "/libxl/usbback"
+
+#define USBHUB_CLASS_CODE 0x09
+
+static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
+                                            libxl_device_usbctrl *usbctrl)
+{
+    int rc;
+
+    if (!usbctrl->version)
+        usbctrl->version = 2;
+
+    if (!usbctrl->ports)
+        usbctrl->ports = 8;
+
+    rc = libxl__resolve_domid(gc, usbctrl->backend_domname,
+                              &usbctrl->backend_domid);
+    return rc;
+}
+
+static void libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid,
+                                       libxl_device_usbctrl *usbctrl,
+                                       libxl__device *device)
+{
+    device->backend_devid   = usbctrl->devid;
+    device->backend_domid   = usbctrl->backend_domid;
+    device->backend_kind    = LIBXL__DEVICE_KIND_VUSB;
+    device->devid           = usbctrl->devid;
+    device->domid           = domid;
+    device->kind            = LIBXL__DEVICE_KIND_VUSB;
+}
+
+static int libxl__usbport_add_xenstore(libxl__gc *gc,
+                                       xs_transaction_t tran,
+                                       uint32_t domid,
+                                       libxl_device_usbctrl *usbctrl)
+{
+    char *path;
+    int i;
+
+    path = GCSPRINTF("%s/backend/vusb/%d/%d/port",
+                     libxl__xs_get_dompath(gc, 0), domid, usbctrl->devid);
+
+    libxl__xs_mkdir(gc, tran, path, NULL, 0);
+
+    for (i = 1; i <= usbctrl->ports; i++) {
+        if (libxl__xs_write_checked(gc, tran, GCSPRINTF("%s/%d", path, i), ""))
+            return ERROR_FAIL;
+    }
+
+    return 0;
+}
+
+static int libxl__usbctrl_add_xenstore(libxl__gc *gc, uint32_t domid,
+                                       libxl_device_usbctrl *usbctrl)
+{
+    flexarray_t *front;
+    flexarray_t *back;
+    libxl__device *device;
+    xs_transaction_t t = XBT_NULL;
+    int rc = 0;
+    libxl_domain_config d_config;
+    libxl_device_usbctrl usbctrl_saved;
+    libxl__domain_userdata_lock *lock = NULL;
+
+    libxl_domain_config_init(&d_config);
+    libxl_device_usbctrl_init(&usbctrl_saved);
+    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl);
+
+    GCNEW(device);
+    libxl__device_from_usbctrl(gc, domid, usbctrl, device);
+
+    front = flexarray_make(gc, 4, 1);
+    back = flexarray_make(gc, 12, 1);
+
+    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid));
+    flexarray_append_pair(back, "online", "1");
+    flexarray_append_pair(back, "state", "1");
+    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d", usbctrl->version));
+    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d", usbctrl->ports));
+    flexarray_append_pair(front, "backend-id", GCSPRINTF("%d", usbctrl->backend_domid));
+    flexarray_append_pair(front, "state", "1");
+
+    lock = libxl__lock_domain_userdata(gc, domid);
+    if (!lock) {
+        rc = ERROR_LOCK_FAIL;
+        goto out;
+    }
+
+    rc = libxl__get_domain_configuration(gc, domid, &d_config);
+    if (rc) goto out;
+
+    DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved, COMPARE_USBCTRL, &d_config);
+
+    for (;;) {
+        rc = libxl__xs_transaction_start(gc, &t);
+        if (rc) goto out;
+
+        rc = libxl__device_exists(gc, t, device);
+        if (rc < 0) goto out;
+        if (rc == 1) {
+            /* already exists in xenstore */
+            LOG(ERROR, "device already exists in xenstore");
+            rc = ERROR_DEVICE_EXISTS;
+            goto out;
+        }
+
+        rc = libxl__set_domain_configuration(gc, domid, &d_config);
+        if (rc) goto out;
+
+        libxl__device_generic_add(gc, t, device,
+                                  libxl__xs_kvs_of_flexarray(gc, back,
+                                                             back->count),
+                                  libxl__xs_kvs_of_flexarray(gc, front,
+                                                             front->count),
+                                  NULL);
+        libxl__usbport_add_xenstore(gc, t, domid, usbctrl);
+        rc = libxl__xs_transaction_commit(gc, &t);
+        if (!rc) break;
+        if (rc < 0) goto out;
+    }
+
+out:
+    if (lock) libxl__unlock_domain_userdata(lock);
+    libxl_device_usbctrl_dispose(&usbctrl_saved);
+    libxl_domain_config_dispose(&d_config);
+    return rc;
+}
+
+int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
+                              libxl_device_usbctrl *usbctrl)
+{
+    int rc = 0;
+
+    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
+    if (rc < 0) goto out;
+
+    if (usbctrl->devid == -1) {
+        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");
+        if (usbctrl->devid < 0) {
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    }
+
+    if (libxl__usbctrl_add_xenstore(gc, domid, usbctrl) < 0){
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+out:
+    return rc;
+}
+
+int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,
+                             libxl_device_usbctrl *usbctrl,
+                             const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    int rc;
+
+    rc = libxl__device_usbctrl_add(gc, domid, usbctrl);
+    libxl__ao_complete(egc, ao, rc);
+    return AO_INPROGRESS;
+}
+
+static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int usbctrl,
+                                  libxl_device_usb **usbs, int *num);
+
+static int do_usb_remove(libxl__gc *gc, uint32_t domid,
+                         libxl_device_usb *usb);
+
+static int
+libxl__device_usbctrl_remove_common(libxl_ctx *ctx, uint32_t domid,
+                                    libxl_device_usbctrl *usbctrl,
+                                    const libxl_asyncop_how *ao_how,
+                                    int force)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    libxl__device *device;
+    libxl__ao_device *aodev;
+    libxl_device_usbctrl *usbctrls = NULL;
+    libxl_device_usbctrl *usbctrl_find = NULL;
+    int numctrl = 0;
+    libxl_device_usb *usbs = NULL;
+    int numusb = 0;
+    int i, rc;
+
+    assert(usbctrl->devid != -1);
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
+    if (!numctrl) {
+        LOG(ERROR, "No USB controller exists in this domain");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    for (i = 0; i < numctrl; i++) {
+        if (usbctrl->devid == usbctrls[i].devid) {
+            usbctrl_find = usbctrls + i;
+            break;
+        }
+    }
+
+    if (!usbctrl_find) {
+        LOG(ERROR, "USB controller %d is not attached to this domain",
+            usbctrl->devid);
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    usbctrl = usbctrl_find;
+
+    GCNEW(device);
+    libxl__device_from_usbctrl(gc, domid, usbctrl, device);
+
+    /* Remove usb devives first */
+    rc  = libxl__device_usb_list(gc, domid, usbctrl->devid, &usbs, &numusb);
+    if (rc) goto out;
+    for (i = 0; i < numusb; i++) {
+        if (do_usb_remove(gc, domid, &usbs[i])) {
+            LOG(ERROR, "do_usb_remove failed");
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    }
+    /* remove usbctrl */
+    GCNEW(aodev);
+    libxl__prepare_ao_device(ao, aodev);
+    aodev->action = LIBXL__DEVICE_ACTION_REMOVE;
+    aodev->dev = device;
+    aodev->callback = device_addrm_aocomplete;
+    aodev->force = force;
+    libxl__initiate_device_remove(egc, aodev);
+
+out:
+    free(usbctrls);
+    free(usbs);
+    if(rc) return AO_ABORT(rc);
+    return AO_INPROGRESS;
+}
+
+int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid,
+                                libxl_device_usbctrl *usbctrl,
+                                const libxl_asyncop_how *ao_how)
+{
+    return libxl__device_usbctrl_remove_common(ctx, domid, usbctrl, ao_how, 0);
+}
+
+int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid,
+                                 libxl_device_usbctrl *usbctrl,
+                                 const libxl_asyncop_how *ao_how)
+{
+    return libxl__device_usbctrl_remove_common(ctx, domid, usbctrl, ao_how, 1);
+}
+
+libxl_device_usbctrl *
+libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num)
+{
+    GC_INIT(ctx);
+    libxl_device_usbctrl *usbctrls = NULL;
+    char *fe_path = NULL;
+    char **dir = NULL;
+    unsigned int ndirs = 0;
+
+    *num = 0;
+
+    fe_path = GCSPRINTF("%s/device/vusb",
+                        libxl__xs_get_dompath(gc, domid));
+    dir = libxl__xs_directory(gc, XBT_NULL, fe_path, &ndirs);
+
+    if (dir && ndirs) {
+        usbctrls = malloc(sizeof(*usbctrls) * ndirs);
+        libxl_device_usbctrl* usbctrl;
+        libxl_device_usbctrl* end = usbctrls + ndirs;
+        for(usbctrl = usbctrls; usbctrl < end; usbctrl++, dir++, (*num)++) {
+            char *tmp;
+            const char *be_path = libxl__xs_read(gc, XBT_NULL,
+                                    GCSPRINTF("%s/%s/backend", fe_path, *dir));
+
+            libxl_device_usbctrl_init(usbctrl);
+            usbctrl->devid = atoi(*dir);
+
+            tmp = libxl__xs_read(gc, XBT_NULL,
+                                 GCSPRINTF("%s/%s/backend-id", fe_path, *dir));
+            if (!tmp) goto outerr;
+            usbctrl->backend_domid = atoi(tmp);
+
+            tmp = libxl__xs_read(gc, XBT_NULL,
+                                 GCSPRINTF("%s/usb-ver", be_path));
+            if (!tmp) goto outerr;
+            usbctrl->version = atoi(tmp);
+
+            tmp = libxl__xs_read(gc, XBT_NULL,
+                                 GCSPRINTF("%s/num-ports", be_path));
+            if (!tmp) goto outerr;
+            usbctrl->ports = atoi(tmp);
+       }
+    }
+
+    return usbctrls;
+
+outerr:
+    LOG(ERROR, "Unable to list USB Controllers");
+    for (int i = 0; i < *num; i++) {
+        libxl_device_usbctrl_dispose(usbctrls + i);
+    }
+    free(usbctrls);
+    *num = 0;
+    return NULL;
+}
+
+int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
+                                libxl_device_usbctrl *usbctrl,
+                                libxl_usbctrlinfo *usbctrlinfo)
+{
+    GC_INIT(ctx);
+    char *dompath, *usbctrlpath;
+    char *val;
+    int rc = 0;
+
+    usbctrlinfo->devid = usbctrl->devid;
+    usbctrlinfo->ports = usbctrl->ports;
+    usbctrlinfo->version = usbctrl->version;
+
+    dompath = libxl__xs_get_dompath(gc, domid);
+    usbctrlpath = GCSPRINTF("%s/device/vusb/%d", dompath, usbctrlinfo->devid);
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/backend", usbctrlpath));
+    usbctrlinfo->backend = libxl__strdup(NOGC, val);
+    if (!usbctrlinfo->backend) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/backend-id", usbctrlpath));
+    usbctrlinfo->backend_id = val ? strtoul(val, NULL, 10) : -1;
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/state", usbctrlpath));
+    usbctrlinfo->state = val ? strtoul(val, NULL, 10) : -1;
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/event-channel", usbctrlpath));
+    usbctrlinfo->evtch = val ? strtoul(val, NULL, 10) : -1;
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/urb-ring-ref", usbctrlpath));
+    usbctrlinfo->ref_urb = val ? strtoul(val, NULL, 10) : -1;
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/conn-ring-ref", usbctrlpath));
+    usbctrlinfo->ref_conn= val ? strtoul(val, NULL, 10) : -1;
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/frontend", usbctrlinfo->backend));
+    usbctrlinfo->frontend = libxl__strdup(NOGC, val);
+
+    val = libxl__xs_read(gc, XBT_NULL,
+                         GCSPRINTF("%s/frontend-id", usbctrlinfo->backend));
+    usbctrlinfo->frontend_id = val ? strtoul(val, NULL, 10) : -1;
+
+out:
+    GC_FREE;
+    return rc;
+}
+
+/* usb device functions */
+
+static int
+libxl__device_usb_assigned_list(libxl__gc *gc,
+                                libxl_device_usb **list, int *num)
+{
+    char **domlist;
+    unsigned int nd = 0, i, j;
+    char *be_path;
+    libxl_device_usb *usb;
+
+    *list = NULL;
+    *num = 0;
+
+    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd);
+    be_path = GCSPRINTF("/local/domain/%d/backend/vusb", LIBXL_TOOLSTACK_DOMID);
+    for (i = 0; i < nd; i++) {
+        char *path, *num_ports, **ctrl_list;
+        unsigned int nc = 0;
+        path = GCSPRINTF("%s/%s", be_path, domlist[i]);
+        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path , &nc);
+
+        for (j = 0; j < nc; j++) {
+            path = GCSPRINTF("%s/%s/%s/num-ports", be_path,
+                             domlist[i], ctrl_list[j]);
+            num_ports = libxl__xs_read(gc, XBT_NULL, path);
+            if (num_ports) {
+                int nport = atoi(num_ports), k;
+                char *devpath, *busid;
+
+                for (k = 1; k <= nport; k++) {
+                    devpath = GCSPRINTF("%s/%s/%s/port/%u", be_path,
+                                        domlist[i], ctrl_list[j], k);
+                    busid = libxl__xs_read(gc, XBT_NULL, devpath);
+                    /* If there are USB device attached, add it to list */
+                    if (busid && strcmp(busid, "") ) {
+                        *list = realloc(*list,
+                                  sizeof(libxl_device_usb) * ((*num) + 1));
+                        if (*list == NULL)
+                            return ERROR_NOMEM;
+                        usb = *list + *num;
+                        usb->ctrl = atoi(ctrl_list[j]);
+                        usb->port = k;
+                        usb->busid = strdup(busid);
+                        (*num)++;
+                    }
+                }
+            }
+        }
+    }
+    libxl__ptr_add(gc, *list);
+
+    return 0;
+}
+
+static bool is_usb_in_array(libxl_device_usb *usbs, int num,
+                            libxl_device_usb *usb)
+{
+    int i;
+
+    for (i = 0; i < num; i++) {
+        if (!strcmp(usbs[i].busid, usb->busid) )
+            return true;
+    }
+
+    return false;
+}
+
+/* check if USB device is already assigned to a domain */
+static bool is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb)
+{
+    libxl_device_usb *usbs;
+    int rc, num;
+
+    rc = libxl__device_usb_assigned_list(gc, &usbs, &num);
+    if (rc) {
+        LOG(ERROR, "Fail to get assigned usb list");
+        return true;
+    }
+
+    if (is_usb_in_array(usbs, num, usb))
+        return true;
+
+    return false;
+}
+
+/* check if USB device type is assignable */
+static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
+{
+    libxl_ctx *ctx = libxl__gc_owner(gc);
+    int classcode;
+    char *filename;
+    void *buf;
+
+    assert(usb->busid);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid);
+    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0)
+        return false;
+
+    sscanf(buf, "%x", &classcode);
+    if (classcode == USBHUB_CLASS_CODE)
+        return false;
+
+    return true;
+}
+
+/* get usb devices under certain usb controller */
+static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int usbctrl,
+                                  libxl_device_usb **usbs, int *num)
+{
+    char *be_path, *num_devs;
+    int n, i;
+
+    *usbs = NULL;
+    *num = 0;
+
+    be_path = GCSPRINTF("%s/backend/vusb/%d/%d",
+                        libxl__xs_get_dompath(gc, 0), domid, usbctrl);
+    num_devs = libxl__xs_read(gc, XBT_NULL,
+                              GCSPRINTF("%s/num-ports", be_path));
+    if (!num_devs)
+        return 0;
+
+    n = atoi(num_devs);
+    *usbs = calloc(n, sizeof(libxl_device_usb));
+
+    for (i = 0; i < n; i++) {
+        char *busid;
+        libxl_device_usb *usb = NULL;
+
+        busid = libxl__xs_read(gc, XBT_NULL,
+                               GCSPRINTF("%s/port/%d", be_path, i + 1));
+        if (busid && strcmp(busid, "")) {
+            usb = *usbs + *num;
+            usb->ctrl = usbctrl;
+            usb->port = i + 1;
+            usb->busid = strdup(busid);
+            (*num)++;
+        }
+    }
+
+    return 0;
+}
+
+/* get all usb devices of the domain */
+static libxl_device_usb *
+libxl_device_usb_list_all(libxl__gc *gc, uint32_t domid, int *num)
+{
+    char **usbctrls;
+    unsigned int nd, i, j;
+    char *be_path;
+    int rc;
+    libxl_device_usb *usbs = NULL;
+
+    *num = 0;
+
+    be_path = GCSPRINTF("/local/domain/%d/backend/vusb/%d",
+                        LIBXL_TOOLSTACK_DOMID, domid);
+    usbctrls = libxl__xs_directory(gc, XBT_NULL, be_path, &nd);
+
+    for (i = 0; i < nd; i++) {
+        int nc = 0;
+        libxl_device_usb *tmp = NULL;
+        rc = libxl__device_usb_list(gc, domid, atoi(usbctrls[i]), &tmp, &nc);
+        if (!nc) continue;
+
+        usbs = realloc(usbs, sizeof(libxl_device_usb)*((*num) + nc));
+        for(j = 0; j < nc; j++) {
+            usbs[*num].ctrl = tmp[j].ctrl;
+            usbs[*num].port = tmp[j].port;
+            usbs[*num].busid = strdup(tmp[j].busid);
+            (*num)++;
+        }
+        free(tmp);
+    }
+    return usbs;
+}
+
+libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid,
+                                        int usbctrl, int *num)
+{
+    GC_INIT(ctx);
+    libxl_device_usb *usbs = NULL;
+
+    libxl__device_usb_list(gc, domid, usbctrl, &usbs, num);
+
+    GC_FREE;
+    return usbs;
+}
+
+/* set default value */
+static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
+{
+    libxl_ctx *ctx = libxl__gc_owner(gc);
+    struct dirent *de;
+    DIR *dir;
+    char *busid = NULL;
+
+    if (bus < 1 || addr < 1)
+        return NULL;
+
+    if (!(dir = opendir(SYSFS_USB_DEV)))
+        return NULL;
+
+    while((de = readdir(dir))) {
+        char *filename;
+        void *buf;
+        int busnum = -1;
+        int devnum = -1;
+
+        if (!de->d_name)
+            continue;
+
+        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
+        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+            sscanf(buf, "%x", &devnum);
+
+        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
+        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+            sscanf(buf, "%x", &busnum);
+
+        if (bus == busnum && addr == devnum) {
+            busid = strdup(de->d_name);
+            break;
+        }
+    }
+
+    closedir(dir);
+    return busid;
+}
+
+/* find first unused controller:port and give that to usb device */
+static int
+libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
+                                      libxl_device_usb *usb)
+{
+    libxl_ctx *ctx = CTX;
+    libxl_device_usbctrl *usbctrls;
+    libxl_device_usb *usbs = NULL;
+    int numctrl, numusb, i, j, rc = -1;
+    char *be_path, *tmp;
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
+    if ( !numctrl)
+        goto out;
+
+    for (i = 0; i < numctrl; i++) {
+        rc = libxl__device_usb_list(gc, domid, usbctrls[i].devid,
+                                    &usbs, &numusb);
+        if (rc) continue;
+
+        if (!usbctrls[i].ports || numusb == usbctrls[i].ports)
+            continue;
+
+        for (j = 1; i <= numusb; j++) {
+            be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                                libxl__xs_get_dompath(gc, 0), domid,
+                                usbctrls[i].devid, j);
+            tmp = libxl__xs_read(gc, XBT_NULL, be_path);
+            if (tmp && !strcmp( tmp, "")) {
+                usb->ctrl = usbctrls[i].devid;
+                usb->port = j;
+                break;
+            }
+        }
+    }
+
+    rc = 0;
+
+out:
+    if (usbctrls)
+        free(usbctrls);
+    if (usbs)
+        free(usbs);
+    return rc;
+}
+
+static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
+                                        libxl_device_usb *usb)
+{
+    char *be_path, *tmp;
+
+    if (usb->ctrl == -1) {
+        int ret = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
+        /* If no existing ctrl to host this usb device, setup a new one */
+        if (ret) {
+            libxl_device_usbctrl usbctrl;
+            libxl_device_usbctrl_init(&usbctrl);
+            libxl__device_usbctrl_add(gc, domid, &usbctrl);
+            usb->ctrl = usbctrl.devid;
+            usb->port = 1;
+            libxl_device_usbctrl_dispose(&usbctrl);
+        }
+    }
+
+    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                        libxl__xs_get_dompath(gc, 0),
+                        domid, usb->ctrl, usb->port);
+    tmp = libxl__xs_read(gc, XBT_NULL, be_path);
+    if (!tmp || strcmp(tmp, "") ){
+        LOG(ERROR, "The controller port isn't available");
+        return ERROR_INVAL;
+    }
+
+    return 0;
+}
+
+/* xenstore usb data */
+static int libxl__device_usb_add_xenstore(libxl__gc *gc, uint32_t domid,
+                                          libxl_device_usb *usb)
+{
+    char *be_path;
+    int rc;
+    libxl_domain_config d_config;
+    libxl_device_usb usb_saved;
+    libxl__domain_userdata_lock *lock = NULL;
+
+    libxl_domain_config_init(&d_config);
+    libxl_device_usb_init(&usb_saved);
+    libxl_device_usb_copy(CTX, &usb_saved, usb);
+
+    lock = libxl__lock_domain_userdata(gc, domid);
+    if (!lock) {
+        rc = ERROR_LOCK_FAIL;
+        goto out;
+    }
+
+    rc = libxl__get_domain_configuration(gc, domid, &d_config);
+    if (rc) goto out;
+
+    DEVICE_ADD(usb, usbs, domid, &usb_saved, COMPARE_USB, &d_config);
+
+    rc = libxl__set_domain_configuration(gc, domid, &d_config);
+    if (rc) goto out;
+
+    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                  libxl__xs_get_dompath(gc, 0), domid, usb->ctrl, usb->port);
+    LOG(DEBUG, "Adding new usb device to xenstore");
+    if (libxl__xs_write_checked(gc, XBT_NULL, be_path, usb->busid)) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = 0;
+
+out:
+    if (lock) libxl__unlock_domain_userdata(lock);
+    libxl_device_usb_dispose(&usb_saved);
+    libxl_domain_config_dispose(&d_config);
+    return rc;
+}
+
+static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t domid,
+                                             libxl_device_usb *usb)
+{
+    char *be_path;
+
+    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
+                        libxl__xs_get_dompath(gc, 0),
+                        domid, usb->ctrl, usb->port);
+    LOG(DEBUG, "Removing USB device from xenstore");
+    if (libxl__xs_write_checked(gc,XBT_NULL, be_path, ""))
+        return ERROR_FAIL;
+
+    return 0;
+}
+
+/* bind/unbind usb device interface */
+static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)
+{
+    char *path, *spath, *dp = NULL;
+    int fd = -1;
+    int rc = 0;
+    struct stat st;
+
+    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf);
+    if (!lstat(spath, &st)) {
+        /* Find the canonical path to the driver. */
+        dp = libxl__zalloc(gc, PATH_MAX);
+        dp = realpath(spath, dp);
+
+        path = GCSPRINTF("%s/unbind", spath);
+        fd = open(path, O_WRONLY);
+        if (fd < 0)
+            return ERROR_FAIL;
+        rc = write(fd, intf, strlen(intf));
+        close(fd);
+        if (rc < 0)
+            return ERROR_FAIL;
+    }
+
+    if (drvpath)
+        *drvpath = dp;
+
+    return 0;
+}
+
+static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath)
+{
+    char *path;
+    struct stat st;
+    int fd, rc = 0;
+
+    path = GCSPRINTF("%s/%s", drvpath, intf);
+    rc = lstat(path, &st);
+    /* already bind, return */
+    if(rc == 0)
+        return 0;
+
+    path = GCSPRINTF("%s/bind", drvpath);
+    fd = open(path, O_WRONLY);
+    if (fd < 0)
+        return ERROR_FAIL;
+
+    rc = write(fd, intf, strlen(intf));
+    close(fd);
+    if (rc < 0)
+        return ERROR_FAIL;
+
+    return 0;
+}
+
+/* Is usb interface bound to usbback? */
+static int usb_intf_is_assigned(libxl__gc *gc, char *intf)
+{
+    char *spath;
+    int rc;
+    struct stat st;
+
+    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf);
+    rc = lstat(spath, &st);
+
+    if(rc == 0)
+        return 1;
+    if (rc < 0 && errno == ENOENT)
+        return 0;
+    LOGE(ERROR, "Accessing %s", spath);
+    return -1;
+}
+
+static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb,
+                                  char ***intfs, int *num)
+{
+    DIR *dir;
+    struct dirent *entry;
+    char *buf;
+    int rc = 0;
+
+    *intfs = NULL;
+    *num = 0;
+
+    buf = GCSPRINTF("%s:", usb->busid);
+
+    if (!(dir = opendir(SYSFS_USB_DEV))) {
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    while ((entry = readdir(dir)) != NULL) {
+        if (!strncmp(entry->d_name, buf, strlen(buf))){
+            *intfs = realloc(*intfs, sizeof(char *) * (*num + 1));
+            if (*intfs == NULL) {
+                rc = ERROR_FAIL;
+                goto out;
+            }
+            (*intfs)[*num] = strdup(entry->d_name);
+            (*num)++;
+        }
+    }
+
+    closedir(dir);
+
+out:
+    return rc;
+}
+
+/* Cann't write '.' or ':' into Xenstore as key. So, change '.' to '_',
+ * change ':' to '-'.
+ */
+static char *usb_interface_encode(char *busid)
+{
+    char *str = strdup(busid);
+    int i, len = strlen(str);
+
+    for (i = 0; i < len; i++) {
+        if (str[i] == '.')
+            str[i] = '_';
+         if (str[i] == ':')
+            str[i] = '-';
+    }
+    return str;
+}
+
+/* unbind usb device from usbback driver, if there are many interfaces
+ * under the usb device, then check each interface, unbind from usbback
+ * driver and rebind to original driver
+ */
+static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
+{
+    char **intfs = NULL;
+    char *path;
+    int num = 0, i;
+    int rc = 0;
+    char *usb_encode = NULL;
+
+    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
+        return ERROR_FAIL;
+
+    usb_encode = usb_interface_encode(usb->busid);
+
+    for (i = 0; i < num; i++){
+        char *intf = intfs[i];
+        char *drvpath = NULL;
+
+        if (usb_intf_is_assigned(gc, intf) > 0) {
+            /* unbind interface from usbback driver */
+            if (unbind_usb_intf(gc, intf, NULL) < 0) {
+                rc = ERROR_FAIL;
+                goto out;
+            }
+        }
+
+        /* bind interface to its originial driver */
+        drvpath = libxl__xs_read(gc, XBT_NULL,
+                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
+                  usb_encode, usb_interface_encode(intf)));
+        if (drvpath) {
+            if (bind_usb_intf(gc, intf, drvpath) < 0) {
+                LOGE(ERROR, "Couldn't bind %s to %s", intf, drvpath);
+                rc = ERROR_FAIL;
+                goto out;
+            }
+        }
+    }
+
+    /* finally, remove xs driver path */
+    path = GCSPRINTF(USBBACK_INFO_PATH"/%s", usb_encode);
+    libxl__xs_rm_checked(gc, XBT_NULL, path);
+
+out:
+    if (intfs) {
+        for (i = 0; i < num; i++)
+            free(intfs[i]);
+        free(intfs);
+    }
+    free(usb_encode);
+    return rc;
+}
+
+/* bind usb device to "usbback" driver, if there are many interfaces
+ * under the usb device, check each interface, unbind from original
+ * driver and bind to usbback driver.
+ */
+static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
+{
+    char **intfs = NULL;
+    int num = 0, i;
+    int rc = 0;
+    char *usb_encode = NULL;
+
+    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
+        return ERROR_FAIL;
+
+    usb_encode = usb_interface_encode(usb->busid);
+
+    for (i = 0; i < num; i++){
+        char *intf = intfs[i];
+        char *path = NULL;
+        char *drvpath = NULL;
+
+        /* already assigned to usbback */
+        if (usb_intf_is_assigned(gc, intf) > 0)
+            continue;
+
+        /* unbind interface from original driver */
+        if (unbind_usb_intf(gc, intf, &drvpath) < 0) {
+            rc = ERROR_FAIL;
+            goto out_rebind;
+        }
+
+        if (drvpath) {
+            /* write driver path to xenstore for later rebinding */
+            path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
+                             usb_encode, usb_interface_encode(intf));
+            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
+                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
+            }
+        }
+
+        /* bind interface to usbback */
+        if (bind_usb_intf(gc, intf, SYSFS_USBBACK_DRIVER) < 0){
+            LOGE(ERROR, "Couldn't bind %s to %s", intf, SYSFS_USBBACK_DRIVER);
+            rc = ERROR_FAIL;
+            goto out_rebind;
+        }
+    }
+
+    goto out;
+
+out_rebind:
+    /* some interfaces might be bound to usbback, unbind it then and
+     * rebind to its original driver
+     */
+    usbback_dev_unassign(gc, usb);
+out:
+    if (intfs) {
+        for (i = 0; i < num; i++)
+            free(intfs[i]);
+        free(intfs);
+    }
+    free(usb_encode);
+    return rc;
+}
+
+static int do_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
+{
+    int rc = 0;
+
+    rc = libxl__device_usb_add_xenstore(gc, domid, usb);
+    if (rc) goto out;
+
+    rc = usbback_dev_assign(gc, usb);
+    if (rc)
+        libxl__device_usb_remove_xenstore(gc, domid, usb);
+
+out:
+    return rc;
+}
+
+int libxl__device_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
+{
+    int rc;
+
+    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
+    if (!usb->busid) {
+        LOG(ERROR, "USB device doesn't exist in sysfs");
+        return ERROR_INVAL;
+    }
+
+    if (!is_usb_assignable(gc, usb)) {
+        LOG(ERROR, "USB device is not assignable.");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /* check usb device is already assigned by pvusb */
+    if (is_usb_assigned(gc, usb)) {
+        LOG(ERROR, "USB device is already attached to a domain.");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = libxl__device_usb_setdefault(gc, domid, usb);
+    if (rc) goto out;
+
+    rc = do_usb_add(gc, domid, usb);
+
+out:
+    return rc;
+}
+
+int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid,
+                         libxl_device_usb *usb,
+                         const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    int rc;
+
+    rc = libxl__device_usb_add(gc, domid, usb);
+    libxl__ao_complete(egc, ao, rc);
+    return AO_INPROGRESS;
+}
+
+static int do_usb_remove(libxl__gc *gc, uint32_t domid,
+                         libxl_device_usb *usb)
+{
+    if (libxl__device_usb_remove_xenstore(gc, domid, usb))
+        return -1;
+
+    usbback_dev_unassign(gc, usb);
+
+    return 0;
+}
+
+static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
+                                    libxl_device_usb *usb)
+{
+    libxl_device_usb *usbs = NULL;
+    libxl_device_usb *usb_find = NULL;
+    int i, num, rc;
+
+    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
+    if (!usb->busid) {
+        LOG(ERROR, "USB device doesn't exist in sysfs");
+        return ERROR_INVAL;
+    }
+
+    usbs = libxl_device_usb_list_all(gc, domid, &num);
+    if (!usbs) {
+       LOG(ERROR, "No USB device attached to this domain");
+       return ERROR_FAIL;
+    }
+
+    for (i = 0; i < num; i++) {
+        if (!strcmp(usb->busid, usbs[i].busid)) {
+            usb_find = usbs + i;
+            break;
+        }
+    }
+
+    /* doesn't find the usb device in domain's usb device list*/
+    if (!usb_find) {
+        LOG(ERROR, "USB device %x.%x is not attached to this domain",
+            usb->hostbus, usb->hostaddr);
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    rc = do_usb_remove(gc, domid, usb_find);
+
+out:
+    free(usbs);
+    return rc;
+}
+
+int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
+                            libxl_device_usb *usb,
+                            const libxl_asyncop_how *ao_how)
+
+{
+    AO_CREATE(ctx, domid, ao_how);
+    int rc;
+
+    rc = libxl__device_usb_remove(gc, domid, usb);
+
+    libxl__ao_complete(egc, ao, rc);
+    return AO_INPROGRESS;
+}
+
+int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid)
+{
+    libxl_ctx *ctx = CTX;
+    libxl_device_usbctrl *usbctrls = NULL;
+    int num, i, rc = 0;
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &num);
+    if (!usbctrls)
+        return 0;
+
+    for (i = 0; i < num; i++) {
+        /* Force remove on shutdown since, on HVM, qemu will not always
+         * respond to SCI interrupt because the guest kernel has shut
+         * down the devices by the time we even get here!
+         */
+        if (libxl__device_usbctrl_remove_common(ctx, domid,
+                                                usbctrls + i, 0, 1) < 0) {
+            rc = ERROR_FAIL;
+            goto out;
+        }
+    }
+
+out:
+    free(usbctrls);
+    return rc;
+}
+
+int libxl_device_usb_getinfo(libxl_ctx *ctx, char *busid, libxl_usbinfo *usbinfo)
+{
+    GC_INIT(ctx);
+    char *filename;
+    void *buf;
+
+    assert(busid);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->devnum);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->busnum);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/idVendor", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->idVendor);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/idProduct", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->idProduct);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/manufacturer", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) {
+        /* replace \n to \0 */
+        usbinfo->manuf = strdup(buf);
+        if (strlen(usbinfo->manuf) > 0)
+            usbinfo->manuf[strlen(usbinfo->manuf) - 1] = '\0';
+    }
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/product", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) {
+       /* replace \n to \0 */
+       usbinfo->prod = strdup(buf);
+       if (strlen(usbinfo->manuf) > 0)
+           usbinfo->manuf[strlen(usbinfo->manuf) - 1] = '\0';
+    }
+
+    GC_FREE;
+    return 0;
+}
+/*
+ * Local variables:
+ * mode: C
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 0866433..a6db614 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -533,6 +533,22 @@ libxl_device_pci = Struct("device_pci", [
     ("seize", bool),
     ])
 
+libxl_device_usbctrl = Struct("device_usbctrl", [
+    ("devid", libxl_devid),
+    ("version", integer),
+    ("ports", integer),
+    ("backend_domid", libxl_domid),
+    ("backend_domname", string),
+   ])
+
+libxl_device_usb = Struct("device_usb", [
+    ("ctrl", libxl_devid),
+    ("port", integer),
+    ("busid", string),
+    ("hostbus",   integer),
+    ("hostaddr",  integer),
+    ])
+
 libxl_device_vtpm = Struct("device_vtpm", [
     ("backend_domid",    libxl_domid),
     ("backend_domname",  string),
@@ -559,6 +575,8 @@ libxl_domain_config = Struct("domain_config", [
     ("disks", Array(libxl_device_disk, "num_disks")),
     ("nics", Array(libxl_device_nic, "num_nics")),
     ("pcidevs", Array(libxl_device_pci, "num_pcidevs")),
+    ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")),
+    ("usbs", Array(libxl_device_usb, "num_usbs")),
     ("vfbs", Array(libxl_device_vfb, "num_vfbs")),
     ("vkbs", Array(libxl_device_vkb, "num_vkbs")),
     ("vtpms", Array(libxl_device_vtpm, "num_vtpms")),
@@ -607,6 +625,29 @@ libxl_vtpminfo = Struct("vtpminfo", [
     ("uuid", libxl_uuid),
     ], dir=DIR_OUT)
 
+libxl_usbctrlinfo = Struct("usbctrlinfo", [
+    ("devid", libxl_devid),
+    ("version", integer),
+    ("ports", integer),
+    ("backend", string),
+    ("backend_id", uint32),
+    ("frontend", string),
+    ("frontend_id", uint32),
+    ("state", integer),
+    ("evtch", integer),
+    ("ref_urb", integer),
+    ("ref_conn", integer),
+    ], dir=DIR_OUT)
+
+libxl_usbinfo = Struct("usbinfo", [
+    ("busnum", integer),
+    ("devnum", integer),
+    ("idVendor", integer),
+    ("idProduct", integer),
+    ("prod", string),
+    ("manuf", string),
+    ], dir=DIR_OUT)
+
 libxl_vcpuinfo = Struct("vcpuinfo", [
     ("vcpuid", uint32),
     ("cpu", uint32),
diff --git a/tools/libxl/libxl_types_internal.idl b/tools/libxl/libxl_types_internal.idl
index 5e55685..696f5f8 100644
--- a/tools/libxl/libxl_types_internal.idl
+++ b/tools/libxl/libxl_types_internal.idl
@@ -22,6 +22,7 @@ libxl__device_kind = Enumeration("device_kind", [
     (6, "VKBD"),
     (7, "CONSOLE"),
     (8, "VTPM"),
+    (9, "VUSB"),
     ])
 
 libxl__console_backend = Enumeration("console_backend", [
-- 
1.8.5.2

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V3 4/6] xl: add pvusb commands
  2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
                   ` (2 preceding siblings ...)
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
@ 2015-04-19  3:50 ` Chunyan Liu
  2015-04-20  8:12   ` Juergen Gross
                     ` (2 more replies)
  2015-04-19  3:50 ` [PATCH V3 5/6] domcreate: support pvusb in configuration file Chunyan Liu
  2015-04-19  3:50 ` [PATCH V3 6/6] refactor codes to unify pvusb and qemu emulated usb Chunyan Liu
  5 siblings, 3 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
usb-attach and usb-detach.

To attach a usb device to guest through pvusb, one could follow
following example:

 #xl usb-ctrl-attach test_vm version=1 num_ports=8

 #xl usb-list test_vm
 will show the usb controllers and port usage under the domain.

 #xl usb-attach test_vm 1.6
 will find the first usable controller:port, and attach usb
 device whose bus address is 1.6 (busnum is 1, devnum is 6)
 to it. One could also specify which <controller> and which <port>.

 #xl usb-detach test_vm 1.6

 #xl usb-ctrl-detach test_vm dev_id
 will destroy the controller with specified dev_id. Dev_id
 can be traced in usb-list info.

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
---
Changes to v2:
  * use bus.addr as user interface instead of busid in usb-attach|detach
  * remove usb-assignable-list interface
  * add documentation

 docs/man/xl.pod.1         |  38 ++++++++
 tools/libxl/xl.h          |   5 +
 tools/libxl/xl_cmdimpl.c  | 238 ++++++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/xl_cmdtable.c |  25 +++++
 4 files changed, 306 insertions(+)

diff --git a/docs/man/xl.pod.1 b/docs/man/xl.pod.1
index 16783c8..d10fbde 100644
--- a/docs/man/xl.pod.1
+++ b/docs/man/xl.pod.1
@@ -1380,6 +1380,44 @@ List pass-through pci devices for a domain.
 
 =back
 
+=head1 USB PASS-THROUGH
+
+=over 4
+
+=item B<usb-ctrl-attach> I<domain-id> [I<version=val>] [I<ports=number>]
+
+Create a new USB controller for the specified domain.
+B<version=val> is the usb controller version, could be 1 (USB1.1) or 2 (USB2.0).
+B<ports=number> is the total ports of the usb controller.
+By default, it will create a USB2.0 controller with 8 ports.
+
+=item B<usb-ctrl-detach> I<domain-id> I<devid>
+
+Destroy a USB controller from the specified domain.
+B<devid> is devid of the USB controller.
+
+=item B<usb-attach> I<domain-id> I<bus.addr> [I<controller=devid> [I<port=number>]]
+
+Hot-plug a new pass-through USB device to the specified domain.
+B<bus.addr> is the busnum.devnum of the physical USB device to pass-through.
+B<controller=devid> B<port=number> is the USB controller:port to hotplug the
+USB device to. By default, it will find the first available controller:port
+and use it; if there is no controller, it will create one.
+
+=item B<usb-detach> I<domain-id> I<bus.addr>
+
+Hot-unplug a previously assigned USB device from a domain. B<bus.addr> is
+busnum.devnum of the physical USB device to be removed from the guest domain.
+
+If B<-f> is specified, B<xl> is going to forcefully remove the device even
+without guest's collaboration.
+
+=item B<usb-list> I<domain-id>
+
+List pass-through usb devices for a domain.
+
+=back
+
 =head1 TMEM
 
 =over 4
diff --git a/tools/libxl/xl.h b/tools/libxl/xl.h
index 5bc138c..2d57ee3 100644
--- a/tools/libxl/xl.h
+++ b/tools/libxl/xl.h
@@ -86,6 +86,11 @@ int main_blockdetach(int argc, char **argv);
 int main_vtpmattach(int argc, char **argv);
 int main_vtpmlist(int argc, char **argv);
 int main_vtpmdetach(int argc, char **argv);
+int main_usbctrl_attach(int argc, char **argv);
+int main_usbctrl_detach(int argc, char **argv);
+int main_usbattach(int argc, char **argv);
+int main_usbdetach(int argc, char **argv);
+int main_usblist(int argc, char **argv);
 int main_uptime(int argc, char **argv);
 int main_claims(int argc, char **argv);
 int main_tmem_list(int argc, char **argv);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 394b55d..4654e1d 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -3196,6 +3196,244 @@ int main_cd_insert(int argc, char **argv)
     return 0;
 }
 
+static void usbinfo_print(libxl_device_usb *usbs, int num) {
+    int i;
+    if ( usbs == NULL )
+         return;
+    for (i = 0; i < num; i++) {
+        libxl_usbinfo usbinfo;
+        libxl_usbinfo_init(&usbinfo);
+
+        if (usbs[i].port )
+            printf("  Port %d:", usbs[i].port);
+        if (!libxl_device_usb_getinfo(ctx, usbs[i].busid, &usbinfo)) {
+            printf(" Bus %03x Device %03x: ID %04x:%04x %s %s\n",
+                    usbinfo.busnum, usbinfo.devnum, usbinfo.idVendor,
+                    usbinfo.idProduct, usbinfo.manuf, usbinfo.prod);
+        }
+        libxl_usbinfo_dispose(&usbinfo);
+    }
+}
+
+int main_usbctrl_attach(int argc, char **argv)
+{
+    uint32_t domid;
+    int opt;
+    char *oparg;
+    libxl_device_usbctrl usbctrl;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-ctrl-attach", 1) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind++]);
+
+    libxl_device_usbctrl_init(&usbctrl);
+
+    while (argc > optind) {
+        if (MATCH_OPTION("version", argv[optind], oparg)) {
+            usbctrl.version = atoi(oparg);
+        } else if (MATCH_OPTION("ports", argv[optind], oparg)) {
+            usbctrl.ports = atoi(oparg);
+        } else {
+            fprintf(stderr, "unrecognized argument `%s'\n", argv[optind]);
+            exit(-1);
+        }
+        optind++;
+    }
+
+    if (dryrun_only) {
+       char* json = libxl_device_usbctrl_to_json(ctx, &usbctrl);
+       printf("usb controller: %s\n", json);
+       free(json);
+       libxl_device_usbctrl_dispose(&usbctrl);
+       if (ferror(stdout) || fflush(stdout)) {
+           perror("stdout");
+           exit(-1);
+       }
+       return 0;
+    }
+
+    if (libxl_device_usbctrl_add(ctx, domid, &usbctrl, 0)) {
+        fprintf(stderr, "libxl_device_usbctrl_add failed.\n");
+        exit(-1);
+    }
+    libxl_device_usbctrl_dispose(&usbctrl);
+    return 0;
+}
+
+int main_usbctrl_detach(int argc, char **argv)
+{
+    uint32_t domid;
+    int opt;
+    libxl_device_usbctrl usbctrl;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-ctrl-detach", 2) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind]);
+
+    libxl_device_usbctrl_init(&usbctrl);
+    usbctrl.devid = atoi(argv[optind+1]);
+
+    if(libxl_device_usbctrl_remove(ctx, domid, &usbctrl, 0)) {
+        fprintf(stderr, "libxl_device_usbctrl_add failed.\n");
+        exit(-1);
+    }
+    libxl_device_usbctrl_dispose(&usbctrl);
+    return 0;
+
+}
+
+int main_usbattach(int argc, char **argv)
+{
+    uint32_t domid;
+    char *devname, *p;
+    int opt;
+    char *oparg;
+    libxl_device_usb usb;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-attach", 2) {
+        /* No options */
+    }
+
+    libxl_device_usb_init(&usb);
+
+    domid = find_domain(argv[optind++]);
+    devname = argv[optind++];
+    p = strchr(devname, '.');
+    if (p) {
+        usb.hostbus = strtoul(devname, NULL, 0);
+        usb.hostaddr = strtoul(p + 1, NULL, 0);
+    }
+
+    if (usb.hostbus < 1 || usb.hostaddr < 1) {
+        fprintf(stderr, "Invalid usb device.\n");
+        exit(-1);
+    }
+
+    while (argc > optind) {
+        if (MATCH_OPTION("controller", argv[optind], oparg)) {
+            usb.ctrl = atoi(oparg);
+        } else if (MATCH_OPTION("port", argv[optind], oparg)) {
+            usb.port = atoi(oparg);
+        } else {
+            fprintf(stderr, "unrecognized argument `%s'\n", argv[optind]);
+            exit(-1);
+        }
+        optind++;
+    }
+
+    if (dryrun_only) {
+        char *json = libxl_device_usb_to_json(ctx, &usb);
+        printf("usb: %s\n", json);
+        free(json);
+        libxl_device_usb_dispose(&usb);
+        if (ferror(stdout) || fflush(stdout)) {
+            perror("stdout");
+            exit(-1);
+        }
+        return 0;
+    }
+
+    if (libxl_device_usb_add(ctx, domid, &usb, 0)) {
+        fprintf(stderr, "libxl_device_usb_add failed.\n");
+        exit(-1);
+    }
+
+    libxl_device_usb_dispose(&usb);
+    return 0;
+}
+
+int main_usbdetach(int argc, char **argv)
+{
+    uint32_t domid;
+    char *devname, *p;
+    int opt;
+    libxl_device_usb usb;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-detach", 2) {
+        /* No options */
+    }
+
+    libxl_device_usb_init(&usb);
+
+    domid = find_domain(argv[optind++]);
+    devname = argv[optind++];
+
+    if (argc > optind) {
+        fprintf(stderr, "Invalid arguments.\n");
+        exit(-1);
+    }
+
+    p = strchr(devname, '.');
+    if (p) {
+        usb.hostbus = strtoul(devname, NULL, 0);
+        usb.hostaddr = strtoul(p + 1, NULL, 0);
+    }
+
+    if (usb.hostbus < 1 || usb.hostaddr < 1) {
+        fprintf(stderr, "Invalid usb device.\n");
+        exit(-1);
+    }
+
+    if (libxl_device_usb_remove(ctx, domid, &usb, 0) ) {
+        fprintf(stderr, "libxl_device_usb_remove failed.\n");
+        exit(-1);
+    }
+    libxl_device_usb_dispose(&usb);
+    return 0;
+}
+
+int main_usblist(int argc, char **argv)
+{
+    uint32_t domid;
+    libxl_device_usbctrl *usbctrls;
+    libxl_device_usb *usbs;
+    libxl_usbctrlinfo usbctrlinfo;
+    int numctrl, numusb, i, opt;
+
+    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-list", 1) {
+        /* No options */
+    }
+
+    domid = find_domain(argv[optind++]);
+
+    if (argc > optind) {
+        fprintf(stderr, "Invalid arguments.\n");
+        exit(-1);
+    }
+
+    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
+    if (!usbctrls) {
+        return 0;
+    }
+
+    for (i = 0; i < numctrl; ++i) {
+        printf("%-6s %-3s %-5s %-7s %-5s %-30s\n",
+                "Devid", "BE", "state", "usb-ver", "ports", "BE-path");
+
+        if (!libxl_device_usbctrl_getinfo(ctx, domid,
+                                &usbctrls[i], &usbctrlinfo)) {
+            printf("%-6d %-3d %-5d %-7d %-5d %-30s\n",
+                    usbctrlinfo.devid,
+                    usbctrlinfo.backend_id, usbctrlinfo.state,
+                    usbctrlinfo.version, usbctrlinfo.ports,
+                    usbctrlinfo.backend);
+
+            usbs = libxl_device_usb_list(ctx, domid, usbctrlinfo.devid, &numusb);
+            usbinfo_print(usbs, numusb);
+
+            libxl_usbctrlinfo_dispose(&usbctrlinfo);
+        }
+        libxl_device_usbctrl_dispose(&usbctrls[i]);
+    }
+
+    free(usbctrls);
+    return 0;
+}
+
 int main_console(int argc, char **argv)
 {
     uint32_t domid;
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index 9284887..da30ae0 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -544,6 +544,31 @@ struct cmd_spec cmd_table[] = {
       "\"local_mem_bandwidth\":     Show local memory bandwidth(KB/s)\n",
     },
 #endif
+    { "usb-ctrl-attach",
+      &main_usbctrl_attach, 1, 1,
+      "Create a virtual USB controller for a domain",
+      "<Domain> [version=<version>] [ports=<number>]",
+    },
+    { "usb-ctrl-detach",
+      &main_usbctrl_detach, 0, 1,
+      "Remove the virtual USB controller specified by <DevId> for a domain",
+      "<Domain> <DevId>",
+    },
+    { "usb-attach",
+      &main_usbattach, 1, 2,
+      "Attach a USB device to a domain",
+      "<Domain> <bus.addr> [controller=<DevId> [port=<port>]]",
+    },
+    { "usb-detach",
+      &main_usbdetach, 0, 1,
+      "Detach a USB device from a domain",
+      "<Domain> <bus.addr>",
+    },
+    { "usb-list",
+      &main_usblist, 0, 0,
+      "List information about USB devices for a domain",
+      "<Domain>",
+    },
 };
 
 int cmdtable_len = sizeof(cmd_table)/sizeof(struct cmd_spec);
-- 
1.8.5.2

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V3 5/6] domcreate: support pvusb in configuration file
  2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
                   ` (3 preceding siblings ...)
  2015-04-19  3:50 ` [PATCH V3 4/6] xl: add pvusb commands Chunyan Liu
@ 2015-04-19  3:50 ` Chunyan Liu
  2015-04-19  3:50 ` [PATCH V3 6/6] refactor codes to unify pvusb and qemu emulated usb Chunyan Liu
  5 siblings, 0 replies; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

Add code to support pvusb in domain config file. One could specify
usbctrl and usb in domain's configuration file and create domain,
then usb controllers will be created and usb device would be attached
to guest automatically.

One could specify usb controllers and usb devices in config file
like this:
usbctrl=['version=2,ports=4', 'version=1, ports=4', ]
usb=['2.1,controller=0,port=1', ]

Signed-off-by: Chunyan Liu <cyliu@suse.com>
Signed-off-by: Simon Cao <caobosimon@gmail.com>
---
Changes to v2:
  *add 'usbctrls' to domain_config
  *add 'usbctrl' to xl.cfg to allow user specify USB controllers in config file
  *add documentation

 docs/man/xl.cfg.pod.5      | 70 ++++++++++++++++++++++++++++++++++++
 tools/libxl/libxl_create.c | 73 ++++++++++++++++++++++++++++++++++++--
 tools/libxl/xl_cmdimpl.c   | 88 +++++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 227 insertions(+), 4 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index f936dfc..c53e2d1 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -645,6 +645,76 @@ assigned slave device.
 
 =back
 
+=item B<usbctrl=[ "USBCTRL_SPEC_STRING", "USBCTRL_SPEC_STRING", ... ]>
+
+Specifies the USB controllers created for this guest. Each
+B<USB_SPEC_STRING> has the form C<KEY=VALUE,KEY=VALUE,...> where:
+
+=over 4
+
+=item B<KEY=VALUE>
+
+Possible B<KEY>s are:
+
+=over 4
+
+=item B<version=VERSION>
+
+Specifies version of the USB controller, could be 1 (USB1.1) or 2 (USB2.0).
+Default is 2 (USB2.0).
+
+=item B<ports=PORTS>
+
+Specifies port number of the USB controller. Default is 8.
+
+Each USB controller will have an index starting from 0. On the same
+controller, each port will have an index starting from 1.
+
+E.g.
+usbctrl=["version=1,ports=4", "version=2,port2=8",]
+The first controller has:
+controller index = 0, and port 1,2,3,4.
+The second controller has:
+controller index = 1, and port 1,2,3,4,5,6,7,8.
+
+=back
+
+=back
+
+=item B<usb=[ "USB_SPEC_STRING", "USB_SPEC_STRING", ... ]>
+
+Specifies the host USB devices to passthrough to this guest. Each
+B<USB_SPEC_STRING> has the form C<bus.addr,KEY=VALUE,KEY=VALUE,...> where:
+
+=over 4
+
+=item B<bus.addr>
+
+Identifies the busnum.devnum of the USB device from the host perspective.
+This is the same scheme as used in the output of C<lsusb> for the device in
+question.
+
+=item B<KEY=VALUE>
+
+Possible B<KEY>s are:
+
+=over 4
+
+=item B<controller=CONTROLLER>
+
+Specifies USB controller index, to which controller the USB device is attached.
+
+=item B<port=PORT>
+
+Specifies USB port index, to which port the USB device is attached. B<port=PORT>
+is valid only when B<controller=CONTROLLER> is specified. Without
+B<controller=CONTROLLER>, it will find the first available USB controller:port
+and use it. If there is no controller at all, it will create one.
+
+=back
+
+=back
+
 =item B<pci=[ "PCI_SPEC_STRING", "PCI_SPEC_STRING", ... ]>
 
 Specifies the host PCI devices to passthrough to this guest. Each B<PCI_SPEC_STRING>
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index e5a343f..d17cdf4 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -749,6 +749,10 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *aodevs,
 
 static void domcreate_attach_vtpms(libxl__egc *egc, libxl__multidev *multidev,
                                    int ret);
+static void domcreate_attach_usbctrls(libxl__egc *egc, libxl__multidev *multidev,
+                                      int ret);
+static void domcreate_attach_usbs(libxl__egc *egc, libxl__multidev *multidev,
+                                   int ret);
 static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *aodevs,
                                  int ret);
 
@@ -1396,13 +1400,13 @@ static void domcreate_attach_vtpms(libxl__egc *egc,
    if (d_config->num_vtpms > 0) {
        /* Attach vtpms */
        libxl__multidev_begin(ao, &dcs->multidev);
-       dcs->multidev.callback = domcreate_attach_pci;
+       dcs->multidev.callback = domcreate_attach_usbctrls;
        libxl__add_vtpms(egc, ao, domid, d_config, &dcs->multidev);
        libxl__multidev_prepared(egc, &dcs->multidev, 0);
        return;
    }
 
-   domcreate_attach_pci(egc, multidev, 0);
+   domcreate_attach_usbctrls(egc, multidev, 0);
    return;
 
 error_out:
@@ -1410,6 +1414,69 @@ error_out:
    domcreate_complete(egc, dcs, ret);
 }
 
+static void domcreate_attach_usbctrls(libxl__egc *egc, libxl__multidev *multidev,
+                                      int ret)
+{
+    libxl__domain_create_state *dcs = CONTAINER_OF(multidev, *dcs, multidev);
+    STATE_AO_GC(dcs->ao);
+    int i;
+    int domid = dcs->guest_domid;
+
+    libxl_domain_config *const d_config = dcs->guest_config;
+
+    if (ret) {
+        LOG(ERROR, "unable to add vtpm devices");
+        goto error_out;
+    }
+
+    for (i = 0; i < d_config->num_usbctrls; i++) {
+        ret = libxl__device_usbctrl_add(gc, domid, &d_config->usbctrls[i]);
+        if (ret < 0) {
+            LOG(ERROR, "libxl__device_usbctrl_add failed: %d", ret);
+            goto error_out;
+        }
+    }
+
+    domcreate_attach_usbs(egc, multidev, 0);
+    return;
+
+error_out:
+   assert(ret);
+   domcreate_complete(egc, dcs, ret);
+}
+
+
+static void domcreate_attach_usbs(libxl__egc *egc, libxl__multidev *multidev,
+                                int ret)
+{
+    libxl__domain_create_state *dcs = CONTAINER_OF(multidev, *dcs, multidev);
+    STATE_AO_GC(dcs->ao);
+    int i;
+    int domid = dcs->guest_domid;
+
+    libxl_domain_config *const d_config = dcs->guest_config;
+
+    if (ret) {
+        LOG(ERROR, "unable to add vtpm devices");
+        goto error_out;
+    }
+
+    for (i = 0; i < d_config->num_usbs; i++) {
+        ret = libxl__device_usb_add(gc, domid, &d_config->usbs[i]);
+        if (ret < 0) {
+            LOG(ERROR, "libxl__device_usb_add failed: %d", ret);
+            goto error_out;
+        }
+    }
+
+    domcreate_attach_pci(egc, multidev, 0);
+    return;
+
+error_out:
+   assert(ret);
+   domcreate_complete(egc, dcs, ret);
+}
+
 static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
                                  int ret)
 {
@@ -1423,7 +1490,7 @@ static void domcreate_attach_pci(libxl__egc *egc, libxl__multidev *multidev,
     libxl_domain_config *const d_config = dcs->guest_config;
 
     if (ret) {
-        LOG(ERROR, "unable to add vtpm devices");
+        LOG(ERROR, "unable to add usb devices");
         goto error_out;
     }
 
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 4654e1d..750377f 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -1168,7 +1168,8 @@ static void parse_config_data(const char *config_source,
     const char *buf;
     long l, vcpus = 0;
     XLU_Config *config;
-    XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms;
+    XLU_ConfigList *cpus, *vbds, *nics, *pcis, *usbctrls, *usbs,
+                   *cvfbs, *cpuids, *vtpms;
     XLU_ConfigList *channels, *ioports, *irqs, *iomem, *viridian;
     int num_ioports, num_irqs, num_iomem, num_cpus, num_viridian;
     int pci_power_mgmt = 0;
@@ -1940,6 +1941,91 @@ skip_vfb:
             libxl_defbool_set(&b_info->u.pv.e820_host, true);
     }
 
+    if (!xlu_cfg_get_list (config, "usbctrl", &usbctrls, 0, 0) ) {
+        d_config->num_usbctrls = 0;
+        d_config->usbctrls = NULL;
+        while ((buf = xlu_cfg_get_listitem (usbctrls, d_config->num_usbctrls))
+               != NULL) {
+            libxl_device_usbctrl *usbctrl;
+            char *buf2 = strdup(buf);
+            char *p, *p2;
+
+            d_config->usbctrls =
+                (libxl_device_usbctrl *) realloc(d_config->usbctrls,
+                sizeof (libxl_device_usbctrl) * (d_config->num_usbctrls + 1));
+            usbctrl = d_config->usbctrls + d_config->num_usbctrls;
+            libxl_device_usbctrl_init(usbctrl);
+
+            p = strtok(buf2, ",");
+            if (!p)
+                goto skip_usbctrl;
+            do {
+                while (*p == ' ')
+                    p++;
+                if ((p2 = strchr(p, '=')) == NULL)
+                    break;
+                *p2 = '\0';
+                if (!strcmp(p, "version")){
+                    usbctrl->version = atoi(p2 + 1);
+                } else if (!strcmp(p, "ports")){
+                    usbctrl->ports = atoi(p2 + 1);
+                } else {
+                    fprintf(stderr, "Unknown string `%s' in usb spec\n", p);
+                    exit(1);
+                }
+            } while ((p = strtok(NULL, ",")) != NULL);
+
+skip_usbctrl:
+            free(buf2);
+            d_config->num_usbctrls++;
+        }
+    }
+
+    if (!xlu_cfg_get_list (config, "usb", &usbs, 0, 0) ) {
+        d_config->num_usbs = 0;
+        d_config->usbs = NULL;
+        while ((buf = xlu_cfg_get_listitem (usbs, d_config->num_usbs)) != NULL) {
+            libxl_device_usb *usb;
+            char *buf2 = strdup(buf);
+            char *p, *p2;
+
+            d_config->usbs = (libxl_device_usb *) realloc(d_config->usbs,
+                                sizeof (libxl_device_usb) * (d_config->num_usbs+1));
+            usb = d_config->usbs + d_config->num_usbs;
+            libxl_device_usb_init(usb);
+
+            p = strtok(buf2, ",");
+            if (!p)
+                goto skip_usb;
+            do {
+                while(*p == ' ')
+                    ++p;
+                if ((p2 = strchr(p, '=')) == NULL) {
+                    char *busaddr = p;
+                    p = strchr(busaddr, '.');
+                    if (p) {
+                        usb->hostbus = strtoul(busaddr, NULL, 0);
+                        usb->hostaddr = strtoul(p + 1, NULL, 0);
+                    }
+                    continue;
+                }
+                *p2 = '\0';
+                if (!strcmp(p, "controller")) {
+                    usb->ctrl = atoi(p2 + 1);
+                } else if (!strcmp(p, "port")) {
+                    usb->port = atoi(p2 + 1);
+                } else {
+                    fprintf(stderr, "Unknown string `%s' in usb spec\n", p);
+                    exit(1);
+                }
+            } while ((p = strtok(NULL, ",")) != NULL);
+
+skip_usb:
+            free(buf2);
+            d_config->num_usbs++;
+        }
+    }
+
     switch (xlu_cfg_get_list(config, "cpuid", &cpuids, 0, 1)) {
     case 0:
         {
-- 
1.8.5.2

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* [PATCH V3 6/6] refactor codes to unify pvusb and qemu emulated usb
  2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
                   ` (4 preceding siblings ...)
  2015-04-19  3:50 ` [PATCH V3 5/6] domcreate: support pvusb in configuration file Chunyan Liu
@ 2015-04-19  3:50 ` Chunyan Liu
  2015-05-21 14:17   ` George Dunlap
  5 siblings, 1 reply; 53+ messages in thread
From: Chunyan Liu @ 2015-04-19  3:50 UTC (permalink / raw)
  To: xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	ian.jackson, caobosimon

Now we have pvusb implementation. To merge with future qemu emulated
usb work, refactor codes:
 - define 'protocol' (type) to indicate pvusb or qemu, add 'protocol' to
usb controller and usb device structures, add 'type' to xl interface
usb-attach|detach.
 - extract common codes for both qemu and pvusb from libxl_pvusb.c to
libxl_usb.c, and adjust libxl_pvusb.c codes accordingly.

Signed-off-by: Chunyan Liu <cyliu@suse.com>
---
 tools/libxl/Makefile         |   2 +-
 tools/libxl/libxl_internal.h |   4 +
 tools/libxl/libxl_pvusb.c    | 169 +-------------------------------
 tools/libxl/libxl_types.idl  |  11 +++
 tools/libxl/libxl_usb.c      | 224 +++++++++++++++++++++++++++++++++++++++++++
 tools/libxl/xl_cmdimpl.c     |  54 +++++++++--
 tools/libxl/xl_cmdtable.c    |   4 +-
 7 files changed, 293 insertions(+), 175 deletions(-)
 create mode 100644 tools/libxl/libxl_usb.c

diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
index d52281f..f786fcf 100644
--- a/tools/libxl/Makefile
+++ b/tools/libxl/Makefile
@@ -95,7 +95,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
 			libxl_internal.o libxl_utils.o libxl_uuid.o \
 			libxl_json.o libxl_aoutils.o libxl_numa.o libxl_vnuma.o \
 			libxl_save_callout.o _libxl_save_msgs_callout.o \
-			libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o $(LIBXL_OBJS-y)
+			libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o libxl_usb.o $(LIBXL_OBJS-y)
 LIBXL_OBJS += libxl_genid.o
 LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
 
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index f426ed8..2ee058b 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -2428,6 +2428,10 @@ _hidden int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
 _hidden int libxl__device_usb_add(libxl__gc *gc, uint32_t domid,
                             libxl_device_usb *usb);
 _hidden int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid);
+_hidden bool pv_is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb);
+_hidden int pv_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb);
+_hidden int pv_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb);
+_hidden int pv_usb_destroy_all(libxl__gc *gc, uint32_t domid);
 
 /* Internal function to connect a vkb device */
 _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid,
diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
index 4e4975a..3dfd2bd 100644
--- a/tools/libxl/libxl_pvusb.c
+++ b/tools/libxl/libxl_pvusb.c
@@ -16,8 +16,6 @@
 
 #define USBBACK_INFO_PATH "/libxl/usbback"
 
-#define USBHUB_CLASS_CODE 0x09
-
 static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
                                             libxl_device_usbctrl *usbctrl)
 {
@@ -338,6 +336,7 @@ int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
     usbctrlinfo->devid = usbctrl->devid;
     usbctrlinfo->ports = usbctrl->ports;
     usbctrlinfo->version = usbctrl->version;
+    usbctrlinfo->protocol = usbctrl->protocol;
 
     dompath = libxl__xs_get_dompath(gc, domid);
     usbctrlpath = GCSPRINTF("%s/device/vusb/%d", dompath, usbctrlinfo->devid);
@@ -451,7 +450,7 @@ static bool is_usb_in_array(libxl_device_usb *usbs, int num,
 }
 
 /* check if USB device is already assigned to a domain */
-static bool is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb)
+bool pv_is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb)
 {
     libxl_device_usb *usbs;
     int rc, num;
@@ -468,27 +467,6 @@ static bool is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb)
     return false;
 }
 
-/* check if USB device type is assignable */
-static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
-{
-    libxl_ctx *ctx = libxl__gc_owner(gc);
-    int classcode;
-    char *filename;
-    void *buf;
-
-    assert(usb->busid);
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid);
-    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0)
-        return false;
-
-    sscanf(buf, "%x", &classcode);
-    if (classcode == USBHUB_CLASS_CODE)
-        return false;
-
-    return true;
-}
-
 /* get usb devices under certain usb controller */
 static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int usbctrl,
                                   libxl_device_usb **usbs, int *num)
@@ -573,47 +551,6 @@ libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid,
     return usbs;
 }
 
-/* set default value */
-static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
-{
-    libxl_ctx *ctx = libxl__gc_owner(gc);
-    struct dirent *de;
-    DIR *dir;
-    char *busid = NULL;
-
-    if (bus < 1 || addr < 1)
-        return NULL;
-
-    if (!(dir = opendir(SYSFS_USB_DEV)))
-        return NULL;
-
-    while((de = readdir(dir))) {
-        char *filename;
-        void *buf;
-        int busnum = -1;
-        int devnum = -1;
-
-        if (!de->d_name)
-            continue;
-
-        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
-        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
-            sscanf(buf, "%x", &devnum);
-
-        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
-        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
-            sscanf(buf, "%x", &busnum);
-
-        if (bus == busnum && addr == devnum) {
-            busid = strdup(de->d_name);
-            break;
-        }
-    }
-
-    closedir(dir);
-    return busid;
-}
-
 /* find first unused controller:port and give that to usb device */
 static int
 libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
@@ -1012,29 +949,10 @@ out:
     return rc;
 }
 
-int libxl__device_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
+int pv_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
 {
     int rc;
 
-    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
-    if (!usb->busid) {
-        LOG(ERROR, "USB device doesn't exist in sysfs");
-        return ERROR_INVAL;
-    }
-
-    if (!is_usb_assignable(gc, usb)) {
-        LOG(ERROR, "USB device is not assignable.");
-        rc = ERROR_FAIL;
-        goto out;
-    }
-
-    /* check usb device is already assigned by pvusb */
-    if (is_usb_assigned(gc, usb)) {
-        LOG(ERROR, "USB device is already attached to a domain.");
-        rc = ERROR_FAIL;
-        goto out;
-    }
-
     rc = libxl__device_usb_setdefault(gc, domid, usb);
     if (rc) goto out;
 
@@ -1044,18 +962,6 @@ out:
     return rc;
 }
 
-int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid,
-                         libxl_device_usb *usb,
-                         const libxl_asyncop_how *ao_how)
-{
-    AO_CREATE(ctx, domid, ao_how);
-    int rc;
-
-    rc = libxl__device_usb_add(gc, domid, usb);
-    libxl__ao_complete(egc, ao, rc);
-    return AO_INPROGRESS;
-}
-
 static int do_usb_remove(libxl__gc *gc, uint32_t domid,
                          libxl_device_usb *usb)
 {
@@ -1067,19 +973,12 @@ static int do_usb_remove(libxl__gc *gc, uint32_t domid,
     return 0;
 }
 
-static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
-                                    libxl_device_usb *usb)
+int pv_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
 {
     libxl_device_usb *usbs = NULL;
     libxl_device_usb *usb_find = NULL;
     int i, num, rc;
 
-    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
-    if (!usb->busid) {
-        LOG(ERROR, "USB device doesn't exist in sysfs");
-        return ERROR_INVAL;
-    }
-
     usbs = libxl_device_usb_list_all(gc, domid, &num);
     if (!usbs) {
        LOG(ERROR, "No USB device attached to this domain");
@@ -1108,21 +1007,7 @@ out:
     return rc;
 }
 
-int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
-                            libxl_device_usb *usb,
-                            const libxl_asyncop_how *ao_how)
-
-{
-    AO_CREATE(ctx, domid, ao_how);
-    int rc;
-
-    rc = libxl__device_usb_remove(gc, domid, usb);
-
-    libxl__ao_complete(egc, ao, rc);
-    return AO_INPROGRESS;
-}
-
-int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid)
+int pv_usb_destroy_all(libxl__gc *gc, uint32_t domid)
 {
     libxl_ctx *ctx = CTX;
     libxl_device_usbctrl *usbctrls = NULL;
@@ -1148,50 +1033,6 @@ out:
     free(usbctrls);
     return rc;
 }
-
-int libxl_device_usb_getinfo(libxl_ctx *ctx, char *busid, libxl_usbinfo *usbinfo)
-{
-    GC_INIT(ctx);
-    char *filename;
-    void *buf;
-
-    assert(busid);
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", busid);
-    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
-        sscanf(buf, "%x", &usbinfo->devnum);
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
-    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
-        sscanf(buf, "%x", &usbinfo->busnum);
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/idVendor", busid);
-    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
-        sscanf(buf, "%x", &usbinfo->idVendor);
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/idProduct", busid);
-    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
-        sscanf(buf, "%x", &usbinfo->idProduct);
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/manufacturer", busid);
-    if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) {
-        /* replace \n to \0 */
-        usbinfo->manuf = strdup(buf);
-        if (strlen(usbinfo->manuf) > 0)
-            usbinfo->manuf[strlen(usbinfo->manuf) - 1] = '\0';
-    }
-
-    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/product", busid);
-    if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) {
-       /* replace \n to \0 */
-       usbinfo->prod = strdup(buf);
-       if (strlen(usbinfo->manuf) > 0)
-           usbinfo->manuf[strlen(usbinfo->manuf) - 1] = '\0';
-    }
-
-    GC_FREE;
-    return 0;
-}
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index a6db614..df56303 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -533,7 +533,15 @@ libxl_device_pci = Struct("device_pci", [
     ("seize", bool),
     ])
 
+libxl_usb_protocol = Enumeration("usb_protocol", [
+    (0, "AUTO"),
+    (1, "PV"),
+    (2, "DEVICEMODEL"),
+    ], init_val = "LIBXL_USB_PROTOCOL_PV")
+
+
 libxl_device_usbctrl = Struct("device_usbctrl", [
+    ("protocol", libxl_usb_protocol),
     ("devid", libxl_devid),
     ("version", integer),
     ("ports", integer),
@@ -542,6 +550,7 @@ libxl_device_usbctrl = Struct("device_usbctrl", [
    ])
 
 libxl_device_usb = Struct("device_usb", [
+    ("protocol", libxl_usb_protocol),
     ("ctrl", libxl_devid),
     ("port", integer),
     ("busid", string),
@@ -626,6 +635,7 @@ libxl_vtpminfo = Struct("vtpminfo", [
     ], dir=DIR_OUT)
 
 libxl_usbctrlinfo = Struct("usbctrlinfo", [
+    ("protocol", libxl_usb_protocol),
     ("devid", libxl_devid),
     ("version", integer),
     ("ports", integer),
@@ -640,6 +650,7 @@ libxl_usbctrlinfo = Struct("usbctrlinfo", [
     ], dir=DIR_OUT)
 
 libxl_usbinfo = Struct("usbinfo", [
+    ("protocol", libxl_usb_protocol),
     ("busnum", integer),
     ("devnum", integer),
     ("idVendor", integer),
diff --git a/tools/libxl/libxl_usb.c b/tools/libxl/libxl_usb.c
new file mode 100644
index 0000000..ad96d15
--- /dev/null
+++ b/tools/libxl/libxl_usb.c
@@ -0,0 +1,224 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU Lesser General Public License as published
+ * by the Free Software Foundation; version 2.1 only. with the special
+ * exception on linking described in file LICENSE.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU Lesser General Public License for more details.
+ */
+
+#include "libxl_osdeps.h" /* must come before any other headers */
+
+#include "libxl_internal.h"
+
+#define USBHUB_CLASS_CODE 0x09
+
+static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
+{
+    libxl_ctx *ctx = libxl__gc_owner(gc);
+    struct dirent *de;
+    DIR *dir;
+    char *busid = NULL;
+
+    if (bus < 1 || addr < 1)
+        return NULL;
+
+    if (!(dir = opendir(SYSFS_USB_DEV)))
+        return NULL;
+
+    while((de = readdir(dir))) {
+        char *filename;
+        void *buf;
+        int busnum = -1;
+        int devnum = -1;
+
+        if (!de->d_name)
+            continue;
+
+        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
+        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+            sscanf(buf, "%x", &devnum);
+
+        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
+        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+            sscanf(buf, "%x", &busnum);
+
+        if (bus == busnum && addr == devnum) {
+            busid = strdup(de->d_name);
+            break;
+        }
+    }
+
+    closedir(dir);
+    return busid;
+}
+
+static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
+{
+    libxl_ctx *ctx = libxl__gc_owner(gc);
+    int classcode;
+    char *filename;
+    void *buf;
+
+    assert(usb->busid);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid);
+    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0)
+        return false;
+
+    sscanf(buf, "%x", &classcode);
+    if (classcode == USBHUB_CLASS_CODE)
+        return false;
+
+    return true;
+}
+
+int libxl__device_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
+{
+    int rc;
+
+    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
+    if (!usb->busid) {
+        LOG(ERROR, "USB device doesn't exist in sysfs");
+        return ERROR_INVAL;
+    }
+
+    if (!is_usb_assignable(gc, usb)) {
+        LOG(ERROR, "USB device is not assignable.");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /* check usb device is already assigned by pvusb */
+    if (pv_is_usb_assigned(gc, usb)) {
+        LOG(ERROR, "USB device is already attached to a domain.");
+        rc = ERROR_FAIL;
+        goto out;
+    }
+
+    /* TODO check usb device is already assigned by qemu */
+
+    switch (usb->protocol) {
+    case LIBXL_USB_PROTOCOL_PV:
+        rc = pv_usb_add(gc, domid, usb);
+        break;
+    default:
+        /* not supported */
+        rc = ERROR_FAIL;
+        break;
+    }
+
+out:
+    return rc;
+}
+
+int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid,
+                         libxl_device_usb *usb,
+                         const libxl_asyncop_how *ao_how)
+{
+    AO_CREATE(ctx, domid, ao_how);
+    int rc;
+
+    rc = libxl__device_usb_add(gc, domid, usb);
+    libxl__ao_complete(egc, ao, rc);
+    return AO_INPROGRESS;
+}
+
+static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
+                                    libxl_device_usb *usb)
+{
+    int rc;
+
+    switch (usb->protocol) {
+    case LIBXL_USB_PROTOCOL_PV:
+        usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
+        if (!usb->busid) {
+            LOG(ERROR, "USB device doesn't exist in sysfs");
+            return ERROR_INVAL;
+        }
+
+        rc = pv_usb_remove(gc, domid, usb);
+        break;
+    default:
+        /* not supported */
+        rc = ERROR_FAIL;
+        break;
+    }
+
+    return rc;
+}
+
+int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
+                            libxl_device_usb *usb,
+                            const libxl_asyncop_how *ao_how)
+
+{
+    AO_CREATE(ctx, domid, ao_how);
+    int rc;
+
+    rc = libxl__device_usb_remove(gc, domid, usb);
+
+    libxl__ao_complete(egc, ao, rc);
+    return AO_INPROGRESS;
+}
+
+int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid)
+{
+    int rc;
+
+    /* destroy all usb devices handled by pvusb */
+    rc = pv_usb_destroy_all(gc, domid);
+    if (rc) goto out;
+
+    /* TODO destroy all usb devices handled by qemu */
+
+out:
+    return rc;
+}
+
+int libxl_device_usb_getinfo(libxl_ctx *ctx, char *busid, libxl_usbinfo *usbinfo)
+{
+    GC_INIT(ctx);
+    char *filename;
+    void *buf;
+
+    assert(busid);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->devnum);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->busnum);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/idVendor", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->idVendor);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/idProduct", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
+        sscanf(buf, "%x", &usbinfo->idProduct);
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/manufacturer", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) {
+        /* replace \n to \0 */
+        usbinfo->manuf = strdup(buf);
+        if (strlen(usbinfo->manuf) > 0)
+            usbinfo->manuf[strlen(usbinfo->manuf) - 1] = '\0';
+    }
+
+    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/product", busid);
+    if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) {
+       /* replace \n to \0 */
+       usbinfo->prod = strdup(buf);
+       if (strlen(usbinfo->manuf) > 0)
+           usbinfo->manuf[strlen(usbinfo->manuf) - 1] = '\0';
+    }
+
+    GC_FREE;
+    return 0;
+}
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 750377f..f41ac6c 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -3400,7 +3400,16 @@ int main_usbattach(int argc, char **argv)
     }
 
     while (argc > optind) {
-        if (MATCH_OPTION("controller", argv[optind], oparg)) {
+        if (MATCH_OPTION("type", argv[optind], oparg)) {
+            if (!strcmp(oparg, "pv")) {
+               usb.protocol = LIBXL_USB_PROTOCOL_PV;
+            } else if (!strcmp(oparg, "qemu")) {
+               usb.protocol = LIBXL_USB_PROTOCOL_DEVICEMODEL;
+            } else {
+               fprintf(stderr, "unrecognized type `%s'\n", oparg);
+               exit(-1);
+            }
+        } else if (MATCH_OPTION("controller", argv[optind], oparg)) {
             usb.ctrl = atoi(oparg);
         } else if (MATCH_OPTION("port", argv[optind], oparg)) {
             usb.port = atoi(oparg);
@@ -3435,7 +3444,7 @@ int main_usbattach(int argc, char **argv)
 int main_usbdetach(int argc, char **argv)
 {
     uint32_t domid;
-    char *devname, *p;
+    char *devname, *p, *oparg;
     int opt;
     libxl_device_usb usb;
 
@@ -3448,9 +3457,21 @@ int main_usbdetach(int argc, char **argv)
     domid = find_domain(argv[optind++]);
     devname = argv[optind++];
 
-    if (argc > optind) {
-        fprintf(stderr, "Invalid arguments.\n");
-        exit(-1);
+    while (argc > optind) {
+        if (MATCH_OPTION("type", argv[optind], oparg)) {
+            if (!strcmp(oparg, "pv")) {
+               usb.protocol = LIBXL_USB_PROTOCOL_PV;
+            } else if (!strcmp(oparg, "qemu")) {
+               usb.protocol = LIBXL_USB_PROTOCOL_DEVICEMODEL;
+            } else {
+               fprintf(stderr, "unrecognized type `%s'\n", oparg);
+               exit(-1);
+            }
+        } else {
+            fprintf(stderr, "unrecognized argument `%s'\n", argv[optind]);
+            exit(-1);
+        }
+        optind++;
     }
 
     p = strchr(devname, '.');
@@ -3472,6 +3493,18 @@ int main_usbdetach(int argc, char **argv)
     return 0;
 }
 
+static const char *get_usb_protocol_string(libxl_usb_protocol type)
+{
+    switch (type) {
+    case LIBXL_USB_PROTOCOL_PV:
+        return "pv";
+    case LIBXL_USB_PROTOCOL_DEVICEMODEL:
+        return "qemu";
+    default:
+        return "invalid";
+    }
+}
+
 int main_usblist(int argc, char **argv)
 {
     uint32_t domid;
@@ -3491,19 +3524,21 @@ int main_usblist(int argc, char **argv)
         exit(-1);
     }
 
+    /* list pvusb info */
     usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
     if (!usbctrls) {
         return 0;
     }
 
     for (i = 0; i < numctrl; ++i) {
-        printf("%-6s %-3s %-5s %-7s %-5s %-30s\n",
-                "Devid", "BE", "state", "usb-ver", "ports", "BE-path");
+        printf("%-6s %-6s %-3s %-5s %-7s %-5s %-30s\n",
+                "Devid", "Type", "BE", "state", "usb-ver", "ports", "BE-path");
 
         if (!libxl_device_usbctrl_getinfo(ctx, domid,
                                 &usbctrls[i], &usbctrlinfo)) {
-            printf("%-6d %-3d %-5d %-7d %-5d %-30s\n",
+            printf("%-6d %-6s %-3d %-5d %-7d %-5d %-30s\n",
                     usbctrlinfo.devid,
+                    get_usb_protocol_string(usbctrlinfo.protocol),
                     usbctrlinfo.backend_id, usbctrlinfo.state,
                     usbctrlinfo.version, usbctrlinfo.ports,
                     usbctrlinfo.backend);
@@ -3517,6 +3552,9 @@ int main_usblist(int argc, char **argv)
     }
 
     free(usbctrls);
+
+    /* TODO list qemu usb device info*/
+
     return 0;
 }
 
diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
index da30ae0..4054911 100644
--- a/tools/libxl/xl_cmdtable.c
+++ b/tools/libxl/xl_cmdtable.c
@@ -557,12 +557,12 @@ struct cmd_spec cmd_table[] = {
     { "usb-attach",
       &main_usbattach, 1, 2,
       "Attach a USB device to a domain",
-      "<Domain> <bus.addr> [controller=<DevId> [port=<port>]]",
+      "<Domain> <bus.addr> [type=pv|qemu] [controller=<DevId> [port=<port>]]",
     },
     { "usb-detach",
       &main_usbdetach, 0, 1,
       "Detach a USB device from a domain",
-      "<Domain> <bus.addr>",
+      "<Domain> <bus.addr> [type=pv|qemu]",
     },
     { "usb-list",
       &main_usblist, 0, 0,
-- 
1.8.5.2

^ permalink raw reply related	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
@ 2015-04-20  5:53   ` Juergen Gross
  2015-05-18 13:55   ` Olaf Hering
                     ` (4 subsequent siblings)
  5 siblings, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-04-20  5:53 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, ian.jackson,
	caobosimon

On 04/19/2015 05:50 AM, Chunyan Liu wrote:
> Add pvusb APIs, including:
>   - attach/detach (create/destroy) virtual usb controller.
>   - attach/detach usb device
>   - list usb controller and usb devices
>   - some other helper functions
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
> ---
> Changes to v2:
>    * remove qemu emulated usb related definitions, keep the work pure pvusb.
>      In last patch, will do all refactor work to unify pvusb and qemu emulated
>      usb.
>    * use bus.addr as user interface instead of busid to do usb-attach|detach
>    * remove usb-assignable-list APIs and some other unnecessary APIs
>    * reuse libxl_read_file_contents function instead of another new function
>      to handle getting sysfs file content
>    * fix build on different platforms as pci does
>    * fix many coding style problems
>    * address other comments in last version
>    * adjust codes to let it look better
>
>   tools/libxl/Makefile                 |    2 +-
>   tools/libxl/libxl.c                  |    2 +
>   tools/libxl/libxl.h                  |   45 ++
>   tools/libxl/libxl_internal.h         |   11 +-
>   tools/libxl/libxl_osdeps.h           |   13 +
>   tools/libxl/libxl_pvusb.c            | 1201 ++++++++++++++++++++++++++++++++++
>   tools/libxl/libxl_types.idl          |   41 ++
>   tools/libxl/libxl_types_internal.idl |    1 +
>   8 files changed, 1314 insertions(+), 2 deletions(-)
>   create mode 100644 tools/libxl/libxl_pvusb.c
>
...
> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
> new file mode 100644
> index 0000000..4e4975a
> --- /dev/null
> +++ b/tools/libxl/libxl_pvusb.c
...
> +static int libxl__usbctrl_add_xenstore(libxl__gc *gc, uint32_t domid,
> +                                       libxl_device_usbctrl *usbctrl)
> +{
> +    flexarray_t *front;
> +    flexarray_t *back;
> +    libxl__device *device;
> +    xs_transaction_t t = XBT_NULL;
> +    int rc = 0;
> +    libxl_domain_config d_config;
> +    libxl_device_usbctrl usbctrl_saved;
> +    libxl__domain_userdata_lock *lock = NULL;
> +
> +    libxl_domain_config_init(&d_config);
> +    libxl_device_usbctrl_init(&usbctrl_saved);
> +    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl);
> +
> +    GCNEW(device);
> +    libxl__device_from_usbctrl(gc, domid, usbctrl, device);
> +
> +    front = flexarray_make(gc, 4, 1);
> +    back = flexarray_make(gc, 12, 1);
> +
> +    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid));
> +    flexarray_append_pair(back, "online", "1");
> +    flexarray_append_pair(back, "state", "1");
> +    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d", usbctrl->version));
> +    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d", usbctrl->ports));
> +    flexarray_append_pair(front, "backend-id", GCSPRINTF("%d", usbctrl->backend_domid));
> +    flexarray_append_pair(front, "state", "1");
> +
> +    lock = libxl__lock_domain_userdata(gc, domid);
> +    if (!lock) {
> +        rc = ERROR_LOCK_FAIL;
> +        goto out;
> +    }
> +
> +    rc = libxl__get_domain_configuration(gc, domid, &d_config);
> +    if (rc) goto out;
> +
> +    DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved, COMPARE_USBCTRL, &d_config);
> +
> +    for (;;) {
> +        rc = libxl__xs_transaction_start(gc, &t);
> +        if (rc) goto out;
> +
> +        rc = libxl__device_exists(gc, t, device);
> +        if (rc < 0) goto out;
> +        if (rc == 1) {
> +            /* already exists in xenstore */
> +            LOG(ERROR, "device already exists in xenstore");
> +            rc = ERROR_DEVICE_EXISTS;
> +            goto out;
> +        }
> +
> +        rc = libxl__set_domain_configuration(gc, domid, &d_config);
> +        if (rc) goto out;
> +
> +        libxl__device_generic_add(gc, t, device,
> +                                  libxl__xs_kvs_of_flexarray(gc, back,
> +                                                             back->count),
> +                                  libxl__xs_kvs_of_flexarray(gc, front,
> +                                                             front->count),
> +                                  NULL);
> +        libxl__usbport_add_xenstore(gc, t, domid, usbctrl);

Still no rc check.

> +        rc = libxl__xs_transaction_commit(gc, &t);
> +        if (!rc) break;
> +        if (rc < 0) goto out;
> +    }
> +
> +out:
> +    if (lock) libxl__unlock_domain_userdata(lock);
> +    libxl_device_usbctrl_dispose(&usbctrl_saved);
> +    libxl_domain_config_dispose(&d_config);
> +    return rc;
> +}
...
> +static int
> +libxl__device_usbctrl_remove_common(libxl_ctx *ctx, uint32_t domid,
> +                                    libxl_device_usbctrl *usbctrl,
> +                                    const libxl_asyncop_how *ao_how,
> +                                    int force)
> +{
> +    AO_CREATE(ctx, domid, ao_how);
> +    libxl__device *device;
> +    libxl__ao_device *aodev;
> +    libxl_device_usbctrl *usbctrls = NULL;
> +    libxl_device_usbctrl *usbctrl_find = NULL;
> +    int numctrl = 0;
> +    libxl_device_usb *usbs = NULL;
> +    int numusb = 0;
> +    int i, rc;
> +
> +    assert(usbctrl->devid != -1);
> +
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> +    if (!numctrl) {
> +        LOG(ERROR, "No USB controller exists in this domain");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    for (i = 0; i < numctrl; i++) {
> +        if (usbctrl->devid == usbctrls[i].devid) {
> +            usbctrl_find = usbctrls + i;
> +            break;
> +        }
> +    }
> +
> +    if (!usbctrl_find) {
> +        LOG(ERROR, "USB controller %d is not attached to this domain",
> +            usbctrl->devid);
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    usbctrl = usbctrl_find;
> +
> +    GCNEW(device);
> +    libxl__device_from_usbctrl(gc, domid, usbctrl, device);
> +
> +    /* Remove usb devives first */
> +    rc  = libxl__device_usb_list(gc, domid, usbctrl->devid, &usbs, &numusb);
> +    if (rc) goto out;
> +    for (i = 0; i < numusb; i++) {
> +        if (do_usb_remove(gc, domid, &usbs[i])) {
> +            LOG(ERROR, "do_usb_remove failed");
> +            rc = ERROR_FAIL;
> +            goto out;
> +        }
> +    }
> +    /* remove usbctrl */
> +    GCNEW(aodev);
> +    libxl__prepare_ao_device(ao, aodev);
> +    aodev->action = LIBXL__DEVICE_ACTION_REMOVE;
> +    aodev->dev = device;
> +    aodev->callback = device_addrm_aocomplete;
> +    aodev->force = force;
> +    libxl__initiate_device_remove(egc, aodev);
> +
> +out:
> +    free(usbctrls);
> +    free(usbs);
> +    if(rc) return AO_ABORT(rc);

Coding style.

> +    return AO_INPROGRESS;
> +}
...
> +libxl_device_usbctrl *
> +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num)
> +{
> +    GC_INIT(ctx);
> +    libxl_device_usbctrl *usbctrls = NULL;
> +    char *fe_path = NULL;
> +    char **dir = NULL;
> +    unsigned int ndirs = 0;
> +
> +    *num = 0;
> +
> +    fe_path = GCSPRINTF("%s/device/vusb",
> +                        libxl__xs_get_dompath(gc, domid));
> +    dir = libxl__xs_directory(gc, XBT_NULL, fe_path, &ndirs);
> +
> +    if (dir && ndirs) {
> +        usbctrls = malloc(sizeof(*usbctrls) * ndirs);
> +        libxl_device_usbctrl* usbctrl;
> +        libxl_device_usbctrl* end = usbctrls + ndirs;
> +        for(usbctrl = usbctrls; usbctrl < end; usbctrl++, dir++, (*num)++) {

Coding style.

> +            char *tmp;
> +            const char *be_path = libxl__xs_read(gc, XBT_NULL,
> +                                    GCSPRINTF("%s/%s/backend", fe_path, *dir));
> +
> +            libxl_device_usbctrl_init(usbctrl);
> +            usbctrl->devid = atoi(*dir);
> +
> +            tmp = libxl__xs_read(gc, XBT_NULL,
> +                                 GCSPRINTF("%s/%s/backend-id", fe_path, *dir));
> +            if (!tmp) goto outerr;
> +            usbctrl->backend_domid = atoi(tmp);
> +
> +            tmp = libxl__xs_read(gc, XBT_NULL,
> +                                 GCSPRINTF("%s/usb-ver", be_path));
> +            if (!tmp) goto outerr;
> +            usbctrl->version = atoi(tmp);
> +
> +            tmp = libxl__xs_read(gc, XBT_NULL,
> +                                 GCSPRINTF("%s/num-ports", be_path));
> +            if (!tmp) goto outerr;
> +            usbctrl->ports = atoi(tmp);
> +       }
> +    }
> +
> +    return usbctrls;
> +
> +outerr:
> +    LOG(ERROR, "Unable to list USB Controllers");
> +    for (int i = 0; i < *num; i++) {
> +        libxl_device_usbctrl_dispose(usbctrls + i);
> +    }
> +    free(usbctrls);
> +    *num = 0;
> +    return NULL;
> +}
...
> +/* get all usb devices of the domain */
> +static libxl_device_usb *
> +libxl_device_usb_list_all(libxl__gc *gc, uint32_t domid, int *num)
> +{
> +    char **usbctrls;
> +    unsigned int nd, i, j;
> +    char *be_path;
> +    int rc;
> +    libxl_device_usb *usbs = NULL;
> +
> +    *num = 0;
> +
> +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb/%d",
> +                        LIBXL_TOOLSTACK_DOMID, domid);
> +    usbctrls = libxl__xs_directory(gc, XBT_NULL, be_path, &nd);
> +
> +    for (i = 0; i < nd; i++) {
> +        int nc = 0;
> +        libxl_device_usb *tmp = NULL;
> +        rc = libxl__device_usb_list(gc, domid, atoi(usbctrls[i]), &tmp, &nc);
> +        if (!nc) continue;
> +
> +        usbs = realloc(usbs, sizeof(libxl_device_usb)*((*num) + nc));
> +        for(j = 0; j < nc; j++) {

Coding style.

> +            usbs[*num].ctrl = tmp[j].ctrl;
> +            usbs[*num].port = tmp[j].port;
> +            usbs[*num].busid = strdup(tmp[j].busid);
> +            (*num)++;
> +        }
> +        free(tmp);
> +    }
> +    return usbs;
> +}
...
> +/* set default value */
> +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
> +{
> +    libxl_ctx *ctx = libxl__gc_owner(gc);
> +    struct dirent *de;
> +    DIR *dir;
> +    char *busid = NULL;
> +
> +    if (bus < 1 || addr < 1)
> +        return NULL;
> +
> +    if (!(dir = opendir(SYSFS_USB_DEV)))
> +        return NULL;
> +
> +    while((de = readdir(dir))) {

Coding style.

> +        char *filename;
> +        void *buf;
> +        int busnum = -1;
> +        int devnum = -1;
> +
> +        if (!de->d_name)
> +            continue;
> +
> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
> +        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
> +            sscanf(buf, "%x", &devnum);
> +
> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
> +        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
> +            sscanf(buf, "%x", &busnum);
> +
> +        if (bus == busnum && addr == devnum) {
> +            busid = strdup(de->d_name);
> +            break;
> +        }
> +    }
> +
> +    closedir(dir);
> +    return busid;
> +}
> +
> +/* find first unused controller:port and give that to usb device */
> +static int
> +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
> +                                      libxl_device_usb *usb)
> +{
> +    libxl_ctx *ctx = CTX;
> +    libxl_device_usbctrl *usbctrls;
> +    libxl_device_usb *usbs = NULL;
> +    int numctrl, numusb, i, j, rc = -1;
> +    char *be_path, *tmp;
> +
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> +    if ( !numctrl)

Coding style.

> +        goto out;
> +
> +    for (i = 0; i < numctrl; i++) {
> +        rc = libxl__device_usb_list(gc, domid, usbctrls[i].devid,
> +                                    &usbs, &numusb);
> +        if (rc) continue;
> +
> +        if (!usbctrls[i].ports || numusb == usbctrls[i].ports)
> +            continue;
> +
> +        for (j = 1; i <= numusb; j++) {
> +            be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
> +                                libxl__xs_get_dompath(gc, 0), domid,
> +                                usbctrls[i].devid, j);
> +            tmp = libxl__xs_read(gc, XBT_NULL, be_path);
> +            if (tmp && !strcmp( tmp, "")) {
> +                usb->ctrl = usbctrls[i].devid;
> +                usb->port = j;
> +                break;
> +            }
> +        }
> +    }
> +
> +    rc = 0;
> +
> +out:
> +    if (usbctrls)
> +        free(usbctrls);
> +    if (usbs)
> +        free(usbs);
> +    return rc;
> +}
> +
> +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
> +                                        libxl_device_usb *usb)
> +{
> +    char *be_path, *tmp;
> +
> +    if (usb->ctrl == -1) {
> +        int ret = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
> +        /* If no existing ctrl to host this usb device, setup a new one */
> +        if (ret) {
> +            libxl_device_usbctrl usbctrl;
> +            libxl_device_usbctrl_init(&usbctrl);
> +            libxl__device_usbctrl_add(gc, domid, &usbctrl);
> +            usb->ctrl = usbctrl.devid;
> +            usb->port = 1;
> +            libxl_device_usbctrl_dispose(&usbctrl);
> +        }
> +    }
> +
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
> +                        libxl__xs_get_dompath(gc, 0),
> +                        domid, usb->ctrl, usb->port);
> +    tmp = libxl__xs_read(gc, XBT_NULL, be_path);
> +    if (!tmp || strcmp(tmp, "") ){

Coding style.

> +        LOG(ERROR, "The controller port isn't available");
> +        return ERROR_INVAL;
> +    }
> +
> +    return 0;
> +}
...
> +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath)
> +{
> +    char *path;
> +    struct stat st;
> +    int fd, rc = 0;
> +
> +    path = GCSPRINTF("%s/%s", drvpath, intf);
> +    rc = lstat(path, &st);
> +    /* already bind, return */
> +    if(rc == 0)

Coding style.

> +        return 0;
> +
> +    path = GCSPRINTF("%s/bind", drvpath);
> +    fd = open(path, O_WRONLY);
> +    if (fd < 0)
> +        return ERROR_FAIL;
> +
> +    rc = write(fd, intf, strlen(intf));
> +    close(fd);
> +    if (rc < 0)
> +        return ERROR_FAIL;
> +
> +    return 0;
> +}
> +
> +/* Is usb interface bound to usbback? */
> +static int usb_intf_is_assigned(libxl__gc *gc, char *intf)
> +{
> +    char *spath;
> +    int rc;
> +    struct stat st;
> +
> +    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf);
> +    rc = lstat(spath, &st);
> +
> +    if(rc == 0)

Coding style.

> +        return 1;
> +    if (rc < 0 && errno == ENOENT)
> +        return 0;
> +    LOGE(ERROR, "Accessing %s", spath);
> +    return -1;
> +}


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-04-19  3:50 ` [PATCH V3 4/6] xl: add pvusb commands Chunyan Liu
@ 2015-04-20  8:12   ` Juergen Gross
  2015-04-21  2:21     ` Chun Yan Liu
  2015-05-20 14:20     ` George Dunlap
  2015-05-20 14:23   ` George Dunlap
  2015-05-20 15:55   ` George Dunlap
  2 siblings, 2 replies; 53+ messages in thread
From: Juergen Gross @ 2015-04-20  8:12 UTC (permalink / raw)
  To: Chunyan Liu, xen-devel
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, ian.jackson,
	caobosimon

On 04/19/2015 05:50 AM, Chunyan Liu wrote:
> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
> usb-attach and usb-detach.
>
> To attach a usb device to guest through pvusb, one could follow
> following example:
>
>   #xl usb-ctrl-attach test_vm version=1 num_ports=8
>
>   #xl usb-list test_vm
>   will show the usb controllers and port usage under the domain.
>
>   #xl usb-attach test_vm 1.6
>   will find the first usable controller:port, and attach usb
>   device whose bus address is 1.6 (busnum is 1, devnum is 6)
>   to it. One could also specify which <controller> and which <port>.
>
>   #xl usb-detach test_vm 1.6
>
>   #xl usb-ctrl-detach test_vm dev_id
>   will destroy the controller with specified dev_id. Dev_id
>   can be traced in usb-list info.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
> ---
> Changes to v2:
>    * use bus.addr as user interface instead of busid in usb-attach|detach
>    * remove usb-assignable-list interface

Why? While lsusb in combination with xl usb-list for each domain will
give the same information, having to iterate through all domains can be
quite annoying.

An alternative would be to accept omitting the domain for xl usb-list
and list all domains with assigned usb devices in this case.

>    * add documentation
>
>   docs/man/xl.pod.1         |  38 ++++++++
>   tools/libxl/xl.h          |   5 +
>   tools/libxl/xl_cmdimpl.c  | 238 ++++++++++++++++++++++++++++++++++++++++++++++
>   tools/libxl/xl_cmdtable.c |  25 +++++
>   4 files changed, 306 insertions(+)
>
...
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -3196,6 +3196,244 @@ int main_cd_insert(int argc, char **argv)
>       return 0;
>   }
>
> +static void usbinfo_print(libxl_device_usb *usbs, int num) {
> +    int i;
> +    if ( usbs == NULL )

Coding style.

> +         return;
> +    for (i = 0; i < num; i++) {
> +        libxl_usbinfo usbinfo;
> +        libxl_usbinfo_init(&usbinfo);
> +
> +        if (usbs[i].port )

Coding style.

> +            printf("  Port %d:", usbs[i].port);
> +        if (!libxl_device_usb_getinfo(ctx, usbs[i].busid, &usbinfo)) {
> +            printf(" Bus %03x Device %03x: ID %04x:%04x %s %s\n",
> +                    usbinfo.busnum, usbinfo.devnum, usbinfo.idVendor,
> +                    usbinfo.idProduct, usbinfo.manuf, usbinfo.prod);
> +        }
> +        libxl_usbinfo_dispose(&usbinfo);
> +    }
> +}
> +
> +int main_usbctrl_attach(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    int opt;
> +    char *oparg;
> +    libxl_device_usbctrl usbctrl;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-ctrl-attach", 1) {
> +        /* No options */
> +    }
> +
> +    domid = find_domain(argv[optind++]);
> +
> +    libxl_device_usbctrl_init(&usbctrl);
> +
> +    while (argc > optind) {
> +        if (MATCH_OPTION("version", argv[optind], oparg)) {
> +            usbctrl.version = atoi(oparg);
> +        } else if (MATCH_OPTION("ports", argv[optind], oparg)) {
> +            usbctrl.ports = atoi(oparg);
> +        } else {
> +            fprintf(stderr, "unrecognized argument `%s'\n", argv[optind]);
> +            exit(-1);

I don't think this is the preferred way of error handling.
Returning with an appropriate error code would be better.

Same applies to all other uses of exit() below.

> +        }
> +        optind++;
> +    }
> +
> +    if (dryrun_only) {
> +       char* json = libxl_device_usbctrl_to_json(ctx, &usbctrl);
> +       printf("usb controller: %s\n", json);
> +       free(json);
> +       libxl_device_usbctrl_dispose(&usbctrl);
> +       if (ferror(stdout) || fflush(stdout)) {
> +           perror("stdout");
> +           exit(-1);
> +       }
> +       return 0;
> +    }
> +
> +    if (libxl_device_usbctrl_add(ctx, domid, &usbctrl, 0)) {
> +        fprintf(stderr, "libxl_device_usbctrl_add failed.\n");
> +        exit(-1);
> +    }
> +    libxl_device_usbctrl_dispose(&usbctrl);
> +    return 0;
> +}
> +
> +int main_usbctrl_detach(int argc, char **argv)
> +{
> +    uint32_t domid;
> +    int opt;
> +    libxl_device_usbctrl usbctrl;
> +
> +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-ctrl-detach", 2) {
> +        /* No options */
> +    }
> +
> +    domid = find_domain(argv[optind]);
> +
> +    libxl_device_usbctrl_init(&usbctrl);
> +    usbctrl.devid = atoi(argv[optind+1]);
> +
> +    if(libxl_device_usbctrl_remove(ctx, domid, &usbctrl, 0)) {

Coding style.

> +        fprintf(stderr, "libxl_device_usbctrl_add failed.\n");
> +        exit(-1);
> +    }
> +    libxl_device_usbctrl_dispose(&usbctrl);
> +    return 0;
> +
> +}

Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 1/6] libxl: export some functions for pvusb use
  2015-04-19  3:50 ` [PATCH V3 1/6] libxl: export some functions for pvusb use Chunyan Liu
@ 2015-04-20 16:25   ` Olaf Hering
  2015-05-18 13:34     ` George Dunlap
  2015-05-18 14:05   ` Wei Liu
  1 sibling, 1 reply; 53+ messages in thread
From: Olaf Hering @ 2015-04-20 16:25 UTC (permalink / raw)
  To: xen-devel; +Cc: george.dunlap, ian.jackson, wei.liu2, ian.campbell, Chunyan Liu

On Sun, Apr 19, Chunyan Liu wrote:

> +++ b/tools/libxl/libxl_internal.h

> +_hidden int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device);
> +_hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,
> +                                 uint32_t *domid);

> +/* generic callback for devices that only need to set ao_complete */
> +_hidden void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev);

If that goes in I may move some or all of the vscsi code in libxl.c into
libxl_vscsi.c.

Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-04-20  8:12   ` Juergen Gross
@ 2015-04-21  2:21     ` Chun Yan Liu
  2015-05-20 14:20     ` George Dunlap
  1 sibling, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-04-21  2:21 UTC (permalink / raw)
  To: xen-devel, Juergen Gross
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, ian.jackson,
	caobosimon



>>> On 4/20/2015 at 04:12 PM, in message <5534B4D8.4010109@suse.com>, Juergen Gross
<jgross@suse.com> wrote: 
> On 04/19/2015 05:50 AM, Chunyan Liu wrote: 
> > Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list, 
> > usb-attach and usb-detach. 
> > 
> > To attach a usb device to guest through pvusb, one could follow 
> > following example: 
> > 
> >   #xl usb-ctrl-attach test_vm version=1 num_ports=8 
> > 
> >   #xl usb-list test_vm 
> >   will show the usb controllers and port usage under the domain. 
> > 
> >   #xl usb-attach test_vm 1.6 
> >   will find the first usable controller:port, and attach usb 
> >   device whose bus address is 1.6 (busnum is 1, devnum is 6) 
> >   to it. One could also specify which <controller> and which <port>. 
> > 
> >   #xl usb-detach test_vm 1.6 
> > 
> >   #xl usb-ctrl-detach test_vm dev_id 
> >   will destroy the controller with specified dev_id. Dev_id 
> >   can be traced in usb-list info. 
> > 
> > Signed-off-by: Chunyan Liu <cyliu@suse.com> 
> > Signed-off-by: Simon Cao <caobosimon@gmail.com> 
> > --- 
> > Changes to v2: 
> >    * use bus.addr as user interface instead of busid in usb-attach|detach 
> >    * remove usb-assignable-list interface 
>  
> Why? While lsusb in combination with xl usb-list for each domain will 
> give the same information, having to iterate through all domains can be 
> quite annoying. 
>  
> An alternative would be to accept omitting the domain for xl usb-list 
> and list all domains with assigned usb devices in this case. 

Ian & George, how do you think?

>  
> >    * add documentation 
> > 
> >   docs/man/xl.pod.1         |  38 ++++++++ 
> >   tools/libxl/xl.h          |   5 + 
> >   tools/libxl/xl_cmdimpl.c  | 238  
> ++++++++++++++++++++++++++++++++++++++++++++++ 
> >   tools/libxl/xl_cmdtable.c |  25 +++++ 
> >   4 files changed, 306 insertions(+) 
> > 
> ... 
> > --- a/tools/libxl/xl_cmdimpl.c 
> > +++ b/tools/libxl/xl_cmdimpl.c 
> > @@ -3196,6 +3196,244 @@ int main_cd_insert(int argc, char **argv) 
> >       return 0; 
> >   } 
> > 
> > +static void usbinfo_print(libxl_device_usb *usbs, int num) { 
> > +    int i; 
> > +    if ( usbs == NULL ) 
>  
> Coding style. 
>  
> > +         return; 
> > +    for (i = 0; i < num; i++) { 
> > +        libxl_usbinfo usbinfo; 
> > +        libxl_usbinfo_init(&usbinfo); 
> > + 
> > +        if (usbs[i].port ) 
>  
> Coding style. 
>  
> > +            printf("  Port %d:", usbs[i].port); 
> > +        if (!libxl_device_usb_getinfo(ctx, usbs[i].busid, &usbinfo)) { 
> > +            printf(" Bus %03x Device %03x: ID %04x:%04x %s %s\n", 
> > +                    usbinfo.busnum, usbinfo.devnum, usbinfo.idVendor, 
> > +                    usbinfo.idProduct, usbinfo.manuf, usbinfo.prod); 
> > +        } 
> > +        libxl_usbinfo_dispose(&usbinfo); 
> > +    } 
> > +} 
> > + 
> > +int main_usbctrl_attach(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    int opt; 
> > +    char *oparg; 
> > +    libxl_device_usbctrl usbctrl; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-ctrl-attach", 1) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    domid = find_domain(argv[optind++]); 
> > + 
> > +    libxl_device_usbctrl_init(&usbctrl); 
> > + 
> > +    while (argc > optind) { 
> > +        if (MATCH_OPTION("version", argv[optind], oparg)) { 
> > +            usbctrl.version = atoi(oparg); 
> > +        } else if (MATCH_OPTION("ports", argv[optind], oparg)) { 
> > +            usbctrl.ports = atoi(oparg); 
> > +        } else { 
> > +            fprintf(stderr, "unrecognized argument `%s'\n", argv[optind]); 
> > +            exit(-1); 
>  
> I don't think this is the preferred way of error handling. 
> Returning with an appropriate error code would be better. 
>  
> Same applies to all other uses of exit() below. 
>  
> > +        } 
> > +        optind++; 
> > +    } 
> > + 
> > +    if (dryrun_only) { 
> > +       char* json = libxl_device_usbctrl_to_json(ctx, &usbctrl); 
> > +       printf("usb controller: %s\n", json); 
> > +       free(json); 
> > +       libxl_device_usbctrl_dispose(&usbctrl); 
> > +       if (ferror(stdout) || fflush(stdout)) { 
> > +           perror("stdout"); 
> > +           exit(-1); 
> > +       } 
> > +       return 0; 
> > +    } 
> > + 
> > +    if (libxl_device_usbctrl_add(ctx, domid, &usbctrl, 0)) { 
> > +        fprintf(stderr, "libxl_device_usbctrl_add failed.\n"); 
> > +        exit(-1); 
> > +    } 
> > +    libxl_device_usbctrl_dispose(&usbctrl); 
> > +    return 0; 
> > +} 
> > + 
> > +int main_usbctrl_detach(int argc, char **argv) 
> > +{ 
> > +    uint32_t domid; 
> > +    int opt; 
> > +    libxl_device_usbctrl usbctrl; 
> > + 
> > +    SWITCH_FOREACH_OPT(opt, "", NULL, "usb-ctrl-detach", 2) { 
> > +        /* No options */ 
> > +    } 
> > + 
> > +    domid = find_domain(argv[optind]); 
> > + 
> > +    libxl_device_usbctrl_init(&usbctrl); 
> > +    usbctrl.devid = atoi(argv[optind+1]); 
> > + 
> > +    if(libxl_device_usbctrl_remove(ctx, domid, &usbctrl, 0)) { 
>  
> Coding style. 
>  
> > +        fprintf(stderr, "libxl_device_usbctrl_add failed.\n"); 
> > +        exit(-1); 
> > +    } 
> > +    libxl_device_usbctrl_dispose(&usbctrl); 
> > +    return 0; 
> > + 
> > +} 
>  
> Juergen 
>  
>  
> _______________________________________________ 
> Xen-devel mailing list 
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 1/6] libxl: export some functions for pvusb use
  2015-04-20 16:25   ` Olaf Hering
@ 2015-05-18 13:34     ` George Dunlap
  0 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-05-18 13:34 UTC (permalink / raw)
  To: Olaf Hering, xen-devel; +Cc: ian.jackson, wei.liu2, ian.campbell, Chunyan Liu

On 04/20/2015 05:25 PM, Olaf Hering wrote:
> On Sun, Apr 19, Chunyan Liu wrote:
> 
>> +++ b/tools/libxl/libxl_internal.h
> 
>> +_hidden int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device);
>> +_hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,
>> +                                 uint32_t *domid);
> 
>> +/* generic callback for devices that only need to set ao_complete */
>> +_hidden void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev);
> 
> If that goes in I may move some or all of the vscsi code in libxl.c into
> libxl_vscsi.c.

It sounds like this would be useful independent of the pvusb stuff.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
  2015-04-20  5:53   ` Juergen Gross
@ 2015-05-18 13:55   ` Olaf Hering
  2015-05-18 18:07   ` Wei Liu
                     ` (3 subsequent siblings)
  5 siblings, 0 replies; 53+ messages in thread
From: Olaf Hering @ 2015-05-18 13:55 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	ian.jackson, caobosimon

On Sun, Apr 19, Chunyan Liu wrote:

> +static int libxl__usbctrl_add_xenstore(libxl__gc *gc, uint32_t domid,
> +                                       libxl_device_usbctrl *usbctrl)
> +{

> +    flexarray_append_pair(back, "state", "1");

> +    flexarray_append_pair(front, "state", "1");

This (and perhaps other places) should be converted to xenbus_state, see
commit 25519c75b7e05fd82d7f2959aaa85518b5564cc3 ("libxl: convert strings
and ints to xenbus_state").

Olaf

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 1/6] libxl: export some functions for pvusb use
  2015-04-19  3:50 ` [PATCH V3 1/6] libxl: export some functions for pvusb use Chunyan Liu
  2015-04-20 16:25   ` Olaf Hering
@ 2015-05-18 14:05   ` Wei Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Wei Liu @ 2015-05-18 14:05 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	ian.jackson, caobosimon

On Sun, Apr 19, 2015 at 11:50:47AM +0800, Chunyan Liu wrote:
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>

On the basis that this can help reduce the length of libxl.c and improve
maintainability by moving stuff out of libxl.c to dedicated files.

Acked-by: Wei Liu <wei.liu2@citrix.com>

> ---
>  tools/libxl/libxl.c          | 6 +++---
>  tools/libxl/libxl_internal.h | 5 +++++
>  2 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index 511eef1..b05d18b 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -1963,7 +1963,7 @@ out:
>  /******************************************************************************/
>  
>  /* generic callback for devices that only need to set ao_complete */
> -static void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev)
> +void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev)
>  {
>      STATE_AO_GC(aodev->ao);
>  
> @@ -1986,7 +1986,7 @@ out:
>  }
>  
>  /* common function to get next device id */
> -static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
> +int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
>  {
>      char *dompath, **l;
>      unsigned int nb;
> @@ -2005,7 +2005,7 @@ static int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device)
>      return nextid;
>  }
>  
> -static int libxl__resolve_domid(libxl__gc *gc, const char *name,
> +int libxl__resolve_domid(libxl__gc *gc, const char *name,
>                                  uint32_t *domid)
>  {
>      if (!name)
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index 9c22309..42eb1b9 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -1080,6 +1080,9 @@ _hidden int libxl__init_console_from_channel(libxl__gc *gc,
>                                               libxl__device_console *console,
>                                               int dev_num,
>                                               libxl_device_channel *channel);
> +_hidden int libxl__device_nextid(libxl__gc *gc, uint32_t domid, char *device);
> +_hidden int libxl__resolve_domid(libxl__gc *gc, const char *name,
> +                                 uint32_t *domid);
>  
>  /*
>   * For each aggregate type which can be used as an input we provide:
> @@ -2208,6 +2211,8 @@ struct libxl__ao_device {
>  
>  /* Starts preparing to add/remove a bunch of devices. */
>  _hidden void libxl__multidev_begin(libxl__ao *ao, libxl__multidev*);
> +/* generic callback for devices that only need to set ao_complete */
> +_hidden void device_addrm_aocomplete(libxl__egc *egc, libxl__ao_device *aodev);
>  
>  /* Prepares to add/remove one of many devices.
>   * Calls libxl__prepare_ao_device on libxl__ao_device argument provided and
> -- 
> 1.8.5.2

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file
  2015-04-19  3:50 ` [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file Chunyan Liu
@ 2015-05-18 14:23   ` Ian Jackson
  2015-05-18 14:28     ` Ian Campbell
  2015-05-18 14:30     ` Wei Liu
  2015-05-18 14:25   ` Wei Liu
  1 sibling, 2 replies; 53+ messages in thread
From: Ian Jackson @ 2015-05-18 14:23 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, xen-devel, caobosimon

Chunyan Liu writes ("[PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file"):
> Sysfs file has size=4096 but actual file content is less than that.

Wow.

Is there any danger that the actual size might be >4096 ?


> Current libxl_read_file_contents will treat it as error when file size
> and actual file content differs, so reading sysfs file content with
> this function always fails. Fix it so that we can reuse this function
> to get sysfs file content in later pvusb work.

I'm uncomfortable with removing an error check from this function for
all its call sites.

I think, sadly, that we are going to need a new function - at least, a
new entrypoint.


We don't want to repeat the whole of libxl__read_file_contents.

Perhaps the bulk should be made into libxl__read_file_contents_core
which takes a boolean instructing whether to tolerate magically
shrinking files ?

Setting that boolean probably ought to arrange to insist that the
function gets eof, in case the file is actually bigger rather than
smaller than the size.


Ian, Wei ?

Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file
  2015-04-19  3:50 ` [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file Chunyan Liu
  2015-05-18 14:23   ` Ian Jackson
@ 2015-05-18 14:25   ` Wei Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Wei Liu @ 2015-05-18 14:25 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	ian.jackson, caobosimon

On Sun, Apr 19, 2015 at 11:50:48AM +0800, Chunyan Liu wrote:
> Sysfs file has size=4096 but actual file content is less than that.
> Current libxl_read_file_contents will treat it as error when file size
> and actual file content differs, so reading sysfs file content with
> this function always fails. Fix it so that we can reuse this function
> to get sysfs file content in later pvusb work.
> 

I'm not sure if I should classify this as a bug in Linux's sysfs
interface.

In any case, we would still like to detect the error case that file size
is changed under our feet.

I have a dumb idea of having a dedicated function that is used to read
sysfs, but I'm not sure if it is too dumb.

I will wait for Ian and Ian's input on this.

Wei.

> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> ---
>  tools/libxl/libxl_utils.c | 5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/tools/libxl/libxl_utils.c b/tools/libxl/libxl_utils.c
> index 9053b27..18ad2b8 100644
> --- a/tools/libxl/libxl_utils.c
> +++ b/tools/libxl/libxl_utils.c
> @@ -363,12 +363,9 @@ int libxl_read_file_contents(libxl_ctx *ctx, const char *filename,
>          if (!data) goto xe;
>  
>          rs = fread(data, 1, datalen, f);
> -        if (rs != datalen) {
> +        if (rs != datalen && !feof(f)) {
>              if (ferror(f))
>                  LOGE(ERROR, "failed to read %s", filename);
> -            else if (feof(f))
> -                LOG(ERROR, "%s changed size while we were reading it",
> -		    filename);
>              else
>                  abort();
>              goto xe;
> -- 
> 1.8.5.2

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file
  2015-05-18 14:23   ` Ian Jackson
@ 2015-05-18 14:28     ` Ian Campbell
  2015-05-18 14:30     ` Wei Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-18 14:28 UTC (permalink / raw)
  To: Ian Jackson
  Cc: lars.kurth, wei.liu2, george.dunlap, Chunyan Liu, xen-devel, caobosimon

On Mon, 2015-05-18 at 15:23 +0100, Ian Jackson wrote:
> Perhaps the bulk should be made into libxl__read_file_contents_core
> which takes a boolean instructing whether to tolerate magically
> shrinking files ?
> 
> Setting that boolean probably ought to arrange to insist that the
> function gets eof, in case the file is actually bigger rather than
> smaller than the size.
> 
> 
> Ian, Wei ?

Sounds ok to me.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file
  2015-05-18 14:23   ` Ian Jackson
  2015-05-18 14:28     ` Ian Campbell
@ 2015-05-18 14:30     ` Wei Liu
  2015-05-19  3:21       ` Chun Yan Liu
  1 sibling, 1 reply; 53+ messages in thread
From: Wei Liu @ 2015-05-18 14:30 UTC (permalink / raw)
  To: Ian Jackson
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, Chunyan Liu,
	xen-devel, caobosimon

On Mon, May 18, 2015 at 03:23:38PM +0100, Ian Jackson wrote:
> Chunyan Liu writes ("[PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file"):
> > Sysfs file has size=4096 but actual file content is less than that.
> 
> Wow.
> 
> Is there any danger that the actual size might be >4096 ?
> 
> 
> > Current libxl_read_file_contents will treat it as error when file size
> > and actual file content differs, so reading sysfs file content with
> > this function always fails. Fix it so that we can reuse this function
> > to get sysfs file content in later pvusb work.
> 
> I'm uncomfortable with removing an error check from this function for
> all its call sites.
> 
> I think, sadly, that we are going to need a new function - at least, a
> new entrypoint.
> 
> 
> We don't want to repeat the whole of libxl__read_file_contents.
> 
> Perhaps the bulk should be made into libxl__read_file_contents_core
> which takes a boolean instructing whether to tolerate magically
> shrinking files ?
> 
> Setting that boolean probably ought to arrange to insist that the
> function gets eof, in case the file is actually bigger rather than
> smaller than the size.
> 
> 
> Ian, Wei ?
> 

Yes, we need a new entry point.

Wei.

> Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
  2015-04-20  5:53   ` Juergen Gross
  2015-05-18 13:55   ` Olaf Hering
@ 2015-05-18 18:07   ` Wei Liu
  2015-05-19  3:20     ` Chun Yan Liu
  2015-05-19 18:06   ` George Dunlap
                     ` (2 subsequent siblings)
  5 siblings, 1 reply; 53+ messages in thread
From: Wei Liu @ 2015-05-18 18:07 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: lars.kurth, wei.liu2, ian.campbell, george.dunlap, xen-devel,
	ian.jackson, caobosimon

(I look at overall code structure this pass. I haven't done a line by
line review.)

On Sun, Apr 19, 2015 at 11:50:49AM +0800, Chunyan Liu wrote:
> Add pvusb APIs, including:
>  - attach/detach (create/destroy) virtual usb controller.
>  - attach/detach usb device
>  - list usb controller and usb devices
>  - some other helper functions
> 
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>
> ---
> Changes to v2:
>   * remove qemu emulated usb related definitions, keep the work pure pvusb.
>     In last patch, will do all refactor work to unify pvusb and qemu emulated
>     usb.
>   * use bus.addr as user interface instead of busid to do usb-attach|detach
>   * remove usb-assignable-list APIs and some other unnecessary APIs
>   * reuse libxl_read_file_contents function instead of another new function
>     to handle getting sysfs file content
>   * fix build on different platforms as pci does
>   * fix many coding style problems
>   * address other comments in last version
>   * adjust codes to let it look better
> 
>  tools/libxl/Makefile                 |    2 +-
>  tools/libxl/libxl.c                  |    2 +
>  tools/libxl/libxl.h                  |   45 ++
>  tools/libxl/libxl_internal.h         |   11 +-
>  tools/libxl/libxl_osdeps.h           |   13 +
>  tools/libxl/libxl_pvusb.c            | 1201 ++++++++++++++++++++++++++++++++++
>  tools/libxl/libxl_types.idl          |   41 ++
>  tools/libxl/libxl_types_internal.idl |    1 +

You also need to document the xenstore keys and values somewhere under
docs directory.

And you forgot to update libxl_retrieve_domain_configuration function.

>  8 files changed, 1314 insertions(+), 2 deletions(-)
>  create mode 100644 tools/libxl/libxl_pvusb.c
> 
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> index 1b16598..d52281f 100644
> --- a/tools/libxl/Makefile
> +++ b/tools/libxl/Makefile
> @@ -95,7 +95,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
>  			libxl_internal.o libxl_utils.o libxl_uuid.o \
>  			libxl_json.o libxl_aoutils.o libxl_numa.o libxl_vnuma.o \
>  			libxl_save_callout.o _libxl_save_msgs_callout.o \
> -			libxl_qmp.o libxl_event.o libxl_fork.o $(LIBXL_OBJS-y)
> +			libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o $(LIBXL_OBJS-y)
>  LIBXL_OBJS += libxl_genid.o
>  LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
>  
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index b05d18b..a7c81d9 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -1611,6 +1611,8 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
>  
>      if (libxl__device_pci_destroy_all(gc, domid) < 0)
>          LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "pci shutdown failed for domid %d", domid);
> +    if (libxl__device_usb_destroy_all(gc, domid) < 0)
> +         LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "usb shutdown failed for domid %d", domid);
>      rc = xc_domain_pause(ctx->xch, domid);
>      if (rc < 0) {
>          LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_pause failed for %d", domid);
> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 6bc75c5..cbe3519 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -114,6 +114,12 @@
>  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
>  
>  /*
> + * LIBXL_HAVE_PVUSB indicates the functions for doing hot-plug of
> + * USB devices through pvusb.
> + */
> +#define LIBXL_HAVE_PVUSB 1
> +
> +/*
>   * LIBXL_HAVE_BUILDINFO_HVM_VENDOR_DEVICE indicates that the
>   * libxl_vendor_device field is present in the hvm sections of
>   * libxl_domain_build_info. This field tells libxl which
> @@ -1224,6 +1230,45 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk,
>                         const libxl_asyncop_how *ao_how)
>                         LIBXL_EXTERNAL_CALLERS_ONLY;
>  
> +/* USB Controllers*/
> +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,
> +                         libxl_device_usbctrl *usbctrl,
> +                         const libxl_asyncop_how *ao_how)
> +                         LIBXL_EXTERNAL_CALLERS_ONLY;
> +
> +int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid,
> +                         libxl_device_usbctrl *usbctrl,
> +                         const libxl_asyncop_how *ao_how)
> +                         LIBXL_EXTERNAL_CALLERS_ONLY;
> +
> +int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid,
> +                         libxl_device_usbctrl *usbctrl,
> +                         const libxl_asyncop_how *ao_how)
> +                         LIBXL_EXTERNAL_CALLERS_ONLY;
> +
> +libxl_device_usbctrl *libxl_device_usbctrl_list(libxl_ctx *ctx,
> +                            uint32_t domid, int *num);
> +
> +
> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
> +                                libxl_device_usbctrl *usbctrl,
> +                                libxl_usbctrlinfo *usbctrlinfo)
> +                                LIBXL_EXTERNAL_CALLERS_ONLY;
> +
> +/* USB Devices */
> +int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
> +                         const libxl_asyncop_how *ao_how)
> +                         LIBXL_EXTERNAL_CALLERS_ONLY;
> +
> +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_usb *usb,
> +                            const libxl_asyncop_how *ao_how)
> +                            LIBXL_EXTERNAL_CALLERS_ONLY;
> +
> +libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid,
> +                                        int usbctrl, int *num);
> +
> +int libxl_device_usb_getinfo(libxl_ctx *ctx, char *intf, libxl_usbinfo *usbinfo)
> +                             LIBXL_EXTERNAL_CALLERS_ONLY;
>  /* Network Interfaces */
>  int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic *nic,
>                           const libxl_asyncop_how *ao_how)
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index 42eb1b9..f426ed8 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -2422,6 +2422,13 @@ _hidden void libxl__device_vtpm_add(libxl__egc *egc, uint32_t domid,
>                                     libxl_device_vtpm *vtpm,
>                                     libxl__ao_device *aodev);
>  
> +/* from libxl_usb */
> +_hidden int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
> +                                      libxl_device_usbctrl *usbctrl);
> +_hidden int libxl__device_usb_add(libxl__gc *gc, uint32_t domid,
> +                            libxl_device_usb *usb);
> +_hidden int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid);
> +
>  /* Internal function to connect a vkb device */
>  _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid,
>                                    libxl_device_vkb *vkb);
> @@ -3628,7 +3635,9 @@ static inline void libxl__update_config_vtpm(libxl__gc *gc,
>  #define COMPARE_PCI(a, b) ((a)->func == (b)->func &&    \
>                             (a)->bus == (b)->bus &&      \
>                             (a)->dev == (b)->dev)
> -
> +#define COMPARE_USB(a, b) (!strcmp((a)->busid, (b)->busid))
> +#define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid)
> + 
>  /* DEVICE_ADD
>   *
>   * Add a device in libxl_domain_config structure
> diff --git a/tools/libxl/libxl_osdeps.h b/tools/libxl/libxl_osdeps.h
> index 08eaf0c..55caf71 100644
> --- a/tools/libxl/libxl_osdeps.h
> +++ b/tools/libxl/libxl_osdeps.h
> @@ -24,6 +24,8 @@
>  #define _GNU_SOURCE
>  
>  #if defined(__NetBSD__)
> +#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
> +#define SYSFS_USBBACK_DRIVER   "/kern/xen/usb"
>  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
>  #define SYSFS_PCIBACK_DRIVER   "/kern/xen/pci"
>  #define NETBACK_NIC_NAME       "xvif%ui%d"
> @@ -31,6 +33,8 @@
>  #elif defined(__OpenBSD__)
>  #include <util.h>
>  #elif defined(__linux__)
> +#define SYSFS_USB_DEV          "/sys/bus/usb/devices"
> +#define SYSFS_USBBACK_DRIVER   "/sys/bus/usb/drivers/usbback"
>  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices"
>  #define SYSFS_PCIBACK_DRIVER   "/sys/bus/pci/drivers/pciback"
>  #define NETBACK_NIC_NAME       "vif%u.%d"
> @@ -38,12 +42,21 @@
>  #elif defined(__sun__)
>  #include <stropts.h>
>  #elif defined(__FreeBSD__)
> +#define SYSFS_USB_DEV          "/dev/null"
> +#define SYSFS_USBBACK_DRIVER   "/dev/null"
>  #define SYSFS_PCI_DEV          "/dev/null"
>  #define SYSFS_PCIBACK_DRIVER   "/dev/null"
>  #define NETBACK_NIC_NAME       "xnb%u.%d"
>  #include <libutil.h>
>  #endif
>  
> +#ifndef SYSFS_USBBACK_DRIVER
> +#error define SYSFS_USBBACK_DRIVER for your platform
> +#endif
> +#ifndef SYSFS_USB_DEV
> +#error define SYSFS_USB_DEV for your platform
> +#endif
> +
>  #ifndef SYSFS_PCIBACK_DRIVER
>  #error define SYSFS_PCIBACK_DRIVER for your platform
>  #endif
> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
> new file mode 100644
> index 0000000..4e4975a
> --- /dev/null
> +++ b/tools/libxl/libxl_pvusb.c
> @@ -0,0 +1,1201 @@
> +/*
> + * This program is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU Lesser General Public License as published
> + * by the Free Software Foundation; version 2.1 only. with the special
> + * exception on linking described in file LICENSE.
> + *
> + * This program is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + * GNU Lesser General Public License for more details.
> + */
> +
> +#include "libxl_osdeps.h" /* must come before any other headers */
> +
> +#include "libxl_internal.h"
> +
> +#define USBBACK_INFO_PATH "/libxl/usbback"
> +
> +#define USBHUB_CLASS_CODE 0x09
> +

I'm not very familiar with how USB is supposed to work. Can you explain
why this particular value is chosen?

> +static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
> +                                            libxl_device_usbctrl *usbctrl)
> +{
> +    int rc;
> +
> +    if (!usbctrl->version)
> +        usbctrl->version = 2;
> +

How hard would it be to implement USB 3? I assume this depends on QEMU's
support? I.e. we just need to specify the version to 3 and it should
just work? Just curious.

> +    if (!usbctrl->ports)
> +        usbctrl->ports = 8;
> +
> +    rc = libxl__resolve_domid(gc, usbctrl->backend_domname,
> +                              &usbctrl->backend_domid);
> +    return rc;
> +}
> +
> +static void libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid,
> +                                       libxl_device_usbctrl *usbctrl,
> +                                       libxl__device *device)
> +{
> +    device->backend_devid   = usbctrl->devid;
> +    device->backend_domid   = usbctrl->backend_domid;
> +    device->backend_kind    = LIBXL__DEVICE_KIND_VUSB;
> +    device->devid           = usbctrl->devid;
> +    device->domid           = domid;
> +    device->kind            = LIBXL__DEVICE_KIND_VUSB;
> +}
> +
> +static int libxl__usbport_add_xenstore(libxl__gc *gc,
> +                                       xs_transaction_t tran,
> +                                       uint32_t domid,
> +                                       libxl_device_usbctrl *usbctrl)
> +{
> +    char *path;
> +    int i;
> +
> +    path = GCSPRINTF("%s/backend/vusb/%d/%d/port",
> +                     libxl__xs_get_dompath(gc, 0), domid, usbctrl->devid);
> +
> +    libxl__xs_mkdir(gc, tran, path, NULL, 0);
> +
> +    for (i = 1; i <= usbctrl->ports; i++) {
> +        if (libxl__xs_write_checked(gc, tran, GCSPRINTF("%s/%d", path, i), ""))
> +            return ERROR_FAIL;
> +    }
> +
> +    return 0;
> +}
> +
> +static int libxl__usbctrl_add_xenstore(libxl__gc *gc, uint32_t domid,
> +                                       libxl_device_usbctrl *usbctrl)
> +{
> +    flexarray_t *front;
> +    flexarray_t *back;
> +    libxl__device *device;
> +    xs_transaction_t t = XBT_NULL;
> +    int rc = 0;
> +    libxl_domain_config d_config;
> +    libxl_device_usbctrl usbctrl_saved;
> +    libxl__domain_userdata_lock *lock = NULL;
> +
> +    libxl_domain_config_init(&d_config);
> +    libxl_device_usbctrl_init(&usbctrl_saved);
> +    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl);
> +
> +    GCNEW(device);
> +    libxl__device_from_usbctrl(gc, domid, usbctrl, device);
> +
> +    front = flexarray_make(gc, 4, 1);
> +    back = flexarray_make(gc, 12, 1);
> +
> +    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid));
> +    flexarray_append_pair(back, "online", "1");
> +    flexarray_append_pair(back, "state", "1");
> +    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d", usbctrl->version));
> +    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d", usbctrl->ports));
> +    flexarray_append_pair(front, "backend-id", GCSPRINTF("%d", usbctrl->backend_domid));

Line too long.

> +    flexarray_append_pair(front, "state", "1");
> +
> +    lock = libxl__lock_domain_userdata(gc, domid);
> +    if (!lock) {
> +        rc = ERROR_LOCK_FAIL;
> +        goto out;
> +    }
> +
> +    rc = libxl__get_domain_configuration(gc, domid, &d_config);
> +    if (rc) goto out;
> +
> +    DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved, COMPARE_USBCTRL, &d_config);
> +

Line too long.

> +    for (;;) {
> +        rc = libxl__xs_transaction_start(gc, &t);
> +        if (rc) goto out;
> +
> +        rc = libxl__device_exists(gc, t, device);
> +        if (rc < 0) goto out;
> +        if (rc == 1) {
> +            /* already exists in xenstore */
> +            LOG(ERROR, "device already exists in xenstore");
> +            rc = ERROR_DEVICE_EXISTS;
> +            goto out;
> +        }
> +
> +        rc = libxl__set_domain_configuration(gc, domid, &d_config);
> +        if (rc) goto out;
> +
> +        libxl__device_generic_add(gc, t, device,
> +                                  libxl__xs_kvs_of_flexarray(gc, back,
> +                                                             back->count),
> +                                  libxl__xs_kvs_of_flexarray(gc, front,
> +                                                             front->count),
> +                                  NULL);
> +        libxl__usbport_add_xenstore(gc, t, domid, usbctrl);
> +        rc = libxl__xs_transaction_commit(gc, &t);
> +        if (!rc) break;
> +        if (rc < 0) goto out;
> +    }
> +

You don't have aodev so you cannot check update_json. Maybe you need
aodev?

That field update_json is set to true when doing hotplug. It's set to
false during domain creation.

The same comment applies to other add functions as well.

> +out:
> +    if (lock) libxl__unlock_domain_userdata(lock);
> +    libxl_device_usbctrl_dispose(&usbctrl_saved);
> +    libxl_domain_config_dispose(&d_config);
> +    return rc;
> +}
> +
> +int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
> +                              libxl_device_usbctrl *usbctrl)
> +{
> +    int rc = 0;
> +
> +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
> +    if (rc < 0) goto out;
> +
> +    if (usbctrl->devid == -1) {
> +        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");
> +        if (usbctrl->devid < 0) {
> +            rc = ERROR_FAIL;
> +            goto out;
> +        }
> +    }
> +
> +    if (libxl__usbctrl_add_xenstore(gc, domid, usbctrl) < 0){
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +out:
> +    return rc;
> +}
> +
> +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,
> +                             libxl_device_usbctrl *usbctrl,
> +                             const libxl_asyncop_how *ao_how)
> +{
> +    AO_CREATE(ctx, domid, ao_how);
> +    int rc;
> +
> +    rc = libxl__device_usbctrl_add(gc, domid, usbctrl);

Hmm... Your remove function is async while this one is sync, why?

> +    libxl__ao_complete(egc, ao, rc);
> +    return AO_INPROGRESS;
> +}
> +
> +static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int usbctrl,
> +                                  libxl_device_usb **usbs, int *num);
> +
> +static int do_usb_remove(libxl__gc *gc, uint32_t domid,
> +                         libxl_device_usb *usb);
> +
> +static int
> +libxl__device_usbctrl_remove_common(libxl_ctx *ctx, uint32_t domid,
> +                                    libxl_device_usbctrl *usbctrl,
> +                                    const libxl_asyncop_how *ao_how,
> +                                    int force)
> +{
> +    AO_CREATE(ctx, domid, ao_how);
> +    libxl__device *device;
> +    libxl__ao_device *aodev;
> +    libxl_device_usbctrl *usbctrls = NULL;
> +    libxl_device_usbctrl *usbctrl_find = NULL;
> +    int numctrl = 0;
> +    libxl_device_usb *usbs = NULL;
> +    int numusb = 0;
> +    int i, rc;
> +
> +    assert(usbctrl->devid != -1);
> +
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> +    if (!numctrl) {
> +        LOG(ERROR, "No USB controller exists in this domain");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    for (i = 0; i < numctrl; i++) {
> +        if (usbctrl->devid == usbctrls[i].devid) {
> +            usbctrl_find = usbctrls + i;
> +            break;
> +        }
> +    }
> +
> +    if (!usbctrl_find) {
> +        LOG(ERROR, "USB controller %d is not attached to this domain",
> +            usbctrl->devid);
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    usbctrl = usbctrl_find;
> +
> +    GCNEW(device);
> +    libxl__device_from_usbctrl(gc, domid, usbctrl, device);
> +
> +    /* Remove usb devives first */
> +    rc  = libxl__device_usb_list(gc, domid, usbctrl->devid, &usbs, &numusb);
> +    if (rc) goto out;
> +    for (i = 0; i < numusb; i++) {
> +        if (do_usb_remove(gc, domid, &usbs[i])) {
> +            LOG(ERROR, "do_usb_remove failed");
> +            rc = ERROR_FAIL;
> +            goto out;
> +        }
> +    }
> +    /* remove usbctrl */
> +    GCNEW(aodev);
> +    libxl__prepare_ao_device(ao, aodev);
> +    aodev->action = LIBXL__DEVICE_ACTION_REMOVE;
> +    aodev->dev = device;
> +    aodev->callback = device_addrm_aocomplete;
> +    aodev->force = force;
> +    libxl__initiate_device_remove(egc, aodev);
> +
> +out:
> +    free(usbctrls);
> +    free(usbs);
> +    if(rc) return AO_ABORT(rc);
> +    return AO_INPROGRESS;
> +}
> +
> +int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid,
> +                                libxl_device_usbctrl *usbctrl,
> +                                const libxl_asyncop_how *ao_how)
> +{
> +    return libxl__device_usbctrl_remove_common(ctx, domid, usbctrl, ao_how, 0);
> +}
> +
> +int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid,
> +                                 libxl_device_usbctrl *usbctrl,
> +                                 const libxl_asyncop_how *ao_how)
> +{
> +    return libxl__device_usbctrl_remove_common(ctx, domid, usbctrl, ao_how, 1);
> +}
> +
> +libxl_device_usbctrl *
> +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num)
> +{
> +    GC_INIT(ctx);
> +    libxl_device_usbctrl *usbctrls = NULL;
> +    char *fe_path = NULL;
> +    char **dir = NULL;
> +    unsigned int ndirs = 0;
> +
> +    *num = 0;
> +
> +    fe_path = GCSPRINTF("%s/device/vusb",
> +                        libxl__xs_get_dompath(gc, domid));
> +    dir = libxl__xs_directory(gc, XBT_NULL, fe_path, &ndirs);
> +
> +    if (dir && ndirs) {
> +        usbctrls = malloc(sizeof(*usbctrls) * ndirs);
> +        libxl_device_usbctrl* usbctrl;
> +        libxl_device_usbctrl* end = usbctrls + ndirs;
> +        for(usbctrl = usbctrls; usbctrl < end; usbctrl++, dir++, (*num)++) {
> +            char *tmp;
> +            const char *be_path = libxl__xs_read(gc, XBT_NULL,
> +                                    GCSPRINTF("%s/%s/backend", fe_path, *dir));
> +
> +            libxl_device_usbctrl_init(usbctrl);
> +            usbctrl->devid = atoi(*dir);
> +
> +            tmp = libxl__xs_read(gc, XBT_NULL,
> +                                 GCSPRINTF("%s/%s/backend-id", fe_path, *dir));
> +            if (!tmp) goto outerr;
> +            usbctrl->backend_domid = atoi(tmp);
> +
> +            tmp = libxl__xs_read(gc, XBT_NULL,
> +                                 GCSPRINTF("%s/usb-ver", be_path));
> +            if (!tmp) goto outerr;
> +            usbctrl->version = atoi(tmp);
> +
> +            tmp = libxl__xs_read(gc, XBT_NULL,
> +                                 GCSPRINTF("%s/num-ports", be_path));
> +            if (!tmp) goto outerr;
> +            usbctrl->ports = atoi(tmp);
> +       }
> +    }
> +
> +    return usbctrls;
> +
> +outerr:
> +    LOG(ERROR, "Unable to list USB Controllers");
> +    for (int i = 0; i < *num; i++) {
> +        libxl_device_usbctrl_dispose(usbctrls + i);
> +    }
> +    free(usbctrls);

It might be useful to provide a function to free the whole list of
device. There are similar functions in libxl.

> +    *num = 0;
> +    return NULL;
> +}
> +
> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
> +                                libxl_device_usbctrl *usbctrl,
> +                                libxl_usbctrlinfo *usbctrlinfo)
> +{
> +    GC_INIT(ctx);
> +    char *dompath, *usbctrlpath;
> +    char *val;
> +    int rc = 0;
> +
> +    usbctrlinfo->devid = usbctrl->devid;
> +    usbctrlinfo->ports = usbctrl->ports;
> +    usbctrlinfo->version = usbctrl->version;
> +
> +    dompath = libxl__xs_get_dompath(gc, domid);
> +    usbctrlpath = GCSPRINTF("%s/device/vusb/%d", dompath, usbctrlinfo->devid);
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/backend", usbctrlpath));
> +    usbctrlinfo->backend = libxl__strdup(NOGC, val);
> +    if (!usbctrlinfo->backend) {
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/backend-id", usbctrlpath));
> +    usbctrlinfo->backend_id = val ? strtoul(val, NULL, 10) : -1;
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/state", usbctrlpath));
> +    usbctrlinfo->state = val ? strtoul(val, NULL, 10) : -1;
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/event-channel", usbctrlpath));
> +    usbctrlinfo->evtch = val ? strtoul(val, NULL, 10) : -1;
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/urb-ring-ref", usbctrlpath));
> +    usbctrlinfo->ref_urb = val ? strtoul(val, NULL, 10) : -1;
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/conn-ring-ref", usbctrlpath));
> +    usbctrlinfo->ref_conn= val ? strtoul(val, NULL, 10) : -1;
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/frontend", usbctrlinfo->backend));
> +    usbctrlinfo->frontend = libxl__strdup(NOGC, val);
> +
> +    val = libxl__xs_read(gc, XBT_NULL,
> +                         GCSPRINTF("%s/frontend-id", usbctrlinfo->backend));
> +    usbctrlinfo->frontend_id = val ? strtoul(val, NULL, 10) : -1;
> +
> +out:
> +    GC_FREE;
> +    return rc;
> +}
> +
> +/* usb device functions */
> +
> +static int
> +libxl__device_usb_assigned_list(libxl__gc *gc,
> +                                libxl_device_usb **list, int *num)
> +{
> +    char **domlist;
> +    unsigned int nd = 0, i, j;
> +    char *be_path;
> +    libxl_device_usb *usb;
> +
> +    *list = NULL;
> +    *num = 0;
> +
> +    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd);
> +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb", LIBXL_TOOLSTACK_DOMID);
> +    for (i = 0; i < nd; i++) {
> +        char *path, *num_ports, **ctrl_list;
> +        unsigned int nc = 0;
> +        path = GCSPRINTF("%s/%s", be_path, domlist[i]);
> +        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path , &nc);
> +
> +        for (j = 0; j < nc; j++) {
> +            path = GCSPRINTF("%s/%s/%s/num-ports", be_path,
> +                             domlist[i], ctrl_list[j]);
> +            num_ports = libxl__xs_read(gc, XBT_NULL, path);
> +            if (num_ports) {
> +                int nport = atoi(num_ports), k;
> +                char *devpath, *busid;
> +
> +                for (k = 1; k <= nport; k++) {
> +                    devpath = GCSPRINTF("%s/%s/%s/port/%u", be_path,
> +                                        domlist[i], ctrl_list[j], k);
> +                    busid = libxl__xs_read(gc, XBT_NULL, devpath);
> +                    /* If there are USB device attached, add it to list */
> +                    if (busid && strcmp(busid, "") ) {
> +                        *list = realloc(*list,
> +                                  sizeof(libxl_device_usb) * ((*num) + 1));
> +                        if (*list == NULL)
> +                            return ERROR_NOMEM;

Use GCREALLOC_ARRAY.

> +                        usb = *list + *num;
> +                        usb->ctrl = atoi(ctrl_list[j]);
> +                        usb->port = k;
> +                        usb->busid = strdup(busid);
> +                        (*num)++;
> +                    }
> +                }
> +            }
> +        }
> +    }
> +    libxl__ptr_add(gc, *list);
> +

No need to do this if you use GCREALLOC_ARRAY.

> +    return 0;
> +}
> +
> +static bool is_usb_in_array(libxl_device_usb *usbs, int num,
> +                            libxl_device_usb *usb)
> +{
> +    int i;
> +
> +    for (i = 0; i < num; i++) {
> +        if (!strcmp(usbs[i].busid, usb->busid) )
> +            return true;
> +    }
> +
> +    return false;
> +}
> +
> +/* check if USB device is already assigned to a domain */
> +static bool is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    libxl_device_usb *usbs;
> +    int rc, num;
> +
> +    rc = libxl__device_usb_assigned_list(gc, &usbs, &num);
> +    if (rc) {
> +        LOG(ERROR, "Fail to get assigned usb list");
> +        return true;
> +    }
> +
> +    if (is_usb_in_array(usbs, num, usb))
> +        return true;
> +
> +    return false;
> +}
> +
> +/* check if USB device type is assignable */
> +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    libxl_ctx *ctx = libxl__gc_owner(gc);
> +    int classcode;
> +    char *filename;
> +    void *buf;
> +
> +    assert(usb->busid);
> +
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid);
> +    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0)
> +        return false;
> +
> +    sscanf(buf, "%x", &classcode);
> +    if (classcode == USBHUB_CLASS_CODE)
> +        return false;
> +
> +    return true;
> +}
> +
> +/* get usb devices under certain usb controller */
> +static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int usbctrl,
> +                                  libxl_device_usb **usbs, int *num)
> +{
> +    char *be_path, *num_devs;
> +    int n, i;
> +
> +    *usbs = NULL;
> +    *num = 0;
> +
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d",
> +                        libxl__xs_get_dompath(gc, 0), domid, usbctrl);
> +    num_devs = libxl__xs_read(gc, XBT_NULL,
> +                              GCSPRINTF("%s/num-ports", be_path));
> +    if (!num_devs)
> +        return 0;
> +
> +    n = atoi(num_devs);
> +    *usbs = calloc(n, sizeof(libxl_device_usb));
> +

libxl__calloc(NOGC, ...)

> +    for (i = 0; i < n; i++) {
> +        char *busid;
> +        libxl_device_usb *usb = NULL;
> +
> +        busid = libxl__xs_read(gc, XBT_NULL,
> +                               GCSPRINTF("%s/port/%d", be_path, i + 1));
> +        if (busid && strcmp(busid, "")) {
> +            usb = *usbs + *num;
> +            usb->ctrl = usbctrl;
> +            usb->port = i + 1;
> +            usb->busid = strdup(busid);
> +            (*num)++;
> +        }
> +    }
> +
> +    return 0;
> +}
> +
> +/* get all usb devices of the domain */
> +static libxl_device_usb *
> +libxl_device_usb_list_all(libxl__gc *gc, uint32_t domid, int *num)
> +{
> +    char **usbctrls;
> +    unsigned int nd, i, j;
> +    char *be_path;
> +    int rc;
> +    libxl_device_usb *usbs = NULL;
> +
> +    *num = 0;
> +
> +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb/%d",
> +                        LIBXL_TOOLSTACK_DOMID, domid);
> +    usbctrls = libxl__xs_directory(gc, XBT_NULL, be_path, &nd);
> +
> +    for (i = 0; i < nd; i++) {
> +        int nc = 0;
> +        libxl_device_usb *tmp = NULL;
> +        rc = libxl__device_usb_list(gc, domid, atoi(usbctrls[i]), &tmp, &nc);
> +        if (!nc) continue;
> +
> +        usbs = realloc(usbs, sizeof(libxl_device_usb)*((*num) + nc));

libxl__realloc

> +        for(j = 0; j < nc; j++) {
> +            usbs[*num].ctrl = tmp[j].ctrl;
> +            usbs[*num].port = tmp[j].port;
> +            usbs[*num].busid = strdup(tmp[j].busid);
> +            (*num)++;
> +        }
> +        free(tmp);
> +    }
> +    return usbs;
> +}
> +
> +libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid,
> +                                        int usbctrl, int *num)
> +{
> +    GC_INIT(ctx);
> +    libxl_device_usb *usbs = NULL;
> +
> +    libxl__device_usb_list(gc, domid, usbctrl, &usbs, num);
> +
> +    GC_FREE;
> +    return usbs;
> +}
> +
> +/* set default value */
> +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr)
> +{
> +    libxl_ctx *ctx = libxl__gc_owner(gc);
> +    struct dirent *de;
> +    DIR *dir;
> +    char *busid = NULL;
> +
> +    if (bus < 1 || addr < 1)
> +        return NULL;
> +
> +    if (!(dir = opendir(SYSFS_USB_DEV)))
> +        return NULL;
> +
> +    while((de = readdir(dir))) {
> +        char *filename;
> +        void *buf;
> +        int busnum = -1;
> +        int devnum = -1;
> +
> +        if (!de->d_name)
> +            continue;
> +
> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name);
> +        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
> +            sscanf(buf, "%x", &devnum);
> +
> +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name);
> +        if (!libxl_read_file_contents(ctx, filename, &buf, NULL))
> +            sscanf(buf, "%x", &busnum);
> +
> +        if (bus == busnum && addr == devnum) {
> +            busid = strdup(de->d_name);
> +            break;
> +        }
> +    }
> +
> +    closedir(dir);
> +    return busid;
> +}
> +
> +/* find first unused controller:port and give that to usb device */
> +static int
> +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
> +                                      libxl_device_usb *usb)
> +{
> +    libxl_ctx *ctx = CTX;
> +    libxl_device_usbctrl *usbctrls;
> +    libxl_device_usb *usbs = NULL;
> +    int numctrl, numusb, i, j, rc = -1;
> +    char *be_path, *tmp;
> +
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> +    if ( !numctrl)
> +        goto out;
> +
> +    for (i = 0; i < numctrl; i++) {
> +        rc = libxl__device_usb_list(gc, domid, usbctrls[i].devid,
> +                                    &usbs, &numusb);
> +        if (rc) continue;
> +
> +        if (!usbctrls[i].ports || numusb == usbctrls[i].ports)
> +            continue;
> +
> +        for (j = 1; i <= numusb; j++) {
> +            be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
> +                                libxl__xs_get_dompath(gc, 0), domid,
> +                                usbctrls[i].devid, j);
> +            tmp = libxl__xs_read(gc, XBT_NULL, be_path);
> +            if (tmp && !strcmp( tmp, "")) {
> +                usb->ctrl = usbctrls[i].devid;
> +                usb->port = j;
> +                break;
> +            }
> +        }
> +    }
> +
> +    rc = 0;
> +
> +out:
> +    if (usbctrls)
> +        free(usbctrls);
> +    if (usbs)
> +        free(usbs);
> +    return rc;
> +}
> +
> +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
> +                                        libxl_device_usb *usb)
> +{
> +    char *be_path, *tmp;
> +
> +    if (usb->ctrl == -1) {
> +        int ret = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
> +        /* If no existing ctrl to host this usb device, setup a new one */
> +        if (ret) {
> +            libxl_device_usbctrl usbctrl;
> +            libxl_device_usbctrl_init(&usbctrl);
> +            libxl__device_usbctrl_add(gc, domid, &usbctrl);
> +            usb->ctrl = usbctrl.devid;
> +            usb->port = 1;
> +            libxl_device_usbctrl_dispose(&usbctrl);
> +        }
> +    }
> +
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
> +                        libxl__xs_get_dompath(gc, 0),
> +                        domid, usb->ctrl, usb->port);
> +    tmp = libxl__xs_read(gc, XBT_NULL, be_path);
> +    if (!tmp || strcmp(tmp, "") ){
> +        LOG(ERROR, "The controller port isn't available");
> +        return ERROR_INVAL;
> +    }
> +
> +    return 0;
> +}
> +
> +/* xenstore usb data */
> +static int libxl__device_usb_add_xenstore(libxl__gc *gc, uint32_t domid,
> +                                          libxl_device_usb *usb)
> +{
> +    char *be_path;
> +    int rc;
> +    libxl_domain_config d_config;
> +    libxl_device_usb usb_saved;
> +    libxl__domain_userdata_lock *lock = NULL;
> +
> +    libxl_domain_config_init(&d_config);
> +    libxl_device_usb_init(&usb_saved);
> +    libxl_device_usb_copy(CTX, &usb_saved, usb);
> +
> +    lock = libxl__lock_domain_userdata(gc, domid);
> +    if (!lock) {
> +        rc = ERROR_LOCK_FAIL;
> +        goto out;
> +    }
> +
> +    rc = libxl__get_domain_configuration(gc, domid, &d_config);
> +    if (rc) goto out;
> +
> +    DEVICE_ADD(usb, usbs, domid, &usb_saved, COMPARE_USB, &d_config);
> +
> +    rc = libxl__set_domain_configuration(gc, domid, &d_config);
> +    if (rc) goto out;
> +
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
> +                  libxl__xs_get_dompath(gc, 0), domid, usb->ctrl, usb->port);
> +    LOG(DEBUG, "Adding new usb device to xenstore");
> +    if (libxl__xs_write_checked(gc, XBT_NULL, be_path, usb->busid)) {
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +

This is not right. See libxl_internal.h, search for "lock json
config" to see the pattern. In short, you should use a xenstore
transaction instead of libxl__xs_write_checked with XBT_NULL.

Also you need to use aodev->update_json.

> +    rc = 0;
> +
> +out:
> +    if (lock) libxl__unlock_domain_userdata(lock);
> +    libxl_device_usb_dispose(&usb_saved);
> +    libxl_domain_config_dispose(&d_config);
> +    return rc;
> +}
> +
> +static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t domid,
> +                                             libxl_device_usb *usb)
> +{
> +    char *be_path;
> +
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d",
> +                        libxl__xs_get_dompath(gc, 0),
> +                        domid, usb->ctrl, usb->port);
> +    LOG(DEBUG, "Removing USB device from xenstore");
> +    if (libxl__xs_write_checked(gc,XBT_NULL, be_path, ""))
> +        return ERROR_FAIL;
> +
> +    return 0;
> +}
> +
> +/* bind/unbind usb device interface */
> +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath)
> +{
> +    char *path, *spath, *dp = NULL;
> +    int fd = -1;
> +    int rc = 0;
> +    struct stat st;
> +
> +    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf);
> +    if (!lstat(spath, &st)) {
> +        /* Find the canonical path to the driver. */
> +        dp = libxl__zalloc(gc, PATH_MAX);
> +        dp = realpath(spath, dp);
> +
> +        path = GCSPRINTF("%s/unbind", spath);
> +        fd = open(path, O_WRONLY);
> +        if (fd < 0)
> +            return ERROR_FAIL;
> +        rc = write(fd, intf, strlen(intf));
> +        close(fd);
> +        if (rc < 0)
> +            return ERROR_FAIL;
> +    }
> +
> +    if (drvpath)
> +        *drvpath = dp;
> +
> +    return 0;
> +}
> +
> +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath)
> +{
> +    char *path;
> +    struct stat st;
> +    int fd, rc = 0;
> +
> +    path = GCSPRINTF("%s/%s", drvpath, intf);
> +    rc = lstat(path, &st);
> +    /* already bind, return */
> +    if(rc == 0)
> +        return 0;
> +
> +    path = GCSPRINTF("%s/bind", drvpath);
> +    fd = open(path, O_WRONLY);
> +    if (fd < 0)
> +        return ERROR_FAIL;
> +
> +    rc = write(fd, intf, strlen(intf));
> +    close(fd);
> +    if (rc < 0)
> +        return ERROR_FAIL;
> +
> +    return 0;
> +}
> +
> +/* Is usb interface bound to usbback? */
> +static int usb_intf_is_assigned(libxl__gc *gc, char *intf)
> +{
> +    char *spath;
> +    int rc;
> +    struct stat st;
> +
> +    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf);
> +    rc = lstat(spath, &st);
> +
> +    if(rc == 0)
> +        return 1;
> +    if (rc < 0 && errno == ENOENT)
> +        return 0;
> +    LOGE(ERROR, "Accessing %s", spath);
> +    return -1;
> +}
> +
> +static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb,
> +                                  char ***intfs, int *num)
> +{
> +    DIR *dir;
> +    struct dirent *entry;
> +    char *buf;
> +    int rc = 0;
> +
> +    *intfs = NULL;
> +    *num = 0;
> +
> +    buf = GCSPRINTF("%s:", usb->busid);
> +
> +    if (!(dir = opendir(SYSFS_USB_DEV))) {
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    while ((entry = readdir(dir)) != NULL) {
> +        if (!strncmp(entry->d_name, buf, strlen(buf))){
> +            *intfs = realloc(*intfs, sizeof(char *) * (*num + 1));
> +            if (*intfs == NULL) {
> +                rc = ERROR_FAIL;
> +                goto out;
> +            }
> +            (*intfs)[*num] = strdup(entry->d_name);
> +            (*num)++;
> +        }
> +    }
> +
> +    closedir(dir);
> +
> +out:
> +    return rc;
> +}
> +
> +/* Cann't write '.' or ':' into Xenstore as key. So, change '.' to '_',

"Can't"

And this behaviour needs to be documented.

> + * change ':' to '-'.
> + */
> +static char *usb_interface_encode(char *busid)
> +{
> +    char *str = strdup(busid);
> +    int i, len = strlen(str);
> +
> +    for (i = 0; i < len; i++) {
> +        if (str[i] == '.')
> +            str[i] = '_';
> +         if (str[i] == ':')

Indentation.

> +            str[i] = '-';
> +    }
> +    return str;
> +}
> +
> +/* unbind usb device from usbback driver, if there are many interfaces
> + * under the usb device, then check each interface, unbind from usbback
> + * driver and rebind to original driver
> + */
> +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    char **intfs = NULL;
> +    char *path;
> +    int num = 0, i;
> +    int rc = 0;
> +    char *usb_encode = NULL;
> +
> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
> +        return ERROR_FAIL;
> +
> +    usb_encode = usb_interface_encode(usb->busid);
> +
> +    for (i = 0; i < num; i++){
> +        char *intf = intfs[i];
> +        char *drvpath = NULL;
> +
> +        if (usb_intf_is_assigned(gc, intf) > 0) {
> +            /* unbind interface from usbback driver */
> +            if (unbind_usb_intf(gc, intf, NULL) < 0) {
> +                rc = ERROR_FAIL;
> +                goto out;
> +            }
> +        }
> +
> +        /* bind interface to its originial driver */
> +        drvpath = libxl__xs_read(gc, XBT_NULL,
> +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
> +                  usb_encode, usb_interface_encode(intf)));
> +        if (drvpath) {
> +            if (bind_usb_intf(gc, intf, drvpath) < 0) {
> +                LOGE(ERROR, "Couldn't bind %s to %s", intf, drvpath);
> +                rc = ERROR_FAIL;
> +                goto out;
> +            }
> +        }
> +    }
> +
> +    /* finally, remove xs driver path */
> +    path = GCSPRINTF(USBBACK_INFO_PATH"/%s", usb_encode);
> +    libxl__xs_rm_checked(gc, XBT_NULL, path);
> +
> +out:
> +    if (intfs) {
> +        for (i = 0; i < num; i++)
> +            free(intfs[i]);
> +        free(intfs);
> +    }
> +    free(usb_encode);
> +    return rc;
> +}
> +
> +/* bind usb device to "usbback" driver, if there are many interfaces
> + * under the usb device, check each interface, unbind from original
> + * driver and bind to usbback driver.
> + */
> +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    char **intfs = NULL;
> +    int num = 0, i;
> +    int rc = 0;
> +    char *usb_encode = NULL;
> +
> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
> +        return ERROR_FAIL;
> +
> +    usb_encode = usb_interface_encode(usb->busid);
> +
> +    for (i = 0; i < num; i++){
> +        char *intf = intfs[i];
> +        char *path = NULL;
> +        char *drvpath = NULL;
> +
> +        /* already assigned to usbback */
> +        if (usb_intf_is_assigned(gc, intf) > 0)
> +            continue;
> +
> +        /* unbind interface from original driver */
> +        if (unbind_usb_intf(gc, intf, &drvpath) < 0) {
> +            rc = ERROR_FAIL;
> +            goto out_rebind;
> +        }
> +
> +        if (drvpath) {
> +            /* write driver path to xenstore for later rebinding */
> +            path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
> +                             usb_encode, usb_interface_encode(intf));
> +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) {
> +                LOG(WARN, "Write of %s to node %s failed", drvpath, path);
> +            }
> +        }
> +
> +        /* bind interface to usbback */
> +        if (bind_usb_intf(gc, intf, SYSFS_USBBACK_DRIVER) < 0){
> +            LOGE(ERROR, "Couldn't bind %s to %s", intf, SYSFS_USBBACK_DRIVER);
> +            rc = ERROR_FAIL;
> +            goto out_rebind;
> +        }
> +    }
> +
> +    goto out;
> +
> +out_rebind:
> +    /* some interfaces might be bound to usbback, unbind it then and
> +     * rebind to its original driver
> +     */
> +    usbback_dev_unassign(gc, usb);
> +out:
> +    if (intfs) {
> +        for (i = 0; i < num; i++)
> +            free(intfs[i]);
> +        free(intfs);
> +    }
> +    free(usb_encode);
> +    return rc;
> +}
> +
> +static int do_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
> +{
> +    int rc = 0;
> +
> +    rc = libxl__device_usb_add_xenstore(gc, domid, usb);
> +    if (rc) goto out;
> +
> +    rc = usbback_dev_assign(gc, usb);
> +    if (rc)
> +        libxl__device_usb_remove_xenstore(gc, domid, usb);
> +
> +out:
> +    return rc;
> +}
> +
> +int libxl__device_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb)
> +{
> +    int rc;
> +
> +    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
> +    if (!usb->busid) {
> +        LOG(ERROR, "USB device doesn't exist in sysfs");
> +        return ERROR_INVAL;
> +    }
> +
> +    if (!is_usb_assignable(gc, usb)) {
> +        LOG(ERROR, "USB device is not assignable.");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    /* check usb device is already assigned by pvusb */
> +    if (is_usb_assigned(gc, usb)) {
> +        LOG(ERROR, "USB device is already attached to a domain.");
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    rc = libxl__device_usb_setdefault(gc, domid, usb);
> +    if (rc) goto out;
> +
> +    rc = do_usb_add(gc, domid, usb);
> +
> +out:
> +    return rc;
> +}
> +
> +int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid,
> +                         libxl_device_usb *usb,
> +                         const libxl_asyncop_how *ao_how)
> +{
> +    AO_CREATE(ctx, domid, ao_how);
> +    int rc;
> +
> +    rc = libxl__device_usb_add(gc, domid, usb);

Hmm... Your remove function is async and this one is sync. See comment
of usbctrl_add.

> +    libxl__ao_complete(egc, ao, rc);
> +    return AO_INPROGRESS;
> +}
> +
> +static int do_usb_remove(libxl__gc *gc, uint32_t domid,
> +                         libxl_device_usb *usb)
> +{
> +    if (libxl__device_usb_remove_xenstore(gc, domid, usb))
> +        return -1;
> +
> +    usbback_dev_unassign(gc, usb);
> +
> +    return 0;
> +}
> +
> +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid,
> +                                    libxl_device_usb *usb)
> +{
> +    libxl_device_usb *usbs = NULL;
> +    libxl_device_usb *usb_find = NULL;
> +    int i, num, rc;
> +
> +    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr);
> +    if (!usb->busid) {
> +        LOG(ERROR, "USB device doesn't exist in sysfs");
> +        return ERROR_INVAL;
> +    }
> +
> +    usbs = libxl_device_usb_list_all(gc, domid, &num);
> +    if (!usbs) {
> +       LOG(ERROR, "No USB device attached to this domain");
> +       return ERROR_FAIL;
> +    }
> +
> +    for (i = 0; i < num; i++) {
> +        if (!strcmp(usb->busid, usbs[i].busid)) {
> +            usb_find = usbs + i;
> +            break;
> +        }
> +    }
> +
> +    /* doesn't find the usb device in domain's usb device list*/
> +    if (!usb_find) {
> +        LOG(ERROR, "USB device %x.%x is not attached to this domain",
> +            usb->hostbus, usb->hostaddr);
> +        rc = ERROR_FAIL;
> +        goto out;
> +    }
> +
> +    rc = do_usb_remove(gc, domid, usb_find);
> +
> +out:
> +    free(usbs);
> +    return rc;
> +}
> +
> +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,
> +                            libxl_device_usb *usb,
> +                            const libxl_asyncop_how *ao_how)
> +
> +{
> +    AO_CREATE(ctx, domid, ao_how);
> +    int rc;
> +
> +    rc = libxl__device_usb_remove(gc, domid, usb);
> +

Same here.

Wei.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-18 18:07   ` Wei Liu
@ 2015-05-19  3:20     ` Chun Yan Liu
  2015-05-19 10:20       ` George Dunlap
  2015-05-20  9:04       ` Wei Liu
  0 siblings, 2 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-05-19  3:20 UTC (permalink / raw)
  To: Wei Liu
  Cc: lars.kurth, ian.campbell, george.dunlap, xen-devel, ian.jackson,
	caobosimon



>>> On 5/19/2015 at 02:07 AM, in message
<20150518180724.GH9503@zion.uk.xensource.com>, Wei Liu <wei.liu2@citrix.com>
wrote: 
> (I look at overall code structure this pass. I haven't done a line by 
> line review.) 
>  
> On Sun, Apr 19, 2015 at 11:50:49AM +0800, Chunyan Liu wrote: 
> > Add pvusb APIs, including: 
> >  - attach/detach (create/destroy) virtual usb controller. 
> >  - attach/detach usb device 
> >  - list usb controller and usb devices 
> >  - some other helper functions 
> >  
> > Signed-off-by: Chunyan Liu <cyliu@suse.com> 
> > Signed-off-by: Simon Cao <caobosimon@gmail.com> 
> > --- 
> > Changes to v2: 
> >   * remove qemu emulated usb related definitions, keep the work pure pvusb. 
> >     In last patch, will do all refactor work to unify pvusb and qemu  
> emulated 
> >     usb. 
> >   * use bus.addr as user interface instead of busid to do usb-attach|detach 
> >   * remove usb-assignable-list APIs and some other unnecessary APIs 
> >   * reuse libxl_read_file_contents function instead of another new function 
> >     to handle getting sysfs file content 
> >   * fix build on different platforms as pci does 
> >   * fix many coding style problems 
> >   * address other comments in last version 
> >   * adjust codes to let it look better 
> >  
> >  tools/libxl/Makefile                 |    2 +- 
> >  tools/libxl/libxl.c                  |    2 + 
> >  tools/libxl/libxl.h                  |   45 ++ 
> >  tools/libxl/libxl_internal.h         |   11 +- 
> >  tools/libxl/libxl_osdeps.h           |   13 + 
> >  tools/libxl/libxl_pvusb.c            | 1201  
> ++++++++++++++++++++++++++++++++++ 
> >  tools/libxl/libxl_types.idl          |   41 ++ 
> >  tools/libxl/libxl_types_internal.idl |    1 + 
>  
> You also need to document the xenstore keys and values somewhere under 
> docs directory. 

Thanks, will add that.

>  
> And you forgot to update libxl_retrieve_domain_configuration function. 

Updating.

>  
> >  8 files changed, 1314 insertions(+), 2 deletions(-) 
> >  create mode 100644 tools/libxl/libxl_pvusb.c 
> >  
> > diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile 
> > index 1b16598..d52281f 100644 
> > --- a/tools/libxl/Makefile 
> > +++ b/tools/libxl/Makefile 
> > @@ -95,7 +95,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o  
> libxl_pci.o \ 
> >  			libxl_internal.o libxl_utils.o libxl_uuid.o \ 
> >  			libxl_json.o libxl_aoutils.o libxl_numa.o libxl_vnuma.o \ 
> >  			libxl_save_callout.o _libxl_save_msgs_callout.o \ 
> > -			libxl_qmp.o libxl_event.o libxl_fork.o $(LIBXL_OBJS-y) 
> > +			libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o $(LIBXL_OBJS-y) 
> >  LIBXL_OBJS += libxl_genid.o 
> >  LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o 
> >   
> > diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c 
> > index b05d18b..a7c81d9 100644 
> > --- a/tools/libxl/libxl.c 
> > +++ b/tools/libxl/libxl.c 
> > @@ -1611,6 +1611,8 @@ void libxl__destroy_domid(libxl__egc *egc,  
> libxl__destroy_domid_state *dis) 
> >   
> >      if (libxl__device_pci_destroy_all(gc, domid) < 0) 
> >          LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "pci shutdown failed for domid  
> %d", domid); 
> > +    if (libxl__device_usb_destroy_all(gc, domid) < 0) 
> > +         LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "usb shutdown failed for domid  
> %d", domid); 
> >      rc = xc_domain_pause(ctx->xch, domid); 
> >      if (rc < 0) { 
> >          LIBXL__LOG_ERRNOVAL(ctx, LIBXL__LOG_ERROR, rc, "xc_domain_pause  
> failed for %d", domid); 
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h 
> > index 6bc75c5..cbe3519 100644 
> > --- a/tools/libxl/libxl.h 
> > +++ b/tools/libxl/libxl.h 
> > @@ -114,6 +114,12 @@ 
> >  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1 
> >   
> >  /* 
> > + * LIBXL_HAVE_PVUSB indicates the functions for doing hot-plug of 
> > + * USB devices through pvusb. 
> > + */ 
> > +#define LIBXL_HAVE_PVUSB 1 
> > + 
> > +/* 
> >   * LIBXL_HAVE_BUILDINFO_HVM_VENDOR_DEVICE indicates that the 
> >   * libxl_vendor_device field is present in the hvm sections of 
> >   * libxl_domain_build_info. This field tells libxl which 
> > @@ -1224,6 +1230,45 @@ int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t  
> domid, libxl_device_disk *disk, 
> >                         const libxl_asyncop_how *ao_how) 
> >                         LIBXL_EXTERNAL_CALLERS_ONLY; 
> >   
> > +/* USB Controllers*/ 
> > +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid, 
> > +                         libxl_device_usbctrl *usbctrl, 
> > +                         const libxl_asyncop_how *ao_how) 
> > +                         LIBXL_EXTERNAL_CALLERS_ONLY; 
> > + 
> > +int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid, 
> > +                         libxl_device_usbctrl *usbctrl, 
> > +                         const libxl_asyncop_how *ao_how) 
> > +                         LIBXL_EXTERNAL_CALLERS_ONLY; 
> > + 
> > +int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid, 
> > +                         libxl_device_usbctrl *usbctrl, 
> > +                         const libxl_asyncop_how *ao_how) 
> > +                         LIBXL_EXTERNAL_CALLERS_ONLY; 
> > + 
> > +libxl_device_usbctrl *libxl_device_usbctrl_list(libxl_ctx *ctx, 
> > +                            uint32_t domid, int *num); 
> > + 
> > + 
> > +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid, 
> > +                                libxl_device_usbctrl *usbctrl, 
> > +                                libxl_usbctrlinfo *usbctrlinfo) 
> > +                                LIBXL_EXTERNAL_CALLERS_ONLY; 
> > + 
> > +/* USB Devices */ 
> > +int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_usb  
> *usb, 
> > +                         const libxl_asyncop_how *ao_how) 
> > +                         LIBXL_EXTERNAL_CALLERS_ONLY; 
> > + 
> > +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid,  
> libxl_device_usb *usb, 
> > +                            const libxl_asyncop_how *ao_how) 
> > +                            LIBXL_EXTERNAL_CALLERS_ONLY; 
> > + 
> > +libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, 
> > +                                        int usbctrl, int *num); 
> > + 
> > +int libxl_device_usb_getinfo(libxl_ctx *ctx, char *intf, libxl_usbinfo  
> *usbinfo) 
> > +                             LIBXL_EXTERNAL_CALLERS_ONLY; 
> >  /* Network Interfaces */ 
> >  int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic  
> *nic, 
> >                           const libxl_asyncop_how *ao_how) 
> > diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h 
> > index 42eb1b9..f426ed8 100644 
> > --- a/tools/libxl/libxl_internal.h 
> > +++ b/tools/libxl/libxl_internal.h 
> > @@ -2422,6 +2422,13 @@ _hidden void libxl__device_vtpm_add(libxl__egc *egc,  
> uint32_t domid, 
> >                                     libxl_device_vtpm *vtpm, 
> >                                     libxl__ao_device *aodev); 
> >   
> > +/* from libxl_usb */ 
> > +_hidden int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid, 
> > +                                      libxl_device_usbctrl *usbctrl); 
> > +_hidden int libxl__device_usb_add(libxl__gc *gc, uint32_t domid, 
> > +                            libxl_device_usb *usb); 
> > +_hidden int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid); 
> > + 
> >  /* Internal function to connect a vkb device */ 
> >  _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid, 
> >                                    libxl_device_vkb *vkb); 
> > @@ -3628,7 +3635,9 @@ static inline void libxl__update_config_vtpm(libxl__gc  
> *gc, 
> >  #define COMPARE_PCI(a, b) ((a)->func == (b)->func &&    \ 
> >                             (a)->bus == (b)->bus &&      \ 
> >                             (a)->dev == (b)->dev) 
> > - 
> > +#define COMPARE_USB(a, b) (!strcmp((a)->busid, (b)->busid)) 
> > +#define COMPARE_USBCTRL(a, b) ((a)->devid == (b)->devid) 
> > +  
> >  /* DEVICE_ADD 
> >   * 
> >   * Add a device in libxl_domain_config structure 
> > diff --git a/tools/libxl/libxl_osdeps.h b/tools/libxl/libxl_osdeps.h 
> > index 08eaf0c..55caf71 100644 
> > --- a/tools/libxl/libxl_osdeps.h 
> > +++ b/tools/libxl/libxl_osdeps.h 
> > @@ -24,6 +24,8 @@ 
> >  #define _GNU_SOURCE 
> >   
> >  #if defined(__NetBSD__) 
> > +#define SYSFS_USB_DEV          "/sys/bus/usb/devices" 
> > +#define SYSFS_USBBACK_DRIVER   "/kern/xen/usb" 
> >  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices" 
> >  #define SYSFS_PCIBACK_DRIVER   "/kern/xen/pci" 
> >  #define NETBACK_NIC_NAME       "xvif%ui%d" 
> > @@ -31,6 +33,8 @@ 
> >  #elif defined(__OpenBSD__) 
> >  #include <util.h> 
> >  #elif defined(__linux__) 
> > +#define SYSFS_USB_DEV          "/sys/bus/usb/devices" 
> > +#define SYSFS_USBBACK_DRIVER   "/sys/bus/usb/drivers/usbback" 
> >  #define SYSFS_PCI_DEV          "/sys/bus/pci/devices" 
> >  #define SYSFS_PCIBACK_DRIVER   "/sys/bus/pci/drivers/pciback" 
> >  #define NETBACK_NIC_NAME       "vif%u.%d" 
> > @@ -38,12 +42,21 @@ 
> >  #elif defined(__sun__) 
> >  #include <stropts.h> 
> >  #elif defined(__FreeBSD__) 
> > +#define SYSFS_USB_DEV          "/dev/null" 
> > +#define SYSFS_USBBACK_DRIVER   "/dev/null" 
> >  #define SYSFS_PCI_DEV          "/dev/null" 
> >  #define SYSFS_PCIBACK_DRIVER   "/dev/null" 
> >  #define NETBACK_NIC_NAME       "xnb%u.%d" 
> >  #include <libutil.h> 
> >  #endif 
> >   
> > +#ifndef SYSFS_USBBACK_DRIVER 
> > +#error define SYSFS_USBBACK_DRIVER for your platform 
> > +#endif 
> > +#ifndef SYSFS_USB_DEV 
> > +#error define SYSFS_USB_DEV for your platform 
> > +#endif 
> > + 
> >  #ifndef SYSFS_PCIBACK_DRIVER 
> >  #error define SYSFS_PCIBACK_DRIVER for your platform 
> >  #endif 
> > diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c 
> > new file mode 100644 
> > index 0000000..4e4975a 
> > --- /dev/null 
> > +++ b/tools/libxl/libxl_pvusb.c 
> > @@ -0,0 +1,1201 @@ 
> > +/* 
> > + * This program is free software; you can redistribute it and/or modify 
> > + * it under the terms of the GNU Lesser General Public License as  
> published 
> > + * by the Free Software Foundation; version 2.1 only. with the special 
> > + * exception on linking described in file LICENSE. 
> > + * 
> > + * This program is distributed in the hope that it will be useful, 
> > + * but WITHOUT ANY WARRANTY; without even the implied warranty of 
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the 
> > + * GNU Lesser General Public License for more details. 
> > + */ 
> > + 
> > +#include "libxl_osdeps.h" /* must come before any other headers */ 
> > + 
> > +#include "libxl_internal.h" 
> > + 
> > +#define USBBACK_INFO_PATH "/libxl/usbback" 
> > + 
> > +#define USBHUB_CLASS_CODE 0x09 
> > + 
>  
> I'm not very familiar with how USB is supposed to work. Can you explain 
> why this particular value is chosen? 

0x09 is the USB class code for USB HUB defined in USB Spec. USB Class code
is  stored under Linux sysfs too. Since USB HUB could not be assigned to guest,
we need to check that.

>  
> > +static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid, 
> > +                                            libxl_device_usbctrl *usbctrl) 
> > +{ 
> > +    int rc; 
> > + 
> > +    if (!usbctrl->version) 
> > +        usbctrl->version = 2; 
> > + 
>  
> How hard would it be to implement USB 3? I assume this depends on QEMU's 
> support? I.e. we just need to specify the version to 3 and it should 
> just work? Just curious. 

This implementation is coworked with PVUSB  driver, so it depends on PVUSB
driver's support. If PVUSB driver supports that, just specify the version to 3 and
can work.

QEMU emulated USB device depends on QEMU's support.

>  
> > +    if (!usbctrl->ports) 
> > +        usbctrl->ports = 8; 
> > + 
> > +    rc = libxl__resolve_domid(gc, usbctrl->backend_domname, 
> > +                              &usbctrl->backend_domid); 
> > +    return rc; 
> > +} 
> > + 
> > +static void libxl__device_from_usbctrl(libxl__gc *gc, uint32_t domid, 
> > +                                       libxl_device_usbctrl *usbctrl, 
> > +                                       libxl__device *device) 
> > +{ 
> > +    device->backend_devid   = usbctrl->devid; 
> > +    device->backend_domid   = usbctrl->backend_domid; 
> > +    device->backend_kind    = LIBXL__DEVICE_KIND_VUSB; 
> > +    device->devid           = usbctrl->devid; 
> > +    device->domid           = domid; 
> > +    device->kind            = LIBXL__DEVICE_KIND_VUSB; 
> > +} 
> > + 
> > +static int libxl__usbport_add_xenstore(libxl__gc *gc, 
> > +                                       xs_transaction_t tran, 
> > +                                       uint32_t domid, 
> > +                                       libxl_device_usbctrl *usbctrl) 
> > +{ 
> > +    char *path; 
> > +    int i; 
> > + 
> > +    path = GCSPRINTF("%s/backend/vusb/%d/%d/port", 
> > +                     libxl__xs_get_dompath(gc, 0), domid, usbctrl->devid); 
> > + 
> > +    libxl__xs_mkdir(gc, tran, path, NULL, 0); 
> > + 
> > +    for (i = 1; i <= usbctrl->ports; i++) { 
> > +        if (libxl__xs_write_checked(gc, tran, GCSPRINTF("%s/%d", path, i),  
> "")) 
> > +            return ERROR_FAIL; 
> > +    } 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +static int libxl__usbctrl_add_xenstore(libxl__gc *gc, uint32_t domid, 
> > +                                       libxl_device_usbctrl *usbctrl) 
> > +{ 
> > +    flexarray_t *front; 
> > +    flexarray_t *back; 
> > +    libxl__device *device; 
> > +    xs_transaction_t t = XBT_NULL; 
> > +    int rc = 0; 
> > +    libxl_domain_config d_config; 
> > +    libxl_device_usbctrl usbctrl_saved; 
> > +    libxl__domain_userdata_lock *lock = NULL; 
> > + 
> > +    libxl_domain_config_init(&d_config); 
> > +    libxl_device_usbctrl_init(&usbctrl_saved); 
> > +    libxl_device_usbctrl_copy(CTX, &usbctrl_saved, usbctrl); 
> > + 
> > +    GCNEW(device); 
> > +    libxl__device_from_usbctrl(gc, domid, usbctrl, device); 
> > + 
> > +    front = flexarray_make(gc, 4, 1); 
> > +    back = flexarray_make(gc, 12, 1); 
> > + 
> > +    flexarray_append_pair(back, "frontend-id", GCSPRINTF("%d", domid)); 
> > +    flexarray_append_pair(back, "online", "1"); 
> > +    flexarray_append_pair(back, "state", "1"); 
> > +    flexarray_append_pair(back, "usb-ver", GCSPRINTF("%d",  
> usbctrl->version)); 
> > +    flexarray_append_pair(back, "num-ports", GCSPRINTF("%d",  
> usbctrl->ports)); 
> > +    flexarray_append_pair(front, "backend-id", GCSPRINTF("%d",  
> usbctrl->backend_domid)); 
>  
> Line too long. 
>  
> > +    flexarray_append_pair(front, "state", "1"); 
> > + 
> > +    lock = libxl__lock_domain_userdata(gc, domid); 
> > +    if (!lock) { 
> > +        rc = ERROR_LOCK_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    rc = libxl__get_domain_configuration(gc, domid, &d_config); 
> > +    if (rc) goto out; 
> > + 
> > +    DEVICE_ADD(usbctrl, usbctrls, domid, &usbctrl_saved, COMPARE_USBCTRL,  
> &d_config); 
> > + 
>  
> Line too long. 
>  
> > +    for (;;) { 
> > +        rc = libxl__xs_transaction_start(gc, &t); 
> > +        if (rc) goto out; 
> > + 
> > +        rc = libxl__device_exists(gc, t, device); 
> > +        if (rc < 0) goto out; 
> > +        if (rc == 1) { 
> > +            /* already exists in xenstore */ 
> > +            LOG(ERROR, "device already exists in xenstore"); 
> > +            rc = ERROR_DEVICE_EXISTS; 
> > +            goto out; 
> > +        } 
> > + 
> > +        rc = libxl__set_domain_configuration(gc, domid, &d_config); 
> > +        if (rc) goto out; 
> > + 
> > +        libxl__device_generic_add(gc, t, device, 
> > +                                  libxl__xs_kvs_of_flexarray(gc, back, 
> > +                                                             back->count), 
> > +                                  libxl__xs_kvs_of_flexarray(gc, front, 
> > +                                                             front->count), 
> > +                                  NULL); 
> > +        libxl__usbport_add_xenstore(gc, t, domid, usbctrl); 
> > +        rc = libxl__xs_transaction_commit(gc, &t); 
> > +        if (!rc) break; 
> > +        if (rc < 0) goto out; 
> > +    } 
> > + 
>  
> You don't have aodev so you cannot check update_json. Maybe you need 
> aodev? 
>  
> That field update_json is set to true when doing hotplug. It's set to 
> false during domain creation. 
>  
> The same comment applies to other add functions as well.

Thanks, I'm updating that. But maybe like pci_add and pci_remove functions,
will add a 'starting' flag to indicate hotplug or creation.
Looking at DEFINE_DEVICE_ADD and DEFINE_DEVICE_REMOVE, usbctrl_add
and usb_add can use DEFINE_DEVICE_ADD; but usbctrl_remove and usb_remove
cannot use DEFINE_DEVICE_REMOVE directly, need some extra handling. So,
finally turned to not use these macros but refer to pci functions.

>  
> > +out: 
> > +    if (lock) libxl__unlock_domain_userdata(lock); 
> > +    libxl_device_usbctrl_dispose(&usbctrl_saved); 
> > +    libxl_domain_config_dispose(&d_config); 
> > +    return rc; 
> > +} 
> > + 
> > +int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid, 
> > +                              libxl_device_usbctrl *usbctrl) 
> > +{ 
> > +    int rc = 0; 
> > + 
> > +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl); 
> > +    if (rc < 0) goto out; 
> > + 
> > +    if (usbctrl->devid == -1) { 
> > +        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb"); 
> > +        if (usbctrl->devid < 0) { 
> > +            rc = ERROR_FAIL; 
> > +            goto out; 
> > +        } 
> > +    } 
> > + 
> > +    if (libxl__usbctrl_add_xenstore(gc, domid, usbctrl) < 0){ 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +out: 
> > +    return rc; 
> > +} 
> > + 
> > +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid, 
> > +                             libxl_device_usbctrl *usbctrl, 
> > +                             const libxl_asyncop_how *ao_how) 
> > +{ 
> > +    AO_CREATE(ctx, domid, ao_how); 
> > +    int rc; 
> > + 
> > +    rc = libxl__device_usbctrl_add(gc, domid, usbctrl); 
>  
> Hmm... Your remove function is async while this one is sync, why?

Hmm, maybe not proper here, just referred to pci_add implementation.
Current calling places only use sync mode.
 
>  
> > +    libxl__ao_complete(egc, ao, rc); 
> > +    return AO_INPROGRESS; 
> > +} 
> > + 
> > +static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int  
> usbctrl, 
> > +                                  libxl_device_usb **usbs, int *num); 
> > + 
> > +static int do_usb_remove(libxl__gc *gc, uint32_t domid, 
> > +                         libxl_device_usb *usb); 
> > + 
> > +static int 
> > +libxl__device_usbctrl_remove_common(libxl_ctx *ctx, uint32_t domid, 
> > +                                    libxl_device_usbctrl *usbctrl, 
> > +                                    const libxl_asyncop_how *ao_how, 
> > +                                    int force) 
> > +{ 
> > +    AO_CREATE(ctx, domid, ao_how); 
> > +    libxl__device *device; 
> > +    libxl__ao_device *aodev; 
> > +    libxl_device_usbctrl *usbctrls = NULL; 
> > +    libxl_device_usbctrl *usbctrl_find = NULL; 
> > +    int numctrl = 0; 
> > +    libxl_device_usb *usbs = NULL; 
> > +    int numusb = 0; 
> > +    int i, rc; 
> > + 
> > +    assert(usbctrl->devid != -1); 
> > + 
> > +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl); 
> > +    if (!numctrl) { 
> > +        LOG(ERROR, "No USB controller exists in this domain"); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    for (i = 0; i < numctrl; i++) { 
> > +        if (usbctrl->devid == usbctrls[i].devid) { 
> > +            usbctrl_find = usbctrls + i; 
> > +            break; 
> > +        } 
> > +    } 
> > + 
> > +    if (!usbctrl_find) { 
> > +        LOG(ERROR, "USB controller %d is not attached to this domain", 
> > +            usbctrl->devid); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    usbctrl = usbctrl_find; 
> > + 
> > +    GCNEW(device); 
> > +    libxl__device_from_usbctrl(gc, domid, usbctrl, device); 
> > + 
> > +    /* Remove usb devives first */ 
> > +    rc  = libxl__device_usb_list(gc, domid, usbctrl->devid, &usbs, &numusb); 
> > +    if (rc) goto out; 
> > +    for (i = 0; i < numusb; i++) { 
> > +        if (do_usb_remove(gc, domid, &usbs[i])) { 
> > +            LOG(ERROR, "do_usb_remove failed"); 
> > +            rc = ERROR_FAIL; 
> > +            goto out; 
> > +        } 
> > +    } 
> > +    /* remove usbctrl */ 
> > +    GCNEW(aodev); 
> > +    libxl__prepare_ao_device(ao, aodev); 
> > +    aodev->action = LIBXL__DEVICE_ACTION_REMOVE; 
> > +    aodev->dev = device; 
> > +    aodev->callback = device_addrm_aocomplete; 
> > +    aodev->force = force; 
> > +    libxl__initiate_device_remove(egc, aodev); 
> > + 
> > +out: 
> > +    free(usbctrls); 
> > +    free(usbs); 
> > +    if(rc) return AO_ABORT(rc); 
> > +    return AO_INPROGRESS; 
> > +} 
> > + 
> > +int libxl_device_usbctrl_remove(libxl_ctx *ctx, uint32_t domid, 
> > +                                libxl_device_usbctrl *usbctrl, 
> > +                                const libxl_asyncop_how *ao_how) 
> > +{ 
> > +    return libxl__device_usbctrl_remove_common(ctx, domid, usbctrl,  
> ao_how, 0); 
> > +} 
> > + 
> > +int libxl_device_usbctrl_destroy(libxl_ctx *ctx, uint32_t domid, 
> > +                                 libxl_device_usbctrl *usbctrl, 
> > +                                 const libxl_asyncop_how *ao_how) 
> > +{ 
> > +    return libxl__device_usbctrl_remove_common(ctx, domid, usbctrl,  
> ao_how, 1); 
> > +} 
> > + 
> > +libxl_device_usbctrl * 
> > +libxl_device_usbctrl_list(libxl_ctx *ctx, uint32_t domid, int *num) 
> > +{ 
> > +    GC_INIT(ctx); 
> > +    libxl_device_usbctrl *usbctrls = NULL; 
> > +    char *fe_path = NULL; 
> > +    char **dir = NULL; 
> > +    unsigned int ndirs = 0; 
> > + 
> > +    *num = 0; 
> > + 
> > +    fe_path = GCSPRINTF("%s/device/vusb", 
> > +                        libxl__xs_get_dompath(gc, domid)); 
> > +    dir = libxl__xs_directory(gc, XBT_NULL, fe_path, &ndirs); 
> > + 
> > +    if (dir && ndirs) { 
> > +        usbctrls = malloc(sizeof(*usbctrls) * ndirs); 
> > +        libxl_device_usbctrl* usbctrl; 
> > +        libxl_device_usbctrl* end = usbctrls + ndirs; 
> > +        for(usbctrl = usbctrls; usbctrl < end; usbctrl++, dir++, (*num)++)  
> { 
> > +            char *tmp; 
> > +            const char *be_path = libxl__xs_read(gc, XBT_NULL, 
> > +                                    GCSPRINTF("%s/%s/backend", fe_path,  
> *dir)); 
> > + 
> > +            libxl_device_usbctrl_init(usbctrl); 
> > +            usbctrl->devid = atoi(*dir); 
> > + 
> > +            tmp = libxl__xs_read(gc, XBT_NULL, 
> > +                                 GCSPRINTF("%s/%s/backend-id", fe_path,  
> *dir)); 
> > +            if (!tmp) goto outerr; 
> > +            usbctrl->backend_domid = atoi(tmp); 
> > + 
> > +            tmp = libxl__xs_read(gc, XBT_NULL, 
> > +                                 GCSPRINTF("%s/usb-ver", be_path)); 
> > +            if (!tmp) goto outerr; 
> > +            usbctrl->version = atoi(tmp); 
> > + 
> > +            tmp = libxl__xs_read(gc, XBT_NULL, 
> > +                                 GCSPRINTF("%s/num-ports", be_path)); 
> > +            if (!tmp) goto outerr; 
> > +            usbctrl->ports = atoi(tmp); 
> > +       } 
> > +    } 
> > + 
> > +    return usbctrls; 
> > + 
> > +outerr: 
> > +    LOG(ERROR, "Unable to list USB Controllers"); 
> > +    for (int i = 0; i < *num; i++) { 
> > +        libxl_device_usbctrl_dispose(usbctrls + i); 
> > +    } 
> > +    free(usbctrls); 
>  
> It might be useful to provide a function to free the whole list of 
> device. There are similar functions in libxl. 

Yeah, that would be useful.

>  
> > +    *num = 0; 
> > +    return NULL; 
> > +} 
> > + 
> > +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid, 
> > +                                libxl_device_usbctrl *usbctrl, 
> > +                                libxl_usbctrlinfo *usbctrlinfo) 
> > +{ 
> > +    GC_INIT(ctx); 
> > +    char *dompath, *usbctrlpath; 
> > +    char *val; 
> > +    int rc = 0; 
> > + 
> > +    usbctrlinfo->devid = usbctrl->devid; 
> > +    usbctrlinfo->ports = usbctrl->ports; 
> > +    usbctrlinfo->version = usbctrl->version; 
> > + 
> > +    dompath = libxl__xs_get_dompath(gc, domid); 
> > +    usbctrlpath = GCSPRINTF("%s/device/vusb/%d", dompath,  
> usbctrlinfo->devid); 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/backend", usbctrlpath)); 
> > +    usbctrlinfo->backend = libxl__strdup(NOGC, val); 
> > +    if (!usbctrlinfo->backend) { 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/backend-id", usbctrlpath)); 
> > +    usbctrlinfo->backend_id = val ? strtoul(val, NULL, 10) : -1; 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/state", usbctrlpath)); 
> > +    usbctrlinfo->state = val ? strtoul(val, NULL, 10) : -1; 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/event-channel", usbctrlpath)); 
> > +    usbctrlinfo->evtch = val ? strtoul(val, NULL, 10) : -1; 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/urb-ring-ref", usbctrlpath)); 
> > +    usbctrlinfo->ref_urb = val ? strtoul(val, NULL, 10) : -1; 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/conn-ring-ref", usbctrlpath)); 
> > +    usbctrlinfo->ref_conn= val ? strtoul(val, NULL, 10) : -1; 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/frontend", usbctrlinfo->backend)); 
> > +    usbctrlinfo->frontend = libxl__strdup(NOGC, val); 
> > + 
> > +    val = libxl__xs_read(gc, XBT_NULL, 
> > +                         GCSPRINTF("%s/frontend-id", usbctrlinfo->backend)); 
> > +    usbctrlinfo->frontend_id = val ? strtoul(val, NULL, 10) : -1; 
> > + 
> > +out: 
> > +    GC_FREE; 
> > +    return rc; 
> > +} 
> > + 
> > +/* usb device functions */ 
> > + 
> > +static int 
> > +libxl__device_usb_assigned_list(libxl__gc *gc, 
> > +                                libxl_device_usb **list, int *num) 
> > +{ 
> > +    char **domlist; 
> > +    unsigned int nd = 0, i, j; 
> > +    char *be_path; 
> > +    libxl_device_usb *usb; 
> > + 
> > +    *list = NULL; 
> > +    *num = 0; 
> > + 
> > +    domlist = libxl__xs_directory(gc, XBT_NULL, "/local/domain", &nd); 
> > +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb",  
> LIBXL_TOOLSTACK_DOMID); 
> > +    for (i = 0; i < nd; i++) { 
> > +        char *path, *num_ports, **ctrl_list; 
> > +        unsigned int nc = 0; 
> > +        path = GCSPRINTF("%s/%s", be_path, domlist[i]); 
> > +        ctrl_list = libxl__xs_directory(gc, XBT_NULL, path , &nc); 
> > + 
> > +        for (j = 0; j < nc; j++) { 
> > +            path = GCSPRINTF("%s/%s/%s/num-ports", be_path, 
> > +                             domlist[i], ctrl_list[j]); 
> > +            num_ports = libxl__xs_read(gc, XBT_NULL, path); 
> > +            if (num_ports) { 
> > +                int nport = atoi(num_ports), k; 
> > +                char *devpath, *busid; 
> > + 
> > +                for (k = 1; k <= nport; k++) { 
> > +                    devpath = GCSPRINTF("%s/%s/%s/port/%u", be_path, 
> > +                                        domlist[i], ctrl_list[j], k); 
> > +                    busid = libxl__xs_read(gc, XBT_NULL, devpath); 
> > +                    /* If there are USB device attached, add it to list */ 
> > +                    if (busid && strcmp(busid, "") ) { 
> > +                        *list = realloc(*list, 
> > +                                  sizeof(libxl_device_usb) * ((*num) +  
> 1)); 
> > +                        if (*list == NULL) 
> > +                            return ERROR_NOMEM; 
>  
> Use GCREALLOC_ARRAY. 

Thanks, will update.

>  
> > +                        usb = *list + *num; 
> > +                        usb->ctrl = atoi(ctrl_list[j]); 
> > +                        usb->port = k; 
> > +                        usb->busid = strdup(busid); 
> > +                        (*num)++; 
> > +                    } 
> > +                } 
> > +            } 
> > +        } 
> > +    } 
> > +    libxl__ptr_add(gc, *list); 
> > + 
>  
> No need to do this if you use GCREALLOC_ARRAY. 
>  
> > +    return 0; 
> > +} 
> > + 
> > +static bool is_usb_in_array(libxl_device_usb *usbs, int num, 
> > +                            libxl_device_usb *usb) 
> > +{ 
> > +    int i; 
> > + 
> > +    for (i = 0; i < num; i++) { 
> > +        if (!strcmp(usbs[i].busid, usb->busid) ) 
> > +            return true; 
> > +    } 
> > + 
> > +    return false; 
> > +} 
> > + 
> > +/* check if USB device is already assigned to a domain */ 
> > +static bool is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> > +    libxl_device_usb *usbs; 
> > +    int rc, num; 
> > + 
> > +    rc = libxl__device_usb_assigned_list(gc, &usbs, &num); 
> > +    if (rc) { 
> > +        LOG(ERROR, "Fail to get assigned usb list"); 
> > +        return true; 
> > +    } 
> > + 
> > +    if (is_usb_in_array(usbs, num, usb)) 
> > +        return true; 
> > + 
> > +    return false; 
> > +} 
> > + 
> > +/* check if USB device type is assignable */ 
> > +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> > +    libxl_ctx *ctx = libxl__gc_owner(gc); 
> > +    int classcode; 
> > +    char *filename; 
> > +    void *buf; 
> > + 
> > +    assert(usb->busid); 
> > + 
> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid); 
> > +    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0) 
> > +        return false; 
> > + 
> > +    sscanf(buf, "%x", &classcode); 
> > +    if (classcode == USBHUB_CLASS_CODE) 
> > +        return false; 
> > + 
> > +    return true; 
> > +} 
> > + 
> > +/* get usb devices under certain usb controller */ 
> > +static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int  
> usbctrl, 
> > +                                  libxl_device_usb **usbs, int *num) 
> > +{ 
> > +    char *be_path, *num_devs; 
> > +    int n, i; 
> > + 
> > +    *usbs = NULL; 
> > +    *num = 0; 
> > + 
> > +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d", 
> > +                        libxl__xs_get_dompath(gc, 0), domid, usbctrl); 
> > +    num_devs = libxl__xs_read(gc, XBT_NULL, 
> > +                              GCSPRINTF("%s/num-ports", be_path)); 
> > +    if (!num_devs) 
> > +        return 0; 
> > + 
> > +    n = atoi(num_devs); 
> > +    *usbs = calloc(n, sizeof(libxl_device_usb)); 
> > + 
>  
> libxl__calloc(NOGC, ...) 
Will update.
>  
> > +    for (i = 0; i < n; i++) { 
> > +        char *busid; 
> > +        libxl_device_usb *usb = NULL; 
> > + 
> > +        busid = libxl__xs_read(gc, XBT_NULL, 
> > +                               GCSPRINTF("%s/port/%d", be_path, i + 1)); 
> > +        if (busid && strcmp(busid, "")) { 
> > +            usb = *usbs + *num; 
> > +            usb->ctrl = usbctrl; 
> > +            usb->port = i + 1; 
> > +            usb->busid = strdup(busid); 
> > +            (*num)++; 
> > +        } 
> > +    } 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +/* get all usb devices of the domain */ 
> > +static libxl_device_usb * 
> > +libxl_device_usb_list_all(libxl__gc *gc, uint32_t domid, int *num) 
> > +{ 
> > +    char **usbctrls; 
> > +    unsigned int nd, i, j; 
> > +    char *be_path; 
> > +    int rc; 
> > +    libxl_device_usb *usbs = NULL; 
> > + 
> > +    *num = 0; 
> > + 
> > +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb/%d", 
> > +                        LIBXL_TOOLSTACK_DOMID, domid); 
> > +    usbctrls = libxl__xs_directory(gc, XBT_NULL, be_path, &nd); 
> > + 
> > +    for (i = 0; i < nd; i++) { 
> > +        int nc = 0; 
> > +        libxl_device_usb *tmp = NULL; 
> > +        rc = libxl__device_usb_list(gc, domid, atoi(usbctrls[i]), &tmp,  
> &nc); 
> > +        if (!nc) continue; 
> > + 
> > +        usbs = realloc(usbs, sizeof(libxl_device_usb)*((*num) + nc)); 
>  
> libxl__realloc 
>  
> > +        for(j = 0; j < nc; j++) { 
> > +            usbs[*num].ctrl = tmp[j].ctrl; 
> > +            usbs[*num].port = tmp[j].port; 
> > +            usbs[*num].busid = strdup(tmp[j].busid); 
> > +            (*num)++; 
> > +        } 
> > +        free(tmp); 
> > +    } 
> > +    return usbs; 
> > +} 
> > + 
> > +libxl_device_usb *libxl_device_usb_list(libxl_ctx *ctx, uint32_t domid, 
> > +                                        int usbctrl, int *num) 
> > +{ 
> > +    GC_INIT(ctx); 
> > +    libxl_device_usb *usbs = NULL; 
> > + 
> > +    libxl__device_usb_list(gc, domid, usbctrl, &usbs, num); 
> > + 
> > +    GC_FREE; 
> > +    return usbs; 
> > +} 
> > + 
> > +/* set default value */ 
> > +static char *usb_busaddr_to_busid(libxl__gc *gc, int bus, int addr) 
> > +{ 
> > +    libxl_ctx *ctx = libxl__gc_owner(gc); 
> > +    struct dirent *de; 
> > +    DIR *dir; 
> > +    char *busid = NULL; 
> > + 
> > +    if (bus < 1 || addr < 1) 
> > +        return NULL; 
> > + 
> > +    if (!(dir = opendir(SYSFS_USB_DEV))) 
> > +        return NULL; 
> > + 
> > +    while((de = readdir(dir))) { 
> > +        char *filename; 
> > +        void *buf; 
> > +        int busnum = -1; 
> > +        int devnum = -1; 
> > + 
> > +        if (!de->d_name) 
> > +            continue; 
> > + 
> > +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/devnum", de->d_name); 
> > +        if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) 
> > +            sscanf(buf, "%x", &devnum); 
> > + 
> > +        filename = GCSPRINTF(SYSFS_USB_DEV"/%s/busnum", de->d_name); 
> > +        if (!libxl_read_file_contents(ctx, filename, &buf, NULL)) 
> > +            sscanf(buf, "%x", &busnum); 
> > + 
> > +        if (bus == busnum && addr == devnum) { 
> > +            busid = strdup(de->d_name); 
> > +            break; 
> > +        } 
> > +    } 
> > + 
> > +    closedir(dir); 
> > +    return busid; 
> > +} 
> > + 
> > +/* find first unused controller:port and give that to usb device */ 
> > +static int 
> > +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid, 
> > +                                      libxl_device_usb *usb) 
> > +{ 
> > +    libxl_ctx *ctx = CTX; 
> > +    libxl_device_usbctrl *usbctrls; 
> > +    libxl_device_usb *usbs = NULL; 
> > +    int numctrl, numusb, i, j, rc = -1; 
> > +    char *be_path, *tmp; 
> > + 
> > +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl); 
> > +    if ( !numctrl) 
> > +        goto out; 
> > + 
> > +    for (i = 0; i < numctrl; i++) { 
> > +        rc = libxl__device_usb_list(gc, domid, usbctrls[i].devid, 
> > +                                    &usbs, &numusb); 
> > +        if (rc) continue; 
> > + 
> > +        if (!usbctrls[i].ports || numusb == usbctrls[i].ports) 
> > +            continue; 
> > + 
> > +        for (j = 1; i <= numusb; j++) { 
> > +            be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> > +                                libxl__xs_get_dompath(gc, 0), domid, 
> > +                                usbctrls[i].devid, j); 
> > +            tmp = libxl__xs_read(gc, XBT_NULL, be_path); 
> > +            if (tmp && !strcmp( tmp, "")) { 
> > +                usb->ctrl = usbctrls[i].devid; 
> > +                usb->port = j; 
> > +                break; 
> > +            } 
> > +        } 
> > +    } 
> > + 
> > +    rc = 0; 
> > + 
> > +out: 
> > +    if (usbctrls) 
> > +        free(usbctrls); 
> > +    if (usbs) 
> > +        free(usbs); 
> > +    return rc; 
> > +} 
> > + 
> > +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid, 
> > +                                        libxl_device_usb *usb) 
> > +{ 
> > +    char *be_path, *tmp; 
> > + 
> > +    if (usb->ctrl == -1) { 
> > +        int ret = libxl__device_usb_set_default_usbctrl(gc, domid, usb); 
> > +        /* If no existing ctrl to host this usb device, setup a new one */ 
> > +        if (ret) { 
> > +            libxl_device_usbctrl usbctrl; 
> > +            libxl_device_usbctrl_init(&usbctrl); 
> > +            libxl__device_usbctrl_add(gc, domid, &usbctrl); 
> > +            usb->ctrl = usbctrl.devid; 
> > +            usb->port = 1; 
> > +            libxl_device_usbctrl_dispose(&usbctrl); 
> > +        } 
> > +    } 
> > + 
> > +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> > +                        libxl__xs_get_dompath(gc, 0), 
> > +                        domid, usb->ctrl, usb->port); 
> > +    tmp = libxl__xs_read(gc, XBT_NULL, be_path); 
> > +    if (!tmp || strcmp(tmp, "") ){ 
> > +        LOG(ERROR, "The controller port isn't available"); 
> > +        return ERROR_INVAL; 
> > +    } 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +/* xenstore usb data */ 
> > +static int libxl__device_usb_add_xenstore(libxl__gc *gc, uint32_t domid, 
> > +                                          libxl_device_usb *usb) 
> > +{ 
> > +    char *be_path; 
> > +    int rc; 
> > +    libxl_domain_config d_config; 
> > +    libxl_device_usb usb_saved; 
> > +    libxl__domain_userdata_lock *lock = NULL; 
> > + 
> > +    libxl_domain_config_init(&d_config); 
> > +    libxl_device_usb_init(&usb_saved); 
> > +    libxl_device_usb_copy(CTX, &usb_saved, usb); 
> > + 
> > +    lock = libxl__lock_domain_userdata(gc, domid); 
> > +    if (!lock) { 
> > +        rc = ERROR_LOCK_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    rc = libxl__get_domain_configuration(gc, domid, &d_config); 
> > +    if (rc) goto out; 
> > + 
> > +    DEVICE_ADD(usb, usbs, domid, &usb_saved, COMPARE_USB, &d_config); 
> > + 
> > +    rc = libxl__set_domain_configuration(gc, domid, &d_config); 
> > +    if (rc) goto out; 
> > + 
> > +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> > +                  libxl__xs_get_dompath(gc, 0), domid, usb->ctrl, usb->port); 
> > +    LOG(DEBUG, "Adding new usb device to xenstore"); 
> > +    if (libxl__xs_write_checked(gc, XBT_NULL, be_path, usb->busid)) { 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
>  
> This is not right. See libxl_internal.h, search for "lock json 
> config" to see the pattern. In short, you should use a xenstore 
> transaction instead of libxl__xs_write_checked with XBT_NULL. 
>  
> Also you need to use aodev->update_json. 
>  
> > +    rc = 0; 
> > + 
> > +out: 
> > +    if (lock) libxl__unlock_domain_userdata(lock); 
> > +    libxl_device_usb_dispose(&usb_saved); 
> > +    libxl_domain_config_dispose(&d_config); 
> > +    return rc; 
> > +} 
> > + 
> > +static int libxl__device_usb_remove_xenstore(libxl__gc *gc, uint32_t  
> domid, 
> > +                                             libxl_device_usb *usb) 
> > +{ 
> > +    char *be_path; 
> > + 
> > +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d/port/%d", 
> > +                        libxl__xs_get_dompath(gc, 0), 
> > +                        domid, usb->ctrl, usb->port); 
> > +    LOG(DEBUG, "Removing USB device from xenstore"); 
> > +    if (libxl__xs_write_checked(gc,XBT_NULL, be_path, "")) 
> > +        return ERROR_FAIL; 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +/* bind/unbind usb device interface */ 
> > +static int unbind_usb_intf(libxl__gc *gc, char *intf, char **drvpath) 
> > +{ 
> > +    char *path, *spath, *dp = NULL; 
> > +    int fd = -1; 
> > +    int rc = 0; 
> > +    struct stat st; 
> > + 
> > +    spath = GCSPRINTF(SYSFS_USB_DEV"/%s/driver", intf); 
> > +    if (!lstat(spath, &st)) { 
> > +        /* Find the canonical path to the driver. */ 
> > +        dp = libxl__zalloc(gc, PATH_MAX); 
> > +        dp = realpath(spath, dp); 
> > + 
> > +        path = GCSPRINTF("%s/unbind", spath); 
> > +        fd = open(path, O_WRONLY); 
> > +        if (fd < 0) 
> > +            return ERROR_FAIL; 
> > +        rc = write(fd, intf, strlen(intf)); 
> > +        close(fd); 
> > +        if (rc < 0) 
> > +            return ERROR_FAIL; 
> > +    } 
> > + 
> > +    if (drvpath) 
> > +        *drvpath = dp; 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +static int bind_usb_intf(libxl__gc *gc, char *intf, char *drvpath) 
> > +{ 
> > +    char *path; 
> > +    struct stat st; 
> > +    int fd, rc = 0; 
> > + 
> > +    path = GCSPRINTF("%s/%s", drvpath, intf); 
> > +    rc = lstat(path, &st); 
> > +    /* already bind, return */ 
> > +    if(rc == 0) 
> > +        return 0; 
> > + 
> > +    path = GCSPRINTF("%s/bind", drvpath); 
> > +    fd = open(path, O_WRONLY); 
> > +    if (fd < 0) 
> > +        return ERROR_FAIL; 
> > + 
> > +    rc = write(fd, intf, strlen(intf)); 
> > +    close(fd); 
> > +    if (rc < 0) 
> > +        return ERROR_FAIL; 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +/* Is usb interface bound to usbback? */ 
> > +static int usb_intf_is_assigned(libxl__gc *gc, char *intf) 
> > +{ 
> > +    char *spath; 
> > +    int rc; 
> > +    struct stat st; 
> > + 
> > +    spath = GCSPRINTF(SYSFS_USBBACK_DRIVER"/%s", intf); 
> > +    rc = lstat(spath, &st); 
> > + 
> > +    if(rc == 0) 
> > +        return 1; 
> > +    if (rc < 0 && errno == ENOENT) 
> > +        return 0; 
> > +    LOGE(ERROR, "Accessing %s", spath); 
> > +    return -1; 
> > +} 
> > + 
> > +static int usb_get_all_interfaces(libxl__gc *gc, libxl_device_usb *usb, 
> > +                                  char ***intfs, int *num) 
> > +{ 
> > +    DIR *dir; 
> > +    struct dirent *entry; 
> > +    char *buf; 
> > +    int rc = 0; 
> > + 
> > +    *intfs = NULL; 
> > +    *num = 0; 
> > + 
> > +    buf = GCSPRINTF("%s:", usb->busid); 
> > + 
> > +    if (!(dir = opendir(SYSFS_USB_DEV))) { 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    while ((entry = readdir(dir)) != NULL) { 
> > +        if (!strncmp(entry->d_name, buf, strlen(buf))){ 
> > +            *intfs = realloc(*intfs, sizeof(char *) * (*num + 1)); 
> > +            if (*intfs == NULL) { 
> > +                rc = ERROR_FAIL; 
> > +                goto out; 
> > +            } 
> > +            (*intfs)[*num] = strdup(entry->d_name); 
> > +            (*num)++; 
> > +        } 
> > +    } 
> > + 
> > +    closedir(dir); 
> > + 
> > +out: 
> > +    return rc; 
> > +} 
> > + 
> > +/* Cann't write '.' or ':' into Xenstore as key. So, change '.' to '_', 
>  
> "Can't" 
>  
> And this behaviour needs to be documented. 
>  
> > + * change ':' to '-'. 
> > + */ 
> > +static char *usb_interface_encode(char *busid) 
> > +{ 
> > +    char *str = strdup(busid); 
> > +    int i, len = strlen(str); 
> > + 
> > +    for (i = 0; i < len; i++) { 
> > +        if (str[i] == '.') 
> > +            str[i] = '_'; 
> > +         if (str[i] == ':') 
>  
> Indentation. 
>  
> > +            str[i] = '-'; 
> > +    } 
> > +    return str; 
> > +} 
> > + 
> > +/* unbind usb device from usbback driver, if there are many interfaces 
> > + * under the usb device, then check each interface, unbind from usbback 
> > + * driver and rebind to original driver 
> > + */ 
> > +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> > +    char **intfs = NULL; 
> > +    char *path; 
> > +    int num = 0, i; 
> > +    int rc = 0; 
> > +    char *usb_encode = NULL; 
> > + 
> > +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0) 
> > +        return ERROR_FAIL; 
> > + 
> > +    usb_encode = usb_interface_encode(usb->busid); 
> > + 
> > +    for (i = 0; i < num; i++){ 
> > +        char *intf = intfs[i]; 
> > +        char *drvpath = NULL; 
> > + 
> > +        if (usb_intf_is_assigned(gc, intf) > 0) { 
> > +            /* unbind interface from usbback driver */ 
> > +            if (unbind_usb_intf(gc, intf, NULL) < 0) { 
> > +                rc = ERROR_FAIL; 
> > +                goto out; 
> > +            } 
> > +        } 
> > + 
> > +        /* bind interface to its originial driver */ 
> > +        drvpath = libxl__xs_read(gc, XBT_NULL, 
> > +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path", 
> > +                  usb_encode, usb_interface_encode(intf))); 
> > +        if (drvpath) { 
> > +            if (bind_usb_intf(gc, intf, drvpath) < 0) { 
> > +                LOGE(ERROR, "Couldn't bind %s to %s", intf, drvpath); 
> > +                rc = ERROR_FAIL; 
> > +                goto out; 
> > +            } 
> > +        } 
> > +    } 
> > + 
> > +    /* finally, remove xs driver path */ 
> > +    path = GCSPRINTF(USBBACK_INFO_PATH"/%s", usb_encode); 
> > +    libxl__xs_rm_checked(gc, XBT_NULL, path); 
> > + 
> > +out: 
> > +    if (intfs) { 
> > +        for (i = 0; i < num; i++) 
> > +            free(intfs[i]); 
> > +        free(intfs); 
> > +    } 
> > +    free(usb_encode); 
> > +    return rc; 
> > +} 
> > + 
> > +/* bind usb device to "usbback" driver, if there are many interfaces 
> > + * under the usb device, check each interface, unbind from original 
> > + * driver and bind to usbback driver. 
> > + */ 
> > +static int usbback_dev_assign(libxl__gc *gc, libxl_device_usb *usb) 
> > +{ 
> > +    char **intfs = NULL; 
> > +    int num = 0, i; 
> > +    int rc = 0; 
> > +    char *usb_encode = NULL; 
> > + 
> > +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0) 
> > +        return ERROR_FAIL; 
> > + 
> > +    usb_encode = usb_interface_encode(usb->busid); 
> > + 
> > +    for (i = 0; i < num; i++){ 
> > +        char *intf = intfs[i]; 
> > +        char *path = NULL; 
> > +        char *drvpath = NULL; 
> > + 
> > +        /* already assigned to usbback */ 
> > +        if (usb_intf_is_assigned(gc, intf) > 0) 
> > +            continue; 
> > + 
> > +        /* unbind interface from original driver */ 
> > +        if (unbind_usb_intf(gc, intf, &drvpath) < 0) { 
> > +            rc = ERROR_FAIL; 
> > +            goto out_rebind; 
> > +        } 
> > + 
> > +        if (drvpath) { 
> > +            /* write driver path to xenstore for later rebinding */ 
> > +            path = GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path", 
> > +                             usb_encode, usb_interface_encode(intf)); 
> > +            if (libxl__xs_write_checked(gc, XBT_NULL, path, drvpath) < 0) { 
> > +                LOG(WARN, "Write of %s to node %s failed", drvpath, path); 
> > +            } 
> > +        } 
> > + 
> > +        /* bind interface to usbback */ 
> > +        if (bind_usb_intf(gc, intf, SYSFS_USBBACK_DRIVER) < 0){ 
> > +            LOGE(ERROR, "Couldn't bind %s to %s", intf,  
> SYSFS_USBBACK_DRIVER); 
> > +            rc = ERROR_FAIL; 
> > +            goto out_rebind; 
> > +        } 
> > +    } 
> > + 
> > +    goto out; 
> > + 
> > +out_rebind: 
> > +    /* some interfaces might be bound to usbback, unbind it then and 
> > +     * rebind to its original driver 
> > +     */ 
> > +    usbback_dev_unassign(gc, usb); 
> > +out: 
> > +    if (intfs) { 
> > +        for (i = 0; i < num; i++) 
> > +            free(intfs[i]); 
> > +        free(intfs); 
> > +    } 
> > +    free(usb_encode); 
> > +    return rc; 
> > +} 
> > + 
> > +static int do_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb  
> *usb) 
> > +{ 
> > +    int rc = 0; 
> > + 
> > +    rc = libxl__device_usb_add_xenstore(gc, domid, usb); 
> > +    if (rc) goto out; 
> > + 
> > +    rc = usbback_dev_assign(gc, usb); 
> > +    if (rc) 
> > +        libxl__device_usb_remove_xenstore(gc, domid, usb); 
> > + 
> > +out: 
> > +    return rc; 
> > +} 
> > + 
> > +int libxl__device_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb  
> *usb) 
> > +{ 
> > +    int rc; 
> > + 
> > +    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr); 
> > +    if (!usb->busid) { 
> > +        LOG(ERROR, "USB device doesn't exist in sysfs"); 
> > +        return ERROR_INVAL; 
> > +    } 
> > + 
> > +    if (!is_usb_assignable(gc, usb)) { 
> > +        LOG(ERROR, "USB device is not assignable."); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    /* check usb device is already assigned by pvusb */ 
> > +    if (is_usb_assigned(gc, usb)) { 
> > +        LOG(ERROR, "USB device is already attached to a domain."); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    rc = libxl__device_usb_setdefault(gc, domid, usb); 
> > +    if (rc) goto out; 
> > + 
> > +    rc = do_usb_add(gc, domid, usb); 
> > + 
> > +out: 
> > +    return rc; 
> > +} 
> > + 
> > +int libxl_device_usb_add(libxl_ctx *ctx, uint32_t domid, 
> > +                         libxl_device_usb *usb, 
> > +                         const libxl_asyncop_how *ao_how) 
> > +{ 
> > +    AO_CREATE(ctx, domid, ao_how); 
> > +    int rc; 
> > + 
> > +    rc = libxl__device_usb_add(gc, domid, usb); 
>  
> Hmm... Your remove function is async and this one is sync. See comment 
> of usbctrl_add. 
>  
> > +    libxl__ao_complete(egc, ao, rc); 
> > +    return AO_INPROGRESS; 
> > +} 
> > + 
> > +static int do_usb_remove(libxl__gc *gc, uint32_t domid, 
> > +                         libxl_device_usb *usb) 
> > +{ 
> > +    if (libxl__device_usb_remove_xenstore(gc, domid, usb)) 
> > +        return -1; 
> > + 
> > +    usbback_dev_unassign(gc, usb); 
> > + 
> > +    return 0; 
> > +} 
> > + 
> > +static int libxl__device_usb_remove(libxl__gc *gc, uint32_t domid, 
> > +                                    libxl_device_usb *usb) 
> > +{ 
> > +    libxl_device_usb *usbs = NULL; 
> > +    libxl_device_usb *usb_find = NULL; 
> > +    int i, num, rc; 
> > + 
> > +    usb->busid = usb_busaddr_to_busid(gc, usb->hostbus, usb->hostaddr); 
> > +    if (!usb->busid) { 
> > +        LOG(ERROR, "USB device doesn't exist in sysfs"); 
> > +        return ERROR_INVAL; 
> > +    } 
> > + 
> > +    usbs = libxl_device_usb_list_all(gc, domid, &num); 
> > +    if (!usbs) { 
> > +       LOG(ERROR, "No USB device attached to this domain"); 
> > +       return ERROR_FAIL; 
> > +    } 
> > + 
> > +    for (i = 0; i < num; i++) { 
> > +        if (!strcmp(usb->busid, usbs[i].busid)) { 
> > +            usb_find = usbs + i; 
> > +            break; 
> > +        } 
> > +    } 
> > + 
> > +    /* doesn't find the usb device in domain's usb device list*/ 
> > +    if (!usb_find) { 
> > +        LOG(ERROR, "USB device %x.%x is not attached to this domain", 
> > +            usb->hostbus, usb->hostaddr); 
> > +        rc = ERROR_FAIL; 
> > +        goto out; 
> > +    } 
> > + 
> > +    rc = do_usb_remove(gc, domid, usb_find); 
> > + 
> > +out: 
> > +    free(usbs); 
> > +    return rc; 
> > +} 
> > + 
> > +int libxl_device_usb_remove(libxl_ctx *ctx, uint32_t domid, 
> > +                            libxl_device_usb *usb, 
> > +                            const libxl_asyncop_how *ao_how) 
> > + 
> > +{ 
> > +    AO_CREATE(ctx, domid, ao_how); 
> > +    int rc; 
> > + 
> > +    rc = libxl__device_usb_remove(gc, domid, usb); 
> > + 
>  
> Same here. 
>  
> Wei. 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file
  2015-05-18 14:30     ` Wei Liu
@ 2015-05-19  3:21       ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-05-19  3:21 UTC (permalink / raw)
  To: Wei Liu, Ian Jackson
  Cc: lars.kurth, ian.campbell, george.dunlap, xen-devel, caobosimon



>>> On 5/18/2015 at 10:30 PM, in message
<20150518143008.GE9503@zion.uk.xensource.com>, Wei Liu <wei.liu2@citrix.com>
wrote: 
> On Mon, May 18, 2015 at 03:23:38PM +0100, Ian Jackson wrote: 
> > Chunyan Liu writes ("[PATCH V3 2/6] libxl_read_file_contents: fix reading  
> sysfs file"): 
> > > Sysfs file has size=4096 but actual file content is less than that. 
> >  
> > Wow. 
> >  
> > Is there any danger that the actual size might be >4096 ? 
> >  
> >  
> > > Current libxl_read_file_contents will treat it as error when file size 
> > > and actual file content differs, so reading sysfs file content with 
> > > this function always fails. Fix it so that we can reuse this function 
> > > to get sysfs file content in later pvusb work. 
> >  
> > I'm uncomfortable with removing an error check from this function for 
> > all its call sites. 
> >  
> > I think, sadly, that we are going to need a new function - at least, a 
> > new entrypoint. 
> >  
> >  
> > We don't want to repeat the whole of libxl__read_file_contents. 
> >  
> > Perhaps the bulk should be made into libxl__read_file_contents_core 
> > which takes a boolean instructing whether to tolerate magically 
> > shrinking files ? 
> >  
> > Setting that boolean probably ought to arrange to insist that the 
> > function gets eof, in case the file is actually bigger rather than 
> > smaller than the size. 
> >  
> >  
> > Ian, Wei ? 
> >  
>  
> Yes, we need a new entry point. 

Will update. Thanks!

>  
> Wei. 
>  
> > Ian. 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-19  3:20     ` Chun Yan Liu
@ 2015-05-19 10:20       ` George Dunlap
  2015-05-19 11:31         ` Jürgen Groß
  2015-05-20  9:04       ` Wei Liu
  1 sibling, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-19 10:20 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Tue, May 19, 2015 at 4:20 AM, Chun Yan Liu <cyliu@suse.com> wrote:
>>
>> > +static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
>> > +                                            libxl_device_usbctrl *usbctrl)
>> > +{
>> > +    int rc;
>> > +
>> > +    if (!usbctrl->version)
>> > +        usbctrl->version = 2;
>> > +
>>
>> How hard would it be to implement USB 3? I assume this depends on QEMU's
>> support? I.e. we just need to specify the version to 3 and it should
>> just work? Just curious.
>
> This implementation is coworked with PVUSB  driver, so it depends on PVUSB
> driver's support. If PVUSB driver supports that, just specify the version to 3 and
> can work.

I sort of got the idea that from the pvusb driver's perspective, the
1/2/3 thing was mostly just a number -- one that's important to the
USB device itself and the USB driver for those devices in the guest,
but less so for the pvusb layer.  Is that not the case?

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-19 10:20       ` George Dunlap
@ 2015-05-19 11:31         ` Jürgen Groß
  0 siblings, 0 replies; 53+ messages in thread
From: Jürgen Groß @ 2015-05-19 11:31 UTC (permalink / raw)
  To: George Dunlap, Chun Yan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On 05/19/2015 12:20 PM, George Dunlap wrote:
> On Tue, May 19, 2015 at 4:20 AM, Chun Yan Liu <cyliu@suse.com> wrote:
>>>
>>>> +static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
>>>> +                                            libxl_device_usbctrl *usbctrl)
>>>> +{
>>>> +    int rc;
>>>> +
>>>> +    if (!usbctrl->version)
>>>> +        usbctrl->version = 2;
>>>> +
>>>
>>> How hard would it be to implement USB 3? I assume this depends on QEMU's
>>> support? I.e. we just need to specify the version to 3 and it should
>>> just work? Just curious.
>>
>> This implementation is coworked with PVUSB  driver, so it depends on PVUSB
>> driver's support. If PVUSB driver supports that, just specify the version to 3 and
>> can work.
>
> I sort of got the idea that from the pvusb driver's perspective, the
> 1/2/3 thing was mostly just a number -- one that's important to the
> USB device itself and the USB driver for those devices in the guest,
> but less so for the pvusb layer.  Is that not the case?

I think at least for support of USB3.0 streams there is some work to do.
I haven't done any thorough research on this topic, but as libusb had to
be changed for support of USB3.0 as well, I'd expect this to be the case
for the pvusb interface, too.

Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
                     ` (2 preceding siblings ...)
  2015-05-18 18:07   ` Wei Liu
@ 2015-05-19 18:06   ` George Dunlap
  2015-05-19 18:24     ` Ian Campbell
  2015-06-08  9:06     ` 答复: " Chun Yan Liu
  2015-05-19 18:16   ` George Dunlap
  2015-05-19 18:20   ` George Dunlap
  5 siblings, 2 replies; 53+ messages in thread
From: George Dunlap @ 2015-05-19 18:06 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add pvusb APIs, including:
>  - attach/detach (create/destroy) virtual usb controller.
>  - attach/detach usb device
>  - list usb controller and usb devices
>  - some other helper functions
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>

OK, getting closer. :-)

A number of comments:

> diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> index 6bc75c5..cbe3519 100644
> --- a/tools/libxl/libxl.h
> +++ b/tools/libxl/libxl.h
> @@ -114,6 +114,12 @@
>  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
>
>  /*
> + * LIBXL_HAVE_PVUSB indicates the functions for doing hot-plug of
> + * USB devices through pvusb.
> + */
> +#define LIBXL_HAVE_PVUSB 1

It seems like we should document somewhere how we expect these to be
used -- namely the connection between usbctrl and usb devices.  In
particular, that you can either add a usbctrl, and then add a usb
device to it specifically; or if you don't specify a usbctrl when
calling usb_add, it will find the most reasonable one to add it to,
even creating one for you if you didn't have one.


> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
> new file mode 100644
> index 0000000..4e4975a
> --- /dev/null
> +++ b/tools/libxl/libxl_pvusb.c

> +int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
> +                              libxl_device_usbctrl *usbctrl)
> +{
> +    int rc = 0;
> +
> +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
> +    if (rc < 0) goto out;
> +
> +    if (usbctrl->devid == -1) {

Hmm, I was doing to say that this shouldn't be a magic constant, but
it looks like that's what all the other devices do :-/

> +static bool is_usb_in_array(libxl_device_usb *usbs, int num,
> +                            libxl_device_usb *usb)
> +{
> +    int i;
> +
> +    for (i = 0; i < num; i++) {
> +        if (!strcmp(usbs[i].busid, usb->busid) )

Here (and elsewhere) you should probably use the COMPARE_USB() macro
you define in this patch.

> +/* check if USB device type is assignable */
> +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    libxl_ctx *ctx = libxl__gc_owner(gc);
> +    int classcode;
> +    char *filename;
> +    void *buf;
> +
> +    assert(usb->busid);
> +
> +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid);
> +    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0)
> +        return false;
> +
> +    sscanf(buf, "%x", &classcode);
> +    if (classcode == USBHUB_CLASS_CODE)
> +        return false;
> +
> +    return true;

Just checking, is it really the case that *all* USB classes are
assignable, *except* for hubs?

This is a minor thing, but this might be simper to do this:

 return classcode != USBHUB_CLASS_CODE;

> +/* get usb devices under certain usb controller */
> +static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int usbctrl,
> +                                  libxl_device_usb **usbs, int *num)
> +{
> +    char *be_path, *num_devs;
> +    int n, i;
> +
> +    *usbs = NULL;
> +    *num = 0;
> +
> +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d",
> +                        libxl__xs_get_dompath(gc, 0), domid, usbctrl);
> +    num_devs = libxl__xs_read(gc, XBT_NULL,
> +                              GCSPRINTF("%s/num-ports", be_path));
> +    if (!num_devs)
> +        return 0;
> +
> +    n = atoi(num_devs);
> +    *usbs = calloc(n, sizeof(libxl_device_usb));
> +
> +    for (i = 0; i < n; i++) {
> +        char *busid;
> +        libxl_device_usb *usb = NULL;
> +
> +        busid = libxl__xs_read(gc, XBT_NULL,
> +                               GCSPRINTF("%s/port/%d", be_path, i + 1));
> +        if (busid && strcmp(busid, "")) {
> +            usb = *usbs + *num;
> +            usb->ctrl = usbctrl;
> +            usb->port = i + 1;
> +            usb->busid = strdup(busid);

This needs to populate the hostbus / hostaddr as well; busid is pretty
useless to users / external callers.

> +/* get all usb devices of the domain */
> +static libxl_device_usb *
> +libxl_device_usb_list_all(libxl__gc *gc, uint32_t domid, int *num)
> +{
> +    char **usbctrls;
> +    unsigned int nd, i, j;
> +    char *be_path;
> +    int rc;
> +    libxl_device_usb *usbs = NULL;
> +
> +    *num = 0;
> +
> +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb/%d",
> +                        LIBXL_TOOLSTACK_DOMID, domid);
> +    usbctrls = libxl__xs_directory(gc, XBT_NULL, be_path, &nd);
> +
> +    for (i = 0; i < nd; i++) {
> +        int nc = 0;
> +        libxl_device_usb *tmp = NULL;
> +        rc = libxl__device_usb_list(gc, domid, atoi(usbctrls[i]), &tmp, &nc);
> +        if (!nc) continue;
> +
> +        usbs = realloc(usbs, sizeof(libxl_device_usb)*((*num) + nc));
> +        for(j = 0; j < nc; j++) {
> +            usbs[*num].ctrl = tmp[j].ctrl;
> +            usbs[*num].port = tmp[j].port;
> +            usbs[*num].busid = strdup(tmp[j].busid);

This needs to copy the hostaddr and busaddr as well, as these are
primarily what an external caller will want.

> +/* find first unused controller:port and give that to usb device */
> +static int
> +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
> +                                      libxl_device_usb *usb)
> +{
> +    libxl_ctx *ctx = CTX;
> +    libxl_device_usbctrl *usbctrls;
> +    libxl_device_usb *usbs = NULL;
> +    int numctrl, numusb, i, j, rc = -1;
> +    char *be_path, *tmp;
> +
> +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> +    if ( !numctrl)
> +        goto out;
> +
> +    for (i = 0; i < numctrl; i++) {
> +        rc = libxl__device_usb_list(gc, domid, usbctrls[i].devid,
> +                                    &usbs, &numusb);
> +        if (rc) continue;
> +
> +        if (!usbctrls[i].ports || numusb == usbctrls[i].ports)
> +            continue;
> +
> +        for (j = 1; i <= numusb; j++) {

Port numbers start at 1, do they?  Interesting...

Er, but isn't the middle thing just plain wrong?  For one, you want to
be comparing j not i.  I can't see that i is updated inside the loop,
so ATM this will loop forever.

For two, you want to compare to usbctrls[i].ports (the total number of
ports), not to numusb (the number of currently assigned devices).

> +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
> +                                        libxl_device_usb *usb)
> +{
> +    char *be_path, *tmp;
> +
> +    if (usb->ctrl == -1) {

Oh, right -- libxl_devid's default to -1, don't they?  I'll save the
grumbling about the lack of a #define here then. (Or rather, I'll
grumble at the library rather than you.)

> +        int ret = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
> +        /* If no existing ctrl to host this usb device, setup a new one */
> +        if (ret) {
> +            libxl_device_usbctrl usbctrl;
> +            libxl_device_usbctrl_init(&usbctrl);
> +            libxl__device_usbctrl_add(gc, domid, &usbctrl);

I think in the previous round I asked what would happen if this
failed, and you said you would fail later, but that you'd change it to
check for an error here and bail out earlier.

> +/* Cann't write '.' or ':' into Xenstore as key. So, change '.' to '_',
> + * change ':' to '-'.
> + */
> +static char *usb_interface_encode(char *busid)

Maybe usb_interface_xenstore_encode, to make it clear that you're just
encoding it to store in xenstore?

> +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
> +{
> +    char **intfs = NULL;
> +    char *path;
> +    int num = 0, i;
> +    int rc = 0;
> +    char *usb_encode = NULL;
> +
> +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
> +        return ERROR_FAIL;
> +
> +    usb_encode = usb_interface_encode(usb->busid);
> +
> +    for (i = 0; i < num; i++){
> +        char *intf = intfs[i];
> +        char *drvpath = NULL;
> +
> +        if (usb_intf_is_assigned(gc, intf) > 0) {
> +            /* unbind interface from usbback driver */
> +            if (unbind_usb_intf(gc, intf, NULL) < 0) {
> +                rc = ERROR_FAIL;
> +                goto out;
> +            }
> +        }
> +
> +        /* bind interface to its originial driver */
> +        drvpath = libxl__xs_read(gc, XBT_NULL,
> +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
> +                  usb_encode, usb_interface_encode(intf)));
> +        if (drvpath) {
> +            if (bind_usb_intf(gc, intf, drvpath) < 0) {
> +                LOGE(ERROR, "Couldn't bind %s to %s", intf, drvpath);
> +                rc = ERROR_FAIL;
> +                goto out;

I think this function probably shouldn't fail if it can't re-bind to
the original driver.  If nothing else, this will be bad because now
the USB device has at least one of its interfaces unbound from
usbback, but the other ones still bound to it.  All interfaces should
be unbound from usbback regardless.

I also think that while a warning should be logged, that the function
as a whole should return success if it managed to unbind, even if it
didn't manage to rebind.  (But feel free to argue otherwise.)

> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 0866433..a6db614 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -533,6 +533,22 @@ libxl_device_pci = Struct("device_pci", [
>      ("seize", bool),
>      ])
>
> +libxl_device_usbctrl = Struct("device_usbctrl", [
> +    ("devid", libxl_devid),
> +    ("version", integer),
> +    ("ports", integer),
> +    ("backend_domid", libxl_domid),
> +    ("backend_domname", string),
> +   ])
> +
> +libxl_device_usb = Struct("device_usb", [
> +    ("ctrl", libxl_devid),
> +    ("port", integer),
> +    ("busid", string),

So it looks like "busid" is purely an internal string that you're
putting in the device struct for convenience, so that you don't have
to either keep translating bus:addr into the sysfs node, or pass
around the second string as well.  Is that right?

I think libxl does something similar to this with "backend_domname"
and "backend_domid"; but in that case I believe you *can* specify the
backend_domid if you want to.  In this case, you're exposing a struct
element which is clobbered immediately, at least by usb_add and
usb_remove.

Maintainers, what do you think?

Other than that, I think this patch as-is looks in pretty good shape
(haven't taken a close look at the rest in the series yet); the
primary things are making the "hostbus:hostaddr" actually the primary
interface; at a minimum filling in that information when doing
queries.  I'd ideally getting rid of "busid" from the external
interface altogether.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
                     ` (3 preceding siblings ...)
  2015-05-19 18:06   ` George Dunlap
@ 2015-05-19 18:16   ` George Dunlap
  2015-06-08 11:15     ` 答复: " Chun Yan Liu
  2015-05-19 18:20   ` George Dunlap
  5 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-19 18:16 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> +int libxl_device_usb_getinfo(libxl_ctx *ctx, char *busid, libxl_usbinfo *usbinfo)
> +{

Sorry, missed something.  This function is wrong.  It gives you
information about a *host* USB device; but should properly give you
information about a *guest* USB device.  The
libxl_device_disk_getinfo() counterpart, for example, takes a domid
and a virtual device (from within a libxl_disk structure) and returns
information about that virtual device.

Hrm... which makes me wonder whether we should use <ctrl,port> as a
unique "handle" for usb devices in the interface, or have a separate
devid for the devices themselves.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
                     ` (4 preceding siblings ...)
  2015-05-19 18:16   ` George Dunlap
@ 2015-05-19 18:20   ` George Dunlap
  5 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-05-19 18:20 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> +int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
> +                                libxl_device_usbctrl *usbctrl,
> +                                libxl_usbctrlinfo *usbctrlinfo)
> +{
> +    GC_INIT(ctx);
> +    char *dompath, *usbctrlpath;
> +    char *val;
> +    int rc = 0;
> +
> +    usbctrlinfo->devid = usbctrl->devid;
> +    usbctrlinfo->ports = usbctrl->ports;
> +    usbctrlinfo->version = usbctrl->version;

...and following on from the previous mail, it looks like the other
device_*_getinfo() functions *only* read devid -- which means you just
have to make the struct and fill in devid, and getinfo() will tell
*you* the ports, the version, and so on.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-19 18:06   ` George Dunlap
@ 2015-05-19 18:24     ` Ian Campbell
  2015-06-08  9:06     ` 答复: " Chun Yan Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-19 18:24 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Chunyan Liu, xen-devel, Ian Jackson, Simon Cao

On Tue, 2015-05-19 at 19:06 +0100, George Dunlap wrote:
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index 6bc75c5..cbe3519 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -114,6 +114,12 @@
> >  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
> >
> >  /*
> > + * LIBXL_HAVE_PVUSB indicates the functions for doing hot-plug of
> > + * USB devices through pvusb.
> > + */
> > +#define LIBXL_HAVE_PVUSB 1
> 
> It seems like we should document somewhere how we expect these to be
> used -- namely the connection between usbctrl and usb devices.  In
> particular, that you can either add a usbctrl, and then add a usb
> device to it specifically; or if you don't specify a usbctrl when
> calling usb_add, it will find the most reasonable one to add it to,
> even creating one for you if you didn't have one.

(I've not checked if this exists in the series yet, but...)

I'd like to see some docs added to libxl.h as an addendum to the
existing:

 * Devices
 * =======
 *
 * Each device is represented by a libxl_device_<TYPE> data structure
 ...

stuff which describes the general form of interfaces used for those
classes of device which distinguish the controller from the devices
themselves in the libxl API.

The new docs should then be applicable to e..g the pvscsi stuff as well.

The goal is to avoid having numerous subtly different style of device
interaction in the libxl API by grouping them into the least number of
logical groups (thus far 2: ones with explicit controllers and those
without) each with a consistent API.

Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-19  3:20     ` Chun Yan Liu
  2015-05-19 10:20       ` George Dunlap
@ 2015-05-20  9:04       ` Wei Liu
  2015-05-20  9:12         ` Ian Campbell
                           ` (2 more replies)
  1 sibling, 3 replies; 53+ messages in thread
From: Wei Liu @ 2015-05-20  9:04 UTC (permalink / raw)
  To: Chun Yan Liu
  Cc: lars.kurth, Wei Liu, ian.campbell, george.dunlap, xen-devel,
	ian.jackson, caobosimon

On Mon, May 18, 2015 at 09:20:43PM -0600, Chun Yan Liu wrote:
[...]
> >  
> > > +    for (;;) { 
> > > +        rc = libxl__xs_transaction_start(gc, &t); 
> > > +        if (rc) goto out; 
> > > + 
> > > +        rc = libxl__device_exists(gc, t, device); 
> > > +        if (rc < 0) goto out; 
> > > +        if (rc == 1) { 
> > > +            /* already exists in xenstore */ 
> > > +            LOG(ERROR, "device already exists in xenstore"); 
> > > +            rc = ERROR_DEVICE_EXISTS; 
> > > +            goto out; 
> > > +        } 
> > > + 
> > > +        rc = libxl__set_domain_configuration(gc, domid, &d_config); 
> > > +        if (rc) goto out; 
> > > + 
> > > +        libxl__device_generic_add(gc, t, device, 
> > > +                                  libxl__xs_kvs_of_flexarray(gc, back, 
> > > +                                                             back->count), 
> > > +                                  libxl__xs_kvs_of_flexarray(gc, front, 
> > > +                                                             front->count), 
> > > +                                  NULL); 
> > > +        libxl__usbport_add_xenstore(gc, t, domid, usbctrl); 
> > > +        rc = libxl__xs_transaction_commit(gc, &t); 
> > > +        if (!rc) break; 
> > > +        if (rc < 0) goto out; 
> > > +    } 
> > > + 
> >  
> > You don't have aodev so you cannot check update_json. Maybe you need 
> > aodev? 
> >  
> > That field update_json is set to true when doing hotplug. It's set to 
> > false during domain creation. 
> >  
> > The same comment applies to other add functions as well.
> 
> Thanks, I'm updating that. But maybe like pci_add and pci_remove functions,
> will add a 'starting' flag to indicate hotplug or creation.
> Looking at DEFINE_DEVICE_ADD and DEFINE_DEVICE_REMOVE, usbctrl_add
> and usb_add can use DEFINE_DEVICE_ADD; but usbctrl_remove and usb_remove
> cannot use DEFINE_DEVICE_REMOVE directly, need some extra handling. So,
> finally turned to not use these macros but refer to pci functions.
> 

TBH I prefer to have only one way to deal with devices.  I personally
prefer the async style that every other devices use. Maybe that's just
because I mostly worked with those.

I don't know the full history of libxl_pci.c so I will wait for Ian and
Ian's comments on which way to go.

I think one merit of determining whether to use sync or async is that
whether the operation is long running (slow). Long running one should be
asyn.  I guess usb passthrough is not slow so we are probably fine in
this regard.

BTW if you find the macros can't meet your need you can either extend
them or not use them.

> >  
> > > +out: 
> > > +    if (lock) libxl__unlock_domain_userdata(lock); 
> > > +    libxl_device_usbctrl_dispose(&usbctrl_saved); 
> > > +    libxl_domain_config_dispose(&d_config); 
> > > +    return rc; 
> > > +} 
> > > + 
> > > +int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid, 
> > > +                              libxl_device_usbctrl *usbctrl) 
> > > +{ 
> > > +    int rc = 0; 
> > > + 
> > > +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl); 
> > > +    if (rc < 0) goto out; 
> > > + 
> > > +    if (usbctrl->devid == -1) { 
> > > +        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb"); 
> > > +        if (usbctrl->devid < 0) { 
> > > +            rc = ERROR_FAIL; 
> > > +            goto out; 
> > > +        } 
> > > +    } 
> > > + 
> > > +    if (libxl__usbctrl_add_xenstore(gc, domid, usbctrl) < 0){ 
> > > +        rc = ERROR_FAIL; 
> > > +        goto out; 
> > > +    } 
> > > + 
> > > +out: 
> > > +    return rc; 
> > > +} 
> > > + 
> > > +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid, 
> > > +                             libxl_device_usbctrl *usbctrl, 
> > > +                             const libxl_asyncop_how *ao_how) 
> > > +{ 
> > > +    AO_CREATE(ctx, domid, ao_how); 
> > > +    int rc; 
> > > + 
> > > +    rc = libxl__device_usbctrl_add(gc, domid, usbctrl); 
> >  
> > Hmm... Your remove function is async while this one is sync, why?
> 
> Hmm, maybe not proper here, just referred to pci_add implementation.
> Current calling places only use sync mode.
>  

Yeah, I only ask for consistency.

Wei.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-20  9:04       ` Wei Liu
@ 2015-05-20  9:12         ` Ian Campbell
  2015-05-20  9:30         ` Chun Yan Liu
  2015-06-11 16:54         ` Ian Jackson
  2 siblings, 0 replies; 53+ messages in thread
From: Ian Campbell @ 2015-05-20  9:12 UTC (permalink / raw)
  To: Wei Liu
  Cc: lars.kurth, george.dunlap, Chun Yan Liu, xen-devel, ian.jackson,
	caobosimon

On Wed, 2015-05-20 at 10:04 +0100, Wei Liu wrote:
> On Mon, May 18, 2015 at 09:20:43PM -0600, Chun Yan Liu wrote:
> [...]
> > >  
> > > > +    for (;;) { 
> > > > +        rc = libxl__xs_transaction_start(gc, &t); 
> > > > +        if (rc) goto out; 
> > > > + 
> > > > +        rc = libxl__device_exists(gc, t, device); 
> > > > +        if (rc < 0) goto out; 
> > > > +        if (rc == 1) { 
> > > > +            /* already exists in xenstore */ 
> > > > +            LOG(ERROR, "device already exists in xenstore"); 
> > > > +            rc = ERROR_DEVICE_EXISTS; 
> > > > +            goto out; 
> > > > +        } 
> > > > + 
> > > > +        rc = libxl__set_domain_configuration(gc, domid, &d_config); 
> > > > +        if (rc) goto out; 
> > > > + 
> > > > +        libxl__device_generic_add(gc, t, device, 
> > > > +                                  libxl__xs_kvs_of_flexarray(gc, back, 
> > > > +                                                             back->count), 
> > > > +                                  libxl__xs_kvs_of_flexarray(gc, front, 
> > > > +                                                             front->count), 
> > > > +                                  NULL); 
> > > > +        libxl__usbport_add_xenstore(gc, t, domid, usbctrl); 
> > > > +        rc = libxl__xs_transaction_commit(gc, &t); 
> > > > +        if (!rc) break; 
> > > > +        if (rc < 0) goto out; 
> > > > +    } 
> > > > + 
> > >  
> > > You don't have aodev so you cannot check update_json. Maybe you need 
> > > aodev? 
> > >  
> > > That field update_json is set to true when doing hotplug. It's set to 
> > > false during domain creation. 
> > >  
> > > The same comment applies to other add functions as well.
> > 
> > Thanks, I'm updating that. But maybe like pci_add and pci_remove functions,
> > will add a 'starting' flag to indicate hotplug or creation.
> > Looking at DEFINE_DEVICE_ADD and DEFINE_DEVICE_REMOVE, usbctrl_add
> > and usb_add can use DEFINE_DEVICE_ADD; but usbctrl_remove and usb_remove
> > cannot use DEFINE_DEVICE_REMOVE directly, need some extra handling. So,
> > finally turned to not use these macros but refer to pci functions.
> > 
> 
> TBH I prefer to have only one way to deal with devices.  I personally
> prefer the async style that every other devices use. Maybe that's just
> because I mostly worked with those.
> 
> I don't know the full history of libxl_pci.c so I will wait for Ian and
> Ian's comments on which way to go.

libxl_pci.c should not be used as an example of the right way to do
things (to say the least).

> I think one merit of determining whether to use sync or async is that
> whether the operation is long running (slow). Long running one should be
> asyn.  I guess usb passthrough is not slow so we are probably fine in
> this regard.

Device add/remove is expected to be async at least at the public API
level , all the others appear to be and it seems logical to me that they
would be.

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-20  9:04       ` Wei Liu
  2015-05-20  9:12         ` Ian Campbell
@ 2015-05-20  9:30         ` Chun Yan Liu
  2015-06-11 16:54         ` Ian Jackson
  2 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-05-20  9:30 UTC (permalink / raw)
  To: Wei Liu
  Cc: lars.kurth, ian.campbell, george.dunlap, xen-devel, ian.jackson,
	caobosimon



>>> On 5/20/2015 at 05:04 PM, in message
<20150520090407.GW26335@zion.uk.xensource.com>, Wei Liu <wei.liu2@citrix.com>
wrote: 
> On Mon, May 18, 2015 at 09:20:43PM -0600, Chun Yan Liu wrote: 
> [...] 
> > >   
> > > > +    for (;;) {  
> > > > +        rc = libxl__xs_transaction_start(gc, &t);  
> > > > +        if (rc) goto out;  
> > > > +  
> > > > +        rc = libxl__device_exists(gc, t, device);  
> > > > +        if (rc < 0) goto out;  
> > > > +        if (rc == 1) {  
> > > > +            /* already exists in xenstore */  
> > > > +            LOG(ERROR, "device already exists in xenstore");  
> > > > +            rc = ERROR_DEVICE_EXISTS;  
> > > > +            goto out;  
> > > > +        }  
> > > > +  
> > > > +        rc = libxl__set_domain_configuration(gc, domid, &d_config);  
> > > > +        if (rc) goto out;  
> > > > +  
> > > > +        libxl__device_generic_add(gc, t, device,  
> > > > +                                  libxl__xs_kvs_of_flexarray(gc, back,  
> > > > +                                                             back->count),  
>  
> > > > +                                  libxl__xs_kvs_of_flexarray(gc, front,  
> > > > +                                                              
> front->count),  
> > > > +                                  NULL);  
> > > > +        libxl__usbport_add_xenstore(gc, t, domid, usbctrl);  
> > > > +        rc = libxl__xs_transaction_commit(gc, &t);  
> > > > +        if (!rc) break;  
> > > > +        if (rc < 0) goto out;  
> > > > +    }  
> > > > +  
> > >   
> > > You don't have aodev so you cannot check update_json. Maybe you need  
> > > aodev?  
> > >   
> > > That field update_json is set to true when doing hotplug. It's set to  
> > > false during domain creation.  
> > >   
> > > The same comment applies to other add functions as well. 
> >  
> > Thanks, I'm updating that. But maybe like pci_add and pci_remove functions, 
> > will add a 'starting' flag to indicate hotplug or creation. 
> > Looking at DEFINE_DEVICE_ADD and DEFINE_DEVICE_REMOVE, usbctrl_add 
> > and usb_add can use DEFINE_DEVICE_ADD; but usbctrl_remove and usb_remove 
> > cannot use DEFINE_DEVICE_REMOVE directly, need some extra handling. So, 
> > finally turned to not use these macros but refer to pci functions. 
> >  
>  
> TBH I prefer to have only one way to deal with devices.  I personally 
> prefer the async style that every other devices use. Maybe that's just 
> because I mostly worked with those. 
>  
> I don't know the full history of libxl_pci.c so I will wait for Ian and 
> Ian's comments on which way to go. 
>  
> I think one merit of determining whether to use sync or async is that 
> whether the operation is long running (slow). Long running one should be 
> asyn.  I guess usb passthrough is not slow so we are probably fine in 
> this regard. 
>  
> BTW if you find the macros can't meet your need you can either extend 
> them or not use them. 

Got you and Ian. I'll update codes then.

Chunyan

>  
> > >   
> > > > +out:  
> > > > +    if (lock) libxl__unlock_domain_userdata(lock);  
> > > > +    libxl_device_usbctrl_dispose(&usbctrl_saved);  
> > > > +    libxl_domain_config_dispose(&d_config);  
> > > > +    return rc;  
> > > > +}  
> > > > +  
> > > > +int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,  
> > > > +                              libxl_device_usbctrl *usbctrl)  
> > > > +{  
> > > > +    int rc = 0;  
> > > > +  
> > > > +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);  
> > > > +    if (rc < 0) goto out;  
> > > > +  
> > > > +    if (usbctrl->devid == -1) {  
> > > > +        usbctrl->devid = libxl__device_nextid(gc, domid, "vusb");  
> > > > +        if (usbctrl->devid < 0) {  
> > > > +            rc = ERROR_FAIL;  
> > > > +            goto out;  
> > > > +        }  
> > > > +    }  
> > > > +  
> > > > +    if (libxl__usbctrl_add_xenstore(gc, domid, usbctrl) < 0){  
> > > > +        rc = ERROR_FAIL;  
> > > > +        goto out;  
> > > > +    }  
> > > > +  
> > > > +out:  
> > > > +    return rc;  
> > > > +}  
> > > > +  
> > > > +int libxl_device_usbctrl_add(libxl_ctx *ctx, uint32_t domid,  
> > > > +                             libxl_device_usbctrl *usbctrl,  
> > > > +                             const libxl_asyncop_how *ao_how)  
> > > > +{  
> > > > +    AO_CREATE(ctx, domid, ao_how);  
> > > > +    int rc;  
> > > > +  
> > > > +    rc = libxl__device_usbctrl_add(gc, domid, usbctrl);  
> > >   
> > > Hmm... Your remove function is async while this one is sync, why? 
> >  
> > Hmm, maybe not proper here, just referred to pci_add implementation. 
> > Current calling places only use sync mode. 
> >   
>  
> Yeah, I only ask for consistency. 
>  
> Wei. 
>  
>  

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-04-20  8:12   ` Juergen Gross
  2015-04-21  2:21     ` Chun Yan Liu
@ 2015-05-20 14:20     ` George Dunlap
  2015-05-20 14:33       ` Juergen Gross
  1 sibling, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-20 14:20 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com> wrote:
> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>
>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>> usb-attach and usb-detach.
>>
>> To attach a usb device to guest through pvusb, one could follow
>> following example:
>>
>>   #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>
>>   #xl usb-list test_vm
>>   will show the usb controllers and port usage under the domain.
>>
>>   #xl usb-attach test_vm 1.6
>>   will find the first usable controller:port, and attach usb
>>   device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>   to it. One could also specify which <controller> and which <port>.
>>
>>   #xl usb-detach test_vm 1.6
>>
>>   #xl usb-ctrl-detach test_vm dev_id
>>   will destroy the controller with specified dev_id. Dev_id
>>   can be traced in usb-list info.
>>
>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>> ---
>> Changes to v2:
>>    * use bus.addr as user interface instead of busid in usb-attach|detach
>>    * remove usb-assignable-list interface
>
>
> Why? While lsusb in combination with xl usb-list for each domain will
> give the same information, having to iterate through all domains can be
> quite annoying.
>
> An alternative would be to accept omitting the domain for xl usb-list
> and list all domains with assigned usb devices in this case.

I don't understand what information it is that you want.  Do you want
a list of devices *not already assigned* to domains?

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-04-19  3:50 ` [PATCH V3 4/6] xl: add pvusb commands Chunyan Liu
  2015-04-20  8:12   ` Juergen Gross
@ 2015-05-20 14:23   ` George Dunlap
  2015-05-20 15:46     ` George Dunlap
  2015-05-20 15:55   ` George Dunlap
  2 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-20 14:23 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
> usb-attach and usb-detach.
>
> To attach a usb device to guest through pvusb, one could follow
> following example:
>
>  #xl usb-ctrl-attach test_vm version=1 num_ports=8
>
>  #xl usb-list test_vm
>  will show the usb controllers and port usage under the domain.
>
>  #xl usb-attach test_vm 1.6
>  will find the first usable controller:port, and attach usb
>  device whose bus address is 1.6 (busnum is 1, devnum is 6)
>  to it. One could also specify which <controller> and which <port>.
>
>  #xl usb-detach test_vm 1.6
>
>  #xl usb-ctrl-detach test_vm dev_id
>  will destroy the controller with specified dev_id. Dev_id
>  can be traced in usb-list info.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>
> Signed-off-by: Simon Cao <caobosimon@gmail.com>

Thanks, Chunyan.  The interface looks good to me, and a skim through
the code doesn't turn up any obvious problems.  Someone more familiar
with the libxl coding conventions will have to give it a stricter
review.

I'll wait on the ack until we've discussed Juergen's point.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-20 14:20     ` George Dunlap
@ 2015-05-20 14:33       ` Juergen Gross
  2015-05-20 14:41         ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-20 14:33 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/20/2015 04:20 PM, George Dunlap wrote:
> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com> wrote:
>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>
>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>> usb-attach and usb-detach.
>>>
>>> To attach a usb device to guest through pvusb, one could follow
>>> following example:
>>>
>>>    #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>
>>>    #xl usb-list test_vm
>>>    will show the usb controllers and port usage under the domain.
>>>
>>>    #xl usb-attach test_vm 1.6
>>>    will find the first usable controller:port, and attach usb
>>>    device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>    to it. One could also specify which <controller> and which <port>.
>>>
>>>    #xl usb-detach test_vm 1.6
>>>
>>>    #xl usb-ctrl-detach test_vm dev_id
>>>    will destroy the controller with specified dev_id. Dev_id
>>>    can be traced in usb-list info.
>>>
>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>> ---
>>> Changes to v2:
>>>     * use bus.addr as user interface instead of busid in usb-attach|detach
>>>     * remove usb-assignable-list interface
>>
>>
>> Why? While lsusb in combination with xl usb-list for each domain will
>> give the same information, having to iterate through all domains can be
>> quite annoying.
>>
>> An alternative would be to accept omitting the domain for xl usb-list
>> and list all domains with assigned usb devices in this case.
>
> I don't understand what information it is that you want.  Do you want
> a list of devices *not already assigned* to domains?

Yes.

Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-20 14:33       ` Juergen Gross
@ 2015-05-20 14:41         ` George Dunlap
  2015-05-20 14:55           ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-20 14:41 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On Wed, May 20, 2015 at 3:33 PM, Juergen Gross <jgross@suse.com> wrote:
> On 05/20/2015 04:20 PM, George Dunlap wrote:
>>
>> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com> wrote:
>>>
>>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>>
>>>>
>>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>>> usb-attach and usb-detach.
>>>>
>>>> To attach a usb device to guest through pvusb, one could follow
>>>> following example:
>>>>
>>>>    #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>>
>>>>    #xl usb-list test_vm
>>>>    will show the usb controllers and port usage under the domain.
>>>>
>>>>    #xl usb-attach test_vm 1.6
>>>>    will find the first usable controller:port, and attach usb
>>>>    device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>>    to it. One could also specify which <controller> and which <port>.
>>>>
>>>>    #xl usb-detach test_vm 1.6
>>>>
>>>>    #xl usb-ctrl-detach test_vm dev_id
>>>>    will destroy the controller with specified dev_id. Dev_id
>>>>    can be traced in usb-list info.
>>>>
>>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>>> ---
>>>> Changes to v2:
>>>>     * use bus.addr as user interface instead of busid in
>>>> usb-attach|detach
>>>>     * remove usb-assignable-list interface
>>>
>>>
>>>
>>> Why? While lsusb in combination with xl usb-list for each domain will
>>> give the same information, having to iterate through all domains can be
>>> quite annoying.
>>>
>>> An alternative would be to accept omitting the domain for xl usb-list
>>> and list all domains with assigned usb devices in this case.
>>
>>
>> I don't understand what information it is that you want.  Do you want
>> a list of devices *not already assigned* to domains?
>
>
> Yes.

...and why do you need that, instead of just remembering what you'd
assigned to whom?

We don't really have the equivalent for pci either.  That is, if a
device shows up in "lspci" but not in "pci-assignable-list", that may
be either because 1) I hasn't yet been assigned to pciback (and this
is available to be assigned to a domain), or 2) because it's already
been assigned to a domain.  Someone new coming to the system would
need to check all VMs to see which devices hadn't yet been assigned.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-20 14:41         ` George Dunlap
@ 2015-05-20 14:55           ` Juergen Gross
  2015-05-20 15:25             ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-20 14:55 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/20/2015 04:41 PM, George Dunlap wrote:
> On Wed, May 20, 2015 at 3:33 PM, Juergen Gross <jgross@suse.com> wrote:
>> On 05/20/2015 04:20 PM, George Dunlap wrote:
>>>
>>> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com> wrote:
>>>>
>>>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>>>
>>>>>
>>>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>>>> usb-attach and usb-detach.
>>>>>
>>>>> To attach a usb device to guest through pvusb, one could follow
>>>>> following example:
>>>>>
>>>>>     #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>>>
>>>>>     #xl usb-list test_vm
>>>>>     will show the usb controllers and port usage under the domain.
>>>>>
>>>>>     #xl usb-attach test_vm 1.6
>>>>>     will find the first usable controller:port, and attach usb
>>>>>     device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>>>     to it. One could also specify which <controller> and which <port>.
>>>>>
>>>>>     #xl usb-detach test_vm 1.6
>>>>>
>>>>>     #xl usb-ctrl-detach test_vm dev_id
>>>>>     will destroy the controller with specified dev_id. Dev_id
>>>>>     can be traced in usb-list info.
>>>>>
>>>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>>>> ---
>>>>> Changes to v2:
>>>>>      * use bus.addr as user interface instead of busid in
>>>>> usb-attach|detach
>>>>>      * remove usb-assignable-list interface
>>>>
>>>>
>>>>
>>>> Why? While lsusb in combination with xl usb-list for each domain will
>>>> give the same information, having to iterate through all domains can be
>>>> quite annoying.
>>>>
>>>> An alternative would be to accept omitting the domain for xl usb-list
>>>> and list all domains with assigned usb devices in this case.
>>>
>>>
>>> I don't understand what information it is that you want.  Do you want
>>> a list of devices *not already assigned* to domains?
>>
>>
>> Yes.
>
> ...and why do you need that, instead of just remembering what you'd
> assigned to whom?
>
> We don't really have the equivalent for pci either.  That is, if a
> device shows up in "lspci" but not in "pci-assignable-list", that may
> be either because 1) I hasn't yet been assigned to pciback (and this
> is available to be assigned to a domain), or 2) because it's already
> been assigned to a domain.  Someone new coming to the system would
> need to check all VMs to see which devices hadn't yet been assigned.

So this is a problem of pci-assignable-list, which isn't present for
USB devices. Any USB device not already assigned to a VM would be listed
as before with "xm usb-assignable-list".

Additionally all systems support hotplug of USB devices - the list of
available USB-devices can change rather often. If you unplug e.g. a
memory stick which has been assigned to a VM and stick it in again it
might show up under a different address and has to be reassigned.
Remembering having it already assigned won't help in this case as much
as a simple command to list the devices ready for assignment.


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-20 14:55           ` Juergen Gross
@ 2015-05-20 15:25             ` George Dunlap
  2015-05-21  3:35               ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-20 15:25 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/20/2015 03:55 PM, Juergen Gross wrote:
> On 05/20/2015 04:41 PM, George Dunlap wrote:
>> On Wed, May 20, 2015 at 3:33 PM, Juergen Gross <jgross@suse.com> wrote:
>>> On 05/20/2015 04:20 PM, George Dunlap wrote:
>>>>
>>>> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com> wrote:
>>>>>
>>>>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>>>>
>>>>>>
>>>>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>>>>> usb-attach and usb-detach.
>>>>>>
>>>>>> To attach a usb device to guest through pvusb, one could follow
>>>>>> following example:
>>>>>>
>>>>>>     #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>>>>
>>>>>>     #xl usb-list test_vm
>>>>>>     will show the usb controllers and port usage under the domain.
>>>>>>
>>>>>>     #xl usb-attach test_vm 1.6
>>>>>>     will find the first usable controller:port, and attach usb
>>>>>>     device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>>>>     to it. One could also specify which <controller> and which
>>>>>> <port>.
>>>>>>
>>>>>>     #xl usb-detach test_vm 1.6
>>>>>>
>>>>>>     #xl usb-ctrl-detach test_vm dev_id
>>>>>>     will destroy the controller with specified dev_id. Dev_id
>>>>>>     can be traced in usb-list info.
>>>>>>
>>>>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>>>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>>>>> ---
>>>>>> Changes to v2:
>>>>>>      * use bus.addr as user interface instead of busid in
>>>>>> usb-attach|detach
>>>>>>      * remove usb-assignable-list interface
>>>>>
>>>>>
>>>>>
>>>>> Why? While lsusb in combination with xl usb-list for each domain will
>>>>> give the same information, having to iterate through all domains
>>>>> can be
>>>>> quite annoying.
>>>>>
>>>>> An alternative would be to accept omitting the domain for xl usb-list
>>>>> and list all domains with assigned usb devices in this case.
>>>>
>>>>
>>>> I don't understand what information it is that you want.  Do you want
>>>> a list of devices *not already assigned* to domains?
>>>
>>>
>>> Yes.
>>
>> ...and why do you need that, instead of just remembering what you'd
>> assigned to whom?
>>
>> We don't really have the equivalent for pci either.  That is, if a
>> device shows up in "lspci" but not in "pci-assignable-list", that may
>> be either because 1) I hasn't yet been assigned to pciback (and this
>> is available to be assigned to a domain), or 2) because it's already
>> been assigned to a domain.  Someone new coming to the system would
>> need to check all VMs to see which devices hadn't yet been assigned.
> 
> So this is a problem of pci-assignable-list, which isn't present for
> USB devices. Any USB device not already assigned to a VM would be listed
> as before with "xm usb-assignable-list".
> 
> Additionally all systems support hotplug of USB devices - the list of
> available USB-devices can change rather often. If you unplug e.g. a
> memory stick which has been assigned to a VM and stick it in again it
> might show up under a different address and has to be reassigned.
> Remembering having it already assigned won't help in this case as much
> as a simple command to list the devices ready for assignment.

OK. :-)  Yes, I can see that having USB devices disappear and appear as
a different bus:host would make such a command particularly useful.

But then we have the problem that "assignable" means something different
in each case.  In pci-assignable-list, it means "Devices which have been
assigned to pciback but not yet been attached to a domain".  In your
suggested command, it means "Devices which have not yet been assigned
either to pvusbback or to a domain."

When I introduced pci-assignable-* I didn't realize that "assignable"
was already in one of the xm commands with a different meaning.  I
introduced it because until that point, neither xm nor xl had a way of
attaching a device to pciback.

So we basically have three options:
1. Keep both names the same
2. Rename usb-assignable-list to something else (usb-available-list?)
3. Rename pci-assignable-* to something else

#1 would be the most backwards-compatible.  But I think it's really bad
going forward, since you have two commands that look like they should do
similar things that don't.

#2 and #3 would both solve the potential confusion issue.

#2 would be the easiest on current xl users.  It's a bit annoying
interface-wise going forward, since "assignable" is probably the best
word for what Juergen wants; and it's not 100% backwards-compatible with
the previous xm usb commands.

#3 would give us probably the most consistent naming thing going
forward, but would be a pretty major breakage for current xl users.

I'm inclined to suggest #2 as the best balance between not disrupting
current users and not confusing future users.

Wei / Ian, any opinions?

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-20 14:23   ` George Dunlap
@ 2015-05-20 15:46     ` George Dunlap
  0 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-05-20 15:46 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Wed, May 20, 2015 at 3:23 PM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>> usb-attach and usb-detach.
>>
>> To attach a usb device to guest through pvusb, one could follow
>> following example:
>>
>>  #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>
>>  #xl usb-list test_vm
>>  will show the usb controllers and port usage under the domain.
>>
>>  #xl usb-attach test_vm 1.6
>>  will find the first usable controller:port, and attach usb
>>  device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>  to it. One could also specify which <controller> and which <port>.
>>
>>  #xl usb-detach test_vm 1.6
>>
>>  #xl usb-ctrl-detach test_vm dev_id
>>  will destroy the controller with specified dev_id. Dev_id
>>  can be traced in usb-list info.
>>
>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>
> Thanks, Chunyan.  The interface looks good to me, and a skim through
> the code doesn't turn up any obvious problems.  Someone more familiar
> with the libxl coding conventions will have to give it a stricter
> review.

Actually, sorry to take this back.  I looked at what I wrote in v2 of
the series, and I think what I suggested there was better. :-)

I pointed out that most devices -- disk, vif, pci, &c -- take the same
thing in their command-line arguments as you put in the config file.
They have parse_disk_config(), parse_vif_config(), and
xlu_parse_bdf(), each of which are called from both
parse_config_data() and the respective add commands.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-04-19  3:50 ` [PATCH V3 4/6] xl: add pvusb commands Chunyan Liu
  2015-04-20  8:12   ` Juergen Gross
  2015-05-20 14:23   ` George Dunlap
@ 2015-05-20 15:55   ` George Dunlap
  2 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-05-20 15:55 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
> usb-attach and usb-detach.
>
> To attach a usb device to guest through pvusb, one could follow
> following example:
>
>  #xl usb-ctrl-attach test_vm version=1 num_ports=8

One more nit: this shuold be "ports=8", not "num_ports=8".

(It looks like it's only wrong here in the description; it's
consistently "ports" throughout the rest of this patch and the next
one.)

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-20 15:25             ` George Dunlap
@ 2015-05-21  3:35               ` Juergen Gross
  2015-05-21 10:37                 ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-21  3:35 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/20/2015 05:25 PM, George Dunlap wrote:
> On 05/20/2015 03:55 PM, Juergen Gross wrote:
>> On 05/20/2015 04:41 PM, George Dunlap wrote:
>>> On Wed, May 20, 2015 at 3:33 PM, Juergen Gross <jgross@suse.com> wrote:
>>>> On 05/20/2015 04:20 PM, George Dunlap wrote:
>>>>>
>>>>> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com> wrote:
>>>>>>
>>>>>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>>>>>
>>>>>>>
>>>>>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>>>>>> usb-attach and usb-detach.
>>>>>>>
>>>>>>> To attach a usb device to guest through pvusb, one could follow
>>>>>>> following example:
>>>>>>>
>>>>>>>      #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>>>>>
>>>>>>>      #xl usb-list test_vm
>>>>>>>      will show the usb controllers and port usage under the domain.
>>>>>>>
>>>>>>>      #xl usb-attach test_vm 1.6
>>>>>>>      will find the first usable controller:port, and attach usb
>>>>>>>      device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>>>>>      to it. One could also specify which <controller> and which
>>>>>>> <port>.
>>>>>>>
>>>>>>>      #xl usb-detach test_vm 1.6
>>>>>>>
>>>>>>>      #xl usb-ctrl-detach test_vm dev_id
>>>>>>>      will destroy the controller with specified dev_id. Dev_id
>>>>>>>      can be traced in usb-list info.
>>>>>>>
>>>>>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>>>>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>>>>>> ---
>>>>>>> Changes to v2:
>>>>>>>       * use bus.addr as user interface instead of busid in
>>>>>>> usb-attach|detach
>>>>>>>       * remove usb-assignable-list interface
>>>>>>
>>>>>>
>>>>>>
>>>>>> Why? While lsusb in combination with xl usb-list for each domain will
>>>>>> give the same information, having to iterate through all domains
>>>>>> can be
>>>>>> quite annoying.
>>>>>>
>>>>>> An alternative would be to accept omitting the domain for xl usb-list
>>>>>> and list all domains with assigned usb devices in this case.
>>>>>
>>>>>
>>>>> I don't understand what information it is that you want.  Do you want
>>>>> a list of devices *not already assigned* to domains?
>>>>
>>>>
>>>> Yes.
>>>
>>> ...and why do you need that, instead of just remembering what you'd
>>> assigned to whom?
>>>
>>> We don't really have the equivalent for pci either.  That is, if a
>>> device shows up in "lspci" but not in "pci-assignable-list", that may
>>> be either because 1) I hasn't yet been assigned to pciback (and this
>>> is available to be assigned to a domain), or 2) because it's already
>>> been assigned to a domain.  Someone new coming to the system would
>>> need to check all VMs to see which devices hadn't yet been assigned.
>>
>> So this is a problem of pci-assignable-list, which isn't present for
>> USB devices. Any USB device not already assigned to a VM would be listed
>> as before with "xm usb-assignable-list".
>>
>> Additionally all systems support hotplug of USB devices - the list of
>> available USB-devices can change rather often. If you unplug e.g. a
>> memory stick which has been assigned to a VM and stick it in again it
>> might show up under a different address and has to be reassigned.
>> Remembering having it already assigned won't help in this case as much
>> as a simple command to list the devices ready for assignment.
>
> OK. :-)  Yes, I can see that having USB devices disappear and appear as
> a different bus:host would make such a command particularly useful.
>
> But then we have the problem that "assignable" means something different
> in each case.  In pci-assignable-list, it means "Devices which have been
> assigned to pciback but not yet been attached to a domain".  In your
> suggested command, it means "Devices which have not yet been assigned
> either to pvusbback or to a domain."

For USB devices there isn't such an action as "assign it to pvusbback".
You just assign a USB device to a domain. One command, no steps to
prepare that action (besides the possibility to define a USB controller
first).

> When I introduced pci-assignable-* I didn't realize that "assignable"
> was already in one of the xm commands with a different meaning.  I
> introduced it because until that point, neither xm nor xl had a way of
> attaching a device to pciback.
>
> So we basically have three options:
> 1. Keep both names the same
> 2. Rename usb-assignable-list to something else (usb-available-list?)
> 3. Rename pci-assignable-* to something else
>
> #1 would be the most backwards-compatible.  But I think it's really bad
> going forward, since you have two commands that look like they should do
> similar things that don't.

Actually they do. :-)


Juergen

>
> #2 and #3 would both solve the potential confusion issue.
>
> #2 would be the easiest on current xl users.  It's a bit annoying
> interface-wise going forward, since "assignable" is probably the best
> word for what Juergen wants; and it's not 100% backwards-compatible with
> the previous xm usb commands.
>
> #3 would give us probably the most consistent naming thing going
> forward, but would be a pretty major breakage for current xl users.
>
> I'm inclined to suggest #2 as the best balance between not disrupting
> current users and not confusing future users.
>
> Wei / Ian, any opinions?
>
>   -George
>
>

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21  3:35               ` Juergen Gross
@ 2015-05-21 10:37                 ` George Dunlap
  2015-05-21 10:52                   ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-21 10:37 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 04:35 AM, Juergen Gross wrote:
> On 05/20/2015 05:25 PM, George Dunlap wrote:
>> On 05/20/2015 03:55 PM, Juergen Gross wrote:
>>> On 05/20/2015 04:41 PM, George Dunlap wrote:
>>>> On Wed, May 20, 2015 at 3:33 PM, Juergen Gross <jgross@suse.com> wrote:
>>>>> On 05/20/2015 04:20 PM, George Dunlap wrote:
>>>>>>
>>>>>> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com>
>>>>>> wrote:
>>>>>>>
>>>>>>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>>>>>>> usb-attach and usb-detach.
>>>>>>>>
>>>>>>>> To attach a usb device to guest through pvusb, one could follow
>>>>>>>> following example:
>>>>>>>>
>>>>>>>>      #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>>>>>>
>>>>>>>>      #xl usb-list test_vm
>>>>>>>>      will show the usb controllers and port usage under the domain.
>>>>>>>>
>>>>>>>>      #xl usb-attach test_vm 1.6
>>>>>>>>      will find the first usable controller:port, and attach usb
>>>>>>>>      device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>>>>>>      to it. One could also specify which <controller> and which
>>>>>>>> <port>.
>>>>>>>>
>>>>>>>>      #xl usb-detach test_vm 1.6
>>>>>>>>
>>>>>>>>      #xl usb-ctrl-detach test_vm dev_id
>>>>>>>>      will destroy the controller with specified dev_id. Dev_id
>>>>>>>>      can be traced in usb-list info.
>>>>>>>>
>>>>>>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>>>>>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>>>>>>> ---
>>>>>>>> Changes to v2:
>>>>>>>>       * use bus.addr as user interface instead of busid in
>>>>>>>> usb-attach|detach
>>>>>>>>       * remove usb-assignable-list interface
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Why? While lsusb in combination with xl usb-list for each domain
>>>>>>> will
>>>>>>> give the same information, having to iterate through all domains
>>>>>>> can be
>>>>>>> quite annoying.
>>>>>>>
>>>>>>> An alternative would be to accept omitting the domain for xl
>>>>>>> usb-list
>>>>>>> and list all domains with assigned usb devices in this case.
>>>>>>
>>>>>>
>>>>>> I don't understand what information it is that you want.  Do you want
>>>>>> a list of devices *not already assigned* to domains?
>>>>>
>>>>>
>>>>> Yes.
>>>>
>>>> ...and why do you need that, instead of just remembering what you'd
>>>> assigned to whom?
>>>>
>>>> We don't really have the equivalent for pci either.  That is, if a
>>>> device shows up in "lspci" but not in "pci-assignable-list", that may
>>>> be either because 1) I hasn't yet been assigned to pciback (and this
>>>> is available to be assigned to a domain), or 2) because it's already
>>>> been assigned to a domain.  Someone new coming to the system would
>>>> need to check all VMs to see which devices hadn't yet been assigned.
>>>
>>> So this is a problem of pci-assignable-list, which isn't present for
>>> USB devices. Any USB device not already assigned to a VM would be listed
>>> as before with "xm usb-assignable-list".
>>>
>>> Additionally all systems support hotplug of USB devices - the list of
>>> available USB-devices can change rather often. If you unplug e.g. a
>>> memory stick which has been assigned to a VM and stick it in again it
>>> might show up under a different address and has to be reassigned.
>>> Remembering having it already assigned won't help in this case as much
>>> as a simple command to list the devices ready for assignment.
>>
>> OK. :-)  Yes, I can see that having USB devices disappear and appear as
>> a different bus:host would make such a command particularly useful.
>>
>> But then we have the problem that "assignable" means something different
>> in each case.  In pci-assignable-list, it means "Devices which have been
>> assigned to pciback but not yet been attached to a domain".  In your
>> suggested command, it means "Devices which have not yet been assigned
>> either to pvusbback or to a domain."
> 
> For USB devices there isn't such an action as "assign it to pvusbback".
> You just assign a USB device to a domain. One command, no steps to
> prepare that action (besides the possibility to define a USB controller
> first).

You can do the same thing (now) for pci devices, by setting "seize=1" in
the device spec (which will cause xl to first assign them to pciback,
then to the guest, just as the usb code does now).

> 
>> When I introduced pci-assignable-* I didn't realize that "assignable"
>> was already in one of the xm commands with a different meaning.  I
>> introduced it because until that point, neither xm nor xl had a way of
>> attaching a device to pciback.
>>
>> So we basically have three options:
>> 1. Keep both names the same
>> 2. Rename usb-assignable-list to something else (usb-available-list?)
>> 3. Rename pci-assignable-* to something else
>>
>> #1 would be the most backwards-compatible.  But I think it's really bad
>> going forward, since you have two commands that look like they should do
>> similar things that don't.
> 
> Actually they do. :-)

Actually, they don't. :-)

>From your description, "xm usb-assignable-list" would list *all* USB
devices on the system which were available to be assigned to a guest.

"xl pci-assignable-list" does *not* list all PCI devices on a system
which are available to be assigned.  It lists the ones which can be
assigned without the "seize=1" parameter -- the ones you've already done
something with.  It won't tell you about the other devices on the system
which have not yet been assigned to pciback.

Yes, from a pedantic perspective, both will tell you on which devices
you can run "X-attach" without any extra arguments.  But from a
practical perspective, "xm usb-assignable-list" tells you something
practical about the state of devices on the whole system; and "xl
pci-assignable-list" tells you a technical quirk about devices are in a
half-way state between not being assigned and actually being assigned.

(I'm about 45% of the opinion that we should make pci behave like the
proposed USB patches -- i.e., make seize=1 the default, and just
deprecate the whole "pci-assignable" state altogether.  My only
hesitation is that it removes one level of safety check against doing
something like unplugging the host's main disk controller or something.)

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 10:37                 ` George Dunlap
@ 2015-05-21 10:52                   ` Juergen Gross
  2015-05-21 11:11                     ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-21 10:52 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 12:37 PM, George Dunlap wrote:
> On 05/21/2015 04:35 AM, Juergen Gross wrote:
>> On 05/20/2015 05:25 PM, George Dunlap wrote:
>>> On 05/20/2015 03:55 PM, Juergen Gross wrote:
>>>> On 05/20/2015 04:41 PM, George Dunlap wrote:
>>>>> On Wed, May 20, 2015 at 3:33 PM, Juergen Gross <jgross@suse.com> wrote:
>>>>>> On 05/20/2015 04:20 PM, George Dunlap wrote:
>>>>>>>
>>>>>>> On Mon, Apr 20, 2015 at 9:12 AM, Juergen Gross <jgross@suse.com>
>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 04/19/2015 05:50 AM, Chunyan Liu wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Add pvusb commands: usb-ctrl-attach, usb-ctrl-detach, usb-list,
>>>>>>>>> usb-attach and usb-detach.
>>>>>>>>>
>>>>>>>>> To attach a usb device to guest through pvusb, one could follow
>>>>>>>>> following example:
>>>>>>>>>
>>>>>>>>>       #xl usb-ctrl-attach test_vm version=1 num_ports=8
>>>>>>>>>
>>>>>>>>>       #xl usb-list test_vm
>>>>>>>>>       will show the usb controllers and port usage under the domain.
>>>>>>>>>
>>>>>>>>>       #xl usb-attach test_vm 1.6
>>>>>>>>>       will find the first usable controller:port, and attach usb
>>>>>>>>>       device whose bus address is 1.6 (busnum is 1, devnum is 6)
>>>>>>>>>       to it. One could also specify which <controller> and which
>>>>>>>>> <port>.
>>>>>>>>>
>>>>>>>>>       #xl usb-detach test_vm 1.6
>>>>>>>>>
>>>>>>>>>       #xl usb-ctrl-detach test_vm dev_id
>>>>>>>>>       will destroy the controller with specified dev_id. Dev_id
>>>>>>>>>       can be traced in usb-list info.
>>>>>>>>>
>>>>>>>>> Signed-off-by: Chunyan Liu <cyliu@suse.com>
>>>>>>>>> Signed-off-by: Simon Cao <caobosimon@gmail.com>
>>>>>>>>> ---
>>>>>>>>> Changes to v2:
>>>>>>>>>        * use bus.addr as user interface instead of busid in
>>>>>>>>> usb-attach|detach
>>>>>>>>>        * remove usb-assignable-list interface
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Why? While lsusb in combination with xl usb-list for each domain
>>>>>>>> will
>>>>>>>> give the same information, having to iterate through all domains
>>>>>>>> can be
>>>>>>>> quite annoying.
>>>>>>>>
>>>>>>>> An alternative would be to accept omitting the domain for xl
>>>>>>>> usb-list
>>>>>>>> and list all domains with assigned usb devices in this case.
>>>>>>>
>>>>>>>
>>>>>>> I don't understand what information it is that you want.  Do you want
>>>>>>> a list of devices *not already assigned* to domains?
>>>>>>
>>>>>>
>>>>>> Yes.
>>>>>
>>>>> ...and why do you need that, instead of just remembering what you'd
>>>>> assigned to whom?
>>>>>
>>>>> We don't really have the equivalent for pci either.  That is, if a
>>>>> device shows up in "lspci" but not in "pci-assignable-list", that may
>>>>> be either because 1) I hasn't yet been assigned to pciback (and this
>>>>> is available to be assigned to a domain), or 2) because it's already
>>>>> been assigned to a domain.  Someone new coming to the system would
>>>>> need to check all VMs to see which devices hadn't yet been assigned.
>>>>
>>>> So this is a problem of pci-assignable-list, which isn't present for
>>>> USB devices. Any USB device not already assigned to a VM would be listed
>>>> as before with "xm usb-assignable-list".
>>>>
>>>> Additionally all systems support hotplug of USB devices - the list of
>>>> available USB-devices can change rather often. If you unplug e.g. a
>>>> memory stick which has been assigned to a VM and stick it in again it
>>>> might show up under a different address and has to be reassigned.
>>>> Remembering having it already assigned won't help in this case as much
>>>> as a simple command to list the devices ready for assignment.
>>>
>>> OK. :-)  Yes, I can see that having USB devices disappear and appear as
>>> a different bus:host would make such a command particularly useful.
>>>
>>> But then we have the problem that "assignable" means something different
>>> in each case.  In pci-assignable-list, it means "Devices which have been
>>> assigned to pciback but not yet been attached to a domain".  In your
>>> suggested command, it means "Devices which have not yet been assigned
>>> either to pvusbback or to a domain."
>>
>> For USB devices there isn't such an action as "assign it to pvusbback".
>> You just assign a USB device to a domain. One command, no steps to
>> prepare that action (besides the possibility to define a USB controller
>> first).
>
> You can do the same thing (now) for pci devices, by setting "seize=1" in
> the device spec (which will cause xl to first assign them to pciback,
> then to the guest, just as the usb code does now).
>
>>
>>> When I introduced pci-assignable-* I didn't realize that "assignable"
>>> was already in one of the xm commands with a different meaning.  I
>>> introduced it because until that point, neither xm nor xl had a way of
>>> attaching a device to pciback.
>>>
>>> So we basically have three options:
>>> 1. Keep both names the same
>>> 2. Rename usb-assignable-list to something else (usb-available-list?)
>>> 3. Rename pci-assignable-* to something else
>>>
>>> #1 would be the most backwards-compatible.  But I think it's really bad
>>> going forward, since you have two commands that look like they should do
>>> similar things that don't.
>>
>> Actually they do. :-)
>
> Actually, they don't. :-)
>
>  From your description, "xm usb-assignable-list" would list *all* USB
> devices on the system which were available to be assigned to a guest.
>
> "xl pci-assignable-list" does *not* list all PCI devices on a system
> which are available to be assigned.  It lists the ones which can be
> assigned without the "seize=1" parameter -- the ones you've already done
> something with.  It won't tell you about the other devices on the system
> which have not yet been assigned to pciback.

So "xl pci-assignable-list" is suppressing some of the PCI devices which
in theory could be assigned. I don't think this "weird" behaviour should
be mimicked by "xl usb-assignable-list".

> Yes, from a pedantic perspective, both will tell you on which devices
> you can run "X-attach" without any extra arguments.  But from a
> practical perspective, "xm usb-assignable-list" tells you something
> practical about the state of devices on the whole system; and "xl
> pci-assignable-list" tells you a technical quirk about devices are in a
> half-way state between not being assigned and actually being assigned.

I can't believe you are suggesting to use "a technical quirk" as a good
example for future development. Just because a user interface isn't
perfect shouldn't result in other interfaces to behave in the same
imperfect way.

> (I'm about 45% of the opinion that we should make pci behave like the
> proposed USB patches -- i.e., make seize=1 the default, and just
> deprecate the whole "pci-assignable" state altogether.  My only
> hesitation is that it removes one level of safety check against doing
> something like unplugging the host's main disk controller or something.)

You could add a configuration item in xl.conf for setting the default
behaviour, defaulting to "default_seize=1". :-)


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 10:52                   ` Juergen Gross
@ 2015-05-21 11:11                     ` George Dunlap
  2015-05-21 11:58                       ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-21 11:11 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 11:52 AM, Juergen Gross wrote:
>>  From your description, "xm usb-assignable-list" would list *all* USB
>> devices on the system which were available to be assigned to a guest.
>>
>> "xl pci-assignable-list" does *not* list all PCI devices on a system
>> which are available to be assigned.  It lists the ones which can be
>> assigned without the "seize=1" parameter -- the ones you've already done
>> something with.  It won't tell you about the other devices on the system
>> which have not yet been assigned to pciback.
> 
> So "xl pci-assignable-list" is suppressing some of the PCI devices which
> in theory could be assigned. I don't think this "weird" behaviour should
> be mimicked by "xl usb-assignable-list".

That's not really an accurate characterization.  I introduced the
concept of "assignable" because under xm, you had to manually much about
in sysfs yourself to assign the device to pciback before attaching it to
a guest.  So "pci-assignable-add" takes a device and assigns it to
pciback; "pci-attach" attaches an assignable device to the guest.
"pci-assignable-list" lists the devices which have been made
"assignable" under this new definition.

> 
>> Yes, from a pedantic perspective, both will tell you on which devices
>> you can run "X-attach" without any extra arguments.  But from a
>> practical perspective, "xm usb-assignable-list" tells you something
>> practical about the state of devices on the whole system; and "xl
>> pci-assignable-list" tells you a technical quirk about devices are in a
>> half-way state between not being assigned and actually being assigned.
> 
> I can't believe you are suggesting to use "a technical quirk" as a good
> example for future development. Just because a user interface isn't
> perfect shouldn't result in other interfaces to behave in the same
> imperfect way.

You seem to have missed in my tone that I think "xm usb-assignable-list"
behavior is more useful.  I said that "xm usb-assignable-list" gave you
practical information, and I said that "xl pci-assignable-list" gives
you about a technical quirk.  I also said that "pci-assignable" a state
half-way in between being assigned and not assigned, which I personally
think portrays it as rather clunky.

I never claimed that we should make the new usb-*-list command mimic
pci-assignable-list.  What I'm responding to is your claim that they do
similar things, and so implying that it should be OK for them to have
similar names.  They do not do the similar things, and therefore they
must not have similar names.

So, as I said in the previous e-mail:
* I think that it would definitely be useful to have the "xm
usb-assignable-list" functionality.
* But we cannot give it the same name as the current "xl
pci-assignable-list" functionality, since they behave differently
* I think "assignable" is the best name for what "xm
usb-assignable-list" does; however,
* We have existing users to consider; I think choosing a different name
(like "xl usb-available-list") will have the lowest negative impact on
existing users.

>> (I'm about 45% of the opinion that we should make pci behave like the
>> proposed USB patches -- i.e., make seize=1 the default, and just
>> deprecate the whole "pci-assignable" state altogether.  My only
>> hesitation is that it removes one level of safety check against doing
>> something like unplugging the host's main disk controller or something.)
> 
> You could add a configuration item in xl.conf for setting the default
> behaviour, defaulting to "default_seize=1". :-)

I think we have that already, actually.  (If not at least we discussed
it.)  But unfortunately that doesn't help us in the current situation,
as the "pci-assignable" state still exists, and we still need commands
to deal with it.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 11:11                     ` George Dunlap
@ 2015-05-21 11:58                       ` Juergen Gross
  2015-05-21 13:01                         ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-21 11:58 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 01:11 PM, George Dunlap wrote:
> On 05/21/2015 11:52 AM, Juergen Gross wrote:
>>>   From your description, "xm usb-assignable-list" would list *all* USB
>>> devices on the system which were available to be assigned to a guest.
>>>
>>> "xl pci-assignable-list" does *not* list all PCI devices on a system
>>> which are available to be assigned.  It lists the ones which can be
>>> assigned without the "seize=1" parameter -- the ones you've already done
>>> something with.  It won't tell you about the other devices on the system
>>> which have not yet been assigned to pciback.
>>
>> So "xl pci-assignable-list" is suppressing some of the PCI devices which
>> in theory could be assigned. I don't think this "weird" behaviour should
>> be mimicked by "xl usb-assignable-list".
>
> That's not really an accurate characterization.  I introduced the
> concept of "assignable" because under xm, you had to manually much about
> in sysfs yourself to assign the device to pciback before attaching it to
> a guest.  So "pci-assignable-add" takes a device and assigns it to
> pciback; "pci-attach" attaches an assignable device to the guest.
> "pci-assignable-list" lists the devices which have been made
> "assignable" under this new definition.
>
>>
>>> Yes, from a pedantic perspective, both will tell you on which devices
>>> you can run "X-attach" without any extra arguments.  But from a
>>> practical perspective, "xm usb-assignable-list" tells you something
>>> practical about the state of devices on the whole system; and "xl
>>> pci-assignable-list" tells you a technical quirk about devices are in a
>>> half-way state between not being assigned and actually being assigned.
>>
>> I can't believe you are suggesting to use "a technical quirk" as a good
>> example for future development. Just because a user interface isn't
>> perfect shouldn't result in other interfaces to behave in the same
>> imperfect way.
>
> You seem to have missed in my tone that I think "xm usb-assignable-list"
> behavior is more useful.  I said that "xm usb-assignable-list" gave you
> practical information, and I said that "xl pci-assignable-list" gives
> you about a technical quirk.  I also said that "pci-assignable" a state
> half-way in between being assigned and not assigned, which I personally
> think portrays it as rather clunky.

I didn't want to offend you, sorry if you felt that way.

> I never claimed that we should make the new usb-*-list command mimic
> pci-assignable-list.  What I'm responding to is your claim that they do
> similar things, and so implying that it should be OK for them to have
> similar names.  They do not do the similar things, and therefore they
> must not have similar names.
>
> So, as I said in the previous e-mail:
> * I think that it would definitely be useful to have the "xm
> usb-assignable-list" functionality.
> * But we cannot give it the same name as the current "xl
> pci-assignable-list" functionality, since they behave differently
> * I think "assignable" is the best name for what "xm
> usb-assignable-list" does; however,
> * We have existing users to consider; I think choosing a different name
> (like "xl usb-available-list") will have the lowest negative impact on
> existing users.

There might be existing users who know about "xm usb-assignable-list".

OTOH I don't care giving it another name, as long as the functionality
is available.


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 11:58                       ` Juergen Gross
@ 2015-05-21 13:01                         ` George Dunlap
  2015-05-21 13:08                           ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-21 13:01 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 12:58 PM, Juergen Gross wrote:
>>>> Yes, from a pedantic perspective, both will tell you on which devices
>>>> you can run "X-attach" without any extra arguments.  But from a
>>>> practical perspective, "xm usb-assignable-list" tells you something
>>>> practical about the state of devices on the whole system; and "xl
>>>> pci-assignable-list" tells you a technical quirk about devices are in a
>>>> half-way state between not being assigned and actually being assigned.
>>>
>>> I can't believe you are suggesting to use "a technical quirk" as a good
>>> example for future development. Just because a user interface isn't
>>> perfect shouldn't result in other interfaces to behave in the same
>>> imperfect way.
>>
>> You seem to have missed in my tone that I think "xm usb-assignable-list"
>> behavior is more useful.  I said that "xm usb-assignable-list" gave you
>> practical information, and I said that "xl pci-assignable-list" gives
>> you about a technical quirk.  I also said that "pci-assignable" a state
>> half-way in between being assigned and not assigned, which I personally
>> think portrays it as rather clunky.
> 
> I didn't want to offend you, sorry if you felt that way.

Misunderstandings happen -- I certainly misread what people write
sometimes.  It's mildly annoying but no harm done. :-)

>> I never claimed that we should make the new usb-*-list command mimic
>> pci-assignable-list.  What I'm responding to is your claim that they do
>> similar things, and so implying that it should be OK for them to have
>> similar names.  They do not do the similar things, and therefore they
>> must not have similar names.
>>
>> So, as I said in the previous e-mail:
>> * I think that it would definitely be useful to have the "xm
>> usb-assignable-list" functionality.
>> * But we cannot give it the same name as the current "xl
>> pci-assignable-list" functionality, since they behave differently
>> * I think "assignable" is the best name for what "xm
>> usb-assignable-list" does; however,
>> * We have existing users to consider; I think choosing a different name
>> (like "xl usb-available-list") will have the lowest negative impact on
>> existing users.
> 
> There might be existing users who know about "xm usb-assignable-list".

Yes -- unfortunately something has to give: either we confuse new users
with two names that sound similar but do something different, or we
confuse former xm users by renaming their function, or we confuse
current xl users by renaming their function.  There's badness to be had
whichever one we choose.  I think renaming the xm function is the least
bad of all the options.

> OTOH I don't care giving it another name, as long as the functionality
> is available.

OK -- what about "usb-available-list"?  Any objections / alternate
suggestions from anyone?

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 13:01                         ` George Dunlap
@ 2015-05-21 13:08                           ` Juergen Gross
  2015-05-21 13:43                             ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-21 13:08 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 03:01 PM, George Dunlap wrote:
> On 05/21/2015 12:58 PM, Juergen Gross wrote:
>> There might be existing users who know about "xm usb-assignable-list".
>
> Yes -- unfortunately something has to give: either we confuse new users
> with two names that sound similar but do something different, or we
> confuse former xm users by renaming their function, or we confuse
> current xl users by renaming their function.  There's badness to be had
> whichever one we choose.  I think renaming the xm function is the least
> bad of all the options.
>
>> OTOH I don't care giving it another name, as long as the functionality
>> is available.
>
> OK -- what about "usb-available-list"?  Any objections / alternate
> suggestions from anyone?

Hmm, just another idea:

xl usb-list -a

could list all domains with assigned USB-devices and the currently not
assigned devices as well. This would avoid the need for another command
name. It would even be possible to omit the "-a".


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 13:08                           ` Juergen Gross
@ 2015-05-21 13:43                             ` George Dunlap
  2015-05-21 13:55                               ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-21 13:43 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 02:08 PM, Juergen Gross wrote:
> On 05/21/2015 03:01 PM, George Dunlap wrote:
>> On 05/21/2015 12:58 PM, Juergen Gross wrote:
>>> There might be existing users who know about "xm usb-assignable-list".
>>
>> Yes -- unfortunately something has to give: either we confuse new users
>> with two names that sound similar but do something different, or we
>> confuse former xm users by renaming their function, or we confuse
>> current xl users by renaming their function.  There's badness to be had
>> whichever one we choose.  I think renaming the xm function is the least
>> bad of all the options.
>>
>>> OTOH I don't care giving it another name, as long as the functionality
>>> is available.
>>
>> OK -- what about "usb-available-list"?  Any objections / alternate
>> suggestions from anyone?
> 
> Hmm, just another idea:
> 
> xl usb-list -a
> 
> could list all domains with assigned USB-devices and the currently not
> assigned devices as well. This would avoid the need for another command
> name. It would even be possible to omit the "-a".

That works for me too, I guess.

Can I suggest, though, that work on that functionality be detached from
getting the core pvusb functionality in?  I can't really effectively do
the qemu side until it is in due to the cost of rebasing, and I would
really like to have both in for 4.6 if possible.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 13:43                             ` George Dunlap
@ 2015-05-21 13:55                               ` Juergen Gross
  2015-05-21 14:00                                 ` George Dunlap
  0 siblings, 1 reply; 53+ messages in thread
From: Juergen Gross @ 2015-05-21 13:55 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 03:43 PM, George Dunlap wrote:
> On 05/21/2015 02:08 PM, Juergen Gross wrote:
>> On 05/21/2015 03:01 PM, George Dunlap wrote:
>>> On 05/21/2015 12:58 PM, Juergen Gross wrote:
>>>> There might be existing users who know about "xm usb-assignable-list".
>>>
>>> Yes -- unfortunately something has to give: either we confuse new users
>>> with two names that sound similar but do something different, or we
>>> confuse former xm users by renaming their function, or we confuse
>>> current xl users by renaming their function.  There's badness to be had
>>> whichever one we choose.  I think renaming the xm function is the least
>>> bad of all the options.
>>>
>>>> OTOH I don't care giving it another name, as long as the functionality
>>>> is available.
>>>
>>> OK -- what about "usb-available-list"?  Any objections / alternate
>>> suggestions from anyone?
>>
>> Hmm, just another idea:
>>
>> xl usb-list -a
>>
>> could list all domains with assigned USB-devices and the currently not
>> assigned devices as well. This would avoid the need for another command
>> name. It would even be possible to omit the "-a".
>
> That works for me too, I guess.
>
> Can I suggest, though, that work on that functionality be detached from
> getting the core pvusb functionality in?  I can't really effectively do
> the qemu side until it is in due to the cost of rebasing, and I would
> really like to have both in for 4.6 if possible.

Sure.

I hope to have my pvusb backend in qemu ready until then, too. :-)
In case the performance isn't too bad I'll have to make some changes to
the libxl part as well, but this will result in some deletions only
(qemu will do driver unbinding, this will no longer be required to be
done by libxl).


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 13:55                               ` Juergen Gross
@ 2015-05-21 14:00                                 ` George Dunlap
  2015-05-21 14:14                                   ` Juergen Gross
  0 siblings, 1 reply; 53+ messages in thread
From: George Dunlap @ 2015-05-21 14:00 UTC (permalink / raw)
  To: Juergen Gross
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 02:55 PM, Juergen Gross wrote:
> On 05/21/2015 03:43 PM, George Dunlap wrote:
>> On 05/21/2015 02:08 PM, Juergen Gross wrote:
>>> On 05/21/2015 03:01 PM, George Dunlap wrote:
>>>> On 05/21/2015 12:58 PM, Juergen Gross wrote:
>>>>> There might be existing users who know about "xm usb-assignable-list".
>>>>
>>>> Yes -- unfortunately something has to give: either we confuse new users
>>>> with two names that sound similar but do something different, or we
>>>> confuse former xm users by renaming their function, or we confuse
>>>> current xl users by renaming their function.  There's badness to be had
>>>> whichever one we choose.  I think renaming the xm function is the least
>>>> bad of all the options.
>>>>
>>>>> OTOH I don't care giving it another name, as long as the functionality
>>>>> is available.
>>>>
>>>> OK -- what about "usb-available-list"?  Any objections / alternate
>>>> suggestions from anyone?
>>>
>>> Hmm, just another idea:
>>>
>>> xl usb-list -a
>>>
>>> could list all domains with assigned USB-devices and the currently not
>>> assigned devices as well. This would avoid the need for another command
>>> name. It would even be possible to omit the "-a".
>>
>> That works for me too, I guess.
>>
>> Can I suggest, though, that work on that functionality be detached from
>> getting the core pvusb functionality in?  I can't really effectively do
>> the qemu side until it is in due to the cost of rebasing, and I would
>> really like to have both in for 4.6 if possible.
> 
> Sure.
> 
> I hope to have my pvusb backend in qemu ready until then, too. :-)
> In case the performance isn't too bad I'll have to make some changes to
> the libxl part as well, but this will result in some deletions only
> (qemu will do driver unbinding, this will no longer be required to be
> done by libxl).

Right -- just like it's not necessary when qemu does emulated device
passthrough. :-)

Out of curiosity, what's the motivation for doing pvusb in qemu?  Will
there be an option, for instance, to switch usb devices from emulated
over to pv when the magic port is written to (as is done with disks &
network currently)?

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 4/6] xl: add pvusb commands
  2015-05-21 14:00                                 ` George Dunlap
@ 2015-05-21 14:14                                   ` Juergen Gross
  0 siblings, 0 replies; 53+ messages in thread
From: Juergen Gross @ 2015-05-21 14:14 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, Chunyan Liu, xen-devel,
	Ian Jackson, Simon Cao

On 05/21/2015 04:00 PM, George Dunlap wrote:
> On 05/21/2015 02:55 PM, Juergen Gross wrote:
>> On 05/21/2015 03:43 PM, George Dunlap wrote:
>>> On 05/21/2015 02:08 PM, Juergen Gross wrote:
>>>> On 05/21/2015 03:01 PM, George Dunlap wrote:
>>>>> On 05/21/2015 12:58 PM, Juergen Gross wrote:
>>>>>> There might be existing users who know about "xm usb-assignable-list".
>>>>>
>>>>> Yes -- unfortunately something has to give: either we confuse new users
>>>>> with two names that sound similar but do something different, or we
>>>>> confuse former xm users by renaming their function, or we confuse
>>>>> current xl users by renaming their function.  There's badness to be had
>>>>> whichever one we choose.  I think renaming the xm function is the least
>>>>> bad of all the options.
>>>>>
>>>>>> OTOH I don't care giving it another name, as long as the functionality
>>>>>> is available.
>>>>>
>>>>> OK -- what about "usb-available-list"?  Any objections / alternate
>>>>> suggestions from anyone?
>>>>
>>>> Hmm, just another idea:
>>>>
>>>> xl usb-list -a
>>>>
>>>> could list all domains with assigned USB-devices and the currently not
>>>> assigned devices as well. This would avoid the need for another command
>>>> name. It would even be possible to omit the "-a".
>>>
>>> That works for me too, I guess.
>>>
>>> Can I suggest, though, that work on that functionality be detached from
>>> getting the core pvusb functionality in?  I can't really effectively do
>>> the qemu side until it is in due to the cost of rebasing, and I would
>>> really like to have both in for 4.6 if possible.
>>
>> Sure.
>>
>> I hope to have my pvusb backend in qemu ready until then, too. :-)
>> In case the performance isn't too bad I'll have to make some changes to
>> the libxl part as well, but this will result in some deletions only
>> (qemu will do driver unbinding, this will no longer be required to be
>> done by libxl).
>
> Right -- just like it's not necessary when qemu does emulated device
> passthrough. :-)
>
> Out of curiosity, what's the motivation for doing pvusb in qemu?  Will
> there be an option, for instance, to switch usb devices from emulated
> over to pv when the magic port is written to (as is done with disks &
> network currently)?

Doing it in qemu was a request when I sent my patches to do it in the
kernel. :-)

Doing pvusb in qemu really has advantages. Just by using a newer Xen
you'll have pvusb backend support in Dom0. No need for a kernel update.

Up to now I have no plan doing a switch of emulated USB devices to pv.
TBH I don't think this is an important scenario. In case someone wants
to add support for it I won't object. :-)


Juergen

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 6/6] refactor codes to unify pvusb and qemu emulated usb
  2015-04-19  3:50 ` [PATCH V3 6/6] refactor codes to unify pvusb and qemu emulated usb Chunyan Liu
@ 2015-05-21 14:17   ` George Dunlap
  0 siblings, 0 replies; 53+ messages in thread
From: George Dunlap @ 2015-05-21 14:17 UTC (permalink / raw)
  To: Chunyan Liu
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao

On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> Now we have pvusb implementation. To merge with future qemu emulated
> usb work, refactor codes:
>  - define 'protocol' (type) to indicate pvusb or qemu, add 'protocol' to
> usb controller and usb device structures, add 'type' to xl interface
> usb-attach|detach.
>  - extract common codes for both qemu and pvusb from libxl_pvusb.c to
> libxl_usb.c, and adjust libxl_pvusb.c codes accordingly.
>
> Signed-off-by: Chunyan Liu <cyliu@suse.com>

Thank you for doing this, Chunyan.

A couple of things.

First, I'm not really a fan of having a patch in a series go through
and rearrange things like this; it seems like it would be better to
fold these changes into patches 3-5 of this series.

Second, I've got a number of comments on the "refactoring" bit -- what
you move into libxl_usb.c and how.  But the refactoring bit isn't
really for your feature, and I'll probably have to adjust it anyway
when I actually go to add the qemu side; it seems a bit unfair and a
bit inefficient to ask you to redo the refactoring at this point.

On the other hand, I don't like the idea of checking in an interface
that I know I don't want set in stone, particularly with less than 2
months left until the code freeze.

So what about this: Take the interface-related changes from this patch
(with my comments) and move them back to the earlier patches in the
series; and then just put the minimum amount of code in those patches
to handle them.  The interface is an enum, so if there's only one
option, you should be able to assume that protocol==PV; and it should
be OK internally to assume that all controllers and devices are
actually PV.  Leave the file named "libxl_pvusb.c", but don't bother
(yet) adding a libxl_usb.c.  I'll do that when I add the HVM side of
things.

What do you think?

Further comments below...

> ---
>  tools/libxl/Makefile         |   2 +-
>  tools/libxl/libxl_internal.h |   4 +
>  tools/libxl/libxl_pvusb.c    | 169 +-------------------------------
>  tools/libxl/libxl_types.idl  |  11 +++
>  tools/libxl/libxl_usb.c      | 224 +++++++++++++++++++++++++++++++++++++++++++
>  tools/libxl/xl_cmdimpl.c     |  54 +++++++++--
>  tools/libxl/xl_cmdtable.c    |   4 +-
>  7 files changed, 293 insertions(+), 175 deletions(-)
>  create mode 100644 tools/libxl/libxl_usb.c
>
> diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile
> index d52281f..f786fcf 100644
> --- a/tools/libxl/Makefile
> +++ b/tools/libxl/Makefile
> @@ -95,7 +95,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \
>                         libxl_internal.o libxl_utils.o libxl_uuid.o \
>                         libxl_json.o libxl_aoutils.o libxl_numa.o libxl_vnuma.o \
>                         libxl_save_callout.o _libxl_save_msgs_callout.o \
> -                       libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o $(LIBXL_OBJS-y)
> +                       libxl_qmp.o libxl_event.o libxl_fork.o libxl_pvusb.o libxl_usb.o $(LIBXL_OBJS-y)
>  LIBXL_OBJS += libxl_genid.o
>  LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o
>
> diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
> index f426ed8..2ee058b 100644
> --- a/tools/libxl/libxl_internal.h
> +++ b/tools/libxl/libxl_internal.h
> @@ -2428,6 +2428,10 @@ _hidden int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
>  _hidden int libxl__device_usb_add(libxl__gc *gc, uint32_t domid,
>                              libxl_device_usb *usb);
>  _hidden int libxl__device_usb_destroy_all(libxl__gc *gc, uint32_t domid);
> +_hidden bool pv_is_usb_assigned(libxl__gc *gc, libxl_device_usb *usb);
> +_hidden int pv_usb_add(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb);
> +_hidden int pv_usb_remove(libxl__gc *gc, uint32_t domid, libxl_device_usb *usb);
> +_hidden int pv_usb_destroy_all(libxl__gc *gc, uint32_t domid);
>
>  /* Internal function to connect a vkb device */
>  _hidden int libxl__device_vkb_add(libxl__gc *gc, uint32_t domid,
> diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
> index 4e4975a..3dfd2bd 100644
> --- a/tools/libxl/libxl_pvusb.c
> +++ b/tools/libxl/libxl_pvusb.c
> @@ -16,8 +16,6 @@
>
>  #define USBBACK_INFO_PATH "/libxl/usbback"
>
> -#define USBHUB_CLASS_CODE 0x09
> -
>  static int libxl__device_usbctrl_setdefault(libxl__gc *gc, uint32_t domid,
>                                              libxl_device_usbctrl *usbctrl)
>  {
> @@ -338,6 +336,7 @@ int libxl_device_usbctrl_getinfo(libxl_ctx *ctx, uint32_t domid,
>      usbctrlinfo->devid = usbctrl->devid;
>      usbctrlinfo->ports = usbctrl->ports;
>      usbctrlinfo->version = usbctrl->version;
> +    usbctrlinfo->protocol = usbctrl->protocol;

Simillar to my comment on patch 3/6: this should only read
usbctrl->devid.  Here you can assume protocol=PV for the time being.

> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index a6db614..df56303 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -533,7 +533,15 @@ libxl_device_pci = Struct("device_pci", [
>      ("seize", bool),
>      ])
>
> +libxl_usb_protocol = Enumeration("usb_protocol", [
> +    (0, "AUTO"),
> +    (1, "PV"),
> +    (2, "DEVICEMODEL"),
> +    ], init_val = "LIBXL_USB_PROTOCOL_PV")

I think this should probably default to AUTO, and then have something
which changes AUTO to PV unconditionally (which I'll refactor later).

> +
> +
>  libxl_device_usbctrl = Struct("device_usbctrl", [
> +    ("protocol", libxl_usb_protocol),
>      ("devid", libxl_devid),
>      ("version", integer),
>      ("ports", integer),
> @@ -542,6 +550,7 @@ libxl_device_usbctrl = Struct("device_usbctrl", [
>     ])
>
>  libxl_device_usb = Struct("device_usb", [
> +    ("protocol", libxl_usb_protocol),
>      ("ctrl", libxl_devid),
>      ("port", integer),
>      ("busid", string),

I don't think we need a protocol for usb -- just for usbctrl.  I think
the protocol for usb should be defined by the controller it's using.

> @@ -626,6 +635,7 @@ libxl_vtpminfo = Struct("vtpminfo", [
>      ], dir=DIR_OUT)
>
>  libxl_usbctrlinfo = Struct("usbctrlinfo", [
> +    ("protocol", libxl_usb_protocol),
>      ("devid", libxl_devid),
>      ("version", integer),
>      ("ports", integer),
> @@ -640,6 +650,7 @@ libxl_usbctrlinfo = Struct("usbctrlinfo", [
>      ], dir=DIR_OUT)
>
>  libxl_usbinfo = Struct("usbinfo", [
> +    ("protocol", libxl_usb_protocol),
>      ("busnum", integer),
>      ("devnum", integer),
>      ("idVendor", integer),

Same here -- I don't think we need the protocol for "usbinfo" if we
have the controller id.

> diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
> index 750377f..f41ac6c 100644
> --- a/tools/libxl/xl_cmdimpl.c
> +++ b/tools/libxl/xl_cmdimpl.c
> @@ -3400,7 +3400,16 @@ int main_usbattach(int argc, char **argv)
>      }
>
>      while (argc > optind) {
> -        if (MATCH_OPTION("controller", argv[optind], oparg)) {
> +        if (MATCH_OPTION("type", argv[optind], oparg)) {
> +            if (!strcmp(oparg, "pv")) {
> +               usb.protocol = LIBXL_USB_PROTOCOL_PV;
> +            } else if (!strcmp(oparg, "qemu")) {
> +               usb.protocol = LIBXL_USB_PROTOCOL_DEVICEMODEL;
> +            } else {
> +               fprintf(stderr, "unrecognized type `%s'\n", oparg);
> +               exit(-1);
> +            }

We want to move this from main_usbattach to main_usbctrl_attach.  No
need to specify the type on detach -- libxl should be able to look
that up.

And here you want to use libxl_usb_protocol_from_string(), which is
generated automatically by the idl.


> +static const char *get_usb_protocol_string(libxl_usb_protocol type)
> +{
> +    switch (type) {
> +    case LIBXL_USB_PROTOCOL_PV:
> +        return "pv";
> +    case LIBXL_USB_PROTOCOL_DEVICEMODEL:
> +        return "qemu";
> +    default:
> +        return "invalid";
> +    }
> +}

And you don't need this, because the IDL will automatically generate
libxl_usb_protocol_to_string()...

>          if (!libxl_device_usbctrl_getinfo(ctx, domid,
>                                  &usbctrls[i], &usbctrlinfo)) {
> -            printf("%-6d %-3d %-5d %-7d %-5d %-30s\n",
> +            printf("%-6d %-6s %-3d %-5d %-7d %-5d %-30s\n",
>                      usbctrlinfo.devid,
> +                    get_usb_protocol_string(usbctrlinfo.protocol),
>                      usbctrlinfo.backend_id, usbctrlinfo.state,
>                      usbctrlinfo.version, usbctrlinfo.ports,
>                      usbctrlinfo.backend);

...which you can use here.

> @@ -3517,6 +3552,9 @@ int main_usblist(int argc, char **argv)
>      }
>
>      free(usbctrls);
> +
> +    /* TODO list qemu usb device info*/
> +

No need for this.

>      return 0;
>  }
>
> diff --git a/tools/libxl/xl_cmdtable.c b/tools/libxl/xl_cmdtable.c
> index da30ae0..4054911 100644
> --- a/tools/libxl/xl_cmdtable.c
> +++ b/tools/libxl/xl_cmdtable.c
> @@ -557,12 +557,12 @@ struct cmd_spec cmd_table[] = {
>      { "usb-attach",
>        &main_usbattach, 1, 2,
>        "Attach a USB device to a domain",
> -      "<Domain> <bus.addr> [controller=<DevId> [port=<port>]]",
> +      "<Domain> <bus.addr> [type=pv|qemu] [controller=<DevId> [port=<port>]]",

I think for now you should just leave this as "pv".  Only advertise
types that are actually supported.

 -George

^ permalink raw reply	[flat|nested] 53+ messages in thread

* 答复: Re:  [PATCH V3 3/6] libxl: add pvusb API
  2015-05-19 18:06   ` George Dunlap
  2015-05-19 18:24     ` Ian Campbell
@ 2015-06-08  9:06     ` Chun Yan Liu
  1 sibling, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-06-08  9:06 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao



>>> 在 2:06 的 2015/5/20 上,在讯息
<CAFLBxZZHFFWo6Tm7602y3+x5kX65-w4obFS1VdVb8KqnzdAmDg@mail.gmail.com> 中,George
Dunlap <George.Dunlap@eu.citrix.com> 写入:
> On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> > Add pvusb APIs, including:
> >  - attach/detach (create/destroy) virtual usb controller.
> >  - attach/detach usb device
> >  - list usb controller and usb devices
> >  - some other helper functions
> >
> > Signed-off-by: Chunyan Liu <cyliu@suse.com>
> > Signed-off-by: Simon Cao <caobosimon@gmail.com>
> 
> OK, getting closer. :-)
> 
> A number of comments:

Hi, George, thanks very much!
I'm so sorry for missing the message and not reply immediately.
Before sending new version, I'm answering some of your questions here.
And there are a couple of comments, I still have some hesitation to follow.
All others, I agree and will update as you suggested.

> 
> > diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
> > index 6bc75c5..cbe3519 100644
> > --- a/tools/libxl/libxl.h
> > +++ b/tools/libxl/libxl.h
> > @@ -114,6 +114,12 @@
> >  #define LIBXL_HAVE_DOMAIN_NODEAFFINITY 1
> >
> >  /*
> > + * LIBXL_HAVE_PVUSB indicates the functions for doing hot-plug of
> > + * USB devices through pvusb.
> > + */
> > +#define LIBXL_HAVE_PVUSB 1
> 
> It seems like we should document somewhere how we expect these to be
> used -- namely the connection between usbctrl and usb devices.  In
> particular, that you can either add a usbctrl, and then add a usb
> device to it specifically; or if you don't specify a usbctrl when
> calling usb_add, it will find the most reasonable one to add it to,
> even creating one for you if you didn't have one.
> 
> 
> > diff --git a/tools/libxl/libxl_pvusb.c b/tools/libxl/libxl_pvusb.c
> > new file mode 100644
> > index 0000000..4e4975a
> > --- /dev/null
> > +++ b/tools/libxl/libxl_pvusb.c
> 
> > +int libxl__device_usbctrl_add(libxl__gc *gc, uint32_t domid,
> > +                              libxl_device_usbctrl *usbctrl)
> > +{
> > +    int rc = 0;
> > +
> > +    rc = libxl__device_usbctrl_setdefault(gc, domid, usbctrl);
> > +    if (rc < 0) goto out;
> > +
> > +    if (usbctrl->devid == -1) {
> 
> Hmm, I was doing to say that this shouldn't be a magic constant, but
> it looks like that's what all the other devices do :-/
> 
> > +static bool is_usb_in_array(libxl_device_usb *usbs, int num,
> > +                            libxl_device_usb *usb)
> > +{
> > +    int i;
> > +
> > +    for (i = 0; i < num; i++) {
> > +        if (!strcmp(usbs[i].busid, usb->busid) )
> 
> Here (and elsewhere) you should probably use the COMPARE_USB() macro
> you define in this patch.
> 
> > +/* check if USB device type is assignable */
> > +static bool is_usb_assignable(libxl__gc *gc, libxl_device_usb *usb)
> > +{
> > +    libxl_ctx *ctx = libxl__gc_owner(gc);
> > +    int classcode;
> > +    char *filename;
> > +    void *buf;
> > +
> > +    assert(usb->busid);
> > +
> > +    filename = GCSPRINTF(SYSFS_USB_DEV"/%s/bDeviceClass", usb->busid);
> > +    if (libxl_read_file_contents(ctx, filename, &buf, NULL) < 0)
> > +        return false;
> > +
> > +    sscanf(buf, "%x", &classcode);
> > +    if (classcode == USBHUB_CLASS_CODE)
> > +        return false;
> > +
> > +    return true;
> 
> Just checking, is it really the case that *all* USB classes are
> assignable, *except* for hubs?

Referring to xm pvusb implementation, only HUB is excluded, so I
just keep the same.

> 
> This is a minor thing, but this might be simper to do this:
> 
>  return classcode != USBHUB_CLASS_CODE;
> 
> > +/* get usb devices under certain usb controller */
> > +static int libxl__device_usb_list(libxl__gc *gc, uint32_t domid, int 
> usbctrl,
> > +                                  libxl_device_usb **usbs, int *num)
> > +{
> > +    char *be_path, *num_devs;
> > +    int n, i;
> > +
> > +    *usbs = NULL;
> > +    *num = 0;
> > +
> > +    be_path = GCSPRINTF("%s/backend/vusb/%d/%d",
> > +                        libxl__xs_get_dompath(gc, 0), domid, usbctrl);
> > +    num_devs = libxl__xs_read(gc, XBT_NULL,
> > +                              GCSPRINTF("%s/num-ports", be_path));
> > +    if (!num_devs)
> > +        return 0;
> > +
> > +    n = atoi(num_devs);
> > +    *usbs = calloc(n, sizeof(libxl_device_usb));
> > +
> > +    for (i = 0; i < n; i++) {
> > +        char *busid;
> > +        libxl_device_usb *usb = NULL;
> > +
> > +        busid = libxl__xs_read(gc, XBT_NULL,
> > +                               GCSPRINTF("%s/port/%d", be_path, i + 1));
> > +        if (busid && strcmp(busid, "")) {
> > +            usb = *usbs + *num;
> > +            usb->ctrl = usbctrl;
> > +            usb->port = i + 1;
> > +            usb->busid = strdup(busid);
> 
> This needs to populate the hostbus / hostaddr as well; busid is pretty
> useless to users / external callers.

I thought about that when implementing, but finally not added to codes considering:
* for all libxl pvusb internal usage, busid is enough.
* for toolstack usage, if we want to expose users useful information about bus:addr,
vendorID:devieID, we have libxl_device_usb_getinfo API, with this API callers can get
all information including hostbus, hostaddr.

If that couldn't satisfy qemu usage, I can add a translating to fill in hostbus and
hostaddr too.

> 
> > +/* get all usb devices of the domain */
> > +static libxl_device_usb *
> > +libxl_device_usb_list_all(libxl__gc *gc, uint32_t domid, int *num)
> > +{
> > +    char **usbctrls;
> > +    unsigned int nd, i, j;
> > +    char *be_path;
> > +    int rc;
> > +    libxl_device_usb *usbs = NULL;
> > +
> > +    *num = 0;
> > +
> > +    be_path = GCSPRINTF("/local/domain/%d/backend/vusb/%d",
> > +                        LIBXL_TOOLSTACK_DOMID, domid);
> > +    usbctrls = libxl__xs_directory(gc, XBT_NULL, be_path, &nd);
> > +
> > +    for (i = 0; i < nd; i++) {
> > +        int nc = 0;
> > +        libxl_device_usb *tmp = NULL;
> > +        rc = libxl__device_usb_list(gc, domid, atoi(usbctrls[i]), &tmp, 
> &nc);
> > +        if (!nc) continue;
> > +
> > +        usbs = realloc(usbs, sizeof(libxl_device_usb)*((*num) + nc));
> > +        for(j = 0; j < nc; j++) {
> > +            usbs[*num].ctrl = tmp[j].ctrl;
> > +            usbs[*num].port = tmp[j].port;
> > +            usbs[*num].busid = strdup(tmp[j].busid);
> 
> This needs to copy the hostaddr and busaddr as well, as these are
> primarily what an external caller will want.

Same to above.

> 
> > +/* find first unused controller:port and give that to usb device */
> > +static int
> > +libxl__device_usb_set_default_usbctrl(libxl__gc *gc, uint32_t domid,
> > +                                      libxl_device_usb *usb)
> > +{
> > +    libxl_ctx *ctx = CTX;
> > +    libxl_device_usbctrl *usbctrls;
> > +    libxl_device_usb *usbs = NULL;
> > +    int numctrl, numusb, i, j, rc = -1;
> > +    char *be_path, *tmp;
> > +
> > +    usbctrls = libxl_device_usbctrl_list(ctx, domid, &numctrl);
> > +    if ( !numctrl)
> > +        goto out;
> > +
> > +    for (i = 0; i < numctrl; i++) {
> > +        rc = libxl__device_usb_list(gc, domid, usbctrls[i].devid,
> > +                                    &usbs, &numusb);
> > +        if (rc) continue;
> > +
> > +        if (!usbctrls[i].ports || numusb == usbctrls[i].ports)
> > +            continue;
> > +
> > +        for (j = 1; i <= numusb; j++) {
> 
> Port numbers start at 1, do they?  Interesting...

Keep same as xm implementation. I think that should be proved right.

> 
> Er, but isn't the middle thing just plain wrong?  For one, you want to
> be comparing j not i.  I can't see that i is updated inside the loop,
> so ATM this will loop forever.
> 
> For two, you want to compare to usbctrls[i].ports (the total number of
> ports), not to numusb (the number of currently assigned devices).
> 
> > +static int libxl__device_usb_setdefault(libxl__gc *gc, uint32_t domid,
> > +                                        libxl_device_usb *usb)
> > +{
> > +    char *be_path, *tmp;
> > +
> > +    if (usb->ctrl == -1) {
> 
> Oh, right -- libxl_devid's default to -1, don't they?  I'll save the
> grumbling about the lack of a #define here then. (Or rather, I'll
> grumble at the library rather than you.)
> 
> > +        int ret = libxl__device_usb_set_default_usbctrl(gc, domid, usb);
> > +        /* If no existing ctrl to host this usb device, setup a new one */
> > +        if (ret) {
> > +            libxl_device_usbctrl usbctrl;
> > +            libxl_device_usbctrl_init(&usbctrl);
> > +            libxl__device_usbctrl_add(gc, domid, &usbctrl);
> 
> I think in the previous round I asked what would happen if this
> failed, and you said you would fail later, but that you'd change it to
> check for an error here and bail out earlier.
> 
> > +/* Cann't write '.' or ':' into Xenstore as key. So, change '.' to '_',
> > + * change ':' to '-'.
> > + */
> > +static char *usb_interface_encode(char *busid)
> 
> Maybe usb_interface_xenstore_encode, to make it clear that you're just
> encoding it to store in xenstore?
> 
> > +static int usbback_dev_unassign(libxl__gc *gc, libxl_device_usb *usb)
> > +{
> > +    char **intfs = NULL;
> > +    char *path;
> > +    int num = 0, i;
> > +    int rc = 0;
> > +    char *usb_encode = NULL;
> > +
> > +    if (usb_get_all_interfaces(gc, usb, &intfs, &num) < 0)
> > +        return ERROR_FAIL;
> > +
> > +    usb_encode = usb_interface_encode(usb->busid);
> > +
> > +    for (i = 0; i < num; i++){
> > +        char *intf = intfs[i];
> > +        char *drvpath = NULL;
> > +
> > +        if (usb_intf_is_assigned(gc, intf) > 0) {
> > +            /* unbind interface from usbback driver */
> > +            if (unbind_usb_intf(gc, intf, NULL) < 0) {
> > +                rc = ERROR_FAIL;
> > +                goto out;
> > +            }
> > +        }
> > +
> > +        /* bind interface to its originial driver */
> > +        drvpath = libxl__xs_read(gc, XBT_NULL,
> > +                  GCSPRINTF(USBBACK_INFO_PATH"/%s/%s/driver_path",
> > +                  usb_encode, usb_interface_encode(intf)));
> > +        if (drvpath) {
> > +            if (bind_usb_intf(gc, intf, drvpath) < 0) {
> > +                LOGE(ERROR, "Couldn't bind %s to %s", intf, drvpath);
> > +                rc = ERROR_FAIL;
> > +                goto out;
> 
> I think this function probably shouldn't fail if it can't re-bind to
> the original driver.  If nothing else, this will be bad because now
> the USB device has at least one of its interfaces unbound from
> usbback, but the other ones still bound to it.  All interfaces should
> be unbound from usbback regardless.
> 
> I also think that while a warning should be logged, that the function
> as a whole should return success if it managed to unbind, even if it
> didn't manage to rebind.  (But feel free to argue otherwise.)
> 
> > diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> > index 0866433..a6db614 100644
> > --- a/tools/libxl/libxl_types.idl
> > +++ b/tools/libxl/libxl_types.idl
> > @@ -533,6 +533,22 @@ libxl_device_pci = Struct("device_pci", [
> >      ("seize", bool),
> >      ])
> >
> > +libxl_device_usbctrl = Struct("device_usbctrl", [
> > +    ("devid", libxl_devid),
> > +    ("version", integer),
> > +    ("ports", integer),
> > +    ("backend_domid", libxl_domid),
> > +    ("backend_domname", string),
> > +   ])
> > +
> > +libxl_device_usb = Struct("device_usb", [
> > +    ("ctrl", libxl_devid),
> > +    ("port", integer),
> > +    ("busid", string),
> 
> So it looks like "busid" is purely an internal string that you're
> putting in the device struct for convenience, so that you don't have
> to either keep translating bus:addr into the sysfs node, or pass
> around the second string as well.  Is that right?

Yes, right. Actually in libxl pvusb, internally it uses busid, hostbus:hostaddr
is only a toolstack interface.

It is 'busid' saved in xenstore for pvusb driver's usage, and everywhere comparing
work (e.g. check usb device existence before removing a usb device) it uses
'busid' to compare. So having a struct element is very convenient, otherwise, there
are many places that need to translate from busid to/from bus:addr.

Personally I still prefer to keep 'busid', but if that's really unacceptable,
I can also update.

- Chunyan

> 
> I think libxl does something similar to this with "backend_domname"
> and "backend_domid"; but in that case I believe you *can* specify the
> backend_domid if you want to.  In this case, you're exposing a struct
> element which is clobbered immediately, at least by usb_add and
> usb_remove.
> 
> Maintainers, what do you think?
> 
> Other than that, I think this patch as-is looks in pretty good shape
> (haven't taken a close look at the rest in the series yet); the
> primary things are making the "hostbus:hostaddr" actually the primary
> interface; at a minimum filling in that information when doing
> queries.  I'd ideally getting rid of "busid" from the external
> interface altogether.
> 
>  -George
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org 
> http://lists.xen.org/xen-devel 
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 53+ messages in thread

* 答复: Re:  [PATCH V3 3/6] libxl: add pvusb API
  2015-05-19 18:16   ` George Dunlap
@ 2015-06-08 11:15     ` Chun Yan Liu
  0 siblings, 0 replies; 53+ messages in thread
From: Chun Yan Liu @ 2015-06-08 11:15 UTC (permalink / raw)
  To: George Dunlap
  Cc: Lars Kurth, Wei Liu, Ian Campbell, xen-devel, Ian Jackson, Simon Cao



>>> 在 2:16 的 2015/5/20 上,在讯息
<CAFLBxZaw3zCsQ-8tJPH+3cVG4xS-J15_NA930hcRy8x4gRbinw@mail.gmail.com> 中,George
Dunlap <George.Dunlap@eu.citrix.com> 写入:
> On Sun, Apr 19, 2015 at 4:50 AM, Chunyan Liu <cyliu@suse.com> wrote:
> > +int libxl_device_usb_getinfo(libxl_ctx *ctx, char *busid, libxl_usbinfo 
> *usbinfo)
> > +{
> 
> Sorry, missed something.  This function is wrong.  It gives you
> information about a *host* USB device; but should properly give you
> information about a *guest* USB device.  The
> libxl_device_disk_getinfo() counterpart, for example, takes a domid
> and a virtual device (from within a libxl_disk structure) and returns
> information about that virtual device.

Maybe the name should be changed as libxl_device_usb_host_info.
Returning *host* USB device info itself is what we need, should not changed.

<ctrl, port> is the most important information to the virtual device. But like
in 'xl list', it will first list USB controller, and then each port, USB device
appears in some port, it doesn't need to show <ctrl, port> info again, but
host information is more useful. e.g.:
#xl usb-list test
Devid Type BE state usb-ver ports BE-path
0        pv     0     4      1         4      /local/domain/0/backend/vusb/1/0
    port 1: Bus 003 Device 005 ID:  8087:07dc Intel Corp.


- Chunyan


> 
> Hrm... which makes me wonder whether we should use <ctrl,port> as a
> unique "handle" for usb devices in the interface, or have a separate
> devid for the devices themselves.
> 
>  -George
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 53+ messages in thread

* Re: [PATCH V3 3/6] libxl: add pvusb API
  2015-05-20  9:04       ` Wei Liu
  2015-05-20  9:12         ` Ian Campbell
  2015-05-20  9:30         ` Chun Yan Liu
@ 2015-06-11 16:54         ` Ian Jackson
  2 siblings, 0 replies; 53+ messages in thread
From: Ian Jackson @ 2015-06-11 16:54 UTC (permalink / raw)
  To: Wei Liu
  Cc: lars.kurth, ian.campbell, george.dunlap, Chun Yan Liu, xen-devel,
	caobosimon

Wei Liu writes ("Re: [PATCH V3 3/6] libxl: add pvusb API"):
> On Mon, May 18, 2015 at 09:20:43PM -0600, Chun Yan Liu wrote:
> > Thanks, I'm updating that. But maybe like pci_add and pci_remove functions,
> > will add a 'starting' flag to indicate hotplug or creation.
> > Looking at DEFINE_DEVICE_ADD and DEFINE_DEVICE_REMOVE, usbctrl_add
> > and usb_add can use DEFINE_DEVICE_ADD; but usbctrl_remove and usb_remove
> > cannot use DEFINE_DEVICE_REMOVE directly, need some extra handling. So,
> > finally turned to not use these macros but refer to pci functions.
> 
> TBH I prefer to have only one way to deal with devices.  I personally
> prefer the async style that every other devices use. Maybe that's just
> because I mostly worked with those.
> 
> I don't know the full history of libxl_pci.c so I will wait for Ian and
> Ian's comments on which way to go.

libxl_pci.c is rather unfortunate in a few respects.  It does a number
of things things synchronously that it shouldn't.

Certainly the external API for all device add/remove functions should
be async.

Ian.

^ permalink raw reply	[flat|nested] 53+ messages in thread

end of thread, other threads:[~2015-06-11 16:54 UTC | newest]

Thread overview: 53+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-04-19  3:50 [PATCH V3 0/6] libxl pvusb toolstack work Chunyan Liu
2015-04-19  3:50 ` [PATCH V3 1/6] libxl: export some functions for pvusb use Chunyan Liu
2015-04-20 16:25   ` Olaf Hering
2015-05-18 13:34     ` George Dunlap
2015-05-18 14:05   ` Wei Liu
2015-04-19  3:50 ` [PATCH V3 2/6] libxl_read_file_contents: fix reading sysfs file Chunyan Liu
2015-05-18 14:23   ` Ian Jackson
2015-05-18 14:28     ` Ian Campbell
2015-05-18 14:30     ` Wei Liu
2015-05-19  3:21       ` Chun Yan Liu
2015-05-18 14:25   ` Wei Liu
2015-04-19  3:50 ` [PATCH V3 3/6] libxl: add pvusb API Chunyan Liu
2015-04-20  5:53   ` Juergen Gross
2015-05-18 13:55   ` Olaf Hering
2015-05-18 18:07   ` Wei Liu
2015-05-19  3:20     ` Chun Yan Liu
2015-05-19 10:20       ` George Dunlap
2015-05-19 11:31         ` Jürgen Groß
2015-05-20  9:04       ` Wei Liu
2015-05-20  9:12         ` Ian Campbell
2015-05-20  9:30         ` Chun Yan Liu
2015-06-11 16:54         ` Ian Jackson
2015-05-19 18:06   ` George Dunlap
2015-05-19 18:24     ` Ian Campbell
2015-06-08  9:06     ` 答复: " Chun Yan Liu
2015-05-19 18:16   ` George Dunlap
2015-06-08 11:15     ` 答复: " Chun Yan Liu
2015-05-19 18:20   ` George Dunlap
2015-04-19  3:50 ` [PATCH V3 4/6] xl: add pvusb commands Chunyan Liu
2015-04-20  8:12   ` Juergen Gross
2015-04-21  2:21     ` Chun Yan Liu
2015-05-20 14:20     ` George Dunlap
2015-05-20 14:33       ` Juergen Gross
2015-05-20 14:41         ` George Dunlap
2015-05-20 14:55           ` Juergen Gross
2015-05-20 15:25             ` George Dunlap
2015-05-21  3:35               ` Juergen Gross
2015-05-21 10:37                 ` George Dunlap
2015-05-21 10:52                   ` Juergen Gross
2015-05-21 11:11                     ` George Dunlap
2015-05-21 11:58                       ` Juergen Gross
2015-05-21 13:01                         ` George Dunlap
2015-05-21 13:08                           ` Juergen Gross
2015-05-21 13:43                             ` George Dunlap
2015-05-21 13:55                               ` Juergen Gross
2015-05-21 14:00                                 ` George Dunlap
2015-05-21 14:14                                   ` Juergen Gross
2015-05-20 14:23   ` George Dunlap
2015-05-20 15:46     ` George Dunlap
2015-05-20 15:55   ` George Dunlap
2015-04-19  3:50 ` [PATCH V3 5/6] domcreate: support pvusb in configuration file Chunyan Liu
2015-04-19  3:50 ` [PATCH V3 6/6] refactor codes to unify pvusb and qemu emulated usb Chunyan Liu
2015-05-21 14:17   ` George Dunlap

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.