All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH RFC 0/9] basic channel IO passthrough infrastructure based on vfio
@ 2016-04-29 12:13 ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

The patch series introduce a basic channel I/O passthrough
infrastructure based on vfio.
- Focus on supporting dasd-eckd(cu_type/dev_type = 0x3990/0x3390) as
  the target device. 
- Support new qemu parameters in the style of:
   "-machine s390-ccw-virtio(,s390-map-css=on|off) ... 
    -device vfio-ccw,id=xx,hostid=xx(,guestid=xx)".
  We want to support real (i.e. not virtual) channel devices even for
  guests that do not support MCSS-E (where guests may see devices from
  any channel subsystem image at once). As all virtio-ccw devices are in
  css 0xfe (and show up in the default css 0 for guests not activating
  MCSS-E), we need an option to map e.g. passed-through channel devices
  from their real css (0-3, or 0 for hosts not activating MCSS-E) into
  the default css, that is what the new machine option s390-map-css is
  added.
- According to the user-space ccw format, which was introduced by the
  vfio-ccw patches of the kernel part, assemble the ccw program in qemu.
  This 'ccw program' would then be transported to the kernel via a
  dedicated ioctl for further processing(e.g. translation in kernel
  space), and issuing it to the real device.
  Overview of the user-space ccw translation(translate 'guest ccw
  program' to 'user-space ccw program'):
  1. Allocate a 4K memory buffer in user-space to store all of the ccw
     program information.
     * Lower 2k of the buffer are used to store a maximum of 256 ccws.
       These ccws are copied from 'guest ccw program' and placed one
       after another.
     * Upper 2k of the buffer are used to store a maximum of 256
       corresponding cda data sets, each having a length of 8 bytes.
  2. For TIC ccw.
     * Locate the TIC target ccw inside the ccw area, and calculate its
       offset.
     * Store the offset to ccw.cda.
  3. For Direct ccw.
     * Find the cda entry with the same index as the ccw.
     * Store the user virtual address of the original ccw.cda to the cda
       entry.
     * Store the offset of the cda entry to ccw.cda.
  4. For IDAL ccw.
     * Find the cda entry with the same index as the ccw.
     * Prepare the user-space idaws. Store the virtual address of the
       idaws to the cda entry.
     * Store the offset of the cda entry to ccw.cda.
  5. Append a NOOP to the chain end.
  6. Expectations for the user-space ccw program I/O result:
     CPA of SCSW should be set to the offset of the ccw area of the
     current ccw.

Xiao Feng Ren (9):
  vfio: linux-headers update for vfio-ccw
  vfio: No-IOMMU mode support
  s390x/css: introduce ccw chain interfaces
  s390x/css: add s390-map-css machine option
  s390x/css: realize css_sch_build_schib
  s390x/css: device and bus support for s390-ccw passthrough
  vfio/ccw: vfio based ccw passthrough driver
  s390x/css: introduce and realize ccw-request callback
  s390x/css: ccws translation infrastructure

 default-configs/s390x-softmmu.mak  |   1 +
 hw/s390x/Makefile.objs             |   1 +
 hw/s390x/css.c                     | 283 ++++++++++++++++++++++--
 hw/s390x/css.h                     |  12 +
 hw/s390x/s390-ccw-bus.c            | 114 ++++++++++
 hw/s390x/s390-ccw-bus.h            |  30 +++
 hw/s390x/s390-ccw.c                | 189 ++++++++++++++++
 hw/s390x/s390-ccw.h                |  41 ++++
 hw/s390x/s390-ccwchain.c           | 441 +++++++++++++++++++++++++++++++++++++
 hw/s390x/s390-ccwchain.h           |  28 +++
 hw/s390x/s390-virtio-ccw.c         |  26 +++
 hw/s390x/virtio-ccw.c              |   1 +
 hw/s390x/virtio-ccw.h              |   2 -
 hw/vfio/Makefile.objs              |   1 +
 hw/vfio/ccw.c                      | 230 +++++++++++++++++++
 hw/vfio/common.c                   |  66 ++++--
 include/hw/s390x/s390-virtio-ccw.h |   1 +
 include/hw/vfio/vfio-common.h      |   3 +
 linux-headers/linux/vfio.h         |  32 +++
 qemu-options.hx                    |   6 +-
 target-s390x/cpu.h                 |  10 +
 target-s390x/ioinst.c              |   9 +
 22 files changed, 1493 insertions(+), 34 deletions(-)
 create mode 100644 hw/s390x/s390-ccw-bus.c
 create mode 100644 hw/s390x/s390-ccw-bus.h
 create mode 100644 hw/s390x/s390-ccw.c
 create mode 100644 hw/s390x/s390-ccw.h
 create mode 100644 hw/s390x/s390-ccwchain.c
 create mode 100644 hw/s390x/s390-ccwchain.h
 create mode 100644 hw/vfio/ccw.c

-- 
2.6.6

^ permalink raw reply	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 0/9] basic channel IO passthrough infrastructure based on vfio
@ 2016-04-29 12:13 ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

The patch series introduce a basic channel I/O passthrough
infrastructure based on vfio.
- Focus on supporting dasd-eckd(cu_type/dev_type = 0x3990/0x3390) as
  the target device. 
- Support new qemu parameters in the style of:
   "-machine s390-ccw-virtio(,s390-map-css=on|off) ... 
    -device vfio-ccw,id=xx,hostid=xx(,guestid=xx)".
  We want to support real (i.e. not virtual) channel devices even for
  guests that do not support MCSS-E (where guests may see devices from
  any channel subsystem image at once). As all virtio-ccw devices are in
  css 0xfe (and show up in the default css 0 for guests not activating
  MCSS-E), we need an option to map e.g. passed-through channel devices
  from their real css (0-3, or 0 for hosts not activating MCSS-E) into
  the default css, that is what the new machine option s390-map-css is
  added.
- According to the user-space ccw format, which was introduced by the
  vfio-ccw patches of the kernel part, assemble the ccw program in qemu.
  This 'ccw program' would then be transported to the kernel via a
  dedicated ioctl for further processing(e.g. translation in kernel
  space), and issuing it to the real device.
  Overview of the user-space ccw translation(translate 'guest ccw
  program' to 'user-space ccw program'):
  1. Allocate a 4K memory buffer in user-space to store all of the ccw
     program information.
     * Lower 2k of the buffer are used to store a maximum of 256 ccws.
       These ccws are copied from 'guest ccw program' and placed one
       after another.
     * Upper 2k of the buffer are used to store a maximum of 256
       corresponding cda data sets, each having a length of 8 bytes.
  2. For TIC ccw.
     * Locate the TIC target ccw inside the ccw area, and calculate its
       offset.
     * Store the offset to ccw.cda.
  3. For Direct ccw.
     * Find the cda entry with the same index as the ccw.
     * Store the user virtual address of the original ccw.cda to the cda
       entry.
     * Store the offset of the cda entry to ccw.cda.
  4. For IDAL ccw.
     * Find the cda entry with the same index as the ccw.
     * Prepare the user-space idaws. Store the virtual address of the
       idaws to the cda entry.
     * Store the offset of the cda entry to ccw.cda.
  5. Append a NOOP to the chain end.
  6. Expectations for the user-space ccw program I/O result:
     CPA of SCSW should be set to the offset of the ccw area of the
     current ccw.

Xiao Feng Ren (9):
  vfio: linux-headers update for vfio-ccw
  vfio: No-IOMMU mode support
  s390x/css: introduce ccw chain interfaces
  s390x/css: add s390-map-css machine option
  s390x/css: realize css_sch_build_schib
  s390x/css: device and bus support for s390-ccw passthrough
  vfio/ccw: vfio based ccw passthrough driver
  s390x/css: introduce and realize ccw-request callback
  s390x/css: ccws translation infrastructure

 default-configs/s390x-softmmu.mak  |   1 +
 hw/s390x/Makefile.objs             |   1 +
 hw/s390x/css.c                     | 283 ++++++++++++++++++++++--
 hw/s390x/css.h                     |  12 +
 hw/s390x/s390-ccw-bus.c            | 114 ++++++++++
 hw/s390x/s390-ccw-bus.h            |  30 +++
 hw/s390x/s390-ccw.c                | 189 ++++++++++++++++
 hw/s390x/s390-ccw.h                |  41 ++++
 hw/s390x/s390-ccwchain.c           | 441 +++++++++++++++++++++++++++++++++++++
 hw/s390x/s390-ccwchain.h           |  28 +++
 hw/s390x/s390-virtio-ccw.c         |  26 +++
 hw/s390x/virtio-ccw.c              |   1 +
 hw/s390x/virtio-ccw.h              |   2 -
 hw/vfio/Makefile.objs              |   1 +
 hw/vfio/ccw.c                      | 230 +++++++++++++++++++
 hw/vfio/common.c                   |  66 ++++--
 include/hw/s390x/s390-virtio-ccw.h |   1 +
 include/hw/vfio/vfio-common.h      |   3 +
 linux-headers/linux/vfio.h         |  32 +++
 qemu-options.hx                    |   6 +-
 target-s390x/cpu.h                 |  10 +
 target-s390x/ioinst.c              |   9 +
 22 files changed, 1493 insertions(+), 34 deletions(-)
 create mode 100644 hw/s390x/s390-ccw-bus.c
 create mode 100644 hw/s390x/s390-ccw-bus.h
 create mode 100644 hw/s390x/s390-ccw.c
 create mode 100644 hw/s390x/s390-ccw.h
 create mode 100644 hw/s390x/s390-ccwchain.c
 create mode 100644 hw/s390x/s390-ccwchain.h
 create mode 100644 hw/vfio/ccw.c

-- 
2.6.6

^ permalink raw reply	[flat|nested] 22+ messages in thread

* [PATCH RFC 1/9] vfio: linux-headers update for vfio-ccw
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

This is a placeholder for a linux-headers update.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 linux-headers/linux/vfio.h | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 759b850..4059e3e 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -198,6 +198,7 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_PCI	(1 << 1)	/* vfio-pci device */
 #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2)	/* vfio-platform device */
 #define VFIO_DEVICE_FLAGS_AMBA  (1 << 3)	/* vfio-amba device */
+#define VFIO_DEVICE_FLAGS_CCW   (1 << 4)	/* vfio-ccw device */
 	__u32	num_regions;	/* Max region index + 1 */
 	__u32	num_irqs;	/* Max IRQ index + 1 */
 };
@@ -686,6 +687,37 @@ struct vfio_iommu_spapr_tce_remove {
 };
 #define VFIO_IOMMU_SPAPR_TCE_REMOVE	_IO(VFIO_TYPE, VFIO_BASE + 20)
 
+/**
+ * VFIO_DEVICE_CCW_HOT_RESET - _IOW(VFIO_TYPE, VFIO_BASE + 21)
+ *
+ * Hot reset the channel I/O device. All state of the subchannel will be
+ * cleared.
+ */
+#define VFIO_DEVICE_CCW_HOT_RESET       _IO(VFIO_TYPE, VFIO_BASE + 21)
+
+/**
+ * VFIO_DEVICE_CCW_CMD_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 22,
+ *                                     struct vfio_ccw_cmd)
+ *
+ * Issue a user-space ccw program for translation and performing channel I/O
+ * operations.
+ */
+struct vfio_ccw_cmd {
+        __u32 argsz;
+        __u8 cssid;
+        __u8 ssid;
+        __u16 devno;
+#define ORB_AREA_SIZE 12
+        __u8 orb_area[ORB_AREA_SIZE];
+#define SCSW_AREA_SIZE 12
+        __u8 scsw_area[SCSW_AREA_SIZE];
+#define IRB_AREA_SIZE 96
+        __u8 irb_area[IRB_AREA_SIZE];
+        __u32 ccwchain_nr;
+        __u64 ccwchain_buf;
+} __attribute__((packed));
+#define VFIO_DEVICE_CCW_CMD_REQUEST    _IO(VFIO_TYPE, VFIO_BASE + 22)
+
 /* ***************************************************************** */
 
 #endif /* VFIO_H */
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 1/9] vfio: linux-headers update for vfio-ccw
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

This is a placeholder for a linux-headers update.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 linux-headers/linux/vfio.h | 32 ++++++++++++++++++++++++++++++++
 1 file changed, 32 insertions(+)

diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 759b850..4059e3e 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -198,6 +198,7 @@ struct vfio_device_info {
 #define VFIO_DEVICE_FLAGS_PCI	(1 << 1)	/* vfio-pci device */
 #define VFIO_DEVICE_FLAGS_PLATFORM (1 << 2)	/* vfio-platform device */
 #define VFIO_DEVICE_FLAGS_AMBA  (1 << 3)	/* vfio-amba device */
+#define VFIO_DEVICE_FLAGS_CCW   (1 << 4)	/* vfio-ccw device */
 	__u32	num_regions;	/* Max region index + 1 */
 	__u32	num_irqs;	/* Max IRQ index + 1 */
 };
@@ -686,6 +687,37 @@ struct vfio_iommu_spapr_tce_remove {
 };
 #define VFIO_IOMMU_SPAPR_TCE_REMOVE	_IO(VFIO_TYPE, VFIO_BASE + 20)
 
+/**
+ * VFIO_DEVICE_CCW_HOT_RESET - _IOW(VFIO_TYPE, VFIO_BASE + 21)
+ *
+ * Hot reset the channel I/O device. All state of the subchannel will be
+ * cleared.
+ */
+#define VFIO_DEVICE_CCW_HOT_RESET       _IO(VFIO_TYPE, VFIO_BASE + 21)
+
+/**
+ * VFIO_DEVICE_CCW_CMD_REQUEST - _IOWR(VFIO_TYPE, VFIO_BASE + 22,
+ *                                     struct vfio_ccw_cmd)
+ *
+ * Issue a user-space ccw program for translation and performing channel I/O
+ * operations.
+ */
+struct vfio_ccw_cmd {
+        __u32 argsz;
+        __u8 cssid;
+        __u8 ssid;
+        __u16 devno;
+#define ORB_AREA_SIZE 12
+        __u8 orb_area[ORB_AREA_SIZE];
+#define SCSW_AREA_SIZE 12
+        __u8 scsw_area[SCSW_AREA_SIZE];
+#define IRB_AREA_SIZE 96
+        __u8 irb_area[IRB_AREA_SIZE];
+        __u32 ccwchain_nr;
+        __u64 ccwchain_buf;
+} __attribute__((packed));
+#define VFIO_DEVICE_CCW_CMD_REQUEST    _IO(VFIO_TYPE, VFIO_BASE + 22)
+
 /* ***************************************************************** */
 
 #endif /* VFIO_H */
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 2/9] vfio: No-IOMMU mode support
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Add qemu support for the newly introduced VFIO No-IOMMU driver.

We need to add special handling for:
- Group character device is /dev/vfio/noiommu-$GROUP.
- No-IOMMU does not rely on a memory listener.
- No IOMMU will be set for its group, so no need to call
  vfio_kvm_device_add_group.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/vfio/common.c              | 66 ++++++++++++++++++++++++++++++++++---------
 include/hw/vfio/vfio-common.h |  2 ++
 2 files changed, 55 insertions(+), 13 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index f27db36..656c303 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -789,6 +789,33 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
     container = g_malloc0(sizeof(*container));
     container->space = space;
     container->fd = fd;
+    container->noiommu = group->noiommu;
+
+    if (container->noiommu) {
+        ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd);
+        if (ret) {
+            error_report("vfio: failed to set group container: %m");
+            ret = -errno;
+            goto free_container_exit;
+        }
+
+        ret = ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_NOIOMMU_IOMMU);
+        if (!ret) {
+            error_report("vfio: No available IOMMU models");
+            ret = -EINVAL;
+            goto free_container_exit;
+        }
+
+        ret = ioctl(fd, VFIO_SET_IOMMU, VFIO_NOIOMMU_IOMMU);
+        if (ret) {
+            error_report("vfio: failed to set iommu for container: %m");
+            ret = -errno;
+            goto free_container_exit;
+        }
+
+        goto listener_register;
+    }
+
     if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) ||
         ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) {
         bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU);
@@ -878,14 +905,16 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
         goto free_container_exit;
     }
 
-    container->listener = vfio_memory_listener;
-
-    memory_listener_register(&container->listener, container->space->as);
-
-    if (container->error) {
-        ret = container->error;
-        error_report("vfio: memory listener initialization failed for container");
-        goto listener_release_exit;
+listener_register:
+    if (!container->noiommu) {
+        container->listener = vfio_memory_listener;
+        memory_listener_register(&container->listener, container->space->as);
+        if (container->error) {
+            ret = container->error;
+            error_report("vfio: memory listener initialization failed for "
+                         "container");
+            goto listener_release_exit;
+        }
     }
 
     container->initialized = true;
@@ -898,7 +927,9 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
 
     return 0;
 listener_release_exit:
-    vfio_listener_release(container);
+    if (!container->noiommu) {
+        vfio_listener_release(container);
+    }
 
 free_container_exit:
     g_free(container);
@@ -928,7 +959,9 @@ static void vfio_disconnect_container(VFIOGroup *group)
         VFIOAddressSpace *space = container->space;
         VFIOGuestIOMMU *giommu, *tmp;
 
-        vfio_listener_release(container);
+        if (!container->noiommu) {
+            vfio_listener_release(container);
+        }
         QLIST_REMOVE(container, next);
 
         QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) {
@@ -969,8 +1002,13 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
     snprintf(path, sizeof(path), "/dev/vfio/%d", groupid);
     group->fd = qemu_open(path, O_RDWR);
     if (group->fd < 0) {
-        error_report("vfio: error opening %s: %m", path);
-        goto free_group_exit;
+        snprintf(path, sizeof(path), "/dev/vfio/noiommu-%d", groupid);
+        group->fd = qemu_open(path, O_RDWR);
+        if (group->fd < 0) {
+            error_report("vfio: error opening %s: %m", path);
+            goto free_group_exit;
+        }
+        group->noiommu = 1;
     }
 
     if (ioctl(group->fd, VFIO_GROUP_GET_STATUS, &status)) {
@@ -999,7 +1037,9 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
 
     QLIST_INSERT_HEAD(&vfio_group_list, group, next);
 
-    vfio_kvm_device_add_group(group);
+    if (!group->noiommu) {
+        vfio_kvm_device_add_group(group);
+    }
 
     return group;
 
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index eb0e1b0..85c2a74 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -72,6 +72,7 @@ struct VFIOGroup;
 typedef struct VFIOContainer {
     VFIOAddressSpace *space;
     int fd; /* /dev/vfio/vfio, empowered by the attached groups */
+    bool noiommu;
     MemoryListener listener;
     int error;
     bool initialized;
@@ -121,6 +122,7 @@ struct VFIODeviceOps {
 typedef struct VFIOGroup {
     int fd;
     int groupid;
+    bool noiommu;
     VFIOContainer *container;
     QLIST_HEAD(, VFIODevice) device_list;
     QLIST_ENTRY(VFIOGroup) next;
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 2/9] vfio: No-IOMMU mode support
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Add qemu support for the newly introduced VFIO No-IOMMU driver.

We need to add special handling for:
- Group character device is /dev/vfio/noiommu-$GROUP.
- No-IOMMU does not rely on a memory listener.
- No IOMMU will be set for its group, so no need to call
  vfio_kvm_device_add_group.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/vfio/common.c              | 66 ++++++++++++++++++++++++++++++++++---------
 include/hw/vfio/vfio-common.h |  2 ++
 2 files changed, 55 insertions(+), 13 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index f27db36..656c303 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -789,6 +789,33 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
     container = g_malloc0(sizeof(*container));
     container->space = space;
     container->fd = fd;
+    container->noiommu = group->noiommu;
+
+    if (container->noiommu) {
+        ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd);
+        if (ret) {
+            error_report("vfio: failed to set group container: %m");
+            ret = -errno;
+            goto free_container_exit;
+        }
+
+        ret = ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_NOIOMMU_IOMMU);
+        if (!ret) {
+            error_report("vfio: No available IOMMU models");
+            ret = -EINVAL;
+            goto free_container_exit;
+        }
+
+        ret = ioctl(fd, VFIO_SET_IOMMU, VFIO_NOIOMMU_IOMMU);
+        if (ret) {
+            error_report("vfio: failed to set iommu for container: %m");
+            ret = -errno;
+            goto free_container_exit;
+        }
+
+        goto listener_register;
+    }
+
     if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) ||
         ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) {
         bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU);
@@ -878,14 +905,16 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
         goto free_container_exit;
     }
 
-    container->listener = vfio_memory_listener;
-
-    memory_listener_register(&container->listener, container->space->as);
-
-    if (container->error) {
-        ret = container->error;
-        error_report("vfio: memory listener initialization failed for container");
-        goto listener_release_exit;
+listener_register:
+    if (!container->noiommu) {
+        container->listener = vfio_memory_listener;
+        memory_listener_register(&container->listener, container->space->as);
+        if (container->error) {
+            ret = container->error;
+            error_report("vfio: memory listener initialization failed for "
+                         "container");
+            goto listener_release_exit;
+        }
     }
 
     container->initialized = true;
@@ -898,7 +927,9 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
 
     return 0;
 listener_release_exit:
-    vfio_listener_release(container);
+    if (!container->noiommu) {
+        vfio_listener_release(container);
+    }
 
 free_container_exit:
     g_free(container);
@@ -928,7 +959,9 @@ static void vfio_disconnect_container(VFIOGroup *group)
         VFIOAddressSpace *space = container->space;
         VFIOGuestIOMMU *giommu, *tmp;
 
-        vfio_listener_release(container);
+        if (!container->noiommu) {
+            vfio_listener_release(container);
+        }
         QLIST_REMOVE(container, next);
 
         QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) {
@@ -969,8 +1002,13 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
     snprintf(path, sizeof(path), "/dev/vfio/%d", groupid);
     group->fd = qemu_open(path, O_RDWR);
     if (group->fd < 0) {
-        error_report("vfio: error opening %s: %m", path);
-        goto free_group_exit;
+        snprintf(path, sizeof(path), "/dev/vfio/noiommu-%d", groupid);
+        group->fd = qemu_open(path, O_RDWR);
+        if (group->fd < 0) {
+            error_report("vfio: error opening %s: %m", path);
+            goto free_group_exit;
+        }
+        group->noiommu = 1;
     }
 
     if (ioctl(group->fd, VFIO_GROUP_GET_STATUS, &status)) {
@@ -999,7 +1037,9 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
 
     QLIST_INSERT_HEAD(&vfio_group_list, group, next);
 
-    vfio_kvm_device_add_group(group);
+    if (!group->noiommu) {
+        vfio_kvm_device_add_group(group);
+    }
 
     return group;
 
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index eb0e1b0..85c2a74 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -72,6 +72,7 @@ struct VFIOGroup;
 typedef struct VFIOContainer {
     VFIOAddressSpace *space;
     int fd; /* /dev/vfio/vfio, empowered by the attached groups */
+    bool noiommu;
     MemoryListener listener;
     int error;
     bool initialized;
@@ -121,6 +122,7 @@ struct VFIODeviceOps {
 typedef struct VFIOGroup {
     int fd;
     int groupid;
+    bool noiommu;
     VFIOContainer *container;
     QLIST_HEAD(, VFIODevice) device_list;
     QLIST_ENTRY(VFIOGroup) next;
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 3/9] s390x/css: introduce ccw chain interfaces
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Introduce CcwChain structures and helper functions that can be used
to translate a guest ccw program to a user-space ccw program.
The following limitations apply:
1. Support only prefetch enable mode.
2. Support chain for idal(c64) ccws.
3. Detect chain end only by CC and DC flags.

This work prepares a user-space ccw program according to the rules
below:
1. Alloc a 4K memory buffer to store all of the ccw program information.
   * Lower 2k of the buffer are used to store a maximum of 256 ccws, these
     ccws are copied from 'guest ccw program' and placed one after another.
   * Upper 2k of the buffer are used to store a maximum of 256 corresponding
     cda data sets, each having a length of 8 bytes.
2. For TIC ccw.
   * Locate the TIC target ccw inside the ccw area, and calculate its offset.
   * Store the offset to ccw.cda.
3. For Direct ccw.
   * Find the cda entry with the same index as the ccw.
   * Store the user virtual address of the original ccw.cda to the cda entry.
   * Store the offset of the cda entry to ccw.cda.
4. For IDAL ccw.
   * Find the cda entry with the same index as the ccw.
   * Prepare the user-space idaws. Store the virtual address of the idaws to
     the cda entry.
   * Store the offset of the cda entry to ccw.cda.
5. Append a NOOP to the chain end.
6. Expectations for the user-space ccw program I/O result:
   CPA of SCSW should be set to the offset of the ccw area of the current ccw.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/Makefile.objs   |   1 +
 hw/s390x/s390-ccwchain.c | 441 +++++++++++++++++++++++++++++++++++++++++++++++
 hw/s390x/s390-ccwchain.h |  28 +++
 3 files changed, 470 insertions(+)
 create mode 100644 hw/s390x/s390-ccwchain.c
 create mode 100644 hw/s390x/s390-ccwchain.h

diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs
index 2203617..35b5d27 100644
--- a/hw/s390x/Makefile.objs
+++ b/hw/s390x/Makefile.objs
@@ -11,3 +11,4 @@ obj-y += virtio-ccw.o
 obj-y += s390-pci-bus.o s390-pci-inst.o
 obj-y += s390-skeys.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
+obj-y += s390-ccwchain.o
diff --git a/hw/s390x/s390-ccwchain.c b/hw/s390x/s390-ccwchain.c
new file mode 100644
index 0000000..e62869d
--- /dev/null
+++ b/hw/s390x/s390-ccwchain.c
@@ -0,0 +1,441 @@
+ /*
+  * s390 ccwchain interface
+  *
+  * Copyright 2016 IBM Corp.
+  * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+  *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+  *
+  * This work is licensed under the terms of the GNU GPL, version
+  * 2 or (at your option) any later version. See the COPYING file
+  * in the top-level directory.
+  */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "s390-ccwchain.h"
+
+#define IDA_SIZE_LOG 12 /* 12 for 4k */
+#define IDA_BLOCK_SIZE (1L << IDA_SIZE_LOG)
+#define CCWCHAIN_LEN_MAX 256
+#define CCW1_SIZE 3 /* sizeof(CCW1) == (1 << 3) */
+#define CDA_ITEM_SIZE 3 /* sizeof(uint64_t) == (1 << 3) */
+
+#define ccw_is_test(_ccw) (((_ccw)->cmd_code & 0x0F) == 0)
+
+#define ccw_is_noop(_ccw) ((_ccw)->cmd_code == CCW_CMD_NOOP)
+
+#define ccw_is_tic(_ccw) ((_ccw)->cmd_code == CCW_CMD_TIC)
+
+#define ccw_is_idal(_ccw) ((_ccw)->flags & CCW_FLAG_IDA)
+
+#define ccw_is_chain(_ccw) ((_ccw)->flags & (CCW_FLAG_CC | CCW_FLAG_DC))
+
+typedef struct IdaWords {
+    uint32_t nr;
+    uint64_t *ida_word;
+} IdaWords;
+
+typedef struct CcwChainBuffer {
+    CCW1 ccw[CCWCHAIN_LEN_MAX];
+    uint64_t cda[CCWCHAIN_LEN_MAX];
+} CcwChainBuffer;
+
+typedef struct CcwChain {
+    QTAILQ_ENTRY(CcwChain) entry;
+    uint32_t nr;
+    hwaddr gpa;
+    CCW1 *ccw;
+    IdaWords *ida_words;
+} CcwChain;
+
+typedef struct CcwChainList {
+    QTAILQ_HEAD(, CcwChain) list;
+    CcwChainBuffer buf;
+    uint32_t nr; /* Number of the CCWs in the whole list. */
+} CcwChainList;
+
+static int ccwchain_translate(CcwChain *chain, CcwChainList *ccwchain_list);
+
+static inline void *ccwchain_hva_get(hwaddr gpa)
+{
+    hwaddr len = 1;
+    return cpu_physical_memory_map(gpa, &len, 1);
+}
+
+static inline void ccwchain_hva_put(void *hva)
+{
+    cpu_physical_memory_unmap(hva, 1, 1, 1);
+}
+
+static inline void ccwchain_idaword_put(IdaWords *ida_words)
+{
+    uint64_t *ida_word;
+    uint32_t cnt;
+
+    ida_word = ida_words->ida_word;
+    cnt = ida_words->nr;
+    while (cnt--) {
+        ccwchain_hva_put((void *)*ida_word);
+        ida_word++;
+    }
+}
+
+static inline void ccwchain_direct_put(CCW1 *ccw, CcwChainList *ccwchain_list)
+{
+    void *cda_hva;
+
+    cda_hva = (void *)ccwchain_list->buf.cda[ccw->cda >> CDA_ITEM_SIZE];
+    ccwchain_hva_put(cda_hva);
+}
+
+static inline uint32_t ccwchain_ccw_offset(CcwChain *chain,
+                                           CcwChainList *ccwchain_list)
+{
+    return (void *)chain->ccw - (void *)(&ccwchain_list->buf.ccw);
+}
+
+static void ccwchain_free(CcwChain *chain, CcwChainList *ccwchain_list)
+{
+    struct IdaWords *ida_words;
+    CCW1 *ccw;
+    uint32_t idx;
+
+    for (idx = 0; idx < chain->nr; idx++) {
+        ccw = chain->ccw + idx;
+        ida_words = chain->ida_words + idx;
+        if (ccw_is_idal(ccw)) {
+            ccwchain_idaword_put(ida_words);
+        } else if (!ccw_is_test(ccw) && !ccw_is_noop(ccw) &&
+                   !ccw_is_tic(ccw)) {
+            ccwchain_direct_put(ccw, ccwchain_list);
+        }
+        g_free(ida_words->ida_word);
+    }
+
+    QTAILQ_REMOVE(&ccwchain_list->list, chain, entry);
+    g_free(chain);
+}
+
+/*
+ * ccwchain_calc_length - calculate the length of the ccwchain.
+ *
+ * This is the chain length not considering any TICs.
+ * You need to do a new round for each TIC target.
+ *
+ * Returns: the length of the ccwchain.
+ */
+static int ccwchain_calc_length(hwaddr gpa)
+{
+    CCW1 *ccw;
+    int cnt;
+
+    cnt = 0;
+    do {
+        ccw = (CCW1 *)ccwchain_hva_get(gpa);
+        if (!ccw) {
+            return -EFAULT;
+        }
+        cnt++;
+
+        if (!ccw_is_chain(ccw)) {
+            /*
+             * An extra count is needed to reserve a space for
+             * appending an extra NOOP to the chain tail.
+             */
+            cnt++;
+            break;
+        }
+
+        gpa = gpa + sizeof(*ccw);
+        ccwchain_hva_put(ccw);
+    } while (cnt < CCWCHAIN_LEN_MAX + 1);
+
+    if (cnt > CCWCHAIN_LEN_MAX) {
+        cnt = 0;
+    }
+    return cnt;
+}
+
+static CcwChain *ccwchain_alloc(int nr)
+{
+    CcwChain *chain;
+    void *data;
+    size_t size;
+
+    size = sizeof(*chain) + sizeof(*chain->ida_words) * nr;
+    chain = g_malloc0(size);
+    data = (uint8_t *)chain + sizeof(*chain);
+    chain->ida_words = (IdaWords *)data;
+    chain->nr = nr;
+
+    return chain;
+}
+
+static CcwChain *ccwchain_copy_from_guest(hwaddr gpa,
+                                          CcwChainList *ccwchain_list)
+{
+    CcwChain *chain;
+    int nr;
+
+    nr = ccwchain_calc_length(gpa);
+    if (nr <= 0) {
+        return NULL;
+    }
+    if ((ccwchain_list->nr + nr) > CCWCHAIN_LEN_MAX) {
+        return NULL;
+    }
+
+    chain = ccwchain_alloc(nr);
+    if (!chain) {
+        return NULL;
+    }
+
+    chain->gpa = gpa;
+    chain->ccw = (void *)(&ccwchain_list->buf) +
+                 ccwchain_list->nr * sizeof(*chain->ccw);
+    cpu_physical_memory_read(chain->gpa,
+                             chain->ccw,
+                             sizeof(*chain->ccw) * chain->nr);
+    ccwchain_list->nr += chain->nr;
+    QTAILQ_INSERT_TAIL(&ccwchain_list->list, chain, entry);
+
+    return chain;
+}
+
+static bool tic_target_chain_exists(CCW1 *tic, CcwChainList *ccwchain_list)
+{
+    CcwChain *chain;
+    uint32_t ccw_head, ccw_tail;
+
+    QTAILQ_FOREACH(chain, &ccwchain_list->list, entry) {
+        ccw_head = chain->gpa;
+        ccw_tail = ccw_head + sizeof(*chain->ccw) * chain->nr;
+
+        if ((ccw_head <= tic->cda) && (tic->cda <= ccw_tail)) {
+            tic->cda = ccwchain_ccw_offset(chain, ccwchain_list);
+            return true;
+        }
+    }
+    return false;
+}
+
+static int ccwchain_translate_loop_tic(CCW1 *tic,
+                                       CcwChainList *ccwchain_list)
+{
+    CcwChain *chain;
+    int ret;
+
+    /* May transfer to an existing chain. */
+    if (tic_target_chain_exists(tic, ccwchain_list)) {
+        return 0;
+    }
+
+    /* It's a new chain then. */
+    chain = ccwchain_copy_from_guest((uint64_t)tic->cda, ccwchain_list);
+    if (!chain) {
+        return -EFAULT;
+    }
+
+    tic->cda = ccwchain_ccw_offset(chain, ccwchain_list);
+    /* Translate the new ccwchain now. */
+    ret = ccwchain_translate(chain, ccwchain_list);
+    if (ret) {
+        ccwchain_free(chain, ccwchain_list);
+    }
+
+    return ret;
+}
+
+static inline uint32_t idal_nr_words(uint64_t gaddr, uint32_t length)
+{
+    return ((gaddr & (IDA_BLOCK_SIZE - 1)) + length + (IDA_BLOCK_SIZE - 1))
+            >> IDA_SIZE_LOG;
+}
+
+static int copy_idaws_from_guest(CCW1 *ccw, IdaWords *ida_words)
+{
+    uint32_t idaws_len;
+    uint64_t data_addr;
+
+    cpu_physical_memory_read(ccw->cda, &data_addr, 8);
+    ida_words->nr = idal_nr_words(data_addr, ccw->count);
+    idaws_len = ida_words->nr * sizeof(*ida_words->ida_word);
+    ida_words->ida_word = g_malloc0(idaws_len);
+    cpu_physical_memory_read(ccw->cda, ida_words->ida_word, idaws_len);
+
+    return 0;
+}
+
+/*
+ * Translate the guest address in the idaws to user-space virtual address.
+ */
+static int translate_idaws(IdaWords *ida_words)
+{
+    uint64_t *ida_word;
+    uint32_t cnt;
+
+    cnt = ida_words->nr;
+    ida_word = ida_words->ida_word;
+    while (cnt--) {
+        *ida_word = (uint64_t)ccwchain_hva_get(be64_to_cpu(*ida_word));
+        if (!(void *)*ida_word) {
+            return -EFAULT;
+        }
+        ida_word++;
+    }
+
+    return 0;
+}
+
+static void ccwchain_update_cda(CCW1 *ccw,
+                               uint64_t cda_data,
+                               CcwChainList *ccwchain_list)
+{
+    uint32_t index;
+
+    index = ((void *)ccw - (void *)&ccwchain_list->buf.ccw) >> CCW1_SIZE;
+    ccwchain_list->buf.cda[index] = cda_data;
+    ccw->cda = index << CDA_ITEM_SIZE;
+}
+
+static int ccwchain_translate_idal(CCW1 *ccw,
+                                   IdaWords *ida_words,
+                                   CcwChainList *ccwchain_list)
+{
+    int ret;
+
+    ret = copy_idaws_from_guest(ccw, ida_words);
+    if (ret) {
+        return ret;
+    }
+
+    ccwchain_update_cda(ccw, (uint64_t)ida_words->ida_word, ccwchain_list);
+
+    return translate_idaws(ida_words);
+}
+
+static int ccwchain_translate_direct(CCW1 *ccw, CcwChainList *ccwchain_list)
+{
+    void *cda;
+
+    cda = ccwchain_hva_get(ccw->cda);
+    if (!cda) {
+        return -EFAULT;
+    }
+
+    ccwchain_update_cda(ccw, (uint64_t)cda, ccwchain_list);
+
+    return 0;
+}
+
+static int ccwchain_translate(CcwChain *chain, CcwChainList *ccwchain_list)
+{
+    CCW1 *ccw;
+    IdaWords *ida_words;
+    int i, ret;
+
+    for (i = 0; i < chain->nr - 1; i++) {
+        ccw = chain->ccw + i;
+        ccw->count = be16_to_cpu(ccw->count);
+        ccw->cda = be32_to_cpu(ccw->cda);
+        if (ccw_is_test(ccw)) {
+            continue;
+        } else if (ccw_is_noop(ccw)) {
+            continue;
+        } else if (ccw_is_tic(ccw)) {
+            ret = ccwchain_translate_loop_tic(ccw, ccwchain_list);
+        } else if (ccw_is_idal(ccw)) {
+            ida_words = chain->ida_words + i;
+            ret = ccwchain_translate_idal(ccw, ida_words, ccwchain_list);
+        } else {
+            ret = ccwchain_translate_direct(ccw, ccwchain_list);
+        }
+        if (ret) {
+            return ret;
+        }
+    }
+
+    /*
+     * CSS may bypass the last ccw via status-modifier and lead to an
+     * unpredictable behavior. To avoid this, we put a NOOP after the
+     * last ccw.
+     */
+    ccw = chain->ccw + i;
+    ccw->cmd_code = CCW_CMD_NOOP;
+    ccw->flags = 0;
+    ccw->count = 0;
+    ccw->cda = 0;
+
+    return 0;
+}
+
+int ccwchain_translate_to_userspace(TransChainData *chain_data)
+{
+    CcwChain *chain = NULL;
+    CcwChainList *ccwchain_list;
+    int ret;
+
+    ccwchain_list = g_malloc0(sizeof(*ccwchain_list));
+    QTAILQ_INIT(&ccwchain_list->list);
+    chain_data->ccwchain_list = (uint64_t)ccwchain_list;
+    chain_data->ccwchain_buf = (uint64_t)&ccwchain_list->buf;
+
+    chain = ccwchain_copy_from_guest(chain_data->cpa_gpa, ccwchain_list);
+    if (!chain) {
+        ccwchain_list_free(chain_data);
+        return -EFAULT;
+    }
+
+    ret = ccwchain_translate(chain, ccwchain_list);
+    if (ret) {
+        ccwchain_list_free(chain_data);
+        return ret;
+    }
+    chain_data->ccwchain_nr = ccwchain_list->nr;
+
+    return ret;
+}
+
+void ccwchain_update_scsw(SCSW *scsw, TransChainData *chain_data)
+{
+    CcwChainList *ccwchain_list;
+    CcwChain *chain;
+    uint32_t ccw_cnt, ccw_idx;
+    hwaddr cpa;
+
+    cpa = 0;
+    ccw_cnt = 0;
+    ccwchain_list = (void *)chain_data->ccwchain_list;
+    ccw_idx = scsw->cpa >> CCW1_SIZE;
+    QTAILQ_FOREACH(chain, &ccwchain_list->list, entry) {
+        if (ccw_cnt + chain->nr > ccw_idx) {
+            /*
+             * TODO:
+             * When the NOOP introduced by us was hit, we should propagate
+             * this information to the guest.
+             */
+            cpa = chain->gpa + (ccw_idx - ccw_cnt) * sizeof(*chain->ccw);
+            break;
+        }
+
+        ccw_cnt += chain->nr;
+    }
+    scsw->cpa = (uint32_t)cpa;
+}
+
+void ccwchain_list_free(TransChainData *chain_data)
+{
+    CcwChainList *ccwchain_list;
+    CcwChain *chain;
+
+    ccwchain_list = (void *)chain_data->ccwchain_list;
+    if (!ccwchain_list) {
+        return;
+    }
+
+    while ((chain = QTAILQ_FIRST(&ccwchain_list->list)) != NULL) {
+        ccwchain_free(chain, ccwchain_list);
+    }
+
+    g_free(ccwchain_list);
+}
diff --git a/hw/s390x/s390-ccwchain.h b/hw/s390x/s390-ccwchain.h
new file mode 100644
index 0000000..b2352de
--- /dev/null
+++ b/hw/s390x/s390-ccwchain.h
@@ -0,0 +1,28 @@
+/*
+ * ccwchain interfaces
+ *
+ * Copyright IBM Corp. 2016
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License (version 2 only)
+ * as published by the Free Software Foundation.
+ *
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ */
+
+#ifndef S390_CCWCHAIN_H
+#define S390_CCWCHAIN_H
+
+typedef struct TransChainData {
+    hwaddr cpa_gpa;
+    uint64_t ccwchain_list;
+    uint64_t ccwchain_buf;
+    uint32_t ccwchain_nr;
+} TransChainData;
+
+int ccwchain_translate_to_userspace(TransChainData *chain_data);
+void ccwchain_update_scsw(SCSW *scsw, TransChainData *chain_data);
+void ccwchain_list_free(TransChainData *chain_data);
+
+#endif
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 3/9] s390x/css: introduce ccw chain interfaces
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Introduce CcwChain structures and helper functions that can be used
to translate a guest ccw program to a user-space ccw program.
The following limitations apply:
1. Support only prefetch enable mode.
2. Support chain for idal(c64) ccws.
3. Detect chain end only by CC and DC flags.

This work prepares a user-space ccw program according to the rules
below:
1. Alloc a 4K memory buffer to store all of the ccw program information.
   * Lower 2k of the buffer are used to store a maximum of 256 ccws, these
     ccws are copied from 'guest ccw program' and placed one after another.
   * Upper 2k of the buffer are used to store a maximum of 256 corresponding
     cda data sets, each having a length of 8 bytes.
2. For TIC ccw.
   * Locate the TIC target ccw inside the ccw area, and calculate its offset.
   * Store the offset to ccw.cda.
3. For Direct ccw.
   * Find the cda entry with the same index as the ccw.
   * Store the user virtual address of the original ccw.cda to the cda entry.
   * Store the offset of the cda entry to ccw.cda.
4. For IDAL ccw.
   * Find the cda entry with the same index as the ccw.
   * Prepare the user-space idaws. Store the virtual address of the idaws to
     the cda entry.
   * Store the offset of the cda entry to ccw.cda.
5. Append a NOOP to the chain end.
6. Expectations for the user-space ccw program I/O result:
   CPA of SCSW should be set to the offset of the ccw area of the current ccw.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/Makefile.objs   |   1 +
 hw/s390x/s390-ccwchain.c | 441 +++++++++++++++++++++++++++++++++++++++++++++++
 hw/s390x/s390-ccwchain.h |  28 +++
 3 files changed, 470 insertions(+)
 create mode 100644 hw/s390x/s390-ccwchain.c
 create mode 100644 hw/s390x/s390-ccwchain.h

diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs
index 2203617..35b5d27 100644
--- a/hw/s390x/Makefile.objs
+++ b/hw/s390x/Makefile.objs
@@ -11,3 +11,4 @@ obj-y += virtio-ccw.o
 obj-y += s390-pci-bus.o s390-pci-inst.o
 obj-y += s390-skeys.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
+obj-y += s390-ccwchain.o
diff --git a/hw/s390x/s390-ccwchain.c b/hw/s390x/s390-ccwchain.c
new file mode 100644
index 0000000..e62869d
--- /dev/null
+++ b/hw/s390x/s390-ccwchain.c
@@ -0,0 +1,441 @@
+ /*
+  * s390 ccwchain interface
+  *
+  * Copyright 2016 IBM Corp.
+  * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+  *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+  *
+  * This work is licensed under the terms of the GNU GPL, version
+  * 2 or (at your option) any later version. See the COPYING file
+  * in the top-level directory.
+  */
+
+#include "qemu/osdep.h"
+#include "cpu.h"
+#include "s390-ccwchain.h"
+
+#define IDA_SIZE_LOG 12 /* 12 for 4k */
+#define IDA_BLOCK_SIZE (1L << IDA_SIZE_LOG)
+#define CCWCHAIN_LEN_MAX 256
+#define CCW1_SIZE 3 /* sizeof(CCW1) == (1 << 3) */
+#define CDA_ITEM_SIZE 3 /* sizeof(uint64_t) == (1 << 3) */
+
+#define ccw_is_test(_ccw) (((_ccw)->cmd_code & 0x0F) == 0)
+
+#define ccw_is_noop(_ccw) ((_ccw)->cmd_code == CCW_CMD_NOOP)
+
+#define ccw_is_tic(_ccw) ((_ccw)->cmd_code == CCW_CMD_TIC)
+
+#define ccw_is_idal(_ccw) ((_ccw)->flags & CCW_FLAG_IDA)
+
+#define ccw_is_chain(_ccw) ((_ccw)->flags & (CCW_FLAG_CC | CCW_FLAG_DC))
+
+typedef struct IdaWords {
+    uint32_t nr;
+    uint64_t *ida_word;
+} IdaWords;
+
+typedef struct CcwChainBuffer {
+    CCW1 ccw[CCWCHAIN_LEN_MAX];
+    uint64_t cda[CCWCHAIN_LEN_MAX];
+} CcwChainBuffer;
+
+typedef struct CcwChain {
+    QTAILQ_ENTRY(CcwChain) entry;
+    uint32_t nr;
+    hwaddr gpa;
+    CCW1 *ccw;
+    IdaWords *ida_words;
+} CcwChain;
+
+typedef struct CcwChainList {
+    QTAILQ_HEAD(, CcwChain) list;
+    CcwChainBuffer buf;
+    uint32_t nr; /* Number of the CCWs in the whole list. */
+} CcwChainList;
+
+static int ccwchain_translate(CcwChain *chain, CcwChainList *ccwchain_list);
+
+static inline void *ccwchain_hva_get(hwaddr gpa)
+{
+    hwaddr len = 1;
+    return cpu_physical_memory_map(gpa, &len, 1);
+}
+
+static inline void ccwchain_hva_put(void *hva)
+{
+    cpu_physical_memory_unmap(hva, 1, 1, 1);
+}
+
+static inline void ccwchain_idaword_put(IdaWords *ida_words)
+{
+    uint64_t *ida_word;
+    uint32_t cnt;
+
+    ida_word = ida_words->ida_word;
+    cnt = ida_words->nr;
+    while (cnt--) {
+        ccwchain_hva_put((void *)*ida_word);
+        ida_word++;
+    }
+}
+
+static inline void ccwchain_direct_put(CCW1 *ccw, CcwChainList *ccwchain_list)
+{
+    void *cda_hva;
+
+    cda_hva = (void *)ccwchain_list->buf.cda[ccw->cda >> CDA_ITEM_SIZE];
+    ccwchain_hva_put(cda_hva);
+}
+
+static inline uint32_t ccwchain_ccw_offset(CcwChain *chain,
+                                           CcwChainList *ccwchain_list)
+{
+    return (void *)chain->ccw - (void *)(&ccwchain_list->buf.ccw);
+}
+
+static void ccwchain_free(CcwChain *chain, CcwChainList *ccwchain_list)
+{
+    struct IdaWords *ida_words;
+    CCW1 *ccw;
+    uint32_t idx;
+
+    for (idx = 0; idx < chain->nr; idx++) {
+        ccw = chain->ccw + idx;
+        ida_words = chain->ida_words + idx;
+        if (ccw_is_idal(ccw)) {
+            ccwchain_idaword_put(ida_words);
+        } else if (!ccw_is_test(ccw) && !ccw_is_noop(ccw) &&
+                   !ccw_is_tic(ccw)) {
+            ccwchain_direct_put(ccw, ccwchain_list);
+        }
+        g_free(ida_words->ida_word);
+    }
+
+    QTAILQ_REMOVE(&ccwchain_list->list, chain, entry);
+    g_free(chain);
+}
+
+/*
+ * ccwchain_calc_length - calculate the length of the ccwchain.
+ *
+ * This is the chain length not considering any TICs.
+ * You need to do a new round for each TIC target.
+ *
+ * Returns: the length of the ccwchain.
+ */
+static int ccwchain_calc_length(hwaddr gpa)
+{
+    CCW1 *ccw;
+    int cnt;
+
+    cnt = 0;
+    do {
+        ccw = (CCW1 *)ccwchain_hva_get(gpa);
+        if (!ccw) {
+            return -EFAULT;
+        }
+        cnt++;
+
+        if (!ccw_is_chain(ccw)) {
+            /*
+             * An extra count is needed to reserve a space for
+             * appending an extra NOOP to the chain tail.
+             */
+            cnt++;
+            break;
+        }
+
+        gpa = gpa + sizeof(*ccw);
+        ccwchain_hva_put(ccw);
+    } while (cnt < CCWCHAIN_LEN_MAX + 1);
+
+    if (cnt > CCWCHAIN_LEN_MAX) {
+        cnt = 0;
+    }
+    return cnt;
+}
+
+static CcwChain *ccwchain_alloc(int nr)
+{
+    CcwChain *chain;
+    void *data;
+    size_t size;
+
+    size = sizeof(*chain) + sizeof(*chain->ida_words) * nr;
+    chain = g_malloc0(size);
+    data = (uint8_t *)chain + sizeof(*chain);
+    chain->ida_words = (IdaWords *)data;
+    chain->nr = nr;
+
+    return chain;
+}
+
+static CcwChain *ccwchain_copy_from_guest(hwaddr gpa,
+                                          CcwChainList *ccwchain_list)
+{
+    CcwChain *chain;
+    int nr;
+
+    nr = ccwchain_calc_length(gpa);
+    if (nr <= 0) {
+        return NULL;
+    }
+    if ((ccwchain_list->nr + nr) > CCWCHAIN_LEN_MAX) {
+        return NULL;
+    }
+
+    chain = ccwchain_alloc(nr);
+    if (!chain) {
+        return NULL;
+    }
+
+    chain->gpa = gpa;
+    chain->ccw = (void *)(&ccwchain_list->buf) +
+                 ccwchain_list->nr * sizeof(*chain->ccw);
+    cpu_physical_memory_read(chain->gpa,
+                             chain->ccw,
+                             sizeof(*chain->ccw) * chain->nr);
+    ccwchain_list->nr += chain->nr;
+    QTAILQ_INSERT_TAIL(&ccwchain_list->list, chain, entry);
+
+    return chain;
+}
+
+static bool tic_target_chain_exists(CCW1 *tic, CcwChainList *ccwchain_list)
+{
+    CcwChain *chain;
+    uint32_t ccw_head, ccw_tail;
+
+    QTAILQ_FOREACH(chain, &ccwchain_list->list, entry) {
+        ccw_head = chain->gpa;
+        ccw_tail = ccw_head + sizeof(*chain->ccw) * chain->nr;
+
+        if ((ccw_head <= tic->cda) && (tic->cda <= ccw_tail)) {
+            tic->cda = ccwchain_ccw_offset(chain, ccwchain_list);
+            return true;
+        }
+    }
+    return false;
+}
+
+static int ccwchain_translate_loop_tic(CCW1 *tic,
+                                       CcwChainList *ccwchain_list)
+{
+    CcwChain *chain;
+    int ret;
+
+    /* May transfer to an existing chain. */
+    if (tic_target_chain_exists(tic, ccwchain_list)) {
+        return 0;
+    }
+
+    /* It's a new chain then. */
+    chain = ccwchain_copy_from_guest((uint64_t)tic->cda, ccwchain_list);
+    if (!chain) {
+        return -EFAULT;
+    }
+
+    tic->cda = ccwchain_ccw_offset(chain, ccwchain_list);
+    /* Translate the new ccwchain now. */
+    ret = ccwchain_translate(chain, ccwchain_list);
+    if (ret) {
+        ccwchain_free(chain, ccwchain_list);
+    }
+
+    return ret;
+}
+
+static inline uint32_t idal_nr_words(uint64_t gaddr, uint32_t length)
+{
+    return ((gaddr & (IDA_BLOCK_SIZE - 1)) + length + (IDA_BLOCK_SIZE - 1))
+            >> IDA_SIZE_LOG;
+}
+
+static int copy_idaws_from_guest(CCW1 *ccw, IdaWords *ida_words)
+{
+    uint32_t idaws_len;
+    uint64_t data_addr;
+
+    cpu_physical_memory_read(ccw->cda, &data_addr, 8);
+    ida_words->nr = idal_nr_words(data_addr, ccw->count);
+    idaws_len = ida_words->nr * sizeof(*ida_words->ida_word);
+    ida_words->ida_word = g_malloc0(idaws_len);
+    cpu_physical_memory_read(ccw->cda, ida_words->ida_word, idaws_len);
+
+    return 0;
+}
+
+/*
+ * Translate the guest address in the idaws to user-space virtual address.
+ */
+static int translate_idaws(IdaWords *ida_words)
+{
+    uint64_t *ida_word;
+    uint32_t cnt;
+
+    cnt = ida_words->nr;
+    ida_word = ida_words->ida_word;
+    while (cnt--) {
+        *ida_word = (uint64_t)ccwchain_hva_get(be64_to_cpu(*ida_word));
+        if (!(void *)*ida_word) {
+            return -EFAULT;
+        }
+        ida_word++;
+    }
+
+    return 0;
+}
+
+static void ccwchain_update_cda(CCW1 *ccw,
+                               uint64_t cda_data,
+                               CcwChainList *ccwchain_list)
+{
+    uint32_t index;
+
+    index = ((void *)ccw - (void *)&ccwchain_list->buf.ccw) >> CCW1_SIZE;
+    ccwchain_list->buf.cda[index] = cda_data;
+    ccw->cda = index << CDA_ITEM_SIZE;
+}
+
+static int ccwchain_translate_idal(CCW1 *ccw,
+                                   IdaWords *ida_words,
+                                   CcwChainList *ccwchain_list)
+{
+    int ret;
+
+    ret = copy_idaws_from_guest(ccw, ida_words);
+    if (ret) {
+        return ret;
+    }
+
+    ccwchain_update_cda(ccw, (uint64_t)ida_words->ida_word, ccwchain_list);
+
+    return translate_idaws(ida_words);
+}
+
+static int ccwchain_translate_direct(CCW1 *ccw, CcwChainList *ccwchain_list)
+{
+    void *cda;
+
+    cda = ccwchain_hva_get(ccw->cda);
+    if (!cda) {
+        return -EFAULT;
+    }
+
+    ccwchain_update_cda(ccw, (uint64_t)cda, ccwchain_list);
+
+    return 0;
+}
+
+static int ccwchain_translate(CcwChain *chain, CcwChainList *ccwchain_list)
+{
+    CCW1 *ccw;
+    IdaWords *ida_words;
+    int i, ret;
+
+    for (i = 0; i < chain->nr - 1; i++) {
+        ccw = chain->ccw + i;
+        ccw->count = be16_to_cpu(ccw->count);
+        ccw->cda = be32_to_cpu(ccw->cda);
+        if (ccw_is_test(ccw)) {
+            continue;
+        } else if (ccw_is_noop(ccw)) {
+            continue;
+        } else if (ccw_is_tic(ccw)) {
+            ret = ccwchain_translate_loop_tic(ccw, ccwchain_list);
+        } else if (ccw_is_idal(ccw)) {
+            ida_words = chain->ida_words + i;
+            ret = ccwchain_translate_idal(ccw, ida_words, ccwchain_list);
+        } else {
+            ret = ccwchain_translate_direct(ccw, ccwchain_list);
+        }
+        if (ret) {
+            return ret;
+        }
+    }
+
+    /*
+     * CSS may bypass the last ccw via status-modifier and lead to an
+     * unpredictable behavior. To avoid this, we put a NOOP after the
+     * last ccw.
+     */
+    ccw = chain->ccw + i;
+    ccw->cmd_code = CCW_CMD_NOOP;
+    ccw->flags = 0;
+    ccw->count = 0;
+    ccw->cda = 0;
+
+    return 0;
+}
+
+int ccwchain_translate_to_userspace(TransChainData *chain_data)
+{
+    CcwChain *chain = NULL;
+    CcwChainList *ccwchain_list;
+    int ret;
+
+    ccwchain_list = g_malloc0(sizeof(*ccwchain_list));
+    QTAILQ_INIT(&ccwchain_list->list);
+    chain_data->ccwchain_list = (uint64_t)ccwchain_list;
+    chain_data->ccwchain_buf = (uint64_t)&ccwchain_list->buf;
+
+    chain = ccwchain_copy_from_guest(chain_data->cpa_gpa, ccwchain_list);
+    if (!chain) {
+        ccwchain_list_free(chain_data);
+        return -EFAULT;
+    }
+
+    ret = ccwchain_translate(chain, ccwchain_list);
+    if (ret) {
+        ccwchain_list_free(chain_data);
+        return ret;
+    }
+    chain_data->ccwchain_nr = ccwchain_list->nr;
+
+    return ret;
+}
+
+void ccwchain_update_scsw(SCSW *scsw, TransChainData *chain_data)
+{
+    CcwChainList *ccwchain_list;
+    CcwChain *chain;
+    uint32_t ccw_cnt, ccw_idx;
+    hwaddr cpa;
+
+    cpa = 0;
+    ccw_cnt = 0;
+    ccwchain_list = (void *)chain_data->ccwchain_list;
+    ccw_idx = scsw->cpa >> CCW1_SIZE;
+    QTAILQ_FOREACH(chain, &ccwchain_list->list, entry) {
+        if (ccw_cnt + chain->nr > ccw_idx) {
+            /*
+             * TODO:
+             * When the NOOP introduced by us was hit, we should propagate
+             * this information to the guest.
+             */
+            cpa = chain->gpa + (ccw_idx - ccw_cnt) * sizeof(*chain->ccw);
+            break;
+        }
+
+        ccw_cnt += chain->nr;
+    }
+    scsw->cpa = (uint32_t)cpa;
+}
+
+void ccwchain_list_free(TransChainData *chain_data)
+{
+    CcwChainList *ccwchain_list;
+    CcwChain *chain;
+
+    ccwchain_list = (void *)chain_data->ccwchain_list;
+    if (!ccwchain_list) {
+        return;
+    }
+
+    while ((chain = QTAILQ_FIRST(&ccwchain_list->list)) != NULL) {
+        ccwchain_free(chain, ccwchain_list);
+    }
+
+    g_free(ccwchain_list);
+}
diff --git a/hw/s390x/s390-ccwchain.h b/hw/s390x/s390-ccwchain.h
new file mode 100644
index 0000000..b2352de
--- /dev/null
+++ b/hw/s390x/s390-ccwchain.h
@@ -0,0 +1,28 @@
+/*
+ * ccwchain interfaces
+ *
+ * Copyright IBM Corp. 2016
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License (version 2 only)
+ * as published by the Free Software Foundation.
+ *
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ */
+
+#ifndef S390_CCWCHAIN_H
+#define S390_CCWCHAIN_H
+
+typedef struct TransChainData {
+    hwaddr cpa_gpa;
+    uint64_t ccwchain_list;
+    uint64_t ccwchain_buf;
+    uint32_t ccwchain_nr;
+} TransChainData;
+
+int ccwchain_translate_to_userspace(TransChainData *chain_data);
+void ccwchain_update_scsw(SCSW *scsw, TransChainData *chain_data);
+void ccwchain_list_free(TransChainData *chain_data);
+
+#endif
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 4/9] s390x/css: add s390-map-css machine option
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

We want to support real (i.e. not virtual) channel devices
even for guests that do not support MCSS-E (where guests may
see devices from any channel subsystem image at once). As all
virtio-ccw devices are in css 0xfe (and show up in the default
css 0 for guests not activating MCSS-E), we need an option to
map e.g. passed-through channel devices from their real css (0-3,
or 0 for hosts not activating MCSS-E) into the default css.
This will be exploited in a later patch.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/s390-virtio-ccw.c         | 23 +++++++++++++++++++++++
 include/hw/s390x/s390-virtio-ccw.h |  1 +
 qemu-options.hx                    |  6 +++++-
 target-s390x/cpu.h                 | 10 ++++++++++
 4 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index e3df9c7..c4706b7 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -237,6 +237,21 @@ static inline void machine_set_dea_key_wrap(Object *obj, bool value,
     ms->dea_key_wrap = value;
 }
 
+static inline bool machine_get_map_css(Object *obj, Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    return ms->s390_map_css;
+}
+
+static inline void machine_set_map_css(Object *obj, bool value,
+                                       Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    ms->s390_map_css = value;
+}
+
 static inline void s390_machine_initfn(Object *obj)
 {
     object_property_add_bool(obj, "aes-key-wrap",
@@ -254,6 +269,14 @@ static inline void s390_machine_initfn(Object *obj)
             "enable/disable DEA key wrapping using the CPACF wrapping key",
             NULL);
     object_property_set_bool(obj, true, "dea-key-wrap", NULL);
+
+    object_property_add_bool(obj, "s390-map-css",
+                             machine_get_map_css,
+                             machine_set_map_css, NULL);
+    object_property_set_description(obj, "s390-map-css",
+            "enable/disable mapping passed-through ccw devices into the "
+            "virtual css", NULL);
+    object_property_set_bool(obj, false, "s390-map-css", NULL);
 }
 
 static const TypeInfo ccw_machine_info = {
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index ab08332..e99f1dd 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -28,6 +28,7 @@ typedef struct S390CcwMachineState {
     /*< public >*/
     bool aes_key_wrap;
     bool dea_key_wrap;
+    bool s390_map_css;
 } S390CcwMachineState;
 
 typedef struct S390CcwMachineClass {
diff --git a/qemu-options.hx b/qemu-options.hx
index 6106520..ce055a7 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -44,7 +44,8 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
     "                dea-key-wrap=on|off controls support for DEA key wrapping (default=on)\n"
     "                suppress-vmdesc=on|off disables self-describing migration (default=off)\n"
     "                nvdimm=on|off controls NVDIMM support (default=off)\n"
-    "                enforce-config-section=on|off enforce configuration section migration (default=off)\n",
+    "                enforce-config-section=on|off enforce configuration section migration (default=off)\n"
+    "                s390-map-css=on|off controls support for mapping into virtual css (default=off)\n",
     QEMU_ARCH_ALL)
 STEXI
 @item -machine [type=]@var{name}[,prop=@var{value}[,...]]
@@ -85,6 +86,9 @@ controls whether DEA wrapping keys will be created to allow
 execution of DEA cryptographic functions.  The default is on.
 @item nvdimm=on|off
 Enables or disables NVDIMM support. The default is off.
+@item s390-map-css=on|off
+Enables or disables mapping passed-through ccw devices into the virtual css.
+The default is off.
 @end table
 ETEXI
 
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 6d97c08..aab0481 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -1279,6 +1279,16 @@ static inline bool vregs_needed(void *opaque)
 }
 #endif
 
+static inline bool s390_get_map_css(void)
+{
+    if (object_property_get_bool(OBJECT(qdev_get_machine()), "s390-map-css",
+                                 NULL)) {
+        return true;
+    }
+
+    return false;
+}
+
 /* machine check interruption code */
 
 /* subclasses */
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 4/9] s390x/css: add s390-map-css machine option
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

We want to support real (i.e. not virtual) channel devices
even for guests that do not support MCSS-E (where guests may
see devices from any channel subsystem image at once). As all
virtio-ccw devices are in css 0xfe (and show up in the default
css 0 for guests not activating MCSS-E), we need an option to
map e.g. passed-through channel devices from their real css (0-3,
or 0 for hosts not activating MCSS-E) into the default css.
This will be exploited in a later patch.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/s390-virtio-ccw.c         | 23 +++++++++++++++++++++++
 include/hw/s390x/s390-virtio-ccw.h |  1 +
 qemu-options.hx                    |  6 +++++-
 target-s390x/cpu.h                 | 10 ++++++++++
 4 files changed, 39 insertions(+), 1 deletion(-)

diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index e3df9c7..c4706b7 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -237,6 +237,21 @@ static inline void machine_set_dea_key_wrap(Object *obj, bool value,
     ms->dea_key_wrap = value;
 }
 
+static inline bool machine_get_map_css(Object *obj, Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    return ms->s390_map_css;
+}
+
+static inline void machine_set_map_css(Object *obj, bool value,
+                                       Error **errp)
+{
+    S390CcwMachineState *ms = S390_CCW_MACHINE(obj);
+
+    ms->s390_map_css = value;
+}
+
 static inline void s390_machine_initfn(Object *obj)
 {
     object_property_add_bool(obj, "aes-key-wrap",
@@ -254,6 +269,14 @@ static inline void s390_machine_initfn(Object *obj)
             "enable/disable DEA key wrapping using the CPACF wrapping key",
             NULL);
     object_property_set_bool(obj, true, "dea-key-wrap", NULL);
+
+    object_property_add_bool(obj, "s390-map-css",
+                             machine_get_map_css,
+                             machine_set_map_css, NULL);
+    object_property_set_description(obj, "s390-map-css",
+            "enable/disable mapping passed-through ccw devices into the "
+            "virtual css", NULL);
+    object_property_set_bool(obj, false, "s390-map-css", NULL);
 }
 
 static const TypeInfo ccw_machine_info = {
diff --git a/include/hw/s390x/s390-virtio-ccw.h b/include/hw/s390x/s390-virtio-ccw.h
index ab08332..e99f1dd 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -28,6 +28,7 @@ typedef struct S390CcwMachineState {
     /*< public >*/
     bool aes_key_wrap;
     bool dea_key_wrap;
+    bool s390_map_css;
 } S390CcwMachineState;
 
 typedef struct S390CcwMachineClass {
diff --git a/qemu-options.hx b/qemu-options.hx
index 6106520..ce055a7 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -44,7 +44,8 @@ DEF("machine", HAS_ARG, QEMU_OPTION_machine, \
     "                dea-key-wrap=on|off controls support for DEA key wrapping (default=on)\n"
     "                suppress-vmdesc=on|off disables self-describing migration (default=off)\n"
     "                nvdimm=on|off controls NVDIMM support (default=off)\n"
-    "                enforce-config-section=on|off enforce configuration section migration (default=off)\n",
+    "                enforce-config-section=on|off enforce configuration section migration (default=off)\n"
+    "                s390-map-css=on|off controls support for mapping into virtual css (default=off)\n",
     QEMU_ARCH_ALL)
 STEXI
 @item -machine [type=]@var{name}[,prop=@var{value}[,...]]
@@ -85,6 +86,9 @@ controls whether DEA wrapping keys will be created to allow
 execution of DEA cryptographic functions.  The default is on.
 @item nvdimm=on|off
 Enables or disables NVDIMM support. The default is off.
+@item s390-map-css=on|off
+Enables or disables mapping passed-through ccw devices into the virtual css.
+The default is off.
 @end table
 ETEXI
 
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 6d97c08..aab0481 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -1279,6 +1279,16 @@ static inline bool vregs_needed(void *opaque)
 }
 #endif
 
+static inline bool s390_get_map_css(void)
+{
+    if (object_property_get_bool(OBJECT(qdev_get_machine()), "s390-map-css",
+                                 NULL)) {
+        return true;
+    }
+
+    return false;
+}
+
 /* machine check interruption code */
 
 /* subclasses */
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 5/9] s390x/css: realize css_sch_build_schib
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

The S390 virtual css support already has a mechanism to build virtual
schib and provide virtual subchannels to the guest. However, to
pass-through ccw device to a guest, we need to introduce a new
mechanism to build its schib according to the real device information.
Thus we realize a new css_sch_build_schib function to do this. And to
reuse the existing code, we refactor css_add_virtual_chpid to
css_add_chpid.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/css.c | 153 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 hw/s390x/css.h |   1 +
 2 files changed, 151 insertions(+), 3 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 3a1d919..5d02ad3 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -11,6 +11,7 @@
 
 #include "qemu/osdep.h"
 #include <hw/qdev.h>
+#include "qemu/error-report.h"
 #include "qemu/bitops.h"
 #include "exec/address-spaces.h"
 #include "cpu.h"
@@ -1236,7 +1237,8 @@ bool css_schid_final(int m, uint8_t cssid, uint8_t ssid, uint16_t schid)
                                  (MAX_SCHID + 1) / sizeof(unsigned long));
 }
 
-static int css_add_virtual_chpid(uint8_t cssid, uint8_t chpid, uint8_t type)
+static int css_add_chpid(uint8_t cssid, uint8_t chpid, uint8_t type,
+                         bool is_virt)
 {
     CssImage *css;
 
@@ -1253,7 +1255,7 @@ static int css_add_virtual_chpid(uint8_t cssid, uint8_t chpid, uint8_t type)
     }
     css->chpids[chpid].in_use = 1;
     css->chpids[chpid].type = type;
-    css->chpids[chpid].is_virtual = 1;
+    css->chpids[chpid].is_virtual = is_virt;
 
     css_generate_chp_crws(cssid, chpid);
 
@@ -1277,7 +1279,7 @@ void css_sch_build_virtual_schib(SubchDev *sch, uint8_t chpid, uint8_t type)
     p->pam = 0x80;
     p->chpid[0] = chpid;
     if (!css->chpids[chpid].in_use) {
-        css_add_virtual_chpid(sch->cssid, chpid, type);
+        css_add_chpid(sch->cssid, chpid, type, true);
     }
 
     memset(s, 0, sizeof(SCSW));
@@ -1644,3 +1646,148 @@ void css_reset(void)
     channel_subsys.max_cssid = 0;
     channel_subsys.max_ssid = 0;
 }
+
+static int css_sch_get_chpids(SubchDev *sch, const char *hschid, uint32_t cssid)
+{
+    char *fid_path;
+    FILE *fd;
+    uint32_t chpid[8];
+    int i;
+    PMCW *p = &sch->curr_status.pmcw;
+
+    fid_path = g_strdup_printf("/sys/devices/css%x/%s/chpids",
+                               cssid, hschid);
+    fd = fopen(fid_path, "r");
+    if (fd == NULL) {
+        error_report("%s: open %s failed", __func__, fid_path);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    if (fscanf(fd, "%x %x %x %x %x %x %x %x",
+        &chpid[0], &chpid[1], &chpid[2], &chpid[3],
+        &chpid[4], &chpid[5], &chpid[6], &chpid[7]) != 8) {
+        fclose(fd);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    for (i = 0; i < ARRAY_SIZE(p->chpid); i++) {
+        p->chpid[i] = chpid[i];
+    }
+
+    fclose(fd);
+    g_free(fid_path);
+
+    return 0;
+}
+
+static int css_sch_get_path_masks(SubchDev *sch, const char *hschid,
+                                  uint32_t cssid)
+{
+    char *fid_path;
+    FILE *fd;
+    uint32_t pim, pam, pom;
+    PMCW *p = &sch->curr_status.pmcw;
+
+    fid_path = g_strdup_printf("/sys/devices/css%x/%s/pimpampom",
+                               cssid, hschid);
+    fd = fopen(fid_path, "r");
+    if (fd == NULL) {
+        error_report("%s: open %s failed", __func__, fid_path);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    if (fscanf(fd, "%x %x %x", &pim, &pam, &pom) != 3) {
+        fclose(fd);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    p->pim = pim;
+    p->pam = pam;
+    p->pom = pom;
+    fclose(fd);
+    g_free(fid_path);
+
+    return 0;
+}
+
+static int css_sch_get_chpid_type(uint8_t chpid, uint32_t *type, uint32_t cssid)
+{
+    char *fid_path;
+    FILE *fd;
+
+    fid_path = g_strdup_printf("/sys/devices/css%x/chp0.%02x/type",
+                               cssid, chpid);
+    fd = fopen(fid_path, "r");
+    if (fd == NULL) {
+        error_report("%s: open %s failed", __func__, fid_path);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    if (fscanf(fd, "%x", type) != 1) {
+        fclose(fd);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    fclose(fd);
+    g_free(fid_path);
+
+    return 0;
+}
+
+int css_sch_build_schib(SubchDev *sch, const char *hschid)
+{
+    PMCW *p = &sch->curr_status.pmcw;
+    SCSW *s = &sch->curr_status.scsw;
+    int i, ret;
+    uint32_t type, cssid;
+    CssImage *css = channel_subsys.css[sch->cssid];
+
+    /* We are dealing with I/O subchannels only. */
+    assert(css != NULL);
+    memset(p, 0, sizeof(PMCW));
+    p->flags |= PMCW_FLAGS_MASK_DNV;
+    p->devno = sch->devno;
+
+    ret = sscanf(hschid, "%1x", &cssid);
+    if (ret != 1) {
+        error_report("%s: Get cssid from host schid %s failed",
+                     __func__, hschid);
+    }
+
+    /* Grab path mask from sysfs. */
+    ret = css_sch_get_path_masks(sch, hschid, cssid);
+    if (ret) {
+        return ret;
+    }
+
+    /* Grab chpids from sysfs. */
+    ret = css_sch_get_chpids(sch, hschid, cssid);
+    if (ret) {
+        return ret;
+    }
+
+   /* Build chpid type. */
+    for (i = 0; i < ARRAY_SIZE(p->chpid); i++) {
+        if (p->chpid[i] && !css->chpids[p->chpid[i]].in_use) {
+            ret = css_sch_get_chpid_type(p->chpid[i], &type, cssid);
+            if (ret) {
+                return ret;
+            }
+            css_add_chpid(sch->cssid, p->chpid[i], type, false);
+        }
+    }
+
+    memset(s, 0, sizeof(SCSW));
+    sch->curr_status.mba = 0;
+    for (i = 0; i < ARRAY_SIZE(sch->curr_status.mda); i++) {
+        sch->curr_status.mda[i] = 0;
+    }
+
+    return 0;
+}
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index a320eea..bd45951 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -119,6 +119,7 @@ void css_generate_chp_crws(uint8_t cssid, uint8_t chpid);
 void css_generate_css_crws(uint8_t cssid);
 void css_clear_sei_pending(void);
 void css_adapter_interrupt(uint8_t isc);
+int css_sch_build_schib(SubchDev *sch, const char *hschid);
 
 #define CSS_IO_ADAPTER_VIRTIO 1
 int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 5/9] s390x/css: realize css_sch_build_schib
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

The S390 virtual css support already has a mechanism to build virtual
schib and provide virtual subchannels to the guest. However, to
pass-through ccw device to a guest, we need to introduce a new
mechanism to build its schib according to the real device information.
Thus we realize a new css_sch_build_schib function to do this. And to
reuse the existing code, we refactor css_add_virtual_chpid to
css_add_chpid.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/css.c | 153 +++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 hw/s390x/css.h |   1 +
 2 files changed, 151 insertions(+), 3 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 3a1d919..5d02ad3 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -11,6 +11,7 @@
 
 #include "qemu/osdep.h"
 #include <hw/qdev.h>
+#include "qemu/error-report.h"
 #include "qemu/bitops.h"
 #include "exec/address-spaces.h"
 #include "cpu.h"
@@ -1236,7 +1237,8 @@ bool css_schid_final(int m, uint8_t cssid, uint8_t ssid, uint16_t schid)
                                  (MAX_SCHID + 1) / sizeof(unsigned long));
 }
 
-static int css_add_virtual_chpid(uint8_t cssid, uint8_t chpid, uint8_t type)
+static int css_add_chpid(uint8_t cssid, uint8_t chpid, uint8_t type,
+                         bool is_virt)
 {
     CssImage *css;
 
@@ -1253,7 +1255,7 @@ static int css_add_virtual_chpid(uint8_t cssid, uint8_t chpid, uint8_t type)
     }
     css->chpids[chpid].in_use = 1;
     css->chpids[chpid].type = type;
-    css->chpids[chpid].is_virtual = 1;
+    css->chpids[chpid].is_virtual = is_virt;
 
     css_generate_chp_crws(cssid, chpid);
 
@@ -1277,7 +1279,7 @@ void css_sch_build_virtual_schib(SubchDev *sch, uint8_t chpid, uint8_t type)
     p->pam = 0x80;
     p->chpid[0] = chpid;
     if (!css->chpids[chpid].in_use) {
-        css_add_virtual_chpid(sch->cssid, chpid, type);
+        css_add_chpid(sch->cssid, chpid, type, true);
     }
 
     memset(s, 0, sizeof(SCSW));
@@ -1644,3 +1646,148 @@ void css_reset(void)
     channel_subsys.max_cssid = 0;
     channel_subsys.max_ssid = 0;
 }
+
+static int css_sch_get_chpids(SubchDev *sch, const char *hschid, uint32_t cssid)
+{
+    char *fid_path;
+    FILE *fd;
+    uint32_t chpid[8];
+    int i;
+    PMCW *p = &sch->curr_status.pmcw;
+
+    fid_path = g_strdup_printf("/sys/devices/css%x/%s/chpids",
+                               cssid, hschid);
+    fd = fopen(fid_path, "r");
+    if (fd == NULL) {
+        error_report("%s: open %s failed", __func__, fid_path);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    if (fscanf(fd, "%x %x %x %x %x %x %x %x",
+        &chpid[0], &chpid[1], &chpid[2], &chpid[3],
+        &chpid[4], &chpid[5], &chpid[6], &chpid[7]) != 8) {
+        fclose(fd);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    for (i = 0; i < ARRAY_SIZE(p->chpid); i++) {
+        p->chpid[i] = chpid[i];
+    }
+
+    fclose(fd);
+    g_free(fid_path);
+
+    return 0;
+}
+
+static int css_sch_get_path_masks(SubchDev *sch, const char *hschid,
+                                  uint32_t cssid)
+{
+    char *fid_path;
+    FILE *fd;
+    uint32_t pim, pam, pom;
+    PMCW *p = &sch->curr_status.pmcw;
+
+    fid_path = g_strdup_printf("/sys/devices/css%x/%s/pimpampom",
+                               cssid, hschid);
+    fd = fopen(fid_path, "r");
+    if (fd == NULL) {
+        error_report("%s: open %s failed", __func__, fid_path);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    if (fscanf(fd, "%x %x %x", &pim, &pam, &pom) != 3) {
+        fclose(fd);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    p->pim = pim;
+    p->pam = pam;
+    p->pom = pom;
+    fclose(fd);
+    g_free(fid_path);
+
+    return 0;
+}
+
+static int css_sch_get_chpid_type(uint8_t chpid, uint32_t *type, uint32_t cssid)
+{
+    char *fid_path;
+    FILE *fd;
+
+    fid_path = g_strdup_printf("/sys/devices/css%x/chp0.%02x/type",
+                               cssid, chpid);
+    fd = fopen(fid_path, "r");
+    if (fd == NULL) {
+        error_report("%s: open %s failed", __func__, fid_path);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    if (fscanf(fd, "%x", type) != 1) {
+        fclose(fd);
+        g_free(fid_path);
+        return -EINVAL;
+    }
+
+    fclose(fd);
+    g_free(fid_path);
+
+    return 0;
+}
+
+int css_sch_build_schib(SubchDev *sch, const char *hschid)
+{
+    PMCW *p = &sch->curr_status.pmcw;
+    SCSW *s = &sch->curr_status.scsw;
+    int i, ret;
+    uint32_t type, cssid;
+    CssImage *css = channel_subsys.css[sch->cssid];
+
+    /* We are dealing with I/O subchannels only. */
+    assert(css != NULL);
+    memset(p, 0, sizeof(PMCW));
+    p->flags |= PMCW_FLAGS_MASK_DNV;
+    p->devno = sch->devno;
+
+    ret = sscanf(hschid, "%1x", &cssid);
+    if (ret != 1) {
+        error_report("%s: Get cssid from host schid %s failed",
+                     __func__, hschid);
+    }
+
+    /* Grab path mask from sysfs. */
+    ret = css_sch_get_path_masks(sch, hschid, cssid);
+    if (ret) {
+        return ret;
+    }
+
+    /* Grab chpids from sysfs. */
+    ret = css_sch_get_chpids(sch, hschid, cssid);
+    if (ret) {
+        return ret;
+    }
+
+   /* Build chpid type. */
+    for (i = 0; i < ARRAY_SIZE(p->chpid); i++) {
+        if (p->chpid[i] && !css->chpids[p->chpid[i]].in_use) {
+            ret = css_sch_get_chpid_type(p->chpid[i], &type, cssid);
+            if (ret) {
+                return ret;
+            }
+            css_add_chpid(sch->cssid, p->chpid[i], type, false);
+        }
+    }
+
+    memset(s, 0, sizeof(SCSW));
+    sch->curr_status.mba = 0;
+    for (i = 0; i < ARRAY_SIZE(sch->curr_status.mda); i++) {
+        sch->curr_status.mda[i] = 0;
+    }
+
+    return 0;
+}
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index a320eea..bd45951 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -119,6 +119,7 @@ void css_generate_chp_crws(uint8_t cssid, uint8_t chpid);
 void css_generate_css_crws(uint8_t cssid);
 void css_clear_sei_pending(void);
 void css_adapter_interrupt(uint8_t isc);
+int css_sch_build_schib(SubchDev *sch, const char *hschid);
 
 #define CSS_IO_ADAPTER_VIRTIO 1
 int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 6/9] s390x/css: device and bus support for s390-ccw passthrough
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

In order to support ccw device pass-through, we introduce a s390
ccw device called "s390-ccw" to hold the real device info. As the
existing virtual-css-bridge and virtual-css-bus are designed to
support virtio_ccw devices for S390x virtio machine. We want to put
passthroughed devices on a different bus, so we implement a css bridge
called "s390-ccw-bridge", and a css bus called "s390-ccw-bus", which
is the groundwork of the passed-through vfio_ccw devices.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/Makefile.objs     |   2 +-
 hw/s390x/css.h             |   3 +
 hw/s390x/s390-ccw-bus.c    | 114 +++++++++++++++++++++++++++++
 hw/s390x/s390-ccw-bus.h    |  30 ++++++++
 hw/s390x/s390-ccw.c        | 177 +++++++++++++++++++++++++++++++++++++++++++++
 hw/s390x/s390-ccw.h        |  40 ++++++++++
 hw/s390x/s390-virtio-ccw.c |   3 +
 hw/s390x/virtio-ccw.h      |   2 -
 8 files changed, 368 insertions(+), 3 deletions(-)
 create mode 100644 hw/s390x/s390-ccw-bus.c
 create mode 100644 hw/s390x/s390-ccw-bus.h
 create mode 100644 hw/s390x/s390-ccw.c
 create mode 100644 hw/s390x/s390-ccw.h

diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs
index 35b5d27..214802f 100644
--- a/hw/s390x/Makefile.objs
+++ b/hw/s390x/Makefile.objs
@@ -11,4 +11,4 @@ obj-y += virtio-ccw.o
 obj-y += s390-pci-bus.o s390-pci-inst.o
 obj-y += s390-skeys.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
-obj-y += s390-ccwchain.o
+obj-y += s390-ccwchain.o s390-ccw-bus.o s390-ccw.o
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index bd45951..c280226 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -15,6 +15,7 @@
 #include "hw/s390x/adapter.h"
 #include "hw/s390x/s390_flic.h"
 #include "ioinst.h"
+#include "hw/hw.h"
 
 /* Channel subsystem constants. */
 #define MAX_SCHID 65535
@@ -24,6 +25,8 @@
 
 #define MAX_CIWS 62
 
+#define VIRTUAL_CSSID 0xfe
+
 typedef struct CIW {
     uint8_t type;
     uint8_t command;
diff --git a/hw/s390x/s390-ccw-bus.c b/hw/s390x/s390-ccw-bus.c
new file mode 100644
index 0000000..19f81ac
--- /dev/null
+++ b/hw/s390x/s390-ccw-bus.c
@@ -0,0 +1,114 @@
+/*
+ * s390 CCW BUS
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version
+ * 2 or (at your option) any later version. See the COPYING file
+ * in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "css.h"
+#include "s390-ccw-bus.h"
+#include "s390-ccw.h"
+
+static void s390_ccw_unplug(HotplugHandler *hotplug_dev,
+                            DeviceState *dev, Error **errp)
+{
+    S390CCWDevice *vcdev = DO_UPCAST(S390CCWDevice, parent, dev);
+    SubchDev *sch = vcdev->sch;
+
+    /*
+     * We should arrive here only for device_del, since we don't
+     * support direct hot(un)plug of channels.
+     */
+    assert(sch != NULL);
+    /* Subchannel is now disabled and no longer valid. */
+    sch->curr_status.pmcw.flags &= ~(PMCW_FLAGS_MASK_ENA |
+                                     PMCW_FLAGS_MASK_DNV);
+
+    css_generate_sch_crws(sch->cssid, sch->ssid, sch->schid, 1, 0);
+
+    object_unparent(OBJECT(dev));
+}
+
+static void s390_ccw_bus_reset(BusState *qbus)
+{
+    /* This should actually be modelled via the generic css. */
+    css_reset();
+}
+
+static void s390_ccw_bus_class_init(ObjectClass *klass, void *data)
+{
+    BusClass *k = BUS_CLASS(klass);
+
+    k->reset = s390_ccw_bus_reset;
+}
+
+static const TypeInfo s390_ccw_bus_info = {
+    .name          = TYPE_S390_CCW_BUS,
+    .parent        = TYPE_BUS,
+    .instance_size = sizeof(S390CCWBus),
+    .class_init    = s390_ccw_bus_class_init,
+};
+
+static int s390_ccw_bridge_init(SysBusDevice *dev)
+{
+    /* Do nothing. */
+    return 0;
+}
+
+static void s390_ccw_bridge_class_init(ObjectClass *klass, void *data)
+{
+    SysBusDeviceClass *k = SYS_BUS_DEVICE_CLASS(klass);
+    HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(klass);
+
+    hc->unplug = s390_ccw_unplug;
+    k->init = s390_ccw_bridge_init;
+}
+
+static const TypeInfo s390_ccw_bridge_info = {
+    .name          = TYPE_S390_CCW_BRIDGE,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(SysBusDevice),
+    .class_init    = s390_ccw_bridge_class_init,
+    .interfaces    = (InterfaceInfo[]) {
+        { TYPE_HOTPLUG_HANDLER },
+        { }
+    },
+};
+
+/********************* Create s390 CCW Bridge and Bus ************/
+S390CCWBus *s390_ccw_bus_init(void)
+{
+    S390CCWBus *cbus;
+    BusState *bus;
+    DeviceState *dev;
+
+    /* Create bridge device. */
+    dev = qdev_create(NULL, TYPE_S390_CCW_BRIDGE);
+    qdev_init_nofail(dev);
+
+    /* Create bus on bridge device. */
+    bus = qbus_create(TYPE_S390_CCW_BUS, dev, "s390-ccw");
+    cbus = S390_CCW_BUS(bus);
+
+    cbus->map_vir_css = s390_get_map_css();
+
+    /* Enable hotplugging. */
+    qbus_set_hotplug_handler(bus, dev, &error_abort);
+
+    return cbus;
+}
+
+static void s390_ccw_register(void)
+{
+    type_register_static(&s390_ccw_bus_info);
+    type_register_static(&s390_ccw_bridge_info);
+}
+
+type_init(s390_ccw_register)
diff --git a/hw/s390x/s390-ccw-bus.h b/hw/s390x/s390-ccw-bus.h
new file mode 100644
index 0000000..493c8f5
--- /dev/null
+++ b/hw/s390x/s390-ccw-bus.h
@@ -0,0 +1,30 @@
+/*
+ * s390 CCW Bus and Bridge definitions
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_S390_CCW_BUS_H
+#define HW_S390_CCW_BUS_H
+
+/* s390 CCW Bus Type. */
+typedef struct S390CCWBus {
+    BusState parent_obj;
+    bool map_vir_css;
+} S390CCWBus;
+
+#define TYPE_S390_CCW_BUS "s390-ccw-bus"
+#define S390_CCW_BUS(obj) \
+    OBJECT_CHECK(S390CCWBus, (obj), TYPE_S390_CCW_BUS)
+
+/* s390 CCW Bridge Type. */
+#define TYPE_S390_CCW_BRIDGE "s390-ccw-bridge"
+
+S390CCWBus *s390_ccw_bus_init(void);
+#endif
diff --git a/hw/s390x/s390-ccw.c b/hw/s390x/s390-ccw.c
new file mode 100644
index 0000000..7b378f9
--- /dev/null
+++ b/hw/s390x/s390-ccw.c
@@ -0,0 +1,177 @@
+/*
+ * s390 CCW Assignment Support
+ *
+ * Copyright 2016 IBM Corp
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2
+ * or (at your option) any later version. See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "libgen.h"
+#include "css.h"
+#include "s390-ccw-bus.h"
+#include "s390-ccw.h"
+
+static void s390_ccw_realize(S390CCWDevice *cdev, Error **errp)
+{
+    unsigned int cssid, ssid, devno, schid;
+    bool found;
+    int ret;
+    BusState *qbus;
+    S390CCWBus *cbus;
+    SubchDev *sch;
+    DeviceState *parent = DEVICE(cdev);
+    char dummy_char;
+    char *symlink_target = NULL;
+    char *path = NULL;
+    char *hschid;
+    GError *gerror;
+
+    /* Store host id. */
+    ret = sscanf(cdev->hostid, "%x.%x.%04x%c",
+                 &cdev->hcssid, &cdev->hssid, &cdev->hdevno, &dummy_char);
+    if (ret != 3) {
+        error_setg(errp, "Malformed hostid parameter '%s'", cdev->hostid);
+        return;
+    }
+
+    /* Store guest id. */
+    if (cdev->guestid) {
+        ret = sscanf(cdev->guestid, "%x.%x.%04x%c",
+                     &cssid, &ssid, &devno, &dummy_char);
+        if (ret != 3) {
+            error_setg(errp, "Malformed guestid parameter '%s'",
+                       cdev->guestid);
+            return;
+        }
+        if (cssid == VIRTUAL_CSSID) {
+            error_setg(errp, "Malformed cssid %x of guestid", cssid);
+            return;
+        }
+
+        ret = css_create_css_image(cssid, false);
+        if (ret == -EINVAL) {
+            error_setg(errp, "Invalid cssid: %x", cssid);
+            return;
+        }
+    } else {
+        cssid = cdev->hcssid;
+        ssid = cdev->hssid;
+        devno = cdev->hdevno;
+    }
+
+    qbus = qdev_get_parent_bus(parent);
+    cbus = S390_CCW_BUS(qbus);
+    if (cbus->map_vir_css) {
+        cssid = VIRTUAL_CSSID;
+    }
+
+    if (cssid > MAX_CSSID || ssid > MAX_SSID) {
+        error_setg(errp, "Invalid cssid or ssid: cssid %x, ssid %x",
+                   cssid, ssid);
+        return;
+    }
+
+    if (css_devno_used(cssid, ssid, devno)) {
+        error_setg(errp, "Device %x.%x.%04x already exists",
+                   cssid, ssid, devno);
+        return;
+    }
+
+    sch = g_new0(SubchDev, 1);
+    sch->driver_data = cdev;
+    sch->cssid = cssid;
+    sch->ssid = ssid;
+    sch->devno = devno;
+
+    cdev->sch = NULL;
+
+    /* Find free subchannel id. */
+    found = false;
+    for (schid = 0; schid <= MAX_SCHID; schid++) {
+        if (!css_find_subch(1, sch->cssid, sch->ssid, schid)) {
+            sch->schid = schid;
+            css_subch_assign(sch->cssid, sch->ssid, sch->schid,
+                             sch->devno, sch);
+            found = true;
+            break;
+        }
+    }
+    if (!found) {
+        error_setg(errp, "No free subchannel found for %x.%x.%04x",
+                   sch->cssid, sch->ssid, sch->devno);
+        goto out_err;
+    }
+
+    /* Build initial schib. */
+    path = g_strdup_printf("/sys/bus/ccw/devices/%s", cdev->hostid);
+    symlink_target = g_file_read_link(path, &gerror);
+    if (!symlink_target) {
+        error_setg(errp, "%s: Failed to readlink from: %s",
+                   __func__, path);
+        css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
+        goto out_err;
+    }
+    hschid = basename(dirname(symlink_target));
+    ret = css_sch_build_schib(sch, hschid);
+    if (ret) {
+        error_setg(errp, "%s: Failed to build initial schib: %d",
+                   __func__, ret);
+        css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
+        goto out_err;
+    }
+    css_generate_sch_crws(sch->cssid, sch->ssid, sch->schid,
+                          parent->hotplugged, 1);
+
+    cdev->sch = sch;
+
+out_err:
+    if (!cdev->sch) {
+        g_free(sch);
+    }
+    g_free(symlink_target);
+    g_free(path);
+}
+
+static void s390_ccw_unrealize(S390CCWDevice *cdev, Error **errp)
+{
+    SubchDev *sch = cdev->sch;
+
+    if (sch) {
+        css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
+        g_free(sch);
+        cdev->sch = NULL;
+    }
+}
+
+static void s390_ccw_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    S390CCWDeviceClass *cdc = S390_CCW_DEVICE_CLASS(klass);
+
+    dc->bus_type = TYPE_S390_CCW_BUS;
+    cdc->realize = s390_ccw_realize;
+    cdc->unrealize = s390_ccw_unrealize;
+}
+
+static const TypeInfo s390_ccw_info = {
+    .name          = TYPE_S390_CCW,
+    .parent        = TYPE_DEVICE,
+    .instance_size = sizeof(S390CCWDevice),
+    .abstract      = true,
+    .class_size    = sizeof(S390CCWDeviceClass),
+    .class_init    = s390_ccw_class_init,
+};
+
+static void register_s390_ccw_type(void)
+{
+    type_register_static(&s390_ccw_info);
+}
+
+type_init(register_s390_ccw_type)
diff --git a/hw/s390x/s390-ccw.h b/hw/s390x/s390-ccw.h
new file mode 100644
index 0000000..a3cd9df
--- /dev/null
+++ b/hw/s390x/s390-ccw.h
@@ -0,0 +1,40 @@
+/*
+ * s390 CCW Assignment Support
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_S390_CCW_H
+#define HW_S390_CCW_H
+
+#define TYPE_S390_CCW "s390-ccw"
+#define S390_CCW_DEVICE(obj) \
+    OBJECT_CHECK(S390CCWDevice, (obj), TYPE_S390_CCW)
+#define S390_CCW_DEVICE_CLASS(klass) \
+    OBJECT_CLASS_CHECK(S390CCWDeviceClass, (klass), TYPE_S390_CCW)
+#define S390_CCW_DEVICE_GET_CLASS(obj) \
+    OBJECT_GET_CLASS(S390CCWDeviceClass, (obj), TYPE_S390_CCW)
+
+typedef struct S390CCWDevice {
+    DeviceState parent;
+    SubchDev *sch;
+    char *hostid;
+    char *guestid;
+    unsigned int hcssid;
+    unsigned int hssid;
+    unsigned int hdevno;
+} S390CCWDevice;
+
+typedef struct S390CCWDeviceClass {
+    DeviceClass parent_class;
+    void (*realize)(S390CCWDevice *dev, Error **errp);
+    void (*unrealize)(S390CCWDevice *dev, Error **errp);
+} S390CCWDeviceClass;
+
+#endif
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index c4706b7..1637006 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -23,6 +23,7 @@
 #include "virtio-ccw.h"
 #include "qemu/config-file.h"
 #include "s390-pci-bus.h"
+#include "s390-ccw-bus.h"
 #include "hw/s390x/storage-keys.h"
 #include "hw/compat.h"
 #include "hw/s390x/s390-virtio-ccw.h"
@@ -117,6 +118,8 @@ static void ccw_init(MachineState *machine)
                       machine->initrd_filename, "s390-ccw.img", true);
     s390_flic_init();
 
+    s390_ccw_bus_init();
+
     dev = qdev_create(NULL, TYPE_S390_PCI_HOST_BRIDGE);
     object_property_add_child(qdev_get_machine(), TYPE_S390_PCI_HOST_BRIDGE,
                               OBJECT(dev), NULL);
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 66c831b..90b7769 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -26,8 +26,6 @@
 
 #include "css.h"
 
-#define VIRTUAL_CSSID 0xfe
-
 #define VIRTIO_CCW_CU_TYPE 0x3832
 #define VIRTIO_CCW_CHPID_TYPE 0x32
 
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 6/9] s390x/css: device and bus support for s390-ccw passthrough
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

In order to support ccw device pass-through, we introduce a s390
ccw device called "s390-ccw" to hold the real device info. As the
existing virtual-css-bridge and virtual-css-bus are designed to
support virtio_ccw devices for S390x virtio machine. We want to put
passthroughed devices on a different bus, so we implement a css bridge
called "s390-ccw-bridge", and a css bus called "s390-ccw-bus", which
is the groundwork of the passed-through vfio_ccw devices.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/Makefile.objs     |   2 +-
 hw/s390x/css.h             |   3 +
 hw/s390x/s390-ccw-bus.c    | 114 +++++++++++++++++++++++++++++
 hw/s390x/s390-ccw-bus.h    |  30 ++++++++
 hw/s390x/s390-ccw.c        | 177 +++++++++++++++++++++++++++++++++++++++++++++
 hw/s390x/s390-ccw.h        |  40 ++++++++++
 hw/s390x/s390-virtio-ccw.c |   3 +
 hw/s390x/virtio-ccw.h      |   2 -
 8 files changed, 368 insertions(+), 3 deletions(-)
 create mode 100644 hw/s390x/s390-ccw-bus.c
 create mode 100644 hw/s390x/s390-ccw-bus.h
 create mode 100644 hw/s390x/s390-ccw.c
 create mode 100644 hw/s390x/s390-ccw.h

diff --git a/hw/s390x/Makefile.objs b/hw/s390x/Makefile.objs
index 35b5d27..214802f 100644
--- a/hw/s390x/Makefile.objs
+++ b/hw/s390x/Makefile.objs
@@ -11,4 +11,4 @@ obj-y += virtio-ccw.o
 obj-y += s390-pci-bus.o s390-pci-inst.o
 obj-y += s390-skeys.o
 obj-$(CONFIG_KVM) += s390-skeys-kvm.o
-obj-y += s390-ccwchain.o
+obj-y += s390-ccwchain.o s390-ccw-bus.o s390-ccw.o
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index bd45951..c280226 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -15,6 +15,7 @@
 #include "hw/s390x/adapter.h"
 #include "hw/s390x/s390_flic.h"
 #include "ioinst.h"
+#include "hw/hw.h"
 
 /* Channel subsystem constants. */
 #define MAX_SCHID 65535
@@ -24,6 +25,8 @@
 
 #define MAX_CIWS 62
 
+#define VIRTUAL_CSSID 0xfe
+
 typedef struct CIW {
     uint8_t type;
     uint8_t command;
diff --git a/hw/s390x/s390-ccw-bus.c b/hw/s390x/s390-ccw-bus.c
new file mode 100644
index 0000000..19f81ac
--- /dev/null
+++ b/hw/s390x/s390-ccw-bus.c
@@ -0,0 +1,114 @@
+/*
+ * s390 CCW BUS
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version
+ * 2 or (at your option) any later version. See the COPYING file
+ * in the top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "css.h"
+#include "s390-ccw-bus.h"
+#include "s390-ccw.h"
+
+static void s390_ccw_unplug(HotplugHandler *hotplug_dev,
+                            DeviceState *dev, Error **errp)
+{
+    S390CCWDevice *vcdev = DO_UPCAST(S390CCWDevice, parent, dev);
+    SubchDev *sch = vcdev->sch;
+
+    /*
+     * We should arrive here only for device_del, since we don't
+     * support direct hot(un)plug of channels.
+     */
+    assert(sch != NULL);
+    /* Subchannel is now disabled and no longer valid. */
+    sch->curr_status.pmcw.flags &= ~(PMCW_FLAGS_MASK_ENA |
+                                     PMCW_FLAGS_MASK_DNV);
+
+    css_generate_sch_crws(sch->cssid, sch->ssid, sch->schid, 1, 0);
+
+    object_unparent(OBJECT(dev));
+}
+
+static void s390_ccw_bus_reset(BusState *qbus)
+{
+    /* This should actually be modelled via the generic css. */
+    css_reset();
+}
+
+static void s390_ccw_bus_class_init(ObjectClass *klass, void *data)
+{
+    BusClass *k = BUS_CLASS(klass);
+
+    k->reset = s390_ccw_bus_reset;
+}
+
+static const TypeInfo s390_ccw_bus_info = {
+    .name          = TYPE_S390_CCW_BUS,
+    .parent        = TYPE_BUS,
+    .instance_size = sizeof(S390CCWBus),
+    .class_init    = s390_ccw_bus_class_init,
+};
+
+static int s390_ccw_bridge_init(SysBusDevice *dev)
+{
+    /* Do nothing. */
+    return 0;
+}
+
+static void s390_ccw_bridge_class_init(ObjectClass *klass, void *data)
+{
+    SysBusDeviceClass *k = SYS_BUS_DEVICE_CLASS(klass);
+    HotplugHandlerClass *hc = HOTPLUG_HANDLER_CLASS(klass);
+
+    hc->unplug = s390_ccw_unplug;
+    k->init = s390_ccw_bridge_init;
+}
+
+static const TypeInfo s390_ccw_bridge_info = {
+    .name          = TYPE_S390_CCW_BRIDGE,
+    .parent        = TYPE_SYS_BUS_DEVICE,
+    .instance_size = sizeof(SysBusDevice),
+    .class_init    = s390_ccw_bridge_class_init,
+    .interfaces    = (InterfaceInfo[]) {
+        { TYPE_HOTPLUG_HANDLER },
+        { }
+    },
+};
+
+/********************* Create s390 CCW Bridge and Bus ************/
+S390CCWBus *s390_ccw_bus_init(void)
+{
+    S390CCWBus *cbus;
+    BusState *bus;
+    DeviceState *dev;
+
+    /* Create bridge device. */
+    dev = qdev_create(NULL, TYPE_S390_CCW_BRIDGE);
+    qdev_init_nofail(dev);
+
+    /* Create bus on bridge device. */
+    bus = qbus_create(TYPE_S390_CCW_BUS, dev, "s390-ccw");
+    cbus = S390_CCW_BUS(bus);
+
+    cbus->map_vir_css = s390_get_map_css();
+
+    /* Enable hotplugging. */
+    qbus_set_hotplug_handler(bus, dev, &error_abort);
+
+    return cbus;
+}
+
+static void s390_ccw_register(void)
+{
+    type_register_static(&s390_ccw_bus_info);
+    type_register_static(&s390_ccw_bridge_info);
+}
+
+type_init(s390_ccw_register)
diff --git a/hw/s390x/s390-ccw-bus.h b/hw/s390x/s390-ccw-bus.h
new file mode 100644
index 0000000..493c8f5
--- /dev/null
+++ b/hw/s390x/s390-ccw-bus.h
@@ -0,0 +1,30 @@
+/*
+ * s390 CCW Bus and Bridge definitions
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_S390_CCW_BUS_H
+#define HW_S390_CCW_BUS_H
+
+/* s390 CCW Bus Type. */
+typedef struct S390CCWBus {
+    BusState parent_obj;
+    bool map_vir_css;
+} S390CCWBus;
+
+#define TYPE_S390_CCW_BUS "s390-ccw-bus"
+#define S390_CCW_BUS(obj) \
+    OBJECT_CHECK(S390CCWBus, (obj), TYPE_S390_CCW_BUS)
+
+/* s390 CCW Bridge Type. */
+#define TYPE_S390_CCW_BRIDGE "s390-ccw-bridge"
+
+S390CCWBus *s390_ccw_bus_init(void);
+#endif
diff --git a/hw/s390x/s390-ccw.c b/hw/s390x/s390-ccw.c
new file mode 100644
index 0000000..7b378f9
--- /dev/null
+++ b/hw/s390x/s390-ccw.c
@@ -0,0 +1,177 @@
+/*
+ * s390 CCW Assignment Support
+ *
+ * Copyright 2016 IBM Corp
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2
+ * or (at your option) any later version. See the COPYING file in the
+ * top-level directory.
+ */
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "libgen.h"
+#include "css.h"
+#include "s390-ccw-bus.h"
+#include "s390-ccw.h"
+
+static void s390_ccw_realize(S390CCWDevice *cdev, Error **errp)
+{
+    unsigned int cssid, ssid, devno, schid;
+    bool found;
+    int ret;
+    BusState *qbus;
+    S390CCWBus *cbus;
+    SubchDev *sch;
+    DeviceState *parent = DEVICE(cdev);
+    char dummy_char;
+    char *symlink_target = NULL;
+    char *path = NULL;
+    char *hschid;
+    GError *gerror;
+
+    /* Store host id. */
+    ret = sscanf(cdev->hostid, "%x.%x.%04x%c",
+                 &cdev->hcssid, &cdev->hssid, &cdev->hdevno, &dummy_char);
+    if (ret != 3) {
+        error_setg(errp, "Malformed hostid parameter '%s'", cdev->hostid);
+        return;
+    }
+
+    /* Store guest id. */
+    if (cdev->guestid) {
+        ret = sscanf(cdev->guestid, "%x.%x.%04x%c",
+                     &cssid, &ssid, &devno, &dummy_char);
+        if (ret != 3) {
+            error_setg(errp, "Malformed guestid parameter '%s'",
+                       cdev->guestid);
+            return;
+        }
+        if (cssid == VIRTUAL_CSSID) {
+            error_setg(errp, "Malformed cssid %x of guestid", cssid);
+            return;
+        }
+
+        ret = css_create_css_image(cssid, false);
+        if (ret == -EINVAL) {
+            error_setg(errp, "Invalid cssid: %x", cssid);
+            return;
+        }
+    } else {
+        cssid = cdev->hcssid;
+        ssid = cdev->hssid;
+        devno = cdev->hdevno;
+    }
+
+    qbus = qdev_get_parent_bus(parent);
+    cbus = S390_CCW_BUS(qbus);
+    if (cbus->map_vir_css) {
+        cssid = VIRTUAL_CSSID;
+    }
+
+    if (cssid > MAX_CSSID || ssid > MAX_SSID) {
+        error_setg(errp, "Invalid cssid or ssid: cssid %x, ssid %x",
+                   cssid, ssid);
+        return;
+    }
+
+    if (css_devno_used(cssid, ssid, devno)) {
+        error_setg(errp, "Device %x.%x.%04x already exists",
+                   cssid, ssid, devno);
+        return;
+    }
+
+    sch = g_new0(SubchDev, 1);
+    sch->driver_data = cdev;
+    sch->cssid = cssid;
+    sch->ssid = ssid;
+    sch->devno = devno;
+
+    cdev->sch = NULL;
+
+    /* Find free subchannel id. */
+    found = false;
+    for (schid = 0; schid <= MAX_SCHID; schid++) {
+        if (!css_find_subch(1, sch->cssid, sch->ssid, schid)) {
+            sch->schid = schid;
+            css_subch_assign(sch->cssid, sch->ssid, sch->schid,
+                             sch->devno, sch);
+            found = true;
+            break;
+        }
+    }
+    if (!found) {
+        error_setg(errp, "No free subchannel found for %x.%x.%04x",
+                   sch->cssid, sch->ssid, sch->devno);
+        goto out_err;
+    }
+
+    /* Build initial schib. */
+    path = g_strdup_printf("/sys/bus/ccw/devices/%s", cdev->hostid);
+    symlink_target = g_file_read_link(path, &gerror);
+    if (!symlink_target) {
+        error_setg(errp, "%s: Failed to readlink from: %s",
+                   __func__, path);
+        css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
+        goto out_err;
+    }
+    hschid = basename(dirname(symlink_target));
+    ret = css_sch_build_schib(sch, hschid);
+    if (ret) {
+        error_setg(errp, "%s: Failed to build initial schib: %d",
+                   __func__, ret);
+        css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
+        goto out_err;
+    }
+    css_generate_sch_crws(sch->cssid, sch->ssid, sch->schid,
+                          parent->hotplugged, 1);
+
+    cdev->sch = sch;
+
+out_err:
+    if (!cdev->sch) {
+        g_free(sch);
+    }
+    g_free(symlink_target);
+    g_free(path);
+}
+
+static void s390_ccw_unrealize(S390CCWDevice *cdev, Error **errp)
+{
+    SubchDev *sch = cdev->sch;
+
+    if (sch) {
+        css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
+        g_free(sch);
+        cdev->sch = NULL;
+    }
+}
+
+static void s390_ccw_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+    S390CCWDeviceClass *cdc = S390_CCW_DEVICE_CLASS(klass);
+
+    dc->bus_type = TYPE_S390_CCW_BUS;
+    cdc->realize = s390_ccw_realize;
+    cdc->unrealize = s390_ccw_unrealize;
+}
+
+static const TypeInfo s390_ccw_info = {
+    .name          = TYPE_S390_CCW,
+    .parent        = TYPE_DEVICE,
+    .instance_size = sizeof(S390CCWDevice),
+    .abstract      = true,
+    .class_size    = sizeof(S390CCWDeviceClass),
+    .class_init    = s390_ccw_class_init,
+};
+
+static void register_s390_ccw_type(void)
+{
+    type_register_static(&s390_ccw_info);
+}
+
+type_init(register_s390_ccw_type)
diff --git a/hw/s390x/s390-ccw.h b/hw/s390x/s390-ccw.h
new file mode 100644
index 0000000..a3cd9df
--- /dev/null
+++ b/hw/s390x/s390-ccw.h
@@ -0,0 +1,40 @@
+/*
+ * s390 CCW Assignment Support
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_S390_CCW_H
+#define HW_S390_CCW_H
+
+#define TYPE_S390_CCW "s390-ccw"
+#define S390_CCW_DEVICE(obj) \
+    OBJECT_CHECK(S390CCWDevice, (obj), TYPE_S390_CCW)
+#define S390_CCW_DEVICE_CLASS(klass) \
+    OBJECT_CLASS_CHECK(S390CCWDeviceClass, (klass), TYPE_S390_CCW)
+#define S390_CCW_DEVICE_GET_CLASS(obj) \
+    OBJECT_GET_CLASS(S390CCWDeviceClass, (obj), TYPE_S390_CCW)
+
+typedef struct S390CCWDevice {
+    DeviceState parent;
+    SubchDev *sch;
+    char *hostid;
+    char *guestid;
+    unsigned int hcssid;
+    unsigned int hssid;
+    unsigned int hdevno;
+} S390CCWDevice;
+
+typedef struct S390CCWDeviceClass {
+    DeviceClass parent_class;
+    void (*realize)(S390CCWDevice *dev, Error **errp);
+    void (*unrealize)(S390CCWDevice *dev, Error **errp);
+} S390CCWDeviceClass;
+
+#endif
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index c4706b7..1637006 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -23,6 +23,7 @@
 #include "virtio-ccw.h"
 #include "qemu/config-file.h"
 #include "s390-pci-bus.h"
+#include "s390-ccw-bus.h"
 #include "hw/s390x/storage-keys.h"
 #include "hw/compat.h"
 #include "hw/s390x/s390-virtio-ccw.h"
@@ -117,6 +118,8 @@ static void ccw_init(MachineState *machine)
                       machine->initrd_filename, "s390-ccw.img", true);
     s390_flic_init();
 
+    s390_ccw_bus_init();
+
     dev = qdev_create(NULL, TYPE_S390_PCI_HOST_BRIDGE);
     object_property_add_child(qdev_get_machine(), TYPE_S390_PCI_HOST_BRIDGE,
                               OBJECT(dev), NULL);
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 66c831b..90b7769 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -26,8 +26,6 @@
 
 #include "css.h"
 
-#define VIRTUAL_CSSID 0xfe
-
 #define VIRTIO_CCW_CU_TYPE 0x3832
 #define VIRTIO_CCW_CHPID_TYPE 0x32
 
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 7/9] vfio/ccw: vfio based ccw passthrough driver
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

We use the No-IOMMU mode of VFIO to realize the ccw device
passthrough, implement a vfio based ccw passthrough
driver called "vfio-ccw".

Support qemu parameters in the style of:
"-device vfio-ccw,id=xx,hostid=xx(,guestid=xx)"

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 default-configs/s390x-softmmu.mak |   1 +
 hw/vfio/Makefile.objs             |   1 +
 hw/vfio/ccw.c                     | 188 ++++++++++++++++++++++++++++++++++++++
 include/hw/vfio/vfio-common.h     |   1 +
 4 files changed, 191 insertions(+)
 create mode 100644 hw/vfio/ccw.c

diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak
index 36e15de..5576b0a 100644
--- a/default-configs/s390x-softmmu.mak
+++ b/default-configs/s390x-softmmu.mak
@@ -4,4 +4,5 @@ CONFIG_VIRTIO=y
 CONFIG_SCLPCONSOLE=y
 CONFIG_S390_FLIC=y
 CONFIG_S390_FLIC_KVM=$(CONFIG_KVM)
+CONFIG_VFIO_CCW=$(CONFIG_LINUX)
 CONFIG_WDT_DIAG288=y
diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs
index ceddbb8..2daf4ae 100644
--- a/hw/vfio/Makefile.objs
+++ b/hw/vfio/Makefile.objs
@@ -1,6 +1,7 @@
 ifeq ($(CONFIG_LINUX), y)
 obj-$(CONFIG_SOFTMMU) += common.o
 obj-$(CONFIG_PCI) += pci.o pci-quirks.o
+obj-$(CONFIG_VFIO_CCW) += ccw.o
 obj-$(CONFIG_SOFTMMU) += platform.o
 obj-$(CONFIG_SOFTMMU) += calxeda-xgmac.o
 obj-$(CONFIG_SOFTMMU) += amd-xgbe.o
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
new file mode 100644
index 0000000..501a0e8
--- /dev/null
+++ b/hw/vfio/ccw.c
@@ -0,0 +1,188 @@
+/*
+ * vfio based ccw device assignment support
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or(at
+ * your option) any version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include <linux/vfio.h>
+#include <sys/ioctl.h>
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "hw/vfio/vfio.h"
+#include "hw/vfio/vfio-common.h"
+#include "hw/s390x/s390-ccw.h"
+
+typedef struct VFIOCCWDevice {
+    S390CCWDevice cdev;
+    VFIODevice vdev;
+} VFIOCCWDevice;
+
+static void vfio_ccw_compute_needs_reset(VFIODevice *vdev)
+{
+    vdev->needs_reset = true;
+}
+
+static int vfio_ccw_hot_reset_multi(VFIODevice *vdev)
+{
+    VFIOCCWDevice *vcdev = container_of(vdev, VFIOCCWDevice, vdev);
+
+    return ioctl(vcdev->vdev.fd, VFIO_DEVICE_CCW_HOT_RESET);
+}
+
+/*
+ * vfio_eoi would be called by vfio_region_read and vfio_region_write
+ * which are not used by us for now.
+ * Let's leave this empty in case we need to realize it later.
+ */
+static void vfio_ccw_eoi(VFIODevice *vdev)
+{
+    /* Do nothing. */
+}
+
+static VFIODeviceOps vfio_ccw_ops = {
+    .vfio_compute_needs_reset = vfio_ccw_compute_needs_reset,
+    .vfio_hot_reset_multi = vfio_ccw_hot_reset_multi,
+    .vfio_eoi = vfio_ccw_eoi,
+};
+
+static void vfio_put_device(VFIOCCWDevice *vcdev)
+{
+    g_free(vcdev->vdev.name);
+    vfio_put_base_device(&vcdev->vdev);
+}
+
+static void vfio_ccw_realize(DeviceState *dev, Error **errp)
+{
+    VFIODevice *vbasedev;
+    VFIOGroup *group;
+    S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent, dev);
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    S390CCWDeviceClass *cdc = S390_CCW_DEVICE_GET_CLASS(cdev);
+    char *path = NULL;
+    char *tmp_path = NULL;
+    char *iommu_group_path = NULL;
+    struct stat st;
+    int  groupid, ret;
+    GError *gerror;
+
+    /* Call the class init function for ccw device. */
+    if (cdc->realize) {
+        cdc->realize(cdev, errp);
+        if (*errp) {
+            return;
+        }
+    }
+
+    /* Check that host device exists. */
+    path = g_strdup_printf("/sys/bus/ccw/devices/%x.%x.%04x",
+                           cdev->hcssid, cdev->hssid, cdev->hdevno);
+    if (stat(path, &st) < 0) {
+        error_setg(errp, "vfio: no such host device %s", path);
+        goto out_err;
+    }
+
+    vcdev->vdev.ops = &vfio_ccw_ops;
+    vcdev->vdev.type = VFIO_DEVICE_TYPE_CCW;
+    vcdev->vdev.name = g_strdup_printf("%02x.%02x.%04x", cdev->hcssid,
+                                       cdev->hssid, cdev->hdevno);
+
+    tmp_path = g_strconcat(path, "/iommu_group", NULL);
+
+    iommu_group_path = g_file_read_link(tmp_path, &gerror);
+    if (!iommu_group_path) {
+        error_setg(errp, "vfio: error no iommu_group for device");
+        goto out_err;
+    }
+
+    if (sscanf(basename(iommu_group_path), "%d", &groupid) != 1) {
+        error_setg(errp, "vfio: error reading %s:%m", iommu_group_path);
+        goto out_err;
+    }
+
+    group = vfio_get_group(groupid, NULL);
+    if (!group) {
+        error_setg(errp, "vfio: failed to get the group %d", groupid);
+        goto out_err;
+    }
+
+    QLIST_FOREACH(vbasedev, &group->device_list, next) {
+        if (strcmp(vbasedev->name, vcdev->vdev.name) == 0) {
+            error_setg(errp, "vfio: device %s has already been attached",
+                       basename(path));
+            vfio_put_group(group);
+            goto out_err;
+        }
+    }
+
+    ret = vfio_get_device(group, basename(path) , &vcdev->vdev);
+    if (ret) {
+        error_setg(errp, "vfio: failed to get device %s", basename(path));
+        vfio_put_group(group);
+        goto out_err;
+    }
+
+out_err:
+    g_free(path);
+    g_free(tmp_path);
+    g_free(iommu_group_path);
+}
+
+static void vfio_ccw_unrealize(DeviceState *dev, Error **errp)
+{
+    S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent, dev);
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    S390CCWDeviceClass *cdc = S390_CCW_DEVICE_GET_CLASS(cdev);
+    VFIOGroup *group = vcdev->vdev.group;
+
+    if (cdc->unrealize) {
+        cdc->unrealize(cdev, errp);
+    }
+
+    vfio_put_device(vcdev);
+    vfio_put_group(group);
+}
+
+static Property vfio_ccw_properties[] = {
+    DEFINE_PROP_STRING("hostid", VFIOCCWDevice, cdev.hostid),
+    DEFINE_PROP_STRING("guestid", VFIOCCWDevice, cdev.guestid),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static const VMStateDescription vfio_ccw_vmstate = {
+    .name = "vfio-ccw",
+    .unmigratable = 1,
+};
+
+static void vfio_ccw_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->props = vfio_ccw_properties;
+    dc->vmsd = &vfio_ccw_vmstate;
+    dc->desc = "VFIO-based CCW device assignment";
+    dc->realize = vfio_ccw_realize;
+    dc->unrealize = vfio_ccw_unrealize;
+}
+
+static const TypeInfo vfio_ccw_info = {
+    .name = "vfio-ccw",
+    .parent = TYPE_S390_CCW,
+    .instance_size = sizeof(VFIOCCWDevice),
+    .class_init = vfio_ccw_class_init,
+};
+
+static void register_vfio_ccw_type(void)
+{
+    type_register_static(&vfio_ccw_info);
+}
+
+type_init(register_vfio_ccw_type)
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 85c2a74..b751506 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -41,6 +41,7 @@
 enum {
     VFIO_DEVICE_TYPE_PCI = 0,
     VFIO_DEVICE_TYPE_PLATFORM = 1,
+    VFIO_DEVICE_TYPE_CCW = 2,
 };
 
 typedef struct VFIOMmap {
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 7/9] vfio/ccw: vfio based ccw passthrough driver
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

We use the No-IOMMU mode of VFIO to realize the ccw device
passthrough, implement a vfio based ccw passthrough
driver called "vfio-ccw".

Support qemu parameters in the style of:
"-device vfio-ccw,id=xx,hostid=xx(,guestid=xx)"

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 default-configs/s390x-softmmu.mak |   1 +
 hw/vfio/Makefile.objs             |   1 +
 hw/vfio/ccw.c                     | 188 ++++++++++++++++++++++++++++++++++++++
 include/hw/vfio/vfio-common.h     |   1 +
 4 files changed, 191 insertions(+)
 create mode 100644 hw/vfio/ccw.c

diff --git a/default-configs/s390x-softmmu.mak b/default-configs/s390x-softmmu.mak
index 36e15de..5576b0a 100644
--- a/default-configs/s390x-softmmu.mak
+++ b/default-configs/s390x-softmmu.mak
@@ -4,4 +4,5 @@ CONFIG_VIRTIO=y
 CONFIG_SCLPCONSOLE=y
 CONFIG_S390_FLIC=y
 CONFIG_S390_FLIC_KVM=$(CONFIG_KVM)
+CONFIG_VFIO_CCW=$(CONFIG_LINUX)
 CONFIG_WDT_DIAG288=y
diff --git a/hw/vfio/Makefile.objs b/hw/vfio/Makefile.objs
index ceddbb8..2daf4ae 100644
--- a/hw/vfio/Makefile.objs
+++ b/hw/vfio/Makefile.objs
@@ -1,6 +1,7 @@
 ifeq ($(CONFIG_LINUX), y)
 obj-$(CONFIG_SOFTMMU) += common.o
 obj-$(CONFIG_PCI) += pci.o pci-quirks.o
+obj-$(CONFIG_VFIO_CCW) += ccw.o
 obj-$(CONFIG_SOFTMMU) += platform.o
 obj-$(CONFIG_SOFTMMU) += calxeda-xgmac.o
 obj-$(CONFIG_SOFTMMU) += amd-xgbe.o
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
new file mode 100644
index 0000000..501a0e8
--- /dev/null
+++ b/hw/vfio/ccw.c
@@ -0,0 +1,188 @@
+/*
+ * vfio based ccw device assignment support
+ *
+ * Copyright 2016 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or(at
+ * your option) any version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include <linux/vfio.h>
+#include <sys/ioctl.h>
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/sysbus.h"
+#include "hw/vfio/vfio.h"
+#include "hw/vfio/vfio-common.h"
+#include "hw/s390x/s390-ccw.h"
+
+typedef struct VFIOCCWDevice {
+    S390CCWDevice cdev;
+    VFIODevice vdev;
+} VFIOCCWDevice;
+
+static void vfio_ccw_compute_needs_reset(VFIODevice *vdev)
+{
+    vdev->needs_reset = true;
+}
+
+static int vfio_ccw_hot_reset_multi(VFIODevice *vdev)
+{
+    VFIOCCWDevice *vcdev = container_of(vdev, VFIOCCWDevice, vdev);
+
+    return ioctl(vcdev->vdev.fd, VFIO_DEVICE_CCW_HOT_RESET);
+}
+
+/*
+ * vfio_eoi would be called by vfio_region_read and vfio_region_write
+ * which are not used by us for now.
+ * Let's leave this empty in case we need to realize it later.
+ */
+static void vfio_ccw_eoi(VFIODevice *vdev)
+{
+    /* Do nothing. */
+}
+
+static VFIODeviceOps vfio_ccw_ops = {
+    .vfio_compute_needs_reset = vfio_ccw_compute_needs_reset,
+    .vfio_hot_reset_multi = vfio_ccw_hot_reset_multi,
+    .vfio_eoi = vfio_ccw_eoi,
+};
+
+static void vfio_put_device(VFIOCCWDevice *vcdev)
+{
+    g_free(vcdev->vdev.name);
+    vfio_put_base_device(&vcdev->vdev);
+}
+
+static void vfio_ccw_realize(DeviceState *dev, Error **errp)
+{
+    VFIODevice *vbasedev;
+    VFIOGroup *group;
+    S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent, dev);
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    S390CCWDeviceClass *cdc = S390_CCW_DEVICE_GET_CLASS(cdev);
+    char *path = NULL;
+    char *tmp_path = NULL;
+    char *iommu_group_path = NULL;
+    struct stat st;
+    int  groupid, ret;
+    GError *gerror;
+
+    /* Call the class init function for ccw device. */
+    if (cdc->realize) {
+        cdc->realize(cdev, errp);
+        if (*errp) {
+            return;
+        }
+    }
+
+    /* Check that host device exists. */
+    path = g_strdup_printf("/sys/bus/ccw/devices/%x.%x.%04x",
+                           cdev->hcssid, cdev->hssid, cdev->hdevno);
+    if (stat(path, &st) < 0) {
+        error_setg(errp, "vfio: no such host device %s", path);
+        goto out_err;
+    }
+
+    vcdev->vdev.ops = &vfio_ccw_ops;
+    vcdev->vdev.type = VFIO_DEVICE_TYPE_CCW;
+    vcdev->vdev.name = g_strdup_printf("%02x.%02x.%04x", cdev->hcssid,
+                                       cdev->hssid, cdev->hdevno);
+
+    tmp_path = g_strconcat(path, "/iommu_group", NULL);
+
+    iommu_group_path = g_file_read_link(tmp_path, &gerror);
+    if (!iommu_group_path) {
+        error_setg(errp, "vfio: error no iommu_group for device");
+        goto out_err;
+    }
+
+    if (sscanf(basename(iommu_group_path), "%d", &groupid) != 1) {
+        error_setg(errp, "vfio: error reading %s:%m", iommu_group_path);
+        goto out_err;
+    }
+
+    group = vfio_get_group(groupid, NULL);
+    if (!group) {
+        error_setg(errp, "vfio: failed to get the group %d", groupid);
+        goto out_err;
+    }
+
+    QLIST_FOREACH(vbasedev, &group->device_list, next) {
+        if (strcmp(vbasedev->name, vcdev->vdev.name) == 0) {
+            error_setg(errp, "vfio: device %s has already been attached",
+                       basename(path));
+            vfio_put_group(group);
+            goto out_err;
+        }
+    }
+
+    ret = vfio_get_device(group, basename(path) , &vcdev->vdev);
+    if (ret) {
+        error_setg(errp, "vfio: failed to get device %s", basename(path));
+        vfio_put_group(group);
+        goto out_err;
+    }
+
+out_err:
+    g_free(path);
+    g_free(tmp_path);
+    g_free(iommu_group_path);
+}
+
+static void vfio_ccw_unrealize(DeviceState *dev, Error **errp)
+{
+    S390CCWDevice *cdev = DO_UPCAST(S390CCWDevice, parent, dev);
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    S390CCWDeviceClass *cdc = S390_CCW_DEVICE_GET_CLASS(cdev);
+    VFIOGroup *group = vcdev->vdev.group;
+
+    if (cdc->unrealize) {
+        cdc->unrealize(cdev, errp);
+    }
+
+    vfio_put_device(vcdev);
+    vfio_put_group(group);
+}
+
+static Property vfio_ccw_properties[] = {
+    DEFINE_PROP_STRING("hostid", VFIOCCWDevice, cdev.hostid),
+    DEFINE_PROP_STRING("guestid", VFIOCCWDevice, cdev.guestid),
+    DEFINE_PROP_END_OF_LIST(),
+};
+
+static const VMStateDescription vfio_ccw_vmstate = {
+    .name = "vfio-ccw",
+    .unmigratable = 1,
+};
+
+static void vfio_ccw_class_init(ObjectClass *klass, void *data)
+{
+    DeviceClass *dc = DEVICE_CLASS(klass);
+
+    dc->props = vfio_ccw_properties;
+    dc->vmsd = &vfio_ccw_vmstate;
+    dc->desc = "VFIO-based CCW device assignment";
+    dc->realize = vfio_ccw_realize;
+    dc->unrealize = vfio_ccw_unrealize;
+}
+
+static const TypeInfo vfio_ccw_info = {
+    .name = "vfio-ccw",
+    .parent = TYPE_S390_CCW,
+    .instance_size = sizeof(VFIOCCWDevice),
+    .class_init = vfio_ccw_class_init,
+};
+
+static void register_vfio_ccw_type(void)
+{
+    type_register_static(&vfio_ccw_info);
+}
+
+type_init(register_vfio_ccw_type)
diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
index 85c2a74..b751506 100644
--- a/include/hw/vfio/vfio-common.h
+++ b/include/hw/vfio/vfio-common.h
@@ -41,6 +41,7 @@
 enum {
     VFIO_DEVICE_TYPE_PCI = 0,
     VFIO_DEVICE_TYPE_PLATFORM = 1,
+    VFIO_DEVICE_TYPE_CCW = 2,
 };
 
 typedef struct VFIOMmap {
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 8/9] s390x/css: introduce and realize ccw-request callback
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Introduce a new callback on ccw device to handle ccw-request.
Realize the callback in vfio-ccw device.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/s390-ccw.h |  1 +
 hw/vfio/ccw.c       | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)

diff --git a/hw/s390x/s390-ccw.h b/hw/s390x/s390-ccw.h
index a3cd9df..92d076d 100644
--- a/hw/s390x/s390-ccw.h
+++ b/hw/s390x/s390-ccw.h
@@ -29,6 +29,7 @@ typedef struct S390CCWDevice {
     unsigned int hcssid;
     unsigned int hssid;
     unsigned int hdevno;
+    int (*handle_request) (ORB *, IRB *, SCSW *, void *);
 } S390CCWDevice;
 
 typedef struct S390CCWDeviceClass {
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
index 501a0e8..6b23291 100644
--- a/hw/vfio/ccw.c
+++ b/hw/vfio/ccw.c
@@ -20,12 +20,52 @@
 #include "hw/vfio/vfio.h"
 #include "hw/vfio/vfio-common.h"
 #include "hw/s390x/s390-ccw.h"
+#include "hw/s390x/s390-ccwchain.h"
 
 typedef struct VFIOCCWDevice {
     S390CCWDevice cdev;
     VFIODevice vdev;
 } VFIOCCWDevice;
 
+static int vfio_ccw_handle_request(ORB *orb, IRB *irb, SCSW *scsw, void *data)
+{
+    struct vfio_ccw_cmd cmd_data;
+    int ret;
+    TransChainData trans_data;
+    S390CCWDevice *cdev = data;
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+
+    QEMU_BUILD_BUG_ON(sizeof(cmd_data.orb_area) != sizeof(ORB));
+    QEMU_BUILD_BUG_ON(sizeof(cmd_data.scsw_area) != sizeof(SCSW));
+    QEMU_BUILD_BUG_ON(sizeof(cmd_data.irb_area) != sizeof(IRB));
+
+    trans_data.cpa_gpa = orb->cpa;
+    ret = ccwchain_translate_to_userspace(&trans_data);
+    if (ret) {
+        return ret;
+    }
+    cmd_data.argsz = sizeof(cmd_data);
+    cmd_data.cssid = cdev->hcssid;
+    cmd_data.ssid = cdev->hssid;
+    cmd_data.devno = cdev->hdevno;
+    cmd_data.ccwchain_nr = trans_data.ccwchain_nr;
+    cmd_data.ccwchain_buf = trans_data.ccwchain_buf;
+
+    memcpy(cmd_data.orb_area, orb, sizeof(ORB));
+    memcpy(cmd_data.scsw_area, scsw, sizeof(SCSW));
+
+    if (ioctl(vcdev->vdev.fd, VFIO_DEVICE_CCW_CMD_REQUEST, &cmd_data) == -1) {
+        ccwchain_list_free(&trans_data);
+        return -errno;
+    }
+
+    memcpy(irb, cmd_data.irb_area, sizeof(IRB));
+    ccwchain_update_scsw(&(irb->scsw), &trans_data);
+    ccwchain_list_free(&trans_data);
+
+    return 0;
+}
+
 static void vfio_ccw_compute_needs_reset(VFIODevice *vdev)
 {
     vdev->needs_reset = true;
@@ -75,6 +115,8 @@ static void vfio_ccw_realize(DeviceState *dev, Error **errp)
     GError *gerror;
 
     /* Call the class init function for ccw device. */
+    cdev->handle_request = vfio_ccw_handle_request;
+
     if (cdc->realize) {
         cdc->realize(cdev, errp);
         if (*errp) {
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 8/9] s390x/css: introduce and realize ccw-request callback
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Introduce a new callback on ccw device to handle ccw-request.
Realize the callback in vfio-ccw device.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/s390-ccw.h |  1 +
 hw/vfio/ccw.c       | 42 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 43 insertions(+)

diff --git a/hw/s390x/s390-ccw.h b/hw/s390x/s390-ccw.h
index a3cd9df..92d076d 100644
--- a/hw/s390x/s390-ccw.h
+++ b/hw/s390x/s390-ccw.h
@@ -29,6 +29,7 @@ typedef struct S390CCWDevice {
     unsigned int hcssid;
     unsigned int hssid;
     unsigned int hdevno;
+    int (*handle_request) (ORB *, IRB *, SCSW *, void *);
 } S390CCWDevice;
 
 typedef struct S390CCWDeviceClass {
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
index 501a0e8..6b23291 100644
--- a/hw/vfio/ccw.c
+++ b/hw/vfio/ccw.c
@@ -20,12 +20,52 @@
 #include "hw/vfio/vfio.h"
 #include "hw/vfio/vfio-common.h"
 #include "hw/s390x/s390-ccw.h"
+#include "hw/s390x/s390-ccwchain.h"
 
 typedef struct VFIOCCWDevice {
     S390CCWDevice cdev;
     VFIODevice vdev;
 } VFIOCCWDevice;
 
+static int vfio_ccw_handle_request(ORB *orb, IRB *irb, SCSW *scsw, void *data)
+{
+    struct vfio_ccw_cmd cmd_data;
+    int ret;
+    TransChainData trans_data;
+    S390CCWDevice *cdev = data;
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+
+    QEMU_BUILD_BUG_ON(sizeof(cmd_data.orb_area) != sizeof(ORB));
+    QEMU_BUILD_BUG_ON(sizeof(cmd_data.scsw_area) != sizeof(SCSW));
+    QEMU_BUILD_BUG_ON(sizeof(cmd_data.irb_area) != sizeof(IRB));
+
+    trans_data.cpa_gpa = orb->cpa;
+    ret = ccwchain_translate_to_userspace(&trans_data);
+    if (ret) {
+        return ret;
+    }
+    cmd_data.argsz = sizeof(cmd_data);
+    cmd_data.cssid = cdev->hcssid;
+    cmd_data.ssid = cdev->hssid;
+    cmd_data.devno = cdev->hdevno;
+    cmd_data.ccwchain_nr = trans_data.ccwchain_nr;
+    cmd_data.ccwchain_buf = trans_data.ccwchain_buf;
+
+    memcpy(cmd_data.orb_area, orb, sizeof(ORB));
+    memcpy(cmd_data.scsw_area, scsw, sizeof(SCSW));
+
+    if (ioctl(vcdev->vdev.fd, VFIO_DEVICE_CCW_CMD_REQUEST, &cmd_data) == -1) {
+        ccwchain_list_free(&trans_data);
+        return -errno;
+    }
+
+    memcpy(irb, cmd_data.irb_area, sizeof(IRB));
+    ccwchain_update_scsw(&(irb->scsw), &trans_data);
+    ccwchain_list_free(&trans_data);
+
+    return 0;
+}
+
 static void vfio_ccw_compute_needs_reset(VFIODevice *vdev)
 {
     vdev->needs_reset = true;
@@ -75,6 +115,8 @@ static void vfio_ccw_realize(DeviceState *dev, Error **errp)
     GError *gerror;
 
     /* Call the class init function for ccw device. */
+    cdev->handle_request = vfio_ccw_handle_request;
+
     if (cdc->realize) {
         cdc->realize(cdev, errp);
         if (*errp) {
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [PATCH RFC 9/9] s390x/css: ccws translation infrastructure
  2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 12:13   ` Xiao Feng Ren
  -1 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Implement a basic infrastructure of handling channel I/O instruction
interception for passed through devices:
1. Branch the code path of instruction interception handling by
   SubChannel type.
2. For a passed through device, issue the ORB to do ccw translation
   and perform an I/O operation.
3. Inject an I/O interrupt to notify guest the I/O result.
4. Assign different condition code based on the I/O result, or
   trigger a program check.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/css.c        | 130 ++++++++++++++++++++++++++++++++++++++++++++------
 hw/s390x/css.h        |   8 ++++
 hw/s390x/s390-ccw.c   |  12 +++++
 hw/s390x/virtio-ccw.c |   1 +
 target-s390x/ioinst.c |   9 ++++
 5 files changed, 145 insertions(+), 15 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 5d02ad3..a46e63b 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -463,7 +463,7 @@ static int css_interpret_ccw(SubchDev *sch, hwaddr ccw_addr,
     return ret;
 }
 
-static void sch_handle_start_func(SubchDev *sch, ORB *orb)
+static void sch_handle_start_func_virtual(SubchDev *sch, ORB *orb)
 {
 
     PMCW *p = &sch->curr_status.pmcw;
@@ -558,13 +558,79 @@ static void sch_handle_start_func(SubchDev *sch, ORB *orb)
 
 }
 
+static void copy_scsw_to_guest(SCSW *dest, const SCSW *src)
+{
+    dest->flags = cpu_to_be16(src->flags);
+    dest->ctrl = cpu_to_be16(src->ctrl);
+    dest->cpa = cpu_to_be32(src->cpa);
+    dest->dstat = src->dstat;
+    dest->cstat = src->cstat;
+    dest->count = cpu_to_be16(src->count);
+}
+
+static int sch_handle_start_func_passthrough(SubchDev *sch, ORB *orb)
+{
+
+    PMCW *p = &sch->curr_status.pmcw;
+    SCSW *s = &sch->curr_status.scsw;
+    int ret;
+
+    if (!(s->ctrl & SCSW_ACTL_SUSP)) {
+        assert(orb != NULL);
+        p->intparm = orb->intparm;
+    }
+
+    /* Prepare the irb for the guest. */
+    memset(&sch->irb, 0, sizeof(IRB));
+
+    /*
+     * TODO:
+     * Only support prefetch enable mode.
+     * Only support 64bit addressing idal.
+     */
+    if (!(orb->ctrl0 & ORB_CTRL0_MASK_PFCH) ||
+        !(orb->ctrl0 & ORB_CTRL0_MASK_C64)) {
+        return -EINVAL;
+    }
+
+    ret = s390_ccw_cmd_request(orb, &sch->irb, s, sch->driver_data);
+    switch (ret) {
+    case 0:
+        /* Update control block via irb. */
+        copy_scsw_to_guest(s, &sch->irb.scsw);
+        break;
+    case -EBUSY:
+        break;
+    case -ENODEV:
+        break;
+    case -EACCES:
+        /* Let's reflect an inaccessible host device by cc 3. */
+        ret = -ENODEV;
+        break;
+    case -ENOMEM:
+       /* No memory, generate deferred cc 1. */
+       s->flags &= ~SCSW_FLAGS_MASK_CC;
+       s->flags |= (1 << 8);
+       s->ctrl &= ~SCSW_CTRL_MASK_STCTL;
+       s->ctrl |= SCSW_STCTL_ALERT | SCSW_STCTL_STATUS_PEND;
+       break;
+    default:
+       /* All other return codes will trigger a program check,
+        * or set cc to 1.
+        */
+       break;
+    };
+
+    return ret;
+}
+
 /*
  * On real machines, this would run asynchronously to the main vcpus.
  * We might want to make some parts of the ssch handling (interpreting
  * read/writes) asynchronous later on if we start supporting more than
  * our current very simple devices.
  */
-static void do_subchannel_work(SubchDev *sch, ORB *orb)
+static void do_subchannel_work_virtual(SubchDev *sch, ORB *orb)
 {
 
     SCSW *s = &sch->curr_status.scsw;
@@ -574,7 +640,7 @@ static void do_subchannel_work(SubchDev *sch, ORB *orb)
     } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
         sch_handle_halt_func(sch);
     } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
-        sch_handle_start_func(sch, orb);
+        sch_handle_start_func_virtual(sch, orb);
     } else {
         /* Cannot happen. */
         return;
@@ -582,6 +648,51 @@ static void do_subchannel_work(SubchDev *sch, ORB *orb)
     css_inject_io_interrupt(sch);
 }
 
+static int do_subchannel_work_passthrough(SubchDev *sch, ORB *orb)
+{
+    int ret;
+    SCSW *s = &sch->curr_status.scsw;
+
+    if (s->ctrl & SCSW_FCTL_CLEAR_FUNC) {
+        /* TODO: Clear handling */
+        sch_handle_clear_func(sch);
+        ret = 0;
+    } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
+        /* TODO: Halt handling */
+        sch_handle_halt_func(sch);
+        ret = 0;
+    } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
+        ret = sch_handle_start_func_passthrough(sch, orb);
+    } else {
+        /* Cannot happen. */
+        return -ENODEV;
+    }
+
+    css_inject_io_interrupt(sch);
+
+    return ret;
+}
+
+static int do_subchannel_work(SubchDev *sch, ORB *orb)
+{
+    int ret;
+
+    switch (sch->type) {
+    case SUBCH_TYPE_VIRTUAL:
+        do_subchannel_work_virtual(sch, orb);
+        ret = 0;
+        break;
+    case SUBCH_TYPE_PASSTHROUGH:
+        ret = do_subchannel_work_passthrough(sch, orb);
+        break;
+    default:
+        assert(false);
+        ret = -EINVAL;
+    };
+
+    return ret;
+}
+
 static void copy_pmcw_to_guest(PMCW *dest, const PMCW *src)
 {
     int i;
@@ -602,16 +713,6 @@ static void copy_pmcw_to_guest(PMCW *dest, const PMCW *src)
     dest->chars = cpu_to_be32(src->chars);
 }
 
-static void copy_scsw_to_guest(SCSW *dest, const SCSW *src)
-{
-    dest->flags = cpu_to_be16(src->flags);
-    dest->ctrl = cpu_to_be16(src->ctrl);
-    dest->cpa = cpu_to_be32(src->cpa);
-    dest->dstat = src->dstat;
-    dest->cstat = src->cstat;
-    dest->count = cpu_to_be16(src->count);
-}
-
 static void copy_schib_to_guest(SCHIB *dest, const SCHIB *src)
 {
     int i;
@@ -898,8 +999,7 @@ int css_do_ssch(SubchDev *sch, ORB *orb)
     s->ctrl |= (SCSW_FCTL_START_FUNC | SCSW_ACTL_START_PEND);
     s->flags &= ~SCSW_FLAGS_MASK_PNO;
 
-    do_subchannel_work(sch, orb);
-    ret = 0;
+    ret = do_subchannel_work(sch, orb);
 
 out:
     return ret;
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index c280226..5f16edc 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -76,6 +76,8 @@ struct SubchDev {
     uint8_t ssid;
     uint16_t schid;
     uint16_t devno;
+    uint8_t type;
+    IRB irb;
     SCHIB curr_status;
     uint8_t sense_data[32];
     hwaddr channel_prog;
@@ -99,6 +101,11 @@ typedef struct IndAddr {
     QTAILQ_ENTRY(IndAddr) sibling;
 } IndAddr;
 
+enum {
+    SUBCH_TYPE_VIRTUAL      = 0,
+    SUBCH_TYPE_PASSTHROUGH  = 1,
+};
+
 IndAddr *get_indicator(hwaddr ind_addr, int len);
 void release_indicator(AdapterInfo *adapter, IndAddr *indicator);
 int map_indicator(AdapterInfo *adapter, IndAddr *indicator);
@@ -123,6 +130,7 @@ void css_generate_css_crws(uint8_t cssid);
 void css_clear_sei_pending(void);
 void css_adapter_interrupt(uint8_t isc);
 int css_sch_build_schib(SubchDev *sch, const char *hschid);
+int s390_ccw_cmd_request(ORB *orb, IRB *irb, SCSW *scsw, void *data);
 
 #define CSS_IO_ADAPTER_VIRTIO 1
 int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
diff --git a/hw/s390x/s390-ccw.c b/hw/s390x/s390-ccw.c
index 7b378f9..ed6f242 100644
--- a/hw/s390x/s390-ccw.c
+++ b/hw/s390x/s390-ccw.c
@@ -18,6 +18,17 @@
 #include "s390-ccw-bus.h"
 #include "s390-ccw.h"
 
+int s390_ccw_cmd_request(ORB *orb, IRB *irb, SCSW *scsw, void *data)
+{
+    S390CCWDevice *cdev = data;
+
+    if (cdev->handle_request) {
+        return cdev->handle_request(orb, irb, scsw, data);
+    } else {
+        return -ENOSYS;
+    }
+}
+
 static void s390_ccw_realize(S390CCWDevice *cdev, Error **errp)
 {
     unsigned int cssid, ssid, devno, schid;
@@ -89,6 +100,7 @@ static void s390_ccw_realize(S390CCWDevice *cdev, Error **errp)
     sch->cssid = cssid;
     sch->ssid = ssid;
     sch->devno = devno;
+    sch->type = SUBCH_TYPE_PASSTHROUGH;
 
     cdev->sch = NULL;
 
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index d51642d..621befb 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -758,6 +758,7 @@ static void virtio_ccw_device_realize(VirtioCcwDevice *dev, Error **errp)
     sch->channel_prog = 0x0;
     sch->last_cmd_valid = false;
     sch->thinint_active = false;
+    sch->type = SUBCH_TYPE_VIRTUAL;
     /*
      * Use a device number if provided. Otherwise, fall back to subchannel
      * number.
diff --git a/target-s390x/ioinst.c b/target-s390x/ioinst.c
index 142ff93..59f15af 100644
--- a/target-s390x/ioinst.c
+++ b/target-s390x/ioinst.c
@@ -244,6 +244,15 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb)
     case -EBUSY:
         cc = 2;
         break;
+    case -EFAULT:
+        /*
+         * TODO:
+         * I'm wondering whether there is something better
+         * to do for us here (like setting some device or
+         * subchannel status).
+         */
+        program_interrupt(env, PGM_ADDRESSING, 4);
+        return;
     case 0:
         cc = 0;
         break;
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* [Qemu-devel] [PATCH RFC 9/9] s390x/css: ccws translation infrastructure
@ 2016-04-29 12:13   ` Xiao Feng Ren
  0 siblings, 0 replies; 22+ messages in thread
From: Xiao Feng Ren @ 2016-04-29 12:13 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: bjsdjshi, renxiaof, cornelia.huck, borntraeger, agraf, alex.williamson

Implement a basic infrastructure of handling channel I/O instruction
interception for passed through devices:
1. Branch the code path of instruction interception handling by
   SubChannel type.
2. For a passed through device, issue the ORB to do ccw translation
   and perform an I/O operation.
3. Inject an I/O interrupt to notify guest the I/O result.
4. Assign different condition code based on the I/O result, or
   trigger a program check.

Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
---
 hw/s390x/css.c        | 130 ++++++++++++++++++++++++++++++++++++++++++++------
 hw/s390x/css.h        |   8 ++++
 hw/s390x/s390-ccw.c   |  12 +++++
 hw/s390x/virtio-ccw.c |   1 +
 target-s390x/ioinst.c |   9 ++++
 5 files changed, 145 insertions(+), 15 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 5d02ad3..a46e63b 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -463,7 +463,7 @@ static int css_interpret_ccw(SubchDev *sch, hwaddr ccw_addr,
     return ret;
 }
 
-static void sch_handle_start_func(SubchDev *sch, ORB *orb)
+static void sch_handle_start_func_virtual(SubchDev *sch, ORB *orb)
 {
 
     PMCW *p = &sch->curr_status.pmcw;
@@ -558,13 +558,79 @@ static void sch_handle_start_func(SubchDev *sch, ORB *orb)
 
 }
 
+static void copy_scsw_to_guest(SCSW *dest, const SCSW *src)
+{
+    dest->flags = cpu_to_be16(src->flags);
+    dest->ctrl = cpu_to_be16(src->ctrl);
+    dest->cpa = cpu_to_be32(src->cpa);
+    dest->dstat = src->dstat;
+    dest->cstat = src->cstat;
+    dest->count = cpu_to_be16(src->count);
+}
+
+static int sch_handle_start_func_passthrough(SubchDev *sch, ORB *orb)
+{
+
+    PMCW *p = &sch->curr_status.pmcw;
+    SCSW *s = &sch->curr_status.scsw;
+    int ret;
+
+    if (!(s->ctrl & SCSW_ACTL_SUSP)) {
+        assert(orb != NULL);
+        p->intparm = orb->intparm;
+    }
+
+    /* Prepare the irb for the guest. */
+    memset(&sch->irb, 0, sizeof(IRB));
+
+    /*
+     * TODO:
+     * Only support prefetch enable mode.
+     * Only support 64bit addressing idal.
+     */
+    if (!(orb->ctrl0 & ORB_CTRL0_MASK_PFCH) ||
+        !(orb->ctrl0 & ORB_CTRL0_MASK_C64)) {
+        return -EINVAL;
+    }
+
+    ret = s390_ccw_cmd_request(orb, &sch->irb, s, sch->driver_data);
+    switch (ret) {
+    case 0:
+        /* Update control block via irb. */
+        copy_scsw_to_guest(s, &sch->irb.scsw);
+        break;
+    case -EBUSY:
+        break;
+    case -ENODEV:
+        break;
+    case -EACCES:
+        /* Let's reflect an inaccessible host device by cc 3. */
+        ret = -ENODEV;
+        break;
+    case -ENOMEM:
+       /* No memory, generate deferred cc 1. */
+       s->flags &= ~SCSW_FLAGS_MASK_CC;
+       s->flags |= (1 << 8);
+       s->ctrl &= ~SCSW_CTRL_MASK_STCTL;
+       s->ctrl |= SCSW_STCTL_ALERT | SCSW_STCTL_STATUS_PEND;
+       break;
+    default:
+       /* All other return codes will trigger a program check,
+        * or set cc to 1.
+        */
+       break;
+    };
+
+    return ret;
+}
+
 /*
  * On real machines, this would run asynchronously to the main vcpus.
  * We might want to make some parts of the ssch handling (interpreting
  * read/writes) asynchronous later on if we start supporting more than
  * our current very simple devices.
  */
-static void do_subchannel_work(SubchDev *sch, ORB *orb)
+static void do_subchannel_work_virtual(SubchDev *sch, ORB *orb)
 {
 
     SCSW *s = &sch->curr_status.scsw;
@@ -574,7 +640,7 @@ static void do_subchannel_work(SubchDev *sch, ORB *orb)
     } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
         sch_handle_halt_func(sch);
     } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
-        sch_handle_start_func(sch, orb);
+        sch_handle_start_func_virtual(sch, orb);
     } else {
         /* Cannot happen. */
         return;
@@ -582,6 +648,51 @@ static void do_subchannel_work(SubchDev *sch, ORB *orb)
     css_inject_io_interrupt(sch);
 }
 
+static int do_subchannel_work_passthrough(SubchDev *sch, ORB *orb)
+{
+    int ret;
+    SCSW *s = &sch->curr_status.scsw;
+
+    if (s->ctrl & SCSW_FCTL_CLEAR_FUNC) {
+        /* TODO: Clear handling */
+        sch_handle_clear_func(sch);
+        ret = 0;
+    } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
+        /* TODO: Halt handling */
+        sch_handle_halt_func(sch);
+        ret = 0;
+    } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
+        ret = sch_handle_start_func_passthrough(sch, orb);
+    } else {
+        /* Cannot happen. */
+        return -ENODEV;
+    }
+
+    css_inject_io_interrupt(sch);
+
+    return ret;
+}
+
+static int do_subchannel_work(SubchDev *sch, ORB *orb)
+{
+    int ret;
+
+    switch (sch->type) {
+    case SUBCH_TYPE_VIRTUAL:
+        do_subchannel_work_virtual(sch, orb);
+        ret = 0;
+        break;
+    case SUBCH_TYPE_PASSTHROUGH:
+        ret = do_subchannel_work_passthrough(sch, orb);
+        break;
+    default:
+        assert(false);
+        ret = -EINVAL;
+    };
+
+    return ret;
+}
+
 static void copy_pmcw_to_guest(PMCW *dest, const PMCW *src)
 {
     int i;
@@ -602,16 +713,6 @@ static void copy_pmcw_to_guest(PMCW *dest, const PMCW *src)
     dest->chars = cpu_to_be32(src->chars);
 }
 
-static void copy_scsw_to_guest(SCSW *dest, const SCSW *src)
-{
-    dest->flags = cpu_to_be16(src->flags);
-    dest->ctrl = cpu_to_be16(src->ctrl);
-    dest->cpa = cpu_to_be32(src->cpa);
-    dest->dstat = src->dstat;
-    dest->cstat = src->cstat;
-    dest->count = cpu_to_be16(src->count);
-}
-
 static void copy_schib_to_guest(SCHIB *dest, const SCHIB *src)
 {
     int i;
@@ -898,8 +999,7 @@ int css_do_ssch(SubchDev *sch, ORB *orb)
     s->ctrl |= (SCSW_FCTL_START_FUNC | SCSW_ACTL_START_PEND);
     s->flags &= ~SCSW_FLAGS_MASK_PNO;
 
-    do_subchannel_work(sch, orb);
-    ret = 0;
+    ret = do_subchannel_work(sch, orb);
 
 out:
     return ret;
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index c280226..5f16edc 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -76,6 +76,8 @@ struct SubchDev {
     uint8_t ssid;
     uint16_t schid;
     uint16_t devno;
+    uint8_t type;
+    IRB irb;
     SCHIB curr_status;
     uint8_t sense_data[32];
     hwaddr channel_prog;
@@ -99,6 +101,11 @@ typedef struct IndAddr {
     QTAILQ_ENTRY(IndAddr) sibling;
 } IndAddr;
 
+enum {
+    SUBCH_TYPE_VIRTUAL      = 0,
+    SUBCH_TYPE_PASSTHROUGH  = 1,
+};
+
 IndAddr *get_indicator(hwaddr ind_addr, int len);
 void release_indicator(AdapterInfo *adapter, IndAddr *indicator);
 int map_indicator(AdapterInfo *adapter, IndAddr *indicator);
@@ -123,6 +130,7 @@ void css_generate_css_crws(uint8_t cssid);
 void css_clear_sei_pending(void);
 void css_adapter_interrupt(uint8_t isc);
 int css_sch_build_schib(SubchDev *sch, const char *hschid);
+int s390_ccw_cmd_request(ORB *orb, IRB *irb, SCSW *scsw, void *data);
 
 #define CSS_IO_ADAPTER_VIRTIO 1
 int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
diff --git a/hw/s390x/s390-ccw.c b/hw/s390x/s390-ccw.c
index 7b378f9..ed6f242 100644
--- a/hw/s390x/s390-ccw.c
+++ b/hw/s390x/s390-ccw.c
@@ -18,6 +18,17 @@
 #include "s390-ccw-bus.h"
 #include "s390-ccw.h"
 
+int s390_ccw_cmd_request(ORB *orb, IRB *irb, SCSW *scsw, void *data)
+{
+    S390CCWDevice *cdev = data;
+
+    if (cdev->handle_request) {
+        return cdev->handle_request(orb, irb, scsw, data);
+    } else {
+        return -ENOSYS;
+    }
+}
+
 static void s390_ccw_realize(S390CCWDevice *cdev, Error **errp)
 {
     unsigned int cssid, ssid, devno, schid;
@@ -89,6 +100,7 @@ static void s390_ccw_realize(S390CCWDevice *cdev, Error **errp)
     sch->cssid = cssid;
     sch->ssid = ssid;
     sch->devno = devno;
+    sch->type = SUBCH_TYPE_PASSTHROUGH;
 
     cdev->sch = NULL;
 
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index d51642d..621befb 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -758,6 +758,7 @@ static void virtio_ccw_device_realize(VirtioCcwDevice *dev, Error **errp)
     sch->channel_prog = 0x0;
     sch->last_cmd_valid = false;
     sch->thinint_active = false;
+    sch->type = SUBCH_TYPE_VIRTUAL;
     /*
      * Use a device number if provided. Otherwise, fall back to subchannel
      * number.
diff --git a/target-s390x/ioinst.c b/target-s390x/ioinst.c
index 142ff93..59f15af 100644
--- a/target-s390x/ioinst.c
+++ b/target-s390x/ioinst.c
@@ -244,6 +244,15 @@ void ioinst_handle_ssch(S390CPU *cpu, uint64_t reg1, uint32_t ipb)
     case -EBUSY:
         cc = 2;
         break;
+    case -EFAULT:
+        /*
+         * TODO:
+         * I'm wondering whether there is something better
+         * to do for us here (like setting some device or
+         * subchannel status).
+         */
+        program_interrupt(env, PGM_ADDRESSING, 4);
+        return;
     case 0:
         cc = 0;
         break;
-- 
2.6.6

^ permalink raw reply related	[flat|nested] 22+ messages in thread

* Re: [PATCH RFC 2/9] vfio: No-IOMMU mode support
  2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
@ 2016-04-29 16:21     ` Alex Williamson
  -1 siblings, 0 replies; 22+ messages in thread
From: Alex Williamson @ 2016-04-29 16:21 UTC (permalink / raw)
  To: Xiao Feng Ren
  Cc: kvm, linux-s390, qemu-devel, bjsdjshi, cornelia.huck, borntraeger, agraf

On Fri, 29 Apr 2016 14:13:16 +0200
Xiao Feng Ren <renxiaof@linux.vnet.ibm.com> wrote:

> Add qemu support for the newly introduced VFIO No-IOMMU driver.
> 
> We need to add special handling for:
> - Group character device is /dev/vfio/noiommu-$GROUP.
> - No-IOMMU does not rely on a memory listener.
> - No IOMMU will be set for its group, so no need to call
>   vfio_kvm_device_add_group.
> 
> Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
> ---
>  hw/vfio/common.c              | 66 ++++++++++++++++++++++++++++++++++---------
>  include/hw/vfio/vfio-common.h |  2 ++
>  2 files changed, 55 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index f27db36..656c303 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -789,6 +789,33 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
>      container = g_malloc0(sizeof(*container));
>      container->space = space;
>      container->fd = fd;
> +    container->noiommu = group->noiommu;
> +
> +    if (container->noiommu) {
> +        ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd);
> +        if (ret) {
> +            error_report("vfio: failed to set group container: %m");
> +            ret = -errno;
> +            goto free_container_exit;
> +        }
> +
> +        ret = ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_NOIOMMU_IOMMU);
> +        if (!ret) {
> +            error_report("vfio: No available IOMMU models");
> +            ret = -EINVAL;
> +            goto free_container_exit;
> +        }
> +
> +        ret = ioctl(fd, VFIO_SET_IOMMU, VFIO_NOIOMMU_IOMMU);
> +        if (ret) {
> +            error_report("vfio: failed to set iommu for container: %m");
> +            ret = -errno;
> +            goto free_container_exit;
> +        }
> +
> +        goto listener_register;
> +    }
> +
>      if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) ||
>          ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) {
>          bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU);
> @@ -878,14 +905,16 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
>          goto free_container_exit;
>      }
>  
> -    container->listener = vfio_memory_listener;
> -
> -    memory_listener_register(&container->listener, container->space->as);
> -
> -    if (container->error) {
> -        ret = container->error;
> -        error_report("vfio: memory listener initialization failed for container");
> -        goto listener_release_exit;
> +listener_register:
> +    if (!container->noiommu) {
> +        container->listener = vfio_memory_listener;
> +        memory_listener_register(&container->listener, container->space->as);
> +        if (container->error) {
> +            ret = container->error;
> +            error_report("vfio: memory listener initialization failed for "
> +                         "container");
> +            goto listener_release_exit;
> +        }
>      }
>  
>      container->initialized = true;
> @@ -898,7 +927,9 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
>  
>      return 0;
>  listener_release_exit:
> -    vfio_listener_release(container);
> +    if (!container->noiommu) {
> +        vfio_listener_release(container);
> +    }
>  
>  free_container_exit:
>      g_free(container);
> @@ -928,7 +959,9 @@ static void vfio_disconnect_container(VFIOGroup *group)
>          VFIOAddressSpace *space = container->space;
>          VFIOGuestIOMMU *giommu, *tmp;
>  
> -        vfio_listener_release(container);
> +        if (!container->noiommu) {
> +            vfio_listener_release(container);
> +        }
>          QLIST_REMOVE(container, next);
>  
>          QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) {
> @@ -969,8 +1002,13 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
>      snprintf(path, sizeof(path), "/dev/vfio/%d", groupid);
>      group->fd = qemu_open(path, O_RDWR);
>      if (group->fd < 0) {
> -        error_report("vfio: error opening %s: %m", path);
> -        goto free_group_exit;
> +        snprintf(path, sizeof(path), "/dev/vfio/noiommu-%d", groupid);
> +        group->fd = qemu_open(path, O_RDWR);
> +        if (group->fd < 0) {
> +            error_report("vfio: error opening %s: %m", path);
> +            goto free_group_exit;
> +        }
> +        group->noiommu = 1;

No, this just can't happen.  There is absolutely no way that falling
back to a noiommu interface is the correct thing to do in most
situations. It cannot be automatic or I will have vfio-pci users lined
up trying to do PCI device assignment with this code.

>      }
>  
>      if (ioctl(group->fd, VFIO_GROUP_GET_STATUS, &status)) {
> @@ -999,7 +1037,9 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
>  
>      QLIST_INSERT_HEAD(&vfio_group_list, group, next);
>  
> -    vfio_kvm_device_add_group(group);
> +    if (!group->noiommu) {
> +        vfio_kvm_device_add_group(group);
> +    }

Why?  Notifying KVM of an assigned device doesn't necessarily imply
some connection to the IOMMU.

>  
>      return group;
>  
> diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
> index eb0e1b0..85c2a74 100644
> --- a/include/hw/vfio/vfio-common.h
> +++ b/include/hw/vfio/vfio-common.h
> @@ -72,6 +72,7 @@ struct VFIOGroup;
>  typedef struct VFIOContainer {
>      VFIOAddressSpace *space;
>      int fd; /* /dev/vfio/vfio, empowered by the attached groups */
> +    bool noiommu;
>      MemoryListener listener;
>      int error;
>      bool initialized;
> @@ -121,6 +122,7 @@ struct VFIODeviceOps {
>  typedef struct VFIOGroup {
>      int fd;
>      int groupid;
> +    bool noiommu;
>      VFIOContainer *container;
>      QLIST_HEAD(, VFIODevice) device_list;
>      QLIST_ENTRY(VFIOGroup) next;

^ permalink raw reply	[flat|nested] 22+ messages in thread

* Re: [Qemu-devel] [PATCH RFC 2/9] vfio: No-IOMMU mode support
@ 2016-04-29 16:21     ` Alex Williamson
  0 siblings, 0 replies; 22+ messages in thread
From: Alex Williamson @ 2016-04-29 16:21 UTC (permalink / raw)
  To: Xiao Feng Ren
  Cc: kvm, linux-s390, qemu-devel, bjsdjshi, cornelia.huck, borntraeger, agraf

On Fri, 29 Apr 2016 14:13:16 +0200
Xiao Feng Ren <renxiaof@linux.vnet.ibm.com> wrote:

> Add qemu support for the newly introduced VFIO No-IOMMU driver.
> 
> We need to add special handling for:
> - Group character device is /dev/vfio/noiommu-$GROUP.
> - No-IOMMU does not rely on a memory listener.
> - No IOMMU will be set for its group, so no need to call
>   vfio_kvm_device_add_group.
> 
> Signed-off-by: Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
> ---
>  hw/vfio/common.c              | 66 ++++++++++++++++++++++++++++++++++---------
>  include/hw/vfio/vfio-common.h |  2 ++
>  2 files changed, 55 insertions(+), 13 deletions(-)
> 
> diff --git a/hw/vfio/common.c b/hw/vfio/common.c
> index f27db36..656c303 100644
> --- a/hw/vfio/common.c
> +++ b/hw/vfio/common.c
> @@ -789,6 +789,33 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
>      container = g_malloc0(sizeof(*container));
>      container->space = space;
>      container->fd = fd;
> +    container->noiommu = group->noiommu;
> +
> +    if (container->noiommu) {
> +        ret = ioctl(group->fd, VFIO_GROUP_SET_CONTAINER, &fd);
> +        if (ret) {
> +            error_report("vfio: failed to set group container: %m");
> +            ret = -errno;
> +            goto free_container_exit;
> +        }
> +
> +        ret = ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_NOIOMMU_IOMMU);
> +        if (!ret) {
> +            error_report("vfio: No available IOMMU models");
> +            ret = -EINVAL;
> +            goto free_container_exit;
> +        }
> +
> +        ret = ioctl(fd, VFIO_SET_IOMMU, VFIO_NOIOMMU_IOMMU);
> +        if (ret) {
> +            error_report("vfio: failed to set iommu for container: %m");
> +            ret = -errno;
> +            goto free_container_exit;
> +        }
> +
> +        goto listener_register;
> +    }
> +
>      if (ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1_IOMMU) ||
>          ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU)) {
>          bool v2 = !!ioctl(fd, VFIO_CHECK_EXTENSION, VFIO_TYPE1v2_IOMMU);
> @@ -878,14 +905,16 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
>          goto free_container_exit;
>      }
>  
> -    container->listener = vfio_memory_listener;
> -
> -    memory_listener_register(&container->listener, container->space->as);
> -
> -    if (container->error) {
> -        ret = container->error;
> -        error_report("vfio: memory listener initialization failed for container");
> -        goto listener_release_exit;
> +listener_register:
> +    if (!container->noiommu) {
> +        container->listener = vfio_memory_listener;
> +        memory_listener_register(&container->listener, container->space->as);
> +        if (container->error) {
> +            ret = container->error;
> +            error_report("vfio: memory listener initialization failed for "
> +                         "container");
> +            goto listener_release_exit;
> +        }
>      }
>  
>      container->initialized = true;
> @@ -898,7 +927,9 @@ static int vfio_connect_container(VFIOGroup *group, AddressSpace *as)
>  
>      return 0;
>  listener_release_exit:
> -    vfio_listener_release(container);
> +    if (!container->noiommu) {
> +        vfio_listener_release(container);
> +    }
>  
>  free_container_exit:
>      g_free(container);
> @@ -928,7 +959,9 @@ static void vfio_disconnect_container(VFIOGroup *group)
>          VFIOAddressSpace *space = container->space;
>          VFIOGuestIOMMU *giommu, *tmp;
>  
> -        vfio_listener_release(container);
> +        if (!container->noiommu) {
> +            vfio_listener_release(container);
> +        }
>          QLIST_REMOVE(container, next);
>  
>          QLIST_FOREACH_SAFE(giommu, &container->giommu_list, giommu_next, tmp) {
> @@ -969,8 +1002,13 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
>      snprintf(path, sizeof(path), "/dev/vfio/%d", groupid);
>      group->fd = qemu_open(path, O_RDWR);
>      if (group->fd < 0) {
> -        error_report("vfio: error opening %s: %m", path);
> -        goto free_group_exit;
> +        snprintf(path, sizeof(path), "/dev/vfio/noiommu-%d", groupid);
> +        group->fd = qemu_open(path, O_RDWR);
> +        if (group->fd < 0) {
> +            error_report("vfio: error opening %s: %m", path);
> +            goto free_group_exit;
> +        }
> +        group->noiommu = 1;

No, this just can't happen.  There is absolutely no way that falling
back to a noiommu interface is the correct thing to do in most
situations. It cannot be automatic or I will have vfio-pci users lined
up trying to do PCI device assignment with this code.

>      }
>  
>      if (ioctl(group->fd, VFIO_GROUP_GET_STATUS, &status)) {
> @@ -999,7 +1037,9 @@ VFIOGroup *vfio_get_group(int groupid, AddressSpace *as)
>  
>      QLIST_INSERT_HEAD(&vfio_group_list, group, next);
>  
> -    vfio_kvm_device_add_group(group);
> +    if (!group->noiommu) {
> +        vfio_kvm_device_add_group(group);
> +    }

Why?  Notifying KVM of an assigned device doesn't necessarily imply
some connection to the IOMMU.

>  
>      return group;
>  
> diff --git a/include/hw/vfio/vfio-common.h b/include/hw/vfio/vfio-common.h
> index eb0e1b0..85c2a74 100644
> --- a/include/hw/vfio/vfio-common.h
> +++ b/include/hw/vfio/vfio-common.h
> @@ -72,6 +72,7 @@ struct VFIOGroup;
>  typedef struct VFIOContainer {
>      VFIOAddressSpace *space;
>      int fd; /* /dev/vfio/vfio, empowered by the attached groups */
> +    bool noiommu;
>      MemoryListener listener;
>      int error;
>      bool initialized;
> @@ -121,6 +122,7 @@ struct VFIODeviceOps {
>  typedef struct VFIOGroup {
>      int fd;
>      int groupid;
> +    bool noiommu;
>      VFIOContainer *container;
>      QLIST_HEAD(, VFIODevice) device_list;
>      QLIST_ENTRY(VFIOGroup) next;

^ permalink raw reply	[flat|nested] 22+ messages in thread

end of thread, other threads:[~2016-04-29 16:22 UTC | newest]

Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-04-29 12:13 [PATCH RFC 0/9] basic channel IO passthrough infrastructure based on vfio Xiao Feng Ren
2016-04-29 12:13 ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 1/9] vfio: linux-headers update for vfio-ccw Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 2/9] vfio: No-IOMMU mode support Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 16:21   ` Alex Williamson
2016-04-29 16:21     ` [Qemu-devel] " Alex Williamson
2016-04-29 12:13 ` [PATCH RFC 3/9] s390x/css: introduce ccw chain interfaces Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 4/9] s390x/css: add s390-map-css machine option Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 5/9] s390x/css: realize css_sch_build_schib Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 6/9] s390x/css: device and bus support for s390-ccw passthrough Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 7/9] vfio/ccw: vfio based ccw passthrough driver Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 8/9] s390x/css: introduce and realize ccw-request callback Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren
2016-04-29 12:13 ` [PATCH RFC 9/9] s390x/css: ccws translation infrastructure Xiao Feng Ren
2016-04-29 12:13   ` [Qemu-devel] " Xiao Feng Ren

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.