[PATCH 0/2] vfio-ccw: support hsch/csch (QEMU part)

[PATCH 0/2] vfio-ccw: support hsch/csch (QEMU part)

[Cornelia Huck]

[This is the QEMU part, git tree is available at
https://github.com/cohuck/qemu vfio-ccw-caps

The companion Linux kernel patches are available at
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/vfio-ccw.git vfio-ccw-caps]

Currently, vfio-ccw only relays START SUBCHANNEL requests to the real
device. This tends to work well for the most common 'good path' scenarios;
however, as we emulate {HALT,CLEAR} SUBCHANNEL in QEMU, things like
clearing pending requests at the device is currently not supported.
This may be a problem for e.g. error recovery.

This patch series makes use of the newly introduced async command region
to issue hsch/csch; if it is not present, continue to emulate hsch/csch,
as before.

[I'm not quite happy with how this async processing hooks up in css.c;
ideas welcome.]

Very lightly tested (I can interact with a dasd as before; I have not
found a reliable way to trigger hsch/csch in the Linux dasd guest driver.)

Cornelia Huck (2):
  vfio-ccw: new capability chain support
  vfio-ccw: support async command subregion

 hw/s390x/css.c                 |  27 ++++++--
 hw/vfio/ccw.c                  | 109 ++++++++++++++++++++++++++++++++-
 include/hw/s390x/s390-ccw.h    |   3 +
 linux-headers/linux/vfio.h     |   4 ++
 linux-headers/linux/vfio_ccw.h |  12 ++++
 5 files changed, 149 insertions(+), 6 deletions(-)

-- 
2.17.2

[Qemu-devel] [PATCH 0/2] vfio-ccw: support hsch/csch (QEMU part)

[Cornelia Huck]

[This is the QEMU part, git tree is available at
https://github.com/cohuck/qemu vfio-ccw-caps

The companion Linux kernel patches are available at
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/vfio-ccw.git vfio-ccw-caps]

Currently, vfio-ccw only relays START SUBCHANNEL requests to the real
device. This tends to work well for the most common 'good path' scenarios;
however, as we emulate {HALT,CLEAR} SUBCHANNEL in QEMU, things like
clearing pending requests at the device is currently not supported.
This may be a problem for e.g. error recovery.

This patch series makes use of the newly introduced async command region
to issue hsch/csch; if it is not present, continue to emulate hsch/csch,
as before.

[I'm not quite happy with how this async processing hooks up in css.c;
ideas welcome.]

Very lightly tested (I can interact with a dasd as before; I have not
found a reliable way to trigger hsch/csch in the Linux dasd guest driver.)

Cornelia Huck (2):
  vfio-ccw: new capability chain support
  vfio-ccw: support async command subregion

 hw/s390x/css.c                 |  27 ++++++--
 hw/vfio/ccw.c                  | 109 ++++++++++++++++++++++++++++++++-
 include/hw/s390x/s390-ccw.h    |   3 +
 linux-headers/linux/vfio.h     |   4 ++
 linux-headers/linux/vfio_ccw.h |  12 ++++
 5 files changed, 149 insertions(+), 6 deletions(-)

-- 
2.17.2

[PATCH 1/2] vfio-ccw: new capability chain support

[Cornelia Huck]

To be replaced with a real linux-headers update.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
---
 linux-headers/linux/vfio.h     |  4 ++++
 linux-headers/linux/vfio_ccw.h | 12 ++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index ceb6453394..c9ba8d52a0 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -297,12 +297,16 @@ struct vfio_region_info_cap_type {
 
 #define VFIO_REGION_TYPE_PCI_VENDOR_TYPE	(1 << 31)
 #define VFIO_REGION_TYPE_PCI_VENDOR_MASK	(0xffff)
+#define VFIO_REGION_TYPE_CCW			(1 << 30)
 
 /* 8086 Vendor sub-types */
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION	(1)
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG	(2)
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG	(3)
 
+/* ccw sub-types */
+#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD	(1)
+
 /*
  * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
  * which allows direct access to non-MSIX registers which happened to be within
diff --git a/linux-headers/linux/vfio_ccw.h b/linux-headers/linux/vfio_ccw.h
index 5bf96c3812..fcc3e69ef5 100644
--- a/linux-headers/linux/vfio_ccw.h
+++ b/linux-headers/linux/vfio_ccw.h
@@ -12,6 +12,7 @@
 
 #include <linux/types.h>
 
+/* used for START SUBCHANNEL, always present */
 struct ccw_io_region {
 #define ORB_AREA_SIZE 12
 	__u8	orb_area[ORB_AREA_SIZE];
@@ -22,4 +23,15 @@ struct ccw_io_region {
 	__u32	ret_code;
 } __attribute__((packed));
 
+/*
+ * used for processing commands that trigger asynchronous actions
+ * Note: this is controlled by a capability
+ */
+#define VFIO_CCW_ASYNC_CMD_HSCH (1 << 0)
+#define VFIO_CCW_ASYNC_CMD_CSCH (1 << 1)
+struct ccw_cmd_region {
+	__u32 command;
+	__u32 ret_code;
+} __attribute__((packed));
+
 #endif
-- 
2.17.2

[Qemu-devel] [PATCH 1/2] vfio-ccw: new capability chain support

[Cornelia Huck]

To be replaced with a real linux-headers update.

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
---
 linux-headers/linux/vfio.h     |  4 ++++
 linux-headers/linux/vfio_ccw.h | 12 ++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index ceb6453394..c9ba8d52a0 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -297,12 +297,16 @@ struct vfio_region_info_cap_type {
 
 #define VFIO_REGION_TYPE_PCI_VENDOR_TYPE	(1 << 31)
 #define VFIO_REGION_TYPE_PCI_VENDOR_MASK	(0xffff)
+#define VFIO_REGION_TYPE_CCW			(1 << 30)
 
 /* 8086 Vendor sub-types */
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION	(1)
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG	(2)
 #define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG	(3)
 
+/* ccw sub-types */
+#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD	(1)
+
 /*
  * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
  * which allows direct access to non-MSIX registers which happened to be within
diff --git a/linux-headers/linux/vfio_ccw.h b/linux-headers/linux/vfio_ccw.h
index 5bf96c3812..fcc3e69ef5 100644
--- a/linux-headers/linux/vfio_ccw.h
+++ b/linux-headers/linux/vfio_ccw.h
@@ -12,6 +12,7 @@
 
 #include <linux/types.h>
 
+/* used for START SUBCHANNEL, always present */
 struct ccw_io_region {
 #define ORB_AREA_SIZE 12
 	__u8	orb_area[ORB_AREA_SIZE];
@@ -22,4 +23,15 @@ struct ccw_io_region {
 	__u32	ret_code;
 } __attribute__((packed));
 
+/*
+ * used for processing commands that trigger asynchronous actions
+ * Note: this is controlled by a capability
+ */
+#define VFIO_CCW_ASYNC_CMD_HSCH (1 << 0)
+#define VFIO_CCW_ASYNC_CMD_CSCH (1 << 1)
+struct ccw_cmd_region {
+	__u32 command;
+	__u32 ret_code;
+} __attribute__((packed));
+
 #endif
-- 
2.17.2

[PATCH 2/2] vfio-ccw: support async command subregion

[Cornelia Huck]

A vfio-ccw device may provide an async command subregion for
issuing halt/clear subchannel requests. If it is present, use
it for sending halt/clear request to the device; if not, fall
back to emulation (as done today).

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
---
 hw/s390x/css.c              |  27 +++++++--
 hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
 include/hw/s390x/s390-ccw.h |   3 +
 3 files changed, 133 insertions(+), 6 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 04ec5cc970..0897c041c5 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -22,6 +22,7 @@
 #include "trace.h"
 #include "hw/s390x/s390_flic.h"
 #include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/s390-ccw.h"
 
 typedef struct CrwContainer {
     CRW crw;
@@ -1194,6 +1195,26 @@ static void sch_handle_start_func_virtual(SubchDev *sch)
 
 }
 
+static void sch_handle_halt_func_passthrough(SubchDev *sch)
+{
+    int ret;
+
+    ret = vfio_ccw_handle_halt(sch);
+    if (ret == -ENOSYS) {
+        sch_handle_halt_func(sch);
+    }
+}
+
+static void sch_handle_clear_func_passthrough(SubchDev *sch)
+{
+    int ret;
+
+    ret = vfio_ccw_handle_clear(sch);
+    if (ret == -ENOSYS) {
+        sch_handle_clear_func(sch);
+    }
+}
+
 static IOInstEnding sch_handle_start_func_passthrough(SubchDev *sch)
 {
 
@@ -1237,11 +1258,9 @@ IOInstEnding do_subchannel_work_passthrough(SubchDev *sch)
     SCSW *s = &sch->curr_status.scsw;
 
     if (s->ctrl & SCSW_FCTL_CLEAR_FUNC) {
-        /* TODO: Clear handling */
-        sch_handle_clear_func(sch);
+        sch_handle_clear_func_passthrough(sch);
     } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
-        /* TODO: Halt handling */
-        sch_handle_halt_func(sch);
+        sch_handle_halt_func_passthrough(sch);
     } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
         return sch_handle_start_func_passthrough(sch);
     }
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
index 9246729a75..dd0fecf168 100644
--- a/hw/vfio/ccw.c
+++ b/hw/vfio/ccw.c
@@ -2,9 +2,12 @@
  * vfio based subchannel assignment support
  *
  * Copyright 2017 IBM Corp.
+ * Copyright 2018 Red Hat, Inc.
+ *
  * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
  *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
  *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *            Cornelia Huck <cohuck@redhat.com>
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or (at
  * your option) any later version. See the COPYING file in the top-level
@@ -32,6 +35,9 @@ typedef struct VFIOCCWDevice {
     uint64_t io_region_size;
     uint64_t io_region_offset;
     struct ccw_io_region *io_region;
+    uint64_t async_cmd_region_size;
+    uint64_t async_cmd_region_offset;
+    struct ccw_cmd_region *async_cmd_region;
     EventNotifier io_notifier;
     bool force_orb_pfch;
     bool warned_orb_pfch;
@@ -114,6 +120,87 @@ again:
     }
 }
 
+int vfio_ccw_handle_clear(SubchDev *sch)
+{
+    S390CCWDevice *cdev = sch->driver_data;
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    struct ccw_cmd_region *region = vcdev->async_cmd_region;
+    int ret;
+
+    if (!vcdev->async_cmd_region) {
+        /* Async command region not available, fall back to emulation */
+        return -ENOSYS;
+    }
+
+    memset(region, 0, sizeof(*region));
+    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
+
+again:
+    ret = pwrite(vcdev->vdev.fd, region,
+                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
+    if (ret != vcdev->async_cmd_region_size) {
+        if (errno == EAGAIN) {
+            goto again;
+        }
+        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
+        ret = -errno;
+    } else {
+        ret = region->ret_code;
+    }
+    switch (ret) {
+    case 0:
+    case -ENODEV:
+    case -EACCES:
+        return 0;
+    case -EFAULT:
+    default:
+        sch_gen_unit_exception(sch);
+        css_inject_io_interrupt(sch);
+        return 0;
+    }
+}
+
+int vfio_ccw_handle_halt(SubchDev *sch)
+{
+    S390CCWDevice *cdev = sch->driver_data;
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    struct ccw_cmd_region *region = vcdev->async_cmd_region;
+    int ret;
+
+    if (!vcdev->async_cmd_region) {
+        /* Async command region not available, fall back to emulation */
+        return -ENOSYS;
+    }
+
+    memset(region, 0, sizeof(*region));
+    region->command = VFIO_CCW_ASYNC_CMD_HSCH;
+
+again:
+    ret = pwrite(vcdev->vdev.fd, region,
+                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
+    if (ret != vcdev->async_cmd_region_size) {
+        if (errno == EAGAIN) {
+            goto again;
+        }
+        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
+        ret = -errno;
+    } else {
+        ret = region->ret_code;
+    }
+    switch (ret) {
+    case 0:
+    case -EBUSY:
+    case -ENODEV:
+    case -EACCES:
+        return 0;
+    case -EFAULT:
+    default:
+        sch_gen_unit_exception(sch);
+        css_inject_io_interrupt(sch);
+        return 0;
+    }
+}
+
 static void vfio_ccw_reset(DeviceState *dev)
 {
     CcwDevice *ccw_dev = DO_UPCAST(CcwDevice, parent_obj, dev);
@@ -285,9 +372,13 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp)
         return;
     }
 
+    /*
+     * We always expect at least the I/O region to be present. We also
+     * may have a variable number of regions governed by capabilities.
+     */
     if (vdev->num_regions < VFIO_CCW_CONFIG_REGION_INDEX + 1) {
-        error_setg(errp, "vfio: Unexpected number of the I/O region %u",
-                   vdev->num_regions);
+        error_setg(errp, "vfio: too few regions (%u), expected at least %u",
+                   vdev->num_regions, VFIO_CCW_CONFIG_REGION_INDEX + 1);
         return;
     }
 
@@ -307,6 +398,20 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp)
     vcdev->io_region_offset = info->offset;
     vcdev->io_region = g_malloc0(info->size);
 
+    /* check for the optional async command region */
+    ret = vfio_get_dev_region_info(vdev, VFIO_REGION_TYPE_CCW,
+                                   VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD, &info);
+    if (!ret) {
+        vcdev->async_cmd_region_size = info->size;
+        if (sizeof(*vcdev->async_cmd_region) != vcdev->async_cmd_region_size) {
+            error_setg(errp, "vfio: Unexpected size of the async cmd region");
+            g_free(info);
+            return;
+        }
+        vcdev->async_cmd_region_offset = info->offset;
+        vcdev->async_cmd_region = g_malloc0(info->size);
+    }
+
     g_free(info);
 }
 
diff --git a/include/hw/s390x/s390-ccw.h b/include/hw/s390x/s390-ccw.h
index 7d15a1a5d4..38d54a7a69 100644
--- a/include/hw/s390x/s390-ccw.h
+++ b/include/hw/s390x/s390-ccw.h
@@ -36,4 +36,7 @@ typedef struct S390CCWDeviceClass {
     IOInstEnding (*handle_request) (SubchDev *sch);
 } S390CCWDeviceClass;
 
+int vfio_ccw_handle_clear(SubchDev *sch);
+int vfio_ccw_handle_halt(SubchDev *sch);
+
 #endif
-- 
2.17.2

[Qemu-devel] [PATCH 2/2] vfio-ccw: support async command subregion

[Cornelia Huck]

A vfio-ccw device may provide an async command subregion for
issuing halt/clear subchannel requests. If it is present, use
it for sending halt/clear request to the device; if not, fall
back to emulation (as done today).

Signed-off-by: Cornelia Huck <cohuck@redhat.com>
---
 hw/s390x/css.c              |  27 +++++++--
 hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
 include/hw/s390x/s390-ccw.h |   3 +
 3 files changed, 133 insertions(+), 6 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 04ec5cc970..0897c041c5 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -22,6 +22,7 @@
 #include "trace.h"
 #include "hw/s390x/s390_flic.h"
 #include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/s390-ccw.h"
 
 typedef struct CrwContainer {
     CRW crw;
@@ -1194,6 +1195,26 @@ static void sch_handle_start_func_virtual(SubchDev *sch)
 
 }
 
+static void sch_handle_halt_func_passthrough(SubchDev *sch)
+{
+    int ret;
+
+    ret = vfio_ccw_handle_halt(sch);
+    if (ret == -ENOSYS) {
+        sch_handle_halt_func(sch);
+    }
+}
+
+static void sch_handle_clear_func_passthrough(SubchDev *sch)
+{
+    int ret;
+
+    ret = vfio_ccw_handle_clear(sch);
+    if (ret == -ENOSYS) {
+        sch_handle_clear_func(sch);
+    }
+}
+
 static IOInstEnding sch_handle_start_func_passthrough(SubchDev *sch)
 {
 
@@ -1237,11 +1258,9 @@ IOInstEnding do_subchannel_work_passthrough(SubchDev *sch)
     SCSW *s = &sch->curr_status.scsw;
 
     if (s->ctrl & SCSW_FCTL_CLEAR_FUNC) {
-        /* TODO: Clear handling */
-        sch_handle_clear_func(sch);
+        sch_handle_clear_func_passthrough(sch);
     } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
-        /* TODO: Halt handling */
-        sch_handle_halt_func(sch);
+        sch_handle_halt_func_passthrough(sch);
     } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
         return sch_handle_start_func_passthrough(sch);
     }
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
index 9246729a75..dd0fecf168 100644
--- a/hw/vfio/ccw.c
+++ b/hw/vfio/ccw.c
@@ -2,9 +2,12 @@
  * vfio based subchannel assignment support
  *
  * Copyright 2017 IBM Corp.
+ * Copyright 2018 Red Hat, Inc.
+ *
  * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
  *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
  *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *            Cornelia Huck <cohuck@redhat.com>
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or (at
  * your option) any later version. See the COPYING file in the top-level
@@ -32,6 +35,9 @@ typedef struct VFIOCCWDevice {
     uint64_t io_region_size;
     uint64_t io_region_offset;
     struct ccw_io_region *io_region;
+    uint64_t async_cmd_region_size;
+    uint64_t async_cmd_region_offset;
+    struct ccw_cmd_region *async_cmd_region;
     EventNotifier io_notifier;
     bool force_orb_pfch;
     bool warned_orb_pfch;
@@ -114,6 +120,87 @@ again:
     }
 }
 
+int vfio_ccw_handle_clear(SubchDev *sch)
+{
+    S390CCWDevice *cdev = sch->driver_data;
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    struct ccw_cmd_region *region = vcdev->async_cmd_region;
+    int ret;
+
+    if (!vcdev->async_cmd_region) {
+        /* Async command region not available, fall back to emulation */
+        return -ENOSYS;
+    }
+
+    memset(region, 0, sizeof(*region));
+    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
+
+again:
+    ret = pwrite(vcdev->vdev.fd, region,
+                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
+    if (ret != vcdev->async_cmd_region_size) {
+        if (errno == EAGAIN) {
+            goto again;
+        }
+        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
+        ret = -errno;
+    } else {
+        ret = region->ret_code;
+    }
+    switch (ret) {
+    case 0:
+    case -ENODEV:
+    case -EACCES:
+        return 0;
+    case -EFAULT:
+    default:
+        sch_gen_unit_exception(sch);
+        css_inject_io_interrupt(sch);
+        return 0;
+    }
+}
+
+int vfio_ccw_handle_halt(SubchDev *sch)
+{
+    S390CCWDevice *cdev = sch->driver_data;
+    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
+    struct ccw_cmd_region *region = vcdev->async_cmd_region;
+    int ret;
+
+    if (!vcdev->async_cmd_region) {
+        /* Async command region not available, fall back to emulation */
+        return -ENOSYS;
+    }
+
+    memset(region, 0, sizeof(*region));
+    region->command = VFIO_CCW_ASYNC_CMD_HSCH;
+
+again:
+    ret = pwrite(vcdev->vdev.fd, region,
+                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
+    if (ret != vcdev->async_cmd_region_size) {
+        if (errno == EAGAIN) {
+            goto again;
+        }
+        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
+        ret = -errno;
+    } else {
+        ret = region->ret_code;
+    }
+    switch (ret) {
+    case 0:
+    case -EBUSY:
+    case -ENODEV:
+    case -EACCES:
+        return 0;
+    case -EFAULT:
+    default:
+        sch_gen_unit_exception(sch);
+        css_inject_io_interrupt(sch);
+        return 0;
+    }
+}
+
 static void vfio_ccw_reset(DeviceState *dev)
 {
     CcwDevice *ccw_dev = DO_UPCAST(CcwDevice, parent_obj, dev);
@@ -285,9 +372,13 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp)
         return;
     }
 
+    /*
+     * We always expect at least the I/O region to be present. We also
+     * may have a variable number of regions governed by capabilities.
+     */
     if (vdev->num_regions < VFIO_CCW_CONFIG_REGION_INDEX + 1) {
-        error_setg(errp, "vfio: Unexpected number of the I/O region %u",
-                   vdev->num_regions);
+        error_setg(errp, "vfio: too few regions (%u), expected at least %u",
+                   vdev->num_regions, VFIO_CCW_CONFIG_REGION_INDEX + 1);
         return;
     }
 
@@ -307,6 +398,20 @@ static void vfio_ccw_get_region(VFIOCCWDevice *vcdev, Error **errp)
     vcdev->io_region_offset = info->offset;
     vcdev->io_region = g_malloc0(info->size);
 
+    /* check for the optional async command region */
+    ret = vfio_get_dev_region_info(vdev, VFIO_REGION_TYPE_CCW,
+                                   VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD, &info);
+    if (!ret) {
+        vcdev->async_cmd_region_size = info->size;
+        if (sizeof(*vcdev->async_cmd_region) != vcdev->async_cmd_region_size) {
+            error_setg(errp, "vfio: Unexpected size of the async cmd region");
+            g_free(info);
+            return;
+        }
+        vcdev->async_cmd_region_offset = info->offset;
+        vcdev->async_cmd_region = g_malloc0(info->size);
+    }
+
     g_free(info);
 }
 
diff --git a/include/hw/s390x/s390-ccw.h b/include/hw/s390x/s390-ccw.h
index 7d15a1a5d4..38d54a7a69 100644
--- a/include/hw/s390x/s390-ccw.h
+++ b/include/hw/s390x/s390-ccw.h
@@ -36,4 +36,7 @@ typedef struct S390CCWDeviceClass {
     IOInstEnding (*handle_request) (SubchDev *sch);
 } S390CCWDeviceClass;
 
+int vfio_ccw_handle_clear(SubchDev *sch);
+int vfio_ccw_handle_halt(SubchDev *sch);
+
 #endif
-- 
2.17.2

Re: [PATCH 1/2] vfio-ccw: new capability chain support

[Pierre Morel]

On 22/11/2018 17:54, Cornelia Huck wrote:
> To be replaced with a real linux-headers update.
> 
> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> ---
>   linux-headers/linux/vfio.h     |  4 ++++
>   linux-headers/linux/vfio_ccw.h | 12 ++++++++++++
>   2 files changed, 16 insertions(+)
> 
> diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
> index ceb6453394..c9ba8d52a0 100644
> --- a/linux-headers/linux/vfio.h
> +++ b/linux-headers/linux/vfio.h
> @@ -297,12 +297,16 @@ struct vfio_region_info_cap_type {
>   
>   #define VFIO_REGION_TYPE_PCI_VENDOR_TYPE	(1 << 31)
>   #define VFIO_REGION_TYPE_PCI_VENDOR_MASK	(0xffff)
> +#define VFIO_REGION_TYPE_CCW			(1 << 30)
>   
>   /* 8086 Vendor sub-types */
>   #define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION	(1)
>   #define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG	(2)
>   #define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG	(3)
>   
> +/* ccw sub-types */
> +#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD	(1)
> +
>   /*
>    * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
>    * which allows direct access to non-MSIX registers which happened to be within
> diff --git a/linux-headers/linux/vfio_ccw.h b/linux-headers/linux/vfio_ccw.h
> index 5bf96c3812..fcc3e69ef5 100644
> --- a/linux-headers/linux/vfio_ccw.h
> +++ b/linux-headers/linux/vfio_ccw.h
> @@ -12,6 +12,7 @@
>   
>   #include <linux/types.h>
>   
> +/* used for START SUBCHANNEL, always present */
>   struct ccw_io_region {
>   #define ORB_AREA_SIZE 12
>   	__u8	orb_area[ORB_AREA_SIZE];
> @@ -22,4 +23,15 @@ struct ccw_io_region {
>   	__u32	ret_code;
>   } __attribute__((packed));
>   
> +/*
> + * used for processing commands that trigger asynchronous actions
> + * Note: this is controlled by a capability
> + */
> +#define VFIO_CCW_ASYNC_CMD_HSCH (1 << 0)
> +#define VFIO_CCW_ASYNC_CMD_CSCH (1 << 1)
> +struct ccw_cmd_region {
> +	__u32 command;
> +	__u32 ret_code;
> +} __attribute__((packed));
> +
>   #endif
> 

LGTM

-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany

Re: [Qemu-devel] [PATCH 1/2] vfio-ccw: new capability chain support

[Pierre Morel]

On 22/11/2018 17:54, Cornelia Huck wrote:
> To be replaced with a real linux-headers update.
> 
> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> ---
>   linux-headers/linux/vfio.h     |  4 ++++
>   linux-headers/linux/vfio_ccw.h | 12 ++++++++++++
>   2 files changed, 16 insertions(+)
> 
> diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
> index ceb6453394..c9ba8d52a0 100644
> --- a/linux-headers/linux/vfio.h
> +++ b/linux-headers/linux/vfio.h
> @@ -297,12 +297,16 @@ struct vfio_region_info_cap_type {
>   
>   #define VFIO_REGION_TYPE_PCI_VENDOR_TYPE	(1 << 31)
>   #define VFIO_REGION_TYPE_PCI_VENDOR_MASK	(0xffff)
> +#define VFIO_REGION_TYPE_CCW			(1 << 30)
>   
>   /* 8086 Vendor sub-types */
>   #define VFIO_REGION_SUBTYPE_INTEL_IGD_OPREGION	(1)
>   #define VFIO_REGION_SUBTYPE_INTEL_IGD_HOST_CFG	(2)
>   #define VFIO_REGION_SUBTYPE_INTEL_IGD_LPC_CFG	(3)
>   
> +/* ccw sub-types */
> +#define VFIO_REGION_SUBTYPE_CCW_ASYNC_CMD	(1)
> +
>   /*
>    * The MSIX mappable capability informs that MSIX data of a BAR can be mmapped
>    * which allows direct access to non-MSIX registers which happened to be within
> diff --git a/linux-headers/linux/vfio_ccw.h b/linux-headers/linux/vfio_ccw.h
> index 5bf96c3812..fcc3e69ef5 100644
> --- a/linux-headers/linux/vfio_ccw.h
> +++ b/linux-headers/linux/vfio_ccw.h
> @@ -12,6 +12,7 @@
>   
>   #include <linux/types.h>
>   
> +/* used for START SUBCHANNEL, always present */
>   struct ccw_io_region {
>   #define ORB_AREA_SIZE 12
>   	__u8	orb_area[ORB_AREA_SIZE];
> @@ -22,4 +23,15 @@ struct ccw_io_region {
>   	__u32	ret_code;
>   } __attribute__((packed));
>   
> +/*
> + * used for processing commands that trigger asynchronous actions
> + * Note: this is controlled by a capability
> + */
> +#define VFIO_CCW_ASYNC_CMD_HSCH (1 << 0)
> +#define VFIO_CCW_ASYNC_CMD_CSCH (1 << 1)
> +struct ccw_cmd_region {
> +	__u32 command;
> +	__u32 ret_code;
> +} __attribute__((packed));
> +
>   #endif
> 

LGTM

-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany

Re: [PATCH 2/2] vfio-ccw: support async command subregion

[Pierre Morel]

On 22/11/2018 17:54, Cornelia Huck wrote:
> A vfio-ccw device may provide an async command subregion for
> issuing halt/clear subchannel requests. If it is present, use
> it for sending halt/clear request to the device; if not, fall
> back to emulation (as done today).
> 
> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> ---
>   hw/s390x/css.c              |  27 +++++++--
>   hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
>   include/hw/s390x/s390-ccw.h |   3 +
>   3 files changed, 133 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/s390x/css.c b/hw/s390x/css.c
> index 04ec5cc970..0897c041c5 100644
> --- a/hw/s390x/css.c
> +++ b/hw/s390x/css.c
> @@ -22,6 +22,7 @@
>   #include "trace.h"
>   #include "hw/s390x/s390_flic.h"
>   #include "hw/s390x/s390-virtio-ccw.h"
> +#include "hw/s390x/s390-ccw.h"
>   
>   typedef struct CrwContainer {
>       CRW crw;
> @@ -1194,6 +1195,26 @@ static void sch_handle_start_func_virtual(SubchDev *sch)
>   
>   }
>   
> +static void sch_handle_halt_func_passthrough(SubchDev *sch)
> +{
> +    int ret;
> +
> +    ret = vfio_ccw_handle_halt(sch);
> +    if (ret == -ENOSYS) {
> +        sch_handle_halt_func(sch);
> +    }
> +}
> +
> +static void sch_handle_clear_func_passthrough(SubchDev *sch)
> +{
> +    int ret;
> +
> +    ret = vfio_ccw_handle_clear(sch);
> +    if (ret == -ENOSYS) {
> +        sch_handle_clear_func(sch);
> +    }
> +}
> +
>   static IOInstEnding sch_handle_start_func_passthrough(SubchDev *sch)
>   {
>   
> @@ -1237,11 +1258,9 @@ IOInstEnding do_subchannel_work_passthrough(SubchDev *sch)
>       SCSW *s = &sch->curr_status.scsw;
>   
>       if (s->ctrl & SCSW_FCTL_CLEAR_FUNC) {
> -        /* TODO: Clear handling */
> -        sch_handle_clear_func(sch);
> +        sch_handle_clear_func_passthrough(sch);
>       } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
> -        /* TODO: Halt handling */
> -        sch_handle_halt_func(sch);
> +        sch_handle_halt_func_passthrough(sch);
>       } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
>           return sch_handle_start_func_passthrough(sch);
>       }
> diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
> index 9246729a75..dd0fecf168 100644
> --- a/hw/vfio/ccw.c
> +++ b/hw/vfio/ccw.c
> @@ -2,9 +2,12 @@
>    * vfio based subchannel assignment support
>    *
>    * Copyright 2017 IBM Corp.
> + * Copyright 2018 Red Hat, Inc.
> + *
>    * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
>    *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
>    *            Pierre Morel <pmorel@linux.vnet.ibm.com>
> + *            Cornelia Huck <cohuck@redhat.com>
>    *
>    * This work is licensed under the terms of the GNU GPL, version 2 or (at
>    * your option) any later version. See the COPYING file in the top-level
> @@ -32,6 +35,9 @@ typedef struct VFIOCCWDevice {
>       uint64_t io_region_size;
>       uint64_t io_region_offset;
>       struct ccw_io_region *io_region;
> +    uint64_t async_cmd_region_size;
> +    uint64_t async_cmd_region_offset;
> +    struct ccw_cmd_region *async_cmd_region;
>       EventNotifier io_notifier;
>       bool force_orb_pfch;
>       bool warned_orb_pfch;
> @@ -114,6 +120,87 @@ again:
>       }
>   }
>   
> +int vfio_ccw_handle_clear(SubchDev *sch)
> +{
> +    S390CCWDevice *cdev = sch->driver_data;
> +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
> +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
> +    int ret;
> +
> +    if (!vcdev->async_cmd_region) {
> +        /* Async command region not available, fall back to emulation */
> +        return -ENOSYS;
> +    }
> +
> +    memset(region, 0, sizeof(*region));
> +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
> +
> +again:
> +    ret = pwrite(vcdev->vdev.fd, region,
> +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
> +    if (ret != vcdev->async_cmd_region_size) {
> +        if (errno == EAGAIN) {


Where do the EAGAIN come from?


> +            goto again;
> +        }
> +        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
> +        ret = -errno;
> +    } else {
> +        ret = region->ret_code;
> +    }
> +    switch (ret) {
> +    case 0:
> +    case -ENODEV:
> +    case -EACCES:

should never happen?

> +        return 0;
> +    case -EFAULT:
> +    default:
> +        sch_gen_unit_exception(sch);
> +        css_inject_io_interrupt(sch);
> +        return 0;
> +    }
> +}
> +

otherwise LGTM

-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany

Re: [Qemu-devel] [PATCH 2/2] vfio-ccw: support async command subregion

[Pierre Morel]

On 22/11/2018 17:54, Cornelia Huck wrote:
> A vfio-ccw device may provide an async command subregion for
> issuing halt/clear subchannel requests. If it is present, use
> it for sending halt/clear request to the device; if not, fall
> back to emulation (as done today).
> 
> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> ---
>   hw/s390x/css.c              |  27 +++++++--
>   hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
>   include/hw/s390x/s390-ccw.h |   3 +
>   3 files changed, 133 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/s390x/css.c b/hw/s390x/css.c
> index 04ec5cc970..0897c041c5 100644
> --- a/hw/s390x/css.c
> +++ b/hw/s390x/css.c
> @@ -22,6 +22,7 @@
>   #include "trace.h"
>   #include "hw/s390x/s390_flic.h"
>   #include "hw/s390x/s390-virtio-ccw.h"
> +#include "hw/s390x/s390-ccw.h"
>   
>   typedef struct CrwContainer {
>       CRW crw;
> @@ -1194,6 +1195,26 @@ static void sch_handle_start_func_virtual(SubchDev *sch)
>   
>   }
>   
> +static void sch_handle_halt_func_passthrough(SubchDev *sch)
> +{
> +    int ret;
> +
> +    ret = vfio_ccw_handle_halt(sch);
> +    if (ret == -ENOSYS) {
> +        sch_handle_halt_func(sch);
> +    }
> +}
> +
> +static void sch_handle_clear_func_passthrough(SubchDev *sch)
> +{
> +    int ret;
> +
> +    ret = vfio_ccw_handle_clear(sch);
> +    if (ret == -ENOSYS) {
> +        sch_handle_clear_func(sch);
> +    }
> +}
> +
>   static IOInstEnding sch_handle_start_func_passthrough(SubchDev *sch)
>   {
>   
> @@ -1237,11 +1258,9 @@ IOInstEnding do_subchannel_work_passthrough(SubchDev *sch)
>       SCSW *s = &sch->curr_status.scsw;
>   
>       if (s->ctrl & SCSW_FCTL_CLEAR_FUNC) {
> -        /* TODO: Clear handling */
> -        sch_handle_clear_func(sch);
> +        sch_handle_clear_func_passthrough(sch);
>       } else if (s->ctrl & SCSW_FCTL_HALT_FUNC) {
> -        /* TODO: Halt handling */
> -        sch_handle_halt_func(sch);
> +        sch_handle_halt_func_passthrough(sch);
>       } else if (s->ctrl & SCSW_FCTL_START_FUNC) {
>           return sch_handle_start_func_passthrough(sch);
>       }
> diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
> index 9246729a75..dd0fecf168 100644
> --- a/hw/vfio/ccw.c
> +++ b/hw/vfio/ccw.c
> @@ -2,9 +2,12 @@
>    * vfio based subchannel assignment support
>    *
>    * Copyright 2017 IBM Corp.
> + * Copyright 2018 Red Hat, Inc.
> + *
>    * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
>    *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
>    *            Pierre Morel <pmorel@linux.vnet.ibm.com>
> + *            Cornelia Huck <cohuck@redhat.com>
>    *
>    * This work is licensed under the terms of the GNU GPL, version 2 or (at
>    * your option) any later version. See the COPYING file in the top-level
> @@ -32,6 +35,9 @@ typedef struct VFIOCCWDevice {
>       uint64_t io_region_size;
>       uint64_t io_region_offset;
>       struct ccw_io_region *io_region;
> +    uint64_t async_cmd_region_size;
> +    uint64_t async_cmd_region_offset;
> +    struct ccw_cmd_region *async_cmd_region;
>       EventNotifier io_notifier;
>       bool force_orb_pfch;
>       bool warned_orb_pfch;
> @@ -114,6 +120,87 @@ again:
>       }
>   }
>   
> +int vfio_ccw_handle_clear(SubchDev *sch)
> +{
> +    S390CCWDevice *cdev = sch->driver_data;
> +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
> +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
> +    int ret;
> +
> +    if (!vcdev->async_cmd_region) {
> +        /* Async command region not available, fall back to emulation */
> +        return -ENOSYS;
> +    }
> +
> +    memset(region, 0, sizeof(*region));
> +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
> +
> +again:
> +    ret = pwrite(vcdev->vdev.fd, region,
> +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
> +    if (ret != vcdev->async_cmd_region_size) {
> +        if (errno == EAGAIN) {


Where do the EAGAIN come from?


> +            goto again;
> +        }
> +        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
> +        ret = -errno;
> +    } else {
> +        ret = region->ret_code;
> +    }
> +    switch (ret) {
> +    case 0:
> +    case -ENODEV:
> +    case -EACCES:

should never happen?

> +        return 0;
> +    case -EFAULT:
> +    default:
> +        sch_gen_unit_exception(sch);
> +        css_inject_io_interrupt(sch);
> +        return 0;
> +    }
> +}
> +

otherwise LGTM

-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany

Re: [PATCH 2/2] vfio-ccw: support async command subregion

[Cornelia Huck]

On Fri, 23 Nov 2018 15:13:12 +0100
Pierre Morel <pmorel@linux.ibm.com> wrote:

> On 22/11/2018 17:54, Cornelia Huck wrote:
> > A vfio-ccw device may provide an async command subregion for
> > issuing halt/clear subchannel requests. If it is present, use
> > it for sending halt/clear request to the device; if not, fall
> > back to emulation (as done today).
> > 
> > Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> > ---
> >   hw/s390x/css.c              |  27 +++++++--
> >   hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
> >   include/hw/s390x/s390-ccw.h |   3 +
> >   3 files changed, 133 insertions(+), 6 deletions(-)
> > 

> > @@ -114,6 +120,87 @@ again:
> >       }
> >   }
> >   
> > +int vfio_ccw_handle_clear(SubchDev *sch)
> > +{
> > +    S390CCWDevice *cdev = sch->driver_data;
> > +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
> > +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
> > +    int ret;
> > +
> > +    if (!vcdev->async_cmd_region) {
> > +        /* Async command region not available, fall back to emulation */
> > +        return -ENOSYS;
> > +    }
> > +
> > +    memset(region, 0, sizeof(*region));
> > +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
> > +
> > +again:
> > +    ret = pwrite(vcdev->vdev.fd, region,
> > +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
> > +    if (ret != vcdev->async_cmd_region_size) {
> > +        if (errno == EAGAIN) {  
> 
> 
> Where do the EAGAIN come from?

It might be set by pwrite.

> 
> 
> > +            goto again;
> > +        }
> > +        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);

I should also fix up this message here and for hsch as well :)

> > +        ret = -errno;
> > +    } else {
> > +        ret = region->ret_code;
> > +    }
> > +    switch (ret) {
> > +    case 0:
> > +    case -ENODEV:
> > +    case -EACCES:  
> 
> should never happen?

It should not happen; but it can nevertheless be returned, so...

> 
> > +        return 0;
> > +    case -EFAULT:
> > +    default:
> > +        sch_gen_unit_exception(sch);
> > +        css_inject_io_interrupt(sch);
> > +        return 0;
> > +    }
> > +}
> > +  
> 
> otherwise LGTM

Thanks!

Re: [Qemu-devel] [PATCH 2/2] vfio-ccw: support async command subregion

[Cornelia Huck]

On Fri, 23 Nov 2018 15:13:12 +0100
Pierre Morel <pmorel@linux.ibm.com> wrote:

> On 22/11/2018 17:54, Cornelia Huck wrote:
> > A vfio-ccw device may provide an async command subregion for
> > issuing halt/clear subchannel requests. If it is present, use
> > it for sending halt/clear request to the device; if not, fall
> > back to emulation (as done today).
> > 
> > Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> > ---
> >   hw/s390x/css.c              |  27 +++++++--
> >   hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
> >   include/hw/s390x/s390-ccw.h |   3 +
> >   3 files changed, 133 insertions(+), 6 deletions(-)
> > 

> > @@ -114,6 +120,87 @@ again:
> >       }
> >   }
> >   
> > +int vfio_ccw_handle_clear(SubchDev *sch)
> > +{
> > +    S390CCWDevice *cdev = sch->driver_data;
> > +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
> > +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
> > +    int ret;
> > +
> > +    if (!vcdev->async_cmd_region) {
> > +        /* Async command region not available, fall back to emulation */
> > +        return -ENOSYS;
> > +    }
> > +
> > +    memset(region, 0, sizeof(*region));
> > +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
> > +
> > +again:
> > +    ret = pwrite(vcdev->vdev.fd, region,
> > +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
> > +    if (ret != vcdev->async_cmd_region_size) {
> > +        if (errno == EAGAIN) {  
> 
> 
> Where do the EAGAIN come from?

It might be set by pwrite.

> 
> 
> > +            goto again;
> > +        }
> > +        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);

I should also fix up this message here and for hsch as well :)

> > +        ret = -errno;
> > +    } else {
> > +        ret = region->ret_code;
> > +    }
> > +    switch (ret) {
> > +    case 0:
> > +    case -ENODEV:
> > +    case -EACCES:  
> 
> should never happen?

It should not happen; but it can nevertheless be returned, so...

> 
> > +        return 0;
> > +    case -EFAULT:
> > +    default:
> > +        sch_gen_unit_exception(sch);
> > +        css_inject_io_interrupt(sch);
> > +        return 0;
> > +    }
> > +}
> > +  
> 
> otherwise LGTM

Thanks!

Re: [PATCH 2/2] vfio-ccw: support async command subregion

[Pierre Morel]

On 26/11/2018 10:58, Cornelia Huck wrote:
> On Fri, 23 Nov 2018 15:13:12 +0100
> Pierre Morel <pmorel@linux.ibm.com> wrote:
> 
>> On 22/11/2018 17:54, Cornelia Huck wrote:
>>> A vfio-ccw device may provide an async command subregion for
>>> issuing halt/clear subchannel requests. If it is present, use
>>> it for sending halt/clear request to the device; if not, fall
>>> back to emulation (as done today).
>>>
>>> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
>>> ---
>>>    hw/s390x/css.c              |  27 +++++++--
>>>    hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
>>>    include/hw/s390x/s390-ccw.h |   3 +
>>>    3 files changed, 133 insertions(+), 6 deletions(-)
>>>
> 
>>> @@ -114,6 +120,87 @@ again:
>>>        }
>>>    }
>>>    
>>> +int vfio_ccw_handle_clear(SubchDev *sch)
>>> +{
>>> +    S390CCWDevice *cdev = sch->driver_data;
>>> +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
>>> +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
>>> +    int ret;
>>> +
>>> +    if (!vcdev->async_cmd_region) {
>>> +        /* Async command region not available, fall back to emulation */
>>> +        return -ENOSYS;
>>> +    }
>>> +
>>> +    memset(region, 0, sizeof(*region));
>>> +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
>>> +
>>> +again:
>>> +    ret = pwrite(vcdev->vdev.fd, region,
>>> +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
>>> +    if (ret != vcdev->async_cmd_region_size) {
>>> +        if (errno == EAGAIN) {
>>
>>
>> Where do the EAGAIN come from?
> 
> It might be set by pwrite.

I saw that the man indicate this, and so we are legitimate to handle the 
fail case, but I did not find EAGAIN in the path of the write for 
accessing devices and I did not find it in the access to the CSS.

If we do not set it explicitly from the driver, the concern I have is: 
isn't it dangerous to try again and shouldn't we better abort?


> 
>>
>>
>>> +            goto again;
>>> +        }
>>> +        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
> 
> I should also fix up this message here and for hsch as well :)
> 
>>> +        ret = -errno;
>>> +    } else {
>>> +        ret = region->ret_code;
>>> +    }
>>> +    switch (ret) {
>>> +    case 0:
>>> +    case -ENODEV:
>>> +    case -EACCES:
>>
>> should never happen?
> 
> It should not happen; but it can nevertheless be returned, so...

I understand: nothing to clear :)


> 
>>
>>> +        return 0;
>>> +    case -EFAULT:
>>> +    default:
>>> +        sch_gen_unit_exception(sch);
>>> +        css_inject_io_interrupt(sch);
>>> +        return 0;
>>> +    }
>>> +}
>>> +
>>
>> otherwise LGTM
> 
> Thanks!
> 






-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany

Re: [Qemu-devel] [PATCH 2/2] vfio-ccw: support async command subregion

[Pierre Morel]

On 26/11/2018 10:58, Cornelia Huck wrote:
> On Fri, 23 Nov 2018 15:13:12 +0100
> Pierre Morel <pmorel@linux.ibm.com> wrote:
> 
>> On 22/11/2018 17:54, Cornelia Huck wrote:
>>> A vfio-ccw device may provide an async command subregion for
>>> issuing halt/clear subchannel requests. If it is present, use
>>> it for sending halt/clear request to the device; if not, fall
>>> back to emulation (as done today).
>>>
>>> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
>>> ---
>>>    hw/s390x/css.c              |  27 +++++++--
>>>    hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
>>>    include/hw/s390x/s390-ccw.h |   3 +
>>>    3 files changed, 133 insertions(+), 6 deletions(-)
>>>
> 
>>> @@ -114,6 +120,87 @@ again:
>>>        }
>>>    }
>>>    
>>> +int vfio_ccw_handle_clear(SubchDev *sch)
>>> +{
>>> +    S390CCWDevice *cdev = sch->driver_data;
>>> +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
>>> +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
>>> +    int ret;
>>> +
>>> +    if (!vcdev->async_cmd_region) {
>>> +        /* Async command region not available, fall back to emulation */
>>> +        return -ENOSYS;
>>> +    }
>>> +
>>> +    memset(region, 0, sizeof(*region));
>>> +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
>>> +
>>> +again:
>>> +    ret = pwrite(vcdev->vdev.fd, region,
>>> +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
>>> +    if (ret != vcdev->async_cmd_region_size) {
>>> +        if (errno == EAGAIN) {
>>
>>
>> Where do the EAGAIN come from?
> 
> It might be set by pwrite.

I saw that the man indicate this, and so we are legitimate to handle the 
fail case, but I did not find EAGAIN in the path of the write for 
accessing devices and I did not find it in the access to the CSS.

If we do not set it explicitly from the driver, the concern I have is: 
isn't it dangerous to try again and shouldn't we better abort?


> 
>>
>>
>>> +            goto again;
>>> +        }
>>> +        error_report("vfio-ccw: wirte I/O region failed with errno=%d", errno);
> 
> I should also fix up this message here and for hsch as well :)
> 
>>> +        ret = -errno;
>>> +    } else {
>>> +        ret = region->ret_code;
>>> +    }
>>> +    switch (ret) {
>>> +    case 0:
>>> +    case -ENODEV:
>>> +    case -EACCES:
>>
>> should never happen?
> 
> It should not happen; but it can nevertheless be returned, so...

I understand: nothing to clear :)


> 
>>
>>> +        return 0;
>>> +    case -EFAULT:
>>> +    default:
>>> +        sch_gen_unit_exception(sch);
>>> +        css_inject_io_interrupt(sch);
>>> +        return 0;
>>> +    }
>>> +}
>>> +
>>
>> otherwise LGTM
> 
> Thanks!
> 






-- 
Pierre Morel
Linux/KVM/QEMU in Böblingen - Germany

Re: [PATCH 2/2] vfio-ccw: support async command subregion

[Cornelia Huck]

On Mon, 26 Nov 2018 19:01:55 +0100
Pierre Morel <pmorel@linux.ibm.com> wrote:

> On 26/11/2018 10:58, Cornelia Huck wrote:
> > On Fri, 23 Nov 2018 15:13:12 +0100
> > Pierre Morel <pmorel@linux.ibm.com> wrote:
> >   
> >> On 22/11/2018 17:54, Cornelia Huck wrote:  
> >>> A vfio-ccw device may provide an async command subregion for
> >>> issuing halt/clear subchannel requests. If it is present, use
> >>> it for sending halt/clear request to the device; if not, fall
> >>> back to emulation (as done today).
> >>>
> >>> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> >>> ---
> >>>    hw/s390x/css.c              |  27 +++++++--
> >>>    hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
> >>>    include/hw/s390x/s390-ccw.h |   3 +
> >>>    3 files changed, 133 insertions(+), 6 deletions(-)
> >>>  
> >   
> >>> @@ -114,6 +120,87 @@ again:
> >>>        }
> >>>    }
> >>>    
> >>> +int vfio_ccw_handle_clear(SubchDev *sch)
> >>> +{
> >>> +    S390CCWDevice *cdev = sch->driver_data;
> >>> +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
> >>> +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
> >>> +    int ret;
> >>> +
> >>> +    if (!vcdev->async_cmd_region) {
> >>> +        /* Async command region not available, fall back to emulation */
> >>> +        return -ENOSYS;
> >>> +    }
> >>> +
> >>> +    memset(region, 0, sizeof(*region));
> >>> +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
> >>> +
> >>> +again:
> >>> +    ret = pwrite(vcdev->vdev.fd, region,
> >>> +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
> >>> +    if (ret != vcdev->async_cmd_region_size) {
> >>> +        if (errno == EAGAIN) {  
> >>
> >>
> >> Where do the EAGAIN come from?  
> > 
> > It might be set by pwrite.  
> 
> I saw that the man indicate this, and so we are legitimate to handle the 
> fail case, but I did not find EAGAIN in the path of the write for 
> accessing devices and I did not find it in the access to the CSS.
> 
> If we do not set it explicitly from the driver, the concern I have is: 
> isn't it dangerous to try again and shouldn't we better abort?

If it is set by the reason stated in the man page, retry sounds like
the sensible thing, doesn't it? I don't think I've yet seen it in
practice, though.

[I don't think we should need to comb through the whole code path to
find out what might happen or not, at some point we'll just have to
trust the documentation.]

Re: [Qemu-devel] [PATCH 2/2] vfio-ccw: support async command subregion

[Cornelia Huck]

On Mon, 26 Nov 2018 19:01:55 +0100
Pierre Morel <pmorel@linux.ibm.com> wrote:

> On 26/11/2018 10:58, Cornelia Huck wrote:
> > On Fri, 23 Nov 2018 15:13:12 +0100
> > Pierre Morel <pmorel@linux.ibm.com> wrote:
> >   
> >> On 22/11/2018 17:54, Cornelia Huck wrote:  
> >>> A vfio-ccw device may provide an async command subregion for
> >>> issuing halt/clear subchannel requests. If it is present, use
> >>> it for sending halt/clear request to the device; if not, fall
> >>> back to emulation (as done today).
> >>>
> >>> Signed-off-by: Cornelia Huck <cohuck@redhat.com>
> >>> ---
> >>>    hw/s390x/css.c              |  27 +++++++--
> >>>    hw/vfio/ccw.c               | 109 +++++++++++++++++++++++++++++++++++-
> >>>    include/hw/s390x/s390-ccw.h |   3 +
> >>>    3 files changed, 133 insertions(+), 6 deletions(-)
> >>>  
> >   
> >>> @@ -114,6 +120,87 @@ again:
> >>>        }
> >>>    }
> >>>    
> >>> +int vfio_ccw_handle_clear(SubchDev *sch)
> >>> +{
> >>> +    S390CCWDevice *cdev = sch->driver_data;
> >>> +    VFIOCCWDevice *vcdev = DO_UPCAST(VFIOCCWDevice, cdev, cdev);
> >>> +    struct ccw_cmd_region *region = vcdev->async_cmd_region;
> >>> +    int ret;
> >>> +
> >>> +    if (!vcdev->async_cmd_region) {
> >>> +        /* Async command region not available, fall back to emulation */
> >>> +        return -ENOSYS;
> >>> +    }
> >>> +
> >>> +    memset(region, 0, sizeof(*region));
> >>> +    region->command = VFIO_CCW_ASYNC_CMD_CSCH;
> >>> +
> >>> +again:
> >>> +    ret = pwrite(vcdev->vdev.fd, region,
> >>> +                 vcdev->async_cmd_region_size, vcdev->async_cmd_region_offset);
> >>> +    if (ret != vcdev->async_cmd_region_size) {
> >>> +        if (errno == EAGAIN) {  
> >>
> >>
> >> Where do the EAGAIN come from?  
> > 
> > It might be set by pwrite.  
> 
> I saw that the man indicate this, and so we are legitimate to handle the 
> fail case, but I did not find EAGAIN in the path of the write for 
> accessing devices and I did not find it in the access to the CSS.
> 
> If we do not set it explicitly from the driver, the concern I have is: 
> isn't it dangerous to try again and shouldn't we better abort?

If it is set by the reason stated in the man page, retry sounds like
the sensible thing, doesn't it? I don't think I've yet seen it in
practice, though.

[I don't think we should need to comb through the whole code path to
find out what might happen or not, at some point we'll just have to
trust the documentation.]