[Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Jason J. Herne]

This is to support booting from vfio-ccw dasd devices. We basically implement
the real hardware ipl procedure. This allows for booting Linux guests on
vfio-ccw devices.

vfio-ccw's channel program prefetch algorithm complicates ipl because most ipl
channel programs dynamically modify themselves. Details on the ipl process and
how we worked around this issue can be found in docs/devel/s390-dasd-ipl.txt.

NOTE: It has been a while, but I've finally chased down my infamous "reset bug".
On subsystem reset (I see this right after host ipl) we sometimes end up getting
an unexpected unit check status from a dasd device. This causes the first start
subchannel instruction to fail due to the pending unit check status. My solution
to this problem, as advised by the kernel folks, is to simply retry my ssch
instructions before declaring failure when unexpected unit checks happen. In the
event of a persistent error, after two retries we'll give up and print some
useful error info for the user.

Changelog
==========

rfc -> v1
---------
- Retry start subchannel operations on unexpected unit check status
- Print more complete state information on i/o error
- Fixed asm constraints (Q -> QS)
- Align lowcore to 8k boundary
- Initialize next_cpa in run_dynamic_ccw_program to avoid compiler warning
- Cleanup: Rework handling of i/o interrupts in assembler
- Fix netboot image (include cio.o in make file)
- Misc cleanups

Jason J. Herne (15):
  s390 vfio-ccw: Add bootindex property and IPLB data
  s390-bios: decouple cio setup from virtio
  s390-bios: decouple common boot logic from virtio
  s390-bios: Extend find_dev() for non-virtio devices
  s390-bios: Factor finding boot device out of virtio code path
  s390-bios: Clean up cio.h
  s390-bios: Decouple channel i/o logic from virtio
  s390-bios: Map low core memory
  s390-bios: ptr2u32 and u32toptr
  s390-bios: Support for running format-0/1 channel programs
  s390-bios: cio error handling
  s390-bios: Refactor virtio to run channel programs via cio
  s390-bios: Use control unit type to determine boot method
  s390-bios: Add channel command codes/structs needed for dasd-ipl
  s390-bios: Support booting from real dasd device

 docs/devel/s390-dasd-ipl.txt     | 132 ++++++++++++++
 hw/s390x/ipl.c                   |  15 ++
 hw/s390x/s390-ccw.c              |   9 +
 hw/vfio/ccw.c                    |  13 +-
 include/hw/s390x/s390-ccw.h      |   1 +
 include/hw/s390x/vfio-ccw.h      |  38 ++++
 pc-bios/s390-ccw/Makefile        |   2 +-
 pc-bios/s390-ccw/cio.c           | 374 +++++++++++++++++++++++++++++++++++++++
 pc-bios/s390-ccw/cio.h           | 236 +++++++++++++++++++-----
 pc-bios/s390-ccw/dasd-ipl.c      | 249 ++++++++++++++++++++++++++
 pc-bios/s390-ccw/dasd-ipl.h      |  16 ++
 pc-bios/s390-ccw/libc.h          |  23 +++
 pc-bios/s390-ccw/main.c          | 161 +++++++++++------
 pc-bios/s390-ccw/netboot.mak     |   2 +-
 pc-bios/s390-ccw/netmain.c       |   1 +
 pc-bios/s390-ccw/s390-arch.h     | 113 ++++++++++++
 pc-bios/s390-ccw/s390-ccw.h      |  10 +-
 pc-bios/s390-ccw/start.S         |  33 +++-
 pc-bios/s390-ccw/virtio-blkdev.c |   1 +
 pc-bios/s390-ccw/virtio.c        |  46 +----
 roms/SLOF                        |   2 +-
 tests/boot-serial-test.c         |   2 +-
 22 files changed, 1308 insertions(+), 171 deletions(-)
 create mode 100644 docs/devel/s390-dasd-ipl.txt
 create mode 100644 include/hw/s390x/vfio-ccw.h
 create mode 100644 pc-bios/s390-ccw/cio.c
 create mode 100644 pc-bios/s390-ccw/dasd-ipl.c
 create mode 100644 pc-bios/s390-ccw/dasd-ipl.h
 create mode 100644 pc-bios/s390-ccw/s390-arch.h

--
2.7.4

[Qemu-devel] [PATCH 01/15] s390 vfio-ccw: Add bootindex property and IPLB data

[Jason J. Herne]

Add bootindex property and iplb data for vfio-ccw devices. This allows us to
forward boot information into the bios for vfio-ccw devices.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
---
 hw/s390x/ipl.c              | 15 +++++++++++++++
 hw/s390x/s390-ccw.c         |  9 +++++++++
 hw/vfio/ccw.c               | 13 +------------
 include/hw/s390x/s390-ccw.h |  1 +
 include/hw/s390x/vfio-ccw.h | 38 ++++++++++++++++++++++++++++++++++++++
 roms/SLOF                   |  2 +-
 6 files changed, 65 insertions(+), 13 deletions(-)
 create mode 100644 include/hw/s390x/vfio-ccw.h

diff --git a/hw/s390x/ipl.c b/hw/s390x/ipl.c
index 21f64ad..31592e7 100644
--- a/hw/s390x/ipl.c
+++ b/hw/s390x/ipl.c
@@ -19,6 +19,7 @@
 #include "hw/loader.h"
 #include "hw/boards.h"
 #include "hw/s390x/virtio-ccw.h"
+#include "hw/s390x/vfio-ccw.h"
 #include "hw/s390x/css.h"
 #include "hw/s390x/ebcdic.h"
 #include "ipl.h"
@@ -311,8 +312,13 @@ static CcwDevice *s390_get_ccw_device(DeviceState *dev_st)
         VirtioCcwDevice *virtio_ccw_dev = (VirtioCcwDevice *)
             object_dynamic_cast(OBJECT(qdev_get_parent_bus(dev_st)->parent),
                                 TYPE_VIRTIO_CCW_DEVICE);
+        VFIOCCWDevice *vc = (VFIOCCWDevice *)
+            object_dynamic_cast(OBJECT(dev_st), TYPE_VFIO_CCW);
+
         if (virtio_ccw_dev) {
             ccw_dev = CCW_DEVICE(virtio_ccw_dev);
+        } else if (vc) {
+            ccw_dev = CCW_DEVICE(vc);
         } else {
             SCSIDevice *sd = (SCSIDevice *)
                 object_dynamic_cast(OBJECT(dev_st),
@@ -347,6 +353,8 @@ static bool s390_gen_initial_iplb(S390IPLState *ipl)
     if (ccw_dev) {
         SCSIDevice *sd = (SCSIDevice *) object_dynamic_cast(OBJECT(dev_st),
                                                             TYPE_SCSI_DEVICE);
+        VFIOCCWDevice *vc = (VFIOCCWDevice *)
+            object_dynamic_cast(OBJECT(dev_st), TYPE_VFIO_CCW);
 
         if (sd) {
             ipl->iplb.len = cpu_to_be32(S390_IPLB_MIN_QEMU_SCSI_LEN);
@@ -358,6 +366,13 @@ static bool s390_gen_initial_iplb(S390IPLState *ipl)
             ipl->iplb.scsi.channel = cpu_to_be16(sd->channel);
             ipl->iplb.scsi.devno = cpu_to_be16(ccw_dev->sch->devno);
             ipl->iplb.scsi.ssid = ccw_dev->sch->ssid & 3;
+        } else if (vc) {
+            CcwDevice *ccw_dev = CCW_DEVICE(vc);
+
+            ipl->iplb.len = cpu_to_be32(S390_IPLB_MIN_CCW_LEN);
+            ipl->iplb.pbt = S390_IPL_TYPE_CCW;
+            ipl->iplb.ccw.devno = cpu_to_be16(ccw_dev->sch->devno);
+            ipl->iplb.ccw.ssid = ccw_dev->sch->ssid & 3;
         } else {
             VirtIONet *vn = (VirtIONet *) object_dynamic_cast(OBJECT(dev_st),
                                                               TYPE_VIRTIO_NET);
diff --git a/hw/s390x/s390-ccw.c b/hw/s390x/s390-ccw.c
index cad91ee..f5f025d 100644
--- a/hw/s390x/s390-ccw.c
+++ b/hw/s390x/s390-ccw.c
@@ -124,6 +124,14 @@ static void s390_ccw_unrealize(S390CCWDevice *cdev, Error **errp)
     g_free(cdev->mdevid);
 }
 
+static void s390_ccw_instance_init(Object *obj)
+{
+    S390CCWDevice *dev = S390_CCW_DEVICE(obj);
+
+    device_add_bootindex_property(obj, &dev->bootindex, "bootindex",
+                                  "/disk@0,0", DEVICE(obj), NULL);
+}
+
 static void s390_ccw_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -137,6 +145,7 @@ static void s390_ccw_class_init(ObjectClass *klass, void *data)
 static const TypeInfo s390_ccw_info = {
     .name          = TYPE_S390_CCW,
     .parent        = TYPE_CCW_DEVICE,
+    .instance_init = s390_ccw_instance_init,
     .instance_size = sizeof(S390CCWDevice),
     .class_size    = sizeof(S390CCWDeviceClass),
     .class_init    = s390_ccw_class_init,
diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
index 9246729..6f480c6 100644
--- a/hw/vfio/ccw.c
+++ b/hw/vfio/ccw.c
@@ -20,23 +20,12 @@
 #include "hw/sysbus.h"
 #include "hw/vfio/vfio.h"
 #include "hw/vfio/vfio-common.h"
+#include "hw/vfio/ccw.h"
 #include "hw/s390x/s390-ccw.h"
 #include "hw/s390x/ccw-device.h"
 #include "exec/address-spaces.h"
 #include "qemu/error-report.h"
 
-#define TYPE_VFIO_CCW "vfio-ccw"
-typedef struct VFIOCCWDevice {
-    S390CCWDevice cdev;
-    VFIODevice vdev;
-    uint64_t io_region_size;
-    uint64_t io_region_offset;
-    struct ccw_io_region *io_region;
-    EventNotifier io_notifier;
-    bool force_orb_pfch;
-    bool warned_orb_pfch;
-} VFIOCCWDevice;
-
 static inline void warn_once_pfch(VFIOCCWDevice *vcdev, SubchDev *sch,
                                   const char *msg)
 {
diff --git a/include/hw/s390x/s390-ccw.h b/include/hw/s390x/s390-ccw.h
index 7d15a1a..901d805 100644
--- a/include/hw/s390x/s390-ccw.h
+++ b/include/hw/s390x/s390-ccw.h
@@ -27,6 +27,7 @@ typedef struct S390CCWDevice {
     CcwDevice parent_obj;
     CssDevId hostid;
     char *mdevid;
+    int32_t bootindex;
 } S390CCWDevice;
 
 typedef struct S390CCWDeviceClass {
diff --git a/include/hw/s390x/vfio-ccw.h b/include/hw/s390x/vfio-ccw.h
new file mode 100644
index 0000000..a7d699d
--- /dev/null
+++ b/include/hw/s390x/vfio-ccw.h
@@ -0,0 +1,38 @@
+/*
+ * vfio based subchannel assignment support
+ *
+ * Copyright 2018 IBM Corp.
+ * Author(s): Dong Jia Shi <bjsdjshi@linux.vnet.ibm.com>
+ *            Xiao Feng Ren <renxiaof@linux.vnet.ibm.com>
+ *            Pierre Morel <pmorel@linux.vnet.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef HW_VFIO_CCW_H
+#define HW_VFIO_CCW_H
+
+#include "hw/vfio/vfio-common.h"
+#include "hw/s390x/s390-ccw.h"
+#include "hw/s390x/ccw-device.h"
+
+#define TYPE_VFIO_CCW "vfio-ccw"
+#define VFIO_CCW(obj) \
+        OBJECT_CHECK(VFIOCCWDevice, (obj), TYPE_VFIO_CCW)
+
+
+#define TYPE_VFIO_CCW "vfio-ccw"
+typedef struct VFIOCCWDevice {
+    S390CCWDevice cdev;
+    VFIODevice vdev;
+    uint64_t io_region_size;
+    uint64_t io_region_offset;
+    struct ccw_io_region *io_region;
+    EventNotifier io_notifier;
+    bool force_orb_pfch;
+    bool warned_orb_pfch;
+} VFIOCCWDevice;
+
+#endif
diff --git a/roms/SLOF b/roms/SLOF
index 9b7ab2f..2317427 160000
--- a/roms/SLOF
+++ b/roms/SLOF
@@ -1 +1 @@
-Subproject commit 9b7ab2fa020341dee8bf9df6c9cf40003e0136df
+Subproject commit 2317427ce76006723f7ae103a6998ab41dd79c68
-- 
2.7.4

[Qemu-devel] [PATCH 02/15] s390-bios: decouple cio setup from virtio

[Jason J. Herne]

Move channel i/o setup code out to a separate function. This decouples cio
setup from the virtio code path and allows us to make use of it for booting
dasd devices.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com>
---
 pc-bios/s390-ccw/main.c | 20 +++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)

diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index 544851d..e82fe2c 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -99,6 +99,18 @@ static void menu_setup(void)
     }
 }
 
+/*
+ * Initialize the channel I/O subsystem so we can talk to our ipl/boot device.
+ */
+static void css_setup(void)
+{
+    /*
+     * Unconditionally enable mss support. In every sane configuration this
+     * will succeed; and even if it doesn't, stsch_err() can handle it.
+     */
+    enable_mss_facility();
+}
+
 static void virtio_setup(void)
 {
     Schib schib;
@@ -109,13 +121,6 @@ static void virtio_setup(void)
     VDev *vdev = virtio_get_device();
     QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
 
-    /*
-     * We unconditionally enable mss support. In every sane configuration,
-     * this will succeed; and even if it doesn't, stsch_err() can deal
-     * with the consequences.
-     */
-    enable_mss_facility();
-
     sclp_get_loadparm_ascii(loadparm_str);
     memcpy(ldp + 10, loadparm_str, LOADPARM_LEN);
     sclp_print(ldp);
@@ -168,6 +173,7 @@ static void virtio_setup(void)
 int main(void)
 {
     sclp_setup();
+    css_setup();
     virtio_setup();
 
     zipl_load(); /* no return */
-- 
2.7.4

[Qemu-devel] [PATCH 03/15] s390-bios: decouple common boot logic from virtio

[Jason J. Herne]

Create a boot_setup function to handle getting boot information from
the machine/hypervisor. This decouples common boot logic from the
virtio code path and allows us to make use of it for the real dasd boot
scenario.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com
Reviewed-by: Thomas Huth <thuth@redhat.com>
---
 pc-bios/s390-ccw/main.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)

diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index e82fe2c..67df421 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -14,16 +14,17 @@
 
 char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE)));
 static SubChannelId blk_schid = { .one = 1 };
-IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));
 static char loadparm_str[LOADPARM_LEN + 1] = { 0, 0, 0, 0, 0, 0, 0, 0, 0 };
 QemuIplParameters qipl;
+IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));
+static bool have_iplb;
 
 #define LOADPARM_PROMPT "PROMPT  "
 #define LOADPARM_EMPTY  "        "
 #define BOOT_MENU_FLAG_MASK (QIPL_FLAG_BM_OPTS_CMD | QIPL_FLAG_BM_OPTS_ZIPL)
 
 /*
- * Priniciples of Operations (SA22-7832-09) chapter 17 requires that
+ * Principles of Operations (SA22-7832-09) chapter 17 requires that
  * a subsystem-identification is at 184-187 and bytes 188-191 are zero
  * after list-directed-IPL and ccw-IPL.
  */
@@ -111,23 +112,33 @@ static void css_setup(void)
     enable_mss_facility();
 }
 
+/*
+ * Collect various pieces of information from the hypervisor/hardware that
+ * we'll use to determine exactly how we'll boot.
+ */
+static void boot_setup(void)
+{
+    char lpmsg[] = "LOADPARM=[________]\n";
+
+    sclp_get_loadparm_ascii(loadparm_str);
+    memcpy(lpmsg + 10, loadparm_str, 8);
+    sclp_print(lpmsg);
+
+    have_iplb = store_iplb(&iplb);
+}
+
 static void virtio_setup(void)
 {
     Schib schib;
     int ssid;
     bool found = false;
     uint16_t dev_no;
-    char ldp[] = "LOADPARM=[________]\n";
     VDev *vdev = virtio_get_device();
     QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
 
-    sclp_get_loadparm_ascii(loadparm_str);
-    memcpy(ldp + 10, loadparm_str, LOADPARM_LEN);
-    sclp_print(ldp);
-
     memcpy(&qipl, early_qipl, sizeof(QemuIplParameters));
 
-    if (store_iplb(&iplb)) {
+    if (have_iplb) {
         switch (iplb.pbt) {
         case S390_IPL_TYPE_CCW:
             dev_no = iplb.ccw.devno;
@@ -174,6 +185,7 @@ int main(void)
 {
     sclp_setup();
     css_setup();
+    boot_setup();
     virtio_setup();
 
     zipl_load(); /* no return */
-- 
2.7.4

[Qemu-devel] [PATCH 04/15] s390-bios: Extend find_dev() for non-virtio devices

[Jason J. Herne]

We need a method for finding the subchannel of a dasd device. Let's
modify find_dev to handle this since it mostly does what we need. Up to
this point find_dev has been specific to only virtio devices.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
---
 pc-bios/s390-ccw/main.c | 16 +++++++++++-----
 1 file changed, 11 insertions(+), 5 deletions(-)

diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index 67df421..7e3f65e 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -49,6 +49,12 @@ unsigned int get_loadparm_index(void)
     return atoui(loadparm_str);
 }
 
+/*
+ * Find the subchannel connected to the given device (dev_no) and fill in the
+ * subchannel information block (schib) with the connected subchannel's info.
+ * NOTE: The global variable blk_schid is updated to contain the subchannel
+ * information.
+ */
 static bool find_dev(Schib *schib, int dev_no)
 {
     int i, r;
@@ -62,15 +68,15 @@ static bool find_dev(Schib *schib, int dev_no)
         if (!schib->pmcw.dnv) {
             continue;
         }
-        if (!virtio_is_supported(blk_schid)) {
-            continue;
-        }
+
         /* Skip net devices since no IPLB is created and therefore no
-         * no network bootloader has been loaded
+         * network bootloader has been loaded
          */
-        if (virtio_get_device_type() == VIRTIO_ID_NET && dev_no < 0) {
+        if (virtio_is_supported(blk_schid) &&
+            virtio_get_device_type() == VIRTIO_ID_NET && dev_no < 0) {
             continue;
         }
+
         if ((dev_no < 0) || (schib->pmcw.dev == dev_no)) {
             return true;
         }
-- 
2.7.4

[Qemu-devel] [PATCH 05/15] s390-bios: Factor finding boot device out of virtio code path

[Jason J. Herne]

Make a new routine find_boot_device to locate the boot device for all
cases. not just virtio.

In one case no boot device is specified and a suitable boot device can not
be auto detected. The error message for this case was specific to virtio
devices. We update this message to remove virtio specific wording.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/main.c  | 87 ++++++++++++++++++++++++++----------------------
 tests/boot-serial-test.c |  2 +-
 2 files changed, 49 insertions(+), 40 deletions(-)

diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index 7e3f65e..2457752 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -55,17 +55,18 @@ unsigned int get_loadparm_index(void)
  * NOTE: The global variable blk_schid is updated to contain the subchannel
  * information.
  */
-static bool find_dev(Schib *schib, int dev_no)
+static bool find_subch(int dev_no)
 {
+    Schib schib;
     int i, r;
 
     for (i = 0; i < 0x10000; i++) {
         blk_schid.sch_no = i;
-        r = stsch_err(blk_schid, schib);
+        r = stsch_err(blk_schid, &schib);
         if ((r == 3) || (r == -EIO)) {
             break;
         }
-        if (!schib->pmcw.dnv) {
+        if (!schib.pmcw.dnv) {
             continue;
         }
 
@@ -77,7 +78,7 @@ static bool find_dev(Schib *schib, int dev_no)
             continue;
         }
 
-        if ((dev_no < 0) || (schib->pmcw.dev == dev_no)) {
+        if ((dev_no < 0) || (schib.pmcw.dev == dev_no)) {
             return true;
         }
     }
@@ -133,56 +134,63 @@ static void boot_setup(void)
     have_iplb = store_iplb(&iplb);
 }
 
-static void virtio_setup(void)
+static void find_boot_device(void)
 {
-    Schib schib;
-    int ssid;
-    bool found = false;
-    uint16_t dev_no;
     VDev *vdev = virtio_get_device();
-    QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
-
-    memcpy(&qipl, early_qipl, sizeof(QemuIplParameters));
+    int ssid;
+    bool found;
 
-    if (have_iplb) {
-        switch (iplb.pbt) {
-        case S390_IPL_TYPE_CCW:
-            dev_no = iplb.ccw.devno;
-            debug_print_int("device no. ", dev_no);
-            blk_schid.ssid = iplb.ccw.ssid & 0x3;
-            debug_print_int("ssid ", blk_schid.ssid);
-            found = find_dev(&schib, dev_no);
-            break;
-        case S390_IPL_TYPE_QEMU_SCSI:
-            vdev->scsi_device_selected = true;
-            vdev->selected_scsi_device.channel = iplb.scsi.channel;
-            vdev->selected_scsi_device.target = iplb.scsi.target;
-            vdev->selected_scsi_device.lun = iplb.scsi.lun;
-            blk_schid.ssid = iplb.scsi.ssid & 0x3;
-            found = find_dev(&schib, iplb.scsi.devno);
-            break;
-        default:
-            panic("List-directed IPL not supported yet!\n");
-        }
-        menu_setup();
-    } else {
+    if (!have_iplb) {
         for (ssid = 0; ssid < 0x3; ssid++) {
             blk_schid.ssid = ssid;
-            found = find_dev(&schib, -1);
+            found = find_subch(-1);
             if (found) {
-                break;
+                return;
             }
         }
+        panic("Could not find a suitable boot device (none specified)\n");
     }
 
-    IPL_assert(found, "No virtio device found");
+    switch (iplb.pbt) {
+    case S390_IPL_TYPE_CCW:
+        debug_print_int("device no. ", iplb.ccw.devno);
+        blk_schid.ssid = iplb.ccw.ssid & 0x3;
+        debug_print_int("ssid ", blk_schid.ssid);
+        found = find_subch(iplb.ccw.devno);
+        break;
+    case S390_IPL_TYPE_QEMU_SCSI:
+        vdev->scsi_device_selected = true;
+        vdev->selected_scsi_device.channel = iplb.scsi.channel;
+        vdev->selected_scsi_device.target = iplb.scsi.target;
+        vdev->selected_scsi_device.lun = iplb.scsi.lun;
+        blk_schid.ssid = iplb.scsi.ssid & 0x3;
+        found = find_subch(iplb.scsi.devno);
+        break;
+    default:
+        panic("List-directed IPL not supported yet!\n");
+    }
+
+    if (!found) {
+        IPL_assert(found, "Boot device not found\n");
+    }
+}
+
+static void virtio_setup(void)
+{
+    VDev *vdev = virtio_get_device();
+    QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
+
+    memcpy(&qipl, early_qipl, sizeof(QemuIplParameters));
+
+    if (have_iplb) {
+        menu_setup();
+    }
 
     if (virtio_get_device_type() == VIRTIO_ID_NET) {
         sclp_print("Network boot device detected\n");
         vdev->netboot_start_addr = qipl.netboot_start_addr;
     } else {
         virtio_blk_setup_device(blk_schid);
-
         IPL_assert(virtio_ipl_disk_is_valid(), "No valid IPL device detected");
     }
 }
@@ -192,8 +200,9 @@ int main(void)
     sclp_setup();
     css_setup();
     boot_setup();
-    virtio_setup();
+    find_boot_device();
 
+    virtio_setup();
     zipl_load(); /* no return */
 
     panic("Failed to load OS from hard disk\n");
diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c
index 8ec6aed..064f5de 100644
--- a/tests/boot-serial-test.c
+++ b/tests/boot-serial-test.c
@@ -112,7 +112,7 @@ static testdef_t tests[] = {
     { "sparc", "SS-4", "", "MB86904" },
     { "sparc", "SS-600MP", "", "TMS390Z55" },
     { "sparc64", "sun4u", "", "UltraSPARC" },
-    { "s390x", "s390-ccw-virtio", "", "virtio device" },
+    { "s390x", "s390-ccw-virtio", "", "device" },
     { "m68k", "mcf5208evb", "", "TT", sizeof(kernel_mcf5208), kernel_mcf5208 },
     { "microblaze", "petalogix-s3adsp1800", "", "TT",
       sizeof(kernel_pls3adsp1800), kernel_pls3adsp1800 },
-- 
2.7.4

[Qemu-devel] [PATCH 06/15] s390-bios: Clean up cio.h

[Jason J. Herne]

Add proper typedefs to all structs and modify all bit fields to use consistent
formatting.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
Reviewed-by: Collin Walling <walling@linux.ibm.com>
---
 pc-bios/s390-ccw/cio.h      | 86 ++++++++++++++++++++++-----------------------
 pc-bios/s390-ccw/s390-ccw.h |  8 -----
 2 files changed, 43 insertions(+), 51 deletions(-)

diff --git a/pc-bios/s390-ccw/cio.h b/pc-bios/s390-ccw/cio.h
index 1a0795f..a48eee5 100644
--- a/pc-bios/s390-ccw/cio.h
+++ b/pc-bios/s390-ccw/cio.h
@@ -53,12 +53,12 @@ struct schib_config {
     __u64 mba;
     __u32 intparm;
     __u16 mbi;
-    __u32 isc:3;
-    __u32 ena:1;
-    __u32 mme:2;
-    __u32 mp:1;
-    __u32 csense:1;
-    __u32 mbfc:1;
+    __u32 isc    : 3;
+    __u32 ena    : 1;
+    __u32 mme    : 2;
+    __u32 mp     : 1;
+    __u32 csense : 1;
+    __u32 mbfc   : 1;
 } __attribute__ ((packed));
 
 struct scsw {
@@ -77,41 +77,41 @@ struct scsw {
 /*
  * subchannel information block
  */
-struct schib {
+typedef struct schib {
     struct pmcw pmcw;     /* path management control word */
     struct scsw scsw;     /* subchannel status word */
     __u64 mba;            /* measurement block address */
     __u8 mda[4];          /* model dependent area */
-} __attribute__ ((packed,aligned(4)));
+} __attribute__ ((packed, aligned(4))) Schib;
 
-struct subchannel_id {
+typedef struct subchannel_id {
         __u32 cssid  : 8;
         __u32        : 4;
         __u32 m      : 1;
         __u32 ssid   : 2;
         __u32 one    : 1;
         __u32 sch_no : 16;
-} __attribute__ ((packed, aligned(4)));
+} __attribute__ ((packed, aligned(4))) SubChannelId;
 
 struct chsc_header {
     __u16 length;
     __u16 code;
 } __attribute__((packed));
 
-struct chsc_area_sda {
+typedef struct chsc_area_sda {
     struct chsc_header request;
-    __u8 reserved1:4;
-    __u8 format:4;
+    __u8 reserved1  : 4;
+    __u8 format     : 4;
     __u8 reserved2;
     __u16 operation_code;
     __u32 reserved3;
     __u32 reserved4;
     __u32 operation_data_area[252];
     struct chsc_header response;
-    __u32 reserved5:4;
-    __u32 format2:4;
-    __u32 reserved6:24;
-} __attribute__((packed));
+    __u32 reserved5 : 4;
+    __u32 format2   : 4;
+    __u32 reserved6 : 24;
+} __attribute__((packed)) ChscAreaSda;
 
 /*
  * TPI info structure
@@ -128,12 +128,12 @@ struct tpi_info {
 } __attribute__ ((packed, aligned(4)));
 
 /* channel command word (type 1) */
-struct ccw1 {
+typedef struct ccw1 {
     __u8 cmd_code;
     __u8 flags;
     __u16 count;
     __u32 cda;
-} __attribute__ ((packed, aligned(8)));
+} __attribute__ ((packed, aligned(8))) Ccw1;
 
 #define CCW_FLAG_DC              0x80
 #define CCW_FLAG_CC              0x40
@@ -162,27 +162,27 @@ struct ccw1 {
 /*
  * Command-mode operation request block
  */
-struct cmd_orb {
+typedef struct cmd_orb {
     __u32 intparm;    /* interruption parameter */
-    __u32 key:4;      /* flags, like key, suspend control, etc. */
-    __u32 spnd:1;     /* suspend control */
-    __u32 res1:1;     /* reserved */
-    __u32 mod:1;      /* modification control */
-    __u32 sync:1;     /* synchronize control */
-    __u32 fmt:1;      /* format control */
-    __u32 pfch:1;     /* prefetch control */
-    __u32 isic:1;     /* initial-status-interruption control */
-    __u32 alcc:1;     /* address-limit-checking control */
-    __u32 ssic:1;     /* suppress-suspended-interr. control */
-    __u32 res2:1;     /* reserved */
-    __u32 c64:1;      /* IDAW/QDIO 64 bit control  */
-    __u32 i2k:1;      /* IDAW 2/4kB block size control */
-    __u32 lpm:8;      /* logical path mask */
-    __u32 ils:1;      /* incorrect length */
-    __u32 zero:6;     /* reserved zeros */
-    __u32 orbx:1;     /* ORB extension control */
-    __u32 cpa;    /* channel program address */
-}  __attribute__ ((packed, aligned(4)));
+    __u32 key  : 4;   /* flags, like key, suspend control, etc. */
+    __u32 spnd : 1;   /* suspend control */
+    __u32 res1 : 1;   /* reserved */
+    __u32 mod  : 1;   /* modification control */
+    __u32 sync : 1;   /* synchronize control */
+    __u32 fmt  : 1;   /* format control */
+    __u32 pfch : 1;   /* prefetch control */
+    __u32 isic : 1;   /* initial-status-interruption control */
+    __u32 alcc : 1;   /* address-limit-checking control */
+    __u32 ssic : 1;   /* suppress-suspended-interr. control */
+    __u32 res2 : 1;   /* reserved */
+    __u32 c64  : 1;   /* IDAW/QDIO 64 bit control  */
+    __u32 i2k  : 1;   /* IDAW 2/4kB block size control */
+    __u32 lpm  : 8;   /* logical path mask */
+    __u32 ils  : 1;   /* incorrect length */
+    __u32 zero : 6;   /* reserved zeros */
+    __u32 orbx : 1;   /* ORB extension control */
+    __u32 cpa;        /* channel program address */
+}  __attribute__ ((packed, aligned(4))) CmdOrb;
 
 struct ciw {
     __u8 type;
@@ -193,7 +193,7 @@ struct ciw {
 /*
  * sense-id response buffer layout
  */
-struct senseid {
+typedef struct senseid {
     /* common part */
     __u8  reserved;   /* always 0x'FF' */
     __u16 cu_type;    /* control unit type */
@@ -203,15 +203,15 @@ struct senseid {
     __u8  unused;     /* padding byte */
     /* extended part */
     struct ciw ciw[62];
-}  __attribute__ ((packed, aligned(4)));
+}  __attribute__ ((packed, aligned(4))) SenseId;
 
 /* interruption response block */
-struct irb {
+typedef struct irb {
     struct scsw scsw;
     __u32 esw[5];
     __u32 ecw[8];
     __u32 emw[8];
-}  __attribute__ ((packed, aligned(4)));
+}  __attribute__ ((packed, aligned(4))) Irb;
 
 /*
  * Some S390 specific IO instructions as inline
diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h
index 9828aa2..241c6d0 100644
--- a/pc-bios/s390-ccw/s390-ccw.h
+++ b/pc-bios/s390-ccw/s390-ccw.h
@@ -49,14 +49,6 @@ typedef unsigned long long __u64;
 #include "cio.h"
 #include "iplb.h"
 
-typedef struct irb Irb;
-typedef struct ccw1 Ccw1;
-typedef struct cmd_orb CmdOrb;
-typedef struct schib Schib;
-typedef struct chsc_area_sda ChscAreaSda;
-typedef struct senseid SenseId;
-typedef struct subchannel_id SubChannelId;
-
 /* start.s */
 void disabled_wait(void);
 void consume_sclp_int(void);
-- 
2.7.4

[Qemu-devel] [PATCH 07/15] s390-bios: Decouple channel i/o logic from virtio

[Jason J. Herne]

Create a separate library for channel i/o related code. This decouples
channel i/o operations from virtio and allows us to make use of them for
the real dasd boot path.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/Makefile        |  2 +-
 pc-bios/s390-ccw/cio.c           | 41 ++++++++++++++++++++++++++++++++++++++++
 pc-bios/s390-ccw/cio.h           |  3 +++
 pc-bios/s390-ccw/main.c          |  1 +
 pc-bios/s390-ccw/netboot.mak     |  2 +-
 pc-bios/s390-ccw/netmain.c       |  1 +
 pc-bios/s390-ccw/s390-ccw.h      |  1 -
 pc-bios/s390-ccw/virtio-blkdev.c |  1 +
 pc-bios/s390-ccw/virtio.c        | 27 ++------------------------
 9 files changed, 51 insertions(+), 28 deletions(-)
 create mode 100644 pc-bios/s390-ccw/cio.c

diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 1eb316b..12ad9c1 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -10,7 +10,7 @@ $(call set-vpath, $(SRC_PATH)/pc-bios/s390-ccw)
 .PHONY : all clean build-all
 
 OBJECTS = start.o main.o bootmap.o jump2ipl.o sclp.o menu.o \
-	  virtio.o virtio-scsi.o virtio-blkdev.o libc.o
+	  virtio.o virtio-scsi.o virtio-blkdev.o libc.o cio.o
 
 QEMU_CFLAGS := $(filter -W%, $(QEMU_CFLAGS))
 QEMU_CFLAGS += -ffreestanding -fno-delete-null-pointer-checks -msoft-float
diff --git a/pc-bios/s390-ccw/cio.c b/pc-bios/s390-ccw/cio.c
new file mode 100644
index 0000000..095f79b
--- /dev/null
+++ b/pc-bios/s390-ccw/cio.c
@@ -0,0 +1,41 @@
+/*
+ * S390 Channel I/O
+ *
+ * Copyright (c) 2018 Jason J. Herne <jjherne@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "libc.h"
+#include "s390-ccw.h"
+#include "cio.h"
+
+static char chsc_page[PAGE_SIZE] __attribute__((__aligned__(PAGE_SIZE)));
+
+int enable_mss_facility(void)
+{
+    int ret;
+    ChscAreaSda *sda_area = (ChscAreaSda *) chsc_page;
+
+    memset(sda_area, 0, PAGE_SIZE);
+    sda_area->request.length = 0x0400;
+    sda_area->request.code = 0x0031;
+    sda_area->operation_code = 0x2;
+
+    ret = chsc(sda_area);
+    if ((ret == 0) && (sda_area->response.code == 0x0001)) {
+        return 0;
+    }
+    return -EIO;
+}
+
+void enable_subchannel(SubChannelId schid)
+{
+    Schib schib;
+
+    stsch_err(schid, &schib);
+    schib.pmcw.ena = 1;
+    msch(schid, &schib);
+}
diff --git a/pc-bios/s390-ccw/cio.h b/pc-bios/s390-ccw/cio.h
index a48eee5..7b07d75 100644
--- a/pc-bios/s390-ccw/cio.h
+++ b/pc-bios/s390-ccw/cio.h
@@ -213,6 +213,9 @@ typedef struct irb {
     __u32 emw[8];
 }  __attribute__ ((packed, aligned(4))) Irb;
 
+int enable_mss_facility(void);
+void enable_subchannel(SubChannelId schid);
+
 /*
  * Some S390 specific IO instructions as inline
  */
diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index 2457752..1bc61b5 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -10,6 +10,7 @@
 
 #include "libc.h"
 #include "s390-ccw.h"
+#include "cio.h"
 #include "virtio.h"
 
 char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE)));
diff --git a/pc-bios/s390-ccw/netboot.mak b/pc-bios/s390-ccw/netboot.mak
index 14e96b2..5eefb7c 100644
--- a/pc-bios/s390-ccw/netboot.mak
+++ b/pc-bios/s390-ccw/netboot.mak
@@ -1,7 +1,7 @@
 
 SLOF_DIR := $(SRC_PATH)/roms/SLOF
 
-NETOBJS := start.o sclp.o virtio.o virtio-net.o jump2ipl.o netmain.o \
+NETOBJS := start.o sclp.o cio.o virtio.o virtio-net.o jump2ipl.o netmain.o \
 	   libnet.a libc.a
 
 LIBC_INC := -nostdinc -I$(SLOF_DIR)/lib/libc/include
diff --git a/pc-bios/s390-ccw/netmain.c b/pc-bios/s390-ccw/netmain.c
index 0392131..5189c0f 100644
--- a/pc-bios/s390-ccw/netmain.c
+++ b/pc-bios/s390-ccw/netmain.c
@@ -33,6 +33,7 @@
 #include <pxelinux.h>
 
 #include "s390-ccw.h"
+#include "cio.h"
 #include "virtio.h"
 
 #define DEFAULT_BOOT_RETRIES 10
diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h
index 241c6d0..b39ee5d 100644
--- a/pc-bios/s390-ccw/s390-ccw.h
+++ b/pc-bios/s390-ccw/s390-ccw.h
@@ -72,7 +72,6 @@ unsigned long virtio_load_direct(ulong rec_list1, ulong rec_list2,
 bool virtio_is_supported(SubChannelId schid);
 void virtio_blk_setup_device(SubChannelId schid);
 int virtio_read(ulong sector, void *load_addr);
-int enable_mss_facility(void);
 u64 get_clock(void);
 ulong get_second(void);
 
diff --git a/pc-bios/s390-ccw/virtio-blkdev.c b/pc-bios/s390-ccw/virtio-blkdev.c
index 11c5626..d2e7fcd 100644
--- a/pc-bios/s390-ccw/virtio-blkdev.c
+++ b/pc-bios/s390-ccw/virtio-blkdev.c
@@ -10,6 +10,7 @@
 
 #include "libc.h"
 #include "s390-ccw.h"
+#include "cio.h"
 #include "virtio.h"
 #include "virtio-scsi.h"
 
diff --git a/pc-bios/s390-ccw/virtio.c b/pc-bios/s390-ccw/virtio.c
index cdb66f4..aa9da72 100644
--- a/pc-bios/s390-ccw/virtio.c
+++ b/pc-bios/s390-ccw/virtio.c
@@ -10,6 +10,7 @@
 
 #include "libc.h"
 #include "s390-ccw.h"
+#include "cio.h"
 #include "virtio.h"
 #include "virtio-scsi.h"
 #include "bswap.h"
@@ -20,8 +21,6 @@ static VRing block[VIRTIO_MAX_VQS];
 static char ring_area[VIRTIO_RING_SIZE * VIRTIO_MAX_VQS]
                      __attribute__((__aligned__(PAGE_SIZE)));
 
-static char chsc_page[PAGE_SIZE] __attribute__((__aligned__(PAGE_SIZE)));
-
 static VDev vdev = {
     .nr_vqs = 1,
     .vrings = block,
@@ -94,14 +93,9 @@ static int run_ccw(VDev *vdev, int cmd, void *ptr, int len)
 {
     Ccw1 ccw = {};
     CmdOrb orb = {};
-    Schib schib;
     int r;
 
-    /* start command processing */
-    stsch_err(vdev->schid, &schib);
-    /* enable the subchannel for IPL device */
-    schib.pmcw.ena = 1;
-    msch(vdev->schid, &schib);
+    enable_subchannel(vdev->schid);
 
     /* start subchannel command */
     orb.fmt = 1;
@@ -343,20 +337,3 @@ bool virtio_is_supported(SubChannelId schid)
     }
     return false;
 }
-
-int enable_mss_facility(void)
-{
-    int ret;
-    ChscAreaSda *sda_area = (ChscAreaSda *) chsc_page;
-
-    memset(sda_area, 0, PAGE_SIZE);
-    sda_area->request.length = 0x0400;
-    sda_area->request.code = 0x0031;
-    sda_area->operation_code = 0x2;
-
-    ret = chsc(sda_area);
-    if ((ret == 0) && (sda_area->response.code == 0x0001)) {
-        return 0;
-    }
-    return -EIO;
-}
-- 
2.7.4

[Qemu-devel] [PATCH 08/15] s390-bios: Map low core memory

[Jason J. Herne]

Create a new header for basic architecture specific definitions and add a
mapping of low core memory. This mapping will be used by the real dasd boot
process.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/main.c      |   2 +
 pc-bios/s390-ccw/s390-arch.h | 100 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 102 insertions(+)
 create mode 100644 pc-bios/s390-ccw/s390-arch.h

diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index 1bc61b5..fa90aa3 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -9,6 +9,7 @@
  */
 
 #include "libc.h"
+#include "s390-arch.h"
 #include "s390-ccw.h"
 #include "cio.h"
 #include "virtio.h"
@@ -19,6 +20,7 @@ static char loadparm_str[LOADPARM_LEN + 1] = { 0, 0, 0, 0, 0, 0, 0, 0, 0 };
 QemuIplParameters qipl;
 IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));
 static bool have_iplb;
+const LowCore *lowcore; /* Yes, this *is* a pointer to address 0 */
 
 #define LOADPARM_PROMPT "PROMPT  "
 #define LOADPARM_EMPTY  "        "
diff --git a/pc-bios/s390-ccw/s390-arch.h b/pc-bios/s390-ccw/s390-arch.h
new file mode 100644
index 0000000..47eaa04
--- /dev/null
+++ b/pc-bios/s390-ccw/s390-arch.h
@@ -0,0 +1,100 @@
+/*
+ * S390 Basic Architecture
+ *
+ * Copyright (c) 2018 Jason J. Herne <jjherne@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef S390_ARCH_H
+#define S390_ARCH_H
+
+typedef struct PSW {
+    uint64_t mask;
+    uint64_t addr;
+} __attribute__ ((packed, aligned(8))) PSW;
+
+/* Older PSW format used by LPSW instruction */
+typedef struct PSWLegacy {
+    uint32_t mask;
+    uint32_t addr;
+} __attribute__ ((packed, aligned(8))) PSWLegacy;
+
+/* s390 psw bit masks */
+#define PSW_MASK_IOINT      0x0200000000000000ULL
+#define PSW_MASK_WAIT       0x0002000000000000ULL
+#define PSW_MASK_EAMODE     0x0000000100000000ULL
+#define PSW_MASK_BAMODE     0x0000000080000000ULL
+#define PSW_MASK_ZMODE      (PSW_MASK_EAMODE | PSW_MASK_BAMODE)
+
+/* Low core mapping */
+typedef struct LowCore {
+    /* prefix area: defined by architecture */
+    uint64_t        ipl_psw;                  /* 0x000 */
+    uint32_t        ccw1[2];                  /* 0x008 */
+    uint32_t        ccw2[2];                  /* 0x010 */
+    uint8_t         pad1[0x80 - 0x18];        /* 0x018 */
+    uint32_t        ext_params;               /* 0x080 */
+    uint16_t        cpu_addr;                 /* 0x084 */
+    uint16_t        ext_int_code;             /* 0x086 */
+    uint16_t        svc_ilen;                 /* 0x088 */
+    uint16_t        svc_code;                 /* 0x08a */
+    uint16_t        pgm_ilen;                 /* 0x08c */
+    uint16_t        pgm_code;                 /* 0x08e */
+    uint32_t        data_exc_code;            /* 0x090 */
+    uint16_t        mon_class_num;            /* 0x094 */
+    uint16_t        per_perc_atmid;           /* 0x096 */
+    uint64_t        per_address;              /* 0x098 */
+    uint8_t         exc_access_id;            /* 0x0a0 */
+    uint8_t         per_access_id;            /* 0x0a1 */
+    uint8_t         op_access_id;             /* 0x0a2 */
+    uint8_t         ar_access_id;             /* 0x0a3 */
+    uint8_t         pad2[0xA8 - 0xA4];        /* 0x0a4 */
+    uint64_t        trans_exc_code;           /* 0x0a8 */
+    uint64_t        monitor_code;             /* 0x0b0 */
+    uint16_t        subchannel_id;            /* 0x0b8 */
+    uint16_t        subchannel_nr;            /* 0x0ba */
+    uint32_t        io_int_parm;              /* 0x0bc */
+    uint32_t        io_int_word;              /* 0x0c0 */
+    uint8_t         pad3[0xc8 - 0xc4];        /* 0x0c4 */
+    uint32_t        stfl_fac_list;            /* 0x0c8 */
+    uint8_t         pad4[0xe8 - 0xcc];        /* 0x0cc */
+    uint64_t        mcic;                     /* 0x0e8 */
+    uint8_t         pad5[0xf4 - 0xf0];        /* 0x0f0 */
+    uint32_t        external_damage_code;     /* 0x0f4 */
+    uint64_t        failing_storage_address;  /* 0x0f8 */
+    uint8_t         pad6[0x110 - 0x100];      /* 0x100 */
+    uint64_t        per_breaking_event_addr;  /* 0x110 */
+    uint8_t         pad7[0x120 - 0x118];      /* 0x118 */
+    PSW             restart_old_psw;          /* 0x120 */
+    PSW             external_old_psw;         /* 0x130 */
+    PSW             svc_old_psw;              /* 0x140 */
+    PSW             program_old_psw;          /* 0x150 */
+    PSW             mcck_old_psw;             /* 0x160 */
+    PSW             io_old_psw;               /* 0x170 */
+    uint8_t         pad8[0x1a0 - 0x180];      /* 0x180 */
+    PSW             restart_new_psw;          /* 0x1a0 */
+    PSW             external_new_psw;         /* 0x1b0 */
+    PSW             svc_new_psw;              /* 0x1c0 */
+    PSW             program_new_psw;          /* 0x1d0 */
+    PSW             mcck_new_psw;             /* 0x1e0 */
+    PSW             io_new_psw;               /* 0x1f0 */
+    PSW             return_psw;               /* 0x200 */
+    uint8_t         irb[64];                  /* 0x210 */
+    uint64_t        sync_enter_timer;         /* 0x250 */
+    uint64_t        async_enter_timer;        /* 0x258 */
+    uint64_t        exit_timer;               /* 0x260 */
+    uint64_t        last_update_timer;        /* 0x268 */
+    uint64_t        user_timer;               /* 0x270 */
+    uint64_t        system_timer;             /* 0x278 */
+    uint64_t        last_update_clock;        /* 0x280 */
+    uint64_t        steal_clock;              /* 0x288 */
+    PSW             return_mcck_psw;          /* 0x290 */
+    uint8_t         pad9[0xc00 - 0x2a0];      /* 0x2a0 */
+} __attribute__((packed, aligned(8192))) LowCore;
+
+extern const LowCore *lowcore;
+
+#endif
-- 
2.7.4

[Qemu-devel] [PATCH 09/15] s390-bios: ptr2u32 and u32toptr

[Jason J. Herne]

Introduce inline functions to convert between pointers and unsigned 32-bit
ints. These are used to hide the ugliness required to  avoid compiler
warnings.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/libc.h | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/pc-bios/s390-ccw/libc.h b/pc-bios/s390-ccw/libc.h
index 818517f..e198f0b 100644
--- a/pc-bios/s390-ccw/libc.h
+++ b/pc-bios/s390-ccw/libc.h
@@ -19,6 +19,18 @@ typedef unsigned short     uint16_t;
 typedef unsigned int       uint32_t;
 typedef unsigned long long uint64_t;
 
+/* Avoids compiler warnings when casting a pointer to a u32 */
+static inline uint32_t ptr2u32(void *ptr)
+{
+    return (uint32_t)(uint64_t)ptr;
+}
+
+/* Avoids compiler warnings when casting a u32 to a pointer */
+static inline void *u32toptr(uint32_t n)
+{
+    return (void *)(uint64_t)n;
+}
+
 static inline void *memset(void *s, int c, size_t n)
 {
     size_t i;
-- 
2.7.4

[Qemu-devel] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

Add struct for format-0 ccws. Support executing format-0 channel
programs and waiting for their completion before continuing execution.
This will be used for real dasd ipl.

Add cu_type() to channel io library. This will be used to query control
unit type which is used to determine if we are booting a virtio device or a
real dasd device.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/cio.c      | 108 ++++++++++++++++++++++++++++++++++++++
 pc-bios/s390-ccw/cio.h      | 124 ++++++++++++++++++++++++++++++++++++++++++--
 pc-bios/s390-ccw/s390-ccw.h |   1 +
 pc-bios/s390-ccw/start.S    |  33 +++++++++++-
 4 files changed, 261 insertions(+), 5 deletions(-)

diff --git a/pc-bios/s390-ccw/cio.c b/pc-bios/s390-ccw/cio.c
index 095f79b..9019250 100644
--- a/pc-bios/s390-ccw/cio.c
+++ b/pc-bios/s390-ccw/cio.c
@@ -10,6 +10,7 @@
 
 #include "libc.h"
 #include "s390-ccw.h"
+#include "s390-arch.h"
 #include "cio.h"
 
 static char chsc_page[PAGE_SIZE] __attribute__((__aligned__(PAGE_SIZE)));
@@ -39,3 +40,110 @@ void enable_subchannel(SubChannelId schid)
     schib.pmcw.ena = 1;
     msch(schid, &schib);
 }
+
+uint16_t cu_type(SubChannelId schid)
+{
+    Ccw1 sense_id_ccw;
+    SenseId sense_data;
+
+    sense_id_ccw.cmd_code = CCW_CMD_SENSE_ID;
+    sense_id_ccw.cda = ptr2u32(&sense_data);
+    sense_id_ccw.count = sizeof(sense_data);
+
+    if (do_cio(schid, ptr2u32(&sense_id_ccw), CCW_FMT1)) {
+        panic("Failed to run SenseID CCw\n");
+    }
+
+    return sense_data.cu_type;
+}
+
+void basic_sense(SubChannelId schid, SenseData *sd)
+{
+    Ccw1 senseCcw;
+
+    senseCcw.cmd_code = CCW_CMD_BASIC_SENSE;
+    senseCcw.cda = ptr2u32(sd);
+    senseCcw.count = sizeof(*sd);
+
+    if (do_cio(schid, ptr2u32(&senseCcw), CCW_FMT1)) {
+        panic("Failed to run Basic Sense CCW\n");
+    }
+}
+
+static bool irb_error(Irb *irb)
+{
+    /* We have to ignore Incorrect Length (cstat == 0x40) indicators because
+     * real devices expect a 24 byte SenseID  buffer, and virtio devices expect
+     * a much larger buffer. Neither device type can tolerate a buffer size
+     * different from what they expect so they set this indicator.
+     */
+    if (irb->scsw.cstat != 0x00 && irb->scsw.cstat != 0x40) {
+        return true;
+    }
+    return irb->scsw.dstat != 0xc;
+}
+
+/* Executes a channel program at a given subchannel. The request to run the
+ * channel program is sent to the subchannel, we then wait for the interrupt
+ * singaling completion of the I/O operation(s) perfomed by the channel
+ * program. Lastly we verify that the i/o operation completed without error and
+ * that the interrupt we received was for the subchannel used to run the
+ * channel program.
+ *
+ * Note: This function assumes it is running in an environment where no other
+ * cpus are generating or receiving I/O interrupts. So either run it in a
+ * single-cpu environment or make sure all other cpus are not doing I/O and
+ * have I/O interrupts masked off.
+ */
+int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
+{
+    CmdOrb orb = {};
+    Irb irb = {};
+    SenseData sd;
+    int rc, retries = 0;
+
+    IPL_assert(fmt == 0 || fmt == 1, "Invalid ccw format");
+
+    /* ccw_addr must be <= 24 bits and point to at least one whole ccw. */
+    if (fmt == 0) {
+        IPL_assert(ccw_addr <= 0xFFFFFF - 8, "Invalid ccw address");
+    }
+
+    orb.fmt = fmt ;
+    orb.pfch = 1;  /* QEMU's cio implementation requires prefetch */
+    orb.c64 = 1;   /* QEMU's cio implementation requires 64-bit idaws */
+    orb.lpm = 0xFF; /* All paths allowed */
+    orb.cpa = ccw_addr;
+
+    while (true) {
+        rc = ssch(schid, &orb);
+        if (rc) {
+            print_int("ssch failed with rc=", rc);
+            break;
+        }
+
+        consume_io_int();
+
+        /* Clear read */
+        rc = tsch(schid, &irb);
+        if (rc) {
+            print_int("tsch failed with rc=", rc);
+            break;
+        }
+
+        if (!irb_error(&irb)) {
+            break;
+        }
+
+        /* Unexpected unit check. Use sense to clear unit check then retry. */
+        if (unit_check(&irb) && retries <= 2) {
+            basic_sense(schid, &sd);
+            retries++;
+            continue;
+        }
+
+        break;
+    }
+
+    return rc;
+}
diff --git a/pc-bios/s390-ccw/cio.h b/pc-bios/s390-ccw/cio.h
index 7b07d75..5c16635 100644
--- a/pc-bios/s390-ccw/cio.h
+++ b/pc-bios/s390-ccw/cio.h
@@ -70,9 +70,46 @@ struct scsw {
     __u16 count;
 } __attribute__ ((packed));
 
-#define SCSW_FCTL_CLEAR_FUNC 0x1000
-#define SCSW_FCTL_HALT_FUNC 0x2000
+/* Function Control */
 #define SCSW_FCTL_START_FUNC 0x4000
+#define SCSW_FCTL_HALT_FUNC 0x2000
+#define SCSW_FCTL_CLEAR_FUNC 0x1000
+
+/* Activity Control */
+#define SCSW_ACTL_RESUME_PEND   0x0800
+#define SCSW_ACTL_START_PEND    0x0400
+#define SCSW_ACTL_HALT_PEND     0x0200
+#define SCSW_ACTL_CLEAR_PEND    0x0100
+#define SCSW_ACTL_CH_ACTIVE     0x0080
+#define SCSW_ACTL_DEV_ACTIVE    0x0040
+#define SCSW_ACTL_SUSPENDED     0x0020
+
+/* Status Control */
+#define SCSW_SCTL_ALERT         0x0010
+#define SCSW_SCTL_INTERMED      0x0008
+#define SCSW_SCTL_PRIMARY       0x0004
+#define SCSW_SCTL_SECONDARY     0x0002
+#define SCSW_SCTL_STATUS_PEND   0x0001
+
+/* SCSW Device Status Flags */
+#define SCSW_DSTAT_ATTN     0x80
+#define SCSW_DSTAT_STATMOD  0x40
+#define SCSW_DSTAT_CUEND    0x20
+#define SCSW_DSTAT_BUSY     0x10
+#define SCSW_DSTAT_CHEND    0x08
+#define SCSW_DSTAT_DEVEND   0x04
+#define SCSW_DSTAT_UCHK     0x02
+#define SCSW_DSTAT_UEXCP    0x01
+
+/* SCSW Subchannel Status Flags */
+#define SCSW_CSTAT_PCINT    0x80
+#define SCSW_CSTAT_BADLEN   0x40
+#define SCSW_CSTAT_PROGCHK  0x20
+#define SCSW_CSTAT_PROTCHK  0x10
+#define SCSW_CSTAT_CHDCHK   0x08
+#define SCSW_CSTAT_CHCCHK   0x04
+#define SCSW_CSTAT_ICCHK    0x02
+#define SCSW_CSTAT_CHAINCHK 0x01
 
 /*
  * subchannel information block
@@ -127,7 +164,23 @@ struct tpi_info {
     __u32 reserved4  : 12;
 } __attribute__ ((packed, aligned(4)));
 
-/* channel command word (type 1) */
+/* channel command word (format 0) */
+typedef struct ccw0 {
+    __u8 cmd_code;
+    __u32 cda        : 24;
+    __u32 chainData  : 1;
+    __u32 chain      : 1;
+    __u32 sli        : 1;
+    __u32 skip       : 1;
+    __u32 pci        : 1;
+    __u32 ida        : 1;
+    __u32 suspend    : 1;
+    __u32 mida       : 1;
+    __u8 reserved;
+    __u16 count;
+} __attribute__ ((packed, aligned(8))) Ccw0;
+
+/* channel command word (format 1) */
 typedef struct ccw1 {
     __u8 cmd_code;
     __u8 flags;
@@ -135,6 +188,10 @@ typedef struct ccw1 {
     __u32 cda;
 } __attribute__ ((packed, aligned(8))) Ccw1;
 
+/* do_cio() CCW formats */
+#define CCW_FMT0                 0x00
+#define CCW_FMT1                 0x01
+
 #define CCW_FLAG_DC              0x80
 #define CCW_FLAG_CC              0x40
 #define CCW_FLAG_SLI             0x20
@@ -190,6 +247,9 @@ struct ciw {
     __u16 count;
 };
 
+#define CU_TYPE_VIRTIO          0x3832
+#define CU_TYPE_DASD            0x3990
+
 /*
  * sense-id response buffer layout
  */
@@ -205,6 +265,61 @@ typedef struct senseid {
     struct ciw ciw[62];
 }  __attribute__ ((packed, aligned(4))) SenseId;
 
+/* architected values for first sense byte */
+#define SNS0_CMD_REJECT         0x80
+#define SNS0_INTERVENTION_REQ   0x40
+#define SNS0_BUS_OUT_CHECK      0x20
+#define SNS0_EQUIPMENT_CHECK    0x10
+#define SNS0_DATA_CHECK         0x08
+#define SNS0_OVERRUN            0x04
+#define SNS0_INCOMPL_DOMAIN     0x01
+
+/* architectured values for second sense byte */
+#define SNS1_PERM_ERR           0x80
+#define SNS1_INV_TRACK_FORMAT   0x40
+#define SNS1_EOC                0x20
+#define SNS1_MESSAGE_TO_OPER    0x10
+#define SNS1_NO_REC_FOUND       0x08
+#define SNS1_FILE_PROTECTED     0x04
+#define SNS1_WRITE_INHIBITED    0x02
+#define SNS1_INPRECISE_END      0x01
+
+/* architectured values for third sense byte */
+#define SNS2_REQ_INH_WRITE      0x80
+#define SNS2_CORRECTABLE        0x40
+#define SNS2_FIRST_LOG_ERR      0x20
+#define SNS2_ENV_DATA_PRESENT   0x10
+#define SNS2_INPRECISE_END      0x04
+
+/* 24-byte Sense fmt/msg codes */
+#define SENSE24_FMT_PROG_SYS    0x0
+#define SENSE24_FMT_EQUIPMENT   0x2
+#define SENSE24_FMT_CONTROLLER  0x3
+#define SENSE24_FMT_MISC        0xF
+
+#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
+
+/* basic sense response buffer layout */
+typedef struct senseData {
+    uint8_t status[3];
+    uint8_t res_count;
+    uint8_t phys_drive_id;
+    uint8_t low_cyl_addr;
+    uint8_t head_high_cyl_addr;
+    uint8_t fmt_msg;
+    uint64_t fmt_dependent_info[2];
+    uint8_t reserved;
+    uint8_t program_action_code;
+    uint16_t config_info;
+    uint8_t mcode_hicyl;
+    uint8_t cyl_head_addr[3];
+}  __attribute__ ((packed, aligned(4))) SenseData;
+
+#define SENSE24_GET_FMT(sd)     (sd->fmt_msg & 0xF0 >> 4)
+#define SENSE24_GET_MSG(sd)     (sd->fmt_msg & 0x0F)
+
+#define unit_check(irb)     ((irb)->scsw.dstat & SCSW_DSTAT_UCHK)
+
 /* interruption response block */
 typedef struct irb {
     struct scsw scsw;
@@ -215,6 +330,9 @@ typedef struct irb {
 
 int enable_mss_facility(void);
 void enable_subchannel(SubChannelId schid);
+uint16_t cu_type(SubChannelId schid);
+void basic_sense(SubChannelId schid, SenseData *sd);
+int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt);
 
 /*
  * Some S390 specific IO instructions as inline
diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h
index b39ee5d..11bce7d 100644
--- a/pc-bios/s390-ccw/s390-ccw.h
+++ b/pc-bios/s390-ccw/s390-ccw.h
@@ -52,6 +52,7 @@ typedef unsigned long long __u64;
 /* start.s */
 void disabled_wait(void);
 void consume_sclp_int(void);
+void consume_io_int(void);
 
 /* main.c */
 void panic(const char *string);
diff --git a/pc-bios/s390-ccw/start.S b/pc-bios/s390-ccw/start.S
index eb8d024..a48c38f 100644
--- a/pc-bios/s390-ccw/start.S
+++ b/pc-bios/s390-ccw/start.S
@@ -65,12 +65,32 @@ consume_sclp_int:
         /* prepare external call handler */
         larl %r1, external_new_code
         stg %r1, 0x1b8
-        larl %r1, external_new_mask
+        larl %r1, int_new_mask
         mvc 0x1b0(8),0(%r1)
         /* load enabled wait PSW */
         larl %r1, enabled_wait_psw
         lpswe 0(%r1)
 
+/*
+ * void consume_io_int(void)
+ *
+ * eats one I/O interrupt
+ */
+        .globl consume_io_int
+consume_io_int:
+        /* enable I/O interrupts in cr0 */
+        stctg 6,6,0(15)
+        oi 4(15), 0xff
+        lctlg 6,6,0(15)
+        /* prepare external call handler */
+        larl %r1, io_new_code
+        stg %r1, 0x1f8
+        larl %r1, int_new_mask
+        mvc 0x1f0(8),0(%r1)
+        /* load enabled wait PSW */
+        larl %r1, enabled_wait_psw
+        lpswe 0(%r1)
+
 external_new_code:
         /* disable service interrupts in cr0 */
         stctg 0,0,0(15)
@@ -78,10 +98,19 @@ external_new_code:
         lctlg 0,0,0(15)
         br 14
 
+io_new_code:
+        /* disable I/O interrupts in cr6 */
+        stctg 6,6,0(15)
+        ni 4(15), 0x00
+        lctlg 6,6,0(15)
+        br 14
+
+
+
         .align  8
 disabled_wait_psw:
         .quad   0x0002000180000000,0x0000000000000000
 enabled_wait_psw:
         .quad   0x0302000180000000,0x0000000000000000
-external_new_mask:
+int_new_mask:
         .quad   0x0000000180000000
-- 
2.7.4

[Qemu-devel] [PATCH 11/15] s390-bios: cio error handling

[Jason J. Herne]

Add verbose error output for when unexpected i/o errors happen. This eases the
burden of debugging and reporting i/o errors. No error information is printed
in the success case, here is an example of what is output on error:

vfio-ccw device I/O error - Interrupt Response Block Data:
    Function Ctrl : [Start]
    Activity Ctrl : [Start-Pending]
    Status Ctrl : [Alert] [Primary] [Secondary] [Status-Pending]
    Device Status : [Unit-Check]
    Channel Status :
    cpa=: 0x0000000001e67098
    prev_ccw=: 0x0000000000000000
    this_ccw=: 0x0000000000000000

Sense Data (fmt 32-bytes):
    Sense Condition Flags : [Equipment-Check]
    Residual Count     =: 0x0000000000000000
    Phys Drive ID      =: 0x000000000000009e
    low cyl address    =: 0x0000000000000000
    head addr & hi cyl =: 0x0000000000000000
    format/message     =: 0x0000000000000008
    fmt-dependent[0-7] =: 0x0000000000000004
    fmt-dependent[8-15]=: 0xe561282305082fff
    prog action code   =: 0x0000000000000016
    Configuration info =: 0x00000000000040e0
    mcode / hi-cyl     =: 0x0000000000000000
    cyl & head addr [0]=: 0x0000000000000000
    cyl & head addr [1]=: 0x0000000000000000
    cyl & head addr [2]=: 0x0000000000000000

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/cio.c  | 225 ++++++++++++++++++++++++++++++++++++++++++++++++
 pc-bios/s390-ccw/libc.h |  11 +++
 2 files changed, 236 insertions(+)

diff --git a/pc-bios/s390-ccw/cio.c b/pc-bios/s390-ccw/cio.c
index 9019250..c72e13c 100644
--- a/pc-bios/s390-ccw/cio.c
+++ b/pc-bios/s390-ccw/cio.c
@@ -83,6 +83,228 @@ static bool irb_error(Irb *irb)
     return irb->scsw.dstat != 0xc;
 }
 
+static void print_sense_data(SenseData *sd)
+{
+    char msgline[512];
+
+    if (sd->config_info & 0x8000) {
+        sclp_print("Sense Data (fmt 24-bytes):\n");
+    } else {
+        sclp_print("Sense Data (fmt 32-bytes):\n");
+    }
+
+    strcat(msgline, "    Sense Condition Flags :");
+    if (sd->status[0] & SNS0_CMD_REJECT) {
+        strcat(msgline, " [Cmd-Reject]");
+    }
+    if (sd->status[0] & SNS0_INTERVENTION_REQ) {
+        strcat(msgline, " [Intervention-Required]");
+    }
+    if (sd->status[0] & SNS0_BUS_OUT_CHECK) {
+        strcat(msgline, " [Bus-Out-Parity-Check]");
+    }
+    if (sd->status[0] & SNS0_EQUIPMENT_CHECK) {
+        strcat(msgline, " [Equipment-Check]");
+    }
+    if (sd->status[0] & SNS0_DATA_CHECK) {
+        strcat(msgline, " [Data-Check]");
+    }
+    if (sd->status[0] & SNS0_OVERRUN) {
+        strcat(msgline, " [Overrun]");
+    }
+    if (sd->status[0] & SNS0_INCOMPL_DOMAIN) {
+        strcat(msgline, " [Incomplete-Domain]");
+    }
+
+    if (sd->status[1] & SNS1_PERM_ERR) {
+        strcat(msgline, " [Permanent-Error]");
+    }
+    if (sd->status[1] & SNS1_INV_TRACK_FORMAT) {
+        strcat(msgline, " [Invalid-Track-Fmt]");
+    }
+    if (sd->status[1] & SNS1_EOC) {
+        strcat(msgline, " [End-of-Cyl]");
+    }
+    if (sd->status[1] & SNS1_MESSAGE_TO_OPER) {
+        strcat(msgline, " [Operator-Msg]");
+    }
+    if (sd->status[1] & SNS1_NO_REC_FOUND) {
+        strcat(msgline, " [No-Record-Found]");
+    }
+    if (sd->status[1] & SNS1_FILE_PROTECTED) {
+        strcat(msgline, " [File-Protected]");
+    }
+    if (sd->status[1] & SNS1_WRITE_INHIBITED) {
+        strcat(msgline, " [Write-Inhibited]");
+    }
+    if (sd->status[1] & SNS1_INPRECISE_END) {
+        strcat(msgline, " [Imprecise-Ending]");
+    }
+
+    if (sd->status[2] & SNS2_REQ_INH_WRITE) {
+        strcat(msgline, " [Req-Inhibit-Write]");
+    }
+    if (sd->status[2] & SNS2_CORRECTABLE) {
+        strcat(msgline, " [Correctable-Data-Check]");
+    }
+    if (sd->status[2] & SNS2_FIRST_LOG_ERR) {
+        strcat(msgline, " [First-Error-Log]");
+    }
+    if (sd->status[2] & SNS2_ENV_DATA_PRESENT) {
+        strcat(msgline, " [Env-Data-Present]");
+    }
+    if (sd->status[2] & SNS2_INPRECISE_END) {
+        strcat(msgline, " [Imprecise-End]");
+    }
+    strcat(msgline, "\n");
+    sclp_print(msgline);
+
+    print_int("    Residual Count     =", sd->res_count);
+    print_int("    Phys Drive ID      =", sd->phys_drive_id);
+    print_int("    low cyl address    =", sd->low_cyl_addr);
+    print_int("    head addr & hi cyl =", sd->head_high_cyl_addr);
+    print_int("    format/message     =", sd->fmt_msg);
+    print_int("    fmt-dependent[0-7] =", sd->fmt_dependent_info[0]);
+    print_int("    fmt-dependent[8-15]=", sd->fmt_dependent_info[1]);
+    print_int("    prog action code   =", sd->program_action_code);
+    print_int("    Configuration info =", sd->config_info);
+    print_int("    mcode / hi-cyl     =", sd->mcode_hicyl);
+    print_int("    cyl & head addr [0]=", sd->cyl_head_addr[0]);
+    print_int("    cyl & head addr [1]=", sd->cyl_head_addr[1]);
+    print_int("    cyl & head addr [2]=", sd->cyl_head_addr[2]);
+}
+
+static void print_irb_err(Irb *irb)
+{
+    Ccw0 *this_ccw = u32toptr(irb->scsw.cpa);
+    Ccw0 *prev_ccw = u32toptr(irb->scsw.cpa - 8);
+    char msgline[256];
+
+    sclp_print("vfio-ccw device I/O error - Interrupt Response Block Data:\n");
+
+    strcat(msgline, "    Function Ctrl :");
+    if (irb->scsw.ctrl & SCSW_FCTL_START_FUNC) {
+        strcat(msgline, " [Start]");
+    }
+    if (irb->scsw.ctrl & SCSW_FCTL_HALT_FUNC) {
+        strcat(msgline, " [Halt]");
+    }
+    if (irb->scsw.ctrl & SCSW_FCTL_CLEAR_FUNC) {
+        strcat(msgline, " [Clear]");
+    }
+    strcat(msgline, "\n");
+    sclp_print(msgline);
+
+    msgline[0] = '\0';
+    strcat(msgline, "    Activity Ctrl :");
+    if (irb->scsw.ctrl & SCSW_ACTL_RESUME_PEND) {
+        strcat(msgline, " [Resume-Pending]");
+    }
+    if (irb->scsw.ctrl & SCSW_ACTL_START_PEND) {
+        strcat(msgline, " [Start-Pending]");
+    }
+    if (irb->scsw.ctrl & SCSW_ACTL_HALT_PEND) {
+        strcat(msgline, " [Halt-Pending]");
+    }
+    if (irb->scsw.ctrl & SCSW_ACTL_CLEAR_PEND) {
+        strcat(msgline, " [Clear-Pending]");
+    }
+    if (irb->scsw.ctrl & SCSW_ACTL_CH_ACTIVE) {
+        strcat(msgline, " [Channel-Active]");
+    }
+    if (irb->scsw.ctrl & SCSW_ACTL_DEV_ACTIVE) {
+        strcat(msgline, " [Device-Active]");
+    }
+    if (irb->scsw.ctrl & SCSW_ACTL_SUSPENDED) {
+        strcat(msgline, " [Suspended]");
+    }
+    strcat(msgline, "\n");
+    sclp_print(msgline);
+
+    msgline[0] = '\0';
+    strcat(msgline, "    Status Ctrl :");
+    if (irb->scsw.ctrl & SCSW_SCTL_ALERT) {
+        strcat(msgline, " [Alert]");
+    }
+    if (irb->scsw.ctrl & SCSW_SCTL_INTERMED) {
+        strcat(msgline, " [Intermediate]");
+    }
+    if (irb->scsw.ctrl & SCSW_SCTL_PRIMARY) {
+        strcat(msgline, " [Primary]");
+    }
+    if (irb->scsw.ctrl & SCSW_SCTL_SECONDARY) {
+        strcat(msgline, " [Secondary]");
+    }
+    if (irb->scsw.ctrl & SCSW_SCTL_STATUS_PEND) {
+        strcat(msgline, " [Status-Pending]");
+    }
+
+    strcat(msgline, "\n");
+    sclp_print(msgline);
+
+    msgline[0] = '\0';
+    strcat(msgline, "    Device Status :");
+    if (irb->scsw.dstat & SCSW_DSTAT_ATTN) {
+        strcat(msgline, " [Attention]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_STATMOD) {
+        strcat(msgline, " [Status-Modifier]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_CUEND) {
+        strcat(msgline, " [Ctrl-Unit-End]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_BUSY) {
+        strcat(msgline, " [Busy]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_CHEND) {
+        strcat(msgline, " [Channel-End]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_DEVEND) {
+        strcat(msgline, " [Device-End]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_UCHK) {
+        strcat(msgline, " [Unit-Check]");
+    }
+    if (irb->scsw.dstat & SCSW_DSTAT_UEXCP) {
+        strcat(msgline, " [Unit-Exception]");
+    }
+    strcat(msgline, "\n");
+    sclp_print(msgline);
+
+    msgline[0] = '\0';
+    strcat(msgline, "    Channel Status :");
+    if (irb->scsw.cstat & SCSW_CSTAT_PCINT) {
+        strcat(msgline, " [Program-Ctrl-Interruption]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_BADLEN) {
+        strcat(msgline, " [Incorrect-Length]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_PROGCHK) {
+        strcat(msgline, " [Program-Check]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_PROTCHK) {
+        strcat(msgline, " [Protection-Check]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_CHDCHK) {
+        strcat(msgline, " [Channel-Data-Check]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_CHCCHK) {
+        strcat(msgline, " [Channel-Ctrl-Check]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_ICCHK) {
+        strcat(msgline, " [Interface-Ctrl-Check]");
+    }
+    if (irb->scsw.cstat & SCSW_CSTAT_CHAINCHK) {
+        strcat(msgline, " [Chaining-Check]");
+    }
+    strcat(msgline, "\n");
+    sclp_print(msgline);
+
+    print_int("    cpa=", irb->scsw.cpa);
+    print_int("    prev_ccw=", *((uint64_t *)prev_ccw));
+    print_int("    this_ccw=", *((uint64_t *)this_ccw));
+}
+
 /* Executes a channel program at a given subchannel. The request to run the
  * channel program is sent to the subchannel, we then wait for the interrupt
  * singaling completion of the I/O operation(s) perfomed by the channel
@@ -142,6 +364,9 @@ int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
             continue;
         }
 
+        print_irb_err(&irb);
+        basic_sense(schid, &sd);
+        print_sense_data(&sd);
         break;
     }
 
diff --git a/pc-bios/s390-ccw/libc.h b/pc-bios/s390-ccw/libc.h
index e198f0b..01b5de0 100644
--- a/pc-bios/s390-ccw/libc.h
+++ b/pc-bios/s390-ccw/libc.h
@@ -79,6 +79,17 @@ static inline size_t strlen(const char *str)
     return i;
 }
 
+static inline char *strcat(char *dest, const char *src)
+{
+    int i;
+    char *dest_end = dest + strlen(dest);
+
+    for (i = 0; i <= strlen(src); i++) {
+        dest_end[i] = src[i];
+    }
+    return dest;
+}
+
 static inline int isdigit(int c)
 {
     return (c >= '0') && (c <= '9');
-- 
2.7.4

[Qemu-devel] [PATCH 12/15] s390-bios: Refactor virtio to run channel programs via cio

[Jason J. Herne]

Now that we have a Channel I/O library let's modify virtio boot code to
make use of it for running channel programs.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/virtio.c | 21 ++-------------------
 1 file changed, 2 insertions(+), 19 deletions(-)

diff --git a/pc-bios/s390-ccw/virtio.c b/pc-bios/s390-ccw/virtio.c
index aa9da72..5fb49e2 100644
--- a/pc-bios/s390-ccw/virtio.c
+++ b/pc-bios/s390-ccw/virtio.c
@@ -92,30 +92,13 @@ int drain_irqs(SubChannelId schid)
 static int run_ccw(VDev *vdev, int cmd, void *ptr, int len)
 {
     Ccw1 ccw = {};
-    CmdOrb orb = {};
-    int r;
-
-    enable_subchannel(vdev->schid);
-
-    /* start subchannel command */
-    orb.fmt = 1;
-    orb.cpa = (u32)(long)&ccw;
-    orb.lpm = 0x80;
 
     ccw.cmd_code = cmd;
     ccw.cda = (long)ptr;
     ccw.count = len;
 
-    r = ssch(vdev->schid, &orb);
-    /*
-     * XXX Wait until device is done processing the CCW. For now we can
-     *     assume that a simple tsch will have finished the CCW processing,
-     *     but the architecture allows for asynchronous operation
-     */
-    if (!r) {
-        r = drain_irqs(vdev->schid);
-    }
-    return r;
+    enable_subchannel(vdev->schid);
+    return do_cio(vdev->schid, ptr2u32(&ccw), CCW_FMT1);
 }
 
 static void vring_init(VRing *vr, VqInfo *info)
-- 
2.7.4

[Qemu-devel] [PATCH 13/15] s390-bios: Use control unit type to determine boot method

[Jason J. Herne]

The boot method is different depending on which device type we are
booting from. Let's examine the control unit type to determine if we're
a virtio device. We'll eventually add a case to check for a real dasd device
here as well.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/main.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index fa90aa3..5ee02c3 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -200,13 +200,24 @@ static void virtio_setup(void)
 
 int main(void)
 {
+    uint16_t cutype;
+
     sclp_setup();
     css_setup();
     boot_setup();
     find_boot_device();
+    enable_subchannel(blk_schid);
 
-    virtio_setup();
-    zipl_load(); /* no return */
+    cutype = cu_type(blk_schid) ;
+    switch (cutype) {
+    case CU_TYPE_VIRTIO:
+        virtio_setup();
+        zipl_load(); /* no return */
+        break;
+    default:
+        print_int("Attempting to boot from unexpected device type", cutype);
+        panic("");
+    }
 
     panic("Failed to load OS from hard disk\n");
     return 0; /* make compiler happy */
-- 
2.7.4

[Qemu-devel] [PATCH 14/15] s390-bios: Add channel command codes/structs needed for dasd-ipl

[Jason J. Herne]

The dasd IPL procedure needs to execute a few previously unused
channel commands. Let's define them and their associated data
structures.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 pc-bios/s390-ccw/cio.h | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/pc-bios/s390-ccw/cio.h b/pc-bios/s390-ccw/cio.h
index 5c16635..3300295 100644
--- a/pc-bios/s390-ccw/cio.h
+++ b/pc-bios/s390-ccw/cio.h
@@ -200,11 +200,14 @@ typedef struct ccw1 {
 #define CCW_FLAG_IDA             0x04
 #define CCW_FLAG_SUSPEND         0x02
 
+/* Common CCW commands */
+#define CCW_CMD_READ_IPL         0x02
 #define CCW_CMD_NOOP             0x03
 #define CCW_CMD_BASIC_SENSE      0x04
 #define CCW_CMD_TIC              0x08
 #define CCW_CMD_SENSE_ID         0xe4
 
+/* Virtio CCW commands */
 #define CCW_CMD_SET_VQ           0x13
 #define CCW_CMD_VDEV_RESET       0x33
 #define CCW_CMD_READ_FEAT        0x12
@@ -216,6 +219,12 @@ typedef struct ccw1 {
 #define CCW_CMD_SET_CONF_IND     0x53
 #define CCW_CMD_READ_VQ_CONF     0x32
 
+/* DASD CCW commands */
+#define CCW_CMD_DASD_READ             0x06
+#define CCW_CMD_DASD_SEEK             0x07
+#define CCW_CMD_DASD_SEARCH_ID_EQ     0x31
+#define CCW_CMD_DASD_READ_MT          0x86
+
 /*
  * Command-mode operation request block
  */
@@ -328,6 +337,20 @@ typedef struct irb {
     __u32 emw[8];
 }  __attribute__ ((packed, aligned(4))) Irb;
 
+/* Used for SEEK ccw commands */
+typedef struct CcwSeekData {
+    uint16_t reserved;
+    uint16_t cyl;
+    uint16_t head;
+} __attribute__((packed)) CcwSeekData;
+
+/* Used for SEARCH ID ccw commands */
+typedef struct CcwSearchIdData {
+    uint16_t cyl;
+    uint16_t head;
+    uint8_t record;
+} __attribute__((packed)) CcwSearchIdData;
+
 int enable_mss_facility(void);
 void enable_subchannel(SubChannelId schid);
 uint16_t cu_type(SubChannelId schid);
-- 
2.7.4

[Qemu-devel] [PATCH 15/15] s390-bios: Support booting from real dasd device

[Jason J. Herne]

Allows guest to boot from a vfio configured real dasd device.

Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
---
 docs/devel/s390-dasd-ipl.txt | 132 +++++++++++++++++++++++
 pc-bios/s390-ccw/Makefile    |   2 +-
 pc-bios/s390-ccw/dasd-ipl.c  | 249 +++++++++++++++++++++++++++++++++++++++++++
 pc-bios/s390-ccw/dasd-ipl.h  |  16 +++
 pc-bios/s390-ccw/main.c      |   4 +
 pc-bios/s390-ccw/s390-arch.h |  13 +++
 6 files changed, 415 insertions(+), 1 deletion(-)
 create mode 100644 docs/devel/s390-dasd-ipl.txt
 create mode 100644 pc-bios/s390-ccw/dasd-ipl.c
 create mode 100644 pc-bios/s390-ccw/dasd-ipl.h

diff --git a/docs/devel/s390-dasd-ipl.txt b/docs/devel/s390-dasd-ipl.txt
new file mode 100644
index 0000000..84ec7b8
--- /dev/null
+++ b/docs/devel/s390-dasd-ipl.txt
@@ -0,0 +1,132 @@
+*****************************
+***** s390 hardware IPL *****
+*****************************
+
+The s390 hardware IPL process consists of the following steps.
+
+1. A READ IPL ccw is constructed in memory location 0x0.
+    This ccw, by definition, reads the IPL1 record which is located on the disk
+    at cylinder 0 track 0 record 1. Note that the chain flag is on in this ccw
+    so when it is complete another ccw will be fetched and executed from memory
+    location 0x08.
+
+2. Execute the Read IPL ccw at 0x00, thereby reading IPL1 data into 0x00.
+    IPL1 data is 24 bytes in length and consists of the following pieces of
+    information: [psw][read ccw][tic ccw]. When the machine executes the Read
+    IPL ccw it read the 24-bytes of IPL1 to be read into memory starting at
+    location 0x0. Then the ccw program at 0x08 which consists of a read
+    ccw and a tic ccw is automatically executed because of the chain flag from
+    the original READ IPL ccw. The read ccw will read the IPL2 data into memory
+    and the TIC (Tranfer In Channel) will transfer control to the channel
+    program contained in the IPL2 data. The TIC channel command is the
+    equivalent of a branch/jump/goto instruction for channel programs.
+    NOTE: The ccws in IPL1 are defined by the architecture to be format 0.
+
+3. Execute IPL2.
+    The TIC ccw instruction at the end of the IPL1 channel program will begin
+    the execution of the IPL2 channel program. IPL2 is stage-2 of the boot
+    process and will contain a larger channel program than IPL1. The point of
+    IPL2 is to find and load either the operating system or a small program that
+    loads the operating system from disk. At the end of this step all or some of
+    the real operating system is loaded into memory and we are ready to hand
+    control over to the guest operating system. At this point the guest
+    operating system is entirely responsible for loading any more data it might
+    need to function. NOTE: The IPL2 channel program might read data into memory
+    location 0 thereby overwriting the IPL1 psw and channel program. This is ok
+    as long as the data placed in location 0 contains a psw whose instruction
+    address points to the guest operating system code to execute at the end of
+    the IPL/boot process.
+    NOTE: The ccws in IPL2 are defined by the architecture to be format 0.
+
+4. Start executing the guest operating system.
+    The psw that was loaded into memory location 0 as part of the ipl process
+    should contain the needed flags for the operating system we have loaded. The
+    psw's instruction address will point to the location in memory where we want
+    to start executing the operating system. This psw is loaded (via LPSW
+    instruction) causing control to be passed to the operating system code.
+
+In a non-virtualized environment this process, handled entirely by the hardware,
+is kicked off by the user initiating a "Load" procedure from the hardware
+management console. This "Load" procedure crafts a special "Read IPL" ccw in
+memory location 0x0 that reads IPL1. It then executes this ccw thereby kicking
+off the reading of IPL1 data. Since the channel program from IPL1 will be
+written immediately after the special "Read IPL" ccw, the IPL1 channel program
+will be executed immediately (the special read ccw has the chaining bit turned
+on). The TIC at the end of the IPL1 channel program will cause the IPL2 channel
+program to be executed automatically. After this sequence completes the "Load"
+procedure then loads the psw from 0x0.
+
+*****************************************
+***** How this all pertains to Qemu *****
+*****************************************
+
+In theory we should merely have to do the following to IPL/boot a guest
+operating system from a DASD device:
+
+1. Place a "Read IPL" ccw into memory location 0x0 with chaining bit on.
+2. Execute channel program at 0x0.
+3. LPSW 0x0.
+
+However, our emulation of the machine's channel program logic is missing one key
+feature that is required for this process to work: non-prefetch of ccw data.
+
+When we start a channel program we pass the channel subsystem parameters via an
+ORB (Operation Request Block). One of those parameters is a prefetch bit. If the
+bit is on then Qemu is allowed to read the entire channel program from guest
+memory before it starts executing it. This means that any channel commands that
+read additional channel commands will not work as expected because the newly
+read commands will only exist in guest memory and NOT within Qemu's channel
+subsystem memory. Qemu's channel subsystem's implementation currently requires
+this bit to be on for all channel programs. This is a problem because the IPL
+process consists of transferring control from the "Read IPL" ccw immediately to
+the IPL1 channel program that was read by "Read IPL".
+
+Not being able to turn off prefetch will also prevent the TIC at the end of the
+IPL1 channel program from transferring control to the IPL2 channel program.
+
+Lastly, in some cases (the zipl bootloader for example) the IPL2 program also
+tansfers control to another channel program segment immediately after reading it
+from the disk. So we need to be able to handle this case.
+
+**************************
+***** What Qemu does *****
+**************************
+
+Since we are forced to live with prefetch we cannot use the very simple IPL
+procedure we defined in the preceding section. So we compensate by doing the
+following.
+
+1. Place "Read IPL" ccw into memory location 0x0, but turn off chaining bit.
+2. Execute "Read IPL" at 0x0.
+
+   So now IPL1's psw is at 0x0 and IPL1's channel program is at 0x08.
+
+4. Write a custom channel program that will seek to the IPL2 record and then
+   execute the READ and TIC ccws from IPL1.  Normamly the seek is not required
+   because after reading the IPL1 record the disk is automatically positioned
+   to read the very next record which will be IPL2. But since we are not reading
+   both IPL1 and IPL2 as part of the same channel program we must manually set
+   the position.
+
+5. Grab the target address of the TIC instruction from the IPL1 channel program.
+   This address is where the IPL2 channel program starts.
+
+   Now IPL2 is loaded into memory somewhere, and we know the address.
+
+6. Execute the IPL2 channel program at the address obtained in step #5.
+
+   Because this channel program can be dynamic, we must use a special algorithm
+   that detects a READ immediately followed by a TIC and breaks the ccw chain
+   by turning off the chain bit in the READ ccw. When control is returned from
+   the kernel/hardware to the Qemu bios code we immediately issue another start
+   subchannel to execute the remaining TIC instruction. This causes the entire
+   channel program (starting from the TIC) and all needed data to be refetched
+   thereby stepping around the limitation that would otherwise prevent this
+   channel program from executing properly.
+
+   Now the operating system code is loaded somewhere in guest memory and the psw
+   in memory location 0x0 will point to entry code for the guest operating
+   system.
+
+7. LPSW 0x0.
+   LPSW transfers control to the guest operating system and we're done.
diff --git a/pc-bios/s390-ccw/Makefile b/pc-bios/s390-ccw/Makefile
index 12ad9c1..a048b6b 100644
--- a/pc-bios/s390-ccw/Makefile
+++ b/pc-bios/s390-ccw/Makefile
@@ -10,7 +10,7 @@ $(call set-vpath, $(SRC_PATH)/pc-bios/s390-ccw)
 .PHONY : all clean build-all
 
 OBJECTS = start.o main.o bootmap.o jump2ipl.o sclp.o menu.o \
-	  virtio.o virtio-scsi.o virtio-blkdev.o libc.o cio.o
+	  virtio.o virtio-scsi.o virtio-blkdev.o libc.o cio.o dasd-ipl.o
 
 QEMU_CFLAGS := $(filter -W%, $(QEMU_CFLAGS))
 QEMU_CFLAGS += -ffreestanding -fno-delete-null-pointer-checks -msoft-float
diff --git a/pc-bios/s390-ccw/dasd-ipl.c b/pc-bios/s390-ccw/dasd-ipl.c
new file mode 100644
index 0000000..b7ce6d9
--- /dev/null
+++ b/pc-bios/s390-ccw/dasd-ipl.c
@@ -0,0 +1,249 @@
+/*
+ * S390 IPL (boot) from a real DASD device via vfio framework.
+ *
+ * Copyright (c) 2018 Jason J. Herne <jjherne@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#include "libc.h"
+#include "s390-ccw.h"
+#include "s390-arch.h"
+#include "dasd-ipl.h"
+
+static char prefix_page[PAGE_SIZE * 2]
+            __attribute__((__aligned__(PAGE_SIZE * 2)));
+
+static void enable_prefixing(void)
+{
+    memcpy(&prefix_page, (void *)0, 4096);
+    set_prefix(ptr2u32(&prefix_page));
+}
+
+static void disable_prefixing(void)
+{
+    set_prefix(0);
+    /* Copy io interrupt info back to low core */
+    memcpy((void *)0xB8, prefix_page + 0xB8, 12);
+}
+
+static bool is_read_tic_ccw_chain(Ccw0 *ccw)
+{
+    Ccw0 *next_ccw = ccw + 1;
+
+    return ((ccw->cmd_code == CCW_CMD_DASD_READ ||
+            ccw->cmd_code == CCW_CMD_DASD_READ_MT) &&
+            ccw->chain && next_ccw->cmd_code == CCW_CMD_TIC);
+}
+
+static bool dynamic_cp_fixup(uint32_t ccw_addr, uint32_t  *next_cpa)
+{
+    Ccw0 *cur_ccw = (Ccw0 *)(uint64_t)ccw_addr;
+    Ccw0 *tic_ccw;
+
+    while (true) {
+        /* Skip over inline TIC (it might not have the chain bit on)  */
+        if (cur_ccw->cmd_code == CCW_CMD_TIC &&
+            cur_ccw->cda == ptr2u32(cur_ccw) - 8) {
+            cur_ccw += 1;
+            continue;
+        }
+
+        if (!cur_ccw->chain) {
+            break;
+        }
+        if (is_read_tic_ccw_chain(cur_ccw)) {
+            /*
+             * Breaking a chain of CCWs may alter the semantics or even the
+             * validity of a channel program. The heuristic implemented below
+             * seems to work well in practice for the channel programs
+             * generated by zipl.
+             */
+            tic_ccw = cur_ccw + 1;
+            *next_cpa = tic_ccw->cda;
+            cur_ccw->chain = 0;
+            return true;
+        }
+        cur_ccw += 1;
+    }
+    return false;
+}
+
+static int run_dynamic_ccw_program(SubChannelId schid, uint32_t cpa)
+{
+    bool has_next;
+    uint32_t next_cpa = 0;
+    int rc;
+
+    do {
+        has_next = dynamic_cp_fixup(cpa, &next_cpa);
+
+        print_int("executing ccw chain at ", cpa);
+        enable_prefixing();
+        rc = do_cio(schid, cpa, CCW_FMT0);
+        disable_prefixing();
+
+        if (rc) {
+            break;
+        }
+        cpa = next_cpa;
+    } while (has_next);
+
+    return rc;
+}
+
+
+static void make_readipl(void)
+{
+    Ccw0 *ccwIplRead = (Ccw0 *)0x00;
+
+    /* Create Read IPL ccw at address 0 */
+    ccwIplRead->cmd_code = CCW_CMD_READ_IPL;
+    ccwIplRead->cda = 0x00; /* Read into address 0x00 in main memory */
+    ccwIplRead->chain = 0; /* Chain flag */
+    ccwIplRead->count = 0x18; /* Read 0x18 bytes of data */
+}
+
+static void run_readipl(SubChannelId schid)
+{
+    if (do_cio(schid, 0x00, CCW_FMT0)) {
+        panic("dasd-ipl: Failed to run Read IPL channel program");
+    }
+}
+
+/*
+ * The architecture states that IPL1 data should consist of a psw followed by
+ * format-0 READ and TIC CCWs. Let's sanity check.
+ */
+static void check_ipl1(void)
+{
+    Ccw0 *ccwread = (Ccw0 *)0x08;
+    Ccw0 *ccwtic = (Ccw0 *)0x10;
+
+    if (ccwread->cmd_code != CCW_CMD_DASD_READ ||
+        ccwtic->cmd_code != CCW_CMD_TIC) {
+        panic("dasd-ipl: IPL1 data invalid. Is this disk really bootable?\n");
+    }
+}
+
+static void check_ipl2(uint32_t ipl2_addr)
+{
+    Ccw0 *ccw = u32toptr(ipl2_addr);
+
+    if (ipl2_addr == 0x00) {
+        panic("IPL2 address invalid. Is this disk really bootable?\n");
+    }
+    if (ccw->cmd_code == 0x00) {
+        panic("IPL2 ccw data invalid. Is this disk really bootable?\n");
+    }
+}
+
+static uint32_t read_ipl2_addr(void)
+{
+    Ccw0 *ccwtic = (Ccw0 *)0x10;
+
+    return ccwtic->cda;
+}
+
+static void ipl1_fixup(void)
+{
+    Ccw0 *ccwSeek = (Ccw0 *) 0x08;
+    Ccw0 *ccwSearchID = (Ccw0 *) 0x10;
+    Ccw0 *ccwSearchTic = (Ccw0 *) 0x18;
+    Ccw0 *ccwRead = (Ccw0 *) 0x20;
+    CcwSeekData *seekData = (CcwSeekData *) 0x30;
+    CcwSearchIdData *searchData = (CcwSearchIdData *) 0x38;
+
+    /* move IPL1 CCWs to make room for CCWs needed to locate record 2 */
+    memcpy(ccwRead, (void *)0x08, 16);
+
+    /* Disable chaining so we don't TIC to IPL2 channel program */
+    ccwRead->chain = 0x00;
+
+    ccwSeek->cmd_code = CCW_CMD_DASD_SEEK;
+    ccwSeek->cda = ptr2u32(seekData);
+    ccwSeek->chain = 1;
+    ccwSeek->count = sizeof(seekData);
+    seekData->reserved = 0x00;
+    seekData->cyl = 0x00;
+    seekData->head = 0x00;
+
+    ccwSearchID->cmd_code = CCW_CMD_DASD_SEARCH_ID_EQ;
+    ccwSearchID->cda = ptr2u32(searchData);
+    ccwSearchID->chain = 1;
+    ccwSearchID->count = sizeof(searchData);
+    searchData->cyl = 0;
+    searchData->head = 0;
+    searchData->record = 2;
+
+    /* Go back to Search CCW if correct record not yet found */
+    ccwSearchTic->cmd_code = CCW_CMD_TIC;
+    ccwSearchTic->cda = ptr2u32(ccwSearchID);
+}
+
+static void run_ipl1(SubChannelId schid)
+ {
+    uint32_t startAddr = 0x08;
+
+    if (do_cio(schid, startAddr, CCW_FMT0)) {
+        panic("dasd-ipl: Failed to run IPL1 channel program");
+    }
+}
+
+static void run_ipl2(SubChannelId schid, uint32_t addr)
+{
+
+    if (run_dynamic_ccw_program(schid, addr)) {
+        panic("dasd-ipl: Failed to run IPL2 channel program");
+    }
+}
+
+static void lpsw(void *psw_addr)
+{
+    PSWLegacy *pswl = (PSWLegacy *) psw_addr;
+
+    pswl->mask |= PSW_MASK_EAMODE;   /* Force z-mode */
+    pswl->addr |= PSW_MASK_BAMODE;
+    asm volatile("  llgtr 0,0\n llgtr 1,1\n"     /* Some OS's expect to be */
+                 "  llgtr 2,2\n llgtr 3,3\n"     /* in 32-bit mode. Clear  */
+                 "  llgtr 4,4\n llgtr 5,5\n"     /* high part of regs to   */
+                 "  llgtr 6,6\n llgtr 7,7\n"     /* avoid messing up       */
+                 "  llgtr 8,8\n llgtr 9,9\n"     /* instructions that work */
+                 "  llgtr 10,10\n llgtr 11,11\n" /* in both addressing     */
+                 "  llgtr 12,12\n llgtr 13,13\n" /* modes, like servc.     */
+                 "  llgtr 14,14\n llgtr 15,15\n"
+                 "  lpsw %0\n"
+                 : : "Q" (*pswl) : "cc");
+}
+
+/*
+ * Limitations in QEMU's CCW support complicate the IPL process. Details can
+ * be found in docs/devel/s390-dasd-ipl.txt
+ */
+void dasd_ipl(SubChannelId schid)
+{
+    uint32_t ipl2_addr;
+
+    /* Construct Read IPL CCW and run it to read IPL1 from boot disk */
+    make_readipl();
+    run_readipl(schid);
+    ipl2_addr = read_ipl2_addr();
+    check_ipl1();
+
+    /*
+     * Fixup IPL1 channel program to account for QEMU limitations, then run it
+     * to read IPL2 channel program from boot disk.
+     */
+    ipl1_fixup();
+    run_ipl1(schid);
+    check_ipl2(ipl2_addr);
+
+    /*
+     * Run IPL2 channel program to read operating system code from boot disk
+     * then transfer control to the guest operating system
+     */
+    run_ipl2(schid, ipl2_addr);
+    lpsw(0);
+}
diff --git a/pc-bios/s390-ccw/dasd-ipl.h b/pc-bios/s390-ccw/dasd-ipl.h
new file mode 100644
index 0000000..56bba82
--- /dev/null
+++ b/pc-bios/s390-ccw/dasd-ipl.h
@@ -0,0 +1,16 @@
+/*
+ * S390 IPL (boot) from a real DASD device via vfio framework.
+ *
+ * Copyright (c) 2018 Jason J. Herne <jjherne@us.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef DASD_IPL_H
+#define DASD_IPL_H
+
+void dasd_ipl(SubChannelId schid);
+
+#endif /* DASD_IPL_H */
diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
index 5ee02c3..6d72196 100644
--- a/pc-bios/s390-ccw/main.c
+++ b/pc-bios/s390-ccw/main.c
@@ -13,6 +13,7 @@
 #include "s390-ccw.h"
 #include "cio.h"
 #include "virtio.h"
+#include "dasd-ipl.h"
 
 char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE)));
 static SubChannelId blk_schid = { .one = 1 };
@@ -210,6 +211,9 @@ int main(void)
 
     cutype = cu_type(blk_schid) ;
     switch (cutype) {
+    case CU_TYPE_DASD:
+        dasd_ipl(blk_schid); /* no return */
+        break;
     case CU_TYPE_VIRTIO:
         virtio_setup();
         zipl_load(); /* no return */
diff --git a/pc-bios/s390-ccw/s390-arch.h b/pc-bios/s390-ccw/s390-arch.h
index 47eaa04..0438d42 100644
--- a/pc-bios/s390-ccw/s390-arch.h
+++ b/pc-bios/s390-ccw/s390-arch.h
@@ -97,4 +97,17 @@ typedef struct LowCore {
 
 extern const LowCore *lowcore;
 
+static inline void set_prefix(uint32_t address)
+{
+    asm volatile("spx %0" : : "m" (address) : "memory");
+}
+
+static inline uint32_t store_prefix(void)
+{
+    uint32_t address;
+
+    asm volatile("stpx %0" : "=m" (address));
+    return address;
+}
+
 #endif
-- 
2.7.4

Re: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Cornelia Huck]

On Wed, 12 Dec 2018 09:11:03 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

Hm, I think you need to adjust your cc: list. I added some more folks
(and removed Dong Jia, whose address is no longer valid AFAIK).

> This is to support booting from vfio-ccw dasd devices. We basically implement
> the real hardware ipl procedure. This allows for booting Linux guests on
> vfio-ccw devices.
> 
> vfio-ccw's channel program prefetch algorithm complicates ipl because most ipl
> channel programs dynamically modify themselves. Details on the ipl process and
> how we worked around this issue can be found in docs/devel/s390-dasd-ipl.txt.
> 
> NOTE: It has been a while, but I've finally chased down my infamous "reset bug".
> On subsystem reset (I see this right after host ipl) we sometimes end up getting
> an unexpected unit check status from a dasd device. This causes the first start
> subchannel instruction to fail due to the pending unit check status. My solution
> to this problem, as advised by the kernel folks, is to simply retry my ssch
> instructions before declaring failure when unexpected unit checks happen. In the
> event of a persistent error, after two retries we'll give up and print some
> useful error info for the user.

So, is that a status we only see because the vfio-ccw driver keeps the
subchannel enabled (as by the other recent thread)?

Is there any value in distinguishing different unit checks, or is retry
the best strategy in any case?

> 
> Changelog
> ==========
> 
> rfc -> v1
> ---------
> - Retry start subchannel operations on unexpected unit check status
> - Print more complete state information on i/o error
> - Fixed asm constraints (Q -> QS)
> - Align lowcore to 8k boundary
> - Initialize next_cpa in run_dynamic_ccw_program to avoid compiler warning
> - Cleanup: Rework handling of i/o interrupts in assembler
> - Fix netboot image (include cio.o in make file)
> - Misc cleanups
> 
> Jason J. Herne (15):
>   s390 vfio-ccw: Add bootindex property and IPLB data
>   s390-bios: decouple cio setup from virtio
>   s390-bios: decouple common boot logic from virtio
>   s390-bios: Extend find_dev() for non-virtio devices
>   s390-bios: Factor finding boot device out of virtio code path
>   s390-bios: Clean up cio.h
>   s390-bios: Decouple channel i/o logic from virtio
>   s390-bios: Map low core memory
>   s390-bios: ptr2u32 and u32toptr
>   s390-bios: Support for running format-0/1 channel programs
>   s390-bios: cio error handling
>   s390-bios: Refactor virtio to run channel programs via cio
>   s390-bios: Use control unit type to determine boot method
>   s390-bios: Add channel command codes/structs needed for dasd-ipl
>   s390-bios: Support booting from real dasd device
> 
>  docs/devel/s390-dasd-ipl.txt     | 132 ++++++++++++++
>  hw/s390x/ipl.c                   |  15 ++
>  hw/s390x/s390-ccw.c              |   9 +
>  hw/vfio/ccw.c                    |  13 +-
>  include/hw/s390x/s390-ccw.h      |   1 +
>  include/hw/s390x/vfio-ccw.h      |  38 ++++
>  pc-bios/s390-ccw/Makefile        |   2 +-
>  pc-bios/s390-ccw/cio.c           | 374 +++++++++++++++++++++++++++++++++++++++
>  pc-bios/s390-ccw/cio.h           | 236 +++++++++++++++++++-----
>  pc-bios/s390-ccw/dasd-ipl.c      | 249 ++++++++++++++++++++++++++
>  pc-bios/s390-ccw/dasd-ipl.h      |  16 ++
>  pc-bios/s390-ccw/libc.h          |  23 +++
>  pc-bios/s390-ccw/main.c          | 161 +++++++++++------
>  pc-bios/s390-ccw/netboot.mak     |   2 +-
>  pc-bios/s390-ccw/netmain.c       |   1 +
>  pc-bios/s390-ccw/s390-arch.h     | 113 ++++++++++++
>  pc-bios/s390-ccw/s390-ccw.h      |  10 +-
>  pc-bios/s390-ccw/start.S         |  33 +++-
>  pc-bios/s390-ccw/virtio-blkdev.c |   1 +
>  pc-bios/s390-ccw/virtio.c        |  46 +----
>  roms/SLOF                        |   2 +-
>  tests/boot-serial-test.c         |   2 +-
>  22 files changed, 1308 insertions(+), 171 deletions(-)
>  create mode 100644 docs/devel/s390-dasd-ipl.txt
>  create mode 100644 include/hw/s390x/vfio-ccw.h
>  create mode 100644 pc-bios/s390-ccw/cio.c
>  create mode 100644 pc-bios/s390-ccw/dasd-ipl.c
>  create mode 100644 pc-bios/s390-ccw/dasd-ipl.h
>  create mode 100644 pc-bios/s390-ccw/s390-arch.h
> 
> --
> 2.7.4
> 

Re: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Jason J. Herne]

On 12/12/18 9:34 AM, Cornelia Huck wrote:
> On Wed, 12 Dec 2018 09:11:03 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
> Hm, I think you need to adjust your cc: list. I added some more folks
> (and removed Dong Jia, whose address is no longer valid AFAIK).
> 

Correct. I forgot to update my list before I sent.


>> NOTE: It has been a while, but I've finally chased down my infamous "reset bug".
>> On subsystem reset (I see this right after host ipl) we sometimes end up getting
>> an unexpected unit check status from a dasd device. This causes the first start
>> subchannel instruction to fail due to the pending unit check status. My solution
>> to this problem, as advised by the kernel folks, is to simply retry my ssch
>> instructions before declaring failure when unexpected unit checks happen. In the
>> event of a persistent error, after two retries we'll give up and print some
>> useful error info for the user.
> 
> So, is that a status we only see because the vfio-ccw driver keeps the
> subchannel enabled (as by the other recent thread)?
> 
> Is there any value in distinguishing different unit checks, or is retry
> the best strategy in any case?
> 

It is status only, yes. I'm not sure if there is value in treating different unit checks 
differently. I discussed the problem with some of the kernel i/o devs and the suggestion I 
got was to simply retry. In the event of a real I/O error I doubt there is much we'd be 
able to do to recover so I think showing the user all of the relevant info (see patch 
s390-bios: cio error handling) and exiting is the right thing to do.

-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [qemu-s390x] [PATCH 01/15] s390 vfio-ccw: Add bootindex property and IPLB data

[Jason J. Herne]

On 12/12/18 9:11 AM, Jason J. Herne wrote:
> ...
> diff --git a/hw/vfio/ccw.c b/hw/vfio/ccw.c
> index 9246729..6f480c6 100644
> --- a/hw/vfio/ccw.c
> +++ b/hw/vfio/ccw.c
> @@ -20,23 +20,12 @@
>   #include "hw/sysbus.h"
>   #include "hw/vfio/vfio.h"
>   #include "hw/vfio/vfio-common.h"
> +#include "hw/vfio/ccw.h"

Oops #1: This should be:
+ #include "hw/s390x/vfio-ccw.h"

Will fix for v2.

>   #include "hw/s390x/s390-ccw.h"
>   #include "hw/s390x/ccw-device.h"
>   #include "exec/address-spaces.h"
>   #include "qemu/error-report.h"
> 

 > ...
> diff --git a/roms/SLOF b/roms/SLOF
> index 9b7ab2f..2317427 160000
> --- a/roms/SLOF
> +++ b/roms/SLOF
> @@ -1 +1 @@
> -Subproject commit 9b7ab2fa020341dee8bf9df6c9cf40003e0136df
> +Subproject commit 2317427ce76006723f7ae103a6998ab41dd79c68

Oops #2: This snuck into the patch unexpectedly. Removed for v2.

-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[no-reply]

Patchew URL: https://patchew.org/QEMU/1544623878-11248-1-git-send-email-jjherne@linux.ibm.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Type: series
Message-id: 1544623878-11248-1-git-send-email-jjherne@linux.ibm.com
Subject: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
7bc3233 s390-bios: Support booting from real dasd device
f2df85e s390-bios: Add channel command codes/structs needed for dasd-ipl
1fa1e47 s390-bios: Use control unit type to determine boot method
364ad22 s390-bios: Refactor virtio to run channel programs via cio
c3129f5 s390-bios: cio error handling
f33018f s390-bios: Support for running format-0/1 channel programs
3e79f11 s390-bios: ptr2u32 and u32toptr
00376ab s390-bios: Map low core memory
c1313ca s390-bios: Decouple channel i/o logic from virtio
98f3522 s390-bios: Clean up cio.h
3ff76c4 s390-bios: Factor finding boot device out of virtio code path
80e0855 s390-bios: Extend find_dev() for non-virtio devices
2127e05 s390-bios: decouple common boot logic from virtio
e47231c s390-bios: decouple cio setup from virtio
67265dd s390 vfio-ccw: Add bootindex property and IPLB data

=== OUTPUT BEGIN ===
Checking PATCH 1/15: s390 vfio-ccw: Add bootindex property and IPLB data...
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#131: 
new file mode 100644

total: 0 errors, 1 warnings, 133 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 2/15: s390-bios: decouple cio setup from virtio...
Checking PATCH 3/15: s390-bios: decouple common boot logic from virtio...
ERROR: externs should be avoided in .c files
#28: FILE: pc-bios/s390-ccw/main.c:19:
+IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));

total: 1 errors, 0 warnings, 65 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 4/15: s390-bios: Extend find_dev() for non-virtio devices...
Checking PATCH 5/15: s390-bios: Factor finding boot device out of virtio code path...
Checking PATCH 6/15: s390-bios: Clean up cio.h...
ERROR: spaces prohibited around that ':' (ctx:WxW)
#27: FILE: pc-bios/s390-ccw/cio.h:56:
+    __u32 isc    : 3;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#28: FILE: pc-bios/s390-ccw/cio.h:57:
+    __u32 ena    : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#29: FILE: pc-bios/s390-ccw/cio.h:58:
+    __u32 mme    : 2;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#30: FILE: pc-bios/s390-ccw/cio.h:59:
+    __u32 mp     : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#31: FILE: pc-bios/s390-ccw/cio.h:60:
+    __u32 csense : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#32: FILE: pc-bios/s390-ccw/cio.h:61:
+    __u32 mbfc   : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#82: FILE: pc-bios/s390-ccw/cio.h:111:
+    __u32 reserved5 : 4;
                     ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#83: FILE: pc-bios/s390-ccw/cio.h:112:
+    __u32 format2   : 4;
                     ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#84: FILE: pc-bios/s390-ccw/cio.h:113:
+    __u32 reserved6 : 24;
                     ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#130: FILE: pc-bios/s390-ccw/cio.h:167:
+    __u32 key  : 4;   /* flags, like key, suspend control, etc. */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#131: FILE: pc-bios/s390-ccw/cio.h:168:
+    __u32 spnd : 1;   /* suspend control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#132: FILE: pc-bios/s390-ccw/cio.h:169:
+    __u32 res1 : 1;   /* reserved */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#133: FILE: pc-bios/s390-ccw/cio.h:170:
+    __u32 mod  : 1;   /* modification control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#134: FILE: pc-bios/s390-ccw/cio.h:171:
+    __u32 sync : 1;   /* synchronize control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#135: FILE: pc-bios/s390-ccw/cio.h:172:
+    __u32 fmt  : 1;   /* format control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#136: FILE: pc-bios/s390-ccw/cio.h:173:
+    __u32 pfch : 1;   /* prefetch control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#137: FILE: pc-bios/s390-ccw/cio.h:174:
+    __u32 isic : 1;   /* initial-status-interruption control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#138: FILE: pc-bios/s390-ccw/cio.h:175:
+    __u32 alcc : 1;   /* address-limit-checking control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#139: FILE: pc-bios/s390-ccw/cio.h:176:
+    __u32 ssic : 1;   /* suppress-suspended-interr. control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#140: FILE: pc-bios/s390-ccw/cio.h:177:
+    __u32 res2 : 1;   /* reserved */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#141: FILE: pc-bios/s390-ccw/cio.h:178:
+    __u32 c64  : 1;   /* IDAW/QDIO 64 bit control  */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#142: FILE: pc-bios/s390-ccw/cio.h:179:
+    __u32 i2k  : 1;   /* IDAW 2/4kB block size control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#143: FILE: pc-bios/s390-ccw/cio.h:180:
+    __u32 lpm  : 8;   /* logical path mask */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#144: FILE: pc-bios/s390-ccw/cio.h:181:
+    __u32 ils  : 1;   /* incorrect length */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#145: FILE: pc-bios/s390-ccw/cio.h:182:
+    __u32 zero : 6;   /* reserved zeros */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#146: FILE: pc-bios/s390-ccw/cio.h:183:
+    __u32 orbx : 1;   /* ORB extension control */
                ^

total: 26 errors, 0 warnings, 171 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 7/15: s390-bios: Decouple channel i/o logic from virtio...
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#27: 
new file mode 100644

total: 0 errors, 1 warnings, 127 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 8/15: s390-bios: Map low core memory...
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#34: 
new file mode 100644

total: 0 errors, 1 warnings, 114 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 9/15: s390-bios: ptr2u32 and u32toptr...
Checking PATCH 10/15: s390-bios: Support for running format-0/1 channel programs...
Checking PATCH 11/15: s390-bios: cio error handling...
Checking PATCH 12/15: s390-bios: Refactor virtio to run channel programs via cio...
Checking PATCH 13/15: s390-bios: Use control unit type to determine boot method...
Checking PATCH 14/15: s390-bios: Add channel command codes/structs needed for dasd-ipl...
Checking PATCH 15/15: s390-bios: Support booting from real dasd device...
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#12: 
new file mode 100644

total: 0 errors, 1 warnings, 438 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/1544623878-11248-1-git-send-email-jjherne@linux.ibm.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

Re: [Qemu-devel] [PATCH 02/15] s390-bios: decouple cio setup from virtio

[Farhan Ali]

On 12/12/2018 09:11 AM, Jason J. Herne wrote:
> Move channel i/o setup code out to a separate function. This decouples cio
> setup from the virtio code path and allows us to make use of it for booting
> dasd devices.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
> Reviewed-by: Collin Walling <walling@linux.ibm.com>
> ---
>   pc-bios/s390-ccw/main.c | 20 +++++++++++++-------
>   1 file changed, 13 insertions(+), 7 deletions(-)
> 
> diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
> index 544851d..e82fe2c 100644
> --- a/pc-bios/s390-ccw/main.c
> +++ b/pc-bios/s390-ccw/main.c
> @@ -99,6 +99,18 @@ static void menu_setup(void)
>       }
>   }
> 
> +/*
> + * Initialize the channel I/O subsystem so we can talk to our ipl/boot device.
> + */
> +static void css_setup(void)
> +{
> +    /*
> +     * Unconditionally enable mss support. In every sane configuration this
> +     * will succeed; and even if it doesn't, stsch_err() can handle it.
> +     */
> +    enable_mss_facility();
> +}
> +
>   static void virtio_setup(void)
>   {
>       Schib schib;
> @@ -109,13 +121,6 @@ static void virtio_setup(void)
>       VDev *vdev = virtio_get_device();
>       QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
> 
> -    /*
> -     * We unconditionally enable mss support. In every sane configuration,
> -     * this will succeed; and even if it doesn't, stsch_err() can deal
> -     * with the consequences.
> -     */
> -    enable_mss_facility();
> -
>       sclp_get_loadparm_ascii(loadparm_str);
>       memcpy(ldp + 10, loadparm_str, LOADPARM_LEN);
>       sclp_print(ldp);
> @@ -168,6 +173,7 @@ static void virtio_setup(void)
>   int main(void)
>   {
>       sclp_setup();
> +    css_setup();
>       virtio_setup();
> 
>       zipl_load(); /* no return */
> 

Reviewed-by: Farhan Ali <alifm@linux.ibm.com>

Re: [Qemu-devel] [PATCH 03/15] s390-bios: decouple common boot logic from virtio

[Farhan Ali]

On 12/12/2018 09:11 AM, Jason J. Herne wrote:
> Create a boot_setup function to handle getting boot information from
> the machine/hypervisor. This decouples common boot logic from the
> virtio code path and allows us to make use of it for the real dasd boot
> scenario.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
> Reviewed-by: Collin Walling <walling@linux.ibm.com
> Reviewed-by: Thomas Huth <thuth@redhat.com>
> ---
>   pc-bios/s390-ccw/main.c | 28 ++++++++++++++++++++--------
>   1 file changed, 20 insertions(+), 8 deletions(-)
> 
> diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
> index e82fe2c..67df421 100644
> --- a/pc-bios/s390-ccw/main.c
> +++ b/pc-bios/s390-ccw/main.c
> @@ -14,16 +14,17 @@
> 
>   char stack[PAGE_SIZE * 8] __attribute__((__aligned__(PAGE_SIZE)));
>   static SubChannelId blk_schid = { .one = 1 };
> -IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));
>   static char loadparm_str[LOADPARM_LEN + 1] = { 0, 0, 0, 0, 0, 0, 0, 0, 0 };
>   QemuIplParameters qipl;
> +IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));
> +static bool have_iplb;
> 
>   #define LOADPARM_PROMPT "PROMPT  "
>   #define LOADPARM_EMPTY  "        "
>   #define BOOT_MENU_FLAG_MASK (QIPL_FLAG_BM_OPTS_CMD | QIPL_FLAG_BM_OPTS_ZIPL)
> 
>   /*
> - * Priniciples of Operations (SA22-7832-09) chapter 17 requires that
> + * Principles of Operations (SA22-7832-09) chapter 17 requires that
>    * a subsystem-identification is at 184-187 and bytes 188-191 are zero
>    * after list-directed-IPL and ccw-IPL.
>    */
> @@ -111,23 +112,33 @@ static void css_setup(void)
>       enable_mss_facility();
>   }
> 
> +/*
> + * Collect various pieces of information from the hypervisor/hardware that
> + * we'll use to determine exactly how we'll boot.
> + */
> +static void boot_setup(void)
> +{
> +    char lpmsg[] = "LOADPARM=[________]\n";
> +
> +    sclp_get_loadparm_ascii(loadparm_str);
> +    memcpy(lpmsg + 10, loadparm_str, 8);
> +    sclp_print(lpmsg);
> +
> +    have_iplb = store_iplb(&iplb);
> +}
> +
>   static void virtio_setup(void)
>   {
>       Schib schib;
>       int ssid;
>       bool found = false;
>       uint16_t dev_no;
> -    char ldp[] = "LOADPARM=[________]\n";
>       VDev *vdev = virtio_get_device();
>       QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
> 
> -    sclp_get_loadparm_ascii(loadparm_str);
> -    memcpy(ldp + 10, loadparm_str, LOADPARM_LEN);
> -    sclp_print(ldp);
> -
>       memcpy(&qipl, early_qipl, sizeof(QemuIplParameters));
> 
> -    if (store_iplb(&iplb)) {
> +    if (have_iplb) {
>           switch (iplb.pbt) {
>           case S390_IPL_TYPE_CCW:
>               dev_no = iplb.ccw.devno;
> @@ -174,6 +185,7 @@ int main(void)
>   {
>       sclp_setup();
>       css_setup();
> +    boot_setup();
>       virtio_setup();
> 
>       zipl_load(); /* no return */
> 


Reviewed-by: Farhan Ali <alifm@linux.ibm.com>

Re: [Qemu-devel] [PATCH 05/15] s390-bios: Factor finding boot device out of virtio code path

[Farhan Ali]

On 12/12/2018 09:11 AM, Jason J. Herne wrote:
> Make a new routine find_boot_device to locate the boot device for all
> cases. not just virtio.

I don't think there should be a period after cases?

> 
> In one case no boot device is specified and a suitable boot device can not
> be auto detected. The error message for this case was specific to virtio
> devices. We update this message to remove virtio specific wording.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> ---
>   pc-bios/s390-ccw/main.c  | 87 ++++++++++++++++++++++++++----------------------
>   tests/boot-serial-test.c |  2 +-
>   2 files changed, 49 insertions(+), 40 deletions(-)
> 
> diff --git a/pc-bios/s390-ccw/main.c b/pc-bios/s390-ccw/main.c
> index 7e3f65e..2457752 100644
> --- a/pc-bios/s390-ccw/main.c
> +++ b/pc-bios/s390-ccw/main.c
> @@ -55,17 +55,18 @@ unsigned int get_loadparm_index(void)
>    * NOTE: The global variable blk_schid is updated to contain the subchannel
>    * information.
>    */
> -static bool find_dev(Schib *schib, int dev_no)
> +static bool find_subch(int dev_no)
>   {
> +    Schib schib;
>       int i, r;
> 
>       for (i = 0; i < 0x10000; i++) {
>           blk_schid.sch_no = i;
> -        r = stsch_err(blk_schid, schib);
> +        r = stsch_err(blk_schid, &schib);
>           if ((r == 3) || (r == -EIO)) {
>               break;
>           }
> -        if (!schib->pmcw.dnv) {
> +        if (!schib.pmcw.dnv) {
>               continue;
>           }
> 
> @@ -77,7 +78,7 @@ static bool find_dev(Schib *schib, int dev_no)
>               continue;
>           }
> 
> -        if ((dev_no < 0) || (schib->pmcw.dev == dev_no)) {
> +        if ((dev_no < 0) || (schib.pmcw.dev == dev_no)) {
>               return true;
>           }
>       }
> @@ -133,56 +134,63 @@ static void boot_setup(void)
>       have_iplb = store_iplb(&iplb);
>   }
> 
> -static void virtio_setup(void)
> +static void find_boot_device(void)
>   {
> -    Schib schib;
> -    int ssid;
> -    bool found = false;
> -    uint16_t dev_no;
>       VDev *vdev = virtio_get_device();
> -    QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
> -
> -    memcpy(&qipl, early_qipl, sizeof(QemuIplParameters));
> +    int ssid;
> +    bool found;
> 
> -    if (have_iplb) {
> -        switch (iplb.pbt) {
> -        case S390_IPL_TYPE_CCW:
> -            dev_no = iplb.ccw.devno;
> -            debug_print_int("device no. ", dev_no);
> -            blk_schid.ssid = iplb.ccw.ssid & 0x3;
> -            debug_print_int("ssid ", blk_schid.ssid);
> -            found = find_dev(&schib, dev_no);
> -            break;
> -        case S390_IPL_TYPE_QEMU_SCSI:
> -            vdev->scsi_device_selected = true;
> -            vdev->selected_scsi_device.channel = iplb.scsi.channel;
> -            vdev->selected_scsi_device.target = iplb.scsi.target;
> -            vdev->selected_scsi_device.lun = iplb.scsi.lun;
> -            blk_schid.ssid = iplb.scsi.ssid & 0x3;
> -            found = find_dev(&schib, iplb.scsi.devno);
> -            break;
> -        default:
> -            panic("List-directed IPL not supported yet!\n");
> -        }
> -        menu_setup();
> -    } else {
> +    if (!have_iplb) {
>           for (ssid = 0; ssid < 0x3; ssid++) {
>               blk_schid.ssid = ssid;
> -            found = find_dev(&schib, -1);
> +            found = find_subch(-1);
>               if (found) {
> -                break;
> +                return;
>               }
>           }
> +        panic("Could not find a suitable boot device (none specified)\n");
>       }
> 
> -    IPL_assert(found, "No virtio device found");
> +    switch (iplb.pbt) {
> +    case S390_IPL_TYPE_CCW:
> +        debug_print_int("device no. ", iplb.ccw.devno);
> +        blk_schid.ssid = iplb.ccw.ssid & 0x3;
> +        debug_print_int("ssid ", blk_schid.ssid);
> +        found = find_subch(iplb.ccw.devno);
> +        break;
> +    case S390_IPL_TYPE_QEMU_SCSI:
> +        vdev->scsi_device_selected = true;
> +        vdev->selected_scsi_device.channel = iplb.scsi.channel;
> +        vdev->selected_scsi_device.target = iplb.scsi.target;
> +        vdev->selected_scsi_device.lun = iplb.scsi.lun;
> +        blk_schid.ssid = iplb.scsi.ssid & 0x3;
> +        found = find_subch(iplb.scsi.devno);
> +        break;
> +    default:
> +        panic("List-directed IPL not supported yet!\n");
> +    }
> +
> +    if (!found) {
> +        IPL_assert(found, "Boot device not found\n");
> +    }
> +}
> +
> +static void virtio_setup(void)
> +{
> +    VDev *vdev = virtio_get_device();
> +    QemuIplParameters *early_qipl = (QemuIplParameters *)QIPL_ADDRESS;
> +
> +    memcpy(&qipl, early_qipl, sizeof(QemuIplParameters));
> +
> +    if (have_iplb) {
> +        menu_setup();
> +    }
> 
>       if (virtio_get_device_type() == VIRTIO_ID_NET) {
>           sclp_print("Network boot device detected\n");
>           vdev->netboot_start_addr = qipl.netboot_start_addr;
>       } else {
>           virtio_blk_setup_device(blk_schid);
> -
>           IPL_assert(virtio_ipl_disk_is_valid(), "No valid IPL device detected");
>       }
>   }
> @@ -192,8 +200,9 @@ int main(void)
>       sclp_setup();
>       css_setup();
>       boot_setup();
> -    virtio_setup();
> +    find_boot_device();
> 
> +    virtio_setup();
>       zipl_load(); /* no return */
> 
>       panic("Failed to load OS from hard disk\n");
> diff --git a/tests/boot-serial-test.c b/tests/boot-serial-test.c
> index 8ec6aed..064f5de 100644
> --- a/tests/boot-serial-test.c
> +++ b/tests/boot-serial-test.c
> @@ -112,7 +112,7 @@ static testdef_t tests[] = {
>       { "sparc", "SS-4", "", "MB86904" },
>       { "sparc", "SS-600MP", "", "TMS390Z55" },
>       { "sparc64", "sun4u", "", "UltraSPARC" },
> -    { "s390x", "s390-ccw-virtio", "", "virtio device" },
> +    { "s390x", "s390-ccw-virtio", "", "device" },
>       { "m68k", "mcf5208evb", "", "TT", sizeof(kernel_mcf5208), kernel_mcf5208 },
>       { "microblaze", "petalogix-s3adsp1800", "", "TT",
>         sizeof(kernel_pls3adsp1800), kernel_pls3adsp1800 },
> 

Re: [Qemu-devel] [PATCH 01/15] s390 vfio-ccw: Add bootindex property and IPLB data

[Cornelia Huck]

On Wed, 12 Dec 2018 09:11:04 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> Add bootindex property and iplb data for vfio-ccw devices. This allows us to
> forward boot information into the bios for vfio-ccw devices.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> Acked-by: Halil Pasic <pasic@linux.vnet.ibm.com>
> ---
>  hw/s390x/ipl.c              | 15 +++++++++++++++
>  hw/s390x/s390-ccw.c         |  9 +++++++++
>  hw/vfio/ccw.c               | 13 +------------
>  include/hw/s390x/s390-ccw.h |  1 +
>  include/hw/s390x/vfio-ccw.h | 38 ++++++++++++++++++++++++++++++++++++++
>  roms/SLOF                   |  2 +-
>  6 files changed, 65 insertions(+), 13 deletions(-)
>  create mode 100644 include/hw/s390x/vfio-ccw.h
> 
> diff --git a/hw/s390x/ipl.c b/hw/s390x/ipl.c
> index 21f64ad..31592e7 100644
> --- a/hw/s390x/ipl.c
> +++ b/hw/s390x/ipl.c
> @@ -19,6 +19,7 @@
>  #include "hw/loader.h"
>  #include "hw/boards.h"
>  #include "hw/s390x/virtio-ccw.h"
> +#include "hw/s390x/vfio-ccw.h"
>  #include "hw/s390x/css.h"
>  #include "hw/s390x/ebcdic.h"
>  #include "ipl.h"
> @@ -311,8 +312,13 @@ static CcwDevice *s390_get_ccw_device(DeviceState *dev_st)
>          VirtioCcwDevice *virtio_ccw_dev = (VirtioCcwDevice *)
>              object_dynamic_cast(OBJECT(qdev_get_parent_bus(dev_st)->parent),
>                                  TYPE_VIRTIO_CCW_DEVICE);
> +        VFIOCCWDevice *vc = (VFIOCCWDevice *)
> +            object_dynamic_cast(OBJECT(dev_st), TYPE_VFIO_CCW);
> +
>          if (virtio_ccw_dev) {
>              ccw_dev = CCW_DEVICE(virtio_ccw_dev);
> +        } else if (vc) {
> +            ccw_dev = CCW_DEVICE(vc);

Maybe call it "vfio_ccw_dev" instead? But that's definitely bikeshed
territory...

>          } else {
>              SCSIDevice *sd = (SCSIDevice *)
>                  object_dynamic_cast(OBJECT(dev_st),

You already pointed out the other problems yourself :)

Otherwise, looks good.

Re: [Qemu-devel] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Farhan Ali]

On 12/12/2018 09:11 AM, Jason J. Herne wrote:
> Add struct for format-0 ccws. Support executing format-0 channel
> programs and waiting for their completion before continuing execution.
> This will be used for real dasd ipl.
> 
> Add cu_type() to channel io library. This will be used to query control
> unit type which is used to determine if we are booting a virtio device or a
> real dasd device.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> ---
>   pc-bios/s390-ccw/cio.c      | 108 ++++++++++++++++++++++++++++++++++++++
>   pc-bios/s390-ccw/cio.h      | 124 ++++++++++++++++++++++++++++++++++++++++++--
>   pc-bios/s390-ccw/s390-ccw.h |   1 +
>   pc-bios/s390-ccw/start.S    |  33 +++++++++++-
>   4 files changed, 261 insertions(+), 5 deletions(-)
> 
> diff --git a/pc-bios/s390-ccw/cio.c b/pc-bios/s390-ccw/cio.c
> index 095f79b..9019250 100644
> --- a/pc-bios/s390-ccw/cio.c
> +++ b/pc-bios/s390-ccw/cio.c
> @@ -10,6 +10,7 @@
> 
>   #include "libc.h"
>   #include "s390-ccw.h"
> +#include "s390-arch.h"
>   #include "cio.h"
> 
>   static char chsc_page[PAGE_SIZE] __attribute__((__aligned__(PAGE_SIZE)));
> @@ -39,3 +40,110 @@ void enable_subchannel(SubChannelId schid)
>       schib.pmcw.ena = 1;
>       msch(schid, &schib);
>   }
> +
> +uint16_t cu_type(SubChannelId schid)
> +{
> +    Ccw1 sense_id_ccw;
> +    SenseId sense_data;
> +
> +    sense_id_ccw.cmd_code = CCW_CMD_SENSE_ID;
> +    sense_id_ccw.cda = ptr2u32(&sense_data);
> +    sense_id_ccw.count = sizeof(sense_data);
> +
> +    if (do_cio(schid, ptr2u32(&sense_id_ccw), CCW_FMT1)) {
> +        panic("Failed to run SenseID CCw\n");
> +    }
> +
> +    return sense_data.cu_type;
> +}
> +
> +void basic_sense(SubChannelId schid, SenseData *sd)
> +{
> +    Ccw1 senseCcw;
> +
> +    senseCcw.cmd_code = CCW_CMD_BASIC_SENSE;
> +    senseCcw.cda = ptr2u32(sd);
> +    senseCcw.count = sizeof(*sd);
> +
> +    if (do_cio(schid, ptr2u32(&senseCcw), CCW_FMT1)) {
> +        panic("Failed to run Basic Sense CCW\n");
> +    }
> +}
> +
> +static bool irb_error(Irb *irb)
> +{
> +    /* We have to ignore Incorrect Length (cstat == 0x40) indicators because
> +     * real devices expect a 24 byte SenseID  buffer, and virtio devices expect
> +     * a much larger buffer. Neither device type can tolerate a buffer size
> +     * different from what they expect so they set this indicator.
> +     */
> +    if (irb->scsw.cstat != 0x00 && irb->scsw.cstat != 0x40) {
> +        return true;
> +    }
> +    return irb->scsw.dstat != 0xc;
> +}
> +
> +/* Executes a channel program at a given subchannel. The request to run the
> + * channel program is sent to the subchannel, we then wait for the interrupt
> + * singaling completion of the I/O operation(s) perfomed by the channel
> + * program. Lastly we verify that the i/o operation completed without error and
> + * that the interrupt we received was for the subchannel used to run the
> + * channel program.
> + *
> + * Note: This function assumes it is running in an environment where no other
> + * cpus are generating or receiving I/O interrupts. So either run it in a
> + * single-cpu environment or make sure all other cpus are not doing I/O and
> + * have I/O interrupts masked off.
> + */
> +int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
> +{
> +    CmdOrb orb = {};
> +    Irb irb = {};
> +    SenseData sd;
> +    int rc, retries = 0;
> +
> +    IPL_assert(fmt == 0 || fmt == 1, "Invalid ccw format");
> +
> +    /* ccw_addr must be <= 24 bits and point to at least one whole ccw. */
> +    if (fmt == 0) {
> +        IPL_assert(ccw_addr <= 0xFFFFFF - 8, "Invalid ccw address");
> +    }
> +
> +    orb.fmt = fmt ;
> +    orb.pfch = 1;  /* QEMU's cio implementation requires prefetch */
> +    orb.c64 = 1;   /* QEMU's cio implementation requires 64-bit idaws */
> +    orb.lpm = 0xFF; /* All paths allowed */
> +    orb.cpa = ccw_addr;
> +
> +    while (true) {
> +        rc = ssch(schid, &orb);
> +        if (rc) {
> +            print_int("ssch failed with rc=", rc);
> +            break;
> +        }
> +
> +        consume_io_int();
> +
> +        /* Clear read */
> +        rc = tsch(schid, &irb);
> +        if (rc) {
> +            print_int("tsch failed with rc=", rc);
> +            break;
> +        }
> +
> +        if (!irb_error(&irb)) {
> +            break;
> +        }

Now that we enable i/o interrupts, do we still need drain_irqs function 
anymore? Maybe we could refactor that.

My question is more out of curiosity and I know it's out of the score 
for this patch series.

> +
> +        /* Unexpected unit check. Use sense to clear unit check then retry. */
> +        if (unit_check(&irb) && retries <= 2) {
> +            basic_sense(schid, &sd);
> +            retries++;
> +            continue;
> +        }
> +
> +        break;
> +    }
> +
> +    return rc;
> +}
> diff --git a/pc-bios/s390-ccw/cio.h b/pc-bios/s390-ccw/cio.h
> index 7b07d75..5c16635 100644
> --- a/pc-bios/s390-ccw/cio.h
> +++ b/pc-bios/s390-ccw/cio.h
> @@ -70,9 +70,46 @@ struct scsw {
>       __u16 count;
>   } __attribute__ ((packed));
> 
> -#define SCSW_FCTL_CLEAR_FUNC 0x1000
> -#define SCSW_FCTL_HALT_FUNC 0x2000
> +/* Function Control */
>   #define SCSW_FCTL_START_FUNC 0x4000
> +#define SCSW_FCTL_HALT_FUNC 0x2000
> +#define SCSW_FCTL_CLEAR_FUNC 0x1000
> +
> +/* Activity Control */
> +#define SCSW_ACTL_RESUME_PEND   0x0800
> +#define SCSW_ACTL_START_PEND    0x0400
> +#define SCSW_ACTL_HALT_PEND     0x0200
> +#define SCSW_ACTL_CLEAR_PEND    0x0100
> +#define SCSW_ACTL_CH_ACTIVE     0x0080
> +#define SCSW_ACTL_DEV_ACTIVE    0x0040
> +#define SCSW_ACTL_SUSPENDED     0x0020
> +
> +/* Status Control */
> +#define SCSW_SCTL_ALERT         0x0010
> +#define SCSW_SCTL_INTERMED      0x0008
> +#define SCSW_SCTL_PRIMARY       0x0004
> +#define SCSW_SCTL_SECONDARY     0x0002
> +#define SCSW_SCTL_STATUS_PEND   0x0001
> +
> +/* SCSW Device Status Flags */
> +#define SCSW_DSTAT_ATTN     0x80
> +#define SCSW_DSTAT_STATMOD  0x40
> +#define SCSW_DSTAT_CUEND    0x20
> +#define SCSW_DSTAT_BUSY     0x10
> +#define SCSW_DSTAT_CHEND    0x08
> +#define SCSW_DSTAT_DEVEND   0x04
> +#define SCSW_DSTAT_UCHK     0x02
> +#define SCSW_DSTAT_UEXCP    0x01
> +
> +/* SCSW Subchannel Status Flags */
> +#define SCSW_CSTAT_PCINT    0x80
> +#define SCSW_CSTAT_BADLEN   0x40
> +#define SCSW_CSTAT_PROGCHK  0x20
> +#define SCSW_CSTAT_PROTCHK  0x10
> +#define SCSW_CSTAT_CHDCHK   0x08
> +#define SCSW_CSTAT_CHCCHK   0x04
> +#define SCSW_CSTAT_ICCHK    0x02
> +#define SCSW_CSTAT_CHAINCHK 0x01
> 
>   /*
>    * subchannel information block
> @@ -127,7 +164,23 @@ struct tpi_info {
>       __u32 reserved4  : 12;
>   } __attribute__ ((packed, aligned(4)));
> 
> -/* channel command word (type 1) */
> +/* channel command word (format 0) */
> +typedef struct ccw0 {
> +    __u8 cmd_code;
> +    __u32 cda        : 24;
> +    __u32 chainData  : 1;
> +    __u32 chain      : 1;
> +    __u32 sli        : 1;
> +    __u32 skip       : 1;
> +    __u32 pci        : 1;
> +    __u32 ida        : 1;
> +    __u32 suspend    : 1;
> +    __u32 mida       : 1;
> +    __u8 reserved;
> +    __u16 count;
> +} __attribute__ ((packed, aligned(8))) Ccw0;
> +
> +/* channel command word (format 1) */
>   typedef struct ccw1 {
>       __u8 cmd_code;
>       __u8 flags;
> @@ -135,6 +188,10 @@ typedef struct ccw1 {
>       __u32 cda;
>   } __attribute__ ((packed, aligned(8))) Ccw1;
> 
> +/* do_cio() CCW formats */
> +#define CCW_FMT0                 0x00
> +#define CCW_FMT1                 0x01
> +
>   #define CCW_FLAG_DC              0x80
>   #define CCW_FLAG_CC              0x40
>   #define CCW_FLAG_SLI             0x20
> @@ -190,6 +247,9 @@ struct ciw {
>       __u16 count;
>   };
> 
> +#define CU_TYPE_VIRTIO          0x3832
> +#define CU_TYPE_DASD            0x3990
> +
>   /*
>    * sense-id response buffer layout
>    */
> @@ -205,6 +265,61 @@ typedef struct senseid {
>       struct ciw ciw[62];
>   }  __attribute__ ((packed, aligned(4))) SenseId;
> 
> +/* architected values for first sense byte */
> +#define SNS0_CMD_REJECT         0x80
> +#define SNS0_INTERVENTION_REQ   0x40
> +#define SNS0_BUS_OUT_CHECK      0x20
> +#define SNS0_EQUIPMENT_CHECK    0x10
> +#define SNS0_DATA_CHECK         0x08
> +#define SNS0_OVERRUN            0x04
> +#define SNS0_INCOMPL_DOMAIN     0x01
> +
> +/* architectured values for second sense byte */
> +#define SNS1_PERM_ERR           0x80
> +#define SNS1_INV_TRACK_FORMAT   0x40
> +#define SNS1_EOC                0x20
> +#define SNS1_MESSAGE_TO_OPER    0x10
> +#define SNS1_NO_REC_FOUND       0x08
> +#define SNS1_FILE_PROTECTED     0x04
> +#define SNS1_WRITE_INHIBITED    0x02
> +#define SNS1_INPRECISE_END      0x01
> +
> +/* architectured values for third sense byte */
> +#define SNS2_REQ_INH_WRITE      0x80
> +#define SNS2_CORRECTABLE        0x40
> +#define SNS2_FIRST_LOG_ERR      0x20
> +#define SNS2_ENV_DATA_PRESENT   0x10
> +#define SNS2_INPRECISE_END      0x04
> +
> +/* 24-byte Sense fmt/msg codes */
> +#define SENSE24_FMT_PROG_SYS    0x0
> +#define SENSE24_FMT_EQUIPMENT   0x2
> +#define SENSE24_FMT_CONTROLLER  0x3
> +#define SENSE24_FMT_MISC        0xF
> +
> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
> +
> +/* basic sense response buffer layout */
> +typedef struct senseData {
> +    uint8_t status[3];
> +    uint8_t res_count;
> +    uint8_t phys_drive_id;
> +    uint8_t low_cyl_addr;
> +    uint8_t head_high_cyl_addr;
> +    uint8_t fmt_msg;
> +    uint64_t fmt_dependent_info[2];
> +    uint8_t reserved;
> +    uint8_t program_action_code;
> +    uint16_t config_info;
> +    uint8_t mcode_hicyl;
> +    uint8_t cyl_head_addr[3];
> +}  __attribute__ ((packed, aligned(4))) SenseData;
> +
> +#define SENSE24_GET_FMT(sd)     (sd->fmt_msg & 0xF0 >> 4)
> +#define SENSE24_GET_MSG(sd)     (sd->fmt_msg & 0x0F)
> +
> +#define unit_check(irb)     ((irb)->scsw.dstat & SCSW_DSTAT_UCHK)
> +
>   /* interruption response block */
>   typedef struct irb {
>       struct scsw scsw;
> @@ -215,6 +330,9 @@ typedef struct irb {
> 
>   int enable_mss_facility(void);
>   void enable_subchannel(SubChannelId schid);
> +uint16_t cu_type(SubChannelId schid);
> +void basic_sense(SubChannelId schid, SenseData *sd);
> +int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt);
> 
>   /*
>    * Some S390 specific IO instructions as inline
> diff --git a/pc-bios/s390-ccw/s390-ccw.h b/pc-bios/s390-ccw/s390-ccw.h
> index b39ee5d..11bce7d 100644
> --- a/pc-bios/s390-ccw/s390-ccw.h
> +++ b/pc-bios/s390-ccw/s390-ccw.h
> @@ -52,6 +52,7 @@ typedef unsigned long long __u64;
>   /* start.s */
>   void disabled_wait(void);
>   void consume_sclp_int(void);
> +void consume_io_int(void);
> 
>   /* main.c */
>   void panic(const char *string);
> diff --git a/pc-bios/s390-ccw/start.S b/pc-bios/s390-ccw/start.S
> index eb8d024..a48c38f 100644
> --- a/pc-bios/s390-ccw/start.S
> +++ b/pc-bios/s390-ccw/start.S
> @@ -65,12 +65,32 @@ consume_sclp_int:
>           /* prepare external call handler */
>           larl %r1, external_new_code
>           stg %r1, 0x1b8
> -        larl %r1, external_new_mask
> +        larl %r1, int_new_mask
>           mvc 0x1b0(8),0(%r1)
>           /* load enabled wait PSW */
>           larl %r1, enabled_wait_psw
>           lpswe 0(%r1)
> 
> +/*
> + * void consume_io_int(void)
> + *
> + * eats one I/O interrupt
> + */
> +        .globl consume_io_int
> +consume_io_int:
> +        /* enable I/O interrupts in cr0 */
> +        stctg 6,6,0(15)
> +        oi 4(15), 0xff
> +        lctlg 6,6,0(15)
> +        /* prepare external call handler */

shouldn't this be i/o call handler?

> +        larl %r1, io_new_code
> +        stg %r1, 0x1f8
> +        larl %r1, int_new_mask
> +        mvc 0x1f0(8),0(%r1)
> +        /* load enabled wait PSW */
> +        larl %r1, enabled_wait_psw
> +        lpswe 0(%r1)
> +
>   external_new_code:
>           /* disable service interrupts in cr0 */
>           stctg 0,0,0(15)
> @@ -78,10 +98,19 @@ external_new_code:
>           lctlg 0,0,0(15)
>           br 14
> 
> +io_new_code:
> +        /* disable I/O interrupts in cr6 */
> +        stctg 6,6,0(15)
> +        ni 4(15), 0x00
> +        lctlg 6,6,0(15)
> +        br 14
> +
> +
> +
>           .align  8
>   disabled_wait_psw:
>           .quad   0x0002000180000000,0x0000000000000000
>   enabled_wait_psw:
>           .quad   0x0302000180000000,0x0000000000000000
> -external_new_mask:
> +int_new_mask:
>           .quad   0x0000000180000000
> 

Re: [Qemu-devel] [PATCH 11/15] s390-bios: cio error handling

[Farhan Ali]

On 12/12/2018 09:11 AM, Jason J. Herne wrote:
> Add verbose error output for when unexpected i/o errors happen. This eases the
> burden of debugging and reporting i/o errors. No error information is printed
> in the success case, here is an example of what is output on error:
> 
> vfio-ccw device I/O error - Interrupt Response Block Data:
>      Function Ctrl : [Start]
>      Activity Ctrl : [Start-Pending]
>      Status Ctrl : [Alert] [Primary] [Secondary] [Status-Pending]
>      Device Status : [Unit-Check]
>      Channel Status :
>      cpa=: 0x0000000001e67098
>      prev_ccw=: 0x0000000000000000
>      this_ccw=: 0x0000000000000000
> 
> Sense Data (fmt 32-bytes):
>      Sense Condition Flags : [Equipment-Check]
>      Residual Count     =: 0x0000000000000000
>      Phys Drive ID      =: 0x000000000000009e
>      low cyl address    =: 0x0000000000000000
>      head addr & hi cyl =: 0x0000000000000000
>      format/message     =: 0x0000000000000008
>      fmt-dependent[0-7] =: 0x0000000000000004
>      fmt-dependent[8-15]=: 0xe561282305082fff
>      prog action code   =: 0x0000000000000016
>      Configuration info =: 0x00000000000040e0
>      mcode / hi-cyl     =: 0x0000000000000000
>      cyl & head addr [0]=: 0x0000000000000000
>      cyl & head addr [1]=: 0x0000000000000000
>      cyl & head addr [2]=: 0x0000000000000000
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> ---
>   pc-bios/s390-ccw/cio.c  | 225 ++++++++++++++++++++++++++++++++++++++++++++++++
>   pc-bios/s390-ccw/libc.h |  11 +++
>   2 files changed, 236 insertions(+)
> 
> diff --git a/pc-bios/s390-ccw/cio.c b/pc-bios/s390-ccw/cio.c
> index 9019250..c72e13c 100644
> --- a/pc-bios/s390-ccw/cio.c
> +++ b/pc-bios/s390-ccw/cio.c
> @@ -83,6 +83,228 @@ static bool irb_error(Irb *irb)
>       return irb->scsw.dstat != 0xc;
>   }
> 
> +static void print_sense_data(SenseData *sd)
> +{
> +    char msgline[512];
> +
> +    if (sd->config_info & 0x8000) {
> +        sclp_print("Sense Data (fmt 24-bytes):\n");
> +    } else {
> +        sclp_print("Sense Data (fmt 32-bytes):\n");
> +    }
> +
> +    strcat(msgline, "    Sense Condition Flags :");
> +    if (sd->status[0] & SNS0_CMD_REJECT) {
> +        strcat(msgline, " [Cmd-Reject]");
> +    }
> +    if (sd->status[0] & SNS0_INTERVENTION_REQ) {
> +        strcat(msgline, " [Intervention-Required]");
> +    }
> +    if (sd->status[0] & SNS0_BUS_OUT_CHECK) {
> +        strcat(msgline, " [Bus-Out-Parity-Check]");
> +    }
> +    if (sd->status[0] & SNS0_EQUIPMENT_CHECK) {
> +        strcat(msgline, " [Equipment-Check]");
> +    }
> +    if (sd->status[0] & SNS0_DATA_CHECK) {
> +        strcat(msgline, " [Data-Check]");
> +    }
> +    if (sd->status[0] & SNS0_OVERRUN) {
> +        strcat(msgline, " [Overrun]");
> +    }
> +    if (sd->status[0] & SNS0_INCOMPL_DOMAIN) {
> +        strcat(msgline, " [Incomplete-Domain]");
> +    }
> +
> +    if (sd->status[1] & SNS1_PERM_ERR) {
> +        strcat(msgline, " [Permanent-Error]");
> +    }
> +    if (sd->status[1] & SNS1_INV_TRACK_FORMAT) {
> +        strcat(msgline, " [Invalid-Track-Fmt]");
> +    }
> +    if (sd->status[1] & SNS1_EOC) {
> +        strcat(msgline, " [End-of-Cyl]");
> +    }
> +    if (sd->status[1] & SNS1_MESSAGE_TO_OPER) {
> +        strcat(msgline, " [Operator-Msg]");
> +    }
> +    if (sd->status[1] & SNS1_NO_REC_FOUND) {
> +        strcat(msgline, " [No-Record-Found]");
> +    }
> +    if (sd->status[1] & SNS1_FILE_PROTECTED) {
> +        strcat(msgline, " [File-Protected]");
> +    }
> +    if (sd->status[1] & SNS1_WRITE_INHIBITED) {
> +        strcat(msgline, " [Write-Inhibited]");
> +    }
> +    if (sd->status[1] & SNS1_INPRECISE_END) {
> +        strcat(msgline, " [Imprecise-Ending]");
> +    }
> +
> +    if (sd->status[2] & SNS2_REQ_INH_WRITE) {
> +        strcat(msgline, " [Req-Inhibit-Write]");
> +    }
> +    if (sd->status[2] & SNS2_CORRECTABLE) {
> +        strcat(msgline, " [Correctable-Data-Check]");
> +    }
> +    if (sd->status[2] & SNS2_FIRST_LOG_ERR) {
> +        strcat(msgline, " [First-Error-Log]");
> +    }
> +    if (sd->status[2] & SNS2_ENV_DATA_PRESENT) {
> +        strcat(msgline, " [Env-Data-Present]");
> +    }
> +    if (sd->status[2] & SNS2_INPRECISE_END) {
> +        strcat(msgline, " [Imprecise-End]");
> +    }
> +    strcat(msgline, "\n");
> +    sclp_print(msgline);
> +
> +    print_int("    Residual Count     =", sd->res_count);
> +    print_int("    Phys Drive ID      =", sd->phys_drive_id);
> +    print_int("    low cyl address    =", sd->low_cyl_addr);
> +    print_int("    head addr & hi cyl =", sd->head_high_cyl_addr);
> +    print_int("    format/message     =", sd->fmt_msg);
> +    print_int("    fmt-dependent[0-7] =", sd->fmt_dependent_info[0]);
> +    print_int("    fmt-dependent[8-15]=", sd->fmt_dependent_info[1]);
> +    print_int("    prog action code   =", sd->program_action_code);
> +    print_int("    Configuration info =", sd->config_info);
> +    print_int("    mcode / hi-cyl     =", sd->mcode_hicyl);
> +    print_int("    cyl & head addr [0]=", sd->cyl_head_addr[0]);
> +    print_int("    cyl & head addr [1]=", sd->cyl_head_addr[1]);
> +    print_int("    cyl & head addr [2]=", sd->cyl_head_addr[2]);
> +}
> +
> +static void print_irb_err(Irb *irb)
> +{
> +    Ccw0 *this_ccw = u32toptr(irb->scsw.cpa);
> +    Ccw0 *prev_ccw = u32toptr(irb->scsw.cpa - 8);
> +    char msgline[256];
> +
> +    sclp_print("vfio-ccw device I/O error - Interrupt Response Block Data:\n");
> +
> +    strcat(msgline, "    Function Ctrl :");
> +    if (irb->scsw.ctrl & SCSW_FCTL_START_FUNC) {
> +        strcat(msgline, " [Start]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_FCTL_HALT_FUNC) {
> +        strcat(msgline, " [Halt]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_FCTL_CLEAR_FUNC) {
> +        strcat(msgline, " [Clear]");
> +    }
> +    strcat(msgline, "\n");
> +    sclp_print(msgline);
> +
> +    msgline[0] = '\0';
> +    strcat(msgline, "    Activity Ctrl :");
> +    if (irb->scsw.ctrl & SCSW_ACTL_RESUME_PEND) {
> +        strcat(msgline, " [Resume-Pending]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_ACTL_START_PEND) {
> +        strcat(msgline, " [Start-Pending]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_ACTL_HALT_PEND) {
> +        strcat(msgline, " [Halt-Pending]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_ACTL_CLEAR_PEND) {
> +        strcat(msgline, " [Clear-Pending]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_ACTL_CH_ACTIVE) {
> +        strcat(msgline, " [Channel-Active]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_ACTL_DEV_ACTIVE) {
> +        strcat(msgline, " [Device-Active]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_ACTL_SUSPENDED) {
> +        strcat(msgline, " [Suspended]");
> +    }
> +    strcat(msgline, "\n");
> +    sclp_print(msgline);
> +
> +    msgline[0] = '\0';
> +    strcat(msgline, "    Status Ctrl :");
> +    if (irb->scsw.ctrl & SCSW_SCTL_ALERT) {
> +        strcat(msgline, " [Alert]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_SCTL_INTERMED) {
> +        strcat(msgline, " [Intermediate]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_SCTL_PRIMARY) {
> +        strcat(msgline, " [Primary]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_SCTL_SECONDARY) {
> +        strcat(msgline, " [Secondary]");
> +    }
> +    if (irb->scsw.ctrl & SCSW_SCTL_STATUS_PEND) {
> +        strcat(msgline, " [Status-Pending]");
> +    }
> +
> +    strcat(msgline, "\n");
> +    sclp_print(msgline);
> +
> +    msgline[0] = '\0';
> +    strcat(msgline, "    Device Status :");
> +    if (irb->scsw.dstat & SCSW_DSTAT_ATTN) {
> +        strcat(msgline, " [Attention]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_STATMOD) {
> +        strcat(msgline, " [Status-Modifier]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_CUEND) {
> +        strcat(msgline, " [Ctrl-Unit-End]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_BUSY) {
> +        strcat(msgline, " [Busy]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_CHEND) {
> +        strcat(msgline, " [Channel-End]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_DEVEND) {
> +        strcat(msgline, " [Device-End]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_UCHK) {
> +        strcat(msgline, " [Unit-Check]");
> +    }
> +    if (irb->scsw.dstat & SCSW_DSTAT_UEXCP) {
> +        strcat(msgline, " [Unit-Exception]");
> +    }
> +    strcat(msgline, "\n");
> +    sclp_print(msgline);
> +
> +    msgline[0] = '\0';
> +    strcat(msgline, "    Channel Status :");
> +    if (irb->scsw.cstat & SCSW_CSTAT_PCINT) {
> +        strcat(msgline, " [Program-Ctrl-Interruption]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_BADLEN) {
> +        strcat(msgline, " [Incorrect-Length]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_PROGCHK) {
> +        strcat(msgline, " [Program-Check]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_PROTCHK) {
> +        strcat(msgline, " [Protection-Check]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_CHDCHK) {
> +        strcat(msgline, " [Channel-Data-Check]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_CHCCHK) {
> +        strcat(msgline, " [Channel-Ctrl-Check]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_ICCHK) {
> +        strcat(msgline, " [Interface-Ctrl-Check]");
> +    }
> +    if (irb->scsw.cstat & SCSW_CSTAT_CHAINCHK) {
> +        strcat(msgline, " [Chaining-Check]");
> +    }
> +    strcat(msgline, "\n");
> +    sclp_print(msgline);
> +
> +    print_int("    cpa=", irb->scsw.cpa);
> +    print_int("    prev_ccw=", *((uint64_t *)prev_ccw));
> +    print_int("    this_ccw=", *((uint64_t *)this_ccw));
> +}
> +
>   /* Executes a channel program at a given subchannel. The request to run the
>    * channel program is sent to the subchannel, we then wait for the interrupt
>    * singaling completion of the I/O operation(s) perfomed by the channel
> @@ -142,6 +364,9 @@ int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
>               continue;
>           }
> 
> +        print_irb_err(&irb);
> +        basic_sense(schid, &sd);
> +        print_sense_data(&sd);
>           break;
>       }
> 
> diff --git a/pc-bios/s390-ccw/libc.h b/pc-bios/s390-ccw/libc.h
> index e198f0b..01b5de0 100644
> --- a/pc-bios/s390-ccw/libc.h
> +++ b/pc-bios/s390-ccw/libc.h
> @@ -79,6 +79,17 @@ static inline size_t strlen(const char *str)
>       return i;
>   }
> 
> +static inline char *strcat(char *dest, const char *src)
> +{
> +    int i;
> +    char *dest_end = dest + strlen(dest);
> +
> +    for (i = 0; i <= strlen(src); i++) {
> +        dest_end[i] = src[i];
> +    }
> +    return dest;
> +}
> +

We are not doing any sort of buffer overflow checks here, I think it 
would be nice to have some sort of checks.

>   static inline int isdigit(int c)
>   {
>       return (c >= '0') && (c <= '9');
> 

Re: [Qemu-devel] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Cornelia Huck]

On Wed, 12 Dec 2018 09:11:13 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> Add struct for format-0 ccws. Support executing format-0 channel
> programs and waiting for their completion before continuing execution.
> This will be used for real dasd ipl.
> 
> Add cu_type() to channel io library. This will be used to query control
> unit type which is used to determine if we are booting a virtio device or a
> real dasd device.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> ---
>  pc-bios/s390-ccw/cio.c      | 108 ++++++++++++++++++++++++++++++++++++++
>  pc-bios/s390-ccw/cio.h      | 124 ++++++++++++++++++++++++++++++++++++++++++--
>  pc-bios/s390-ccw/s390-ccw.h |   1 +
>  pc-bios/s390-ccw/start.S    |  33 +++++++++++-
>  4 files changed, 261 insertions(+), 5 deletions(-)
> 

(...)

> +static bool irb_error(Irb *irb)
> +{
> +    /* We have to ignore Incorrect Length (cstat == 0x40) indicators because
> +     * real devices expect a 24 byte SenseID  buffer, and virtio devices expect
> +     * a much larger buffer. Neither device type can tolerate a buffer size
> +     * different from what they expect so they set this indicator.

Hm, can't you specify SLI for SenseID?

> +     */
> +    if (irb->scsw.cstat != 0x00 && irb->scsw.cstat != 0x40) {
> +        return true;
> +    }
> +    return irb->scsw.dstat != 0xc;

Also, shouldn't you actually use the #defines you introduce further
down?

> +}
> +
> +/* Executes a channel program at a given subchannel. The request to run the
> + * channel program is sent to the subchannel, we then wait for the interrupt
> + * singaling completion of the I/O operation(s) perfomed by the channel
> + * program. Lastly we verify that the i/o operation completed without error and
> + * that the interrupt we received was for the subchannel used to run the
> + * channel program.
> + *
> + * Note: This function assumes it is running in an environment where no other
> + * cpus are generating or receiving I/O interrupts. So either run it in a
> + * single-cpu environment or make sure all other cpus are not doing I/O and
> + * have I/O interrupts masked off.

Anything about iscs here (cr6)?

> + */
> +int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
> +{
> +    CmdOrb orb = {};
> +    Irb irb = {};
> +    SenseData sd;
> +    int rc, retries = 0;
> +
> +    IPL_assert(fmt == 0 || fmt == 1, "Invalid ccw format");
> +
> +    /* ccw_addr must be <= 24 bits and point to at least one whole ccw. */
> +    if (fmt == 0) {
> +        IPL_assert(ccw_addr <= 0xFFFFFF - 8, "Invalid ccw address");
> +    }
> +
> +    orb.fmt = fmt ;
> +    orb.pfch = 1;  /* QEMU's cio implementation requires prefetch */
> +    orb.c64 = 1;   /* QEMU's cio implementation requires 64-bit idaws */
> +    orb.lpm = 0xFF; /* All paths allowed */
> +    orb.cpa = ccw_addr;
> +
> +    while (true) {
> +        rc = ssch(schid, &orb);

I think we can get here:
- cc 0 -> all ok
- cc 1 -> status pending; could that be an unsolicited interrupt from
  the device? or would we always get a deferred cc 1 in that case?
- cc 2 -> another function pending; Should Not Happen
- cc 3 -> it's dead, Jim

So I'm wondering whether we should consume the status and retry for cc
1. The handling of the others is fine.

> +        if (rc) {
> +            print_int("ssch failed with rc=", rc);
> +            break;
> +        }
> +
> +        consume_io_int();
> +
> +        /* Clear read */

I find that comment confusing. /* collect status */ maybe?

> +        rc = tsch(schid, &irb);

Here we can get:
- cc 0 -> status pending, all ok
- cc 1 -> no status pending, Should Not Happen
- cc 3 -> it's dead, Jim

So this looks fine.

> +        if (rc) {
> +            print_int("tsch failed with rc=", rc);
> +            break;
> +        }
> +
> +        if (!irb_error(&irb)) {
> +            break;
> +        }
> +
> +        /* Unexpected unit check. Use sense to clear unit check then retry. */

The dasds still don't support concurrent sense, do they? Might also be
worth investigating whether some unit checks are more "recoverable"
than others.

I expect we simply want to ignore IFCCs? IIRC, the strategy for those
is "retry, in case it is transient"; but that may take some time. Or
was there some path handling to be considered? (i.e., retrying may
select another path, which may be fine.)

> +        if (unit_check(&irb) && retries <= 2) {
> +            basic_sense(schid, &sd);
> +            retries++;
> +            continue;
> +        }
> +
> +        break;
> +    }
> +
> +    return rc;
> +}

(...)

> @@ -190,6 +247,9 @@ struct ciw {
>      __u16 count;
>  };
>  
> +#define CU_TYPE_VIRTIO          0x3832
> +#define CU_TYPE_DASD            0x3990

No other dasd types we want to support? :) (Not sure if others are out
in the wild. Maybe FBA?)

> +
>  /*
>   * sense-id response buffer layout
>   */
> @@ -205,6 +265,61 @@ typedef struct senseid {
>      struct ciw ciw[62];
>  }  __attribute__ ((packed, aligned(4))) SenseId;
>  
> +/* architected values for first sense byte */
> +#define SNS0_CMD_REJECT         0x80
> +#define SNS0_INTERVENTION_REQ   0x40
> +#define SNS0_BUS_OUT_CHECK      0x20
> +#define SNS0_EQUIPMENT_CHECK    0x10
> +#define SNS0_DATA_CHECK         0x08
> +#define SNS0_OVERRUN            0x04
> +#define SNS0_INCOMPL_DOMAIN     0x01

IIRC, only byte 0 is device independent, and the others below are
(ECKD) dasd specific?

> +
> +/* architectured values for second sense byte */
> +#define SNS1_PERM_ERR           0x80
> +#define SNS1_INV_TRACK_FORMAT   0x40
> +#define SNS1_EOC                0x20
> +#define SNS1_MESSAGE_TO_OPER    0x10
> +#define SNS1_NO_REC_FOUND       0x08
> +#define SNS1_FILE_PROTECTED     0x04
> +#define SNS1_WRITE_INHIBITED    0x02
> +#define SNS1_INPRECISE_END      0x01
> +
> +/* architectured values for third sense byte */
> +#define SNS2_REQ_INH_WRITE      0x80
> +#define SNS2_CORRECTABLE        0x40
> +#define SNS2_FIRST_LOG_ERR      0x20
> +#define SNS2_ENV_DATA_PRESENT   0x10
> +#define SNS2_INPRECISE_END      0x04
> +
> +/* 24-byte Sense fmt/msg codes */
> +#define SENSE24_FMT_PROG_SYS    0x0
> +#define SENSE24_FMT_EQUIPMENT   0x2
> +#define SENSE24_FMT_CONTROLLER  0x3
> +#define SENSE24_FMT_MISC        0xF
> +
> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
> +
> +/* basic sense response buffer layout */
> +typedef struct senseData {
> +    uint8_t status[3];
> +    uint8_t res_count;
> +    uint8_t phys_drive_id;
> +    uint8_t low_cyl_addr;
> +    uint8_t head_high_cyl_addr;
> +    uint8_t fmt_msg;
> +    uint64_t fmt_dependent_info[2];
> +    uint8_t reserved;
> +    uint8_t program_action_code;
> +    uint16_t config_info;
> +    uint8_t mcode_hicyl;
> +    uint8_t cyl_head_addr[3];
> +}  __attribute__ ((packed, aligned(4))) SenseData;

And this looks _really_ dasd specific.

> +
> +#define SENSE24_GET_FMT(sd)     (sd->fmt_msg & 0xF0 >> 4)
> +#define SENSE24_GET_MSG(sd)     (sd->fmt_msg & 0x0F)
> +
> +#define unit_check(irb)     ((irb)->scsw.dstat & SCSW_DSTAT_UCHK)
> +
>  /* interruption response block */
>  typedef struct irb {
>      struct scsw scsw;

(...)

> diff --git a/pc-bios/s390-ccw/start.S b/pc-bios/s390-ccw/start.S
> index eb8d024..a48c38f 100644
> --- a/pc-bios/s390-ccw/start.S
> +++ b/pc-bios/s390-ccw/start.S
> @@ -65,12 +65,32 @@ consume_sclp_int:
>          /* prepare external call handler */
>          larl %r1, external_new_code
>          stg %r1, 0x1b8
> -        larl %r1, external_new_mask
> +        larl %r1, int_new_mask
>          mvc 0x1b0(8),0(%r1)
>          /* load enabled wait PSW */
>          larl %r1, enabled_wait_psw
>          lpswe 0(%r1)
>  
> +/*
> + * void consume_io_int(void)
> + *
> + * eats one I/O interrupt

*nomnom*

> + */
> +        .globl consume_io_int
> +consume_io_int:
> +        /* enable I/O interrupts in cr0 */

cr6?

> +        stctg 6,6,0(15)
> +        oi 4(15), 0xff
> +        lctlg 6,6,0(15)
> +        /* prepare external call handler */

I/O call handler?

> +        larl %r1, io_new_code
> +        stg %r1, 0x1f8
> +        larl %r1, int_new_mask
> +        mvc 0x1f0(8),0(%r1)
> +        /* load enabled wait PSW */
> +        larl %r1, enabled_wait_psw
> +        lpswe 0(%r1)
> +
>  external_new_code:
>          /* disable service interrupts in cr0 */
>          stctg 0,0,0(15)
> @@ -78,10 +98,19 @@ external_new_code:
>          lctlg 0,0,0(15)
>          br 14
>  
> +io_new_code:
> +        /* disable I/O interrupts in cr6 */
> +        stctg 6,6,0(15)

I'm wondering why you are changing cr6 every time you wait for an I/O
interrupt. Just enable the isc(s) you want once, and disable them again
after you're done with all I/O? Simply disabling the I/O interrupts
should be enough to prevent further interrupts popping up. You maybe
want two enabled wait PSWs, one with I/O + external and one with
external only?

> +        ni 4(15), 0x00
> +        lctlg 6,6,0(15)
> +        br 14
> +
> +
> +
>          .align  8
>  disabled_wait_psw:
>          .quad   0x0002000180000000,0x0000000000000000
>  enabled_wait_psw:
>          .quad   0x0302000180000000,0x0000000000000000
> -external_new_mask:
> +int_new_mask:
>          .quad   0x0000000180000000

Re: [Qemu-devel] [PATCH 11/15] s390-bios: cio error handling

[Cornelia Huck]

On Wed, 12 Dec 2018 09:11:14 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> Add verbose error output for when unexpected i/o errors happen. This eases the
> burden of debugging and reporting i/o errors. No error information is printed
> in the success case, here is an example of what is output on error:
> 
> vfio-ccw device I/O error - Interrupt Response Block Data:
>     Function Ctrl : [Start]
>     Activity Ctrl : [Start-Pending]
>     Status Ctrl : [Alert] [Primary] [Secondary] [Status-Pending]
>     Device Status : [Unit-Check]
>     Channel Status :
>     cpa=: 0x0000000001e67098
>     prev_ccw=: 0x0000000000000000
>     this_ccw=: 0x0000000000000000
> 
> Sense Data (fmt 32-bytes):
>     Sense Condition Flags : [Equipment-Check]
>     Residual Count     =: 0x0000000000000000
>     Phys Drive ID      =: 0x000000000000009e
>     low cyl address    =: 0x0000000000000000
>     head addr & hi cyl =: 0x0000000000000000
>     format/message     =: 0x0000000000000008
>     fmt-dependent[0-7] =: 0x0000000000000004
>     fmt-dependent[8-15]=: 0xe561282305082fff
>     prog action code   =: 0x0000000000000016
>     Configuration info =: 0x00000000000040e0
>     mcode / hi-cyl     =: 0x0000000000000000
>     cyl & head addr [0]=: 0x0000000000000000
>     cyl & head addr [1]=: 0x0000000000000000
>     cyl & head addr [2]=: 0x0000000000000000

Even though we currently only support eckd dasd passed through via
vfio-ccw, it is probably not a good idea to hardcode this?

> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> ---
>  pc-bios/s390-ccw/cio.c  | 225 ++++++++++++++++++++++++++++++++++++++++++++++++
>  pc-bios/s390-ccw/libc.h |  11 +++
>  2 files changed, 236 insertions(+)

Re: [Qemu-devel] [PATCH 12/15] s390-bios: Refactor virtio to run channel programs via cio

[Cornelia Huck]

On Wed, 12 Dec 2018 09:11:15 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> Now that we have a Channel I/O library let's modify virtio boot code to
> make use of it for running channel programs.
> 
> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> ---
>  pc-bios/s390-ccw/virtio.c | 21 ++-------------------
>  1 file changed, 2 insertions(+), 19 deletions(-)
> 
> diff --git a/pc-bios/s390-ccw/virtio.c b/pc-bios/s390-ccw/virtio.c
> index aa9da72..5fb49e2 100644
> --- a/pc-bios/s390-ccw/virtio.c
> +++ b/pc-bios/s390-ccw/virtio.c
> @@ -92,30 +92,13 @@ int drain_irqs(SubChannelId schid)
>  static int run_ccw(VDev *vdev, int cmd, void *ptr, int len)
>  {
>      Ccw1 ccw = {};
> -    CmdOrb orb = {};
> -    int r;
> -
> -    enable_subchannel(vdev->schid);
> -
> -    /* start subchannel command */
> -    orb.fmt = 1;
> -    orb.cpa = (u32)(long)&ccw;
> -    orb.lpm = 0x80;
>  
>      ccw.cmd_code = cmd;
>      ccw.cda = (long)ptr;
>      ccw.count = len;
>  
> -    r = ssch(vdev->schid, &orb);
> -    /*
> -     * XXX Wait until device is done processing the CCW. For now we can
> -     *     assume that a simple tsch will have finished the CCW processing,
> -     *     but the architecture allows for asynchronous operation
> -     */
> -    if (!r) {
> -        r = drain_irqs(vdev->schid);
> -    }
> -    return r;
> +    enable_subchannel(vdev->schid);

Preexisting, but I'm wondering why we enable the subchannel here and
not just once?

> +    return do_cio(vdev->schid, ptr2u32(&ccw), CCW_FMT1);
>  }
>  
>  static void vring_init(VRing *vr, VqInfo *info)

Re: [Qemu-devel] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

On 12/13/18 12:21 PM, Cornelia Huck wrote:
> On Wed, 12 Dec 2018 09:11:13 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
>> Add struct for format-0 ccws. Support executing format-0 channel
>> programs and waiting for their completion before continuing execution.
>> This will be used for real dasd ipl.
>>
>> Add cu_type() to channel io library. This will be used to query control
>> unit type which is used to determine if we are booting a virtio device or a
>> real dasd device.
>>
>> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
>> ---
>>   pc-bios/s390-ccw/cio.c      | 108 ++++++++++++++++++++++++++++++++++++++
>>   pc-bios/s390-ccw/cio.h      | 124 ++++++++++++++++++++++++++++++++++++++++++--
>>   pc-bios/s390-ccw/s390-ccw.h |   1 +
>>   pc-bios/s390-ccw/start.S    |  33 +++++++++++-
>>   4 files changed, 261 insertions(+), 5 deletions(-)
>>
> 
> (...)
> 
>> +static bool irb_error(Irb *irb)
>> +{
>> +    /* We have to ignore Incorrect Length (cstat == 0x40) indicators because
>> +     * real devices expect a 24 byte SenseID  buffer, and virtio devices expect
>> +     * a much larger buffer. Neither device type can tolerate a buffer size
>> +     * different from what they expect so they set this indicator.
> 
> Hm, can't you specify SLI for SenseID?
> 

Yes, but this requires modifying run_ccw() in virtio.c to always specify the SLI flag. I'm 
not sure that is the best choice? I suppose I could add an sli argument to run_ccw if 
you'd prefer that.

>> +     */
>> +    if (irb->scsw.cstat != 0x00 && irb->scsw.cstat != 0x40) {
>> +        return true;
>> +    }
>> +    return irb->scsw.dstat != 0xc;
> 
> Also, shouldn't you actually use the #defines you introduce further
> down?
> 

Yep, I added the defines after I wrote this code. I'll fix that.

>> +}
>> +
>> +/* Executes a channel program at a given subchannel. The request to run the
>> + * channel program is sent to the subchannel, we then wait for the interrupt
>> + * singaling completion of the I/O operation(s) perfomed by the channel
>> + * program. Lastly we verify that the i/o operation completed without error and
>> + * that the interrupt we received was for the subchannel used to run the
>> + * channel program.
>> + *
>> + * Note: This function assumes it is running in an environment where no other
>> + * cpus are generating or receiving I/O interrupts. So either run it in a
>> + * single-cpu environment or make sure all other cpus are not doing I/O and
>> + * have I/O interrupts masked off.
> 
> Anything about iscs here (cr6)?
> 

Those details are handled in the assembler code. Do you think I should mention something 
about cr6 here?

>> + */
>> +int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
>> +{
>> +    CmdOrb orb = {};
>> +    Irb irb = {};
>> +    SenseData sd;
>> +    int rc, retries = 0;
>> +
>> +    IPL_assert(fmt == 0 || fmt == 1, "Invalid ccw format");
>> +
>> +    /* ccw_addr must be <= 24 bits and point to at least one whole ccw. */
>> +    if (fmt == 0) {
>> +        IPL_assert(ccw_addr <= 0xFFFFFF - 8, "Invalid ccw address");
>> +    }
>> +
>> +    orb.fmt = fmt ;
>> +    orb.pfch = 1;  /* QEMU's cio implementation requires prefetch */
>> +    orb.c64 = 1;   /* QEMU's cio implementation requires 64-bit idaws */
>> +    orb.lpm = 0xFF; /* All paths allowed */
>> +    orb.cpa = ccw_addr;
>> +
>> +    while (true) {
>> +        rc = ssch(schid, &orb);
> 
> I think we can get here:
> - cc 0 -> all ok
> - cc 1 -> status pending; could that be an unsolicited interrupt from
>    the device? or would we always get a deferred cc 1 in that case?
> - cc 2 -> another function pending; Should Not Happen
> - cc 3 -> it's dead, Jim
> 
> So I'm wondering whether we should consume the status and retry for cc
> 1. The handling of the others is fine.
> 

I took a look at css_do_ssch() in hw/s390x/css.c and it appears as though CC1 is a 
possibility here. I'm not against taking action, but I suspect we would have to clear the 
status with a basic sense (or something) before simply retrying... right?

Is it safe for us to just assume we can clear it and move on? It seems like an edge case 
that we'd be better off failing on. Perhaps let the user try again which will redrive the 
process?


>> +        if (rc) {
>> +            print_int("ssch failed with rc=", rc);
>> +            break;
>> +        }
>> +
>> +        consume_io_int();
>> +
>> +        /* Clear read */
> 
> I find that comment confusing. /* collect status */ maybe?
> 
>> +        rc = tsch(schid, &irb);
> 
> Here we can get:
> - cc 0 -> status pending, all ok
> - cc 1 -> no status pending, Should Not Happen
> - cc 3 -> it's dead, Jim
> 
> So this looks fine.
> 
>> +        if (rc) {
>> +            print_int("tsch failed with rc=", rc);
>> +            break;
>> +        }
>> +
>> +        if (!irb_error(&irb)) {
>> +            break;
>> +        }
>> +
>> +        /* Unexpected unit check. Use sense to clear unit check then retry. */
> 
> The dasds still don't support concurrent sense, do they? Might also be
> worth investigating whether some unit checks are more "recoverable"
> than others.
> 

I wasn't sure on concurrent sense. I'd bet there are situations or environments where it 
won't be supported so it seems safest to assume we don't have it.

We already recover from the one unit check scenario I've discovered in practice (initial 
reset). And the algorithm I chose is to simply retry a few times whenever we're presented 
with unexpected unit check status. This is what the kernel does. It seems fairly robust.


> I expect we simply want to ignore IFCCs? IIRC, the strategy for those
> is "retry, in case it is transient"; but that may take some time. Or
> was there some path handling to be considered? (i.e., retrying may
> select another path, which may be fine.)
> 

Currently we'll give up on IFCC. I think this is the right thing to do. A user can always 
retry if they want. But in reality an IFCC very likely means faulty hardware IIUC.

I've not thought about path management much. I suspect paths changing isn't something we 
should realistically see in the bios. Even still, a retry is really all we can do, so 
assuming path changes result in a unit check then we should be okay there.


>> +        if (unit_check(&irb) && retries <= 2) {
>> +            basic_sense(schid, &sd);
>> +            retries++;
>> +            continue;
>> +        }
>> +
>> +        break;
>> +    }
>> +
>> +    return rc;
>> +}
> 
> (...)
> 
>> @@ -190,6 +247,9 @@ struct ciw {
>>       __u16 count;
>>   };
>>   
>> +#define CU_TYPE_VIRTIO          0x3832
>> +#define CU_TYPE_DASD            0x3990
> 
> No other dasd types we want to support? :) (Not sure if others are out
> in the wild. Maybe FBA?)
>

I have no idea. I assumed 3390 was the only thing we supported. Perhaps 3380? I'd need to 
find a test device, which I could probably do ... I'll look more into this.


>> +
>>   /*
>>    * sense-id response buffer layout
>>    */
>> @@ -205,6 +265,61 @@ typedef struct senseid {
>>       struct ciw ciw[62];
>>   }  __attribute__ ((packed, aligned(4))) SenseId;
>>   
>> +/* architected values for first sense byte */
>> +#define SNS0_CMD_REJECT         0x80
>> +#define SNS0_INTERVENTION_REQ   0x40
>> +#define SNS0_BUS_OUT_CHECK      0x20
>> +#define SNS0_EQUIPMENT_CHECK    0x10
>> +#define SNS0_DATA_CHECK         0x08
>> +#define SNS0_OVERRUN            0x04
>> +#define SNS0_INCOMPL_DOMAIN     0x01
> 
> IIRC, only byte 0 is device independent, and the others below are
> (ECKD) dasd specific?
> 
>> +
>> +/* architectured values for second sense byte */
>> +#define SNS1_PERM_ERR           0x80
>> +#define SNS1_INV_TRACK_FORMAT   0x40
>> +#define SNS1_EOC                0x20
>> +#define SNS1_MESSAGE_TO_OPER    0x10
>> +#define SNS1_NO_REC_FOUND       0x08
>> +#define SNS1_FILE_PROTECTED     0x04
>> +#define SNS1_WRITE_INHIBITED    0x02
>> +#define SNS1_INPRECISE_END      0x01
>> +
>> +/* architectured values for third sense byte */
>> +#define SNS2_REQ_INH_WRITE      0x80
>> +#define SNS2_CORRECTABLE        0x40
>> +#define SNS2_FIRST_LOG_ERR      0x20
>> +#define SNS2_ENV_DATA_PRESENT   0x10
>> +#define SNS2_INPRECISE_END      0x04
>> +
>> +/* 24-byte Sense fmt/msg codes */
>> +#define SENSE24_FMT_PROG_SYS    0x0
>> +#define SENSE24_FMT_EQUIPMENT   0x2
>> +#define SENSE24_FMT_CONTROLLER  0x3
>> +#define SENSE24_FMT_MISC        0xF
>> +
>> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
>> +
>> +/* basic sense response buffer layout */
>> +typedef struct senseData {
>> +    uint8_t status[3];
>> +    uint8_t res_count;
>> +    uint8_t phys_drive_id;
>> +    uint8_t low_cyl_addr;
>> +    uint8_t head_high_cyl_addr;
>> +    uint8_t fmt_msg;
>> +    uint64_t fmt_dependent_info[2];
>> +    uint8_t reserved;
>> +    uint8_t program_action_code;
>> +    uint16_t config_info;
>> +    uint8_t mcode_hicyl;
>> +    uint8_t cyl_head_addr[3];
>> +}  __attribute__ ((packed, aligned(4))) SenseData;
> 
> And this looks _really_ dasd specific.
> 

Yep, I glossed over those details while I was furiously tracking down the reset bug. I'll 
take a look at redesigning this.

>> +
>> +#define SENSE24_GET_FMT(sd)     (sd->fmt_msg & 0xF0 >> 4)
>> +#define SENSE24_GET_MSG(sd)     (sd->fmt_msg & 0x0F)
>> +
>> +#define unit_check(irb)     ((irb)->scsw.dstat & SCSW_DSTAT_UCHK)
>> +
>>   /* interruption response block */
>>   typedef struct irb {
>>       struct scsw scsw;
> 
> (...)
> 
>> diff --git a/pc-bios/s390-ccw/start.S b/pc-bios/s390-ccw/start.S
>> index eb8d024..a48c38f 100644
>> --- a/pc-bios/s390-ccw/start.S
>> +++ b/pc-bios/s390-ccw/start.S
>> @@ -65,12 +65,32 @@ consume_sclp_int:
>>           /* prepare external call handler */
>>           larl %r1, external_new_code
>>           stg %r1, 0x1b8
>> -        larl %r1, external_new_mask
>> +        larl %r1, int_new_mask
>>           mvc 0x1b0(8),0(%r1)
>>           /* load enabled wait PSW */
>>           larl %r1, enabled_wait_psw
>>           lpswe 0(%r1)
>>   
>> +/*
>> + * void consume_io_int(void)
>> + *
>> + * eats one I/O interrupt
> 
> *nomnom*
> 
>> + */
>> +        .globl consume_io_int
>> +consume_io_int:
>> +        /* enable I/O interrupts in cr0 */
> 
> cr6?
> 
>> +        stctg 6,6,0(15)
>> +        oi 4(15), 0xff
>> +        lctlg 6,6,0(15)
>> +        /* prepare external call handler */
> 
> I/O call handler?
> 

Both copy/paste errors. Thanks for catching these. :)

>> +        larl %r1, io_new_code
>> +        stg %r1, 0x1f8
>> +        larl %r1, int_new_mask
>> +        mvc 0x1f0(8),0(%r1)
>> +        /* load enabled wait PSW */
>> +        larl %r1, enabled_wait_psw
>> +        lpswe 0(%r1)
>> +
>>   external_new_code:
>>           /* disable service interrupts in cr0 */
>>           stctg 0,0,0(15)
>> @@ -78,10 +98,19 @@ external_new_code:
>>           lctlg 0,0,0(15)
>>           br 14
>>   
>> +io_new_code:
>> +        /* disable I/O interrupts in cr6 */
>> +        stctg 6,6,0(15)
> 
> I'm wondering why you are changing cr6 every time you wait for an I/O
> interrupt. Just enable the isc(s) you want once, and disable them again
> after you're done with all I/O? Simply disabling the I/O interrupts
> should be enough to prevent further interrupts popping up. You maybe
> want two enabled wait PSWs, one with I/O + external and one with
> external only?
> 

No real reason. We only come through here a hand full of times so performance is not a 
consideration. I guess my thought process was probably to keep the system is as close to 
initial state as possible through the ipl process. Eventually when we hand control to the 
guest OS we want the system as close to undisturbed as possible. If you think I should 
only be setting cr-6 once, it sounds reasonable.


>> +        ni 4(15), 0x00
>> +        lctlg 6,6,0(15)
>> +        br 14
>> +
>> +
>> +
>>           .align  8
>>   disabled_wait_psw:
>>           .quad   0x0002000180000000,0x0000000000000000
>>   enabled_wait_psw:
>>           .quad   0x0302000180000000,0x0000000000000000
>> -external_new_mask:
>> +int_new_mask:
>>           .quad   0x0000000180000000
> 
> 


-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Cornelia Huck]

On Mon, 7 Jan 2019 14:02:45 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 12/13/18 12:21 PM, Cornelia Huck wrote:
> > On Wed, 12 Dec 2018 09:11:13 -0500
> > "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> >   
> >> Add struct for format-0 ccws. Support executing format-0 channel
> >> programs and waiting for their completion before continuing execution.
> >> This will be used for real dasd ipl.
> >>
> >> Add cu_type() to channel io library. This will be used to query control
> >> unit type which is used to determine if we are booting a virtio device or a
> >> real dasd device.
> >>
> >> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> >> ---
> >>   pc-bios/s390-ccw/cio.c      | 108 ++++++++++++++++++++++++++++++++++++++
> >>   pc-bios/s390-ccw/cio.h      | 124 ++++++++++++++++++++++++++++++++++++++++++--
> >>   pc-bios/s390-ccw/s390-ccw.h |   1 +
> >>   pc-bios/s390-ccw/start.S    |  33 +++++++++++-
> >>   4 files changed, 261 insertions(+), 5 deletions(-)
> >>  
> > 
> > (...)
> >   
> >> +static bool irb_error(Irb *irb)
> >> +{
> >> +    /* We have to ignore Incorrect Length (cstat == 0x40) indicators because
> >> +     * real devices expect a 24 byte SenseID  buffer, and virtio devices expect
> >> +     * a much larger buffer. Neither device type can tolerate a buffer size
> >> +     * different from what they expect so they set this indicator.  
> > 
> > Hm, can't you specify SLI for SenseID?
> >   
> 
> Yes, but this requires modifying run_ccw() in virtio.c to always specify the SLI flag. I'm 
> not sure that is the best choice? I suppose I could add an sli argument to run_ccw if 
> you'd prefer that.

Ignoring an error feels wrong :) Telling the function "hey, I don't
know what buffer size you expect, just give me what you have" feels
better. If I read SA22-7204-01 correctly, there's always just a minimum
of sense id data, and how much we get is device dependent. (FWIW, the
Linux kernel does sense id with SLI as well.)

So yes, +1 to adding a sli parameter to run_ccw().

> 
> >> +     */
> >> +    if (irb->scsw.cstat != 0x00 && irb->scsw.cstat != 0x40) {
> >> +        return true;
> >> +    }
> >> +    return irb->scsw.dstat != 0xc;  
> > 
> > Also, shouldn't you actually use the #defines you introduce further
> > down?
> >   
> 
> Yep, I added the defines after I wrote this code. I'll fix that.
> 
> >> +}
> >> +
> >> +/* Executes a channel program at a given subchannel. The request to run the
> >> + * channel program is sent to the subchannel, we then wait for the interrupt
> >> + * singaling completion of the I/O operation(s) perfomed by the channel

s/perfomed/performed/

> >> + * program. Lastly we verify that the i/o operation completed without error and
> >> + * that the interrupt we received was for the subchannel used to run the
> >> + * channel program.
> >> + *
> >> + * Note: This function assumes it is running in an environment where no other
> >> + * cpus are generating or receiving I/O interrupts. So either run it in a
> >> + * single-cpu environment or make sure all other cpus are not doing I/O and
> >> + * have I/O interrupts masked off.  
> > 
> > Anything about iscs here (cr6)?
> >   
> 
> Those details are handled in the assembler code. Do you think I should mention something 
> about cr6 here?

We can probably do without.

> 
> >> + */
> >> +int do_cio(SubChannelId schid, uint32_t ccw_addr, int fmt)
> >> +{
> >> +    CmdOrb orb = {};
> >> +    Irb irb = {};
> >> +    SenseData sd;
> >> +    int rc, retries = 0;
> >> +
> >> +    IPL_assert(fmt == 0 || fmt == 1, "Invalid ccw format");
> >> +
> >> +    /* ccw_addr must be <= 24 bits and point to at least one whole ccw. */
> >> +    if (fmt == 0) {
> >> +        IPL_assert(ccw_addr <= 0xFFFFFF - 8, "Invalid ccw address");
> >> +    }
> >> +
> >> +    orb.fmt = fmt ;
> >> +    orb.pfch = 1;  /* QEMU's cio implementation requires prefetch */
> >> +    orb.c64 = 1;   /* QEMU's cio implementation requires 64-bit idaws */
> >> +    orb.lpm = 0xFF; /* All paths allowed */
> >> +    orb.cpa = ccw_addr;
> >> +
> >> +    while (true) {
> >> +        rc = ssch(schid, &orb);  
> > 
> > I think we can get here:
> > - cc 0 -> all ok
> > - cc 1 -> status pending; could that be an unsolicited interrupt from
> >    the device? or would we always get a deferred cc 1 in that case?
> > - cc 2 -> another function pending; Should Not Happen
> > - cc 3 -> it's dead, Jim
> > 
> > So I'm wondering whether we should consume the status and retry for cc
> > 1. The handling of the others is fine.
> >   
> 
> I took a look at css_do_ssch() in hw/s390x/css.c and it appears as though CC1 is a 
> possibility here. I'm not against taking action, but I suspect we would have to clear the 
> status with a basic sense (or something) before simply retrying... right?

It depends on the status. If you get an unit check, you'll probably
need the basic sense; in other cases, you'll probably want to simply
retry.

> 
> Is it safe for us to just assume we can clear it and move on? It seems like an edge case 
> that we'd be better off failing on. Perhaps let the user try again which will redrive the 
> process?

A very low amount of retries (2?) sounds reasonable: this would keep us
going if there's a state from the device that will be ignored anyway,
and won't get us stuck in a pointless retry loop if something more
involved is going on.

> 
> 
> >> +        if (rc) {
> >> +            print_int("ssch failed with rc=", rc);
> >> +            break;
> >> +        }
> >> +
> >> +        consume_io_int();
> >> +
> >> +        /* Clear read */  
> > 
> > I find that comment confusing. /* collect status */ maybe?
> >   
> >> +        rc = tsch(schid, &irb);  
> > 
> > Here we can get:
> > - cc 0 -> status pending, all ok
> > - cc 1 -> no status pending, Should Not Happen
> > - cc 3 -> it's dead, Jim
> > 
> > So this looks fine.
> >   
> >> +        if (rc) {
> >> +            print_int("tsch failed with rc=", rc);
> >> +            break;
> >> +        }
> >> +
> >> +        if (!irb_error(&irb)) {
> >> +            break;
> >> +        }
> >> +
> >> +        /* Unexpected unit check. Use sense to clear unit check then retry. */  
> > 
> > The dasds still don't support concurrent sense, do they? Might also be
> > worth investigating whether some unit checks are more "recoverable"
> > than others.
> >   
> 
> I wasn't sure on concurrent sense. I'd bet there are situations or environments where it 
> won't be supported so it seems safest to assume we don't have it.

Ok.

> 
> We already recover from the one unit check scenario I've discovered in practice (initial 
> reset). And the algorithm I chose is to simply retry a few times whenever we're presented 
> with unexpected unit check status. This is what the kernel does. It seems fairly robust.

Nod.

> > I expect we simply want to ignore IFCCs? IIRC, the strategy for those
> > is "retry, in case it is transient"; but that may take some time. Or
> > was there some path handling to be considered? (i.e., retrying may
> > select another path, which may be fine.)
> >   
> 
> Currently we'll give up on IFCC. I think this is the right thing to do. A user can always 
> retry if they want. But in reality an IFCC very likely means faulty hardware IIUC.

It could also be a transient link issue. Maybe retry twice, just to
avoid the very tiny blips?

> I've not thought about path management much. I suspect paths changing isn't something we 
> should realistically see in the bios. Even still, a retry is really all we can do, so 
> assuming path changes result in a unit check then we should be okay there.

If you use a full path mask, the channel subsystem might try a
different path (that is working correctly) the next time. I don't think
you want to implement path grouping stuff in the bios, which would mean
a lot of pain for very little gain :)

Thinking about path groups: One scenario we might have is that another
LPAR did a reserve on a dasd and then died. The dasd is then
unaccessible by our LPAR until we do a steal lock. If the device is
bound to the vfio-ccw subchannel driver, we don't have an interface for
that, though (we would need to re-bind to the I/O subchannel driver and
the dasd driver so we can invoke tunedasd). We could add an option to
break the lock from the bios, although that's probably overkill for a
real edge case. Just wanted to mention it :)

> 
> 
> >> +        if (unit_check(&irb) && retries <= 2) {
> >> +            basic_sense(schid, &sd);
> >> +            retries++;
> >> +            continue;
> >> +        }
> >> +
> >> +        break;
> >> +    }
> >> +
> >> +    return rc;
> >> +}  
> > 
> > (...)
> >   
> >> @@ -190,6 +247,9 @@ struct ciw {
> >>       __u16 count;
> >>   };
> >>   
> >> +#define CU_TYPE_VIRTIO          0x3832
> >> +#define CU_TYPE_DASD            0x3990  
> > 
> > No other dasd types we want to support? :) (Not sure if others are out
> > in the wild. Maybe FBA?)
> >  
> 
> I have no idea. I assumed 3390 was the only thing we supported. Perhaps 3380? I'd need to 
> find a test device, which I could probably do ... I'll look more into this.

IIRC, z/VM can hand out FBA devices. I'm not sure if current storage
systems can emulate them.

> 
> 
> >> +
> >>   /*
> >>    * sense-id response buffer layout
> >>    */
> >> @@ -205,6 +265,61 @@ typedef struct senseid {
> >>       struct ciw ciw[62];
> >>   }  __attribute__ ((packed, aligned(4))) SenseId;
> >>   
> >> +/* architected values for first sense byte */
> >> +#define SNS0_CMD_REJECT         0x80
> >> +#define SNS0_INTERVENTION_REQ   0x40
> >> +#define SNS0_BUS_OUT_CHECK      0x20
> >> +#define SNS0_EQUIPMENT_CHECK    0x10
> >> +#define SNS0_DATA_CHECK         0x08
> >> +#define SNS0_OVERRUN            0x04
> >> +#define SNS0_INCOMPL_DOMAIN     0x01  
> > 
> > IIRC, only byte 0 is device independent, and the others below are
> > (ECKD) dasd specific?
> >   
> >> +
> >> +/* architectured values for second sense byte */
> >> +#define SNS1_PERM_ERR           0x80
> >> +#define SNS1_INV_TRACK_FORMAT   0x40
> >> +#define SNS1_EOC                0x20
> >> +#define SNS1_MESSAGE_TO_OPER    0x10
> >> +#define SNS1_NO_REC_FOUND       0x08
> >> +#define SNS1_FILE_PROTECTED     0x04
> >> +#define SNS1_WRITE_INHIBITED    0x02
> >> +#define SNS1_INPRECISE_END      0x01
> >> +
> >> +/* architectured values for third sense byte */
> >> +#define SNS2_REQ_INH_WRITE      0x80
> >> +#define SNS2_CORRECTABLE        0x40
> >> +#define SNS2_FIRST_LOG_ERR      0x20
> >> +#define SNS2_ENV_DATA_PRESENT   0x10
> >> +#define SNS2_INPRECISE_END      0x04
> >> +
> >> +/* 24-byte Sense fmt/msg codes */
> >> +#define SENSE24_FMT_PROG_SYS    0x0
> >> +#define SENSE24_FMT_EQUIPMENT   0x2
> >> +#define SENSE24_FMT_CONTROLLER  0x3
> >> +#define SENSE24_FMT_MISC        0xF
> >> +
> >> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
> >> +
> >> +/* basic sense response buffer layout */
> >> +typedef struct senseData {
> >> +    uint8_t status[3];
> >> +    uint8_t res_count;
> >> +    uint8_t phys_drive_id;
> >> +    uint8_t low_cyl_addr;
> >> +    uint8_t head_high_cyl_addr;
> >> +    uint8_t fmt_msg;
> >> +    uint64_t fmt_dependent_info[2];
> >> +    uint8_t reserved;
> >> +    uint8_t program_action_code;
> >> +    uint16_t config_info;
> >> +    uint8_t mcode_hicyl;
> >> +    uint8_t cyl_head_addr[3];
> >> +}  __attribute__ ((packed, aligned(4))) SenseData;  
> > 
> > And this looks _really_ dasd specific.
> >   
> 
> Yep, I glossed over those details while I was furiously tracking down the reset bug. I'll 
> take a look at redesigning this.

Ok.

> 
> >> +
> >> +#define SENSE24_GET_FMT(sd)     (sd->fmt_msg & 0xF0 >> 4)
> >> +#define SENSE24_GET_MSG(sd)     (sd->fmt_msg & 0x0F)
> >> +
> >> +#define unit_check(irb)     ((irb)->scsw.dstat & SCSW_DSTAT_UCHK)
> >> +
> >>   /* interruption response block */
> >>   typedef struct irb {
> >>       struct scsw scsw;  
> > 
> > (...)
> >   
> >> diff --git a/pc-bios/s390-ccw/start.S b/pc-bios/s390-ccw/start.S
> >> index eb8d024..a48c38f 100644
> >> --- a/pc-bios/s390-ccw/start.S
> >> +++ b/pc-bios/s390-ccw/start.S
> >> @@ -65,12 +65,32 @@ consume_sclp_int:
> >>           /* prepare external call handler */
> >>           larl %r1, external_new_code
> >>           stg %r1, 0x1b8
> >> -        larl %r1, external_new_mask
> >> +        larl %r1, int_new_mask
> >>           mvc 0x1b0(8),0(%r1)
> >>           /* load enabled wait PSW */
> >>           larl %r1, enabled_wait_psw
> >>           lpswe 0(%r1)
> >>   
> >> +/*
> >> + * void consume_io_int(void)
> >> + *
> >> + * eats one I/O interrupt  
> > 
> > *nomnom*
> >   
> >> + */
> >> +        .globl consume_io_int
> >> +consume_io_int:
> >> +        /* enable I/O interrupts in cr0 */  
> > 
> > cr6?
> >   
> >> +        stctg 6,6,0(15)
> >> +        oi 4(15), 0xff
> >> +        lctlg 6,6,0(15)
> >> +        /* prepare external call handler */  
> > 
> > I/O call handler?
> >   
> 
> Both copy/paste errors. Thanks for catching these. :)
> 
> >> +        larl %r1, io_new_code
> >> +        stg %r1, 0x1f8
> >> +        larl %r1, int_new_mask
> >> +        mvc 0x1f0(8),0(%r1)
> >> +        /* load enabled wait PSW */
> >> +        larl %r1, enabled_wait_psw
> >> +        lpswe 0(%r1)
> >> +
> >>   external_new_code:
> >>           /* disable service interrupts in cr0 */
> >>           stctg 0,0,0(15)
> >> @@ -78,10 +98,19 @@ external_new_code:
> >>           lctlg 0,0,0(15)
> >>           br 14
> >>   
> >> +io_new_code:
> >> +        /* disable I/O interrupts in cr6 */
> >> +        stctg 6,6,0(15)  
> > 
> > I'm wondering why you are changing cr6 every time you wait for an I/O
> > interrupt. Just enable the isc(s) you want once, and disable them again
> > after you're done with all I/O? Simply disabling the I/O interrupts
> > should be enough to prevent further interrupts popping up. You maybe
> > want two enabled wait PSWs, one with I/O + external and one with
> > external only?
> >   
> 
> No real reason. We only come through here a hand full of times so performance is not a 
> consideration. I guess my thought process was probably to keep the system is as close to 
> initial state as possible through the ipl process. Eventually when we hand control to the 
> guest OS we want the system as close to undisturbed as possible. If you think I should 
> only be setting cr-6 once, it sounds reasonable.

It just looked a bit odd to me. But I agree that this isn't
performance-sensitive.

> 
> 
> >> +        ni 4(15), 0x00
> >> +        lctlg 6,6,0(15)
> >> +        br 14
> >> +
> >> +
> >> +
> >>           .align  8
> >>   disabled_wait_psw:
> >>           .quad   0x0002000180000000,0x0000000000000000
> >>   enabled_wait_psw:
> >>           .quad   0x0302000180000000,0x0000000000000000
> >> -external_new_mask:
> >> +int_new_mask:
> >>           .quad   0x0000000180000000  
> > 
> >   
> 
> 

Re: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Jason J. Herne]

On 12/12/18 9:34 AM, Cornelia Huck wrote:
...
>>
>> NOTE: It has been a while, but I've finally chased down my infamous "reset bug".
>> On subsystem reset (I see this right after host ipl) we sometimes end up getting
>> an unexpected unit check status from a dasd device. This causes the first start
>> subchannel instruction to fail due to the pending unit check status. My solution
>> to this problem, as advised by the kernel folks, is to simply retry my ssch
>> instructions before declaring failure when unexpected unit checks happen. In the
>> event of a persistent error, after two retries we'll give up and print some
>> useful error info for the user.
> 
> So, is that a status we only see because the vfio-ccw driver keeps the
> subchannel enabled (as by the other recent thread)?
> 
> Is there any value in distinguishing different unit checks, or is retry
> the best strategy in any case?
> 
The status presents on device reset. So when the host kernel IPLs this status will be 
present. The very first attempt to use the device (SSCH, other instructions perhaps?) will 
cause this status to be presented. Sometimes the host kernel must "get there first" and 
clear the status. And other times the guest (by way of Qemu bios) gets there first.

The kernel handles unexpected unit checks by simply retrying a low number of times before 
giving up. Given that bios code is a constant frequency code path, and the kernel has 
already set this precedent, I feel safe with this decision and don't see a ton of value in 
doing much more. If we find a case that requires more handling we can take a look at it.

-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [qemu-s390x] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Halil Pasic]

On Tue, 8 Jan 2019 11:37:56 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 12/12/18 9:34 AM, Cornelia Huck wrote:
> ...
> >>
> >> NOTE: It has been a while, but I've finally chased down my infamous "reset bug".
> >> On subsystem reset (I see this right after host ipl) we sometimes end up getting
> >> an unexpected unit check status from a dasd device. This causes the first start
> >> subchannel instruction to fail due to the pending unit check status. My solution
> >> to this problem, as advised by the kernel folks, is to simply retry my ssch
> >> instructions before declaring failure when unexpected unit checks happen. In the
> >> event of a persistent error, after two retries we'll give up and print some
> >> useful error info for the user.
> > 
> > So, is that a status we only see because the vfio-ccw driver keeps the
> > subchannel enabled (as by the other recent thread)?
> > 
> > Is there any value in distinguishing different unit checks, or is retry
> > the best strategy in any case?
> > 
> The status presents on device reset. So when the host kernel IPLs this status will be 
> present. The very first attempt to use the device (SSCH, other instructions perhaps?) will 
> cause this status to be presented. Sometimes the host kernel must "get there first" and 
> clear the status. And other times the guest (by way of Qemu bios) gets there first.
> 
> The kernel handles unexpected unit checks by simply retrying a low number of times before 
> giving up. Given that bios code is a constant frequency code path, and the kernel has 
> already set this precedent, I feel safe with this decision and don't see a ton of value in 
> doing much more. If we find a case that requires more handling we can take a look at it.
> 

I agree, doing elaborate CIO error handling here does not seem like a
particularly good idea.

Something remotely related -- let me play crazy for a moment: let's say
we pass-through two DASD's to a single guest, one as the IPL disk and
one just so. If I'm not mistaken, the guest is guaranteed to get this
special after reset unit check (let's say freshly constructed VM),
unless there is another OS messing with the same DASD maybe, at least
for the 'just son DASD'. I would even guess that the condition in
question is indicated even for the IPL-DASD (if we thing guest1).

But ccw-passthrough won't get perfect anyway. So I think we can ignore
this side effect of the reset, unless  a need arises not to.

Regards,
Halil

Re: [Qemu-devel] [qemu-s390x] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Cornelia Huck]

On Tue, 8 Jan 2019 18:36:09 +0100
Halil Pasic <pasic@linux.ibm.com> wrote:

> On Tue, 8 Jan 2019 11:37:56 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
> > On 12/12/18 9:34 AM, Cornelia Huck wrote:
> > ...  
> > >>
> > >> NOTE: It has been a while, but I've finally chased down my infamous "reset bug".
> > >> On subsystem reset (I see this right after host ipl) we sometimes end up getting
> > >> an unexpected unit check status from a dasd device. This causes the first start
> > >> subchannel instruction to fail due to the pending unit check status. My solution
> > >> to this problem, as advised by the kernel folks, is to simply retry my ssch
> > >> instructions before declaring failure when unexpected unit checks happen. In the
> > >> event of a persistent error, after two retries we'll give up and print some
> > >> useful error info for the user.  
> > > 
> > > So, is that a status we only see because the vfio-ccw driver keeps the
> > > subchannel enabled (as by the other recent thread)?
> > > 
> > > Is there any value in distinguishing different unit checks, or is retry
> > > the best strategy in any case?
> > >   
> > The status presents on device reset. So when the host kernel IPLs this status will be 
> > present. The very first attempt to use the device (SSCH, other instructions perhaps?) will 
> > cause this status to be presented. Sometimes the host kernel must "get there first" and 
> > clear the status. And other times the guest (by way of Qemu bios) gets there first.
> > 
> > The kernel handles unexpected unit checks by simply retrying a low number of times before 
> > giving up. Given that bios code is a constant frequency code path, and the kernel has 
> > already set this precedent, I feel safe with this decision and don't see a ton of value in 
> > doing much more. If we find a case that requires more handling we can take a look at it.

Yeah, my thinking was "should we check for this particular unit check
so we don't ignore other problems"? But if the kernel simply retries,
let's just do that in the bios as well.

> I agree, doing elaborate CIO error handling here does not seem like a
> particularly good idea.
> 
> Something remotely related -- let me play crazy for a moment: let's say
> we pass-through two DASD's to a single guest, one as the IPL disk and
> one just so. If I'm not mistaken, the guest is guaranteed to get this
> special after reset unit check (let's say freshly constructed VM),
> unless there is another OS messing with the same DASD maybe, at least
> for the 'just son DASD'. I would even guess that the condition in
> question is indicated even for the IPL-DASD (if we thing guest1).

My thinking is that the guest needs to be able to deal with this unit
check if it gets it, and chances are good that it already does the
right thing if the OS has been running on non-KVM as well. We probably
can neglect any differences between IPL and non-IPL, as the unit check
is simply something that the guest *might* see.

> 
> But ccw-passthrough won't get perfect anyway. So I think we can ignore
> this side effect of the reset, unless  a need arises not to.
> 
> Regards,
> Halil
> 
> 

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

On 1/7/19 2:02 PM, Jason J. Herne wrote:
>>> +
>>>   /*
>>>    * sense-id response buffer layout
>>>    */
>>> @@ -205,6 +265,61 @@ typedef struct senseid {
>>>       struct ciw ciw[62];
>>>   }  __attribute__ ((packed, aligned(4))) SenseId;
>>> +/* architected values for first sense byte */
>>> +#define SNS0_CMD_REJECT         0x80
>>> +#define SNS0_INTERVENTION_REQ   0x40
>>> +#define SNS0_BUS_OUT_CHECK      0x20
>>> +#define SNS0_EQUIPMENT_CHECK    0x10
>>> +#define SNS0_DATA_CHECK         0x08
>>> +#define SNS0_OVERRUN            0x04
>>> +#define SNS0_INCOMPL_DOMAIN     0x01
>>
>> IIRC, only byte 0 is device independent, and the others below are
>> (ECKD) dasd specific?
>>
>>> +
>>> +/* architectured values for second sense byte */
>>> +#define SNS1_PERM_ERR           0x80
>>> +#define SNS1_INV_TRACK_FORMAT   0x40
>>> +#define SNS1_EOC                0x20
>>> +#define SNS1_MESSAGE_TO_OPER    0x10
>>> +#define SNS1_NO_REC_FOUND       0x08
>>> +#define SNS1_FILE_PROTECTED     0x04
>>> +#define SNS1_WRITE_INHIBITED    0x02
>>> +#define SNS1_INPRECISE_END      0x01
>>> +
>>> +/* architectured values for third sense byte */
>>> +#define SNS2_REQ_INH_WRITE      0x80
>>> +#define SNS2_CORRECTABLE        0x40
>>> +#define SNS2_FIRST_LOG_ERR      0x20
>>> +#define SNS2_ENV_DATA_PRESENT   0x10
>>> +#define SNS2_INPRECISE_END      0x04
>>> +
>>> +/* 24-byte Sense fmt/msg codes */
>>> +#define SENSE24_FMT_PROG_SYS    0x0
>>> +#define SENSE24_FMT_EQUIPMENT   0x2
>>> +#define SENSE24_FMT_CONTROLLER  0x3
>>> +#define SENSE24_FMT_MISC        0xF
>>> +
>>> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
>>> +
>>> +/* basic sense response buffer layout */
>>> +typedef struct senseData {
>>> +    uint8_t status[3];
>>> +    uint8_t res_count;
>>> +    uint8_t phys_drive_id;
>>> +    uint8_t low_cyl_addr;
>>> +    uint8_t head_high_cyl_addr;
>>> +    uint8_t fmt_msg;
>>> +    uint64_t fmt_dependent_info[2];
>>> +    uint8_t reserved;
>>> +    uint8_t program_action_code;
>>> +    uint16_t config_info;
>>> +    uint8_t mcode_hicyl;
>>> +    uint8_t cyl_head_addr[3];
>>> +}  __attribute__ ((packed, aligned(4))) SenseData;
>>
>> And this looks _really_ dasd specific.
>>
> 
> Yep, I glossed over those details while I was furiously tracking down the reset bug. I'll 
> take a look at redesigning this.

All of my information for creating these data structures came from an internal ECKD DASD 
reference. There are probably some things that could stand a bit of cleanup or renaming. 
Aside from that, considering this is in a DASD only (ECKD DASD only at the moment) code 
path are you okay with my renaming the struct to senseDataECKD or something similar?
I'm not sure what value there is in abstracting sense at the moment. I'm not even sure 
what other device's sense data looks like. Since my description of the SENSE CCW comes 
from an ECKD reference I have not been able to verify any areas of the data that are 
common across device types. Thoughts?


-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Cornelia Huck]

On Wed, 9 Jan 2019 13:10:26 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 1/7/19 2:02 PM, Jason J. Herne wrote:
> >>> +
> >>>   /*
> >>>    * sense-id response buffer layout
> >>>    */
> >>> @@ -205,6 +265,61 @@ typedef struct senseid {
> >>>       struct ciw ciw[62];
> >>>   }  __attribute__ ((packed, aligned(4))) SenseId;
> >>> +/* architected values for first sense byte */
> >>> +#define SNS0_CMD_REJECT         0x80
> >>> +#define SNS0_INTERVENTION_REQ   0x40
> >>> +#define SNS0_BUS_OUT_CHECK      0x20
> >>> +#define SNS0_EQUIPMENT_CHECK    0x10
> >>> +#define SNS0_DATA_CHECK         0x08
> >>> +#define SNS0_OVERRUN            0x04
> >>> +#define SNS0_INCOMPL_DOMAIN     0x01  
> >>
> >> IIRC, only byte 0 is device independent, and the others below are
> >> (ECKD) dasd specific?
> >>  
> >>> +
> >>> +/* architectured values for second sense byte */
> >>> +#define SNS1_PERM_ERR           0x80
> >>> +#define SNS1_INV_TRACK_FORMAT   0x40
> >>> +#define SNS1_EOC                0x20
> >>> +#define SNS1_MESSAGE_TO_OPER    0x10
> >>> +#define SNS1_NO_REC_FOUND       0x08
> >>> +#define SNS1_FILE_PROTECTED     0x04
> >>> +#define SNS1_WRITE_INHIBITED    0x02
> >>> +#define SNS1_INPRECISE_END      0x01
> >>> +
> >>> +/* architectured values for third sense byte */
> >>> +#define SNS2_REQ_INH_WRITE      0x80
> >>> +#define SNS2_CORRECTABLE        0x40
> >>> +#define SNS2_FIRST_LOG_ERR      0x20
> >>> +#define SNS2_ENV_DATA_PRESENT   0x10
> >>> +#define SNS2_INPRECISE_END      0x04
> >>> +
> >>> +/* 24-byte Sense fmt/msg codes */
> >>> +#define SENSE24_FMT_PROG_SYS    0x0
> >>> +#define SENSE24_FMT_EQUIPMENT   0x2
> >>> +#define SENSE24_FMT_CONTROLLER  0x3
> >>> +#define SENSE24_FMT_MISC        0xF
> >>> +
> >>> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
> >>> +
> >>> +/* basic sense response buffer layout */
> >>> +typedef struct senseData {
> >>> +    uint8_t status[3];
> >>> +    uint8_t res_count;
> >>> +    uint8_t phys_drive_id;
> >>> +    uint8_t low_cyl_addr;
> >>> +    uint8_t head_high_cyl_addr;
> >>> +    uint8_t fmt_msg;
> >>> +    uint64_t fmt_dependent_info[2];
> >>> +    uint8_t reserved;
> >>> +    uint8_t program_action_code;
> >>> +    uint16_t config_info;
> >>> +    uint8_t mcode_hicyl;
> >>> +    uint8_t cyl_head_addr[3];
> >>> +}  __attribute__ ((packed, aligned(4))) SenseData;  
> >>
> >> And this looks _really_ dasd specific.
> >>  
> > 
> > Yep, I glossed over those details while I was furiously tracking down the reset bug. I'll 
> > take a look at redesigning this.  
> 
> All of my information for creating these data structures came from an internal ECKD DASD 
> reference. There are probably some things that could stand a bit of cleanup or renaming. 
> Aside from that, considering this is in a DASD only (ECKD DASD only at the moment) code 
> path are you okay with my renaming the struct to senseDataECKD or something similar?

Renaming this makes sense.

> I'm not sure what value there is in abstracting sense at the moment. I'm not even sure 
> what other device's sense data looks like. Since my description of the SENSE CCW comes 
> from an ECKD reference I have not been able to verify any areas of the data that are 
> common across device types. Thoughts?

There's SA22-7204-01 ("Common I/O Device Commands"), which is from 1992
(this is what I have on my disk -- is there anything newer?). It
specifies what bits 0-5 of byte 0 mean and states that bytes 1-31 are
optional and device-specific.

Maybe some other bits have been specified after 1992, but I have not
come across documentation for them.

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

On 1/9/19 1:34 PM, Cornelia Huck wrote:
> On Wed, 9 Jan 2019 13:10:26 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
>> On 1/7/19 2:02 PM, Jason J. Herne wrote:
>>>>> +
>>>>>    /*
>>>>>     * sense-id response buffer layout
>>>>>     */
>>>>> @@ -205,6 +265,61 @@ typedef struct senseid {
>>>>>        struct ciw ciw[62];
>>>>>    }  __attribute__ ((packed, aligned(4))) SenseId;
>>>>> +/* architected values for first sense byte */
>>>>> +#define SNS0_CMD_REJECT         0x80
>>>>> +#define SNS0_INTERVENTION_REQ   0x40
>>>>> +#define SNS0_BUS_OUT_CHECK      0x20
>>>>> +#define SNS0_EQUIPMENT_CHECK    0x10
>>>>> +#define SNS0_DATA_CHECK         0x08
>>>>> +#define SNS0_OVERRUN            0x04
>>>>> +#define SNS0_INCOMPL_DOMAIN     0x01
>>>>
>>>> IIRC, only byte 0 is device independent, and the others below are
>>>> (ECKD) dasd specific?
>>>>   
>>>>> +
>>>>> +/* architectured values for second sense byte */
>>>>> +#define SNS1_PERM_ERR           0x80
>>>>> +#define SNS1_INV_TRACK_FORMAT   0x40
>>>>> +#define SNS1_EOC                0x20
>>>>> +#define SNS1_MESSAGE_TO_OPER    0x10
>>>>> +#define SNS1_NO_REC_FOUND       0x08
>>>>> +#define SNS1_FILE_PROTECTED     0x04
>>>>> +#define SNS1_WRITE_INHIBITED    0x02
>>>>> +#define SNS1_INPRECISE_END      0x01
>>>>> +
>>>>> +/* architectured values for third sense byte */
>>>>> +#define SNS2_REQ_INH_WRITE      0x80
>>>>> +#define SNS2_CORRECTABLE        0x40
>>>>> +#define SNS2_FIRST_LOG_ERR      0x20
>>>>> +#define SNS2_ENV_DATA_PRESENT   0x10
>>>>> +#define SNS2_INPRECISE_END      0x04
>>>>> +
>>>>> +/* 24-byte Sense fmt/msg codes */
>>>>> +#define SENSE24_FMT_PROG_SYS    0x0
>>>>> +#define SENSE24_FMT_EQUIPMENT   0x2
>>>>> +#define SENSE24_FMT_CONTROLLER  0x3
>>>>> +#define SENSE24_FMT_MISC        0xF
>>>>> +
>>>>> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
>>>>> +
>>>>> +/* basic sense response buffer layout */
>>>>> +typedef struct senseData {
>>>>> +    uint8_t status[3];
>>>>> +    uint8_t res_count;
>>>>> +    uint8_t phys_drive_id;
>>>>> +    uint8_t low_cyl_addr;
>>>>> +    uint8_t head_high_cyl_addr;
>>>>> +    uint8_t fmt_msg;
>>>>> +    uint64_t fmt_dependent_info[2];
>>>>> +    uint8_t reserved;
>>>>> +    uint8_t program_action_code;
>>>>> +    uint16_t config_info;
>>>>> +    uint8_t mcode_hicyl;
>>>>> +    uint8_t cyl_head_addr[3];
>>>>> +}  __attribute__ ((packed, aligned(4))) SenseData;
>>>>
>>>> And this looks _really_ dasd specific.
>>>>   
>>>
>>> Yep, I glossed over those details while I was furiously tracking down the reset bug. I'll
>>> take a look at redesigning this.
>>
>> All of my information for creating these data structures came from an internal ECKD DASD
>> reference. There are probably some things that could stand a bit of cleanup or renaming.
>> Aside from that, considering this is in a DASD only (ECKD DASD only at the moment) code
>> path are you okay with my renaming the struct to senseDataECKD or something similar?
> 
> Renaming this makes sense.
> 
>> I'm not sure what value there is in abstracting sense at the moment. I'm not even sure
>> what other device's sense data looks like. Since my description of the SENSE CCW comes
>> from an ECKD reference I have not been able to verify any areas of the data that are
>> common across device types. Thoughts?
> 
> There's SA22-7204-01 ("Common I/O Device Commands"), which is from 1992
> (this is what I have on my disk -- is there anything newer?). It
> specifies what bits 0-5 of byte 0 mean and states that bytes 1-31 are
> optional and device-specific.
> 
> Maybe some other bits have been specified after 1992, but I have not
> come across documentation for them.

That publication is no longer available. According to my quick research it has been 
replaced by an internal only publication. I'll see what I can find.


-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Cornelia Huck]

On Wed, 9 Jan 2019 15:01:19 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 1/9/19 1:34 PM, Cornelia Huck wrote:
> > On Wed, 9 Jan 2019 13:10:26 -0500
> > "Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> >> I'm not sure what value there is in abstracting sense at the moment. I'm not even sure
> >> what other device's sense data looks like. Since my description of the SENSE CCW comes
> >> from an ECKD reference I have not been able to verify any areas of the data that are
> >> common across device types. Thoughts?  
> > 
> > There's SA22-7204-01 ("Common I/O Device Commands"), which is from 1992
> > (this is what I have on my disk -- is there anything newer?). It
> > specifies what bits 0-5 of byte 0 mean and states that bytes 1-31 are
> > optional and device-specific.
> > 
> > Maybe some other bits have been specified after 1992, but I have not
> > come across documentation for them.  
> 
> That publication is no longer available. According to my quick research it has been 
> replaced by an internal only publication. I'll see what I can find.

The publication no longer being available is not good, as it is a
normative reference pointed to by the virtio standard (see
http://docs.oasis-open.org/virtio/virtio/v1.0/cs04/virtio-v1.0-cs04.html#x1-30001)
:(

Is there any chance that at least that old version can be made
available again?

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

On 1/10/19 7:15 AM, Cornelia Huck wrote:
> On Wed, 9 Jan 2019 15:01:19 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
>> On 1/9/19 1:34 PM, Cornelia Huck wrote:
>>> On Wed, 9 Jan 2019 13:10:26 -0500
>>> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
>>>> I'm not sure what value there is in abstracting sense at the moment. I'm not even sure
>>>> what other device's sense data looks like. Since my description of the SENSE CCW comes
>>>> from an ECKD reference I have not been able to verify any areas of the data that are
>>>> common across device types. Thoughts?
>>>
>>> There's SA22-7204-01 ("Common I/O Device Commands"), which is from 1992
>>> (this is what I have on my disk -- is there anything newer?). It
>>> specifies what bits 0-5 of byte 0 mean and states that bytes 1-31 are
>>> optional and device-specific.
>>>
>>> Maybe some other bits have been specified after 1992, but I have not
>>> come across documentation for them.
>>
>> That publication is no longer available. According to my quick research it has been
>> replaced by an internal only publication. I'll see what I can find.
> 
> The publication no longer being available is not good, as it is a
> normative reference pointed to by the virtio standard (see
> http://docs.oasis-open.org/virtio/virtio/v1.0/cs04/virtio-v1.0-cs04.html#x1-30001)
> :(
> 
> Is there any chance that at least that old version can be made
> available again?

A quick web search turns up the following link:
http://publibz.boulder.ibm.com/support/libraryserver/FRAMESET/DZ9AR501/CCONTENTS?D

So I guess it is available, even if the document has been superseded internally.


-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Cornelia Huck]

On Thu, 10 Jan 2019 10:02:48 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 1/10/19 7:15 AM, Cornelia Huck wrote:
> > On Wed, 9 Jan 2019 15:01:19 -0500
> > "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> >   
> >> On 1/9/19 1:34 PM, Cornelia Huck wrote:  
> >>> On Wed, 9 Jan 2019 13:10:26 -0500
> >>> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:  
> >   
> >>>> I'm not sure what value there is in abstracting sense at the moment. I'm not even sure
> >>>> what other device's sense data looks like. Since my description of the SENSE CCW comes
> >>>> from an ECKD reference I have not been able to verify any areas of the data that are
> >>>> common across device types. Thoughts?  
> >>>
> >>> There's SA22-7204-01 ("Common I/O Device Commands"), which is from 1992
> >>> (this is what I have on my disk -- is there anything newer?). It
> >>> specifies what bits 0-5 of byte 0 mean and states that bytes 1-31 are
> >>> optional and device-specific.
> >>>
> >>> Maybe some other bits have been specified after 1992, but I have not
> >>> come across documentation for them.  
> >>
> >> That publication is no longer available. According to my quick research it has been
> >> replaced by an internal only publication. I'll see what I can find.  
> > 
> > The publication no longer being available is not good, as it is a
> > normative reference pointed to by the virtio standard (see
> > http://docs.oasis-open.org/virtio/virtio/v1.0/cs04/virtio-v1.0-cs04.html#x1-30001)
> > :(
> > 
> > Is there any chance that at least that old version can be made
> > available again?  
> 
> A quick web search turns up the following link:
> http://publibz.boulder.ibm.com/support/libraryserver/FRAMESET/DZ9AR501/CCONTENTS?D
> 
> So I guess it is available, even if the document has been superseded internally.

Cool, thanks for checking!

Re: [Qemu-devel] [PATCH 12/15] s390-bios: Refactor virtio to run channel programs via cio

[Jason J. Herne]

On 12/14/18 8:04 AM, Cornelia Huck wrote:
> On Wed, 12 Dec 2018 09:11:15 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
>> Now that we have a Channel I/O library let's modify virtio boot code to
>> make use of it for running channel programs.
>>
>> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
>> ---
>>   pc-bios/s390-ccw/virtio.c | 21 ++-------------------
>>   1 file changed, 2 insertions(+), 19 deletions(-)
>>
>> diff --git a/pc-bios/s390-ccw/virtio.c b/pc-bios/s390-ccw/virtio.c
>> index aa9da72..5fb49e2 100644
>> --- a/pc-bios/s390-ccw/virtio.c
>> +++ b/pc-bios/s390-ccw/virtio.c
>> @@ -92,30 +92,13 @@ int drain_irqs(SubChannelId schid)
>>   static int run_ccw(VDev *vdev, int cmd, void *ptr, int len)
>>   {
>>       Ccw1 ccw = {};
>> -    CmdOrb orb = {};
>> -    int r;
>> -
>> -    enable_subchannel(vdev->schid);
>> -
>> -    /* start subchannel command */
>> -    orb.fmt = 1;
>> -    orb.cpa = (u32)(long)&ccw;
>> -    orb.lpm = 0x80;
>>   
>>       ccw.cmd_code = cmd;
>>       ccw.cda = (long)ptr;
>>       ccw.count = len;
>>   
>> -    r = ssch(vdev->schid, &orb);
>> -    /*
>> -     * XXX Wait until device is done processing the CCW. For now we can
>> -     *     assume that a simple tsch will have finished the CCW processing,
>> -     *     but the architecture allows for asynchronous operation
>> -     */
>> -    if (!r) {
>> -        r = drain_irqs(vdev->schid);
>> -    }
>> -    return r;
>> +    enable_subchannel(vdev->schid);
> 
> Preexisting, but I'm wondering why we enable the subchannel here and
> not just once?
> 
>> +    return do_cio(vdev->schid, ptr2u32(&ccw), CCW_FMT1);
>>   }
>>   
>>   static void vring_init(VRing *vr, VqInfo *info)
> 
> 

Good question. It seems to be needed for the net-boot path. Here is the chain:

netmain.c : main()
netmain,c : net_init()
virtio-net.c : virtio_net_init()
virtio.c : virtio_setup_ccw()
virtio.c : run_ccw()

In this case the subchannel would not already be enabled. I suppose I *could* change it, 
but at this point I'm trying to limit how much more code I'm touching :). This patch set 
already touches a bit more code than I anticipated. Let me know what your thoughts are.

-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [PATCH 12/15] s390-bios: Refactor virtio to run channel programs via cio

[Cornelia Huck]

On Thu, 10 Jan 2019 11:12:47 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 12/14/18 8:04 AM, Cornelia Huck wrote:
> > On Wed, 12 Dec 2018 09:11:15 -0500
> > "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> >   
> >> Now that we have a Channel I/O library let's modify virtio boot code to
> >> make use of it for running channel programs.
> >>
> >> Signed-off-by: Jason J. Herne <jjherne@linux.ibm.com>
> >> ---
> >>   pc-bios/s390-ccw/virtio.c | 21 ++-------------------
> >>   1 file changed, 2 insertions(+), 19 deletions(-)
> >>
> >> diff --git a/pc-bios/s390-ccw/virtio.c b/pc-bios/s390-ccw/virtio.c
> >> index aa9da72..5fb49e2 100644
> >> --- a/pc-bios/s390-ccw/virtio.c
> >> +++ b/pc-bios/s390-ccw/virtio.c
> >> @@ -92,30 +92,13 @@ int drain_irqs(SubChannelId schid)
> >>   static int run_ccw(VDev *vdev, int cmd, void *ptr, int len)
> >>   {
> >>       Ccw1 ccw = {};
> >> -    CmdOrb orb = {};
> >> -    int r;
> >> -
> >> -    enable_subchannel(vdev->schid);
> >> -
> >> -    /* start subchannel command */
> >> -    orb.fmt = 1;
> >> -    orb.cpa = (u32)(long)&ccw;
> >> -    orb.lpm = 0x80;
> >>   
> >>       ccw.cmd_code = cmd;
> >>       ccw.cda = (long)ptr;
> >>       ccw.count = len;
> >>   
> >> -    r = ssch(vdev->schid, &orb);
> >> -    /*
> >> -     * XXX Wait until device is done processing the CCW. For now we can
> >> -     *     assume that a simple tsch will have finished the CCW processing,
> >> -     *     but the architecture allows for asynchronous operation
> >> -     */
> >> -    if (!r) {
> >> -        r = drain_irqs(vdev->schid);
> >> -    }
> >> -    return r;
> >> +    enable_subchannel(vdev->schid);  
> > 
> > Preexisting, but I'm wondering why we enable the subchannel here and
> > not just once?
> >   
> >> +    return do_cio(vdev->schid, ptr2u32(&ccw), CCW_FMT1);
> >>   }
> >>   
> >>   static void vring_init(VRing *vr, VqInfo *info)  
> > 
> >   
> 
> Good question. It seems to be needed for the net-boot path. Here is the chain:
> 
> netmain.c : main()
> netmain,c : net_init()
> virtio-net.c : virtio_net_init()
> virtio.c : virtio_setup_ccw()
> virtio.c : run_ccw()
> 
> In this case the subchannel would not already be enabled. I suppose I *could* change it, 
> but at this point I'm trying to limit how much more code I'm touching :). This patch set 
> already touches a bit more code than I anticipated. Let me know what your thoughts are.

As it doesn't really hurt, I think we can defer this to later. Let's
not make this patch set bigger than needed :)

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

On 1/7/19 2:02 PM, Jason J. Herne wrote:
>>> @@ -190,6 +247,9 @@ struct ciw {
>>>       __u16 count;
>>>   };
>>> +#define CU_TYPE_VIRTIO          0x3832
>>> +#define CU_TYPE_DASD            0x3990
>>
>> No other dasd types we want to support? :) (Not sure if others are out
>> in the wild. Maybe FBA?)
>>
> 
> I have no idea. I assumed 3390 was the only thing we supported. Perhaps 3380? I'd need to 
> find a test device, which I could probably do ... I'll look more into this.

After a few discussions with folks in the lab we've decided that we don't see a ton of 
value in supporting anything other than 3990 at the moment. Anything else would be older 
(3380) and/or rare to see in the wild (and very difficult to test). As for emulated setups 
like z/VM, a user can just use 3390 instead of FBA. So I recommend we move forward with 
3390/3990 support for now. We can always add in others types if/when we need them.

-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Cornelia Huck]

On Mon, 14 Jan 2019 13:44:12 -0500
"Jason J. Herne" <jjherne@linux.ibm.com> wrote:

> On 1/7/19 2:02 PM, Jason J. Herne wrote:
> >>> @@ -190,6 +247,9 @@ struct ciw {
> >>>       __u16 count;
> >>>   };
> >>> +#define CU_TYPE_VIRTIO          0x3832
> >>> +#define CU_TYPE_DASD            0x3990  
> >>
> >> No other dasd types we want to support? :) (Not sure if others are out
> >> in the wild. Maybe FBA?)
> >>  
> > 
> > I have no idea. I assumed 3390 was the only thing we supported. Perhaps 3380? I'd need to 
> > find a test device, which I could probably do ... I'll look more into this.  
> 
> After a few discussions with folks in the lab we've decided that we don't see a ton of 
> value in supporting anything other than 3990 at the moment. Anything else would be older 
> (3380) and/or rare to see in the wild (and very difficult to test). As for emulated setups 
> like z/VM, a user can just use 3390 instead of FBA. So I recommend we move forward with 
> 3390/3990 support for now. We can always add in others types if/when we need them.

Sounds reasonable.

What about calling the #define above CU_TYPE_DASD_3990 instead? Just to
make clear that there are other dasd types out there, but we only
support that particular one (at least at the moment).

Re: [Qemu-devel] [qemu-s390x] [PATCH 10/15] s390-bios: Support for running format-0/1 channel programs

[Jason J. Herne]

On 1/9/19 1:34 PM, Cornelia Huck wrote:
> On Wed, 9 Jan 2019 13:10:26 -0500
> "Jason J. Herne" <jjherne@linux.ibm.com> wrote:
> 
>> On 1/7/19 2:02 PM, Jason J. Herne wrote:
>>>>> +
>>>>>    /*
>>>>>     * sense-id response buffer layout
>>>>>     */
>>>>> @@ -205,6 +265,61 @@ typedef struct senseid {
>>>>>        struct ciw ciw[62];
>>>>>    }  __attribute__ ((packed, aligned(4))) SenseId;
>>>>> +/* architected values for first sense byte */
>>>>> +#define SNS0_CMD_REJECT         0x80
>>>>> +#define SNS0_INTERVENTION_REQ   0x40
>>>>> +#define SNS0_BUS_OUT_CHECK      0x20
>>>>> +#define SNS0_EQUIPMENT_CHECK    0x10
>>>>> +#define SNS0_DATA_CHECK         0x08
>>>>> +#define SNS0_OVERRUN            0x04
>>>>> +#define SNS0_INCOMPL_DOMAIN     0x01
>>>>
>>>> IIRC, only byte 0 is device independent, and the others below are
>>>> (ECKD) dasd specific?
>>>>   
>>>>> +
>>>>> +/* architectured values for second sense byte */
>>>>> +#define SNS1_PERM_ERR           0x80
>>>>> +#define SNS1_INV_TRACK_FORMAT   0x40
>>>>> +#define SNS1_EOC                0x20
>>>>> +#define SNS1_MESSAGE_TO_OPER    0x10
>>>>> +#define SNS1_NO_REC_FOUND       0x08
>>>>> +#define SNS1_FILE_PROTECTED     0x04
>>>>> +#define SNS1_WRITE_INHIBITED    0x02
>>>>> +#define SNS1_INPRECISE_END      0x01
>>>>> +
>>>>> +/* architectured values for third sense byte */
>>>>> +#define SNS2_REQ_INH_WRITE      0x80
>>>>> +#define SNS2_CORRECTABLE        0x40
>>>>> +#define SNS2_FIRST_LOG_ERR      0x20
>>>>> +#define SNS2_ENV_DATA_PRESENT   0x10
>>>>> +#define SNS2_INPRECISE_END      0x04
>>>>> +
>>>>> +/* 24-byte Sense fmt/msg codes */
>>>>> +#define SENSE24_FMT_PROG_SYS    0x0
>>>>> +#define SENSE24_FMT_EQUIPMENT   0x2
>>>>> +#define SENSE24_FMT_CONTROLLER  0x3
>>>>> +#define SENSE24_FMT_MISC        0xF
>>>>> +
>>>>> +#define SENSE24_FMT0_MSG_RESET_NOTIFICATION 0x16
>>>>> +
>>>>> +/* basic sense response buffer layout */
>>>>> +typedef struct senseData {
>>>>> +    uint8_t status[3];
>>>>> +    uint8_t res_count;
>>>>> +    uint8_t phys_drive_id;
>>>>> +    uint8_t low_cyl_addr;
>>>>> +    uint8_t head_high_cyl_addr;
>>>>> +    uint8_t fmt_msg;
>>>>> +    uint64_t fmt_dependent_info[2];
>>>>> +    uint8_t reserved;
>>>>> +    uint8_t program_action_code;
>>>>> +    uint16_t config_info;
>>>>> +    uint8_t mcode_hicyl;
>>>>> +    uint8_t cyl_head_addr[3];
>>>>> +}  __attribute__ ((packed, aligned(4))) SenseData;
>>>>
>>>> And this looks _really_ dasd specific.
>>>>   
>>>
>>> Yep, I glossed over those details while I was furiously tracking down the reset bug. I'll
>>> take a look at redesigning this.
>>
>> All of my information for creating these data structures came from an internal ECKD DASD
>> reference. There are probably some things that could stand a bit of cleanup or renaming.
>> Aside from that, considering this is in a DASD only (ECKD DASD only at the moment) code
>> path are you okay with my renaming the struct to senseDataECKD or something similar?
> 
> Renaming this makes sense.
> 
>> I'm not sure what value there is in abstracting sense at the moment. I'm not even sure
>> what other device's sense data looks like. Since my description of the SENSE CCW comes
>> from an ECKD reference I have not been able to verify any areas of the data that are
>> common across device types. Thoughts?
> 
> There's SA22-7204-01 ("Common I/O Device Commands"), which is from 1992
> (this is what I have on my disk -- is there anything newer?). It
> specifies what bits 0-5 of byte 0 mean and states that bytes 1-31 are
> optional and device-specific.
> 
> Maybe some other bits have been specified after 1992, but I have not
> come across documentation for them.
> 

So everything I'm finding (including SA22-7204-01) only speaks of the first 6 bits being 
device agnostic. It is odd that the entire first byte would not have been reserved. 
Perhaps it is and I'm just not looking at the right documentation. My thought process is 
to have the basic_Sense function change to this:

- void basic_sense(SubChannelId schid, SenseData *sd);
+ void basic_sense(SubChannelId schid, void *sense_data);

The caller is free to map it however they see fit depending on the device type they are 
sensing. I'll rename struct senseData to struct senseDataEckdDasd, break out the first 
status byte and define common constants for the first 6 bits of that byte. This should 
clear up the confusion. I'll add a small comment to explain. Does this sound good?


-- 
-- Jason J. Herne (jjherne@linux.ibm.com)

Re: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[no-reply]

Patchew URL: https://patchew.org/QEMU/1548768562-20007-1-git-send-email-jjherne@linux.ibm.com/



Hi,

This series seems to have some coding style problems. See output below for
more information:

Subject: [Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support
Type: series
Message-id: 1548768562-20007-1-git-send-email-jjherne@linux.ibm.com

=== TEST SCRIPT BEGIN ===
#!/bin/bash
git config --local diff.renamelimit 0
git config --local diff.renames True
git config --local diff.algorithm histogram
./scripts/checkpatch.pl --mailback base..
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
2bd738843d s390-bios: Support booting from real dasd device
a17332c8bf s390-bios: Add channel command codes/structs needed for dasd-ipl
1a0fc7faff s390-bios: Use control unit type to determine boot method
8f5f2c10de s390-bios: Refactor virtio to run channel programs via cio
9ede1038cd s390-bios: cio error handling
ab1f7e87aa s390-bios: Support for running format-0/1 channel programs
1b481ab578 s390-bios: ptr2u32 and u32toptr
0ff2ea42d5 s390-bios: Map low core memory
481d2eb86b s390-bios: Decouple channel i/o logic from virtio
b8bf17c714 s390-bios: Clean up cio.h
1b326192a9 s390-bios: Factor finding boot device out of virtio code path
66a8e27697 s390-bios: Extend find_dev() for non-virtio devices
bb7c632ef8 s390-bios: decouple common boot logic from virtio
c75b9d09eb s390-bios: decouple cio setup from virtio
7edc7ef234 s390 vfio-ccw: Add bootindex property and IPLB data

=== OUTPUT BEGIN ===
1/15 Checking commit 7edc7ef234c8 (s390 vfio-ccw: Add bootindex property and IPLB data)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#130: 
new file mode 100644

total: 0 errors, 1 warnings, 129 lines checked

Patch 1/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
2/15 Checking commit c75b9d09eb61 (s390-bios: decouple cio setup from virtio)
3/15 Checking commit bb7c632ef8b4 (s390-bios: decouple common boot logic from virtio)
ERROR: externs should be avoided in .c files
#30: FILE: pc-bios/s390-ccw/main.c:19:
+IplParameterBlock iplb __attribute__((__aligned__(PAGE_SIZE)));

total: 1 errors, 0 warnings, 65 lines checked

Patch 3/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

4/15 Checking commit 66a8e27697df (s390-bios: Extend find_dev() for non-virtio devices)
5/15 Checking commit 1b326192a91e (s390-bios: Factor finding boot device out of virtio code path)
6/15 Checking commit b8bf17c71419 (s390-bios: Clean up cio.h)
ERROR: spaces prohibited around that ':' (ctx:WxW)
#29: FILE: pc-bios/s390-ccw/cio.h:56:
+    __u32 isc    : 3;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#30: FILE: pc-bios/s390-ccw/cio.h:57:
+    __u32 ena    : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#31: FILE: pc-bios/s390-ccw/cio.h:58:
+    __u32 mme    : 2;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#32: FILE: pc-bios/s390-ccw/cio.h:59:
+    __u32 mp     : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#33: FILE: pc-bios/s390-ccw/cio.h:60:
+    __u32 csense : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#34: FILE: pc-bios/s390-ccw/cio.h:61:
+    __u32 mbfc   : 1;
                  ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#84: FILE: pc-bios/s390-ccw/cio.h:111:
+    __u32 reserved5 : 4;
                     ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#85: FILE: pc-bios/s390-ccw/cio.h:112:
+    __u32 format2   : 4;
                     ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#86: FILE: pc-bios/s390-ccw/cio.h:113:
+    __u32 reserved6 : 24;
                     ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#132: FILE: pc-bios/s390-ccw/cio.h:167:
+    __u32 key  : 4;   /* flags, like key, suspend control, etc. */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#133: FILE: pc-bios/s390-ccw/cio.h:168:
+    __u32 spnd : 1;   /* suspend control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#134: FILE: pc-bios/s390-ccw/cio.h:169:
+    __u32 res1 : 1;   /* reserved */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#135: FILE: pc-bios/s390-ccw/cio.h:170:
+    __u32 mod  : 1;   /* modification control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#136: FILE: pc-bios/s390-ccw/cio.h:171:
+    __u32 sync : 1;   /* synchronize control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#137: FILE: pc-bios/s390-ccw/cio.h:172:
+    __u32 fmt  : 1;   /* format control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#138: FILE: pc-bios/s390-ccw/cio.h:173:
+    __u32 pfch : 1;   /* prefetch control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#139: FILE: pc-bios/s390-ccw/cio.h:174:
+    __u32 isic : 1;   /* initial-status-interruption control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#140: FILE: pc-bios/s390-ccw/cio.h:175:
+    __u32 alcc : 1;   /* address-limit-checking control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#141: FILE: pc-bios/s390-ccw/cio.h:176:
+    __u32 ssic : 1;   /* suppress-suspended-interr. control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#142: FILE: pc-bios/s390-ccw/cio.h:177:
+    __u32 res2 : 1;   /* reserved */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#143: FILE: pc-bios/s390-ccw/cio.h:178:
+    __u32 c64  : 1;   /* IDAW/QDIO 64 bit control  */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#144: FILE: pc-bios/s390-ccw/cio.h:179:
+    __u32 i2k  : 1;   /* IDAW 2/4kB block size control */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#145: FILE: pc-bios/s390-ccw/cio.h:180:
+    __u32 lpm  : 8;   /* logical path mask */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#146: FILE: pc-bios/s390-ccw/cio.h:181:
+    __u32 ils  : 1;   /* incorrect length */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#147: FILE: pc-bios/s390-ccw/cio.h:182:
+    __u32 zero : 6;   /* reserved zeros */
                ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#148: FILE: pc-bios/s390-ccw/cio.h:183:
+    __u32 orbx : 1;   /* ORB extension control */
                ^

total: 26 errors, 0 warnings, 171 lines checked

Patch 6/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

7/15 Checking commit 481d2eb86b1a (s390-bios: Decouple channel i/o logic from virtio)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#28: 
new file mode 100644

total: 0 errors, 1 warnings, 127 lines checked

Patch 7/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
8/15 Checking commit 0ff2ea42d5e9 (s390-bios: Map low core memory)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#35: 
new file mode 100644

total: 0 errors, 1 warnings, 114 lines checked

Patch 8/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
9/15 Checking commit 1b481ab57836 (s390-bios: ptr2u32 and u32toptr)
10/15 Checking commit ab1f7e87aa13 (s390-bios: Support for running format-0/1 channel programs)
ERROR: spaces prohibited around that ':' (ctx:WxW)
#208: FILE: pc-bios/s390-ccw/cio.h:170:
+    __u32 cda        : 24;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#209: FILE: pc-bios/s390-ccw/cio.h:171:
+    __u32 chainData  : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#210: FILE: pc-bios/s390-ccw/cio.h:172:
+    __u32 chain      : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#211: FILE: pc-bios/s390-ccw/cio.h:173:
+    __u32 sli        : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#212: FILE: pc-bios/s390-ccw/cio.h:174:
+    __u32 skip       : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#213: FILE: pc-bios/s390-ccw/cio.h:175:
+    __u32 pci        : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#214: FILE: pc-bios/s390-ccw/cio.h:176:
+    __u32 ida        : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#215: FILE: pc-bios/s390-ccw/cio.h:177:
+    __u32 suspend    : 1;
                      ^

ERROR: spaces prohibited around that ':' (ctx:WxW)
#216: FILE: pc-bios/s390-ccw/cio.h:178:
+    __u32 mida       : 1;
                      ^

total: 9 errors, 0 warnings, 347 lines checked

Patch 10/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

11/15 Checking commit 9ede1038cd66 (s390-bios: cio error handling)
12/15 Checking commit 8f5f2c10dea5 (s390-bios: Refactor virtio to run channel programs via cio)
WARNING: line over 80 characters
#95: FILE: pc-bios/s390-ccw/virtio.c:297:
+            run_ccw(vdev, CCW_CMD_READ_VQ_CONF, &config, sizeof(config), false) == 0,

WARNING: line over 80 characters
#108: FILE: pc-bios/s390-ccw/virtio.c:307:
+        run_ccw(vdev, CCW_CMD_WRITE_STATUS, &status, sizeof(status), false) == 0,

total: 0 errors, 2 warnings, 100 lines checked

Patch 12/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
13/15 Checking commit 1a0fc7faff50 (s390-bios: Use control unit type to determine boot method)
14/15 Checking commit a17332c8bfa1 (s390-bios: Add channel command codes/structs needed for dasd-ipl)
15/15 Checking commit 2bd738843dfe (s390-bios: Support booting from real dasd device)
WARNING: added, moved or deleted file(s), does MAINTAINERS need updating?
#13: 
new file mode 100644

total: 0 errors, 1 warnings, 438 lines checked

Patch 15/15 has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
=== OUTPUT END ===

Test command exited with code: 1


The full log is available at
http://patchew.org/logs/1548768562-20007-1-git-send-email-jjherne@linux.ibm.com/testing.checkpatch/?type=message.
---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@redhat.com

[Qemu-devel] [PATCH 00/15] s390: vfio-ccw dasd ipl support

[Jason J. Herne]

This is to support booting from vfio-ccw dasd devices. We basically implement
the real hardware ipl procedure. This allows for booting Linux guests on
vfio-ccw devices.

vfio-ccw's channel program prefetch algorithm complicates ipl because most ipl
channel programs dynamically modify themselves. Details on the ipl process and
how we worked around this issue can be found in docs/devel/s390-dasd-ipl.txt.

This version brings better error handling, namely retrying I/O operations in the
event of some transient failure conditions. I've also renamed the sense data
structures and error output to indicate that they are for eckd dasd only. A few
other fixups as well, see changelog for details.

Changelog
==========
v2
---------
01/15: s390 vfio-ccw: Add bootindex property and IPLB data
- Remove accidental changes to roms/SLOF
- Fix bad include in hw/vfio/ccw.c
- ipl.c: Rename vc to vfio_ccw_dev

10/15: s390-bios: Support for running format-0/1 channel programs
- cio.c: Retry ssch on cc=1 or interface-control-check status
- cio.c: SLI now used instead of special code in irb_error
- Rename CU_TYPE_DASD to CU_TYPE_DASD_3990
- Rename dasd sense data structures and constants
- cio.c: Fix comment typos

11/15: s390-bios: cio error handling
- - Rename dasd sense data structures and constants

12/15: s390-bios: Refactor virtio to run channel programs via cio
- Use SLI for Sense ID ccws.

rfc -> v1
---------
- Retry start subchannel operations on unexpected unit check status
- Print more complete state information on i/o error
- Fixed asm constraints (Q -> QS)
- Align lowcore to 8k boundary
- Initialize next_cpa in run_dynamic_ccw_program to avoid compiler warning
- Cleanup: Rework handling of i/o interrupts in assembler
- Fix netboot image (include cio.o in make file)
- Misc cleanups

Jason J. Herne (15):
  s390 vfio-ccw: Add bootindex property and IPLB data
  s390-bios: decouple cio setup from virtio
  s390-bios: decouple common boot logic from virtio
  s390-bios: Extend find_dev() for non-virtio devices
  s390-bios: Factor finding boot device out of virtio code path
  s390-bios: Clean up cio.h
  s390-bios: Decouple channel i/o logic from virtio
  s390-bios: Map low core memory
  s390-bios: ptr2u32 and u32toptr
  s390-bios: Support for running format-0/1 channel programs
  s390-bios: cio error handling
  s390-bios: Refactor virtio to run channel programs via cio
  s390-bios: Use control unit type to determine boot method
  s390-bios: Add channel command codes/structs needed for dasd-ipl
  s390-bios: Support booting from real dasd device

 docs/devel/s390-dasd-ipl.txt     | 132 ++++++++++++++
 hw/s390x/ipl.c                   |  14 ++
 hw/s390x/s390-ccw.c              |   9 +
 hw/vfio/ccw.c                    |  13 +-
 include/hw/s390x/s390-ccw.h      |   1 +
 include/hw/s390x/vfio-ccw.h      |  38 ++++
 pc-bios/s390-ccw/Makefile        |   2 +-
 pc-bios/s390-ccw/cio.c           | 380 +++++++++++++++++++++++++++++++++++++++
 pc-bios/s390-ccw/cio.h           | 239 +++++++++++++++++++-----
 pc-bios/s390-ccw/dasd-ipl.c      | 249 +++++++++++++++++++++++++
 pc-bios/s390-ccw/dasd-ipl.h      |  16 ++
 pc-bios/s390-ccw/libc.h          |  23 +++
 pc-bios/s390-ccw/main.c          | 161 +++++++++++------
 pc-bios/s390-ccw/netboot.mak     |   2 +-
 pc-bios/s390-ccw/netmain.c       |   1 +
 pc-bios/s390-ccw/s390-arch.h     | 113 ++++++++++++
 pc-bios/s390-ccw/s390-ccw.h      |  10 +-
 pc-bios/s390-ccw/start.S         |  33 +++-
 pc-bios/s390-ccw/virtio-blkdev.c |   1 +
 pc-bios/s390-ccw/virtio.c        |  73 +++-----
 tests/boot-serial-test.c         |   2 +-
 21 files changed, 1332 insertions(+), 180 deletions(-)
 create mode 100644 docs/devel/s390-dasd-ipl.txt
 create mode 100644 include/hw/s390x/vfio-ccw.h
 create mode 100644 pc-bios/s390-ccw/cio.c
 create mode 100644 pc-bios/s390-ccw/dasd-ipl.c
 create mode 100644 pc-bios/s390-ccw/dasd-ipl.h
 create mode 100644 pc-bios/s390-ccw/s390-arch.h

--
2.7.4