All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/6] qemu: irqfds for s390x
@ 2014-02-25 17:25 ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Hi,

here's the companion patchset to "KVM: irqfds for s390", based on
Christian's s390-next branch.

Patch 1 adds support for adapter interrupts to virtio-ccw. This has been
posted before, modulo some fixes.

Patch 2 grabs the interfaces introduced by the kernel patchset. Will be
replaced by a normal headers update.

Patch 3 introduces simple wrappers for enabling capabilities. We can convert
existing users later on.

Patch 4 registers an I/O adapter when we enable adapter interrupts for a
device. We just have one virtio-ccw adapter for one interruption subclass.

Patch 5 adds reference counting for indicators. This is needed because we
want to map and pin indicator pages and drop them again when the last user
is gone.

Patch 6 finally enables irqfds via irq routing on s390x/virtio-ccw. We've
done most of the work in qemu already, so the kernel basically only needs
to write to the correct memory location and inject an interrupt.

A git branch with the patches is available at

https://github.com/cohuck/qemu.git s390x-irqfd


Cornelia Huck (6):
  s390x/virtio-ccw: Adapter interrupt support.
  linux-headers: add new interfaces
  kvm: add kvm_enable_cap_{vm,vcpu}
  s390x: Add I/O adapter registration.
  s390x/virtio-ccw: reference-counted indicators
  s390x/virtio-ccw: Wire up irq routing and irqfds.

 hw/intc/s390_flic.c          |   66 +++++++++
 hw/s390x/css.c               |   61 ++++++++
 hw/s390x/css.h               |    6 +
 hw/s390x/virtio-ccw.c        |  325 +++++++++++++++++++++++++++++++++++++++---
 hw/s390x/virtio-ccw.h        |   18 ++-
 include/hw/s390x/adapter.h   |   23 +++
 include/qemu/typedefs.h      |    1 +
 include/sysemu/kvm.h         |    6 +
 kvm-all.c                    |   57 +++++++-
 kvm-stub.c                   |    5 +
 linux-headers/asm-s390/kvm.h |   22 +++
 linux-headers/linux/kvm.h    |   16 +++
 target-s390x/cpu.h           |   33 +++++
 target-s390x/ioinst.h        |    2 +
 target-s390x/kvm.c           |   13 +-
 trace-events                 |    1 +
 16 files changed, 627 insertions(+), 28 deletions(-)
 create mode 100644 include/hw/s390x/adapter.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 0/6] qemu: irqfds for s390x
@ 2014-02-25 17:25 ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Hi,

here's the companion patchset to "KVM: irqfds for s390", based on
Christian's s390-next branch.

Patch 1 adds support for adapter interrupts to virtio-ccw. This has been
posted before, modulo some fixes.

Patch 2 grabs the interfaces introduced by the kernel patchset. Will be
replaced by a normal headers update.

Patch 3 introduces simple wrappers for enabling capabilities. We can convert
existing users later on.

Patch 4 registers an I/O adapter when we enable adapter interrupts for a
device. We just have one virtio-ccw adapter for one interruption subclass.

Patch 5 adds reference counting for indicators. This is needed because we
want to map and pin indicator pages and drop them again when the last user
is gone.

Patch 6 finally enables irqfds via irq routing on s390x/virtio-ccw. We've
done most of the work in qemu already, so the kernel basically only needs
to write to the correct memory location and inject an interrupt.

A git branch with the patches is available at

https://github.com/cohuck/qemu.git s390x-irqfd


Cornelia Huck (6):
  s390x/virtio-ccw: Adapter interrupt support.
  linux-headers: add new interfaces
  kvm: add kvm_enable_cap_{vm,vcpu}
  s390x: Add I/O adapter registration.
  s390x/virtio-ccw: reference-counted indicators
  s390x/virtio-ccw: Wire up irq routing and irqfds.

 hw/intc/s390_flic.c          |   66 +++++++++
 hw/s390x/css.c               |   61 ++++++++
 hw/s390x/css.h               |    6 +
 hw/s390x/virtio-ccw.c        |  325 +++++++++++++++++++++++++++++++++++++++---
 hw/s390x/virtio-ccw.h        |   18 ++-
 include/hw/s390x/adapter.h   |   23 +++
 include/qemu/typedefs.h      |    1 +
 include/sysemu/kvm.h         |    6 +
 kvm-all.c                    |   57 +++++++-
 kvm-stub.c                   |    5 +
 linux-headers/asm-s390/kvm.h |   22 +++
 linux-headers/linux/kvm.h    |   16 +++
 target-s390x/cpu.h           |   33 +++++
 target-s390x/ioinst.h        |    2 +
 target-s390x/kvm.c           |   13 +-
 trace-events                 |    1 +
 16 files changed, 627 insertions(+), 28 deletions(-)
 create mode 100644 include/hw/s390x/adapter.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 21+ messages in thread

* [PATCH 1/6] s390x/virtio-ccw: Adapter interrupt support.
  2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:25   ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Handle the new CCW_CMD_SET_IND_ADAPTER command enabling adapter interrupts
on guest request. When active, host->guest notifications will be handled
via global_indicator -> queue indicators instead of queue indicators +
subchannel I/O interrupt. Indicators for virtqueues may be present at an
offset.

Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/css.c        |   10 +++++
 hw/s390x/css.h        |    2 +
 hw/s390x/virtio-ccw.c |  102 +++++++++++++++++++++++++++++++++++++++++++++----
 hw/s390x/virtio-ccw.h |    4 ++
 target-s390x/ioinst.h |    2 +
 target-s390x/kvm.c    |    8 +++-
 trace-events          |    1 +
 7 files changed, 120 insertions(+), 9 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 75b04b4..7074d2b 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -116,6 +116,15 @@ void css_conditional_io_interrupt(SubchDev *sch)
     }
 }
 
+void css_adapter_interrupt(uint8_t isc)
+{
+    S390CPU *cpu = s390_cpu_addr2state(0);
+    uint32_t io_int_word = (isc << 27) | IO_INT_WORD_AI;
+
+    trace_css_adapter_interrupt(isc);
+    s390_io_interrupt(cpu, 0, 0, 0, io_int_word);
+}
+
 static void sch_handle_clear_func(SubchDev *sch)
 {
     PMCW *p = &sch->curr_status.pmcw;
@@ -1259,6 +1268,7 @@ void css_reset_sch(SubchDev *sch)
     sch->channel_prog = 0x0;
     sch->last_cmd_valid = false;
     sch->orb = NULL;
+    sch->thinint_active = false;
 }
 
 void css_reset(void)
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index b536ab5..e9b4405 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -77,6 +77,7 @@ struct SubchDev {
     CCW1 last_cmd;
     bool last_cmd_valid;
     ORB *orb;
+    bool thinint_active;
     /* transport-provided data: */
     int (*ccw_cb) (SubchDev *, CCW1);
     SenseId id;
@@ -97,4 +98,5 @@ void css_queue_crw(uint8_t rsc, uint8_t erc, int chain, uint16_t rsid);
 void css_generate_sch_crws(uint8_t cssid, uint8_t ssid, uint16_t schid,
                            int hotplugged, int add);
 void css_generate_chp_crws(uint8_t cssid, uint8_t chpid);
+void css_adapter_interrupt(uint8_t isc);
 #endif
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index f6e0e3e..a01801e 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -1,7 +1,7 @@
 /*
  * virtio ccw target implementation
  *
- * Copyright 2012 IBM Corp.
+ * Copyright 2012,2014 IBM Corp.
  * Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or (at
@@ -188,6 +188,13 @@ typedef struct VirtioFeatDesc {
     uint8_t index;
 } QEMU_PACKED VirtioFeatDesc;
 
+typedef struct VirtioThinintInfo {
+    hwaddr summary_indicator;
+    hwaddr device_indicator;
+    uint64_t ind_bit;
+    uint8_t isc;
+} QEMU_PACKED VirtioThinintInfo;
+
 /* Specify where the virtqueues for the subchannel are in guest memory. */
 static int virtio_ccw_set_vqs(SubchDev *sch, uint64_t addr, uint32_t align,
                               uint16_t index, uint16_t num)
@@ -237,6 +244,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
     bool check_len;
     int len;
     hwaddr hw_len;
+    VirtioThinintInfo *thinint;
 
     if (!dev) {
         return -EINVAL;
@@ -428,6 +436,11 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = -EINVAL;
             break;
         }
+        if (sch->thinint_active) {
+            /* Trigger a command reject. */
+            ret = -ENOSYS;
+            break;
+        }
         if (!ccw.cda) {
             ret = -EFAULT;
         } else {
@@ -480,6 +493,42 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = 0;
         }
         break;
+    case CCW_CMD_SET_IND_ADAPTER:
+        if (check_len) {
+            if (ccw.count != sizeof(*thinint)) {
+                ret = -EINVAL;
+                break;
+            }
+        } else if (ccw.count < sizeof(*thinint)) {
+            /* Can't execute command. */
+            ret = -EINVAL;
+            break;
+        }
+        len = sizeof(*thinint);
+        hw_len = len;
+        if (!ccw.cda) {
+            ret = -EFAULT;
+        } else if (dev->indicators && !sch->thinint_active) {
+            /* Trigger a command reject. */
+            ret = -ENOSYS;
+        } else {
+            thinint = cpu_physical_memory_map(ccw.cda, &hw_len, 0);
+            if (!thinint) {
+                ret = -EFAULT;
+            } else {
+                len = hw_len;
+                dev->summary_indicator = thinint->summary_indicator;
+                dev->indicators = thinint->device_indicator;
+                dev->thinint_isc = thinint->isc;
+                dev->ind_bit = thinint->ind_bit;
+                cpu_physical_memory_unmap(thinint, hw_len, 0, hw_len);
+                sch->thinint_active = ((dev->indicators != 0) &&
+                                       (dev->summary_indicator != 0));
+                sch->curr_status.scsw.count = ccw.count - len;
+                ret = 0;
+            }
+        }
+        break;
     default:
         ret = -ENOSYS;
         break;
@@ -511,6 +560,7 @@ static int virtio_ccw_device_init(VirtioCcwDevice *dev, VirtIODevice *vdev)
     sch->channel_prog = 0x0;
     sch->last_cmd_valid = false;
     sch->orb = NULL;
+    sch->thinint_active = false;
     /*
      * Use a device number if provided. Otherwise, fall back to subchannel
      * number.
@@ -858,6 +908,28 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)
     return container_of(d, VirtioCcwDevice, parent_obj);
 }
 
+static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc,
+                                     uint8_t to_be_set)
+{
+    uint8_t ind_old, ind_new;
+    hwaddr len = 1;
+    uint8_t *ind_addr;
+
+    ind_addr = cpu_physical_memory_map(ind_loc, &len, 1);
+    if (!ind_addr) {
+        error_report("%s(%x.%x.%04x): unable to access indicator",
+                     __func__, sch->cssid, sch->ssid, sch->schid);
+        return -1;
+    }
+    do {
+        ind_old = *ind_addr;
+        ind_new = ind_old | to_be_set;
+    } while (atomic_cmpxchg(ind_addr, ind_old, ind_new) != ind_old);
+    cpu_physical_memory_unmap(ind_addr, len, 1, len);
+
+    return ind_old;
+}
+
 static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
 {
     VirtioCcwDevice *dev = to_virtio_ccw_dev_fast(d);
@@ -872,9 +944,26 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
         if (!dev->indicators) {
             return;
         }
-        indicators = ldq_phys(&address_space_memory, dev->indicators);
-        indicators |= 1ULL << vector;
-        stq_phys(&address_space_memory, dev->indicators, indicators);
+        if (sch->thinint_active) {
+            /*
+             * In the adapter interrupt case, indicators points to a
+             * memory area that may be (way) larger than 64 bit and
+             * ind_bit indicates the start of the indicators in a big
+             * endian notation.
+             */
+            virtio_set_ind_atomic(sch, dev->indicators +
+                                  (dev->ind_bit + vector) / 8,
+                                  0x80 >> ((dev->ind_bit + vector) % 8));
+            if (!virtio_set_ind_atomic(sch, dev->summary_indicator,
+                                       0x01)) {
+                css_adapter_interrupt(dev->thinint_isc);
+            }
+        } else {
+            indicators = ldq_phys(&address_space_memory, dev->indicators);
+            indicators |= 1ULL << vector;
+            stq_phys(&address_space_memory, dev->indicators, indicators);
+            css_conditional_io_interrupt(sch);
+        }
     } else {
         if (!dev->indicators2) {
             return;
@@ -883,10 +972,8 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
         indicators = ldq_phys(&address_space_memory, dev->indicators2);
         indicators |= 1ULL << vector;
         stq_phys(&address_space_memory, dev->indicators2, indicators);
+        css_conditional_io_interrupt(sch);
     }
-
-    css_conditional_io_interrupt(sch);
-
 }
 
 static unsigned virtio_ccw_get_features(DeviceState *d)
@@ -907,6 +994,7 @@ static void virtio_ccw_reset(DeviceState *d)
     css_reset_sch(dev->sch);
     dev->indicators = 0;
     dev->indicators2 = 0;
+    dev->summary_indicator = 0;
 }
 
 static void virtio_ccw_vmstate_change(DeviceState *d, bool running)
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 00932c7..4393e44 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -38,6 +38,7 @@
 #define CCW_CMD_SET_IND      0x43
 #define CCW_CMD_SET_CONF_IND 0x53
 #define CCW_CMD_READ_VQ_CONF 0x32
+#define CCW_CMD_SET_IND_ADAPTER 0x73
 
 #define TYPE_VIRTIO_CCW_DEVICE "virtio-ccw-device"
 #define VIRTIO_CCW_DEVICE(obj) \
@@ -83,9 +84,12 @@ struct VirtioCcwDevice {
     bool ioeventfd_started;
     bool ioeventfd_disabled;
     uint32_t flags;
+    uint8_t thinint_isc;
     /* Guest provided values: */
     hwaddr indicators;
     hwaddr indicators2;
+    hwaddr summary_indicator;
+    uint64_t ind_bit;
 };
 
 /* virtual css bus type */
diff --git a/target-s390x/ioinst.h b/target-s390x/ioinst.h
index 613da49..5bbc67d 100644
--- a/target-s390x/ioinst.h
+++ b/target-s390x/ioinst.h
@@ -212,6 +212,8 @@ typedef struct IOIntCode {
 #define IO_INT_WORD_ISC(_int_word) ((_int_word & 0x38000000) >> 24)
 #define ISC_TO_ISC_BITS(_isc)      ((0x80 >> _isc) << 24)
 
+#define IO_INT_WORD_AI 0x80000000
+
 int ioinst_disassemble_sch_ident(uint32_t value, int *m, int *cssid, int *ssid,
                                  int *schid);
 void ioinst_handle_xsch(S390CPU *cpu, uint64_t reg1);
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index d3f0d4a..7e8321e 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -868,8 +868,12 @@ void kvm_s390_io_interrupt(S390CPU *cpu, uint16_t subchannel_id,
 {
     uint32_t type;
 
-    type = ((subchannel_id & 0xff00) << 24) |
-        ((subchannel_id & 0x00060) << 22) | (subchannel_nr << 16);
+    if (io_int_word & IO_INT_WORD_AI) {
+        type = KVM_S390_INT_IO(1, 0, 0, 0);
+    } else {
+        type = ((subchannel_id & 0xff00) << 24) |
+            ((subchannel_id & 0x00060) << 22) | (subchannel_nr << 16);
+    }
     kvm_s390_interrupt_internal(cpu, type,
                                 ((uint32_t)subchannel_id << 16) | subchannel_nr,
                                 ((uint64_t)io_int_parm << 32) | io_int_word, 1);
diff --git a/trace-events b/trace-events
index 8bc7cc4..372ea64 100644
--- a/trace-events
+++ b/trace-events
@@ -1157,6 +1157,7 @@ css_chpid_add(uint8_t cssid, uint8_t chpid, uint8_t type) "CSS: add chpid %x.%02
 css_new_image(uint8_t cssid, const char *default_cssid) "CSS: add css image %02x %s"
 css_assign_subch(const char *do_assign, uint8_t cssid, uint8_t ssid, uint16_t schid, uint16_t devno) "CSS: %s %x.%x.%04x (devno %04x)"
 css_io_interrupt(int cssid, int ssid, int schid, uint32_t intparm, uint8_t isc, const char *conditional) "CSS: I/O interrupt on sch %x.%x.%04x (intparm %08x, isc %x) %s"
+css_adapter_interrupt(uint8_t isc) "CSS: adapter I/O interrupt (isc %x)"
 
 # hw/s390x/virtio-ccw.c
 virtio_ccw_interpret_ccw(int cssid, int ssid, int schid, int cmd_code) "VIRTIO-CCW: %x.%x.%04x: interpret command %x"
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 1/6] s390x/virtio-ccw: Adapter interrupt support.
@ 2014-02-25 17:25   ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Handle the new CCW_CMD_SET_IND_ADAPTER command enabling adapter interrupts
on guest request. When active, host->guest notifications will be handled
via global_indicator -> queue indicators instead of queue indicators +
subchannel I/O interrupt. Indicators for virtqueues may be present at an
offset.

Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/css.c        |   10 +++++
 hw/s390x/css.h        |    2 +
 hw/s390x/virtio-ccw.c |  102 +++++++++++++++++++++++++++++++++++++++++++++----
 hw/s390x/virtio-ccw.h |    4 ++
 target-s390x/ioinst.h |    2 +
 target-s390x/kvm.c    |    8 +++-
 trace-events          |    1 +
 7 files changed, 120 insertions(+), 9 deletions(-)

diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 75b04b4..7074d2b 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -116,6 +116,15 @@ void css_conditional_io_interrupt(SubchDev *sch)
     }
 }
 
+void css_adapter_interrupt(uint8_t isc)
+{
+    S390CPU *cpu = s390_cpu_addr2state(0);
+    uint32_t io_int_word = (isc << 27) | IO_INT_WORD_AI;
+
+    trace_css_adapter_interrupt(isc);
+    s390_io_interrupt(cpu, 0, 0, 0, io_int_word);
+}
+
 static void sch_handle_clear_func(SubchDev *sch)
 {
     PMCW *p = &sch->curr_status.pmcw;
@@ -1259,6 +1268,7 @@ void css_reset_sch(SubchDev *sch)
     sch->channel_prog = 0x0;
     sch->last_cmd_valid = false;
     sch->orb = NULL;
+    sch->thinint_active = false;
 }
 
 void css_reset(void)
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index b536ab5..e9b4405 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -77,6 +77,7 @@ struct SubchDev {
     CCW1 last_cmd;
     bool last_cmd_valid;
     ORB *orb;
+    bool thinint_active;
     /* transport-provided data: */
     int (*ccw_cb) (SubchDev *, CCW1);
     SenseId id;
@@ -97,4 +98,5 @@ void css_queue_crw(uint8_t rsc, uint8_t erc, int chain, uint16_t rsid);
 void css_generate_sch_crws(uint8_t cssid, uint8_t ssid, uint16_t schid,
                            int hotplugged, int add);
 void css_generate_chp_crws(uint8_t cssid, uint8_t chpid);
+void css_adapter_interrupt(uint8_t isc);
 #endif
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index f6e0e3e..a01801e 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -1,7 +1,7 @@
 /*
  * virtio ccw target implementation
  *
- * Copyright 2012 IBM Corp.
+ * Copyright 2012,2014 IBM Corp.
  * Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or (at
@@ -188,6 +188,13 @@ typedef struct VirtioFeatDesc {
     uint8_t index;
 } QEMU_PACKED VirtioFeatDesc;
 
+typedef struct VirtioThinintInfo {
+    hwaddr summary_indicator;
+    hwaddr device_indicator;
+    uint64_t ind_bit;
+    uint8_t isc;
+} QEMU_PACKED VirtioThinintInfo;
+
 /* Specify where the virtqueues for the subchannel are in guest memory. */
 static int virtio_ccw_set_vqs(SubchDev *sch, uint64_t addr, uint32_t align,
                               uint16_t index, uint16_t num)
@@ -237,6 +244,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
     bool check_len;
     int len;
     hwaddr hw_len;
+    VirtioThinintInfo *thinint;
 
     if (!dev) {
         return -EINVAL;
@@ -428,6 +436,11 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = -EINVAL;
             break;
         }
+        if (sch->thinint_active) {
+            /* Trigger a command reject. */
+            ret = -ENOSYS;
+            break;
+        }
         if (!ccw.cda) {
             ret = -EFAULT;
         } else {
@@ -480,6 +493,42 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = 0;
         }
         break;
+    case CCW_CMD_SET_IND_ADAPTER:
+        if (check_len) {
+            if (ccw.count != sizeof(*thinint)) {
+                ret = -EINVAL;
+                break;
+            }
+        } else if (ccw.count < sizeof(*thinint)) {
+            /* Can't execute command. */
+            ret = -EINVAL;
+            break;
+        }
+        len = sizeof(*thinint);
+        hw_len = len;
+        if (!ccw.cda) {
+            ret = -EFAULT;
+        } else if (dev->indicators && !sch->thinint_active) {
+            /* Trigger a command reject. */
+            ret = -ENOSYS;
+        } else {
+            thinint = cpu_physical_memory_map(ccw.cda, &hw_len, 0);
+            if (!thinint) {
+                ret = -EFAULT;
+            } else {
+                len = hw_len;
+                dev->summary_indicator = thinint->summary_indicator;
+                dev->indicators = thinint->device_indicator;
+                dev->thinint_isc = thinint->isc;
+                dev->ind_bit = thinint->ind_bit;
+                cpu_physical_memory_unmap(thinint, hw_len, 0, hw_len);
+                sch->thinint_active = ((dev->indicators != 0) &&
+                                       (dev->summary_indicator != 0));
+                sch->curr_status.scsw.count = ccw.count - len;
+                ret = 0;
+            }
+        }
+        break;
     default:
         ret = -ENOSYS;
         break;
@@ -511,6 +560,7 @@ static int virtio_ccw_device_init(VirtioCcwDevice *dev, VirtIODevice *vdev)
     sch->channel_prog = 0x0;
     sch->last_cmd_valid = false;
     sch->orb = NULL;
+    sch->thinint_active = false;
     /*
      * Use a device number if provided. Otherwise, fall back to subchannel
      * number.
@@ -858,6 +908,28 @@ static inline VirtioCcwDevice *to_virtio_ccw_dev_fast(DeviceState *d)
     return container_of(d, VirtioCcwDevice, parent_obj);
 }
 
+static uint8_t virtio_set_ind_atomic(SubchDev *sch, uint64_t ind_loc,
+                                     uint8_t to_be_set)
+{
+    uint8_t ind_old, ind_new;
+    hwaddr len = 1;
+    uint8_t *ind_addr;
+
+    ind_addr = cpu_physical_memory_map(ind_loc, &len, 1);
+    if (!ind_addr) {
+        error_report("%s(%x.%x.%04x): unable to access indicator",
+                     __func__, sch->cssid, sch->ssid, sch->schid);
+        return -1;
+    }
+    do {
+        ind_old = *ind_addr;
+        ind_new = ind_old | to_be_set;
+    } while (atomic_cmpxchg(ind_addr, ind_old, ind_new) != ind_old);
+    cpu_physical_memory_unmap(ind_addr, len, 1, len);
+
+    return ind_old;
+}
+
 static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
 {
     VirtioCcwDevice *dev = to_virtio_ccw_dev_fast(d);
@@ -872,9 +944,26 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
         if (!dev->indicators) {
             return;
         }
-        indicators = ldq_phys(&address_space_memory, dev->indicators);
-        indicators |= 1ULL << vector;
-        stq_phys(&address_space_memory, dev->indicators, indicators);
+        if (sch->thinint_active) {
+            /*
+             * In the adapter interrupt case, indicators points to a
+             * memory area that may be (way) larger than 64 bit and
+             * ind_bit indicates the start of the indicators in a big
+             * endian notation.
+             */
+            virtio_set_ind_atomic(sch, dev->indicators +
+                                  (dev->ind_bit + vector) / 8,
+                                  0x80 >> ((dev->ind_bit + vector) % 8));
+            if (!virtio_set_ind_atomic(sch, dev->summary_indicator,
+                                       0x01)) {
+                css_adapter_interrupt(dev->thinint_isc);
+            }
+        } else {
+            indicators = ldq_phys(&address_space_memory, dev->indicators);
+            indicators |= 1ULL << vector;
+            stq_phys(&address_space_memory, dev->indicators, indicators);
+            css_conditional_io_interrupt(sch);
+        }
     } else {
         if (!dev->indicators2) {
             return;
@@ -883,10 +972,8 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
         indicators = ldq_phys(&address_space_memory, dev->indicators2);
         indicators |= 1ULL << vector;
         stq_phys(&address_space_memory, dev->indicators2, indicators);
+        css_conditional_io_interrupt(sch);
     }
-
-    css_conditional_io_interrupt(sch);
-
 }
 
 static unsigned virtio_ccw_get_features(DeviceState *d)
@@ -907,6 +994,7 @@ static void virtio_ccw_reset(DeviceState *d)
     css_reset_sch(dev->sch);
     dev->indicators = 0;
     dev->indicators2 = 0;
+    dev->summary_indicator = 0;
 }
 
 static void virtio_ccw_vmstate_change(DeviceState *d, bool running)
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 00932c7..4393e44 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -38,6 +38,7 @@
 #define CCW_CMD_SET_IND      0x43
 #define CCW_CMD_SET_CONF_IND 0x53
 #define CCW_CMD_READ_VQ_CONF 0x32
+#define CCW_CMD_SET_IND_ADAPTER 0x73
 
 #define TYPE_VIRTIO_CCW_DEVICE "virtio-ccw-device"
 #define VIRTIO_CCW_DEVICE(obj) \
@@ -83,9 +84,12 @@ struct VirtioCcwDevice {
     bool ioeventfd_started;
     bool ioeventfd_disabled;
     uint32_t flags;
+    uint8_t thinint_isc;
     /* Guest provided values: */
     hwaddr indicators;
     hwaddr indicators2;
+    hwaddr summary_indicator;
+    uint64_t ind_bit;
 };
 
 /* virtual css bus type */
diff --git a/target-s390x/ioinst.h b/target-s390x/ioinst.h
index 613da49..5bbc67d 100644
--- a/target-s390x/ioinst.h
+++ b/target-s390x/ioinst.h
@@ -212,6 +212,8 @@ typedef struct IOIntCode {
 #define IO_INT_WORD_ISC(_int_word) ((_int_word & 0x38000000) >> 24)
 #define ISC_TO_ISC_BITS(_isc)      ((0x80 >> _isc) << 24)
 
+#define IO_INT_WORD_AI 0x80000000
+
 int ioinst_disassemble_sch_ident(uint32_t value, int *m, int *cssid, int *ssid,
                                  int *schid);
 void ioinst_handle_xsch(S390CPU *cpu, uint64_t reg1);
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index d3f0d4a..7e8321e 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -868,8 +868,12 @@ void kvm_s390_io_interrupt(S390CPU *cpu, uint16_t subchannel_id,
 {
     uint32_t type;
 
-    type = ((subchannel_id & 0xff00) << 24) |
-        ((subchannel_id & 0x00060) << 22) | (subchannel_nr << 16);
+    if (io_int_word & IO_INT_WORD_AI) {
+        type = KVM_S390_INT_IO(1, 0, 0, 0);
+    } else {
+        type = ((subchannel_id & 0xff00) << 24) |
+            ((subchannel_id & 0x00060) << 22) | (subchannel_nr << 16);
+    }
     kvm_s390_interrupt_internal(cpu, type,
                                 ((uint32_t)subchannel_id << 16) | subchannel_nr,
                                 ((uint64_t)io_int_parm << 32) | io_int_word, 1);
diff --git a/trace-events b/trace-events
index 8bc7cc4..372ea64 100644
--- a/trace-events
+++ b/trace-events
@@ -1157,6 +1157,7 @@ css_chpid_add(uint8_t cssid, uint8_t chpid, uint8_t type) "CSS: add chpid %x.%02
 css_new_image(uint8_t cssid, const char *default_cssid) "CSS: add css image %02x %s"
 css_assign_subch(const char *do_assign, uint8_t cssid, uint8_t ssid, uint16_t schid, uint16_t devno) "CSS: %s %x.%x.%04x (devno %04x)"
 css_io_interrupt(int cssid, int ssid, int schid, uint32_t intparm, uint8_t isc, const char *conditional) "CSS: I/O interrupt on sch %x.%x.%04x (intparm %08x, isc %x) %s"
+css_adapter_interrupt(uint8_t isc) "CSS: adapter I/O interrupt (isc %x)"
 
 # hw/s390x/virtio-ccw.c
 virtio_ccw_interpret_ccw(int cssid, int ssid, int schid, int cmd_code) "VIRTIO-CCW: %x.%x.%04x: interpret command %x"
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH 2/6] linux-headers: add new interfaces
  2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:25   ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Per-vm capability enablement, adapter interrupt sources, irq routing on s390.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 linux-headers/asm-s390/kvm.h |   22 ++++++++++++++++++++++
 linux-headers/linux/kvm.h    |   16 ++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h
index cb4c1eb..a0761e2 100644
--- a/linux-headers/asm-s390/kvm.h
+++ b/linux-headers/asm-s390/kvm.h
@@ -22,6 +22,8 @@
 #define KVM_DEV_FLIC_CLEAR_IRQS		3
 #define KVM_DEV_FLIC_APF_ENABLE		4
 #define KVM_DEV_FLIC_APF_DISABLE_WAIT	5
+#define KVM_DEV_FLIC_ADAPTER_REGISTER	6
+#define KVM_DEV_FLIC_ADAPTER_MODIFY	7
 /*
  * We can have up to 4*64k pending subchannels + 8 adapter interrupts,
  * as well as up  to ASYNC_PF_PER_VCPU*KVM_MAX_VCPUS pfault done interrupts.
@@ -32,6 +34,26 @@
 #define KVM_S390_MAX_FLOAT_IRQS	266250
 #define KVM_S390_FLIC_MAX_BUFFER	0x2000000
 
+struct kvm_s390_io_adapter {
+	__u32 id;
+	__u8 isc;
+	__u8 maskable;
+	__u8 swap;
+	__u8 pad;
+};
+
+#define KVM_S390_IO_ADAPTER_MASK 1
+#define KVM_S390_IO_ADAPTER_MAP 2
+#define KVM_S390_IO_ADAPTER_UNMAP 3
+
+struct kvm_s390_io_adapter_req {
+	__u32 id;
+	__u8 type;
+	__u8 mask;
+	__u16 pad0;
+	__u64 addr;
+};
+
 /* for KVM_GET_REGS and KVM_SET_REGS */
 struct kvm_regs {
 	/* general purpose regs for s390 */
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index e27a4b3..820b250 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -740,6 +740,8 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_SPAPR_MULTITCE 94
 #define KVM_CAP_EXT_EMUL_CPUID 95
 #define KVM_CAP_HYPERV_TIME 96
+#define KVM_CAP_ENABLE_CAP_VM 97
+#define KVM_CAP_S390_IRQCHIP 98
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -755,9 +757,18 @@ struct kvm_irq_routing_msi {
 	__u32 pad;
 };
 
+struct kvm_irq_routing_s390_adapter {
+	__u64 ind_addr;
+	__u64 summary_addr;
+	__u64 ind_offset;
+	__u32 summary_offset;
+	__u32 adapter_id;
+};
+
 /* gsi routing entry types */
 #define KVM_IRQ_ROUTING_IRQCHIP 1
 #define KVM_IRQ_ROUTING_MSI 2
+#define KVM_IRQ_ROUTING_S390_ADAPTER 3
 
 struct kvm_irq_routing_entry {
 	__u32 gsi;
@@ -767,6 +778,7 @@ struct kvm_irq_routing_entry {
 	union {
 		struct kvm_irq_routing_irqchip irqchip;
 		struct kvm_irq_routing_msi msi;
+		struct kvm_irq_routing_s390_adapter adapter;
 		__u32 pad[8];
 	} u;
 };
@@ -1075,6 +1087,10 @@ struct kvm_s390_ucas_mapping {
 /* Available with KVM_CAP_DEBUGREGS */
 #define KVM_GET_DEBUGREGS         _IOR(KVMIO,  0xa1, struct kvm_debugregs)
 #define KVM_SET_DEBUGREGS         _IOW(KVMIO,  0xa2, struct kvm_debugregs)
+/*
+ * vcpu version available with KVM_ENABLE_CAP
+ * vm version available with KVM_CAP_ENABLE_CAP_VM
+ */
 #define KVM_ENABLE_CAP            _IOW(KVMIO,  0xa3, struct kvm_enable_cap)
 /* Available with KVM_CAP_XSAVE */
 #define KVM_GET_XSAVE		  _IOR(KVMIO,  0xa4, struct kvm_xsave)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 2/6] linux-headers: add new interfaces
@ 2014-02-25 17:25   ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Per-vm capability enablement, adapter interrupt sources, irq routing on s390.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 linux-headers/asm-s390/kvm.h |   22 ++++++++++++++++++++++
 linux-headers/linux/kvm.h    |   16 ++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/linux-headers/asm-s390/kvm.h b/linux-headers/asm-s390/kvm.h
index cb4c1eb..a0761e2 100644
--- a/linux-headers/asm-s390/kvm.h
+++ b/linux-headers/asm-s390/kvm.h
@@ -22,6 +22,8 @@
 #define KVM_DEV_FLIC_CLEAR_IRQS		3
 #define KVM_DEV_FLIC_APF_ENABLE		4
 #define KVM_DEV_FLIC_APF_DISABLE_WAIT	5
+#define KVM_DEV_FLIC_ADAPTER_REGISTER	6
+#define KVM_DEV_FLIC_ADAPTER_MODIFY	7
 /*
  * We can have up to 4*64k pending subchannels + 8 adapter interrupts,
  * as well as up  to ASYNC_PF_PER_VCPU*KVM_MAX_VCPUS pfault done interrupts.
@@ -32,6 +34,26 @@
 #define KVM_S390_MAX_FLOAT_IRQS	266250
 #define KVM_S390_FLIC_MAX_BUFFER	0x2000000
 
+struct kvm_s390_io_adapter {
+	__u32 id;
+	__u8 isc;
+	__u8 maskable;
+	__u8 swap;
+	__u8 pad;
+};
+
+#define KVM_S390_IO_ADAPTER_MASK 1
+#define KVM_S390_IO_ADAPTER_MAP 2
+#define KVM_S390_IO_ADAPTER_UNMAP 3
+
+struct kvm_s390_io_adapter_req {
+	__u32 id;
+	__u8 type;
+	__u8 mask;
+	__u16 pad0;
+	__u64 addr;
+};
+
 /* for KVM_GET_REGS and KVM_SET_REGS */
 struct kvm_regs {
 	/* general purpose regs for s390 */
diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index e27a4b3..820b250 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -740,6 +740,8 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_SPAPR_MULTITCE 94
 #define KVM_CAP_EXT_EMUL_CPUID 95
 #define KVM_CAP_HYPERV_TIME 96
+#define KVM_CAP_ENABLE_CAP_VM 97
+#define KVM_CAP_S390_IRQCHIP 98
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -755,9 +757,18 @@ struct kvm_irq_routing_msi {
 	__u32 pad;
 };
 
+struct kvm_irq_routing_s390_adapter {
+	__u64 ind_addr;
+	__u64 summary_addr;
+	__u64 ind_offset;
+	__u32 summary_offset;
+	__u32 adapter_id;
+};
+
 /* gsi routing entry types */
 #define KVM_IRQ_ROUTING_IRQCHIP 1
 #define KVM_IRQ_ROUTING_MSI 2
+#define KVM_IRQ_ROUTING_S390_ADAPTER 3
 
 struct kvm_irq_routing_entry {
 	__u32 gsi;
@@ -767,6 +778,7 @@ struct kvm_irq_routing_entry {
 	union {
 		struct kvm_irq_routing_irqchip irqchip;
 		struct kvm_irq_routing_msi msi;
+		struct kvm_irq_routing_s390_adapter adapter;
 		__u32 pad[8];
 	} u;
 };
@@ -1075,6 +1087,10 @@ struct kvm_s390_ucas_mapping {
 /* Available with KVM_CAP_DEBUGREGS */
 #define KVM_GET_DEBUGREGS         _IOR(KVMIO,  0xa1, struct kvm_debugregs)
 #define KVM_SET_DEBUGREGS         _IOW(KVMIO,  0xa2, struct kvm_debugregs)
+/*
+ * vcpu version available with KVM_ENABLE_CAP
+ * vm version available with KVM_CAP_ENABLE_CAP_VM
+ */
 #define KVM_ENABLE_CAP            _IOW(KVMIO,  0xa3, struct kvm_enable_cap)
 /* Available with KVM_CAP_XSAVE */
 #define KVM_GET_XSAVE		  _IOR(KVMIO,  0xa4, struct kvm_xsave)
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH 3/6] kvm: add kvm_enable_cap_{vm,vcpu}
  2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:25   ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Provide helper functions for enabling capabilities (on a vcpu and on a vm).

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 include/sysemu/kvm.h |    4 ++++
 kvm-all.c            |   19 ++++++++++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 3b25f27..9d2df88 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -272,6 +272,10 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cpu);
 
 int kvm_check_extension(KVMState *s, unsigned int extension);
 
+int kvm_enable_cap_vm(KVMState *s, unsigned int capability);
+
+int kvm_enable_cap_vcpu(CPUState *cpu, unsigned int capability);
+
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 
diff --git a/kvm-all.c b/kvm-all.c
index 2ca9143..6e6a804 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -499,7 +499,24 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
     return ret;
 }
 
-static int kvm_set_ioeventfd_mmio(int fd, hwaddr addr, uint32_t val,
+int kvm_enable_cap_vm(KVMState *s, unsigned int capability)
+{
+    struct kvm_enable_cap cap = {};
+
+    cap.cap = capability;
+    return kvm_vm_ioctl(s, KVM_ENABLE_CAP, &cap);
+}
+
+int kvm_enable_cap_vcpu(CPUState *cpu, unsigned int capability)
+{
+    struct kvm_enable_cap cap = {};
+
+    cap.cap = capability;
+    return kvm_vcpu_ioctl(cpu, KVM_ENABLE_CAP, &cap);
+}
+
+
+static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val,
                                   bool assign, uint32_t size, bool datamatch)
 {
     int ret;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 3/6] kvm: add kvm_enable_cap_{vm,vcpu}
@ 2014-02-25 17:25   ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Provide helper functions for enabling capabilities (on a vcpu and on a vm).

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 include/sysemu/kvm.h |    4 ++++
 kvm-all.c            |   19 ++++++++++++++++++-
 2 files changed, 22 insertions(+), 1 deletion(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 3b25f27..9d2df88 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -272,6 +272,10 @@ bool kvm_arch_stop_on_emulation_error(CPUState *cpu);
 
 int kvm_check_extension(KVMState *s, unsigned int extension);
 
+int kvm_enable_cap_vm(KVMState *s, unsigned int capability);
+
+int kvm_enable_cap_vcpu(CPUState *cpu, unsigned int capability);
+
 uint32_t kvm_arch_get_supported_cpuid(KVMState *env, uint32_t function,
                                       uint32_t index, int reg);
 
diff --git a/kvm-all.c b/kvm-all.c
index 2ca9143..6e6a804 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -499,7 +499,24 @@ int kvm_check_extension(KVMState *s, unsigned int extension)
     return ret;
 }
 
-static int kvm_set_ioeventfd_mmio(int fd, hwaddr addr, uint32_t val,
+int kvm_enable_cap_vm(KVMState *s, unsigned int capability)
+{
+    struct kvm_enable_cap cap = {};
+
+    cap.cap = capability;
+    return kvm_vm_ioctl(s, KVM_ENABLE_CAP, &cap);
+}
+
+int kvm_enable_cap_vcpu(CPUState *cpu, unsigned int capability)
+{
+    struct kvm_enable_cap cap = {};
+
+    cap.cap = capability;
+    return kvm_vcpu_ioctl(cpu, KVM_ENABLE_CAP, &cap);
+}
+
+
+static int kvm_set_ioeventfd_mmio(int fd, uint32_t addr, uint32_t val,
                                   bool assign, uint32_t size, bool datamatch)
 {
     int ret;
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH 4/6] s390x: Add I/O adapter registration.
  2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:25   ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Register an I/O adapter interrupt source for when virtio-ccw devices start
using adapter interrupts.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/intc/s390_flic.c   |   66 +++++++++++++++++++++++++++++++++++++++++++++++++
 hw/s390x/css.c        |   51 ++++++++++++++++++++++++++++++++++++++
 hw/s390x/css.h        |    4 +++
 hw/s390x/virtio-ccw.c |    4 +++
 hw/s390x/virtio-ccw.h |    1 +
 target-s390x/cpu.h    |   33 +++++++++++++++++++++++++
 6 files changed, 159 insertions(+)

diff --git a/hw/intc/s390_flic.c b/hw/intc/s390_flic.c
index b2ef3e3..d1117fc 100644
--- a/hw/intc/s390_flic.c
+++ b/hw/intc/s390_flic.c
@@ -21,6 +21,18 @@
 #define FLIC_FAILED (-1UL)
 #define FLIC_SAVEVM_VERSION 1
 
+static KVMS390FLICState *s390_get_flic(void)
+{
+    ObjectProperty *op = object_property_find(qdev_get_machine(),
+                                              "s390-flic", NULL);
+
+    if (op) {
+        return op->opaque;
+    } else {
+        return NULL;
+    }
+}
+
 void s390_flic_init(void)
 {
     DeviceState *dev;
@@ -148,6 +160,60 @@ static int __get_all_irqs(KVMS390FLICState *flic,
     return r;
 }
 
+int kvm_s390_register_io_adapter(uint32_t id, uint8_t isc, bool swap,
+                                 bool is_maskable)
+{
+    struct kvm_s390_io_adapter adapter = {
+        .id = id,
+        .isc = isc,
+        .maskable = is_maskable,
+        .swap = swap,
+    };
+    KVMS390FLICState *flic = s390_get_flic();
+    int r, ret;
+    struct kvm_device_attr attr;
+
+    if (!flic) {
+        return -ENOSYS;
+    }
+    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
+        return -ENOSYS;
+    }
+
+    attr.group = KVM_DEV_FLIC_ADAPTER_REGISTER;
+    attr.addr = (uint64_t)&adapter;
+
+    r = ioctl(flic->fd, KVM_SET_DEVICE_ATTR, &attr);
+
+    ret = r ? -errno : 0;
+    return ret;
+}
+
+int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map)
+{
+    struct kvm_s390_io_adapter_req req = {
+        .id = id,
+        .type = do_map ? KVM_S390_IO_ADAPTER_MAP : KVM_S390_IO_ADAPTER_UNMAP,
+        .addr = map_addr,
+    };
+    KVMS390FLICState *flic = s390_get_flic();
+    struct kvm_device_attr attr;
+    int r;
+
+    if (!flic) {
+        return -ENOSYS;
+    }
+    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
+        return -ENOSYS;
+    }
+
+    attr.group = KVM_DEV_FLIC_ADAPTER_MODIFY;
+    attr.addr = (uint64_t)&req;
+
+    r = ioctl(flic->fd, KVM_SET_DEVICE_ATTR, &attr);
+    return r ? -errno : 0;
+}
+
 /**
  * kvm_flic_save - Save pending floating interrupts
  * @f: QEMUFile containing migration state
diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 7074d2b..a6d173f 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -39,6 +39,13 @@ typedef struct CssImage {
     ChpInfo chpids[MAX_CHPID + 1];
 } CssImage;
 
+typedef struct IoAdapter {
+    uint32_t id;
+    uint8_t type;
+    uint8_t isc;
+    QTAILQ_ENTRY(IoAdapter) sibling;
+} IoAdapter;
+
 typedef struct ChannelSubSys {
     QTAILQ_HEAD(, CrwContainer) pending_crws;
     bool do_crw_mchk;
@@ -49,6 +56,7 @@ typedef struct ChannelSubSys {
     uint64_t chnmon_area;
     CssImage *css[MAX_CSSID + 1];
     uint8_t default_cssid;
+    QTAILQ_HEAD(, IoAdapter) io_adapters;
 } ChannelSubSys;
 
 static ChannelSubSys *channel_subsys;
@@ -69,6 +77,48 @@ int css_create_css_image(uint8_t cssid, bool default_image)
     return 0;
 }
 
+int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
+                            bool maskable, uint32_t *id)
+{
+    IoAdapter *adapter;
+    bool found = false;
+    int ret;
+
+    *id = 0;
+    QTAILQ_FOREACH(adapter, &channel_subsys->io_adapters, sibling) {
+        if ((adapter->type == type) && (adapter->isc == isc)) {
+            *id = adapter->id;
+            found = true;
+            ret = 0;
+            break;
+        }
+        if (adapter->id >= *id) {
+            *id = adapter->id + 1;
+        }
+    }
+    if (found) {
+        goto out;
+    }
+    adapter = g_new0(IoAdapter, 1);
+    ret = s390_register_io_adapter(*id, isc, swap, maskable);
+    if (ret == -ENOSYS) {
+        /* Keep adapter even if we didn't register it anywhere. */
+        ret = 0;
+    }
+    if (ret == 0) {
+        adapter->id = *id;
+        adapter->isc = isc;
+        adapter->type = type;
+        QTAILQ_INSERT_TAIL(&channel_subsys->io_adapters, adapter, sibling);
+    } else {
+        g_free(adapter);
+        fprintf(stderr, "Unexpected error %d when registering adapter %d\n",
+                ret, *id);
+    }
+out:
+    return ret;
+}
+
 uint16_t css_build_subchannel_id(SubchDev *sch)
 {
     if (channel_subsys->max_cssid > 0) {
@@ -1239,6 +1289,7 @@ static void css_init(void)
     channel_subsys->do_crw_mchk = true;
     channel_subsys->crws_lost = false;
     channel_subsys->chnmon_active = false;
+    QTAILQ_INIT(&channel_subsys->io_adapters);
 }
 machine_init(css_init);
 
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index e9b4405..380e8e7 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -99,4 +99,8 @@ void css_generate_sch_crws(uint8_t cssid, uint8_t ssid, uint16_t schid,
                            int hotplugged, int add);
 void css_generate_chp_crws(uint8_t cssid, uint8_t chpid);
 void css_adapter_interrupt(uint8_t isc);
+
+#define CSS_IO_ADAPTER_VIRTIO 1
+int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
+                            bool maskable, uint32_t *id);
 #endif
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index a01801e..4125100 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -522,6 +522,10 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                 dev->thinint_isc = thinint->isc;
                 dev->ind_bit = thinint->ind_bit;
                 cpu_physical_memory_unmap(thinint, hw_len, 0, hw_len);
+                ret = css_register_io_adapter(CSS_IO_ADAPTER_VIRTIO,
+                                              dev->thinint_isc, true, false,
+                                              &dev->adapter_id);
+                assert(ret == 0);
                 sch->thinint_active = ((dev->indicators != 0) &&
                                        (dev->summary_indicator != 0));
                 sch->curr_status.scsw.count = ccw.count - len;
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 4393e44..0b70b91 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -85,6 +85,7 @@ struct VirtioCcwDevice {
     bool ioeventfd_disabled;
     uint32_t flags;
     uint8_t thinint_isc;
+    uint32_t adapter_id;
     /* Guest provided values: */
     hwaddr indicators;
     hwaddr indicators2;
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 9673838..7f21d09 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -1071,6 +1071,9 @@ void kvm_s390_enable_css_support(S390CPU *cpu);
 int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch,
                                     int vq, bool assign);
 int kvm_s390_cpu_restart(S390CPU *cpu);
+int kvm_s390_register_io_adapter(uint32_t id, uint8_t isc, bool swap,
+                                 bool maskable);
+int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map);
 #else
 static inline void kvm_s390_io_interrupt(S390CPU *cpu,
                                         uint16_t subchannel_id,
@@ -1095,6 +1098,16 @@ static inline int kvm_s390_cpu_restart(S390CPU *cpu)
 {
     return -ENOSYS;
 }
+static inline int kvm_s390_register_io_adapter(uint32_t id, uint8_t isc,
+                                               bool swap, bool maskable)
+{
+    return -ENOSYS;
+}
+static inline int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr,
+                                          bool do_map)
+{
+    return -ENOSYS;
+}
 #endif
 
 static inline int s390_cpu_restart(S390CPU *cpu)
@@ -1140,4 +1153,24 @@ static inline int s390_assign_subch_ioeventfd(EventNotifier *notifier,
     }
 }
 
+static inline int s390_register_io_adapter(uint32_t id, uint8_t isc, bool swap,
+                                           bool is_maskable)
+{
+    if (kvm_enabled()) {
+        return kvm_s390_register_io_adapter(id, isc, swap, is_maskable);
+    } else {
+        return -ENOSYS;
+    }
+}
+
+static inline int s390_io_adapter_map(uint32_t id, uint64_t map_addr,
+                                      bool do_map)
+{
+    if (kvm_enabled()) {
+        return kvm_s390_io_adapter_map(id, map_addr, do_map);
+    } else {
+        return -ENOSYS;
+    }
+}
+
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 4/6] s390x: Add I/O adapter registration.
@ 2014-02-25 17:25   ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Register an I/O adapter interrupt source for when virtio-ccw devices start
using adapter interrupts.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/intc/s390_flic.c   |   66 +++++++++++++++++++++++++++++++++++++++++++++++++
 hw/s390x/css.c        |   51 ++++++++++++++++++++++++++++++++++++++
 hw/s390x/css.h        |    4 +++
 hw/s390x/virtio-ccw.c |    4 +++
 hw/s390x/virtio-ccw.h |    1 +
 target-s390x/cpu.h    |   33 +++++++++++++++++++++++++
 6 files changed, 159 insertions(+)

diff --git a/hw/intc/s390_flic.c b/hw/intc/s390_flic.c
index b2ef3e3..d1117fc 100644
--- a/hw/intc/s390_flic.c
+++ b/hw/intc/s390_flic.c
@@ -21,6 +21,18 @@
 #define FLIC_FAILED (-1UL)
 #define FLIC_SAVEVM_VERSION 1
 
+static KVMS390FLICState *s390_get_flic(void)
+{
+    ObjectProperty *op = object_property_find(qdev_get_machine(),
+                                              "s390-flic", NULL);
+
+    if (op) {
+        return op->opaque;
+    } else {
+        return NULL;
+    }
+}
+
 void s390_flic_init(void)
 {
     DeviceState *dev;
@@ -148,6 +160,60 @@ static int __get_all_irqs(KVMS390FLICState *flic,
     return r;
 }
 
+int kvm_s390_register_io_adapter(uint32_t id, uint8_t isc, bool swap,
+                                 bool is_maskable)
+{
+    struct kvm_s390_io_adapter adapter = {
+        .id = id,
+        .isc = isc,
+        .maskable = is_maskable,
+        .swap = swap,
+    };
+    KVMS390FLICState *flic = s390_get_flic();
+    int r, ret;
+    struct kvm_device_attr attr;
+
+    if (!flic) {
+        return -ENOSYS;
+    }
+    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
+        return -ENOSYS;
+    }
+
+    attr.group = KVM_DEV_FLIC_ADAPTER_REGISTER;
+    attr.addr = (uint64_t)&adapter;
+
+    r = ioctl(flic->fd, KVM_SET_DEVICE_ATTR, &attr);
+
+    ret = r ? -errno : 0;
+    return ret;
+}
+
+int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map)
+{
+    struct kvm_s390_io_adapter_req req = {
+        .id = id,
+        .type = do_map ? KVM_S390_IO_ADAPTER_MAP : KVM_S390_IO_ADAPTER_UNMAP,
+        .addr = map_addr,
+    };
+    KVMS390FLICState *flic = s390_get_flic();
+    struct kvm_device_attr attr;
+    int r;
+
+    if (!flic) {
+        return -ENOSYS;
+    }
+    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
+        return -ENOSYS;
+    }
+
+    attr.group = KVM_DEV_FLIC_ADAPTER_MODIFY;
+    attr.addr = (uint64_t)&req;
+
+    r = ioctl(flic->fd, KVM_SET_DEVICE_ATTR, &attr);
+    return r ? -errno : 0;
+}
+
 /**
  * kvm_flic_save - Save pending floating interrupts
  * @f: QEMUFile containing migration state
diff --git a/hw/s390x/css.c b/hw/s390x/css.c
index 7074d2b..a6d173f 100644
--- a/hw/s390x/css.c
+++ b/hw/s390x/css.c
@@ -39,6 +39,13 @@ typedef struct CssImage {
     ChpInfo chpids[MAX_CHPID + 1];
 } CssImage;
 
+typedef struct IoAdapter {
+    uint32_t id;
+    uint8_t type;
+    uint8_t isc;
+    QTAILQ_ENTRY(IoAdapter) sibling;
+} IoAdapter;
+
 typedef struct ChannelSubSys {
     QTAILQ_HEAD(, CrwContainer) pending_crws;
     bool do_crw_mchk;
@@ -49,6 +56,7 @@ typedef struct ChannelSubSys {
     uint64_t chnmon_area;
     CssImage *css[MAX_CSSID + 1];
     uint8_t default_cssid;
+    QTAILQ_HEAD(, IoAdapter) io_adapters;
 } ChannelSubSys;
 
 static ChannelSubSys *channel_subsys;
@@ -69,6 +77,48 @@ int css_create_css_image(uint8_t cssid, bool default_image)
     return 0;
 }
 
+int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
+                            bool maskable, uint32_t *id)
+{
+    IoAdapter *adapter;
+    bool found = false;
+    int ret;
+
+    *id = 0;
+    QTAILQ_FOREACH(adapter, &channel_subsys->io_adapters, sibling) {
+        if ((adapter->type == type) && (adapter->isc == isc)) {
+            *id = adapter->id;
+            found = true;
+            ret = 0;
+            break;
+        }
+        if (adapter->id >= *id) {
+            *id = adapter->id + 1;
+        }
+    }
+    if (found) {
+        goto out;
+    }
+    adapter = g_new0(IoAdapter, 1);
+    ret = s390_register_io_adapter(*id, isc, swap, maskable);
+    if (ret == -ENOSYS) {
+        /* Keep adapter even if we didn't register it anywhere. */
+        ret = 0;
+    }
+    if (ret == 0) {
+        adapter->id = *id;
+        adapter->isc = isc;
+        adapter->type = type;
+        QTAILQ_INSERT_TAIL(&channel_subsys->io_adapters, adapter, sibling);
+    } else {
+        g_free(adapter);
+        fprintf(stderr, "Unexpected error %d when registering adapter %d\n",
+                ret, *id);
+    }
+out:
+    return ret;
+}
+
 uint16_t css_build_subchannel_id(SubchDev *sch)
 {
     if (channel_subsys->max_cssid > 0) {
@@ -1239,6 +1289,7 @@ static void css_init(void)
     channel_subsys->do_crw_mchk = true;
     channel_subsys->crws_lost = false;
     channel_subsys->chnmon_active = false;
+    QTAILQ_INIT(&channel_subsys->io_adapters);
 }
 machine_init(css_init);
 
diff --git a/hw/s390x/css.h b/hw/s390x/css.h
index e9b4405..380e8e7 100644
--- a/hw/s390x/css.h
+++ b/hw/s390x/css.h
@@ -99,4 +99,8 @@ void css_generate_sch_crws(uint8_t cssid, uint8_t ssid, uint16_t schid,
                            int hotplugged, int add);
 void css_generate_chp_crws(uint8_t cssid, uint8_t chpid);
 void css_adapter_interrupt(uint8_t isc);
+
+#define CSS_IO_ADAPTER_VIRTIO 1
+int css_register_io_adapter(uint8_t type, uint8_t isc, bool swap,
+                            bool maskable, uint32_t *id);
 #endif
diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index a01801e..4125100 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -522,6 +522,10 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                 dev->thinint_isc = thinint->isc;
                 dev->ind_bit = thinint->ind_bit;
                 cpu_physical_memory_unmap(thinint, hw_len, 0, hw_len);
+                ret = css_register_io_adapter(CSS_IO_ADAPTER_VIRTIO,
+                                              dev->thinint_isc, true, false,
+                                              &dev->adapter_id);
+                assert(ret == 0);
                 sch->thinint_active = ((dev->indicators != 0) &&
                                        (dev->summary_indicator != 0));
                 sch->curr_status.scsw.count = ccw.count - len;
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 4393e44..0b70b91 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -85,6 +85,7 @@ struct VirtioCcwDevice {
     bool ioeventfd_disabled;
     uint32_t flags;
     uint8_t thinint_isc;
+    uint32_t adapter_id;
     /* Guest provided values: */
     hwaddr indicators;
     hwaddr indicators2;
diff --git a/target-s390x/cpu.h b/target-s390x/cpu.h
index 9673838..7f21d09 100644
--- a/target-s390x/cpu.h
+++ b/target-s390x/cpu.h
@@ -1071,6 +1071,9 @@ void kvm_s390_enable_css_support(S390CPU *cpu);
 int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch,
                                     int vq, bool assign);
 int kvm_s390_cpu_restart(S390CPU *cpu);
+int kvm_s390_register_io_adapter(uint32_t id, uint8_t isc, bool swap,
+                                 bool maskable);
+int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map);
 #else
 static inline void kvm_s390_io_interrupt(S390CPU *cpu,
                                         uint16_t subchannel_id,
@@ -1095,6 +1098,16 @@ static inline int kvm_s390_cpu_restart(S390CPU *cpu)
 {
     return -ENOSYS;
 }
+static inline int kvm_s390_register_io_adapter(uint32_t id, uint8_t isc,
+                                               bool swap, bool maskable)
+{
+    return -ENOSYS;
+}
+static inline int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr,
+                                          bool do_map)
+{
+    return -ENOSYS;
+}
 #endif
 
 static inline int s390_cpu_restart(S390CPU *cpu)
@@ -1140,4 +1153,24 @@ static inline int s390_assign_subch_ioeventfd(EventNotifier *notifier,
     }
 }
 
+static inline int s390_register_io_adapter(uint32_t id, uint8_t isc, bool swap,
+                                           bool is_maskable)
+{
+    if (kvm_enabled()) {
+        return kvm_s390_register_io_adapter(id, isc, swap, is_maskable);
+    } else {
+        return -ENOSYS;
+    }
+}
+
+static inline int s390_io_adapter_map(uint32_t id, uint64_t map_addr,
+                                      bool do_map)
+{
+    if (kvm_enabled()) {
+        return kvm_s390_io_adapter_map(id, map_addr, do_map);
+    } else {
+        return -ENOSYS;
+    }
+}
+
 #endif
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH 5/6] s390x/virtio-ccw: reference-counted indicators
  2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:25   ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Make code using the same indicators point to a single allocated structure
that is freed when the last user goes away.

This will be used by the irqfd code to unmap addresses after the last user
is gone.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/virtio-ccw.c |   80 ++++++++++++++++++++++++++++++++++++++-----------
 hw/s390x/virtio-ccw.h |   13 ++++++--
 2 files changed, 73 insertions(+), 20 deletions(-)

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index 4125100..c218acd 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -27,6 +27,38 @@
 #include "virtio-ccw.h"
 #include "trace.h"
 
+static QTAILQ_HEAD(, IndAddr) indicator_addresses =
+    QTAILQ_HEAD_INITIALIZER(indicator_addresses);
+
+static IndAddr *get_indicator(hwaddr ind_addr, int len)
+{
+    IndAddr *indicator;
+
+    QTAILQ_FOREACH(indicator, &indicator_addresses, sibling) {
+        if (indicator->addr == ind_addr) {
+            indicator->refcnt++;
+            return indicator;
+        }
+    }
+    indicator = g_new0(IndAddr, 1);
+    indicator->addr = ind_addr;
+    indicator->len = len;
+    indicator->refcnt = 1;
+    QTAILQ_INSERT_TAIL(&indicator_addresses, indicator, sibling);
+    return indicator;
+}
+
+static void release_indicator(IndAddr *indicator)
+{
+    assert(indicator->refcnt > 0);
+    indicator->refcnt--;
+    if (indicator->refcnt > 0) {
+        return;
+    }
+    QTAILQ_REMOVE(&indicator_addresses, indicator, sibling);
+    g_free(indicator);
+}
+
 static void virtio_ccw_bus_new(VirtioBusState *bus, size_t bus_size,
                                VirtioCcwDevice *dev);
 
@@ -445,7 +477,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = -EFAULT;
         } else {
             indicators = ldq_phys(&address_space_memory, ccw.cda);
-            dev->indicators = indicators;
+            dev->indicators = get_indicator(indicators, sizeof(uint64_t));
             sch->curr_status.scsw.count = ccw.count - sizeof(indicators);
             ret = 0;
         }
@@ -465,7 +497,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = -EFAULT;
         } else {
             indicators = ldq_phys(&address_space_memory, ccw.cda);
-            dev->indicators2 = indicators;
+            dev->indicators2 = get_indicator(indicators, sizeof(uint64_t));
             sch->curr_status.scsw.count = ccw.count - sizeof(indicators);
             ret = 0;
         }
@@ -517,8 +549,10 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                 ret = -EFAULT;
             } else {
                 len = hw_len;
-                dev->summary_indicator = thinint->summary_indicator;
-                dev->indicators = thinint->device_indicator;
+                dev->summary_indicator =
+                    get_indicator(thinint->summary_indicator, sizeof(uint8_t));
+                dev->indicators = get_indicator(thinint->device_indicator,
+                                                thinint->ind_bit / 8 + 1);
                 dev->thinint_isc = thinint->isc;
                 dev->ind_bit = thinint->ind_bit;
                 cpu_physical_memory_unmap(thinint, hw_len, 0, hw_len);
@@ -526,8 +560,8 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                                               dev->thinint_isc, true, false,
                                               &dev->adapter_id);
                 assert(ret == 0);
-                sch->thinint_active = ((dev->indicators != 0) &&
-                                       (dev->summary_indicator != 0));
+                sch->thinint_active = ((dev->indicators != NULL) &&
+                                       (dev->summary_indicator != NULL));
                 sch->curr_status.scsw.count = ccw.count - len;
                 ret = 0;
             }
@@ -558,7 +592,7 @@ static int virtio_ccw_device_init(VirtioCcwDevice *dev, VirtIODevice *vdev)
     sch->driver_data = dev;
     dev->sch = sch;
 
-    dev->indicators = 0;
+    dev->indicators = NULL;
 
     /* Initialize subchannel structure. */
     sch->channel_prog = 0x0;
@@ -698,7 +732,10 @@ static int virtio_ccw_exit(VirtioCcwDevice *dev)
         css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
         g_free(sch);
     }
-    dev->indicators = 0;
+    if (dev->indicators) {
+        release_indicator(dev->indicators);
+        dev->indicators = NULL;
+    }
     return 0;
 }
 
@@ -955,17 +992,17 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
              * ind_bit indicates the start of the indicators in a big
              * endian notation.
              */
-            virtio_set_ind_atomic(sch, dev->indicators +
+            virtio_set_ind_atomic(sch, dev->indicators->addr +
                                   (dev->ind_bit + vector) / 8,
                                   0x80 >> ((dev->ind_bit + vector) % 8));
-            if (!virtio_set_ind_atomic(sch, dev->summary_indicator,
+            if (!virtio_set_ind_atomic(sch, dev->summary_indicator->addr,
                                        0x01)) {
                 css_adapter_interrupt(dev->thinint_isc);
             }
         } else {
-            indicators = ldq_phys(&address_space_memory, dev->indicators);
+            indicators = ldq_phys(&address_space_memory, dev->indicators->addr);
             indicators |= 1ULL << vector;
-            stq_phys(&address_space_memory, dev->indicators, indicators);
+            stq_phys(&address_space_memory, dev->indicators->addr, indicators);
             css_conditional_io_interrupt(sch);
         }
     } else {
@@ -973,9 +1010,9 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
             return;
         }
         vector = 0;
-        indicators = ldq_phys(&address_space_memory, dev->indicators2);
+        indicators = ldq_phys(&address_space_memory, dev->indicators2->addr);
         indicators |= 1ULL << vector;
-        stq_phys(&address_space_memory, dev->indicators2, indicators);
+        stq_phys(&address_space_memory, dev->indicators2->addr, indicators);
         css_conditional_io_interrupt(sch);
     }
 }
@@ -996,9 +1033,18 @@ static void virtio_ccw_reset(DeviceState *d)
     virtio_ccw_stop_ioeventfd(dev);
     virtio_reset(vdev);
     css_reset_sch(dev->sch);
-    dev->indicators = 0;
-    dev->indicators2 = 0;
-    dev->summary_indicator = 0;
+    if (dev->indicators) {
+        release_indicator(dev->indicators);
+        dev->indicators = NULL;
+    }
+    if (dev->indicators2) {
+        release_indicator(dev->indicators2);
+        dev->indicators2 = NULL;
+    }
+    if (dev->summary_indicator) {
+        release_indicator(dev->summary_indicator);
+        dev->summary_indicator = NULL;
+    }
 }
 
 static void virtio_ccw_vmstate_change(DeviceState *d, bool running)
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 0b70b91..d340bf4 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -75,6 +75,13 @@ typedef struct VirtIOCCWDeviceClass {
 #define VIRTIO_CCW_FLAG_USE_IOEVENTFD_BIT 1
 #define VIRTIO_CCW_FLAG_USE_IOEVENTFD   (1 << VIRTIO_CCW_FLAG_USE_IOEVENTFD_BIT)
 
+typedef struct IndAddr {
+    hwaddr addr;
+    unsigned long refcnt;
+    int len;
+    QTAILQ_ENTRY(IndAddr) sibling;
+} IndAddr;
+
 struct VirtioCcwDevice {
     DeviceState parent_obj;
     SubchDev *sch;
@@ -87,9 +94,9 @@ struct VirtioCcwDevice {
     uint8_t thinint_isc;
     uint32_t adapter_id;
     /* Guest provided values: */
-    hwaddr indicators;
-    hwaddr indicators2;
-    hwaddr summary_indicator;
+    IndAddr *indicators;
+    IndAddr *indicators2;
+    IndAddr *summary_indicator;
     uint64_t ind_bit;
 };
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 5/6] s390x/virtio-ccw: reference-counted indicators
@ 2014-02-25 17:25   ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Make code using the same indicators point to a single allocated structure
that is freed when the last user goes away.

This will be used by the irqfd code to unmap addresses after the last user
is gone.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/virtio-ccw.c |   80 ++++++++++++++++++++++++++++++++++++++-----------
 hw/s390x/virtio-ccw.h |   13 ++++++--
 2 files changed, 73 insertions(+), 20 deletions(-)

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index 4125100..c218acd 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -27,6 +27,38 @@
 #include "virtio-ccw.h"
 #include "trace.h"
 
+static QTAILQ_HEAD(, IndAddr) indicator_addresses =
+    QTAILQ_HEAD_INITIALIZER(indicator_addresses);
+
+static IndAddr *get_indicator(hwaddr ind_addr, int len)
+{
+    IndAddr *indicator;
+
+    QTAILQ_FOREACH(indicator, &indicator_addresses, sibling) {
+        if (indicator->addr == ind_addr) {
+            indicator->refcnt++;
+            return indicator;
+        }
+    }
+    indicator = g_new0(IndAddr, 1);
+    indicator->addr = ind_addr;
+    indicator->len = len;
+    indicator->refcnt = 1;
+    QTAILQ_INSERT_TAIL(&indicator_addresses, indicator, sibling);
+    return indicator;
+}
+
+static void release_indicator(IndAddr *indicator)
+{
+    assert(indicator->refcnt > 0);
+    indicator->refcnt--;
+    if (indicator->refcnt > 0) {
+        return;
+    }
+    QTAILQ_REMOVE(&indicator_addresses, indicator, sibling);
+    g_free(indicator);
+}
+
 static void virtio_ccw_bus_new(VirtioBusState *bus, size_t bus_size,
                                VirtioCcwDevice *dev);
 
@@ -445,7 +477,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = -EFAULT;
         } else {
             indicators = ldq_phys(&address_space_memory, ccw.cda);
-            dev->indicators = indicators;
+            dev->indicators = get_indicator(indicators, sizeof(uint64_t));
             sch->curr_status.scsw.count = ccw.count - sizeof(indicators);
             ret = 0;
         }
@@ -465,7 +497,7 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
             ret = -EFAULT;
         } else {
             indicators = ldq_phys(&address_space_memory, ccw.cda);
-            dev->indicators2 = indicators;
+            dev->indicators2 = get_indicator(indicators, sizeof(uint64_t));
             sch->curr_status.scsw.count = ccw.count - sizeof(indicators);
             ret = 0;
         }
@@ -517,8 +549,10 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                 ret = -EFAULT;
             } else {
                 len = hw_len;
-                dev->summary_indicator = thinint->summary_indicator;
-                dev->indicators = thinint->device_indicator;
+                dev->summary_indicator =
+                    get_indicator(thinint->summary_indicator, sizeof(uint8_t));
+                dev->indicators = get_indicator(thinint->device_indicator,
+                                                thinint->ind_bit / 8 + 1);
                 dev->thinint_isc = thinint->isc;
                 dev->ind_bit = thinint->ind_bit;
                 cpu_physical_memory_unmap(thinint, hw_len, 0, hw_len);
@@ -526,8 +560,8 @@ static int virtio_ccw_cb(SubchDev *sch, CCW1 ccw)
                                               dev->thinint_isc, true, false,
                                               &dev->adapter_id);
                 assert(ret == 0);
-                sch->thinint_active = ((dev->indicators != 0) &&
-                                       (dev->summary_indicator != 0));
+                sch->thinint_active = ((dev->indicators != NULL) &&
+                                       (dev->summary_indicator != NULL));
                 sch->curr_status.scsw.count = ccw.count - len;
                 ret = 0;
             }
@@ -558,7 +592,7 @@ static int virtio_ccw_device_init(VirtioCcwDevice *dev, VirtIODevice *vdev)
     sch->driver_data = dev;
     dev->sch = sch;
 
-    dev->indicators = 0;
+    dev->indicators = NULL;
 
     /* Initialize subchannel structure. */
     sch->channel_prog = 0x0;
@@ -698,7 +732,10 @@ static int virtio_ccw_exit(VirtioCcwDevice *dev)
         css_subch_assign(sch->cssid, sch->ssid, sch->schid, sch->devno, NULL);
         g_free(sch);
     }
-    dev->indicators = 0;
+    if (dev->indicators) {
+        release_indicator(dev->indicators);
+        dev->indicators = NULL;
+    }
     return 0;
 }
 
@@ -955,17 +992,17 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
              * ind_bit indicates the start of the indicators in a big
              * endian notation.
              */
-            virtio_set_ind_atomic(sch, dev->indicators +
+            virtio_set_ind_atomic(sch, dev->indicators->addr +
                                   (dev->ind_bit + vector) / 8,
                                   0x80 >> ((dev->ind_bit + vector) % 8));
-            if (!virtio_set_ind_atomic(sch, dev->summary_indicator,
+            if (!virtio_set_ind_atomic(sch, dev->summary_indicator->addr,
                                        0x01)) {
                 css_adapter_interrupt(dev->thinint_isc);
             }
         } else {
-            indicators = ldq_phys(&address_space_memory, dev->indicators);
+            indicators = ldq_phys(&address_space_memory, dev->indicators->addr);
             indicators |= 1ULL << vector;
-            stq_phys(&address_space_memory, dev->indicators, indicators);
+            stq_phys(&address_space_memory, dev->indicators->addr, indicators);
             css_conditional_io_interrupt(sch);
         }
     } else {
@@ -973,9 +1010,9 @@ static void virtio_ccw_notify(DeviceState *d, uint16_t vector)
             return;
         }
         vector = 0;
-        indicators = ldq_phys(&address_space_memory, dev->indicators2);
+        indicators = ldq_phys(&address_space_memory, dev->indicators2->addr);
         indicators |= 1ULL << vector;
-        stq_phys(&address_space_memory, dev->indicators2, indicators);
+        stq_phys(&address_space_memory, dev->indicators2->addr, indicators);
         css_conditional_io_interrupt(sch);
     }
 }
@@ -996,9 +1033,18 @@ static void virtio_ccw_reset(DeviceState *d)
     virtio_ccw_stop_ioeventfd(dev);
     virtio_reset(vdev);
     css_reset_sch(dev->sch);
-    dev->indicators = 0;
-    dev->indicators2 = 0;
-    dev->summary_indicator = 0;
+    if (dev->indicators) {
+        release_indicator(dev->indicators);
+        dev->indicators = NULL;
+    }
+    if (dev->indicators2) {
+        release_indicator(dev->indicators2);
+        dev->indicators2 = NULL;
+    }
+    if (dev->summary_indicator) {
+        release_indicator(dev->summary_indicator);
+        dev->summary_indicator = NULL;
+    }
 }
 
 static void virtio_ccw_vmstate_change(DeviceState *d, bool running)
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index 0b70b91..d340bf4 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -75,6 +75,13 @@ typedef struct VirtIOCCWDeviceClass {
 #define VIRTIO_CCW_FLAG_USE_IOEVENTFD_BIT 1
 #define VIRTIO_CCW_FLAG_USE_IOEVENTFD   (1 << VIRTIO_CCW_FLAG_USE_IOEVENTFD_BIT)
 
+typedef struct IndAddr {
+    hwaddr addr;
+    unsigned long refcnt;
+    int len;
+    QTAILQ_ENTRY(IndAddr) sibling;
+} IndAddr;
+
 struct VirtioCcwDevice {
     DeviceState parent_obj;
     SubchDev *sch;
@@ -87,9 +94,9 @@ struct VirtioCcwDevice {
     uint8_t thinint_isc;
     uint32_t adapter_id;
     /* Guest provided values: */
-    hwaddr indicators;
-    hwaddr indicators2;
-    hwaddr summary_indicator;
+    IndAddr *indicators;
+    IndAddr *indicators2;
+    IndAddr *summary_indicator;
     uint64_t ind_bit;
 };
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [PATCH 6/6] s390x/virtio-ccw: Wire up irq routing and irqfds.
  2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:25   ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck

Make use of the new s390 adapter irq routing support to enable real
in-kernel irqfds for virtio-ccw with adapter interrupts.

Note that s390 doesn't provide the common KVM_CAP_IRQCHIP capability, but
rather needs KVM_CAP_S390_IRQCHIP to be enabled. This is to ensure backward
compatibility.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/virtio-ccw.c      |  167 ++++++++++++++++++++++++++++++++++++++++----
 hw/s390x/virtio-ccw.h      |    2 +
 include/hw/s390x/adapter.h |   23 ++++++
 include/qemu/typedefs.h    |    1 +
 include/sysemu/kvm.h       |    2 +
 kvm-all.c                  |   38 +++++++++-
 kvm-stub.c                 |    5 ++
 target-s390x/kvm.c         |    5 ++
 8 files changed, 230 insertions(+), 13 deletions(-)
 create mode 100644 include/hw/s390x/adapter.h

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index c218acd..29f9762 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -21,6 +21,7 @@
 #include "hw/sysbus.h"
 #include "qemu/bitops.h"
 #include "hw/virtio/virtio-bus.h"
+#include "hw/s390x/adapter.h"
 
 #include "ioinst.h"
 #include "css.h"
@@ -48,7 +49,7 @@ static IndAddr *get_indicator(hwaddr ind_addr, int len)
     return indicator;
 }
 
-static void release_indicator(IndAddr *indicator)
+static void release_indicator(uint32_t adapter_id, IndAddr *indicator)
 {
     assert(indicator->refcnt > 0);
     indicator->refcnt--;
@@ -56,9 +57,33 @@ static void release_indicator(IndAddr *indicator)
         return;
     }
     QTAILQ_REMOVE(&indicator_addresses, indicator, sibling);
+    if (indicator->map) {
+        s390_io_adapter_map(adapter_id,
+                            (uint64_t)(unsigned long)indicator->map, false);
+    }
     g_free(indicator);
 }
 
+static int map_indicator(uint32_t adapter_id, IndAddr *indicator)
+{
+    int ret;
+
+    if (indicator->map) {
+        return 0; /* already mapped is not an error */
+    }
+    indicator->map = (void *)indicator->addr;
+    ret = s390_io_adapter_map(adapter_id,
+                              (uint64_t)(unsigned long)indicator->map, true);
+    if ((ret != 0) && (ret != -ENOSYS)) {
+        goto out_err;
+    }
+    return 0;
+
+out_err:
+    indicator->map = 0;
+    return -EFAULT;
+}
+
 static void virtio_ccw_bus_new(VirtioBusState *bus, size_t bus_size,
                                VirtioCcwDevice *dev);
 
@@ -733,7 +758,7 @@ static int virtio_ccw_exit(VirtioCcwDevice *dev)
         g_free(sch);
     }
     if (dev->indicators) {
-        release_indicator(dev->indicators);
+        release_indicator(dev->adapter_id, dev->indicators);
         dev->indicators = NULL;
     }
     return 0;
@@ -1034,15 +1059,15 @@ static void virtio_ccw_reset(DeviceState *d)
     virtio_reset(vdev);
     css_reset_sch(dev->sch);
     if (dev->indicators) {
-        release_indicator(dev->indicators);
+        release_indicator(dev->adapter_id, dev->indicators);
         dev->indicators = NULL;
     }
     if (dev->indicators2) {
-        release_indicator(dev->indicators2);
+        release_indicator(dev->adapter_id, dev->indicators2);
         dev->indicators2 = NULL;
     }
     if (dev->summary_indicator) {
-        release_indicator(dev->summary_indicator);
+        release_indicator(dev->adapter_id, dev->summary_indicator);
         dev->summary_indicator = NULL;
     }
 }
@@ -1078,6 +1103,100 @@ static int virtio_ccw_set_host_notifier(DeviceState *d, int n, bool assign)
     return virtio_ccw_set_guest2host_notifier(dev, n, assign, false);
 }
 
+static int virtio_ccw_get_adapter_info(VirtioCcwDevice *dev,
+                                       AdapterInfo *adapter)
+{
+    int r;
+
+    if (!dev->sch->thinint_active) {
+        return -EINVAL;
+    }
+
+    r = map_indicator(dev->adapter_id, dev->summary_indicator);
+    if (r) {
+        return r;
+    }
+    r = map_indicator(dev->adapter_id, dev->indicators);
+    if (r) {
+        return r;
+    }
+    adapter->summary_addr = (unsigned long)dev->summary_indicator->map;
+    adapter->ind_addr = (unsigned long)dev->indicators->map;
+    adapter->ind_offset = dev->ind_bit;
+    adapter->summary_offset = 7;
+    adapter->adapter_id = dev->adapter_id;
+
+    return 0;
+}
+
+static int virtio_ccw_setup_irqroutes(VirtioCcwDevice *dev, int nvqs)
+{
+    int i;
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    int ret;
+    AdapterInfo adapter;
+
+    ret = virtio_ccw_get_adapter_info(dev, &adapter);
+    if (ret) {
+        return ret;
+    }
+    for (i = 0; i < nvqs; i++) {
+        if (!virtio_queue_get_num(vdev, i)) {
+            break;
+        }
+        ret = kvm_irqchip_add_adapter_route(kvm_state, &adapter);
+        if (ret < 0) {
+            goto out_undo;
+        }
+        dev->gsi[i] = ret;
+        adapter.ind_offset++;
+    }
+    return 0;
+out_undo:
+    while (--i >= 0) {
+        kvm_irqchip_release_virq(kvm_state, dev->gsi[i]);
+        dev->gsi[i] = -1;
+    }
+    return ret;
+}
+
+static void virtio_ccw_release_irqroutes(VirtioCcwDevice *dev, int nvqs)
+{
+    int i;
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+
+    for (i = 0; i < nvqs; i++) {
+        if (!virtio_queue_get_num(vdev, i)) {
+            break;
+        }
+        if (dev->gsi[i] >= 0) {
+            kvm_irqchip_release_virq(kvm_state, dev->gsi[i]);
+            dev->gsi[i] = -1;
+        }
+    }
+}
+
+static int virtio_ccw_add_irqfd(VirtioCcwDevice *dev, int n)
+{
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    VirtQueue *vq = virtio_get_queue(vdev, n);
+    EventNotifier *notifier = virtio_queue_get_guest_notifier(vq);
+
+    return kvm_irqchip_add_irqfd_notifier(kvm_state, notifier, NULL,
+                                          dev->gsi[n]);
+}
+
+static void virtio_ccw_remove_irqfd(VirtioCcwDevice *dev, int n)
+{
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    VirtQueue *vq = virtio_get_queue(vdev, n);
+    EventNotifier *notifier = virtio_queue_get_guest_notifier(vq);
+    int ret;
+
+    ret = kvm_irqchip_remove_irqfd_notifier(kvm_state, notifier, dev->gsi[n]);
+    assert(ret == 0);
+}
+
 static int virtio_ccw_set_guest_notifier(VirtioCcwDevice *dev, int n,
                                          bool assign, bool with_irqfd)
 {
@@ -1093,11 +1212,17 @@ static int virtio_ccw_set_guest_notifier(VirtioCcwDevice *dev, int n,
             return r;
         }
         virtio_queue_set_guest_notifier_fd_handler(vq, true, with_irqfd);
-        /* We do not support irqfd for classic I/O interrupts, because the
-         * classic interrupts are intermixed with the subchannel status, that
-         * is queried with test subchannel. We want to use vhost, though.
-         * Lets make sure to have vhost running and wire up the irq fd to
-         * land in qemu (and only the irq fd) in this code.
+        if (with_irqfd) {
+            r = virtio_ccw_add_irqfd(dev, n);
+            if (r) {
+                virtio_queue_set_guest_notifier_fd_handler(vq, false,
+                                                           with_irqfd);
+                return r;
+            }
+        }
+        /*
+         * We do not support individual masking for channel devices, so we
+         * need to manually trigger any guest masking callbacks here.
          */
         if (k->guest_notifier_mask) {
             k->guest_notifier_mask(vdev, n, false);
@@ -1111,6 +1236,9 @@ static int virtio_ccw_set_guest_notifier(VirtioCcwDevice *dev, int n,
         if (k->guest_notifier_mask) {
             k->guest_notifier_mask(vdev, n, true);
         }
+        if (with_irqfd) {
+            virtio_ccw_remove_irqfd(dev, n);
+        }
         virtio_queue_set_guest_notifier_fd_handler(vq, false, with_irqfd);
         event_notifier_cleanup(notifier);
     }
@@ -1122,24 +1250,39 @@ static int virtio_ccw_set_guest_notifiers(DeviceState *d, int nvqs,
 {
     VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
     VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    bool with_irqfd = dev->sch->thinint_active && kvm_irqfds_enabled();
     int r, n;
 
+    if (with_irqfd && assigned) {
+        /* irq routes need to be set up before assigning irqfds */
+        r = virtio_ccw_setup_irqroutes(dev, nvqs);
+        if (r < 0) {
+            goto irqroute_error;
+        }
+    }
     for (n = 0; n < nvqs; n++) {
         if (!virtio_queue_get_num(vdev, n)) {
             break;
         }
-        /* false -> true, as soon as irqfd works */
-        r = virtio_ccw_set_guest_notifier(dev, n, assigned, false);
+        r = virtio_ccw_set_guest_notifier(dev, n, assigned, with_irqfd);
         if (r < 0) {
             goto assign_error;
         }
     }
+    if (with_irqfd && !assigned) {
+        /* release irq routes after irqfds have been released */
+        virtio_ccw_release_irqroutes(dev, nvqs);
+    }
     return 0;
 
 assign_error:
     while (--n >= 0) {
         virtio_ccw_set_guest_notifier(dev, n, !assigned, false);
     }
+irqroute_error:
+    if (with_irqfd && assigned) {
+        virtio_ccw_release_irqroutes(dev, nvqs);
+    }
     return r;
 }
 
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index d340bf4..7977a3e 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -77,6 +77,7 @@ typedef struct VirtIOCCWDeviceClass {
 
 typedef struct IndAddr {
     hwaddr addr;
+    void *map;
     unsigned long refcnt;
     int len;
     QTAILQ_ENTRY(IndAddr) sibling;
@@ -93,6 +94,7 @@ struct VirtioCcwDevice {
     uint32_t flags;
     uint8_t thinint_isc;
     uint32_t adapter_id;
+    int gsi[VIRTIO_PCI_QUEUE_MAX];
     /* Guest provided values: */
     IndAddr *indicators;
     IndAddr *indicators2;
diff --git a/include/hw/s390x/adapter.h b/include/hw/s390x/adapter.h
new file mode 100644
index 0000000..7e56724
--- /dev/null
+++ b/include/hw/s390x/adapter.h
@@ -0,0 +1,23 @@
+/*
+ * s390 adapter definitions
+ *
+ * Copyright 2013 IBM Corp.
+ * Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef S390X_ADAPTER_H
+#define S390X_ADAPTER_H
+
+struct AdapterInfo {
+    uint64_t ind_addr;
+    uint64_t summary_addr;
+    uint64_t ind_offset;
+    uint32_t summary_offset;
+    uint32_t adapter_id;
+};
+
+#endif
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 83c9b16..76e6d43 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -72,5 +72,6 @@ typedef struct SHPCDevice SHPCDevice;
 typedef struct FWCfgState FWCfgState;
 typedef struct PcGuestInfo PcGuestInfo;
 typedef struct Range Range;
+typedef struct AdapterInfo AdapterInfo;
 
 #endif /* QEMU_TYPEDEFS_H */
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 9d2df88..7a0c18a 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -317,6 +317,8 @@ int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg);
 int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg);
 void kvm_irqchip_release_virq(KVMState *s, int virq);
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter);
+
 int kvm_irqchip_add_irqfd_notifier(KVMState *s, EventNotifier *n,
                                    EventNotifier *rn, int virq);
 int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq);
diff --git a/kvm-all.c b/kvm-all.c
index 6e6a804..9d6cd45 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -27,6 +27,7 @@
 #include "sysemu/sysemu.h"
 #include "hw/hw.h"
 #include "hw/pci/msi.h"
+#include "hw/s390x/adapter.h"
 #include "exec/gdbstub.h"
 #include "sysemu/kvm.h"
 #include "qemu/bswap.h"
@@ -1262,6 +1263,35 @@ static int kvm_irqchip_assign_irqfd(KVMState *s, int fd, int rfd, int virq,
     return kvm_vm_ioctl(s, KVM_IRQFD, &irqfd);
 }
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
+{
+    struct kvm_irq_routing_entry kroute;
+    int virq;
+
+    if (!kvm_gsi_routing_enabled()) {
+        return -ENOSYS;
+    }
+
+    virq = kvm_irqchip_get_virq(s);
+    if (virq < 0) {
+        return virq;
+    }
+
+    kroute.gsi = virq;
+    kroute.type = KVM_IRQ_ROUTING_S390_ADAPTER;
+    kroute.flags = 0;
+    kroute.u.adapter.summary_addr = adapter->summary_addr;
+    kroute.u.adapter.ind_addr = adapter->ind_addr;
+    kroute.u.adapter.summary_offset = adapter->summary_offset;
+    kroute.u.adapter.ind_offset = adapter->ind_offset;
+    kroute.u.adapter.adapter_id = adapter->adapter_id;
+
+    kvm_add_routing_entry(s, &kroute);
+    kvm_irqchip_commit_routes(s);
+
+    return virq;
+}
+
 #else /* !KVM_CAP_IRQ_ROUTING */
 
 void kvm_init_irq_routing(KVMState *s)
@@ -1282,6 +1312,11 @@ int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
     return -ENOSYS;
 }
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
+{
+    return -ENOSYS;
+}
+
 static int kvm_irqchip_assign_irqfd(KVMState *s, int fd, int virq, bool assign)
 {
     abort();
@@ -1311,7 +1346,8 @@ static int kvm_irqchip_create(KVMState *s)
     int ret;
 
     if (!qemu_opt_get_bool(qemu_get_machine_opts(), "kernel_irqchip", true) ||
-        !kvm_check_extension(s, KVM_CAP_IRQCHIP)) {
+        (!kvm_check_extension(s, KVM_CAP_IRQCHIP) &&
+         kvm_enable_cap_vm(s, KVM_CAP_S390_IRQCHIP))) {
         return 0;
     }
 
diff --git a/kvm-stub.c b/kvm-stub.c
index e979f76..c329ab5 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -136,6 +136,11 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg)
     return -ENOSYS;
 }
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
+{
+    return -ENOSYS;
+}
+
 int kvm_irqchip_add_irqfd_notifier(KVMState *s, EventNotifier *n,
                                    EventNotifier *rn, int virq)
 {
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 7e8321e..6b7e5db 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -898,6 +898,11 @@ void kvm_s390_enable_css_support(S390CPU *cpu)
 
 void kvm_arch_init_irq_routing(KVMState *s)
 {
+    if (kvm_check_extension(s, KVM_CAP_IRQ_ROUTING)) {
+        kvm_irqfds_allowed = true;
+        kvm_gsi_routing_allowed = true;
+        kvm_halt_in_kernel_allowed = false;
+    }
 }
 
 int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* [Qemu-devel] [PATCH 6/6] s390x/virtio-ccw: Wire up irq routing and irqfds.
@ 2014-02-25 17:25   ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:25 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel
  Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini

Make use of the new s390 adapter irq routing support to enable real
in-kernel irqfds for virtio-ccw with adapter interrupts.

Note that s390 doesn't provide the common KVM_CAP_IRQCHIP capability, but
rather needs KVM_CAP_S390_IRQCHIP to be enabled. This is to ensure backward
compatibility.

Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
 hw/s390x/virtio-ccw.c      |  167 ++++++++++++++++++++++++++++++++++++++++----
 hw/s390x/virtio-ccw.h      |    2 +
 include/hw/s390x/adapter.h |   23 ++++++
 include/qemu/typedefs.h    |    1 +
 include/sysemu/kvm.h       |    2 +
 kvm-all.c                  |   38 +++++++++-
 kvm-stub.c                 |    5 ++
 target-s390x/kvm.c         |    5 ++
 8 files changed, 230 insertions(+), 13 deletions(-)
 create mode 100644 include/hw/s390x/adapter.h

diff --git a/hw/s390x/virtio-ccw.c b/hw/s390x/virtio-ccw.c
index c218acd..29f9762 100644
--- a/hw/s390x/virtio-ccw.c
+++ b/hw/s390x/virtio-ccw.c
@@ -21,6 +21,7 @@
 #include "hw/sysbus.h"
 #include "qemu/bitops.h"
 #include "hw/virtio/virtio-bus.h"
+#include "hw/s390x/adapter.h"
 
 #include "ioinst.h"
 #include "css.h"
@@ -48,7 +49,7 @@ static IndAddr *get_indicator(hwaddr ind_addr, int len)
     return indicator;
 }
 
-static void release_indicator(IndAddr *indicator)
+static void release_indicator(uint32_t adapter_id, IndAddr *indicator)
 {
     assert(indicator->refcnt > 0);
     indicator->refcnt--;
@@ -56,9 +57,33 @@ static void release_indicator(IndAddr *indicator)
         return;
     }
     QTAILQ_REMOVE(&indicator_addresses, indicator, sibling);
+    if (indicator->map) {
+        s390_io_adapter_map(adapter_id,
+                            (uint64_t)(unsigned long)indicator->map, false);
+    }
     g_free(indicator);
 }
 
+static int map_indicator(uint32_t adapter_id, IndAddr *indicator)
+{
+    int ret;
+
+    if (indicator->map) {
+        return 0; /* already mapped is not an error */
+    }
+    indicator->map = (void *)indicator->addr;
+    ret = s390_io_adapter_map(adapter_id,
+                              (uint64_t)(unsigned long)indicator->map, true);
+    if ((ret != 0) && (ret != -ENOSYS)) {
+        goto out_err;
+    }
+    return 0;
+
+out_err:
+    indicator->map = 0;
+    return -EFAULT;
+}
+
 static void virtio_ccw_bus_new(VirtioBusState *bus, size_t bus_size,
                                VirtioCcwDevice *dev);
 
@@ -733,7 +758,7 @@ static int virtio_ccw_exit(VirtioCcwDevice *dev)
         g_free(sch);
     }
     if (dev->indicators) {
-        release_indicator(dev->indicators);
+        release_indicator(dev->adapter_id, dev->indicators);
         dev->indicators = NULL;
     }
     return 0;
@@ -1034,15 +1059,15 @@ static void virtio_ccw_reset(DeviceState *d)
     virtio_reset(vdev);
     css_reset_sch(dev->sch);
     if (dev->indicators) {
-        release_indicator(dev->indicators);
+        release_indicator(dev->adapter_id, dev->indicators);
         dev->indicators = NULL;
     }
     if (dev->indicators2) {
-        release_indicator(dev->indicators2);
+        release_indicator(dev->adapter_id, dev->indicators2);
         dev->indicators2 = NULL;
     }
     if (dev->summary_indicator) {
-        release_indicator(dev->summary_indicator);
+        release_indicator(dev->adapter_id, dev->summary_indicator);
         dev->summary_indicator = NULL;
     }
 }
@@ -1078,6 +1103,100 @@ static int virtio_ccw_set_host_notifier(DeviceState *d, int n, bool assign)
     return virtio_ccw_set_guest2host_notifier(dev, n, assign, false);
 }
 
+static int virtio_ccw_get_adapter_info(VirtioCcwDevice *dev,
+                                       AdapterInfo *adapter)
+{
+    int r;
+
+    if (!dev->sch->thinint_active) {
+        return -EINVAL;
+    }
+
+    r = map_indicator(dev->adapter_id, dev->summary_indicator);
+    if (r) {
+        return r;
+    }
+    r = map_indicator(dev->adapter_id, dev->indicators);
+    if (r) {
+        return r;
+    }
+    adapter->summary_addr = (unsigned long)dev->summary_indicator->map;
+    adapter->ind_addr = (unsigned long)dev->indicators->map;
+    adapter->ind_offset = dev->ind_bit;
+    adapter->summary_offset = 7;
+    adapter->adapter_id = dev->adapter_id;
+
+    return 0;
+}
+
+static int virtio_ccw_setup_irqroutes(VirtioCcwDevice *dev, int nvqs)
+{
+    int i;
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    int ret;
+    AdapterInfo adapter;
+
+    ret = virtio_ccw_get_adapter_info(dev, &adapter);
+    if (ret) {
+        return ret;
+    }
+    for (i = 0; i < nvqs; i++) {
+        if (!virtio_queue_get_num(vdev, i)) {
+            break;
+        }
+        ret = kvm_irqchip_add_adapter_route(kvm_state, &adapter);
+        if (ret < 0) {
+            goto out_undo;
+        }
+        dev->gsi[i] = ret;
+        adapter.ind_offset++;
+    }
+    return 0;
+out_undo:
+    while (--i >= 0) {
+        kvm_irqchip_release_virq(kvm_state, dev->gsi[i]);
+        dev->gsi[i] = -1;
+    }
+    return ret;
+}
+
+static void virtio_ccw_release_irqroutes(VirtioCcwDevice *dev, int nvqs)
+{
+    int i;
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+
+    for (i = 0; i < nvqs; i++) {
+        if (!virtio_queue_get_num(vdev, i)) {
+            break;
+        }
+        if (dev->gsi[i] >= 0) {
+            kvm_irqchip_release_virq(kvm_state, dev->gsi[i]);
+            dev->gsi[i] = -1;
+        }
+    }
+}
+
+static int virtio_ccw_add_irqfd(VirtioCcwDevice *dev, int n)
+{
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    VirtQueue *vq = virtio_get_queue(vdev, n);
+    EventNotifier *notifier = virtio_queue_get_guest_notifier(vq);
+
+    return kvm_irqchip_add_irqfd_notifier(kvm_state, notifier, NULL,
+                                          dev->gsi[n]);
+}
+
+static void virtio_ccw_remove_irqfd(VirtioCcwDevice *dev, int n)
+{
+    VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    VirtQueue *vq = virtio_get_queue(vdev, n);
+    EventNotifier *notifier = virtio_queue_get_guest_notifier(vq);
+    int ret;
+
+    ret = kvm_irqchip_remove_irqfd_notifier(kvm_state, notifier, dev->gsi[n]);
+    assert(ret == 0);
+}
+
 static int virtio_ccw_set_guest_notifier(VirtioCcwDevice *dev, int n,
                                          bool assign, bool with_irqfd)
 {
@@ -1093,11 +1212,17 @@ static int virtio_ccw_set_guest_notifier(VirtioCcwDevice *dev, int n,
             return r;
         }
         virtio_queue_set_guest_notifier_fd_handler(vq, true, with_irqfd);
-        /* We do not support irqfd for classic I/O interrupts, because the
-         * classic interrupts are intermixed with the subchannel status, that
-         * is queried with test subchannel. We want to use vhost, though.
-         * Lets make sure to have vhost running and wire up the irq fd to
-         * land in qemu (and only the irq fd) in this code.
+        if (with_irqfd) {
+            r = virtio_ccw_add_irqfd(dev, n);
+            if (r) {
+                virtio_queue_set_guest_notifier_fd_handler(vq, false,
+                                                           with_irqfd);
+                return r;
+            }
+        }
+        /*
+         * We do not support individual masking for channel devices, so we
+         * need to manually trigger any guest masking callbacks here.
          */
         if (k->guest_notifier_mask) {
             k->guest_notifier_mask(vdev, n, false);
@@ -1111,6 +1236,9 @@ static int virtio_ccw_set_guest_notifier(VirtioCcwDevice *dev, int n,
         if (k->guest_notifier_mask) {
             k->guest_notifier_mask(vdev, n, true);
         }
+        if (with_irqfd) {
+            virtio_ccw_remove_irqfd(dev, n);
+        }
         virtio_queue_set_guest_notifier_fd_handler(vq, false, with_irqfd);
         event_notifier_cleanup(notifier);
     }
@@ -1122,24 +1250,39 @@ static int virtio_ccw_set_guest_notifiers(DeviceState *d, int nvqs,
 {
     VirtioCcwDevice *dev = VIRTIO_CCW_DEVICE(d);
     VirtIODevice *vdev = virtio_bus_get_device(&dev->bus);
+    bool with_irqfd = dev->sch->thinint_active && kvm_irqfds_enabled();
     int r, n;
 
+    if (with_irqfd && assigned) {
+        /* irq routes need to be set up before assigning irqfds */
+        r = virtio_ccw_setup_irqroutes(dev, nvqs);
+        if (r < 0) {
+            goto irqroute_error;
+        }
+    }
     for (n = 0; n < nvqs; n++) {
         if (!virtio_queue_get_num(vdev, n)) {
             break;
         }
-        /* false -> true, as soon as irqfd works */
-        r = virtio_ccw_set_guest_notifier(dev, n, assigned, false);
+        r = virtio_ccw_set_guest_notifier(dev, n, assigned, with_irqfd);
         if (r < 0) {
             goto assign_error;
         }
     }
+    if (with_irqfd && !assigned) {
+        /* release irq routes after irqfds have been released */
+        virtio_ccw_release_irqroutes(dev, nvqs);
+    }
     return 0;
 
 assign_error:
     while (--n >= 0) {
         virtio_ccw_set_guest_notifier(dev, n, !assigned, false);
     }
+irqroute_error:
+    if (with_irqfd && assigned) {
+        virtio_ccw_release_irqroutes(dev, nvqs);
+    }
     return r;
 }
 
diff --git a/hw/s390x/virtio-ccw.h b/hw/s390x/virtio-ccw.h
index d340bf4..7977a3e 100644
--- a/hw/s390x/virtio-ccw.h
+++ b/hw/s390x/virtio-ccw.h
@@ -77,6 +77,7 @@ typedef struct VirtIOCCWDeviceClass {
 
 typedef struct IndAddr {
     hwaddr addr;
+    void *map;
     unsigned long refcnt;
     int len;
     QTAILQ_ENTRY(IndAddr) sibling;
@@ -93,6 +94,7 @@ struct VirtioCcwDevice {
     uint32_t flags;
     uint8_t thinint_isc;
     uint32_t adapter_id;
+    int gsi[VIRTIO_PCI_QUEUE_MAX];
     /* Guest provided values: */
     IndAddr *indicators;
     IndAddr *indicators2;
diff --git a/include/hw/s390x/adapter.h b/include/hw/s390x/adapter.h
new file mode 100644
index 0000000..7e56724
--- /dev/null
+++ b/include/hw/s390x/adapter.h
@@ -0,0 +1,23 @@
+/*
+ * s390 adapter definitions
+ *
+ * Copyright 2013 IBM Corp.
+ * Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or (at
+ * your option) any later version. See the COPYING file in the top-level
+ * directory.
+ */
+
+#ifndef S390X_ADAPTER_H
+#define S390X_ADAPTER_H
+
+struct AdapterInfo {
+    uint64_t ind_addr;
+    uint64_t summary_addr;
+    uint64_t ind_offset;
+    uint32_t summary_offset;
+    uint32_t adapter_id;
+};
+
+#endif
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 83c9b16..76e6d43 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -72,5 +72,6 @@ typedef struct SHPCDevice SHPCDevice;
 typedef struct FWCfgState FWCfgState;
 typedef struct PcGuestInfo PcGuestInfo;
 typedef struct Range Range;
+typedef struct AdapterInfo AdapterInfo;
 
 #endif /* QEMU_TYPEDEFS_H */
diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 9d2df88..7a0c18a 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -317,6 +317,8 @@ int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg);
 int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg);
 void kvm_irqchip_release_virq(KVMState *s, int virq);
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter);
+
 int kvm_irqchip_add_irqfd_notifier(KVMState *s, EventNotifier *n,
                                    EventNotifier *rn, int virq);
 int kvm_irqchip_remove_irqfd_notifier(KVMState *s, EventNotifier *n, int virq);
diff --git a/kvm-all.c b/kvm-all.c
index 6e6a804..9d6cd45 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -27,6 +27,7 @@
 #include "sysemu/sysemu.h"
 #include "hw/hw.h"
 #include "hw/pci/msi.h"
+#include "hw/s390x/adapter.h"
 #include "exec/gdbstub.h"
 #include "sysemu/kvm.h"
 #include "qemu/bswap.h"
@@ -1262,6 +1263,35 @@ static int kvm_irqchip_assign_irqfd(KVMState *s, int fd, int rfd, int virq,
     return kvm_vm_ioctl(s, KVM_IRQFD, &irqfd);
 }
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
+{
+    struct kvm_irq_routing_entry kroute;
+    int virq;
+
+    if (!kvm_gsi_routing_enabled()) {
+        return -ENOSYS;
+    }
+
+    virq = kvm_irqchip_get_virq(s);
+    if (virq < 0) {
+        return virq;
+    }
+
+    kroute.gsi = virq;
+    kroute.type = KVM_IRQ_ROUTING_S390_ADAPTER;
+    kroute.flags = 0;
+    kroute.u.adapter.summary_addr = adapter->summary_addr;
+    kroute.u.adapter.ind_addr = adapter->ind_addr;
+    kroute.u.adapter.summary_offset = adapter->summary_offset;
+    kroute.u.adapter.ind_offset = adapter->ind_offset;
+    kroute.u.adapter.adapter_id = adapter->adapter_id;
+
+    kvm_add_routing_entry(s, &kroute);
+    kvm_irqchip_commit_routes(s);
+
+    return virq;
+}
+
 #else /* !KVM_CAP_IRQ_ROUTING */
 
 void kvm_init_irq_routing(KVMState *s)
@@ -1282,6 +1312,11 @@ int kvm_irqchip_add_msi_route(KVMState *s, MSIMessage msg)
     return -ENOSYS;
 }
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
+{
+    return -ENOSYS;
+}
+
 static int kvm_irqchip_assign_irqfd(KVMState *s, int fd, int virq, bool assign)
 {
     abort();
@@ -1311,7 +1346,8 @@ static int kvm_irqchip_create(KVMState *s)
     int ret;
 
     if (!qemu_opt_get_bool(qemu_get_machine_opts(), "kernel_irqchip", true) ||
-        !kvm_check_extension(s, KVM_CAP_IRQCHIP)) {
+        (!kvm_check_extension(s, KVM_CAP_IRQCHIP) &&
+         kvm_enable_cap_vm(s, KVM_CAP_S390_IRQCHIP))) {
         return 0;
     }
 
diff --git a/kvm-stub.c b/kvm-stub.c
index e979f76..c329ab5 100644
--- a/kvm-stub.c
+++ b/kvm-stub.c
@@ -136,6 +136,11 @@ int kvm_irqchip_update_msi_route(KVMState *s, int virq, MSIMessage msg)
     return -ENOSYS;
 }
 
+int kvm_irqchip_add_adapter_route(KVMState *s, AdapterInfo *adapter)
+{
+    return -ENOSYS;
+}
+
 int kvm_irqchip_add_irqfd_notifier(KVMState *s, EventNotifier *n,
                                    EventNotifier *rn, int virq)
 {
diff --git a/target-s390x/kvm.c b/target-s390x/kvm.c
index 7e8321e..6b7e5db 100644
--- a/target-s390x/kvm.c
+++ b/target-s390x/kvm.c
@@ -898,6 +898,11 @@ void kvm_s390_enable_css_support(S390CPU *cpu)
 
 void kvm_arch_init_irq_routing(KVMState *s)
 {
+    if (kvm_check_extension(s, KVM_CAP_IRQ_ROUTING)) {
+        kvm_irqfds_allowed = true;
+        kvm_gsi_routing_allowed = true;
+        kvm_halt_in_kernel_allowed = false;
+    }
 }
 
 int kvm_s390_assign_subch_ioeventfd(EventNotifier *notifier, uint32_t sch,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 21+ messages in thread

* Re: [PATCH 4/6] s390x: Add I/O adapter registration.
  2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
@ 2014-02-26  8:29     ` Christian Borntraeger
  -1 siblings, 0 replies; 21+ messages in thread
From: Christian Borntraeger @ 2014-02-26  8:29 UTC (permalink / raw)
  To: Cornelia Huck, kvm, linux-s390, qemu-devel; +Cc: agraf, pbonzini, gleb

On 25/02/14 18:25, Cornelia Huck wrote:

> +int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map)
> +{
> +    struct kvm_s390_io_adapter_req req = {
> +        .id = id,
> +        .type = do_map ? KVM_S390_IO_ADAPTER_MAP : KVM_S390_IO_ADAPTER_UNMAP,
> +        .addr = map_addr,
> +    };
> +    KVMS390FLICState *flic = s390_get_flic();
> +    struct kvm_device_attr attr;

Can we use designated initializer for attr, e.g.
    struct kvm_device_attr attr = {
        .group = KVM_DEV_FLIC_ADAPTER_MODIFY,
        .addr = (uint64_t)&req,
    }

> +    int r;
> +
> +    if (!flic) {
> +        return -ENOSYS;
> +    }
> +    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
> +        return -ENOSYS;
> +    }
> +
> +    attr.group = KVM_DEV_FLIC_ADAPTER_MODIFY;
> +    attr.addr = (uint64_t)&req;

and not do it here. That will zero out the other fields of attr.

Same for the register code.

Christian

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 4/6] s390x: Add I/O adapter registration.
@ 2014-02-26  8:29     ` Christian Borntraeger
  0 siblings, 0 replies; 21+ messages in thread
From: Christian Borntraeger @ 2014-02-26  8:29 UTC (permalink / raw)
  To: Cornelia Huck, kvm, linux-s390, qemu-devel; +Cc: gleb, pbonzini, agraf

On 25/02/14 18:25, Cornelia Huck wrote:

> +int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map)
> +{
> +    struct kvm_s390_io_adapter_req req = {
> +        .id = id,
> +        .type = do_map ? KVM_S390_IO_ADAPTER_MAP : KVM_S390_IO_ADAPTER_UNMAP,
> +        .addr = map_addr,
> +    };
> +    KVMS390FLICState *flic = s390_get_flic();
> +    struct kvm_device_attr attr;

Can we use designated initializer for attr, e.g.
    struct kvm_device_attr attr = {
        .group = KVM_DEV_FLIC_ADAPTER_MODIFY,
        .addr = (uint64_t)&req,
    }

> +    int r;
> +
> +    if (!flic) {
> +        return -ENOSYS;
> +    }
> +    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
> +        return -ENOSYS;
> +    }
> +
> +    attr.group = KVM_DEV_FLIC_ADAPTER_MODIFY;
> +    attr.addr = (uint64_t)&req;

and not do it here. That will zero out the other fields of attr.

Same for the register code.

Christian

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [PATCH 4/6] s390x: Add I/O adapter registration.
  2014-02-26  8:29     ` [Qemu-devel] " Christian Borntraeger
@ 2014-02-26  8:37       ` Cornelia Huck
  -1 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-26  8:37 UTC (permalink / raw)
  To: Christian Borntraeger; +Cc: kvm, linux-s390, qemu-devel, agraf, pbonzini, gleb

On Wed, 26 Feb 2014 09:29:12 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> On 25/02/14 18:25, Cornelia Huck wrote:
> 
> > +int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map)
> > +{
> > +    struct kvm_s390_io_adapter_req req = {
> > +        .id = id,
> > +        .type = do_map ? KVM_S390_IO_ADAPTER_MAP : KVM_S390_IO_ADAPTER_UNMAP,
> > +        .addr = map_addr,
> > +    };
> > +    KVMS390FLICState *flic = s390_get_flic();
> > +    struct kvm_device_attr attr;
> 
> Can we use designated initializer for attr, e.g.
>     struct kvm_device_attr attr = {
>         .group = KVM_DEV_FLIC_ADAPTER_MODIFY,
>         .addr = (uint64_t)&req,
>     }
> 
> > +    int r;
> > +
> > +    if (!flic) {
> > +        return -ENOSYS;
> > +    }
> > +    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
> > +        return -ENOSYS;
> > +    }
> > +
> > +    attr.group = KVM_DEV_FLIC_ADAPTER_MODIFY;
> > +    attr.addr = (uint64_t)&req;
> 
> and not do it here. That will zero out the other fields of attr.
> 
> Same for the register code.
> 

Hm, yes, I missed that. I'll change it for the next version.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 4/6] s390x: Add I/O adapter registration.
@ 2014-02-26  8:37       ` Cornelia Huck
  0 siblings, 0 replies; 21+ messages in thread
From: Cornelia Huck @ 2014-02-26  8:37 UTC (permalink / raw)
  To: Christian Borntraeger; +Cc: linux-s390, kvm, gleb, qemu-devel, agraf, pbonzini

On Wed, 26 Feb 2014 09:29:12 +0100
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> On 25/02/14 18:25, Cornelia Huck wrote:
> 
> > +int kvm_s390_io_adapter_map(uint32_t id, uint64_t map_addr, bool do_map)
> > +{
> > +    struct kvm_s390_io_adapter_req req = {
> > +        .id = id,
> > +        .type = do_map ? KVM_S390_IO_ADAPTER_MAP : KVM_S390_IO_ADAPTER_UNMAP,
> > +        .addr = map_addr,
> > +    };
> > +    KVMS390FLICState *flic = s390_get_flic();
> > +    struct kvm_device_attr attr;
> 
> Can we use designated initializer for attr, e.g.
>     struct kvm_device_attr attr = {
>         .group = KVM_DEV_FLIC_ADAPTER_MODIFY,
>         .addr = (uint64_t)&req,
>     }
> 
> > +    int r;
> > +
> > +    if (!flic) {
> > +        return -ENOSYS;
> > +    }
> > +    if (!kvm_check_extension(kvm_state, KVM_CAP_IRQ_ROUTING)) {
> > +        return -ENOSYS;
> > +    }
> > +
> > +    attr.group = KVM_DEV_FLIC_ADAPTER_MODIFY;
> > +    attr.addr = (uint64_t)&req;
> 
> and not do it here. That will zero out the other fields of attr.
> 
> Same for the register code.
> 

Hm, yes, I missed that. I'll change it for the next version.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 1/6] s390x/virtio-ccw: Adapter interrupt support.
  2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
  (?)
@ 2014-03-04  9:05   ` Cornelia Huck
  2014-03-04  9:20       ` Christian Borntraeger
  -1 siblings, 1 reply; 21+ messages in thread
From: Cornelia Huck @ 2014-03-04  9:05 UTC (permalink / raw)
  To: kvm, linux-s390, qemu-devel; +Cc: gleb, borntraeger, agraf, pbonzini

On Tue, 25 Feb 2014 18:25:15 +0100
Cornelia Huck <cornelia.huck@de.ibm.com> wrote:

> Handle the new CCW_CMD_SET_IND_ADAPTER command enabling adapter interrupts
> on guest request. When active, host->guest notifications will be handled
> via global_indicator -> queue indicators instead of queue indicators +
> subchannel I/O interrupt. Indicators for virtqueues may be present at an
> offset.
> 
> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
> ---
>  hw/s390x/css.c        |   10 +++++
>  hw/s390x/css.h        |    2 +
>  hw/s390x/virtio-ccw.c |  102 +++++++++++++++++++++++++++++++++++++++++++++----
>  hw/s390x/virtio-ccw.h |    4 ++
>  target-s390x/ioinst.h |    2 +
>  target-s390x/kvm.c    |    8 +++-
>  trace-events          |    1 +
>  7 files changed, 120 insertions(+), 9 deletions(-)
> 

As this is a nicely self-contained patch, I'd like to send a pull
request for this, unless there are any objections.

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 1/6] s390x/virtio-ccw: Adapter interrupt support.
  2014-03-04  9:05   ` Cornelia Huck
@ 2014-03-04  9:20       ` Christian Borntraeger
  0 siblings, 0 replies; 21+ messages in thread
From: Christian Borntraeger @ 2014-03-04  9:20 UTC (permalink / raw)
  To: Cornelia Huck, kvm, linux-s390, qemu-devel; +Cc: gleb, agraf, pbonzini

On 04/03/14 10:05, Cornelia Huck wrote:
> On Tue, 25 Feb 2014 18:25:15 +0100
> Cornelia Huck <cornelia.huck@de.ibm.com> wrote:
> 
>> Handle the new CCW_CMD_SET_IND_ADAPTER command enabling adapter interrupts
>> on guest request. When active, host->guest notifications will be handled
>> via global_indicator -> queue indicators instead of queue indicators +
>> subchannel I/O interrupt. Indicators for virtqueues may be present at an
>> offset.
>>
>> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
>> ---
>>  hw/s390x/css.c        |   10 +++++
>>  hw/s390x/css.h        |    2 +
>>  hw/s390x/virtio-ccw.c |  102 +++++++++++++++++++++++++++++++++++++++++++++----
>>  hw/s390x/virtio-ccw.h |    4 ++
>>  target-s390x/ioinst.h |    2 +
>>  target-s390x/kvm.c    |    8 +++-
>>  trace-events          |    1 +
>>  7 files changed, 120 insertions(+), 9 deletions(-)
>>
> 
> As this is a nicely self-contained patch, I'd like to send a pull
> request for this, unless there are any objections.
> 

ACK from my side. I will send out the guest kernel patch with the next kvm patches
for s390.

Christian

^ permalink raw reply	[flat|nested] 21+ messages in thread

* Re: [Qemu-devel] [PATCH 1/6] s390x/virtio-ccw: Adapter interrupt support.
@ 2014-03-04  9:20       ` Christian Borntraeger
  0 siblings, 0 replies; 21+ messages in thread
From: Christian Borntraeger @ 2014-03-04  9:20 UTC (permalink / raw)
  To: Cornelia Huck, kvm, linux-s390, qemu-devel; +Cc: gleb, pbonzini, agraf

On 04/03/14 10:05, Cornelia Huck wrote:
> On Tue, 25 Feb 2014 18:25:15 +0100
> Cornelia Huck <cornelia.huck@de.ibm.com> wrote:
> 
>> Handle the new CCW_CMD_SET_IND_ADAPTER command enabling adapter interrupts
>> on guest request. When active, host->guest notifications will be handled
>> via global_indicator -> queue indicators instead of queue indicators +
>> subchannel I/O interrupt. Indicators for virtqueues may be present at an
>> offset.
>>
>> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
>> ---
>>  hw/s390x/css.c        |   10 +++++
>>  hw/s390x/css.h        |    2 +
>>  hw/s390x/virtio-ccw.c |  102 +++++++++++++++++++++++++++++++++++++++++++++----
>>  hw/s390x/virtio-ccw.h |    4 ++
>>  target-s390x/ioinst.h |    2 +
>>  target-s390x/kvm.c    |    8 +++-
>>  trace-events          |    1 +
>>  7 files changed, 120 insertions(+), 9 deletions(-)
>>
> 
> As this is a nicely self-contained patch, I'd like to send a pull
> request for this, unless there are any objections.
> 

ACK from my side. I will send out the guest kernel patch with the next kvm patches
for s390.

Christian

^ permalink raw reply	[flat|nested] 21+ messages in thread

end of thread, other threads:[~2014-03-04  9:21 UTC | newest]

Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-25 17:25 [PATCH 0/6] qemu: irqfds for s390x Cornelia Huck
2014-02-25 17:25 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:25 ` [PATCH 1/6] s390x/virtio-ccw: Adapter interrupt support Cornelia Huck
2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
2014-03-04  9:05   ` Cornelia Huck
2014-03-04  9:20     ` Christian Borntraeger
2014-03-04  9:20       ` Christian Borntraeger
2014-02-25 17:25 ` [PATCH 2/6] linux-headers: add new interfaces Cornelia Huck
2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:25 ` [PATCH 3/6] kvm: add kvm_enable_cap_{vm,vcpu} Cornelia Huck
2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:25 ` [PATCH 4/6] s390x: Add I/O adapter registration Cornelia Huck
2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
2014-02-26  8:29   ` Christian Borntraeger
2014-02-26  8:29     ` [Qemu-devel] " Christian Borntraeger
2014-02-26  8:37     ` Cornelia Huck
2014-02-26  8:37       ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:25 ` [PATCH 5/6] s390x/virtio-ccw: reference-counted indicators Cornelia Huck
2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:25 ` [PATCH 6/6] s390x/virtio-ccw: Wire up irq routing and irqfds Cornelia Huck
2014-02-25 17:25   ` [Qemu-devel] " Cornelia Huck

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.