* [PATCH 0/7] KVM: irqfds for s390
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck
Hi,
here's my current patchset enabling irqfds for s390, based on current
kvm/queue.
It introduces adapter interrupts for virtio-ccw (these have been specced
for virtio-1.0 in the virtio oasis standard that is currently under work).
We use two-staged indicators for the queue indicator area (summary indicator)
and the queue indicators. Mainly fixes compared to the last time I posted
this as an RFC (it has been some time...)
For implementing irqfds, we exploit the common infrastructure via adding
a new interrupt routing type for adapter interrupts. Each of these routes
contains several parameters like the matching I/O adapter and the locations
of the summary and queue indicators - one route per virtqueue.
Patches 1 and 2 are relevant for guest support for adapter interrupts.
Patch 1 will be pushed upstream via the s390 architecture tree.
Patch 3 tries to fix a problem in the common irqfd code I managed to trigger;
it might make sense to push this on its own.
Patch 4 introduces infrastructure to enable capabilities on a per-vm basis
(instead of only per-vcpu). It might be of common interest, but will be
used in this patchset to introduce a s390-specific irqchip capability that
needs to be enabled by userspace.
Patch 5 introduces the concept of I/O adapters that act as adapter interrupt
sources. They contain various characteristics, not all of them relevant for
virtio-ccw but of general use for other types of adapter interrupts. They
are registered and modified via the new s390 floating interrupt controller.
Patch 6 finally wires up irqfds for virtio-ccw adapter interrupts. The
capability for this needs to be explicitly enabled by userspace to avoid
older qemus choking on newer kernels.
Patch 7 is providing a stop-gap measure to let us run not-quite-as-small
guests: we gobble up irqroutes fairly quickly. There must be a better way.
A git branch with the patches is available at
git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git kvms390-irqfd
Cornelia Huck (6):
KVM: s390: virtio-ccw adapter interrupt support.
KVM: eventfd: Fix lock order inversion.
KVM: Add per-vm capability enablement.
KVM: s390: adapter interrupt sources
KVM: s390: irq routing for adapter interrupts.
KVM: Bump KVM_MAX_IRQ_ROUTES for s390
Martin Schwidefsky (1):
s390/airq: add support for irq ranges
Documentation/virtual/kvm/api.txt | 27 ++-
Documentation/virtual/kvm/devices/s390_flic.txt | 45 ++++
arch/s390/include/asm/airq.h | 14 +-
arch/s390/include/asm/irq.h | 1 +
arch/s390/include/asm/kvm_host.h | 30 +++
arch/s390/include/uapi/asm/kvm.h | 22 ++
arch/s390/kernel/irq.c | 1 +
arch/s390/kvm/Kconfig | 2 +
arch/s390/kvm/Makefile | 2 +-
arch/s390/kvm/interrupt.c | 287 ++++++++++++++++++++++-
arch/s390/kvm/irq.h | 22 ++
arch/s390/kvm/kvm-s390.c | 42 ++++
arch/s390/kvm/kvm-s390.h | 2 +
drivers/s390/cio/airq.c | 66 ++++--
drivers/s390/kvm/virtio_ccw.c | 276 +++++++++++++++++++++-
include/linux/kvm_host.h | 13 +
include/uapi/linux/kvm.h | 16 ++
virt/kvm/eventfd.c | 8 +-
18 files changed, 827 insertions(+), 49 deletions(-)
create mode 100644 arch/s390/kvm/irq.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 0/7] KVM: irqfds for s390
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Hi,
here's my current patchset enabling irqfds for s390, based on current
kvm/queue.
It introduces adapter interrupts for virtio-ccw (these have been specced
for virtio-1.0 in the virtio oasis standard that is currently under work).
We use two-staged indicators for the queue indicator area (summary indicator)
and the queue indicators. Mainly fixes compared to the last time I posted
this as an RFC (it has been some time...)
For implementing irqfds, we exploit the common infrastructure via adding
a new interrupt routing type for adapter interrupts. Each of these routes
contains several parameters like the matching I/O adapter and the locations
of the summary and queue indicators - one route per virtqueue.
Patches 1 and 2 are relevant for guest support for adapter interrupts.
Patch 1 will be pushed upstream via the s390 architecture tree.
Patch 3 tries to fix a problem in the common irqfd code I managed to trigger;
it might make sense to push this on its own.
Patch 4 introduces infrastructure to enable capabilities on a per-vm basis
(instead of only per-vcpu). It might be of common interest, but will be
used in this patchset to introduce a s390-specific irqchip capability that
needs to be enabled by userspace.
Patch 5 introduces the concept of I/O adapters that act as adapter interrupt
sources. They contain various characteristics, not all of them relevant for
virtio-ccw but of general use for other types of adapter interrupts. They
are registered and modified via the new s390 floating interrupt controller.
Patch 6 finally wires up irqfds for virtio-ccw adapter interrupts. The
capability for this needs to be explicitly enabled by userspace to avoid
older qemus choking on newer kernels.
Patch 7 is providing a stop-gap measure to let us run not-quite-as-small
guests: we gobble up irqroutes fairly quickly. There must be a better way.
A git branch with the patches is available at
git://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git kvms390-irqfd
Cornelia Huck (6):
KVM: s390: virtio-ccw adapter interrupt support.
KVM: eventfd: Fix lock order inversion.
KVM: Add per-vm capability enablement.
KVM: s390: adapter interrupt sources
KVM: s390: irq routing for adapter interrupts.
KVM: Bump KVM_MAX_IRQ_ROUTES for s390
Martin Schwidefsky (1):
s390/airq: add support for irq ranges
Documentation/virtual/kvm/api.txt | 27 ++-
Documentation/virtual/kvm/devices/s390_flic.txt | 45 ++++
arch/s390/include/asm/airq.h | 14 +-
arch/s390/include/asm/irq.h | 1 +
arch/s390/include/asm/kvm_host.h | 30 +++
arch/s390/include/uapi/asm/kvm.h | 22 ++
arch/s390/kernel/irq.c | 1 +
arch/s390/kvm/Kconfig | 2 +
arch/s390/kvm/Makefile | 2 +-
arch/s390/kvm/interrupt.c | 287 ++++++++++++++++++++++-
arch/s390/kvm/irq.h | 22 ++
arch/s390/kvm/kvm-s390.c | 42 ++++
arch/s390/kvm/kvm-s390.h | 2 +
drivers/s390/cio/airq.c | 66 ++++--
drivers/s390/kvm/virtio_ccw.c | 276 +++++++++++++++++++++-
include/linux/kvm_host.h | 13 +
include/uapi/linux/kvm.h | 16 ++
virt/kvm/eventfd.c | 8 +-
18 files changed, 827 insertions(+), 49 deletions(-)
create mode 100644 arch/s390/kvm/irq.h
--
1.7.9.5
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH 1/7] s390/airq: add support for irq ranges
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: borntraeger, agraf, pbonzini, gleb, Martin Schwidefsky, Cornelia Huck
From: Martin Schwidefsky <schwidefsky@de.ibm.com>
Add airq_iv_alloc and airq_iv_free to allocate and free consecutive
ranges of irqs from the interrupt vector.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
arch/s390/include/asm/airq.h | 14 +++++++--
drivers/s390/cio/airq.c | 66 ++++++++++++++++++++++++++----------------
2 files changed, 53 insertions(+), 27 deletions(-)
diff --git a/arch/s390/include/asm/airq.h b/arch/s390/include/asm/airq.h
index 4bbb595..bd93ff6 100644
--- a/arch/s390/include/asm/airq.h
+++ b/arch/s390/include/asm/airq.h
@@ -44,11 +44,21 @@ struct airq_iv {
struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags);
void airq_iv_release(struct airq_iv *iv);
-unsigned long airq_iv_alloc_bit(struct airq_iv *iv);
-void airq_iv_free_bit(struct airq_iv *iv, unsigned long bit);
+unsigned long airq_iv_alloc(struct airq_iv *iv, unsigned long num);
+void airq_iv_free(struct airq_iv *iv, unsigned long bit, unsigned long num);
unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start,
unsigned long end);
+static inline unsigned long airq_iv_alloc_bit(struct airq_iv *iv)
+{
+ return airq_iv_alloc(iv, 1);
+}
+
+static inline void airq_iv_free_bit(struct airq_iv *iv, unsigned long bit)
+{
+ airq_iv_free(iv, bit, 1);
+}
+
static inline unsigned long airq_iv_end(struct airq_iv *iv)
{
return iv->end;
diff --git a/drivers/s390/cio/airq.c b/drivers/s390/cio/airq.c
index f055df0..445564c 100644
--- a/drivers/s390/cio/airq.c
+++ b/drivers/s390/cio/airq.c
@@ -186,55 +186,71 @@ void airq_iv_release(struct airq_iv *iv)
EXPORT_SYMBOL(airq_iv_release);
/**
- * airq_iv_alloc_bit - allocate an irq bit from an interrupt vector
+ * airq_iv_alloc - allocate irq bits from an interrupt vector
* @iv: pointer to an interrupt vector structure
+ * @num: number of consecutive irq bits to allocate
*
- * Returns the bit number of the allocated irq, or -1UL if no bit
- * is available or the AIRQ_IV_ALLOC flag has not been specified
+ * Returns the bit number of the first irq in the allocated block of irqs,
+ * or -1UL if no bit is available or the AIRQ_IV_ALLOC flag has not been
+ * specified
*/
-unsigned long airq_iv_alloc_bit(struct airq_iv *iv)
+unsigned long airq_iv_alloc(struct airq_iv *iv, unsigned long num)
{
- unsigned long bit;
+ unsigned long bit, i;
- if (!iv->avail)
+ if (!iv->avail || num == 0)
return -1UL;
spin_lock(&iv->lock);
bit = find_first_bit_inv(iv->avail, iv->bits);
- if (bit < iv->bits) {
- clear_bit_inv(bit, iv->avail);
- if (bit >= iv->end)
- iv->end = bit + 1;
- } else
+ while (bit + num <= iv->bits) {
+ for (i = 1; i < num; i++)
+ if (!test_bit_inv(bit + i, iv->avail))
+ break;
+ if (i >= num) {
+ /* Found a suitable block of irqs */
+ for (i = 0; i < num; i++)
+ clear_bit_inv(bit + i, iv->avail);
+ if (bit + num >= iv->end)
+ iv->end = bit + num + 1;
+ break;
+ }
+ bit = find_next_bit_inv(iv->avail, iv->bits, bit + i + 1);
+ }
+ if (bit + num > iv->bits)
bit = -1UL;
spin_unlock(&iv->lock);
return bit;
}
-EXPORT_SYMBOL(airq_iv_alloc_bit);
+EXPORT_SYMBOL(airq_iv_alloc);
/**
- * airq_iv_free_bit - free an irq bit of an interrupt vector
+ * airq_iv_free - free irq bits of an interrupt vector
* @iv: pointer to interrupt vector structure
- * @bit: number of the irq bit to free
+ * @bit: number of the first irq bit to free
+ * @num: number of consecutive irq bits to free
*/
-void airq_iv_free_bit(struct airq_iv *iv, unsigned long bit)
+void airq_iv_free(struct airq_iv *iv, unsigned long bit, unsigned long num)
{
- if (!iv->avail)
+ unsigned long i;
+
+ if (!iv->avail || num == 0)
return;
spin_lock(&iv->lock);
- /* Clear (possibly left over) interrupt bit */
- clear_bit_inv(bit, iv->vector);
- /* Make the bit position available again */
- set_bit_inv(bit, iv->avail);
- if (bit == iv->end - 1) {
+ for (i = 0; i < num; i++) {
+ /* Clear (possibly left over) interrupt bit */
+ clear_bit_inv(bit + i, iv->vector);
+ /* Make the bit positions available again */
+ set_bit_inv(bit + i, iv->avail);
+ }
+ if (bit + num >= iv->end) {
/* Find new end of bit-field */
- while (--iv->end > 0)
- if (!test_bit_inv(iv->end - 1, iv->avail))
- break;
+ while (iv->end > 0 && !test_bit_inv(iv->end - 1, iv->avail))
+ iv->end--;
}
spin_unlock(&iv->lock);
}
-EXPORT_SYMBOL(airq_iv_free_bit);
+EXPORT_SYMBOL(airq_iv_free);
/**
* airq_iv_scan - scan interrupt vector for non-zero bits
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 1/7] s390/airq: add support for irq ranges
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, agraf, borntraeger, Cornelia Huck, pbonzini, Martin Schwidefsky
From: Martin Schwidefsky <schwidefsky@de.ibm.com>
Add airq_iv_alloc and airq_iv_free to allocate and free consecutive
ranges of irqs from the interrupt vector.
Signed-off-by: Martin Schwidefsky <schwidefsky@de.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
arch/s390/include/asm/airq.h | 14 +++++++--
drivers/s390/cio/airq.c | 66 ++++++++++++++++++++++++++----------------
2 files changed, 53 insertions(+), 27 deletions(-)
diff --git a/arch/s390/include/asm/airq.h b/arch/s390/include/asm/airq.h
index 4bbb595..bd93ff6 100644
--- a/arch/s390/include/asm/airq.h
+++ b/arch/s390/include/asm/airq.h
@@ -44,11 +44,21 @@ struct airq_iv {
struct airq_iv *airq_iv_create(unsigned long bits, unsigned long flags);
void airq_iv_release(struct airq_iv *iv);
-unsigned long airq_iv_alloc_bit(struct airq_iv *iv);
-void airq_iv_free_bit(struct airq_iv *iv, unsigned long bit);
+unsigned long airq_iv_alloc(struct airq_iv *iv, unsigned long num);
+void airq_iv_free(struct airq_iv *iv, unsigned long bit, unsigned long num);
unsigned long airq_iv_scan(struct airq_iv *iv, unsigned long start,
unsigned long end);
+static inline unsigned long airq_iv_alloc_bit(struct airq_iv *iv)
+{
+ return airq_iv_alloc(iv, 1);
+}
+
+static inline void airq_iv_free_bit(struct airq_iv *iv, unsigned long bit)
+{
+ airq_iv_free(iv, bit, 1);
+}
+
static inline unsigned long airq_iv_end(struct airq_iv *iv)
{
return iv->end;
diff --git a/drivers/s390/cio/airq.c b/drivers/s390/cio/airq.c
index f055df0..445564c 100644
--- a/drivers/s390/cio/airq.c
+++ b/drivers/s390/cio/airq.c
@@ -186,55 +186,71 @@ void airq_iv_release(struct airq_iv *iv)
EXPORT_SYMBOL(airq_iv_release);
/**
- * airq_iv_alloc_bit - allocate an irq bit from an interrupt vector
+ * airq_iv_alloc - allocate irq bits from an interrupt vector
* @iv: pointer to an interrupt vector structure
+ * @num: number of consecutive irq bits to allocate
*
- * Returns the bit number of the allocated irq, or -1UL if no bit
- * is available or the AIRQ_IV_ALLOC flag has not been specified
+ * Returns the bit number of the first irq in the allocated block of irqs,
+ * or -1UL if no bit is available or the AIRQ_IV_ALLOC flag has not been
+ * specified
*/
-unsigned long airq_iv_alloc_bit(struct airq_iv *iv)
+unsigned long airq_iv_alloc(struct airq_iv *iv, unsigned long num)
{
- unsigned long bit;
+ unsigned long bit, i;
- if (!iv->avail)
+ if (!iv->avail || num == 0)
return -1UL;
spin_lock(&iv->lock);
bit = find_first_bit_inv(iv->avail, iv->bits);
- if (bit < iv->bits) {
- clear_bit_inv(bit, iv->avail);
- if (bit >= iv->end)
- iv->end = bit + 1;
- } else
+ while (bit + num <= iv->bits) {
+ for (i = 1; i < num; i++)
+ if (!test_bit_inv(bit + i, iv->avail))
+ break;
+ if (i >= num) {
+ /* Found a suitable block of irqs */
+ for (i = 0; i < num; i++)
+ clear_bit_inv(bit + i, iv->avail);
+ if (bit + num >= iv->end)
+ iv->end = bit + num + 1;
+ break;
+ }
+ bit = find_next_bit_inv(iv->avail, iv->bits, bit + i + 1);
+ }
+ if (bit + num > iv->bits)
bit = -1UL;
spin_unlock(&iv->lock);
return bit;
}
-EXPORT_SYMBOL(airq_iv_alloc_bit);
+EXPORT_SYMBOL(airq_iv_alloc);
/**
- * airq_iv_free_bit - free an irq bit of an interrupt vector
+ * airq_iv_free - free irq bits of an interrupt vector
* @iv: pointer to interrupt vector structure
- * @bit: number of the irq bit to free
+ * @bit: number of the first irq bit to free
+ * @num: number of consecutive irq bits to free
*/
-void airq_iv_free_bit(struct airq_iv *iv, unsigned long bit)
+void airq_iv_free(struct airq_iv *iv, unsigned long bit, unsigned long num)
{
- if (!iv->avail)
+ unsigned long i;
+
+ if (!iv->avail || num == 0)
return;
spin_lock(&iv->lock);
- /* Clear (possibly left over) interrupt bit */
- clear_bit_inv(bit, iv->vector);
- /* Make the bit position available again */
- set_bit_inv(bit, iv->avail);
- if (bit == iv->end - 1) {
+ for (i = 0; i < num; i++) {
+ /* Clear (possibly left over) interrupt bit */
+ clear_bit_inv(bit + i, iv->vector);
+ /* Make the bit positions available again */
+ set_bit_inv(bit + i, iv->avail);
+ }
+ if (bit + num >= iv->end) {
/* Find new end of bit-field */
- while (--iv->end > 0)
- if (!test_bit_inv(iv->end - 1, iv->avail))
- break;
+ while (iv->end > 0 && !test_bit_inv(iv->end - 1, iv->avail))
+ iv->end--;
}
spin_unlock(&iv->lock);
}
-EXPORT_SYMBOL(airq_iv_free_bit);
+EXPORT_SYMBOL(airq_iv_free);
/**
* airq_iv_scan - scan interrupt vector for non-zero bits
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 2/7] KVM: s390: virtio-ccw adapter interrupt support.
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Implement the new CCW_CMD_SET_IND_ADAPTER command and try to enable
adapter interrupts for every device on the first startup. If the host
does not support adapter interrupts, fall back to normal I/O interrupts.
virtio-ccw adapter interrupts use the same isc as normal I/O subchannels
and share a summary indicator for all devices sharing the same indicator
area.
Indicator bits for the individual virtqueues may be contained in the same
indicator area for different devices.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
arch/s390/include/asm/irq.h | 1 +
arch/s390/kernel/irq.c | 1 +
drivers/s390/kvm/virtio_ccw.c | 276 +++++++++++++++++++++++++++++++++++++++--
3 files changed, 268 insertions(+), 10 deletions(-)
diff --git a/arch/s390/include/asm/irq.h b/arch/s390/include/asm/irq.h
index 5f8bcc5..35f0faa 100644
--- a/arch/s390/include/asm/irq.h
+++ b/arch/s390/include/asm/irq.h
@@ -53,6 +53,7 @@ enum interruption_class {
IRQIO_PCI,
IRQIO_MSI,
IRQIO_VIR,
+ IRQIO_VAI,
NMI_NMI,
CPU_RST,
NR_ARCH_IRQS
diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c
index bb27a26..c288ef7 100644
--- a/arch/s390/kernel/irq.c
+++ b/arch/s390/kernel/irq.c
@@ -84,6 +84,7 @@ static const struct irq_class irqclass_sub_desc[NR_ARCH_IRQS] = {
[IRQIO_PCI] = {.name = "PCI", .desc = "[I/O] PCI Interrupt" },
[IRQIO_MSI] = {.name = "MSI", .desc = "[I/O] MSI Interrupt" },
[IRQIO_VIR] = {.name = "VIR", .desc = "[I/O] Virtual I/O Devices"},
+ [IRQIO_VAI] = {.name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"},
[NMI_NMI] = {.name = "NMI", .desc = "[NMI] Machine Check"},
[CPU_RST] = {.name = "RST", .desc = "[CPU] CPU Restart"},
};
diff --git a/drivers/s390/kvm/virtio_ccw.c b/drivers/s390/kvm/virtio_ccw.c
index 0fc5848..09afefe 100644
--- a/drivers/s390/kvm/virtio_ccw.c
+++ b/drivers/s390/kvm/virtio_ccw.c
@@ -1,7 +1,7 @@
/*
* ccw based virtio transport
*
- * Copyright IBM Corp. 2012
+ * Copyright IBM Corp. 2012,2014
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License (version 2 only)
@@ -32,6 +32,8 @@
#include <asm/cio.h>
#include <asm/ccwdev.h>
#include <asm/virtio-ccw.h>
+#include <asm/isc.h>
+#include <asm/airq.h>
/*
* virtio related functions
@@ -58,6 +60,8 @@ struct virtio_ccw_device {
unsigned long indicators;
unsigned long indicators2;
struct vq_config_block *config_block;
+ bool is_thinint;
+ void *airq_info;
};
struct vq_info_block {
@@ -72,15 +76,38 @@ struct virtio_feature_desc {
__u8 index;
} __packed;
+struct virtio_thinint_area {
+ unsigned long summary_indicator;
+ unsigned long indicator;
+ u64 bit_nr;
+ u8 isc;
+} __packed;
+
struct virtio_ccw_vq_info {
struct virtqueue *vq;
int num;
void *queue;
struct vq_info_block *info_block;
+ int bit_nr;
struct list_head node;
long cookie;
};
+#define VIRTIO_AIRQ_ISC IO_SCH_ISC /* inherit from subchannel */
+
+#define VIRTIO_IV_BITS (L1_CACHE_BYTES * 8)
+#define MAX_AIRQ_AREAS 20
+
+static int virtio_ccw_use_airq = 1;
+
+struct airq_info {
+ rwlock_t lock;
+ u8 summary_indicator;
+ struct airq_struct airq;
+ struct airq_iv *aiv;
+};
+static struct airq_info *airq_areas[MAX_AIRQ_AREAS];
+
#define CCW_CMD_SET_VQ 0x13
#define CCW_CMD_VDEV_RESET 0x33
#define CCW_CMD_SET_IND 0x43
@@ -91,6 +118,7 @@ struct virtio_ccw_vq_info {
#define CCW_CMD_WRITE_CONF 0x21
#define CCW_CMD_WRITE_STATUS 0x31
#define CCW_CMD_READ_VQ_CONF 0x32
+#define CCW_CMD_SET_IND_ADAPTER 0x73
#define VIRTIO_CCW_DOING_SET_VQ 0x00010000
#define VIRTIO_CCW_DOING_RESET 0x00040000
@@ -102,6 +130,7 @@ struct virtio_ccw_vq_info {
#define VIRTIO_CCW_DOING_SET_IND 0x01000000
#define VIRTIO_CCW_DOING_READ_VQ_CONF 0x02000000
#define VIRTIO_CCW_DOING_SET_CONF_IND 0x04000000
+#define VIRTIO_CCW_DOING_SET_IND_ADAPTER 0x08000000
#define VIRTIO_CCW_INTPARM_MASK 0xffff0000
static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
@@ -109,6 +138,125 @@ static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
return container_of(vdev, struct virtio_ccw_device, vdev);
}
+static void drop_airq_indicator(struct virtqueue *vq, struct airq_info *info)
+{
+ unsigned long i, flags;
+
+ write_lock_irqsave(&info->lock, flags);
+ for (i = 0; i < airq_iv_end(info->aiv); i++) {
+ if (vq == (void *)airq_iv_get_ptr(info->aiv, i)) {
+ airq_iv_free_bit(info->aiv, i);
+ airq_iv_set_ptr(info->aiv, i, 0);
+ break;
+ }
+ }
+ write_unlock_irqrestore(&info->lock, flags);
+}
+
+static void virtio_airq_handler(struct airq_struct *airq)
+{
+ struct airq_info *info = container_of(airq, struct airq_info, airq);
+ unsigned long ai;
+
+ inc_irq_stat(IRQIO_VAI);
+ read_lock(&info->lock);
+ /* Walk through indicators field, summary indicator active. */
+ for (ai = 0;;) {
+ ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
+ if (ai == -1UL)
+ break;
+ vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
+ }
+ info->summary_indicator = 0;
+ smp_wmb();
+ /* Walk through indicators field, summary indicator not active. */
+ for (ai = 0;;) {
+ ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
+ if (ai == -1UL)
+ break;
+ vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
+ }
+ read_unlock(&info->lock);
+}
+
+static struct airq_info *new_airq_info(void)
+{
+ struct airq_info *info;
+ int rc;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (!info)
+ return NULL;
+ rwlock_init(&info->lock);
+ info->aiv = airq_iv_create(VIRTIO_IV_BITS, AIRQ_IV_ALLOC | AIRQ_IV_PTR);
+ if (!info->aiv) {
+ kfree(info);
+ return NULL;
+ }
+ info->airq.handler = virtio_airq_handler;
+ info->airq.lsi_ptr = &info->summary_indicator;
+ info->airq.lsi_mask = 0xff;
+ info->airq.isc = VIRTIO_AIRQ_ISC;
+ rc = register_adapter_interrupt(&info->airq);
+ if (rc) {
+ airq_iv_release(info->aiv);
+ kfree(info);
+ return NULL;
+ }
+ return info;
+}
+
+static void destroy_airq_info(struct airq_info *info)
+{
+ if (!info)
+ return;
+
+ unregister_adapter_interrupt(&info->airq);
+ airq_iv_release(info->aiv);
+ kfree(info);
+}
+
+static unsigned long get_airq_indicator(struct virtqueue *vqs[], int nvqs,
+ u64 *first, void **airq_info)
+{
+ int i, j;
+ struct airq_info *info;
+ unsigned long indicator_addr = 0;
+ unsigned long bit, flags;
+
+ for (i = 0; i < MAX_AIRQ_AREAS && !indicator_addr; i++) {
+ if (!airq_areas[i])
+ airq_areas[i] = new_airq_info();
+ info = airq_areas[i];
+ if (!info)
+ return 0;
+ write_lock_irqsave(&info->lock, flags);
+ bit = airq_iv_alloc(info->aiv, nvqs);
+ if (bit == -1UL) {
+ /* Not enough vacancies. */
+ write_unlock_irqrestore(&info->lock, flags);
+ continue;
+ }
+ *first = bit;
+ *airq_info = info;
+ indicator_addr = (unsigned long)info->aiv->vector;
+ for (j = 0; j < nvqs; j++) {
+ airq_iv_set_ptr(info->aiv, bit + j,
+ (unsigned long)vqs[j]);
+ }
+ write_unlock_irqrestore(&info->lock, flags);
+ }
+ return indicator_addr;
+}
+
+static void virtio_ccw_drop_indicators(struct virtio_ccw_device *vcdev)
+{
+ struct virtio_ccw_vq_info *info;
+
+ list_for_each_entry(info, &vcdev->virtqueues, node)
+ drop_airq_indicator(info->vq, vcdev->airq_info);
+}
+
static int doing_io(struct virtio_ccw_device *vcdev, __u32 flag)
{
unsigned long flags;
@@ -145,6 +293,51 @@ static int ccw_io_helper(struct virtio_ccw_device *vcdev,
return ret ? ret : vcdev->err;
}
+static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev,
+ struct ccw1 *ccw)
+{
+ int ret;
+ unsigned long *indicatorp = NULL;
+ struct virtio_thinint_area *thinint_area = NULL;
+ struct airq_info *airq_info = vcdev->airq_info;
+
+ if (vcdev->is_thinint) {
+ thinint_area = kzalloc(sizeof(*thinint_area),
+ GFP_DMA | GFP_KERNEL);
+ if (!thinint_area)
+ return;
+ thinint_area->summary_indicator =
+ (unsigned long) &airq_info->summary_indicator;
+ thinint_area->isc = VIRTIO_AIRQ_ISC;
+ ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
+ ccw->count = sizeof(*thinint_area);
+ ccw->cda = (__u32)(unsigned long) thinint_area;
+ } else {
+ indicatorp = kmalloc(sizeof(&vcdev->indicators),
+ GFP_DMA | GFP_KERNEL);
+ if (!indicatorp)
+ return;
+ *indicatorp = 0;
+ ccw->cmd_code = CCW_CMD_SET_IND;
+ ccw->count = sizeof(vcdev->indicators);
+ ccw->cda = (__u32)(unsigned long) indicatorp;
+ }
+ /* Deregister indicators from host. */
+ vcdev->indicators = 0;
+ ccw->flags = 0;
+ ret = ccw_io_helper(vcdev, ccw,
+ vcdev->is_thinint ?
+ VIRTIO_CCW_DOING_SET_IND_ADAPTER :
+ VIRTIO_CCW_DOING_SET_IND);
+ if (ret && (ret != -ENODEV))
+ dev_info(&vcdev->cdev->dev,
+ "Failed to deregister indicators (%d)\n", ret);
+ else if (vcdev->is_thinint)
+ virtio_ccw_drop_indicators(vcdev);
+ kfree(indicatorp);
+ kfree(thinint_area);
+}
+
static inline long do_kvm_notify(struct subchannel_id schid,
unsigned long queue_index,
long cookie)
@@ -232,11 +425,13 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev)
{
struct virtqueue *vq, *n;
struct ccw1 *ccw;
+ struct virtio_ccw_device *vcdev = to_vc_device(vdev);
ccw = kzalloc(sizeof(*ccw), GFP_DMA | GFP_KERNEL);
if (!ccw)
return;
+ virtio_ccw_drop_indicator(vcdev, ccw);
list_for_each_entry_safe(vq, n, &vdev->vqs, list)
virtio_ccw_del_vq(vq, ccw);
@@ -326,6 +521,52 @@ out_err:
return ERR_PTR(err);
}
+static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev,
+ struct virtqueue *vqs[], int nvqs,
+ struct ccw1 *ccw)
+{
+ int ret;
+ struct virtio_thinint_area *thinint_area = NULL;
+ struct airq_info *info;
+
+ thinint_area = kzalloc(sizeof(*thinint_area), GFP_DMA | GFP_KERNEL);
+ if (!thinint_area) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ /* Try to get an indicator. */
+ thinint_area->indicator = get_airq_indicator(vqs, nvqs,
+ &thinint_area->bit_nr,
+ &vcdev->airq_info);
+ if (!thinint_area->indicator) {
+ ret = -ENOSPC;
+ goto out;
+ }
+ info = vcdev->airq_info;
+ thinint_area->summary_indicator =
+ (unsigned long) &info->summary_indicator;
+ thinint_area->isc = VIRTIO_AIRQ_ISC;
+ ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
+ ccw->flags = CCW_FLAG_SLI;
+ ccw->count = sizeof(*thinint_area);
+ ccw->cda = (__u32)(unsigned long)thinint_area;
+ ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND_ADAPTER);
+ if (ret) {
+ dev_warn(&vcdev->cdev->dev,
+ "enabling adapter interrupts = %d\n", ret);
+ if (ret == -EOPNOTSUPP)
+ /*
+ * The host does not support adapter interrupts
+ * for virtio-ccw, stop trying.
+ */
+ virtio_ccw_use_airq = 0;
+ virtio_ccw_drop_indicators(vcdev);
+ }
+out:
+ kfree(thinint_area);
+ return ret;
+}
+
static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
struct virtqueue *vqs[],
vq_callback_t *callbacks[],
@@ -355,15 +596,23 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
if (!indicatorp)
goto out;
*indicatorp = (unsigned long) &vcdev->indicators;
- /* Register queue indicators with host. */
- vcdev->indicators = 0;
- ccw->cmd_code = CCW_CMD_SET_IND;
- ccw->flags = 0;
- ccw->count = sizeof(vcdev->indicators);
- ccw->cda = (__u32)(unsigned long) indicatorp;
- ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
- if (ret)
- goto out;
+ if (vcdev->is_thinint) {
+ ret = virtio_ccw_register_adapter_ind(vcdev, vqs, nvqs, ccw);
+ if (ret)
+ /* no error, just fall back to legacy interrupts */
+ vcdev->is_thinint = 0;
+ }
+ if (!vcdev->is_thinint) {
+ /* Register queue indicators with host. */
+ vcdev->indicators = 0;
+ ccw->cmd_code = CCW_CMD_SET_IND;
+ ccw->flags = 0;
+ ccw->count = sizeof(vcdev->indicators);
+ ccw->cda = (__u32)(unsigned long) indicatorp;
+ ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
+ if (ret)
+ goto out;
+ }
/* Register indicators2 with host for config changes */
*indicatorp = (unsigned long) &vcdev->indicators2;
vcdev->indicators2 = 0;
@@ -663,6 +912,7 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev,
case VIRTIO_CCW_DOING_SET_CONF_IND:
case VIRTIO_CCW_DOING_RESET:
case VIRTIO_CCW_DOING_READ_VQ_CONF:
+ case VIRTIO_CCW_DOING_SET_IND_ADAPTER:
vcdev->curr_io &= ~activity;
wake_up(&vcdev->wait_q);
break;
@@ -778,6 +1028,8 @@ static int virtio_ccw_online(struct ccw_device *cdev)
goto out_free;
}
+ vcdev->is_thinint = virtio_ccw_use_airq; /* at least try */
+
vcdev->vdev.dev.parent = &cdev->dev;
vcdev->vdev.dev.release = virtio_ccw_release_dev;
vcdev->vdev.config = &virtio_ccw_config_ops;
@@ -935,6 +1187,10 @@ module_init(virtio_ccw_init);
static void __exit virtio_ccw_exit(void)
{
+ int i;
+
ccw_driver_unregister(&virtio_ccw_driver);
+ for (i = 0; i < MAX_AIRQ_AREAS; i++)
+ destroy_airq_info(airq_areas[i]);
}
module_exit(virtio_ccw_exit);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 2/7] KVM: s390: virtio-ccw adapter interrupt support.
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Implement the new CCW_CMD_SET_IND_ADAPTER command and try to enable
adapter interrupts for every device on the first startup. If the host
does not support adapter interrupts, fall back to normal I/O interrupts.
virtio-ccw adapter interrupts use the same isc as normal I/O subchannels
and share a summary indicator for all devices sharing the same indicator
area.
Indicator bits for the individual virtqueues may be contained in the same
indicator area for different devices.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
arch/s390/include/asm/irq.h | 1 +
arch/s390/kernel/irq.c | 1 +
drivers/s390/kvm/virtio_ccw.c | 276 +++++++++++++++++++++++++++++++++++++++--
3 files changed, 268 insertions(+), 10 deletions(-)
diff --git a/arch/s390/include/asm/irq.h b/arch/s390/include/asm/irq.h
index 5f8bcc5..35f0faa 100644
--- a/arch/s390/include/asm/irq.h
+++ b/arch/s390/include/asm/irq.h
@@ -53,6 +53,7 @@ enum interruption_class {
IRQIO_PCI,
IRQIO_MSI,
IRQIO_VIR,
+ IRQIO_VAI,
NMI_NMI,
CPU_RST,
NR_ARCH_IRQS
diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c
index bb27a26..c288ef7 100644
--- a/arch/s390/kernel/irq.c
+++ b/arch/s390/kernel/irq.c
@@ -84,6 +84,7 @@ static const struct irq_class irqclass_sub_desc[NR_ARCH_IRQS] = {
[IRQIO_PCI] = {.name = "PCI", .desc = "[I/O] PCI Interrupt" },
[IRQIO_MSI] = {.name = "MSI", .desc = "[I/O] MSI Interrupt" },
[IRQIO_VIR] = {.name = "VIR", .desc = "[I/O] Virtual I/O Devices"},
+ [IRQIO_VAI] = {.name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"},
[NMI_NMI] = {.name = "NMI", .desc = "[NMI] Machine Check"},
[CPU_RST] = {.name = "RST", .desc = "[CPU] CPU Restart"},
};
diff --git a/drivers/s390/kvm/virtio_ccw.c b/drivers/s390/kvm/virtio_ccw.c
index 0fc5848..09afefe 100644
--- a/drivers/s390/kvm/virtio_ccw.c
+++ b/drivers/s390/kvm/virtio_ccw.c
@@ -1,7 +1,7 @@
/*
* ccw based virtio transport
*
- * Copyright IBM Corp. 2012
+ * Copyright IBM Corp. 2012,2014
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License (version 2 only)
@@ -32,6 +32,8 @@
#include <asm/cio.h>
#include <asm/ccwdev.h>
#include <asm/virtio-ccw.h>
+#include <asm/isc.h>
+#include <asm/airq.h>
/*
* virtio related functions
@@ -58,6 +60,8 @@ struct virtio_ccw_device {
unsigned long indicators;
unsigned long indicators2;
struct vq_config_block *config_block;
+ bool is_thinint;
+ void *airq_info;
};
struct vq_info_block {
@@ -72,15 +76,38 @@ struct virtio_feature_desc {
__u8 index;
} __packed;
+struct virtio_thinint_area {
+ unsigned long summary_indicator;
+ unsigned long indicator;
+ u64 bit_nr;
+ u8 isc;
+} __packed;
+
struct virtio_ccw_vq_info {
struct virtqueue *vq;
int num;
void *queue;
struct vq_info_block *info_block;
+ int bit_nr;
struct list_head node;
long cookie;
};
+#define VIRTIO_AIRQ_ISC IO_SCH_ISC /* inherit from subchannel */
+
+#define VIRTIO_IV_BITS (L1_CACHE_BYTES * 8)
+#define MAX_AIRQ_AREAS 20
+
+static int virtio_ccw_use_airq = 1;
+
+struct airq_info {
+ rwlock_t lock;
+ u8 summary_indicator;
+ struct airq_struct airq;
+ struct airq_iv *aiv;
+};
+static struct airq_info *airq_areas[MAX_AIRQ_AREAS];
+
#define CCW_CMD_SET_VQ 0x13
#define CCW_CMD_VDEV_RESET 0x33
#define CCW_CMD_SET_IND 0x43
@@ -91,6 +118,7 @@ struct virtio_ccw_vq_info {
#define CCW_CMD_WRITE_CONF 0x21
#define CCW_CMD_WRITE_STATUS 0x31
#define CCW_CMD_READ_VQ_CONF 0x32
+#define CCW_CMD_SET_IND_ADAPTER 0x73
#define VIRTIO_CCW_DOING_SET_VQ 0x00010000
#define VIRTIO_CCW_DOING_RESET 0x00040000
@@ -102,6 +130,7 @@ struct virtio_ccw_vq_info {
#define VIRTIO_CCW_DOING_SET_IND 0x01000000
#define VIRTIO_CCW_DOING_READ_VQ_CONF 0x02000000
#define VIRTIO_CCW_DOING_SET_CONF_IND 0x04000000
+#define VIRTIO_CCW_DOING_SET_IND_ADAPTER 0x08000000
#define VIRTIO_CCW_INTPARM_MASK 0xffff0000
static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
@@ -109,6 +138,125 @@ static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
return container_of(vdev, struct virtio_ccw_device, vdev);
}
+static void drop_airq_indicator(struct virtqueue *vq, struct airq_info *info)
+{
+ unsigned long i, flags;
+
+ write_lock_irqsave(&info->lock, flags);
+ for (i = 0; i < airq_iv_end(info->aiv); i++) {
+ if (vq == (void *)airq_iv_get_ptr(info->aiv, i)) {
+ airq_iv_free_bit(info->aiv, i);
+ airq_iv_set_ptr(info->aiv, i, 0);
+ break;
+ }
+ }
+ write_unlock_irqrestore(&info->lock, flags);
+}
+
+static void virtio_airq_handler(struct airq_struct *airq)
+{
+ struct airq_info *info = container_of(airq, struct airq_info, airq);
+ unsigned long ai;
+
+ inc_irq_stat(IRQIO_VAI);
+ read_lock(&info->lock);
+ /* Walk through indicators field, summary indicator active. */
+ for (ai = 0;;) {
+ ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
+ if (ai == -1UL)
+ break;
+ vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
+ }
+ info->summary_indicator = 0;
+ smp_wmb();
+ /* Walk through indicators field, summary indicator not active. */
+ for (ai = 0;;) {
+ ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
+ if (ai == -1UL)
+ break;
+ vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
+ }
+ read_unlock(&info->lock);
+}
+
+static struct airq_info *new_airq_info(void)
+{
+ struct airq_info *info;
+ int rc;
+
+ info = kzalloc(sizeof(*info), GFP_KERNEL);
+ if (!info)
+ return NULL;
+ rwlock_init(&info->lock);
+ info->aiv = airq_iv_create(VIRTIO_IV_BITS, AIRQ_IV_ALLOC | AIRQ_IV_PTR);
+ if (!info->aiv) {
+ kfree(info);
+ return NULL;
+ }
+ info->airq.handler = virtio_airq_handler;
+ info->airq.lsi_ptr = &info->summary_indicator;
+ info->airq.lsi_mask = 0xff;
+ info->airq.isc = VIRTIO_AIRQ_ISC;
+ rc = register_adapter_interrupt(&info->airq);
+ if (rc) {
+ airq_iv_release(info->aiv);
+ kfree(info);
+ return NULL;
+ }
+ return info;
+}
+
+static void destroy_airq_info(struct airq_info *info)
+{
+ if (!info)
+ return;
+
+ unregister_adapter_interrupt(&info->airq);
+ airq_iv_release(info->aiv);
+ kfree(info);
+}
+
+static unsigned long get_airq_indicator(struct virtqueue *vqs[], int nvqs,
+ u64 *first, void **airq_info)
+{
+ int i, j;
+ struct airq_info *info;
+ unsigned long indicator_addr = 0;
+ unsigned long bit, flags;
+
+ for (i = 0; i < MAX_AIRQ_AREAS && !indicator_addr; i++) {
+ if (!airq_areas[i])
+ airq_areas[i] = new_airq_info();
+ info = airq_areas[i];
+ if (!info)
+ return 0;
+ write_lock_irqsave(&info->lock, flags);
+ bit = airq_iv_alloc(info->aiv, nvqs);
+ if (bit == -1UL) {
+ /* Not enough vacancies. */
+ write_unlock_irqrestore(&info->lock, flags);
+ continue;
+ }
+ *first = bit;
+ *airq_info = info;
+ indicator_addr = (unsigned long)info->aiv->vector;
+ for (j = 0; j < nvqs; j++) {
+ airq_iv_set_ptr(info->aiv, bit + j,
+ (unsigned long)vqs[j]);
+ }
+ write_unlock_irqrestore(&info->lock, flags);
+ }
+ return indicator_addr;
+}
+
+static void virtio_ccw_drop_indicators(struct virtio_ccw_device *vcdev)
+{
+ struct virtio_ccw_vq_info *info;
+
+ list_for_each_entry(info, &vcdev->virtqueues, node)
+ drop_airq_indicator(info->vq, vcdev->airq_info);
+}
+
static int doing_io(struct virtio_ccw_device *vcdev, __u32 flag)
{
unsigned long flags;
@@ -145,6 +293,51 @@ static int ccw_io_helper(struct virtio_ccw_device *vcdev,
return ret ? ret : vcdev->err;
}
+static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev,
+ struct ccw1 *ccw)
+{
+ int ret;
+ unsigned long *indicatorp = NULL;
+ struct virtio_thinint_area *thinint_area = NULL;
+ struct airq_info *airq_info = vcdev->airq_info;
+
+ if (vcdev->is_thinint) {
+ thinint_area = kzalloc(sizeof(*thinint_area),
+ GFP_DMA | GFP_KERNEL);
+ if (!thinint_area)
+ return;
+ thinint_area->summary_indicator =
+ (unsigned long) &airq_info->summary_indicator;
+ thinint_area->isc = VIRTIO_AIRQ_ISC;
+ ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
+ ccw->count = sizeof(*thinint_area);
+ ccw->cda = (__u32)(unsigned long) thinint_area;
+ } else {
+ indicatorp = kmalloc(sizeof(&vcdev->indicators),
+ GFP_DMA | GFP_KERNEL);
+ if (!indicatorp)
+ return;
+ *indicatorp = 0;
+ ccw->cmd_code = CCW_CMD_SET_IND;
+ ccw->count = sizeof(vcdev->indicators);
+ ccw->cda = (__u32)(unsigned long) indicatorp;
+ }
+ /* Deregister indicators from host. */
+ vcdev->indicators = 0;
+ ccw->flags = 0;
+ ret = ccw_io_helper(vcdev, ccw,
+ vcdev->is_thinint ?
+ VIRTIO_CCW_DOING_SET_IND_ADAPTER :
+ VIRTIO_CCW_DOING_SET_IND);
+ if (ret && (ret != -ENODEV))
+ dev_info(&vcdev->cdev->dev,
+ "Failed to deregister indicators (%d)\n", ret);
+ else if (vcdev->is_thinint)
+ virtio_ccw_drop_indicators(vcdev);
+ kfree(indicatorp);
+ kfree(thinint_area);
+}
+
static inline long do_kvm_notify(struct subchannel_id schid,
unsigned long queue_index,
long cookie)
@@ -232,11 +425,13 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev)
{
struct virtqueue *vq, *n;
struct ccw1 *ccw;
+ struct virtio_ccw_device *vcdev = to_vc_device(vdev);
ccw = kzalloc(sizeof(*ccw), GFP_DMA | GFP_KERNEL);
if (!ccw)
return;
+ virtio_ccw_drop_indicator(vcdev, ccw);
list_for_each_entry_safe(vq, n, &vdev->vqs, list)
virtio_ccw_del_vq(vq, ccw);
@@ -326,6 +521,52 @@ out_err:
return ERR_PTR(err);
}
+static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev,
+ struct virtqueue *vqs[], int nvqs,
+ struct ccw1 *ccw)
+{
+ int ret;
+ struct virtio_thinint_area *thinint_area = NULL;
+ struct airq_info *info;
+
+ thinint_area = kzalloc(sizeof(*thinint_area), GFP_DMA | GFP_KERNEL);
+ if (!thinint_area) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ /* Try to get an indicator. */
+ thinint_area->indicator = get_airq_indicator(vqs, nvqs,
+ &thinint_area->bit_nr,
+ &vcdev->airq_info);
+ if (!thinint_area->indicator) {
+ ret = -ENOSPC;
+ goto out;
+ }
+ info = vcdev->airq_info;
+ thinint_area->summary_indicator =
+ (unsigned long) &info->summary_indicator;
+ thinint_area->isc = VIRTIO_AIRQ_ISC;
+ ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
+ ccw->flags = CCW_FLAG_SLI;
+ ccw->count = sizeof(*thinint_area);
+ ccw->cda = (__u32)(unsigned long)thinint_area;
+ ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND_ADAPTER);
+ if (ret) {
+ dev_warn(&vcdev->cdev->dev,
+ "enabling adapter interrupts = %d\n", ret);
+ if (ret == -EOPNOTSUPP)
+ /*
+ * The host does not support adapter interrupts
+ * for virtio-ccw, stop trying.
+ */
+ virtio_ccw_use_airq = 0;
+ virtio_ccw_drop_indicators(vcdev);
+ }
+out:
+ kfree(thinint_area);
+ return ret;
+}
+
static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
struct virtqueue *vqs[],
vq_callback_t *callbacks[],
@@ -355,15 +596,23 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
if (!indicatorp)
goto out;
*indicatorp = (unsigned long) &vcdev->indicators;
- /* Register queue indicators with host. */
- vcdev->indicators = 0;
- ccw->cmd_code = CCW_CMD_SET_IND;
- ccw->flags = 0;
- ccw->count = sizeof(vcdev->indicators);
- ccw->cda = (__u32)(unsigned long) indicatorp;
- ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
- if (ret)
- goto out;
+ if (vcdev->is_thinint) {
+ ret = virtio_ccw_register_adapter_ind(vcdev, vqs, nvqs, ccw);
+ if (ret)
+ /* no error, just fall back to legacy interrupts */
+ vcdev->is_thinint = 0;
+ }
+ if (!vcdev->is_thinint) {
+ /* Register queue indicators with host. */
+ vcdev->indicators = 0;
+ ccw->cmd_code = CCW_CMD_SET_IND;
+ ccw->flags = 0;
+ ccw->count = sizeof(vcdev->indicators);
+ ccw->cda = (__u32)(unsigned long) indicatorp;
+ ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
+ if (ret)
+ goto out;
+ }
/* Register indicators2 with host for config changes */
*indicatorp = (unsigned long) &vcdev->indicators2;
vcdev->indicators2 = 0;
@@ -663,6 +912,7 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev,
case VIRTIO_CCW_DOING_SET_CONF_IND:
case VIRTIO_CCW_DOING_RESET:
case VIRTIO_CCW_DOING_READ_VQ_CONF:
+ case VIRTIO_CCW_DOING_SET_IND_ADAPTER:
vcdev->curr_io &= ~activity;
wake_up(&vcdev->wait_q);
break;
@@ -778,6 +1028,8 @@ static int virtio_ccw_online(struct ccw_device *cdev)
goto out_free;
}
+ vcdev->is_thinint = virtio_ccw_use_airq; /* at least try */
+
vcdev->vdev.dev.parent = &cdev->dev;
vcdev->vdev.dev.release = virtio_ccw_release_dev;
vcdev->vdev.config = &virtio_ccw_config_ops;
@@ -935,6 +1187,10 @@ module_init(virtio_ccw_init);
static void __exit virtio_ccw_exit(void)
{
+ int i;
+
ccw_driver_unregister(&virtio_ccw_driver);
+ for (i = 0; i < MAX_AIRQ_AREAS; i++)
+ destroy_airq_info(airq_areas[i]);
}
module_exit(virtio_ccw_exit);
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 3/7] KVM: eventfd: Fix lock order inversion.
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck
When registering a new irqfd, we call its ->poll method to collect any
event that might have previously been pending so that we can trigger it.
This is done under the kvm->irqfds.lock, which means the eventfd's ctx
lock is taken under it.
However, if we get a POLLHUP in irqfd_wakeup, we will be called with the
ctx lock held before getting the irqfds.lock to deactivate the irqfd,
causing lockdep to complain.
Calling the ->poll method does not really need the irqfds.lock, so let's
just move it after we've given up the irqfds.lock in kvm_irqfd_assign().
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
virt/kvm/eventfd.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index abe4d60..29c2a04 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -391,19 +391,19 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args)
lockdep_is_held(&kvm->irqfds.lock));
irqfd_update(kvm, irqfd, irq_rt);
- events = f.file->f_op->poll(f.file, &irqfd->pt);
-
list_add_tail(&irqfd->list, &kvm->irqfds.items);
+ spin_unlock_irq(&kvm->irqfds.lock);
+
/*
* Check if there was an event already pending on the eventfd
* before we registered, and trigger it as if we didn't miss it.
*/
+ events = f.file->f_op->poll(f.file, &irqfd->pt);
+
if (events & POLLIN)
schedule_work(&irqfd->inject);
- spin_unlock_irq(&kvm->irqfds.lock);
-
/*
* do not drop the file until the irqfd is fully initialized, otherwise
* we might race against the POLLHUP
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 3/7] KVM: eventfd: Fix lock order inversion.
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
When registering a new irqfd, we call its ->poll method to collect any
event that might have previously been pending so that we can trigger it.
This is done under the kvm->irqfds.lock, which means the eventfd's ctx
lock is taken under it.
However, if we get a POLLHUP in irqfd_wakeup, we will be called with the
ctx lock held before getting the irqfds.lock to deactivate the irqfd,
causing lockdep to complain.
Calling the ->poll method does not really need the irqfds.lock, so let's
just move it after we've given up the irqfds.lock in kvm_irqfd_assign().
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
virt/kvm/eventfd.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index abe4d60..29c2a04 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -391,19 +391,19 @@ kvm_irqfd_assign(struct kvm *kvm, struct kvm_irqfd *args)
lockdep_is_held(&kvm->irqfds.lock));
irqfd_update(kvm, irqfd, irq_rt);
- events = f.file->f_op->poll(f.file, &irqfd->pt);
-
list_add_tail(&irqfd->list, &kvm->irqfds.items);
+ spin_unlock_irq(&kvm->irqfds.lock);
+
/*
* Check if there was an event already pending on the eventfd
* before we registered, and trigger it as if we didn't miss it.
*/
+ events = f.file->f_op->poll(f.file, &irqfd->pt);
+
if (events & POLLIN)
schedule_work(&irqfd->inject);
- spin_unlock_irq(&kvm->irqfds.lock);
-
/*
* do not drop the file until the irqfd is fully initialized, otherwise
* we might race against the POLLHUP
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 4/7] KVM: Add per-vm capability enablement.
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck
Allow KVM_ENABLE_CAP to act on a vm as well as on a vcpu. This makes more
sense when the caller wants to enable a vm-related capability.
s390 will be the first user; wire it up.
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
Documentation/virtual/kvm/api.txt | 6 ++++--
arch/s390/kvm/kvm-s390.c | 24 ++++++++++++++++++++++++
include/uapi/linux/kvm.h | 5 +++++
3 files changed, 33 insertions(+), 2 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 6cd63a9..7c6a578 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -918,9 +918,9 @@ documentation when it pops into existence).
4.37 KVM_ENABLE_CAP
-Capability: KVM_CAP_ENABLE_CAP
+Capability: KVM_CAP_ENABLE_CAP, KVM_CAP_ENABLE_CAP_VM
Architectures: ppc, s390
-Type: vcpu ioctl
+Type: vcpu ioctl, vm ioctl (with KVM_CAP_ENABLE_CAP_VM)
Parameters: struct kvm_enable_cap (in)
Returns: 0 on success; -1 on error
@@ -951,6 +951,8 @@ function properly, this is the place to put them.
__u8 pad[64];
};
+The vcpu ioctl should be used for vcpu-specific capabilities, the vm ioctl
+for vm-wide capabilities.
4.38 KVM_GET_MP_STATE
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index a5da2cc..aef92b1 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -159,6 +159,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_S390_CSS_SUPPORT:
case KVM_CAP_IOEVENTFD:
case KVM_CAP_DEVICE_CTRL:
+ case KVM_CAP_ENABLE_CAP_VM:
r = 1;
break;
case KVM_CAP_NR_VCPUS:
@@ -187,6 +188,21 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,
return 0;
}
+static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+ int r;
+
+ if (cap->flags)
+ return -EINVAL;
+
+ switch (cap->cap) {
+ default:
+ r = -EINVAL;
+ break;
+ }
+ return r;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -204,6 +220,14 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = kvm_s390_inject_vm(kvm, &s390int);
break;
}
+ case KVM_ENABLE_CAP: {
+ struct kvm_enable_cap cap;
+ r = -EFAULT;
+ if (copy_from_user(&cap, argp, sizeof(cap)))
+ break;
+ r = kvm_vm_ioctl_enable_cap(kvm, &cap);
+ break;
+ }
default:
r = -ENOTTY;
}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 7d76401..48bc6db 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -740,6 +740,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_SPAPR_MULTITCE 94
#define KVM_CAP_EXT_EMUL_CPUID 95
#define KVM_CAP_HYPERV_TIME 96
+#define KVM_CAP_ENABLE_CAP_VM 97
#ifdef KVM_CAP_IRQ_ROUTING
@@ -1075,6 +1076,10 @@ struct kvm_s390_ucas_mapping {
/* Available with KVM_CAP_DEBUGREGS */
#define KVM_GET_DEBUGREGS _IOR(KVMIO, 0xa1, struct kvm_debugregs)
#define KVM_SET_DEBUGREGS _IOW(KVMIO, 0xa2, struct kvm_debugregs)
+/*
+ * vcpu version available with KVM_ENABLE_CAP
+ * vm version available with KVM_CAP_ENABLE_CAP_VM
+ */
#define KVM_ENABLE_CAP _IOW(KVMIO, 0xa3, struct kvm_enable_cap)
/* Available with KVM_CAP_XSAVE */
#define KVM_GET_XSAVE _IOR(KVMIO, 0xa4, struct kvm_xsave)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 4/7] KVM: Add per-vm capability enablement.
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Allow KVM_ENABLE_CAP to act on a vm as well as on a vcpu. This makes more
sense when the caller wants to enable a vm-related capability.
s390 will be the first user; wire it up.
Reviewed-by: Thomas Huth <thuth@linux.vnet.ibm.com>
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
Documentation/virtual/kvm/api.txt | 6 ++++--
arch/s390/kvm/kvm-s390.c | 24 ++++++++++++++++++++++++
include/uapi/linux/kvm.h | 5 +++++
3 files changed, 33 insertions(+), 2 deletions(-)
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 6cd63a9..7c6a578 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -918,9 +918,9 @@ documentation when it pops into existence).
4.37 KVM_ENABLE_CAP
-Capability: KVM_CAP_ENABLE_CAP
+Capability: KVM_CAP_ENABLE_CAP, KVM_CAP_ENABLE_CAP_VM
Architectures: ppc, s390
-Type: vcpu ioctl
+Type: vcpu ioctl, vm ioctl (with KVM_CAP_ENABLE_CAP_VM)
Parameters: struct kvm_enable_cap (in)
Returns: 0 on success; -1 on error
@@ -951,6 +951,8 @@ function properly, this is the place to put them.
__u8 pad[64];
};
+The vcpu ioctl should be used for vcpu-specific capabilities, the vm ioctl
+for vm-wide capabilities.
4.38 KVM_GET_MP_STATE
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index a5da2cc..aef92b1 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -159,6 +159,7 @@ int kvm_dev_ioctl_check_extension(long ext)
case KVM_CAP_S390_CSS_SUPPORT:
case KVM_CAP_IOEVENTFD:
case KVM_CAP_DEVICE_CTRL:
+ case KVM_CAP_ENABLE_CAP_VM:
r = 1;
break;
case KVM_CAP_NR_VCPUS:
@@ -187,6 +188,21 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm,
return 0;
}
+static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
+{
+ int r;
+
+ if (cap->flags)
+ return -EINVAL;
+
+ switch (cap->cap) {
+ default:
+ r = -EINVAL;
+ break;
+ }
+ return r;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
@@ -204,6 +220,14 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = kvm_s390_inject_vm(kvm, &s390int);
break;
}
+ case KVM_ENABLE_CAP: {
+ struct kvm_enable_cap cap;
+ r = -EFAULT;
+ if (copy_from_user(&cap, argp, sizeof(cap)))
+ break;
+ r = kvm_vm_ioctl_enable_cap(kvm, &cap);
+ break;
+ }
default:
r = -ENOTTY;
}
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 7d76401..48bc6db 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -740,6 +740,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_SPAPR_MULTITCE 94
#define KVM_CAP_EXT_EMUL_CPUID 95
#define KVM_CAP_HYPERV_TIME 96
+#define KVM_CAP_ENABLE_CAP_VM 97
#ifdef KVM_CAP_IRQ_ROUTING
@@ -1075,6 +1076,10 @@ struct kvm_s390_ucas_mapping {
/* Available with KVM_CAP_DEBUGREGS */
#define KVM_GET_DEBUGREGS _IOR(KVMIO, 0xa1, struct kvm_debugregs)
#define KVM_SET_DEBUGREGS _IOW(KVMIO, 0xa2, struct kvm_debugregs)
+/*
+ * vcpu version available with KVM_ENABLE_CAP
+ * vm version available with KVM_CAP_ENABLE_CAP_VM
+ */
#define KVM_ENABLE_CAP _IOW(KVMIO, 0xa3, struct kvm_enable_cap)
/* Available with KVM_CAP_XSAVE */
#define KVM_GET_XSAVE _IOR(KVMIO, 0xa4, struct kvm_xsave)
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 5/7] KVM: s390: adapter interrupt sources
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Add a new interface to register/deregister sources of adapter interrupts
identified by an unique id via the flic. Adapters may also be maskable
and carry a list of pinned pages.
These adapters will be used by irq routing later.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
Documentation/virtual/kvm/devices/s390_flic.txt | 43 ++++++
arch/s390/include/asm/kvm_host.h | 20 +++
arch/s390/include/uapi/asm/kvm.h | 22 ++++
arch/s390/kvm/interrupt.c | 161 ++++++++++++++++++++++-
arch/s390/kvm/kvm-s390.c | 1 +
arch/s390/kvm/kvm-s390.h | 2 +
6 files changed, 248 insertions(+), 1 deletion(-)
diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virtual/kvm/devices/s390_flic.txt
index 410fa67..db16111 100644
--- a/Documentation/virtual/kvm/devices/s390_flic.txt
+++ b/Documentation/virtual/kvm/devices/s390_flic.txt
@@ -12,6 +12,7 @@ FLIC provides support to
- inspect currently pending interrupts (KVM_FLIC_GET_ALL_IRQS)
- purge all pending floating interrupts (KVM_DEV_FLIC_CLEAR_IRQS)
- enable/disable for the guest transparent async page faults
+- register and modify adapter interrupt sources (KVM_DEV_FLIC_ADAPTER_*)
Groups:
KVM_DEV_FLIC_ENQUEUE
@@ -44,3 +45,45 @@ Groups:
Disables async page faults for the guest and waits until already pending
async page faults are done. This is necessary to trigger a completion interrupt
for every init interrupt before migrating the interrupt list.
+
+ KVM_DEV_FLIC_ADAPTER_REGISTER
+ Register an I/O adapter interrupt source. Takes a kvm_s390_io_adapter
+ describing the adapter to register:
+
+struct kvm_s390_io_adapter {
+ __u32 id;
+ __u8 isc;
+ __u8 maskable;
+ __u8 swap;
+ __u8 pad;
+};
+
+ id contains the unique id for the adapter, isc the I/O interruption subclass
+ to use, maskable whether this adapter may be masked (interrupts turned off)
+ and swap whether the indicators need to be byte swapped.
+
+
+ KVM_DEV_FLIC_ADAPTER_MODIFY
+ Modifies attributes of an existing I/O adapter interrupt source. Takes
+ a kvm_s390_io_adapter_req specifiying the adapter and the operation:
+
+struct kvm_s390_io_adapter_req {
+ __u32 id;
+ __u8 type;
+ __u8 mask;
+ __u16 pad0;
+ __u64 addr;
+};
+
+ id specifies the adapter and type the operation. The supported operations
+ are:
+
+ KVM_S390_IO_ADAPTER_MASK
+ mask or unmask the adapter, as specified in mask
+
+ KVM_S390_IO_ADAPTER_MAP
+ pin a userspace page for the address provided in addr and add it to the
+ list of mappings
+
+ KVM_S390_IO_ADAPTER_UNMAP
+ release a userspace page as specified in addr from the list of mappings
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 2c69ba2..83aa94b 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -19,6 +19,7 @@
#include <linux/kvm.h>
#include <asm/debug.h>
#include <asm/cpu.h>
+#include <asm/isc.h>
#define KVM_MAX_VCPUS 64
#define KVM_USER_MEM_SLOTS 32
@@ -244,6 +245,24 @@ struct kvm_vm_stat {
struct kvm_arch_memory_slot {
};
+struct s390_map_info {
+ struct list_head list;
+ __u64 addr;
+ struct page *page;
+};
+
+struct s390_io_adapter {
+ unsigned int id;
+ int isc;
+ bool maskable;
+ bool masked;
+ bool swap;
+ struct rw_semaphore maps_lock;
+ struct list_head maps;
+};
+
+#define MAX_S390_IO_ADAPTERS (MAX_ISC + 1) * 8
+
struct kvm_arch{
struct sca_block *sca;
debug_info_t *dbf;
@@ -251,6 +270,7 @@ struct kvm_arch{
struct kvm_device *flic;
struct gmap *gmap;
int css_support;
+ struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
};
#define KVM_HVA_ERR_BAD (-1UL)
diff --git a/arch/s390/include/uapi/asm/kvm.h b/arch/s390/include/uapi/asm/kvm.h
index cb4c1eb8..a0761e2 100644
--- a/arch/s390/include/uapi/asm/kvm.h
+++ b/arch/s390/include/uapi/asm/kvm.h
@@ -22,6 +22,8 @@
#define KVM_DEV_FLIC_CLEAR_IRQS 3
#define KVM_DEV_FLIC_APF_ENABLE 4
#define KVM_DEV_FLIC_APF_DISABLE_WAIT 5
+#define KVM_DEV_FLIC_ADAPTER_REGISTER 6
+#define KVM_DEV_FLIC_ADAPTER_MODIFY 7
/*
* We can have up to 4*64k pending subchannels + 8 adapter interrupts,
* as well as up to ASYNC_PF_PER_VCPU*KVM_MAX_VCPUS pfault done interrupts.
@@ -32,6 +34,26 @@
#define KVM_S390_MAX_FLOAT_IRQS 266250
#define KVM_S390_FLIC_MAX_BUFFER 0x2000000
+struct kvm_s390_io_adapter {
+ __u32 id;
+ __u8 isc;
+ __u8 maskable;
+ __u8 swap;
+ __u8 pad;
+};
+
+#define KVM_S390_IO_ADAPTER_MASK 1
+#define KVM_S390_IO_ADAPTER_MAP 2
+#define KVM_S390_IO_ADAPTER_UNMAP 3
+
+struct kvm_s390_io_adapter_req {
+ __u32 id;
+ __u8 type;
+ __u8 mask;
+ __u16 pad0;
+ __u64 addr;
+};
+
/* for KVM_GET_REGS and KVM_SET_REGS */
struct kvm_regs {
/* general purpose regs for s390 */
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 1848080..bfe59be 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -1,7 +1,7 @@
/*
* handling kvm guest interrupts
*
- * Copyright IBM Corp. 2008
+ * Copyright IBM Corp. 2008,2014
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License (version 2 only)
@@ -1051,6 +1051,159 @@ static int enqueue_floating_irq(struct kvm_device *dev,
return r;
}
+static struct s390_io_adapter *get_io_adapter(struct kvm *kvm, unsigned int id)
+{
+ if (id >= MAX_S390_IO_ADAPTERS)
+ return NULL;
+ return kvm->arch.adapters[id];
+}
+
+static int register_io_adapter(struct kvm_device *dev,
+ struct kvm_device_attr *attr)
+{
+ struct s390_io_adapter *adapter;
+ struct kvm_s390_io_adapter adapter_info;
+
+ if (copy_from_user(&adapter_info,
+ (void __user *)attr->addr, sizeof(adapter_info)))
+ return -EFAULT;
+
+ if ((adapter_info.id >= MAX_S390_IO_ADAPTERS) ||
+ (dev->kvm->arch.adapters[adapter_info.id] != NULL))
+ return -EINVAL;
+
+ adapter = kzalloc(sizeof(*adapter), GFP_KERNEL);
+ if (!adapter)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&adapter->maps);
+ init_rwsem(&adapter->maps_lock);
+ adapter->id = adapter_info.id;
+ adapter->isc = adapter_info.isc;
+ adapter->maskable = adapter_info.maskable;
+ adapter->masked = false;
+ adapter->swap = adapter_info.swap;
+ dev->kvm->arch.adapters[adapter->id] = adapter;
+
+ return 0;
+}
+
+int kvm_s390_mask_adapter(struct kvm *kvm, unsigned int id, bool masked)
+{
+ int ret;
+ struct s390_io_adapter *adapter = get_io_adapter(kvm, id);
+
+ if (!adapter || !adapter->maskable)
+ return -EINVAL;
+ ret = adapter->masked;
+ adapter->masked = masked;
+ return ret;
+}
+
+static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr)
+{
+ struct s390_io_adapter *adapter = get_io_adapter(kvm, id);
+ struct s390_map_info *map;
+ int ret;
+
+ if (!adapter || !addr)
+ return -EINVAL;
+
+ map = kzalloc(sizeof(*map), GFP_KERNEL);
+ if (!map) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ INIT_LIST_HEAD(&map->list);
+ map->addr = addr;
+ ret = get_user_pages_fast(addr, 1, 1, &map->page);
+ if (ret < 0)
+ goto out;
+ BUG_ON(ret != 1);
+ down_write(&adapter->maps_lock);
+ list_add_tail(&map->list, &adapter->maps);
+ up_write(&adapter->maps_lock);
+ ret = 0;
+out:
+ if (ret)
+ kfree(map);
+ return ret;
+}
+
+static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr)
+{
+ struct s390_io_adapter *adapter = get_io_adapter(kvm, id);
+ struct s390_map_info *map, *tmp;
+ int found = 0;
+
+ if (!adapter || !addr)
+ return -EINVAL;
+
+ down_write(&adapter->maps_lock);
+ list_for_each_entry_safe(map, tmp, &adapter->maps, list) {
+ if (map->addr == addr) {
+ found = 1;
+ list_del(&map->list);
+ put_page(map->page);
+ kfree(map);
+ break;
+ }
+ }
+ up_write(&adapter->maps_lock);
+
+ return found ? 0 : -EINVAL;
+}
+
+void kvm_s390_destroy_adapters(struct kvm *kvm)
+{
+ int i;
+ struct s390_map_info *map, *tmp;
+
+ for (i = 0; i < MAX_S390_IO_ADAPTERS; i++) {
+ if (!kvm->arch.adapters[i])
+ continue;
+ list_for_each_entry_safe(map, tmp,
+ &kvm->arch.adapters[i]->maps, list) {
+ list_del(&map->list);
+ put_page(map->page);
+ kfree(map);
+ }
+ kfree(kvm->arch.adapters[i]);
+ }
+}
+
+static int modify_io_adapter(struct kvm_device *dev,
+ struct kvm_device_attr *attr)
+{
+ struct kvm_s390_io_adapter_req req;
+ struct s390_io_adapter *adapter;
+ int ret;
+
+ if (copy_from_user(&req, (void __user *)attr->addr, sizeof(req)))
+ return -EFAULT;
+
+ adapter = get_io_adapter(dev->kvm, req.id);
+ if (!adapter)
+ return -EINVAL;
+ switch (req.type) {
+ case KVM_S390_IO_ADAPTER_MASK:
+ ret = kvm_s390_mask_adapter(dev->kvm, req.id, req.mask);
+ if (ret > 0)
+ ret = 0;
+ break;
+ case KVM_S390_IO_ADAPTER_MAP:
+ ret = kvm_s390_adapter_map(dev->kvm, req.id, req.addr);
+ break;
+ case KVM_S390_IO_ADAPTER_UNMAP:
+ ret = kvm_s390_adapter_unmap(dev->kvm, req.id, req.addr);
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static int flic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
int r = 0;
@@ -1079,6 +1232,12 @@ static int flic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
kvm_for_each_vcpu(i, vcpu, dev->kvm)
kvm_clear_async_pf_completion_queue(vcpu);
break;
+ case KVM_DEV_FLIC_ADAPTER_REGISTER:
+ r = register_io_adapter(dev, attr);
+ break;
+ case KVM_DEV_FLIC_ADAPTER_MODIFY:
+ r = modify_io_adapter(dev, attr);
+ break;
default:
r = -EINVAL;
}
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index aef92b1..f0ff213 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -343,6 +343,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
debug_unregister(kvm->arch.dbf);
if (!kvm_is_ucontrol(kvm))
gmap_free(kvm->arch.gmap);
+ kvm_s390_destroy_adapters(kvm);
}
/* Section: vcpu related */
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index ed4750a..5502cc95 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -136,6 +136,7 @@ int __must_check kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu,
int __must_check kvm_s390_inject_program_int(struct kvm_vcpu *vcpu, u16 code);
struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,
u64 cr6, u64 schid);
+int kvm_s390_mask_adapter(struct kvm *kvm, unsigned int id, bool masked);
/* implemented in priv.c */
int kvm_s390_handle_b2(struct kvm_vcpu *vcpu);
@@ -162,5 +163,6 @@ int kvm_s390_handle_diag(struct kvm_vcpu *vcpu);
/* implemented in interrupt.c */
int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu);
int psw_extint_disabled(struct kvm_vcpu *vcpu);
+void kvm_s390_destroy_adapters(struct kvm *kvm);
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 5/7] KVM: s390: adapter interrupt sources
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Add a new interface to register/deregister sources of adapter interrupts
identified by an unique id via the flic. Adapters may also be maskable
and carry a list of pinned pages.
These adapters will be used by irq routing later.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
Documentation/virtual/kvm/devices/s390_flic.txt | 43 ++++++
arch/s390/include/asm/kvm_host.h | 20 +++
arch/s390/include/uapi/asm/kvm.h | 22 ++++
arch/s390/kvm/interrupt.c | 161 ++++++++++++++++++++++-
arch/s390/kvm/kvm-s390.c | 1 +
arch/s390/kvm/kvm-s390.h | 2 +
6 files changed, 248 insertions(+), 1 deletion(-)
diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virtual/kvm/devices/s390_flic.txt
index 410fa67..db16111 100644
--- a/Documentation/virtual/kvm/devices/s390_flic.txt
+++ b/Documentation/virtual/kvm/devices/s390_flic.txt
@@ -12,6 +12,7 @@ FLIC provides support to
- inspect currently pending interrupts (KVM_FLIC_GET_ALL_IRQS)
- purge all pending floating interrupts (KVM_DEV_FLIC_CLEAR_IRQS)
- enable/disable for the guest transparent async page faults
+- register and modify adapter interrupt sources (KVM_DEV_FLIC_ADAPTER_*)
Groups:
KVM_DEV_FLIC_ENQUEUE
@@ -44,3 +45,45 @@ Groups:
Disables async page faults for the guest and waits until already pending
async page faults are done. This is necessary to trigger a completion interrupt
for every init interrupt before migrating the interrupt list.
+
+ KVM_DEV_FLIC_ADAPTER_REGISTER
+ Register an I/O adapter interrupt source. Takes a kvm_s390_io_adapter
+ describing the adapter to register:
+
+struct kvm_s390_io_adapter {
+ __u32 id;
+ __u8 isc;
+ __u8 maskable;
+ __u8 swap;
+ __u8 pad;
+};
+
+ id contains the unique id for the adapter, isc the I/O interruption subclass
+ to use, maskable whether this adapter may be masked (interrupts turned off)
+ and swap whether the indicators need to be byte swapped.
+
+
+ KVM_DEV_FLIC_ADAPTER_MODIFY
+ Modifies attributes of an existing I/O adapter interrupt source. Takes
+ a kvm_s390_io_adapter_req specifiying the adapter and the operation:
+
+struct kvm_s390_io_adapter_req {
+ __u32 id;
+ __u8 type;
+ __u8 mask;
+ __u16 pad0;
+ __u64 addr;
+};
+
+ id specifies the adapter and type the operation. The supported operations
+ are:
+
+ KVM_S390_IO_ADAPTER_MASK
+ mask or unmask the adapter, as specified in mask
+
+ KVM_S390_IO_ADAPTER_MAP
+ pin a userspace page for the address provided in addr and add it to the
+ list of mappings
+
+ KVM_S390_IO_ADAPTER_UNMAP
+ release a userspace page as specified in addr from the list of mappings
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 2c69ba2..83aa94b 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -19,6 +19,7 @@
#include <linux/kvm.h>
#include <asm/debug.h>
#include <asm/cpu.h>
+#include <asm/isc.h>
#define KVM_MAX_VCPUS 64
#define KVM_USER_MEM_SLOTS 32
@@ -244,6 +245,24 @@ struct kvm_vm_stat {
struct kvm_arch_memory_slot {
};
+struct s390_map_info {
+ struct list_head list;
+ __u64 addr;
+ struct page *page;
+};
+
+struct s390_io_adapter {
+ unsigned int id;
+ int isc;
+ bool maskable;
+ bool masked;
+ bool swap;
+ struct rw_semaphore maps_lock;
+ struct list_head maps;
+};
+
+#define MAX_S390_IO_ADAPTERS (MAX_ISC + 1) * 8
+
struct kvm_arch{
struct sca_block *sca;
debug_info_t *dbf;
@@ -251,6 +270,7 @@ struct kvm_arch{
struct kvm_device *flic;
struct gmap *gmap;
int css_support;
+ struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
};
#define KVM_HVA_ERR_BAD (-1UL)
diff --git a/arch/s390/include/uapi/asm/kvm.h b/arch/s390/include/uapi/asm/kvm.h
index cb4c1eb8..a0761e2 100644
--- a/arch/s390/include/uapi/asm/kvm.h
+++ b/arch/s390/include/uapi/asm/kvm.h
@@ -22,6 +22,8 @@
#define KVM_DEV_FLIC_CLEAR_IRQS 3
#define KVM_DEV_FLIC_APF_ENABLE 4
#define KVM_DEV_FLIC_APF_DISABLE_WAIT 5
+#define KVM_DEV_FLIC_ADAPTER_REGISTER 6
+#define KVM_DEV_FLIC_ADAPTER_MODIFY 7
/*
* We can have up to 4*64k pending subchannels + 8 adapter interrupts,
* as well as up to ASYNC_PF_PER_VCPU*KVM_MAX_VCPUS pfault done interrupts.
@@ -32,6 +34,26 @@
#define KVM_S390_MAX_FLOAT_IRQS 266250
#define KVM_S390_FLIC_MAX_BUFFER 0x2000000
+struct kvm_s390_io_adapter {
+ __u32 id;
+ __u8 isc;
+ __u8 maskable;
+ __u8 swap;
+ __u8 pad;
+};
+
+#define KVM_S390_IO_ADAPTER_MASK 1
+#define KVM_S390_IO_ADAPTER_MAP 2
+#define KVM_S390_IO_ADAPTER_UNMAP 3
+
+struct kvm_s390_io_adapter_req {
+ __u32 id;
+ __u8 type;
+ __u8 mask;
+ __u16 pad0;
+ __u64 addr;
+};
+
/* for KVM_GET_REGS and KVM_SET_REGS */
struct kvm_regs {
/* general purpose regs for s390 */
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index 1848080..bfe59be 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -1,7 +1,7 @@
/*
* handling kvm guest interrupts
*
- * Copyright IBM Corp. 2008
+ * Copyright IBM Corp. 2008,2014
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License (version 2 only)
@@ -1051,6 +1051,159 @@ static int enqueue_floating_irq(struct kvm_device *dev,
return r;
}
+static struct s390_io_adapter *get_io_adapter(struct kvm *kvm, unsigned int id)
+{
+ if (id >= MAX_S390_IO_ADAPTERS)
+ return NULL;
+ return kvm->arch.adapters[id];
+}
+
+static int register_io_adapter(struct kvm_device *dev,
+ struct kvm_device_attr *attr)
+{
+ struct s390_io_adapter *adapter;
+ struct kvm_s390_io_adapter adapter_info;
+
+ if (copy_from_user(&adapter_info,
+ (void __user *)attr->addr, sizeof(adapter_info)))
+ return -EFAULT;
+
+ if ((adapter_info.id >= MAX_S390_IO_ADAPTERS) ||
+ (dev->kvm->arch.adapters[adapter_info.id] != NULL))
+ return -EINVAL;
+
+ adapter = kzalloc(sizeof(*adapter), GFP_KERNEL);
+ if (!adapter)
+ return -ENOMEM;
+
+ INIT_LIST_HEAD(&adapter->maps);
+ init_rwsem(&adapter->maps_lock);
+ adapter->id = adapter_info.id;
+ adapter->isc = adapter_info.isc;
+ adapter->maskable = adapter_info.maskable;
+ adapter->masked = false;
+ adapter->swap = adapter_info.swap;
+ dev->kvm->arch.adapters[adapter->id] = adapter;
+
+ return 0;
+}
+
+int kvm_s390_mask_adapter(struct kvm *kvm, unsigned int id, bool masked)
+{
+ int ret;
+ struct s390_io_adapter *adapter = get_io_adapter(kvm, id);
+
+ if (!adapter || !adapter->maskable)
+ return -EINVAL;
+ ret = adapter->masked;
+ adapter->masked = masked;
+ return ret;
+}
+
+static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr)
+{
+ struct s390_io_adapter *adapter = get_io_adapter(kvm, id);
+ struct s390_map_info *map;
+ int ret;
+
+ if (!adapter || !addr)
+ return -EINVAL;
+
+ map = kzalloc(sizeof(*map), GFP_KERNEL);
+ if (!map) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ INIT_LIST_HEAD(&map->list);
+ map->addr = addr;
+ ret = get_user_pages_fast(addr, 1, 1, &map->page);
+ if (ret < 0)
+ goto out;
+ BUG_ON(ret != 1);
+ down_write(&adapter->maps_lock);
+ list_add_tail(&map->list, &adapter->maps);
+ up_write(&adapter->maps_lock);
+ ret = 0;
+out:
+ if (ret)
+ kfree(map);
+ return ret;
+}
+
+static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr)
+{
+ struct s390_io_adapter *adapter = get_io_adapter(kvm, id);
+ struct s390_map_info *map, *tmp;
+ int found = 0;
+
+ if (!adapter || !addr)
+ return -EINVAL;
+
+ down_write(&adapter->maps_lock);
+ list_for_each_entry_safe(map, tmp, &adapter->maps, list) {
+ if (map->addr == addr) {
+ found = 1;
+ list_del(&map->list);
+ put_page(map->page);
+ kfree(map);
+ break;
+ }
+ }
+ up_write(&adapter->maps_lock);
+
+ return found ? 0 : -EINVAL;
+}
+
+void kvm_s390_destroy_adapters(struct kvm *kvm)
+{
+ int i;
+ struct s390_map_info *map, *tmp;
+
+ for (i = 0; i < MAX_S390_IO_ADAPTERS; i++) {
+ if (!kvm->arch.adapters[i])
+ continue;
+ list_for_each_entry_safe(map, tmp,
+ &kvm->arch.adapters[i]->maps, list) {
+ list_del(&map->list);
+ put_page(map->page);
+ kfree(map);
+ }
+ kfree(kvm->arch.adapters[i]);
+ }
+}
+
+static int modify_io_adapter(struct kvm_device *dev,
+ struct kvm_device_attr *attr)
+{
+ struct kvm_s390_io_adapter_req req;
+ struct s390_io_adapter *adapter;
+ int ret;
+
+ if (copy_from_user(&req, (void __user *)attr->addr, sizeof(req)))
+ return -EFAULT;
+
+ adapter = get_io_adapter(dev->kvm, req.id);
+ if (!adapter)
+ return -EINVAL;
+ switch (req.type) {
+ case KVM_S390_IO_ADAPTER_MASK:
+ ret = kvm_s390_mask_adapter(dev->kvm, req.id, req.mask);
+ if (ret > 0)
+ ret = 0;
+ break;
+ case KVM_S390_IO_ADAPTER_MAP:
+ ret = kvm_s390_adapter_map(dev->kvm, req.id, req.addr);
+ break;
+ case KVM_S390_IO_ADAPTER_UNMAP:
+ ret = kvm_s390_adapter_unmap(dev->kvm, req.id, req.addr);
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static int flic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
{
int r = 0;
@@ -1079,6 +1232,12 @@ static int flic_set_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
kvm_for_each_vcpu(i, vcpu, dev->kvm)
kvm_clear_async_pf_completion_queue(vcpu);
break;
+ case KVM_DEV_FLIC_ADAPTER_REGISTER:
+ r = register_io_adapter(dev, attr);
+ break;
+ case KVM_DEV_FLIC_ADAPTER_MODIFY:
+ r = modify_io_adapter(dev, attr);
+ break;
default:
r = -EINVAL;
}
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index aef92b1..f0ff213 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -343,6 +343,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
debug_unregister(kvm->arch.dbf);
if (!kvm_is_ucontrol(kvm))
gmap_free(kvm->arch.gmap);
+ kvm_s390_destroy_adapters(kvm);
}
/* Section: vcpu related */
diff --git a/arch/s390/kvm/kvm-s390.h b/arch/s390/kvm/kvm-s390.h
index ed4750a..5502cc95 100644
--- a/arch/s390/kvm/kvm-s390.h
+++ b/arch/s390/kvm/kvm-s390.h
@@ -136,6 +136,7 @@ int __must_check kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu,
int __must_check kvm_s390_inject_program_int(struct kvm_vcpu *vcpu, u16 code);
struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,
u64 cr6, u64 schid);
+int kvm_s390_mask_adapter(struct kvm *kvm, unsigned int id, bool masked);
/* implemented in priv.c */
int kvm_s390_handle_b2(struct kvm_vcpu *vcpu);
@@ -162,5 +163,6 @@ int kvm_s390_handle_diag(struct kvm_vcpu *vcpu);
/* implemented in interrupt.c */
int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu);
int psw_extint_disabled(struct kvm_vcpu *vcpu);
+void kvm_s390_destroy_adapters(struct kvm *kvm);
#endif
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 6/7] KVM: s390: irq routing for adapter interrupts.
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck
Introduce a new interrupt class for s390 adapter interrupts and enable
irqfds for s390.
This is depending on a new s390 specific vm capability, KVM_CAP_S390_IRQCHIP,
that needs to be enabled by userspace.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
Documentation/virtual/kvm/api.txt | 21 +++-
Documentation/virtual/kvm/devices/s390_flic.txt | 6 +-
arch/s390/include/asm/kvm_host.h | 10 ++
arch/s390/kvm/Kconfig | 2 +
arch/s390/kvm/Makefile | 2 +-
arch/s390/kvm/interrupt.c | 132 ++++++++++++++++++++++-
arch/s390/kvm/irq.h | 22 ++++
arch/s390/kvm/kvm-s390.c | 17 +++
include/linux/kvm_host.h | 9 ++
include/uapi/linux/kvm.h | 11 ++
10 files changed, 222 insertions(+), 10 deletions(-)
create mode 100644 arch/s390/kvm/irq.h
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 7c6a578..f084e29 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -586,8 +586,8 @@ struct kvm_fpu {
4.24 KVM_CREATE_IRQCHIP
-Capability: KVM_CAP_IRQCHIP
-Architectures: x86, ia64, ARM, arm64
+Capability: KVM_CAP_IRQCHIP, KVM_CAP_S390_IRQCHIP (s390)
+Architectures: x86, ia64, ARM, arm64, s390
Type: vm ioctl
Parameters: none
Returns: 0 on success, -1 on error
@@ -596,7 +596,10 @@ Creates an interrupt controller model in the kernel. On x86, creates a virtual
ioapic, a virtual PIC (two PICs, nested), and sets up future vcpus to have a
local APIC. IRQ routing for GSIs 0-15 is set to both PIC and IOAPIC; GSI 16-23
only go to the IOAPIC. On ia64, a IOSAPIC is created. On ARM/arm64, a GIC is
-created.
+created. On s390, a dummy irq routing table is created.
+
+Note that on s390 the KVM_CAP_S390_IRQCHIP vm capability needs to be enabled
+before KVM_CREATE_IRQCHIP can be used.
4.25 KVM_IRQ_LINE
@@ -1322,7 +1325,7 @@ KVM_ASSIGN_DEV_IRQ. Partial deassignment of host or guest IRQ is allowed.
4.52 KVM_SET_GSI_ROUTING
Capability: KVM_CAP_IRQ_ROUTING
-Architectures: x86 ia64
+Architectures: x86 ia64 s390
Type: vm ioctl
Parameters: struct kvm_irq_routing (in)
Returns: 0 on success, -1 on error
@@ -1345,6 +1348,7 @@ struct kvm_irq_routing_entry {
union {
struct kvm_irq_routing_irqchip irqchip;
struct kvm_irq_routing_msi msi;
+ struct kvm_irq_routing_s390_adapter adapter;
__u32 pad[8];
} u;
};
@@ -1352,6 +1356,7 @@ struct kvm_irq_routing_entry {
/* gsi routing entry types */
#define KVM_IRQ_ROUTING_IRQCHIP 1
#define KVM_IRQ_ROUTING_MSI 2
+#define KVM_IRQ_ROUTING_S390_ADAPTER 3
No flags are specified so far, the corresponding field must be set to zero.
@@ -1367,6 +1372,14 @@ struct kvm_irq_routing_msi {
__u32 pad;
};
+struct kvm_irq_routing_s390_adapter {
+ __u64 ind_addr;
+ __u64 summary_addr;
+ __u64 ind_offset;
+ __u32 summary_offset;
+ __u32 adapter_id;
+};
+
4.53 KVM_ASSIGN_SET_MSIX_NR
diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virtual/kvm/devices/s390_flic.txt
index db16111..4ceef53 100644
--- a/Documentation/virtual/kvm/devices/s390_flic.txt
+++ b/Documentation/virtual/kvm/devices/s390_flic.txt
@@ -82,8 +82,10 @@ struct kvm_s390_io_adapter_req {
mask or unmask the adapter, as specified in mask
KVM_S390_IO_ADAPTER_MAP
- pin a userspace page for the address provided in addr and add it to the
+ perform a gmap translation for the guest address provided in addr,
+ pin a userspace page for the translated address and add it to the
list of mappings
KVM_S390_IO_ADAPTER_UNMAP
- release a userspace page as specified in addr from the list of mappings
+ release a userspace page for the translated address specified in addr
+ from the list of mappings
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 83aa94b..3d7bce8 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -24,6 +24,14 @@
#define KVM_MAX_VCPUS 64
#define KVM_USER_MEM_SLOTS 32
+/*
+ * These seem to be used for allocating ->chip in the routing table,
+ * which we don't use. 4096 is an out-of-thin-air value. If we need
+ * to look at ->chip later on, we'll need to revisit this.
+ */
+#define KVM_NR_IRQCHIPS 1
+#define KVM_IRQCHIP_NUM_PINS 4096
+
struct sca_entry {
atomic_t scn;
__u32 reserved;
@@ -247,6 +255,7 @@ struct kvm_arch_memory_slot {
struct s390_map_info {
struct list_head list;
+ __u64 guest_addr;
__u64 addr;
struct page *page;
};
@@ -270,6 +279,7 @@ struct kvm_arch{
struct kvm_device *flic;
struct gmap *gmap;
int css_support;
+ int use_irqchip;
struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
};
diff --git a/arch/s390/kvm/Kconfig b/arch/s390/kvm/Kconfig
index c8bacbc..10d529a 100644
--- a/arch/s390/kvm/Kconfig
+++ b/arch/s390/kvm/Kconfig
@@ -25,6 +25,8 @@ config KVM
select HAVE_KVM_EVENTFD
select KVM_ASYNC_PF
select KVM_ASYNC_PF_SYNC
+ select HAVE_KVM_IRQCHIP
+ select HAVE_KVM_IRQ_ROUTING
---help---
Support hosting paravirtualized guest machines using the SIE
virtualization capability on the mainframe. This should work
diff --git a/arch/s390/kvm/Makefile b/arch/s390/kvm/Makefile
index a47d2c3..d3adb37 100644
--- a/arch/s390/kvm/Makefile
+++ b/arch/s390/kvm/Makefile
@@ -7,7 +7,7 @@
# as published by the Free Software Foundation.
KVM := ../../../virt/kvm
-common-objs = $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/async_pf.o
+common-objs = $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/async_pf.o $(KVM)/irqchip.o
ccflags-y := -Ivirt/kvm -Iarch/s390/kvm
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index bfe59be..e4af3c4 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -13,6 +13,7 @@
#include <linux/interrupt.h>
#include <linux/kvm_host.h>
#include <linux/hrtimer.h>
+#include <linux/mmu_context.h>
#include <linux/signal.h>
#include <linux/slab.h>
#include <asm/asm-offsets.h>
@@ -1115,8 +1116,13 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr)
goto out;
}
INIT_LIST_HEAD(&map->list);
- map->addr = addr;
- ret = get_user_pages_fast(addr, 1, 1, &map->page);
+ map->guest_addr = addr;
+ map->addr = gmap_translate(addr, kvm->arch.gmap);
+ if (map->addr == -EFAULT) {
+ ret = -EFAULT;
+ goto out;
+ }
+ ret = get_user_pages_fast(map->addr, 1, 1, &map->page);
if (ret < 0)
goto out;
BUG_ON(ret != 1);
@@ -1141,7 +1147,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr)
down_write(&adapter->maps_lock);
list_for_each_entry_safe(map, tmp, &adapter->maps, list) {
- if (map->addr == addr) {
+ if (map->guest_addr == addr) {
found = 1;
list_del(&map->list);
put_page(map->page);
@@ -1269,3 +1275,123 @@ struct kvm_device_ops kvm_flic_ops = {
.create = flic_create,
.destroy = flic_destroy,
};
+
+static unsigned long get_ind_bit(__u64 addr, unsigned long bit_nr, bool swap)
+{
+ unsigned long bit;
+
+ bit = bit_nr + (addr % PAGE_SIZE) * 8;
+
+ return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit;
+}
+
+static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter,
+ u64 addr)
+{
+ struct s390_map_info *map;
+
+ if (!adapter)
+ return NULL;
+
+ list_for_each_entry(map, &adapter->maps, list) {
+ if (map->guest_addr == addr)
+ return map;
+ }
+ return NULL;
+}
+
+static int adapter_indicators_set(struct kvm *kvm,
+ struct s390_io_adapter *adapter,
+ struct kvm_s390_adapter_int *adapter_int)
+{
+ unsigned long bit;
+ int summary_set, idx;
+ struct s390_map_info *info;
+ void *map;
+
+ info = get_map_info(adapter, adapter_int->ind_addr);
+ if (!info)
+ return -1;
+ map = page_address(info->page);
+ bit = get_ind_bit(info->addr, adapter_int->ind_offset, adapter->swap);
+ set_bit(bit, map);
+ idx = srcu_read_lock(&kvm->srcu);
+ mark_page_dirty(kvm, info->guest_addr >> PAGE_SHIFT);
+ set_page_dirty_lock(info->page);
+ info = get_map_info(adapter, adapter_int->summary_addr);
+ if (!info) {
+ srcu_read_unlock(&kvm->srcu, idx);
+ return -1;
+ }
+ map = page_address(info->page);
+ bit = get_ind_bit(info->addr, adapter_int->summary_offset,
+ adapter->swap);
+ summary_set = test_and_set_bit(bit, map);
+ mark_page_dirty(kvm, info->guest_addr >> PAGE_SHIFT);
+ set_page_dirty_lock(info->page);
+ srcu_read_unlock(&kvm->srcu, idx);
+ return summary_set ? 0 : 1;
+}
+
+/*
+ * < 0 - not injected due to error
+ * = 0 - coalesced, summary indicator already active
+ * > 0 - injected interrupt
+ */
+static int set_adapter_int(struct kvm_kernel_irq_routing_entry *e,
+ struct kvm *kvm, int irq_source_id, int level,
+ bool line_status)
+{
+ int ret;
+ struct s390_io_adapter *adapter;
+
+ /* We're only interested in the 0->1 transition. */
+ if (!level)
+ return 0;
+ adapter = get_io_adapter(kvm, e->adapter.adapter_id);
+ if (!adapter)
+ return -1;
+ down_read(&adapter->maps_lock);
+ ret = adapter_indicators_set(kvm, adapter, &e->adapter);
+ up_read(&adapter->maps_lock);
+ if ((ret > 0) && !adapter->masked) {
+ struct kvm_s390_interrupt s390int = {
+ .type = KVM_S390_INT_IO(1, 0, 0, 0),
+ .parm = 0,
+ .parm64 = (adapter->isc << 27) | 0x80000000,
+ };
+ ret = kvm_s390_inject_vm(kvm, &s390int);
+ if (ret == 0)
+ ret = 1;
+ }
+ return ret;
+}
+
+int kvm_set_routing_entry(struct kvm_irq_routing_table *rt,
+ struct kvm_kernel_irq_routing_entry *e,
+ const struct kvm_irq_routing_entry *ue)
+{
+ int ret;
+
+ switch (ue->type) {
+ case KVM_IRQ_ROUTING_S390_ADAPTER:
+ e->set = set_adapter_int;
+ e->adapter.summary_addr = ue->u.adapter.summary_addr;
+ e->adapter.ind_addr = ue->u.adapter.ind_addr;
+ e->adapter.summary_offset = ue->u.adapter.summary_offset;
+ e->adapter.ind_offset = ue->u.adapter.ind_offset;
+ e->adapter.adapter_id = ue->u.adapter.adapter_id;
+ ret = 0;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e, struct kvm *kvm,
+ int irq_source_id, int level, bool line_status)
+{
+ return -EINVAL;
+}
diff --git a/arch/s390/kvm/irq.h b/arch/s390/kvm/irq.h
new file mode 100644
index 0000000..d98e415
--- /dev/null
+++ b/arch/s390/kvm/irq.h
@@ -0,0 +1,22 @@
+/*
+ * s390 irqchip routines
+ *
+ * Copyright IBM Corp. 2014
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License (version 2 only)
+ * as published by the Free Software Foundation.
+ *
+ * Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
+ */
+#ifndef __KVM_IRQ_H
+#define __KVM_IRQ_H
+
+#include <linux/kvm_host.h>
+
+static inline int irqchip_in_kernel(struct kvm *kvm)
+{
+ return 1;
+}
+
+#endif
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index f0ff213..216317a 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -196,6 +196,10 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
return -EINVAL;
switch (cap->cap) {
+ case KVM_CAP_S390_IRQCHIP:
+ kvm->arch.use_irqchip = 1;
+ r = 0;
+ break;
default:
r = -EINVAL;
break;
@@ -228,6 +232,18 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = kvm_vm_ioctl_enable_cap(kvm, &cap);
break;
}
+ case KVM_CREATE_IRQCHIP: {
+ struct kvm_irq_routing_entry routing;
+
+ r = -EINVAL;
+ if (kvm->arch.use_irqchip) {
+ /* Set up dummy routing. */
+ memset(&routing, 0, sizeof(routing));
+ kvm_set_irq_routing(kvm, &routing, 0, 0);
+ r = 0;
+ }
+ break;
+ }
default:
r = -ENOTTY;
}
@@ -284,6 +300,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
}
kvm->arch.css_support = 0;
+ kvm->arch.use_irqchip = 0;
return 0;
out_nogmap:
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9816b68..da7510b 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -297,6 +297,14 @@ static inline unsigned long kvm_dirty_bitmap_bytes(struct kvm_memory_slot *memsl
return ALIGN(memslot->npages, BITS_PER_LONG) / 8;
}
+struct kvm_s390_adapter_int {
+ u64 ind_addr;
+ u64 summary_addr;
+ u64 ind_offset;
+ u32 summary_offset;
+ u32 adapter_id;
+};
+
struct kvm_kernel_irq_routing_entry {
u32 gsi;
u32 type;
@@ -309,6 +317,7 @@ struct kvm_kernel_irq_routing_entry {
unsigned pin;
} irqchip;
struct msi_msg msi;
+ struct kvm_s390_adapter_int adapter;
};
struct hlist_node link;
};
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 48bc6db..4a0387b 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -741,6 +741,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_EXT_EMUL_CPUID 95
#define KVM_CAP_HYPERV_TIME 96
#define KVM_CAP_ENABLE_CAP_VM 97
+#define KVM_CAP_S390_IRQCHIP 98
#ifdef KVM_CAP_IRQ_ROUTING
@@ -756,9 +757,18 @@ struct kvm_irq_routing_msi {
__u32 pad;
};
+struct kvm_irq_routing_s390_adapter {
+ __u64 ind_addr;
+ __u64 summary_addr;
+ __u64 ind_offset;
+ __u32 summary_offset;
+ __u32 adapter_id;
+};
+
/* gsi routing entry types */
#define KVM_IRQ_ROUTING_IRQCHIP 1
#define KVM_IRQ_ROUTING_MSI 2
+#define KVM_IRQ_ROUTING_S390_ADAPTER 3
struct kvm_irq_routing_entry {
__u32 gsi;
@@ -768,6 +778,7 @@ struct kvm_irq_routing_entry {
union {
struct kvm_irq_routing_irqchip irqchip;
struct kvm_irq_routing_msi msi;
+ struct kvm_irq_routing_s390_adapter adapter;
__u32 pad[8];
} u;
};
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 6/7] KVM: s390: irq routing for adapter interrupts.
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
Introduce a new interrupt class for s390 adapter interrupts and enable
irqfds for s390.
This is depending on a new s390 specific vm capability, KVM_CAP_S390_IRQCHIP,
that needs to be enabled by userspace.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
Documentation/virtual/kvm/api.txt | 21 +++-
Documentation/virtual/kvm/devices/s390_flic.txt | 6 +-
arch/s390/include/asm/kvm_host.h | 10 ++
arch/s390/kvm/Kconfig | 2 +
arch/s390/kvm/Makefile | 2 +-
arch/s390/kvm/interrupt.c | 132 ++++++++++++++++++++++-
arch/s390/kvm/irq.h | 22 ++++
arch/s390/kvm/kvm-s390.c | 17 +++
include/linux/kvm_host.h | 9 ++
include/uapi/linux/kvm.h | 11 ++
10 files changed, 222 insertions(+), 10 deletions(-)
create mode 100644 arch/s390/kvm/irq.h
diff --git a/Documentation/virtual/kvm/api.txt b/Documentation/virtual/kvm/api.txt
index 7c6a578..f084e29 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -586,8 +586,8 @@ struct kvm_fpu {
4.24 KVM_CREATE_IRQCHIP
-Capability: KVM_CAP_IRQCHIP
-Architectures: x86, ia64, ARM, arm64
+Capability: KVM_CAP_IRQCHIP, KVM_CAP_S390_IRQCHIP (s390)
+Architectures: x86, ia64, ARM, arm64, s390
Type: vm ioctl
Parameters: none
Returns: 0 on success, -1 on error
@@ -596,7 +596,10 @@ Creates an interrupt controller model in the kernel. On x86, creates a virtual
ioapic, a virtual PIC (two PICs, nested), and sets up future vcpus to have a
local APIC. IRQ routing for GSIs 0-15 is set to both PIC and IOAPIC; GSI 16-23
only go to the IOAPIC. On ia64, a IOSAPIC is created. On ARM/arm64, a GIC is
-created.
+created. On s390, a dummy irq routing table is created.
+
+Note that on s390 the KVM_CAP_S390_IRQCHIP vm capability needs to be enabled
+before KVM_CREATE_IRQCHIP can be used.
4.25 KVM_IRQ_LINE
@@ -1322,7 +1325,7 @@ KVM_ASSIGN_DEV_IRQ. Partial deassignment of host or guest IRQ is allowed.
4.52 KVM_SET_GSI_ROUTING
Capability: KVM_CAP_IRQ_ROUTING
-Architectures: x86 ia64
+Architectures: x86 ia64 s390
Type: vm ioctl
Parameters: struct kvm_irq_routing (in)
Returns: 0 on success, -1 on error
@@ -1345,6 +1348,7 @@ struct kvm_irq_routing_entry {
union {
struct kvm_irq_routing_irqchip irqchip;
struct kvm_irq_routing_msi msi;
+ struct kvm_irq_routing_s390_adapter adapter;
__u32 pad[8];
} u;
};
@@ -1352,6 +1356,7 @@ struct kvm_irq_routing_entry {
/* gsi routing entry types */
#define KVM_IRQ_ROUTING_IRQCHIP 1
#define KVM_IRQ_ROUTING_MSI 2
+#define KVM_IRQ_ROUTING_S390_ADAPTER 3
No flags are specified so far, the corresponding field must be set to zero.
@@ -1367,6 +1372,14 @@ struct kvm_irq_routing_msi {
__u32 pad;
};
+struct kvm_irq_routing_s390_adapter {
+ __u64 ind_addr;
+ __u64 summary_addr;
+ __u64 ind_offset;
+ __u32 summary_offset;
+ __u32 adapter_id;
+};
+
4.53 KVM_ASSIGN_SET_MSIX_NR
diff --git a/Documentation/virtual/kvm/devices/s390_flic.txt b/Documentation/virtual/kvm/devices/s390_flic.txt
index db16111..4ceef53 100644
--- a/Documentation/virtual/kvm/devices/s390_flic.txt
+++ b/Documentation/virtual/kvm/devices/s390_flic.txt
@@ -82,8 +82,10 @@ struct kvm_s390_io_adapter_req {
mask or unmask the adapter, as specified in mask
KVM_S390_IO_ADAPTER_MAP
- pin a userspace page for the address provided in addr and add it to the
+ perform a gmap translation for the guest address provided in addr,
+ pin a userspace page for the translated address and add it to the
list of mappings
KVM_S390_IO_ADAPTER_UNMAP
- release a userspace page as specified in addr from the list of mappings
+ release a userspace page for the translated address specified in addr
+ from the list of mappings
diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h
index 83aa94b..3d7bce8 100644
--- a/arch/s390/include/asm/kvm_host.h
+++ b/arch/s390/include/asm/kvm_host.h
@@ -24,6 +24,14 @@
#define KVM_MAX_VCPUS 64
#define KVM_USER_MEM_SLOTS 32
+/*
+ * These seem to be used for allocating ->chip in the routing table,
+ * which we don't use. 4096 is an out-of-thin-air value. If we need
+ * to look at ->chip later on, we'll need to revisit this.
+ */
+#define KVM_NR_IRQCHIPS 1
+#define KVM_IRQCHIP_NUM_PINS 4096
+
struct sca_entry {
atomic_t scn;
__u32 reserved;
@@ -247,6 +255,7 @@ struct kvm_arch_memory_slot {
struct s390_map_info {
struct list_head list;
+ __u64 guest_addr;
__u64 addr;
struct page *page;
};
@@ -270,6 +279,7 @@ struct kvm_arch{
struct kvm_device *flic;
struct gmap *gmap;
int css_support;
+ int use_irqchip;
struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
};
diff --git a/arch/s390/kvm/Kconfig b/arch/s390/kvm/Kconfig
index c8bacbc..10d529a 100644
--- a/arch/s390/kvm/Kconfig
+++ b/arch/s390/kvm/Kconfig
@@ -25,6 +25,8 @@ config KVM
select HAVE_KVM_EVENTFD
select KVM_ASYNC_PF
select KVM_ASYNC_PF_SYNC
+ select HAVE_KVM_IRQCHIP
+ select HAVE_KVM_IRQ_ROUTING
---help---
Support hosting paravirtualized guest machines using the SIE
virtualization capability on the mainframe. This should work
diff --git a/arch/s390/kvm/Makefile b/arch/s390/kvm/Makefile
index a47d2c3..d3adb37 100644
--- a/arch/s390/kvm/Makefile
+++ b/arch/s390/kvm/Makefile
@@ -7,7 +7,7 @@
# as published by the Free Software Foundation.
KVM := ../../../virt/kvm
-common-objs = $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/async_pf.o
+common-objs = $(KVM)/kvm_main.o $(KVM)/eventfd.o $(KVM)/async_pf.o $(KVM)/irqchip.o
ccflags-y := -Ivirt/kvm -Iarch/s390/kvm
diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c
index bfe59be..e4af3c4 100644
--- a/arch/s390/kvm/interrupt.c
+++ b/arch/s390/kvm/interrupt.c
@@ -13,6 +13,7 @@
#include <linux/interrupt.h>
#include <linux/kvm_host.h>
#include <linux/hrtimer.h>
+#include <linux/mmu_context.h>
#include <linux/signal.h>
#include <linux/slab.h>
#include <asm/asm-offsets.h>
@@ -1115,8 +1116,13 @@ static int kvm_s390_adapter_map(struct kvm *kvm, unsigned int id, __u64 addr)
goto out;
}
INIT_LIST_HEAD(&map->list);
- map->addr = addr;
- ret = get_user_pages_fast(addr, 1, 1, &map->page);
+ map->guest_addr = addr;
+ map->addr = gmap_translate(addr, kvm->arch.gmap);
+ if (map->addr == -EFAULT) {
+ ret = -EFAULT;
+ goto out;
+ }
+ ret = get_user_pages_fast(map->addr, 1, 1, &map->page);
if (ret < 0)
goto out;
BUG_ON(ret != 1);
@@ -1141,7 +1147,7 @@ static int kvm_s390_adapter_unmap(struct kvm *kvm, unsigned int id, __u64 addr)
down_write(&adapter->maps_lock);
list_for_each_entry_safe(map, tmp, &adapter->maps, list) {
- if (map->addr == addr) {
+ if (map->guest_addr == addr) {
found = 1;
list_del(&map->list);
put_page(map->page);
@@ -1269,3 +1275,123 @@ struct kvm_device_ops kvm_flic_ops = {
.create = flic_create,
.destroy = flic_destroy,
};
+
+static unsigned long get_ind_bit(__u64 addr, unsigned long bit_nr, bool swap)
+{
+ unsigned long bit;
+
+ bit = bit_nr + (addr % PAGE_SIZE) * 8;
+
+ return swap ? (bit ^ (BITS_PER_LONG - 1)) : bit;
+}
+
+static struct s390_map_info *get_map_info(struct s390_io_adapter *adapter,
+ u64 addr)
+{
+ struct s390_map_info *map;
+
+ if (!adapter)
+ return NULL;
+
+ list_for_each_entry(map, &adapter->maps, list) {
+ if (map->guest_addr == addr)
+ return map;
+ }
+ return NULL;
+}
+
+static int adapter_indicators_set(struct kvm *kvm,
+ struct s390_io_adapter *adapter,
+ struct kvm_s390_adapter_int *adapter_int)
+{
+ unsigned long bit;
+ int summary_set, idx;
+ struct s390_map_info *info;
+ void *map;
+
+ info = get_map_info(adapter, adapter_int->ind_addr);
+ if (!info)
+ return -1;
+ map = page_address(info->page);
+ bit = get_ind_bit(info->addr, adapter_int->ind_offset, adapter->swap);
+ set_bit(bit, map);
+ idx = srcu_read_lock(&kvm->srcu);
+ mark_page_dirty(kvm, info->guest_addr >> PAGE_SHIFT);
+ set_page_dirty_lock(info->page);
+ info = get_map_info(adapter, adapter_int->summary_addr);
+ if (!info) {
+ srcu_read_unlock(&kvm->srcu, idx);
+ return -1;
+ }
+ map = page_address(info->page);
+ bit = get_ind_bit(info->addr, adapter_int->summary_offset,
+ adapter->swap);
+ summary_set = test_and_set_bit(bit, map);
+ mark_page_dirty(kvm, info->guest_addr >> PAGE_SHIFT);
+ set_page_dirty_lock(info->page);
+ srcu_read_unlock(&kvm->srcu, idx);
+ return summary_set ? 0 : 1;
+}
+
+/*
+ * < 0 - not injected due to error
+ * = 0 - coalesced, summary indicator already active
+ * > 0 - injected interrupt
+ */
+static int set_adapter_int(struct kvm_kernel_irq_routing_entry *e,
+ struct kvm *kvm, int irq_source_id, int level,
+ bool line_status)
+{
+ int ret;
+ struct s390_io_adapter *adapter;
+
+ /* We're only interested in the 0->1 transition. */
+ if (!level)
+ return 0;
+ adapter = get_io_adapter(kvm, e->adapter.adapter_id);
+ if (!adapter)
+ return -1;
+ down_read(&adapter->maps_lock);
+ ret = adapter_indicators_set(kvm, adapter, &e->adapter);
+ up_read(&adapter->maps_lock);
+ if ((ret > 0) && !adapter->masked) {
+ struct kvm_s390_interrupt s390int = {
+ .type = KVM_S390_INT_IO(1, 0, 0, 0),
+ .parm = 0,
+ .parm64 = (adapter->isc << 27) | 0x80000000,
+ };
+ ret = kvm_s390_inject_vm(kvm, &s390int);
+ if (ret == 0)
+ ret = 1;
+ }
+ return ret;
+}
+
+int kvm_set_routing_entry(struct kvm_irq_routing_table *rt,
+ struct kvm_kernel_irq_routing_entry *e,
+ const struct kvm_irq_routing_entry *ue)
+{
+ int ret;
+
+ switch (ue->type) {
+ case KVM_IRQ_ROUTING_S390_ADAPTER:
+ e->set = set_adapter_int;
+ e->adapter.summary_addr = ue->u.adapter.summary_addr;
+ e->adapter.ind_addr = ue->u.adapter.ind_addr;
+ e->adapter.summary_offset = ue->u.adapter.summary_offset;
+ e->adapter.ind_offset = ue->u.adapter.ind_offset;
+ e->adapter.adapter_id = ue->u.adapter.adapter_id;
+ ret = 0;
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
+int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e, struct kvm *kvm,
+ int irq_source_id, int level, bool line_status)
+{
+ return -EINVAL;
+}
diff --git a/arch/s390/kvm/irq.h b/arch/s390/kvm/irq.h
new file mode 100644
index 0000000..d98e415
--- /dev/null
+++ b/arch/s390/kvm/irq.h
@@ -0,0 +1,22 @@
+/*
+ * s390 irqchip routines
+ *
+ * Copyright IBM Corp. 2014
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License (version 2 only)
+ * as published by the Free Software Foundation.
+ *
+ * Author(s): Cornelia Huck <cornelia.huck@de.ibm.com>
+ */
+#ifndef __KVM_IRQ_H
+#define __KVM_IRQ_H
+
+#include <linux/kvm_host.h>
+
+static inline int irqchip_in_kernel(struct kvm *kvm)
+{
+ return 1;
+}
+
+#endif
diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c
index f0ff213..216317a 100644
--- a/arch/s390/kvm/kvm-s390.c
+++ b/arch/s390/kvm/kvm-s390.c
@@ -196,6 +196,10 @@ static int kvm_vm_ioctl_enable_cap(struct kvm *kvm, struct kvm_enable_cap *cap)
return -EINVAL;
switch (cap->cap) {
+ case KVM_CAP_S390_IRQCHIP:
+ kvm->arch.use_irqchip = 1;
+ r = 0;
+ break;
default:
r = -EINVAL;
break;
@@ -228,6 +232,18 @@ long kvm_arch_vm_ioctl(struct file *filp,
r = kvm_vm_ioctl_enable_cap(kvm, &cap);
break;
}
+ case KVM_CREATE_IRQCHIP: {
+ struct kvm_irq_routing_entry routing;
+
+ r = -EINVAL;
+ if (kvm->arch.use_irqchip) {
+ /* Set up dummy routing. */
+ memset(&routing, 0, sizeof(routing));
+ kvm_set_irq_routing(kvm, &routing, 0, 0);
+ r = 0;
+ }
+ break;
+ }
default:
r = -ENOTTY;
}
@@ -284,6 +300,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
}
kvm->arch.css_support = 0;
+ kvm->arch.use_irqchip = 0;
return 0;
out_nogmap:
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9816b68..da7510b 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -297,6 +297,14 @@ static inline unsigned long kvm_dirty_bitmap_bytes(struct kvm_memory_slot *memsl
return ALIGN(memslot->npages, BITS_PER_LONG) / 8;
}
+struct kvm_s390_adapter_int {
+ u64 ind_addr;
+ u64 summary_addr;
+ u64 ind_offset;
+ u32 summary_offset;
+ u32 adapter_id;
+};
+
struct kvm_kernel_irq_routing_entry {
u32 gsi;
u32 type;
@@ -309,6 +317,7 @@ struct kvm_kernel_irq_routing_entry {
unsigned pin;
} irqchip;
struct msi_msg msi;
+ struct kvm_s390_adapter_int adapter;
};
struct hlist_node link;
};
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index 48bc6db..4a0387b 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -741,6 +741,7 @@ struct kvm_ppc_smmu_info {
#define KVM_CAP_EXT_EMUL_CPUID 95
#define KVM_CAP_HYPERV_TIME 96
#define KVM_CAP_ENABLE_CAP_VM 97
+#define KVM_CAP_S390_IRQCHIP 98
#ifdef KVM_CAP_IRQ_ROUTING
@@ -756,9 +757,18 @@ struct kvm_irq_routing_msi {
__u32 pad;
};
+struct kvm_irq_routing_s390_adapter {
+ __u64 ind_addr;
+ __u64 summary_addr;
+ __u64 ind_offset;
+ __u32 summary_offset;
+ __u32 adapter_id;
+};
+
/* gsi routing entry types */
#define KVM_IRQ_ROUTING_IRQCHIP 1
#define KVM_IRQ_ROUTING_MSI 2
+#define KVM_IRQ_ROUTING_S390_ADAPTER 3
struct kvm_irq_routing_entry {
__u32 gsi;
@@ -768,6 +778,7 @@ struct kvm_irq_routing_entry {
union {
struct kvm_irq_routing_irqchip irqchip;
struct kvm_irq_routing_msi msi;
+ struct kvm_irq_routing_s390_adapter adapter;
__u32 pad[8];
} u;
};
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH 7/7] KVM: Bump KVM_MAX_IRQ_ROUTES for s390
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-02-25 17:24 ` Cornelia Huck
-1 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: borntraeger, agraf, pbonzini, gleb, Cornelia Huck
The maximum number for irq routes is currently 1024, which is a bit on
the small size for s390: We support up to 4 x 64k virtual devices with
up to 64 queues, and we need one route for each of the queues if we want
to operate it via irqfd.
Let's bump this to 4k on s390 for now, as this at least covers the saner
setups.
We need to find a more general solution, though, as we can't just grow
the routing table indefinitly.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
include/linux/kvm_host.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index da7510b..7d21cf9 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -922,7 +922,11 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
#ifdef CONFIG_HAVE_KVM_IRQ_ROUTING
+#ifdef CONFIG_S390
+#define KVM_MAX_IRQ_ROUTES 4096 //FIXME: we can have more than that...
+#else
#define KVM_MAX_IRQ_ROUTES 1024
+#endif
int kvm_setup_default_irq_routing(struct kvm *kvm);
int kvm_set_irq_routing(struct kvm *kvm,
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [Qemu-devel] [PATCH 7/7] KVM: Bump KVM_MAX_IRQ_ROUTES for s390
@ 2014-02-25 17:24 ` Cornelia Huck
0 siblings, 0 replies; 18+ messages in thread
From: Cornelia Huck @ 2014-02-25 17:24 UTC (permalink / raw)
To: kvm, linux-s390, qemu-devel
Cc: gleb, borntraeger, Cornelia Huck, agraf, pbonzini
The maximum number for irq routes is currently 1024, which is a bit on
the small size for s390: We support up to 4 x 64k virtual devices with
up to 64 queues, and we need one route for each of the queues if we want
to operate it via irqfd.
Let's bump this to 4k on s390 for now, as this at least covers the saner
setups.
We need to find a more general solution, though, as we can't just grow
the routing table indefinitly.
Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
---
include/linux/kvm_host.h | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index da7510b..7d21cf9 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -922,7 +922,11 @@ static inline int mmu_notifier_retry(struct kvm *kvm, unsigned long mmu_seq)
#ifdef CONFIG_HAVE_KVM_IRQ_ROUTING
+#ifdef CONFIG_S390
+#define KVM_MAX_IRQ_ROUTES 4096 //FIXME: we can have more than that...
+#else
#define KVM_MAX_IRQ_ROUTES 1024
+#endif
int kvm_setup_default_irq_routing(struct kvm *kvm);
int kvm_set_irq_routing(struct kvm *kvm,
--
1.7.9.5
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH 2/7] KVM: s390: virtio-ccw adapter interrupt support.
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
@ 2014-03-03 20:47 ` Christian Borntraeger
-1 siblings, 0 replies; 18+ messages in thread
From: Christian Borntraeger @ 2014-03-03 20:47 UTC (permalink / raw)
To: Cornelia Huck, kvm, linux-s390, qemu-devel; +Cc: agraf, pbonzini, gleb
On 25/02/14 18:24, Cornelia Huck wrote:
> Implement the new CCW_CMD_SET_IND_ADAPTER command and try to enable
> adapter interrupts for every device on the first startup. If the host
> does not support adapter interrupts, fall back to normal I/O interrupts.
>
> virtio-ccw adapter interrupts use the same isc as normal I/O subchannels
> and share a summary indicator for all devices sharing the same indicator
> area.
>
> Indicator bits for the individual virtqueues may be contained in the same
> indicator area for different devices.
>
> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
This is s390 only code, so I will queue that for my next KVM:s390 pull request
(I will also queue patch 1. It is also in s390/features, but patch 2
needs it).
For everything else (irqfd etc) I prefer to have some review from the
other platforms, since we change common code.
Christian
> ---
> arch/s390/include/asm/irq.h | 1 +
> arch/s390/kernel/irq.c | 1 +
> drivers/s390/kvm/virtio_ccw.c | 276 +++++++++++++++++++++++++++++++++++++++--
> 3 files changed, 268 insertions(+), 10 deletions(-)
>
> diff --git a/arch/s390/include/asm/irq.h b/arch/s390/include/asm/irq.h
> index 5f8bcc5..35f0faa 100644
> --- a/arch/s390/include/asm/irq.h
> +++ b/arch/s390/include/asm/irq.h
> @@ -53,6 +53,7 @@ enum interruption_class {
> IRQIO_PCI,
> IRQIO_MSI,
> IRQIO_VIR,
> + IRQIO_VAI,
> NMI_NMI,
> CPU_RST,
> NR_ARCH_IRQS
> diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c
> index bb27a26..c288ef7 100644
> --- a/arch/s390/kernel/irq.c
> +++ b/arch/s390/kernel/irq.c
> @@ -84,6 +84,7 @@ static const struct irq_class irqclass_sub_desc[NR_ARCH_IRQS] = {
> [IRQIO_PCI] = {.name = "PCI", .desc = "[I/O] PCI Interrupt" },
> [IRQIO_MSI] = {.name = "MSI", .desc = "[I/O] MSI Interrupt" },
> [IRQIO_VIR] = {.name = "VIR", .desc = "[I/O] Virtual I/O Devices"},
> + [IRQIO_VAI] = {.name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"},
> [NMI_NMI] = {.name = "NMI", .desc = "[NMI] Machine Check"},
> [CPU_RST] = {.name = "RST", .desc = "[CPU] CPU Restart"},
> };
> diff --git a/drivers/s390/kvm/virtio_ccw.c b/drivers/s390/kvm/virtio_ccw.c
> index 0fc5848..09afefe 100644
> --- a/drivers/s390/kvm/virtio_ccw.c
> +++ b/drivers/s390/kvm/virtio_ccw.c
> @@ -1,7 +1,7 @@
> /*
> * ccw based virtio transport
> *
> - * Copyright IBM Corp. 2012
> + * Copyright IBM Corp. 2012,2014
> *
> * This program is free software; you can redistribute it and/or modify
> * it under the terms of the GNU General Public License (version 2 only)
> @@ -32,6 +32,8 @@
> #include <asm/cio.h>
> #include <asm/ccwdev.h>
> #include <asm/virtio-ccw.h>
> +#include <asm/isc.h>
> +#include <asm/airq.h>
>
> /*
> * virtio related functions
> @@ -58,6 +60,8 @@ struct virtio_ccw_device {
> unsigned long indicators;
> unsigned long indicators2;
> struct vq_config_block *config_block;
> + bool is_thinint;
> + void *airq_info;
> };
>
> struct vq_info_block {
> @@ -72,15 +76,38 @@ struct virtio_feature_desc {
> __u8 index;
> } __packed;
>
> +struct virtio_thinint_area {
> + unsigned long summary_indicator;
> + unsigned long indicator;
> + u64 bit_nr;
> + u8 isc;
> +} __packed;
> +
> struct virtio_ccw_vq_info {
> struct virtqueue *vq;
> int num;
> void *queue;
> struct vq_info_block *info_block;
> + int bit_nr;
> struct list_head node;
> long cookie;
> };
>
> +#define VIRTIO_AIRQ_ISC IO_SCH_ISC /* inherit from subchannel */
> +
> +#define VIRTIO_IV_BITS (L1_CACHE_BYTES * 8)
> +#define MAX_AIRQ_AREAS 20
> +
> +static int virtio_ccw_use_airq = 1;
> +
> +struct airq_info {
> + rwlock_t lock;
> + u8 summary_indicator;
> + struct airq_struct airq;
> + struct airq_iv *aiv;
> +};
> +static struct airq_info *airq_areas[MAX_AIRQ_AREAS];
> +
> #define CCW_CMD_SET_VQ 0x13
> #define CCW_CMD_VDEV_RESET 0x33
> #define CCW_CMD_SET_IND 0x43
> @@ -91,6 +118,7 @@ struct virtio_ccw_vq_info {
> #define CCW_CMD_WRITE_CONF 0x21
> #define CCW_CMD_WRITE_STATUS 0x31
> #define CCW_CMD_READ_VQ_CONF 0x32
> +#define CCW_CMD_SET_IND_ADAPTER 0x73
>
> #define VIRTIO_CCW_DOING_SET_VQ 0x00010000
> #define VIRTIO_CCW_DOING_RESET 0x00040000
> @@ -102,6 +130,7 @@ struct virtio_ccw_vq_info {
> #define VIRTIO_CCW_DOING_SET_IND 0x01000000
> #define VIRTIO_CCW_DOING_READ_VQ_CONF 0x02000000
> #define VIRTIO_CCW_DOING_SET_CONF_IND 0x04000000
> +#define VIRTIO_CCW_DOING_SET_IND_ADAPTER 0x08000000
> #define VIRTIO_CCW_INTPARM_MASK 0xffff0000
>
> static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
> @@ -109,6 +138,125 @@ static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
> return container_of(vdev, struct virtio_ccw_device, vdev);
> }
>
> +static void drop_airq_indicator(struct virtqueue *vq, struct airq_info *info)
> +{
> + unsigned long i, flags;
> +
> + write_lock_irqsave(&info->lock, flags);
> + for (i = 0; i < airq_iv_end(info->aiv); i++) {
> + if (vq == (void *)airq_iv_get_ptr(info->aiv, i)) {
> + airq_iv_free_bit(info->aiv, i);
> + airq_iv_set_ptr(info->aiv, i, 0);
> + break;
> + }
> + }
> + write_unlock_irqrestore(&info->lock, flags);
> +}
> +
> +static void virtio_airq_handler(struct airq_struct *airq)
> +{
> + struct airq_info *info = container_of(airq, struct airq_info, airq);
> + unsigned long ai;
> +
> + inc_irq_stat(IRQIO_VAI);
> + read_lock(&info->lock);
> + /* Walk through indicators field, summary indicator active. */
> + for (ai = 0;;) {
> + ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
> + if (ai == -1UL)
> + break;
> + vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
> + }
> + info->summary_indicator = 0;
> + smp_wmb();
> + /* Walk through indicators field, summary indicator not active. */
> + for (ai = 0;;) {
> + ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
> + if (ai == -1UL)
> + break;
> + vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
> + }
> + read_unlock(&info->lock);
> +}
> +
> +static struct airq_info *new_airq_info(void)
> +{
> + struct airq_info *info;
> + int rc;
> +
> + info = kzalloc(sizeof(*info), GFP_KERNEL);
> + if (!info)
> + return NULL;
> + rwlock_init(&info->lock);
> + info->aiv = airq_iv_create(VIRTIO_IV_BITS, AIRQ_IV_ALLOC | AIRQ_IV_PTR);
> + if (!info->aiv) {
> + kfree(info);
> + return NULL;
> + }
> + info->airq.handler = virtio_airq_handler;
> + info->airq.lsi_ptr = &info->summary_indicator;
> + info->airq.lsi_mask = 0xff;
> + info->airq.isc = VIRTIO_AIRQ_ISC;
> + rc = register_adapter_interrupt(&info->airq);
> + if (rc) {
> + airq_iv_release(info->aiv);
> + kfree(info);
> + return NULL;
> + }
> + return info;
> +}
> +
> +static void destroy_airq_info(struct airq_info *info)
> +{
> + if (!info)
> + return;
> +
> + unregister_adapter_interrupt(&info->airq);
> + airq_iv_release(info->aiv);
> + kfree(info);
> +}
> +
> +static unsigned long get_airq_indicator(struct virtqueue *vqs[], int nvqs,
> + u64 *first, void **airq_info)
> +{
> + int i, j;
> + struct airq_info *info;
> + unsigned long indicator_addr = 0;
> + unsigned long bit, flags;
> +
> + for (i = 0; i < MAX_AIRQ_AREAS && !indicator_addr; i++) {
> + if (!airq_areas[i])
> + airq_areas[i] = new_airq_info();
> + info = airq_areas[i];
> + if (!info)
> + return 0;
> + write_lock_irqsave(&info->lock, flags);
> + bit = airq_iv_alloc(info->aiv, nvqs);
> + if (bit == -1UL) {
> + /* Not enough vacancies. */
> + write_unlock_irqrestore(&info->lock, flags);
> + continue;
> + }
> + *first = bit;
> + *airq_info = info;
> + indicator_addr = (unsigned long)info->aiv->vector;
> + for (j = 0; j < nvqs; j++) {
> + airq_iv_set_ptr(info->aiv, bit + j,
> + (unsigned long)vqs[j]);
> + }
> + write_unlock_irqrestore(&info->lock, flags);
> + }
> + return indicator_addr;
> +}
> +
> +static void virtio_ccw_drop_indicators(struct virtio_ccw_device *vcdev)
> +{
> + struct virtio_ccw_vq_info *info;
> +
> + list_for_each_entry(info, &vcdev->virtqueues, node)
> + drop_airq_indicator(info->vq, vcdev->airq_info);
> +}
> +
> static int doing_io(struct virtio_ccw_device *vcdev, __u32 flag)
> {
> unsigned long flags;
> @@ -145,6 +293,51 @@ static int ccw_io_helper(struct virtio_ccw_device *vcdev,
> return ret ? ret : vcdev->err;
> }
>
> +static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev,
> + struct ccw1 *ccw)
> +{
> + int ret;
> + unsigned long *indicatorp = NULL;
> + struct virtio_thinint_area *thinint_area = NULL;
> + struct airq_info *airq_info = vcdev->airq_info;
> +
> + if (vcdev->is_thinint) {
> + thinint_area = kzalloc(sizeof(*thinint_area),
> + GFP_DMA | GFP_KERNEL);
> + if (!thinint_area)
> + return;
> + thinint_area->summary_indicator =
> + (unsigned long) &airq_info->summary_indicator;
> + thinint_area->isc = VIRTIO_AIRQ_ISC;
> + ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
> + ccw->count = sizeof(*thinint_area);
> + ccw->cda = (__u32)(unsigned long) thinint_area;
> + } else {
> + indicatorp = kmalloc(sizeof(&vcdev->indicators),
> + GFP_DMA | GFP_KERNEL);
> + if (!indicatorp)
> + return;
> + *indicatorp = 0;
> + ccw->cmd_code = CCW_CMD_SET_IND;
> + ccw->count = sizeof(vcdev->indicators);
> + ccw->cda = (__u32)(unsigned long) indicatorp;
> + }
> + /* Deregister indicators from host. */
> + vcdev->indicators = 0;
> + ccw->flags = 0;
> + ret = ccw_io_helper(vcdev, ccw,
> + vcdev->is_thinint ?
> + VIRTIO_CCW_DOING_SET_IND_ADAPTER :
> + VIRTIO_CCW_DOING_SET_IND);
> + if (ret && (ret != -ENODEV))
> + dev_info(&vcdev->cdev->dev,
> + "Failed to deregister indicators (%d)\n", ret);
> + else if (vcdev->is_thinint)
> + virtio_ccw_drop_indicators(vcdev);
> + kfree(indicatorp);
> + kfree(thinint_area);
> +}
> +
> static inline long do_kvm_notify(struct subchannel_id schid,
> unsigned long queue_index,
> long cookie)
> @@ -232,11 +425,13 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev)
> {
> struct virtqueue *vq, *n;
> struct ccw1 *ccw;
> + struct virtio_ccw_device *vcdev = to_vc_device(vdev);
>
> ccw = kzalloc(sizeof(*ccw), GFP_DMA | GFP_KERNEL);
> if (!ccw)
> return;
>
> + virtio_ccw_drop_indicator(vcdev, ccw);
>
> list_for_each_entry_safe(vq, n, &vdev->vqs, list)
> virtio_ccw_del_vq(vq, ccw);
> @@ -326,6 +521,52 @@ out_err:
> return ERR_PTR(err);
> }
>
> +static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev,
> + struct virtqueue *vqs[], int nvqs,
> + struct ccw1 *ccw)
> +{
> + int ret;
> + struct virtio_thinint_area *thinint_area = NULL;
> + struct airq_info *info;
> +
> + thinint_area = kzalloc(sizeof(*thinint_area), GFP_DMA | GFP_KERNEL);
> + if (!thinint_area) {
> + ret = -ENOMEM;
> + goto out;
> + }
> + /* Try to get an indicator. */
> + thinint_area->indicator = get_airq_indicator(vqs, nvqs,
> + &thinint_area->bit_nr,
> + &vcdev->airq_info);
> + if (!thinint_area->indicator) {
> + ret = -ENOSPC;
> + goto out;
> + }
> + info = vcdev->airq_info;
> + thinint_area->summary_indicator =
> + (unsigned long) &info->summary_indicator;
> + thinint_area->isc = VIRTIO_AIRQ_ISC;
> + ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
> + ccw->flags = CCW_FLAG_SLI;
> + ccw->count = sizeof(*thinint_area);
> + ccw->cda = (__u32)(unsigned long)thinint_area;
> + ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND_ADAPTER);
> + if (ret) {
> + dev_warn(&vcdev->cdev->dev,
> + "enabling adapter interrupts = %d\n", ret);
> + if (ret == -EOPNOTSUPP)
> + /*
> + * The host does not support adapter interrupts
> + * for virtio-ccw, stop trying.
> + */
> + virtio_ccw_use_airq = 0;
> + virtio_ccw_drop_indicators(vcdev);
> + }
> +out:
> + kfree(thinint_area);
> + return ret;
> +}
> +
> static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
> struct virtqueue *vqs[],
> vq_callback_t *callbacks[],
> @@ -355,15 +596,23 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
> if (!indicatorp)
> goto out;
> *indicatorp = (unsigned long) &vcdev->indicators;
> - /* Register queue indicators with host. */
> - vcdev->indicators = 0;
> - ccw->cmd_code = CCW_CMD_SET_IND;
> - ccw->flags = 0;
> - ccw->count = sizeof(vcdev->indicators);
> - ccw->cda = (__u32)(unsigned long) indicatorp;
> - ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
> - if (ret)
> - goto out;
> + if (vcdev->is_thinint) {
> + ret = virtio_ccw_register_adapter_ind(vcdev, vqs, nvqs, ccw);
> + if (ret)
> + /* no error, just fall back to legacy interrupts */
> + vcdev->is_thinint = 0;
> + }
> + if (!vcdev->is_thinint) {
> + /* Register queue indicators with host. */
> + vcdev->indicators = 0;
> + ccw->cmd_code = CCW_CMD_SET_IND;
> + ccw->flags = 0;
> + ccw->count = sizeof(vcdev->indicators);
> + ccw->cda = (__u32)(unsigned long) indicatorp;
> + ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
> + if (ret)
> + goto out;
> + }
> /* Register indicators2 with host for config changes */
> *indicatorp = (unsigned long) &vcdev->indicators2;
> vcdev->indicators2 = 0;
> @@ -663,6 +912,7 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev,
> case VIRTIO_CCW_DOING_SET_CONF_IND:
> case VIRTIO_CCW_DOING_RESET:
> case VIRTIO_CCW_DOING_READ_VQ_CONF:
> + case VIRTIO_CCW_DOING_SET_IND_ADAPTER:
> vcdev->curr_io &= ~activity;
> wake_up(&vcdev->wait_q);
> break;
> @@ -778,6 +1028,8 @@ static int virtio_ccw_online(struct ccw_device *cdev)
> goto out_free;
> }
>
> + vcdev->is_thinint = virtio_ccw_use_airq; /* at least try */
> +
> vcdev->vdev.dev.parent = &cdev->dev;
> vcdev->vdev.dev.release = virtio_ccw_release_dev;
> vcdev->vdev.config = &virtio_ccw_config_ops;
> @@ -935,6 +1187,10 @@ module_init(virtio_ccw_init);
>
> static void __exit virtio_ccw_exit(void)
> {
> + int i;
> +
> ccw_driver_unregister(&virtio_ccw_driver);
> + for (i = 0; i < MAX_AIRQ_AREAS; i++)
> + destroy_airq_info(airq_areas[i]);
> }
> module_exit(virtio_ccw_exit);
>
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Qemu-devel] [PATCH 2/7] KVM: s390: virtio-ccw adapter interrupt support.
@ 2014-03-03 20:47 ` Christian Borntraeger
0 siblings, 0 replies; 18+ messages in thread
From: Christian Borntraeger @ 2014-03-03 20:47 UTC (permalink / raw)
To: Cornelia Huck, kvm, linux-s390, qemu-devel; +Cc: gleb, pbonzini, agraf
On 25/02/14 18:24, Cornelia Huck wrote:
> Implement the new CCW_CMD_SET_IND_ADAPTER command and try to enable
> adapter interrupts for every device on the first startup. If the host
> does not support adapter interrupts, fall back to normal I/O interrupts.
>
> virtio-ccw adapter interrupts use the same isc as normal I/O subchannels
> and share a summary indicator for all devices sharing the same indicator
> area.
>
> Indicator bits for the individual virtqueues may be contained in the same
> indicator area for different devices.
>
> Signed-off-by: Cornelia Huck <cornelia.huck@de.ibm.com>
This is s390 only code, so I will queue that for my next KVM:s390 pull request
(I will also queue patch 1. It is also in s390/features, but patch 2
needs it).
For everything else (irqfd etc) I prefer to have some review from the
other platforms, since we change common code.
Christian
> ---
> arch/s390/include/asm/irq.h | 1 +
> arch/s390/kernel/irq.c | 1 +
> drivers/s390/kvm/virtio_ccw.c | 276 +++++++++++++++++++++++++++++++++++++++--
> 3 files changed, 268 insertions(+), 10 deletions(-)
>
> diff --git a/arch/s390/include/asm/irq.h b/arch/s390/include/asm/irq.h
> index 5f8bcc5..35f0faa 100644
> --- a/arch/s390/include/asm/irq.h
> +++ b/arch/s390/include/asm/irq.h
> @@ -53,6 +53,7 @@ enum interruption_class {
> IRQIO_PCI,
> IRQIO_MSI,
> IRQIO_VIR,
> + IRQIO_VAI,
> NMI_NMI,
> CPU_RST,
> NR_ARCH_IRQS
> diff --git a/arch/s390/kernel/irq.c b/arch/s390/kernel/irq.c
> index bb27a26..c288ef7 100644
> --- a/arch/s390/kernel/irq.c
> +++ b/arch/s390/kernel/irq.c
> @@ -84,6 +84,7 @@ static const struct irq_class irqclass_sub_desc[NR_ARCH_IRQS] = {
> [IRQIO_PCI] = {.name = "PCI", .desc = "[I/O] PCI Interrupt" },
> [IRQIO_MSI] = {.name = "MSI", .desc = "[I/O] MSI Interrupt" },
> [IRQIO_VIR] = {.name = "VIR", .desc = "[I/O] Virtual I/O Devices"},
> + [IRQIO_VAI] = {.name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"},
> [NMI_NMI] = {.name = "NMI", .desc = "[NMI] Machine Check"},
> [CPU_RST] = {.name = "RST", .desc = "[CPU] CPU Restart"},
> };
> diff --git a/drivers/s390/kvm/virtio_ccw.c b/drivers/s390/kvm/virtio_ccw.c
> index 0fc5848..09afefe 100644
> --- a/drivers/s390/kvm/virtio_ccw.c
> +++ b/drivers/s390/kvm/virtio_ccw.c
> @@ -1,7 +1,7 @@
> /*
> * ccw based virtio transport
> *
> - * Copyright IBM Corp. 2012
> + * Copyright IBM Corp. 2012,2014
> *
> * This program is free software; you can redistribute it and/or modify
> * it under the terms of the GNU General Public License (version 2 only)
> @@ -32,6 +32,8 @@
> #include <asm/cio.h>
> #include <asm/ccwdev.h>
> #include <asm/virtio-ccw.h>
> +#include <asm/isc.h>
> +#include <asm/airq.h>
>
> /*
> * virtio related functions
> @@ -58,6 +60,8 @@ struct virtio_ccw_device {
> unsigned long indicators;
> unsigned long indicators2;
> struct vq_config_block *config_block;
> + bool is_thinint;
> + void *airq_info;
> };
>
> struct vq_info_block {
> @@ -72,15 +76,38 @@ struct virtio_feature_desc {
> __u8 index;
> } __packed;
>
> +struct virtio_thinint_area {
> + unsigned long summary_indicator;
> + unsigned long indicator;
> + u64 bit_nr;
> + u8 isc;
> +} __packed;
> +
> struct virtio_ccw_vq_info {
> struct virtqueue *vq;
> int num;
> void *queue;
> struct vq_info_block *info_block;
> + int bit_nr;
> struct list_head node;
> long cookie;
> };
>
> +#define VIRTIO_AIRQ_ISC IO_SCH_ISC /* inherit from subchannel */
> +
> +#define VIRTIO_IV_BITS (L1_CACHE_BYTES * 8)
> +#define MAX_AIRQ_AREAS 20
> +
> +static int virtio_ccw_use_airq = 1;
> +
> +struct airq_info {
> + rwlock_t lock;
> + u8 summary_indicator;
> + struct airq_struct airq;
> + struct airq_iv *aiv;
> +};
> +static struct airq_info *airq_areas[MAX_AIRQ_AREAS];
> +
> #define CCW_CMD_SET_VQ 0x13
> #define CCW_CMD_VDEV_RESET 0x33
> #define CCW_CMD_SET_IND 0x43
> @@ -91,6 +118,7 @@ struct virtio_ccw_vq_info {
> #define CCW_CMD_WRITE_CONF 0x21
> #define CCW_CMD_WRITE_STATUS 0x31
> #define CCW_CMD_READ_VQ_CONF 0x32
> +#define CCW_CMD_SET_IND_ADAPTER 0x73
>
> #define VIRTIO_CCW_DOING_SET_VQ 0x00010000
> #define VIRTIO_CCW_DOING_RESET 0x00040000
> @@ -102,6 +130,7 @@ struct virtio_ccw_vq_info {
> #define VIRTIO_CCW_DOING_SET_IND 0x01000000
> #define VIRTIO_CCW_DOING_READ_VQ_CONF 0x02000000
> #define VIRTIO_CCW_DOING_SET_CONF_IND 0x04000000
> +#define VIRTIO_CCW_DOING_SET_IND_ADAPTER 0x08000000
> #define VIRTIO_CCW_INTPARM_MASK 0xffff0000
>
> static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
> @@ -109,6 +138,125 @@ static struct virtio_ccw_device *to_vc_device(struct virtio_device *vdev)
> return container_of(vdev, struct virtio_ccw_device, vdev);
> }
>
> +static void drop_airq_indicator(struct virtqueue *vq, struct airq_info *info)
> +{
> + unsigned long i, flags;
> +
> + write_lock_irqsave(&info->lock, flags);
> + for (i = 0; i < airq_iv_end(info->aiv); i++) {
> + if (vq == (void *)airq_iv_get_ptr(info->aiv, i)) {
> + airq_iv_free_bit(info->aiv, i);
> + airq_iv_set_ptr(info->aiv, i, 0);
> + break;
> + }
> + }
> + write_unlock_irqrestore(&info->lock, flags);
> +}
> +
> +static void virtio_airq_handler(struct airq_struct *airq)
> +{
> + struct airq_info *info = container_of(airq, struct airq_info, airq);
> + unsigned long ai;
> +
> + inc_irq_stat(IRQIO_VAI);
> + read_lock(&info->lock);
> + /* Walk through indicators field, summary indicator active. */
> + for (ai = 0;;) {
> + ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
> + if (ai == -1UL)
> + break;
> + vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
> + }
> + info->summary_indicator = 0;
> + smp_wmb();
> + /* Walk through indicators field, summary indicator not active. */
> + for (ai = 0;;) {
> + ai = airq_iv_scan(info->aiv, ai, airq_iv_end(info->aiv));
> + if (ai == -1UL)
> + break;
> + vring_interrupt(0, (void *)airq_iv_get_ptr(info->aiv, ai));
> + }
> + read_unlock(&info->lock);
> +}
> +
> +static struct airq_info *new_airq_info(void)
> +{
> + struct airq_info *info;
> + int rc;
> +
> + info = kzalloc(sizeof(*info), GFP_KERNEL);
> + if (!info)
> + return NULL;
> + rwlock_init(&info->lock);
> + info->aiv = airq_iv_create(VIRTIO_IV_BITS, AIRQ_IV_ALLOC | AIRQ_IV_PTR);
> + if (!info->aiv) {
> + kfree(info);
> + return NULL;
> + }
> + info->airq.handler = virtio_airq_handler;
> + info->airq.lsi_ptr = &info->summary_indicator;
> + info->airq.lsi_mask = 0xff;
> + info->airq.isc = VIRTIO_AIRQ_ISC;
> + rc = register_adapter_interrupt(&info->airq);
> + if (rc) {
> + airq_iv_release(info->aiv);
> + kfree(info);
> + return NULL;
> + }
> + return info;
> +}
> +
> +static void destroy_airq_info(struct airq_info *info)
> +{
> + if (!info)
> + return;
> +
> + unregister_adapter_interrupt(&info->airq);
> + airq_iv_release(info->aiv);
> + kfree(info);
> +}
> +
> +static unsigned long get_airq_indicator(struct virtqueue *vqs[], int nvqs,
> + u64 *first, void **airq_info)
> +{
> + int i, j;
> + struct airq_info *info;
> + unsigned long indicator_addr = 0;
> + unsigned long bit, flags;
> +
> + for (i = 0; i < MAX_AIRQ_AREAS && !indicator_addr; i++) {
> + if (!airq_areas[i])
> + airq_areas[i] = new_airq_info();
> + info = airq_areas[i];
> + if (!info)
> + return 0;
> + write_lock_irqsave(&info->lock, flags);
> + bit = airq_iv_alloc(info->aiv, nvqs);
> + if (bit == -1UL) {
> + /* Not enough vacancies. */
> + write_unlock_irqrestore(&info->lock, flags);
> + continue;
> + }
> + *first = bit;
> + *airq_info = info;
> + indicator_addr = (unsigned long)info->aiv->vector;
> + for (j = 0; j < nvqs; j++) {
> + airq_iv_set_ptr(info->aiv, bit + j,
> + (unsigned long)vqs[j]);
> + }
> + write_unlock_irqrestore(&info->lock, flags);
> + }
> + return indicator_addr;
> +}
> +
> +static void virtio_ccw_drop_indicators(struct virtio_ccw_device *vcdev)
> +{
> + struct virtio_ccw_vq_info *info;
> +
> + list_for_each_entry(info, &vcdev->virtqueues, node)
> + drop_airq_indicator(info->vq, vcdev->airq_info);
> +}
> +
> static int doing_io(struct virtio_ccw_device *vcdev, __u32 flag)
> {
> unsigned long flags;
> @@ -145,6 +293,51 @@ static int ccw_io_helper(struct virtio_ccw_device *vcdev,
> return ret ? ret : vcdev->err;
> }
>
> +static void virtio_ccw_drop_indicator(struct virtio_ccw_device *vcdev,
> + struct ccw1 *ccw)
> +{
> + int ret;
> + unsigned long *indicatorp = NULL;
> + struct virtio_thinint_area *thinint_area = NULL;
> + struct airq_info *airq_info = vcdev->airq_info;
> +
> + if (vcdev->is_thinint) {
> + thinint_area = kzalloc(sizeof(*thinint_area),
> + GFP_DMA | GFP_KERNEL);
> + if (!thinint_area)
> + return;
> + thinint_area->summary_indicator =
> + (unsigned long) &airq_info->summary_indicator;
> + thinint_area->isc = VIRTIO_AIRQ_ISC;
> + ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
> + ccw->count = sizeof(*thinint_area);
> + ccw->cda = (__u32)(unsigned long) thinint_area;
> + } else {
> + indicatorp = kmalloc(sizeof(&vcdev->indicators),
> + GFP_DMA | GFP_KERNEL);
> + if (!indicatorp)
> + return;
> + *indicatorp = 0;
> + ccw->cmd_code = CCW_CMD_SET_IND;
> + ccw->count = sizeof(vcdev->indicators);
> + ccw->cda = (__u32)(unsigned long) indicatorp;
> + }
> + /* Deregister indicators from host. */
> + vcdev->indicators = 0;
> + ccw->flags = 0;
> + ret = ccw_io_helper(vcdev, ccw,
> + vcdev->is_thinint ?
> + VIRTIO_CCW_DOING_SET_IND_ADAPTER :
> + VIRTIO_CCW_DOING_SET_IND);
> + if (ret && (ret != -ENODEV))
> + dev_info(&vcdev->cdev->dev,
> + "Failed to deregister indicators (%d)\n", ret);
> + else if (vcdev->is_thinint)
> + virtio_ccw_drop_indicators(vcdev);
> + kfree(indicatorp);
> + kfree(thinint_area);
> +}
> +
> static inline long do_kvm_notify(struct subchannel_id schid,
> unsigned long queue_index,
> long cookie)
> @@ -232,11 +425,13 @@ static void virtio_ccw_del_vqs(struct virtio_device *vdev)
> {
> struct virtqueue *vq, *n;
> struct ccw1 *ccw;
> + struct virtio_ccw_device *vcdev = to_vc_device(vdev);
>
> ccw = kzalloc(sizeof(*ccw), GFP_DMA | GFP_KERNEL);
> if (!ccw)
> return;
>
> + virtio_ccw_drop_indicator(vcdev, ccw);
>
> list_for_each_entry_safe(vq, n, &vdev->vqs, list)
> virtio_ccw_del_vq(vq, ccw);
> @@ -326,6 +521,52 @@ out_err:
> return ERR_PTR(err);
> }
>
> +static int virtio_ccw_register_adapter_ind(struct virtio_ccw_device *vcdev,
> + struct virtqueue *vqs[], int nvqs,
> + struct ccw1 *ccw)
> +{
> + int ret;
> + struct virtio_thinint_area *thinint_area = NULL;
> + struct airq_info *info;
> +
> + thinint_area = kzalloc(sizeof(*thinint_area), GFP_DMA | GFP_KERNEL);
> + if (!thinint_area) {
> + ret = -ENOMEM;
> + goto out;
> + }
> + /* Try to get an indicator. */
> + thinint_area->indicator = get_airq_indicator(vqs, nvqs,
> + &thinint_area->bit_nr,
> + &vcdev->airq_info);
> + if (!thinint_area->indicator) {
> + ret = -ENOSPC;
> + goto out;
> + }
> + info = vcdev->airq_info;
> + thinint_area->summary_indicator =
> + (unsigned long) &info->summary_indicator;
> + thinint_area->isc = VIRTIO_AIRQ_ISC;
> + ccw->cmd_code = CCW_CMD_SET_IND_ADAPTER;
> + ccw->flags = CCW_FLAG_SLI;
> + ccw->count = sizeof(*thinint_area);
> + ccw->cda = (__u32)(unsigned long)thinint_area;
> + ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND_ADAPTER);
> + if (ret) {
> + dev_warn(&vcdev->cdev->dev,
> + "enabling adapter interrupts = %d\n", ret);
> + if (ret == -EOPNOTSUPP)
> + /*
> + * The host does not support adapter interrupts
> + * for virtio-ccw, stop trying.
> + */
> + virtio_ccw_use_airq = 0;
> + virtio_ccw_drop_indicators(vcdev);
> + }
> +out:
> + kfree(thinint_area);
> + return ret;
> +}
> +
> static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
> struct virtqueue *vqs[],
> vq_callback_t *callbacks[],
> @@ -355,15 +596,23 @@ static int virtio_ccw_find_vqs(struct virtio_device *vdev, unsigned nvqs,
> if (!indicatorp)
> goto out;
> *indicatorp = (unsigned long) &vcdev->indicators;
> - /* Register queue indicators with host. */
> - vcdev->indicators = 0;
> - ccw->cmd_code = CCW_CMD_SET_IND;
> - ccw->flags = 0;
> - ccw->count = sizeof(vcdev->indicators);
> - ccw->cda = (__u32)(unsigned long) indicatorp;
> - ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
> - if (ret)
> - goto out;
> + if (vcdev->is_thinint) {
> + ret = virtio_ccw_register_adapter_ind(vcdev, vqs, nvqs, ccw);
> + if (ret)
> + /* no error, just fall back to legacy interrupts */
> + vcdev->is_thinint = 0;
> + }
> + if (!vcdev->is_thinint) {
> + /* Register queue indicators with host. */
> + vcdev->indicators = 0;
> + ccw->cmd_code = CCW_CMD_SET_IND;
> + ccw->flags = 0;
> + ccw->count = sizeof(vcdev->indicators);
> + ccw->cda = (__u32)(unsigned long) indicatorp;
> + ret = ccw_io_helper(vcdev, ccw, VIRTIO_CCW_DOING_SET_IND);
> + if (ret)
> + goto out;
> + }
> /* Register indicators2 with host for config changes */
> *indicatorp = (unsigned long) &vcdev->indicators2;
> vcdev->indicators2 = 0;
> @@ -663,6 +912,7 @@ static void virtio_ccw_int_handler(struct ccw_device *cdev,
> case VIRTIO_CCW_DOING_SET_CONF_IND:
> case VIRTIO_CCW_DOING_RESET:
> case VIRTIO_CCW_DOING_READ_VQ_CONF:
> + case VIRTIO_CCW_DOING_SET_IND_ADAPTER:
> vcdev->curr_io &= ~activity;
> wake_up(&vcdev->wait_q);
> break;
> @@ -778,6 +1028,8 @@ static int virtio_ccw_online(struct ccw_device *cdev)
> goto out_free;
> }
>
> + vcdev->is_thinint = virtio_ccw_use_airq; /* at least try */
> +
> vcdev->vdev.dev.parent = &cdev->dev;
> vcdev->vdev.dev.release = virtio_ccw_release_dev;
> vcdev->vdev.config = &virtio_ccw_config_ops;
> @@ -935,6 +1187,10 @@ module_init(virtio_ccw_init);
>
> static void __exit virtio_ccw_exit(void)
> {
> + int i;
> +
> ccw_driver_unregister(&virtio_ccw_driver);
> + for (i = 0; i < MAX_AIRQ_AREAS; i++)
> + destroy_airq_info(airq_areas[i]);
> }
> module_exit(virtio_ccw_exit);
>
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2014-03-03 20:48 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-25 17:24 [PATCH 0/7] KVM: irqfds for s390 Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:24 ` [PATCH 1/7] s390/airq: add support for irq ranges Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:24 ` [PATCH 2/7] KVM: s390: virtio-ccw adapter interrupt support Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-03-03 20:47 ` Christian Borntraeger
2014-03-03 20:47 ` [Qemu-devel] " Christian Borntraeger
2014-02-25 17:24 ` [PATCH 3/7] KVM: eventfd: Fix lock order inversion Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:24 ` [PATCH 4/7] KVM: Add per-vm capability enablement Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:24 ` [PATCH 5/7] KVM: s390: adapter interrupt sources Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:24 ` [PATCH 6/7] KVM: s390: irq routing for adapter interrupts Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
2014-02-25 17:24 ` [PATCH 7/7] KVM: Bump KVM_MAX_IRQ_ROUTES for s390 Cornelia Huck
2014-02-25 17:24 ` [Qemu-devel] " Cornelia Huck
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.