All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 00/12] KVM: arm/arm64: move VGIC MMIO to kvm_io_bus
@ 2015-03-13 16:10 ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

This series converts the VGIC MMIO handling routines to the generic
kvm_io_bus framework. The framework is needed for the ioeventfd
functionality, some people on the list wanted to see the VGIC
converted over to use it, too.
Looking at the diffstat it doesn't look too useful after all, but
it cleans up the code a bit and gets rid of an ugly ping-pong
between vgic.c and vgic-v[23]-emul.c.

It's based on Nikolay's work[1], thanks especially for the tedious
first patch.
I totally reworked Nikolay's 3/5. I don't see much point in adding
another MMIO handling layer on top of the already quite convoluted
VGIC MMIO handling.
Instead I let the kvm_io_bus call into the final MMIO dispatcher of
the VGIC directly, which redirects the access to the actual register
handlers. We would loose quite some checking and some neatness of the
current functionality if that would be skipped, so I kept this in.

Unfortunately kvm_io_bus lacks an opaque pointer to pass in some data,
so I worked around this by using container_of.
Now for every struct kvm_mmio_range array a KVM I/O device is
registered (one for VGICv2, 2*nr_vcpus + 1 for VGICv3), using the
struct kvm_io_device variable as an anchor into the new
struct vgic_io_device. This one holds the base address, the
vgic_io_range pointer and (in case of the GICv3 redistributor) the
associated vCPU, so that we can access all instance-specific data
easily.

Patch 2 moves the iodev.h header file around, that solves a problem
when embedding a struct in arm_vgic.h later. That looks like a nice
cleanup anyway, so I added two patches to remove the compiler switch
to add virt/kvm as a include directory. This has been tested for
arm/arm64 and x86. As soon as I get around to compile-test the other
architectures, I can send out the respective patches for those, too.

Patches 5-7 tweak the existing code a bit to make it fit for the
conversion.
Patch 8 contains the framework for the new handling, while
patch 9 and 10 enable the GICv2 and GICv3 emulation, respectively.
Patch 11 finally switches over to the new kvm_io_bus handling (this
is basically Nikolay's 2/5 with some additions), patch 12 removes the
now unneeded code. I split this up to ease reviewing, I could merge
patches as well if needed.

The series goes on top of the kvmarm.git/next branch and was briefly
tested on an arm64 model with a GICv2 and a GICv3 guest and on Midway
(GICv2 guest).

Cheers,
Andre.

[1] https://lists.cs.columbia.edu/pipermail/kvmarm/2015-January/013379.html

Andre Przywara (10):
  KVM: move iodev.h from virt/kvm/ to include/kvm
  KVM: arm/arm64: remove now unneeded include directory from Makefile
  KVM: x86: remove now unneeded include directory from Makefile
  KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range
  KVM: mark kvm->buses as empty once they were destroyed
  KVM: arm/arm64: simplify vgic_find_range() and callers
  KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
  KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO
    handling
  KVM: arm/arm64: remove now obsolete VGIC specific MMIO handling code

Nikolay Nikolaev (2):
  KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the
    callbacks.
  KVM: ARM: on IO mem abort - route the call to KVM MMIO bus

 arch/arm/kvm/Makefile       |    2 +-
 arch/arm/kvm/mmio.c         |   32 +++++++-
 arch/arm64/kvm/Makefile     |    2 +-
 arch/powerpc/kvm/mpic.c     |   12 +--
 arch/powerpc/kvm/powerpc.c  |    4 +-
 arch/s390/kvm/diag.c        |    2 +-
 arch/x86/kvm/Makefile       |    2 +-
 arch/x86/kvm/i8254.c        |   14 ++--
 arch/x86/kvm/i8254.h        |    2 +-
 arch/x86/kvm/i8259.c        |   12 +--
 arch/x86/kvm/ioapic.c       |    8 +-
 arch/x86/kvm/ioapic.h       |    2 +-
 arch/x86/kvm/irq.h          |    2 +-
 arch/x86/kvm/lapic.c        |    4 +-
 arch/x86/kvm/lapic.h        |    2 +-
 arch/x86/kvm/vmx.c          |    2 +-
 arch/x86/kvm/x86.c          |   13 +--
 include/kvm/arm_vgic.h      |   15 +++-
 include/kvm/iodev.h         |   76 ++++++++++++++++++
 include/linux/kvm_host.h    |   10 +--
 virt/kvm/arm/vgic-v2-emul.c |   40 +++-------
 virt/kvm/arm/vgic-v3-emul.c |   77 ++++++++----------
 virt/kvm/arm/vgic.c         |  186 +++++++++++++++++++++++++++++--------------
 virt/kvm/arm/vgic.h         |   21 ++---
 virt/kvm/coalesced_mmio.c   |    7 +-
 virt/kvm/eventfd.c          |    6 +-
 virt/kvm/iodev.h            |   70 ----------------
 virt/kvm/kvm_main.c         |   38 ++++-----
 28 files changed, 379 insertions(+), 284 deletions(-)
 create mode 100644 include/kvm/iodev.h
 delete mode 100644 virt/kvm/iodev.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 00/12] KVM: arm/arm64: move VGIC MMIO to kvm_io_bus
@ 2015-03-13 16:10 ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

This series converts the VGIC MMIO handling routines to the generic
kvm_io_bus framework. The framework is needed for the ioeventfd
functionality, some people on the list wanted to see the VGIC
converted over to use it, too.
Looking at the diffstat it doesn't look too useful after all, but
it cleans up the code a bit and gets rid of an ugly ping-pong
between vgic.c and vgic-v[23]-emul.c.

It's based on Nikolay's work[1], thanks especially for the tedious
first patch.
I totally reworked Nikolay's 3/5. I don't see much point in adding
another MMIO handling layer on top of the already quite convoluted
VGIC MMIO handling.
Instead I let the kvm_io_bus call into the final MMIO dispatcher of
the VGIC directly, which redirects the access to the actual register
handlers. We would loose quite some checking and some neatness of the
current functionality if that would be skipped, so I kept this in.

Unfortunately kvm_io_bus lacks an opaque pointer to pass in some data,
so I worked around this by using container_of.
Now for every struct kvm_mmio_range array a KVM I/O device is
registered (one for VGICv2, 2*nr_vcpus + 1 for VGICv3), using the
struct kvm_io_device variable as an anchor into the new
struct vgic_io_device. This one holds the base address, the
vgic_io_range pointer and (in case of the GICv3 redistributor) the
associated vCPU, so that we can access all instance-specific data
easily.

Patch 2 moves the iodev.h header file around, that solves a problem
when embedding a struct in arm_vgic.h later. That looks like a nice
cleanup anyway, so I added two patches to remove the compiler switch
to add virt/kvm as a include directory. This has been tested for
arm/arm64 and x86. As soon as I get around to compile-test the other
architectures, I can send out the respective patches for those, too.

Patches 5-7 tweak the existing code a bit to make it fit for the
conversion.
Patch 8 contains the framework for the new handling, while
patch 9 and 10 enable the GICv2 and GICv3 emulation, respectively.
Patch 11 finally switches over to the new kvm_io_bus handling (this
is basically Nikolay's 2/5 with some additions), patch 12 removes the
now unneeded code. I split this up to ease reviewing, I could merge
patches as well if needed.

The series goes on top of the kvmarm.git/next branch and was briefly
tested on an arm64 model with a GICv2 and a GICv3 guest and on Midway
(GICv2 guest).

Cheers,
Andre.

[1] https://lists.cs.columbia.edu/pipermail/kvmarm/2015-January/013379.html

Andre Przywara (10):
  KVM: move iodev.h from virt/kvm/ to include/kvm
  KVM: arm/arm64: remove now unneeded include directory from Makefile
  KVM: x86: remove now unneeded include directory from Makefile
  KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range
  KVM: mark kvm->buses as empty once they were destroyed
  KVM: arm/arm64: simplify vgic_find_range() and callers
  KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
  KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO
    handling
  KVM: arm/arm64: remove now obsolete VGIC specific MMIO handling code

Nikolay Nikolaev (2):
  KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the
    callbacks.
  KVM: ARM: on IO mem abort - route the call to KVM MMIO bus

 arch/arm/kvm/Makefile       |    2 +-
 arch/arm/kvm/mmio.c         |   32 +++++++-
 arch/arm64/kvm/Makefile     |    2 +-
 arch/powerpc/kvm/mpic.c     |   12 +--
 arch/powerpc/kvm/powerpc.c  |    4 +-
 arch/s390/kvm/diag.c        |    2 +-
 arch/x86/kvm/Makefile       |    2 +-
 arch/x86/kvm/i8254.c        |   14 ++--
 arch/x86/kvm/i8254.h        |    2 +-
 arch/x86/kvm/i8259.c        |   12 +--
 arch/x86/kvm/ioapic.c       |    8 +-
 arch/x86/kvm/ioapic.h       |    2 +-
 arch/x86/kvm/irq.h          |    2 +-
 arch/x86/kvm/lapic.c        |    4 +-
 arch/x86/kvm/lapic.h        |    2 +-
 arch/x86/kvm/vmx.c          |    2 +-
 arch/x86/kvm/x86.c          |   13 +--
 include/kvm/arm_vgic.h      |   15 +++-
 include/kvm/iodev.h         |   76 ++++++++++++++++++
 include/linux/kvm_host.h    |   10 +--
 virt/kvm/arm/vgic-v2-emul.c |   40 +++-------
 virt/kvm/arm/vgic-v3-emul.c |   77 ++++++++----------
 virt/kvm/arm/vgic.c         |  186 +++++++++++++++++++++++++++++--------------
 virt/kvm/arm/vgic.h         |   21 ++---
 virt/kvm/coalesced_mmio.c   |    7 +-
 virt/kvm/eventfd.c          |    6 +-
 virt/kvm/iodev.h            |   70 ----------------
 virt/kvm/kvm_main.c         |   38 ++++-----
 28 files changed, 379 insertions(+), 284 deletions(-)
 create mode 100644 include/kvm/iodev.h
 delete mode 100644 virt/kvm/iodev.h

-- 
1.7.9.5

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 01/12] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks.
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>

This is needed in e.g. ARM vGIC emulation, where the MMIO handling
depends on the VCPU that does the access.

Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Paolo Bonzini <pbonzini at redhat.com>
---
 arch/powerpc/kvm/mpic.c    |   10 ++++++----
 arch/powerpc/kvm/powerpc.c |    4 ++--
 arch/s390/kvm/diag.c       |    2 +-
 arch/x86/kvm/i8254.c       |   14 +++++++++-----
 arch/x86/kvm/i8259.c       |   12 ++++++------
 arch/x86/kvm/ioapic.c      |    8 ++++----
 arch/x86/kvm/lapic.c       |    4 ++--
 arch/x86/kvm/vmx.c         |    2 +-
 arch/x86/kvm/x86.c         |   13 +++++++------
 include/linux/kvm_host.h   |   10 +++++-----
 virt/kvm/coalesced_mmio.c  |    5 +++--
 virt/kvm/eventfd.c         |    4 ++--
 virt/kvm/iodev.h           |   23 +++++++++++++++--------
 virt/kvm/kvm_main.c        |   32 ++++++++++++++++----------------
 14 files changed, 79 insertions(+), 64 deletions(-)

diff --git a/arch/powerpc/kvm/mpic.c b/arch/powerpc/kvm/mpic.c
index 39b3a8f..8542f07 100644
--- a/arch/powerpc/kvm/mpic.c
+++ b/arch/powerpc/kvm/mpic.c
@@ -1374,8 +1374,9 @@ static int kvm_mpic_write_internal(struct openpic *opp, gpa_t addr, u32 val)
 	return -ENXIO;
 }
 
-static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
-			 int len, void *ptr)
+static int kvm_mpic_read(struct kvm_vcpu *vcpu,
+			 struct kvm_io_device *this,
+			 gpa_t addr, int len, void *ptr)
 {
 	struct openpic *opp = container_of(this, struct openpic, mmio);
 	int ret;
@@ -1415,8 +1416,9 @@ static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
 	return ret;
 }
 
-static int kvm_mpic_write(struct kvm_io_device *this, gpa_t addr,
-			  int len, const void *ptr)
+static int kvm_mpic_write(struct kvm_vcpu *vcpu,
+			  struct kvm_io_device *this,
+			  gpa_t addr, int len, const void *ptr)
 {
 	struct openpic *opp = container_of(this, struct openpic, mmio);
 	int ret;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 27c0fac..24bfe40 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -807,7 +807,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 
-	ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+	ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
 			      bytes, &run->mmio.data);
 
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
@@ -880,7 +880,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 
-	ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+	ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
 			       bytes, &run->mmio.data);
 
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 9254aff..329ec75 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -213,7 +213,7 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
 	 * - gpr 3 contains the virtqueue index (passed as datamatch)
 	 * - gpr 4 contains the index on the bus (optionally)
 	 */
-	ret = kvm_io_bus_write_cookie(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
+	ret = kvm_io_bus_write_cookie(vcpu, KVM_VIRTIO_CCW_NOTIFY_BUS,
 				      vcpu->run->s.regs.gprs[2] & 0xffffffff,
 				      8, &vcpu->run->s.regs.gprs[3],
 				      vcpu->run->s.regs.gprs[4]);
diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c
index 298781d..4dce6f8 100644
--- a/arch/x86/kvm/i8254.c
+++ b/arch/x86/kvm/i8254.c
@@ -443,7 +443,8 @@ static inline int pit_in_range(gpa_t addr)
 		(addr < KVM_PIT_BASE_ADDRESS + KVM_PIT_MEM_LENGTH));
 }
 
-static int pit_ioport_write(struct kvm_io_device *this,
+static int pit_ioport_write(struct kvm_vcpu *vcpu,
+				struct kvm_io_device *this,
 			    gpa_t addr, int len, const void *data)
 {
 	struct kvm_pit *pit = dev_to_pit(this);
@@ -519,7 +520,8 @@ static int pit_ioport_write(struct kvm_io_device *this,
 	return 0;
 }
 
-static int pit_ioport_read(struct kvm_io_device *this,
+static int pit_ioport_read(struct kvm_vcpu *vcpu,
+			   struct kvm_io_device *this,
 			   gpa_t addr, int len, void *data)
 {
 	struct kvm_pit *pit = dev_to_pit(this);
@@ -589,7 +591,8 @@ static int pit_ioport_read(struct kvm_io_device *this,
 	return 0;
 }
 
-static int speaker_ioport_write(struct kvm_io_device *this,
+static int speaker_ioport_write(struct kvm_vcpu *vcpu,
+				struct kvm_io_device *this,
 				gpa_t addr, int len, const void *data)
 {
 	struct kvm_pit *pit = speaker_to_pit(this);
@@ -606,8 +609,9 @@ static int speaker_ioport_write(struct kvm_io_device *this,
 	return 0;
 }
 
-static int speaker_ioport_read(struct kvm_io_device *this,
-			       gpa_t addr, int len, void *data)
+static int speaker_ioport_read(struct kvm_vcpu *vcpu,
+				   struct kvm_io_device *this,
+				   gpa_t addr, int len, void *data)
 {
 	struct kvm_pit *pit = speaker_to_pit(this);
 	struct kvm_kpit_state *pit_state = &pit->pit_state;
diff --git a/arch/x86/kvm/i8259.c b/arch/x86/kvm/i8259.c
index cc31f7c..8ff4eaa 100644
--- a/arch/x86/kvm/i8259.c
+++ b/arch/x86/kvm/i8259.c
@@ -528,42 +528,42 @@ static int picdev_read(struct kvm_pic *s,
 	return 0;
 }
 
-static int picdev_master_write(struct kvm_io_device *dev,
+static int picdev_master_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			       gpa_t addr, int len, const void *val)
 {
 	return picdev_write(container_of(dev, struct kvm_pic, dev_master),
 			    addr, len, val);
 }
 
-static int picdev_master_read(struct kvm_io_device *dev,
+static int picdev_master_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			      gpa_t addr, int len, void *val)
 {
 	return picdev_read(container_of(dev, struct kvm_pic, dev_master),
 			    addr, len, val);
 }
 
-static int picdev_slave_write(struct kvm_io_device *dev,
+static int picdev_slave_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			      gpa_t addr, int len, const void *val)
 {
 	return picdev_write(container_of(dev, struct kvm_pic, dev_slave),
 			    addr, len, val);
 }
 
-static int picdev_slave_read(struct kvm_io_device *dev,
+static int picdev_slave_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			     gpa_t addr, int len, void *val)
 {
 	return picdev_read(container_of(dev, struct kvm_pic, dev_slave),
 			    addr, len, val);
 }
 
-static int picdev_eclr_write(struct kvm_io_device *dev,
+static int picdev_eclr_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			     gpa_t addr, int len, const void *val)
 {
 	return picdev_write(container_of(dev, struct kvm_pic, dev_eclr),
 			    addr, len, val);
 }
 
-static int picdev_eclr_read(struct kvm_io_device *dev,
+static int picdev_eclr_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			    gpa_t addr, int len, void *val)
 {
 	return picdev_read(container_of(dev, struct kvm_pic, dev_eclr),
diff --git a/arch/x86/kvm/ioapic.c b/arch/x86/kvm/ioapic.c
index b1947e0..8bf2e49 100644
--- a/arch/x86/kvm/ioapic.c
+++ b/arch/x86/kvm/ioapic.c
@@ -498,8 +498,8 @@ static inline int ioapic_in_range(struct kvm_ioapic *ioapic, gpa_t addr)
 		 (addr < ioapic->base_address + IOAPIC_MEM_LENGTH)));
 }
 
-static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
-			    void *val)
+static int ioapic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+				gpa_t addr, int len, void *val)
 {
 	struct kvm_ioapic *ioapic = to_ioapic(this);
 	u32 result;
@@ -541,8 +541,8 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	return 0;
 }
 
-static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
-			     const void *val)
+static int ioapic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+				 gpa_t addr, int len, const void *val)
 {
 	struct kvm_ioapic *ioapic = to_ioapic(this);
 	u32 data;
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index e55b5fc..ba57bb7 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1038,7 +1038,7 @@ static int apic_mmio_in_range(struct kvm_lapic *apic, gpa_t addr)
 	    addr < apic->base_address + LAPIC_MMIO_LENGTH;
 }
 
-static int apic_mmio_read(struct kvm_io_device *this,
+static int apic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
 			   gpa_t address, int len, void *data)
 {
 	struct kvm_lapic *apic = to_lapic(this);
@@ -1358,7 +1358,7 @@ static int apic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)
 	return ret;
 }
 
-static int apic_mmio_write(struct kvm_io_device *this,
+static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
 			    gpa_t address, int len, const void *data)
 {
 	struct kvm_lapic *apic = to_lapic(this);
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f7b20b4..317da9b 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5822,7 +5822,7 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu)
 	gpa_t gpa;
 
 	gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
-	if (!kvm_io_bus_write(vcpu->kvm, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
+	if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
 		skip_emulated_instruction(vcpu);
 		return 1;
 	}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bd7a70b..5573d63 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4115,8 +4115,8 @@ static int vcpu_mmio_write(struct kvm_vcpu *vcpu, gpa_t addr, int len,
 	do {
 		n = min(len, 8);
 		if (!(vcpu->arch.apic &&
-		      !kvm_iodevice_write(&vcpu->arch.apic->dev, addr, n, v))
-		    && kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+		      !kvm_iodevice_write(vcpu, &vcpu->arch.apic->dev, addr, n, v))
+		    && kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, n, v))
 			break;
 		handled += n;
 		addr += n;
@@ -4135,8 +4135,9 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t addr, int len, void *v)
 	do {
 		n = min(len, 8);
 		if (!(vcpu->arch.apic &&
-		      !kvm_iodevice_read(&vcpu->arch.apic->dev, addr, n, v))
-		    && kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+		      !kvm_iodevice_read(vcpu, &vcpu->arch.apic->dev,
+					 addr, n, v))
+		    && kvm_io_bus_read(vcpu, KVM_MMIO_BUS, addr, n, v))
 			break;
 		trace_kvm_mmio(KVM_TRACE_MMIO_READ, n, addr, *(u64 *)v);
 		handled += n;
@@ -4630,10 +4631,10 @@ static int kernel_pio(struct kvm_vcpu *vcpu, void *pd)
 	int r;
 
 	if (vcpu->arch.pio.in)
-		r = kvm_io_bus_read(vcpu->kvm, KVM_PIO_BUS, vcpu->arch.pio.port,
+		r = kvm_io_bus_read(vcpu, KVM_PIO_BUS, vcpu->arch.pio.port,
 				    vcpu->arch.pio.size, pd);
 	else
-		r = kvm_io_bus_write(vcpu->kvm, KVM_PIO_BUS,
+		r = kvm_io_bus_write(vcpu, KVM_PIO_BUS,
 				     vcpu->arch.pio.port, vcpu->arch.pio.size,
 				     pd);
 	return r;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index ae9c720..9605e46 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -165,12 +165,12 @@ enum kvm_bus {
 	KVM_NR_BUSES
 };
 
-int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 		     int len, const void *val);
-int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
-			    int len, const void *val, long cookie);
-int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, int len,
-		    void *val);
+int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
+			    gpa_t addr, int len, const void *val, long cookie);
+int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
+		    int len, void *val);
 int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 			    int len, struct kvm_io_device *dev);
 int kvm_io_bus_unregister_dev(struct kvm *kvm, enum kvm_bus bus_idx,
diff --git a/virt/kvm/coalesced_mmio.c b/virt/kvm/coalesced_mmio.c
index 00d8642..c831a40 100644
--- a/virt/kvm/coalesced_mmio.c
+++ b/virt/kvm/coalesced_mmio.c
@@ -60,8 +60,9 @@ static int coalesced_mmio_has_room(struct kvm_coalesced_mmio_dev *dev)
 	return 1;
 }
 
-static int coalesced_mmio_write(struct kvm_io_device *this,
-				gpa_t addr, int len, const void *val)
+static int coalesced_mmio_write(struct kvm_vcpu *vcpu,
+				struct kvm_io_device *this, gpa_t addr,
+				int len, const void *val)
 {
 	struct kvm_coalesced_mmio_dev *dev = to_mmio(this);
 	struct kvm_coalesced_mmio_ring *ring = dev->kvm->coalesced_mmio_ring;
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index fc5f43e..26c72f3 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -715,8 +715,8 @@ ioeventfd_in_range(struct _ioeventfd *p, gpa_t addr, int len, const void *val)
 
 /* MMIO/PIO writes trigger an event if the addr/val match */
 static int
-ioeventfd_write(struct kvm_io_device *this, gpa_t addr, int len,
-		const void *val)
+ioeventfd_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this, gpa_t addr,
+		int len, const void *val)
 {
 	struct _ioeventfd *p = to_ioeventfd(this);
 
diff --git a/virt/kvm/iodev.h b/virt/kvm/iodev.h
index 12fd3ca..9ef709c 100644
--- a/virt/kvm/iodev.h
+++ b/virt/kvm/iodev.h
@@ -20,6 +20,7 @@
 #include <asm/errno.h>
 
 struct kvm_io_device;
+struct kvm_vcpu;
 
 /**
  * kvm_io_device_ops are called under kvm slots_lock.
@@ -27,11 +28,13 @@ struct kvm_io_device;
  * or non-zero to have it passed to the next device.
  **/
 struct kvm_io_device_ops {
-	int (*read)(struct kvm_io_device *this,
+	int (*read)(struct kvm_vcpu *vcpu,
+		    struct kvm_io_device *this,
 		    gpa_t addr,
 		    int len,
 		    void *val);
-	int (*write)(struct kvm_io_device *this,
+	int (*write)(struct kvm_vcpu *vcpu,
+		     struct kvm_io_device *this,
 		     gpa_t addr,
 		     int len,
 		     const void *val);
@@ -49,16 +52,20 @@ static inline void kvm_iodevice_init(struct kvm_io_device *dev,
 	dev->ops = ops;
 }
 
-static inline int kvm_iodevice_read(struct kvm_io_device *dev,
-				    gpa_t addr, int l, void *v)
+static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
+				    struct kvm_io_device *dev, gpa_t addr,
+				    int l, void *v)
 {
-	return dev->ops->read ? dev->ops->read(dev, addr, l, v) : -EOPNOTSUPP;
+	return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
+				: -EOPNOTSUPP;
 }
 
-static inline int kvm_iodevice_write(struct kvm_io_device *dev,
-				     gpa_t addr, int l, const void *v)
+static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
+				     struct kvm_io_device *dev, gpa_t addr,
+				     int l, const void *v)
 {
-	return dev->ops->write ? dev->ops->write(dev, addr, l, v) : -EOPNOTSUPP;
+	return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
+				 : -EOPNOTSUPP;
 }
 
 static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index a109370..ce86493 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2997,7 +2997,7 @@ static int kvm_io_bus_get_first_dev(struct kvm_io_bus *bus,
 	return off;
 }
 
-static int __kvm_io_bus_write(struct kvm_io_bus *bus,
+static int __kvm_io_bus_write(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
 			      struct kvm_io_range *range, const void *val)
 {
 	int idx;
@@ -3008,7 +3008,7 @@ static int __kvm_io_bus_write(struct kvm_io_bus *bus,
 
 	while (idx < bus->dev_count &&
 		kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
-		if (!kvm_iodevice_write(bus->range[idx].dev, range->addr,
+		if (!kvm_iodevice_write(vcpu, bus->range[idx].dev, range->addr,
 					range->len, val))
 			return idx;
 		idx++;
@@ -3018,7 +3018,7 @@ static int __kvm_io_bus_write(struct kvm_io_bus *bus,
 }
 
 /* kvm_io_bus_write - called under kvm->slots_lock */
-int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 		     int len, const void *val)
 {
 	struct kvm_io_bus *bus;
@@ -3030,14 +3030,14 @@ int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		.len = len,
 	};
 
-	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
-	r = __kvm_io_bus_write(bus, &range, val);
+	bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &kvm->srcu);
+	r = __kvm_io_bus_write(vcpu, bus, &range, val);
 	return r < 0 ? r : 0;
 }
 
 /* kvm_io_bus_write_cookie - called under kvm->slots_lock */
-int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
-			    int len, const void *val, long cookie)
+int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
+			    gpa_t addr, int len, const void *val, long cookie)
 {
 	struct kvm_io_bus *bus;
 	struct kvm_io_range range;
@@ -3047,12 +3047,12 @@ int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		.len = len,
 	};
 
-	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
+	bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &kvm->srcu);
 
 	/* First try the device referenced by cookie. */
 	if ((cookie >= 0) && (cookie < bus->dev_count) &&
 	    (kvm_io_bus_cmp(&range, &bus->range[cookie]) == 0))
-		if (!kvm_iodevice_write(bus->range[cookie].dev, addr, len,
+		if (!kvm_iodevice_write(vcpu, bus->range[cookie].dev, addr, len,
 					val))
 			return cookie;
 
@@ -3060,11 +3060,11 @@ int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 	 * cookie contained garbage; fall back to search and return the
 	 * correct cookie value.
 	 */
-	return __kvm_io_bus_write(bus, &range, val);
+	return __kvm_io_bus_write(vcpu, bus, &range, val);
 }
 
-static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
-			     void *val)
+static int __kvm_io_bus_read(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
+			     struct kvm_io_range *range, void *val)
 {
 	int idx;
 
@@ -3074,7 +3074,7 @@ static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
 
 	while (idx < bus->dev_count &&
 		kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
-		if (!kvm_iodevice_read(bus->range[idx].dev, range->addr,
+		if (!kvm_iodevice_read(vcpu, bus->range[idx].dev, range->addr,
 				       range->len, val))
 			return idx;
 		idx++;
@@ -3085,7 +3085,7 @@ static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
 EXPORT_SYMBOL_GPL(kvm_io_bus_write);
 
 /* kvm_io_bus_read - called under kvm->slots_lock */
-int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 		    int len, void *val)
 {
 	struct kvm_io_bus *bus;
@@ -3097,8 +3097,8 @@ int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		.len = len,
 	};
 
-	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
-	r = __kvm_io_bus_read(bus, &range, val);
+	bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &kvm->srcu);
+	r = __kvm_io_bus_read(vcpu, bus, &range, val);
 	return r < 0 ? r : 0;
 }
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 01/12] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks.
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>

This is needed in e.g. ARM vGIC emulation, where the MMIO handling
depends on the VCPU that does the access.

Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
---
 arch/powerpc/kvm/mpic.c    |   10 ++++++----
 arch/powerpc/kvm/powerpc.c |    4 ++--
 arch/s390/kvm/diag.c       |    2 +-
 arch/x86/kvm/i8254.c       |   14 +++++++++-----
 arch/x86/kvm/i8259.c       |   12 ++++++------
 arch/x86/kvm/ioapic.c      |    8 ++++----
 arch/x86/kvm/lapic.c       |    4 ++--
 arch/x86/kvm/vmx.c         |    2 +-
 arch/x86/kvm/x86.c         |   13 +++++++------
 include/linux/kvm_host.h   |   10 +++++-----
 virt/kvm/coalesced_mmio.c  |    5 +++--
 virt/kvm/eventfd.c         |    4 ++--
 virt/kvm/iodev.h           |   23 +++++++++++++++--------
 virt/kvm/kvm_main.c        |   32 ++++++++++++++++----------------
 14 files changed, 79 insertions(+), 64 deletions(-)

diff --git a/arch/powerpc/kvm/mpic.c b/arch/powerpc/kvm/mpic.c
index 39b3a8f..8542f07 100644
--- a/arch/powerpc/kvm/mpic.c
+++ b/arch/powerpc/kvm/mpic.c
@@ -1374,8 +1374,9 @@ static int kvm_mpic_write_internal(struct openpic *opp, gpa_t addr, u32 val)
 	return -ENXIO;
 }
 
-static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
-			 int len, void *ptr)
+static int kvm_mpic_read(struct kvm_vcpu *vcpu,
+			 struct kvm_io_device *this,
+			 gpa_t addr, int len, void *ptr)
 {
 	struct openpic *opp = container_of(this, struct openpic, mmio);
 	int ret;
@@ -1415,8 +1416,9 @@ static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
 	return ret;
 }
 
-static int kvm_mpic_write(struct kvm_io_device *this, gpa_t addr,
-			  int len, const void *ptr)
+static int kvm_mpic_write(struct kvm_vcpu *vcpu,
+			  struct kvm_io_device *this,
+			  gpa_t addr, int len, const void *ptr)
 {
 	struct openpic *opp = container_of(this, struct openpic, mmio);
 	int ret;
diff --git a/arch/powerpc/kvm/powerpc.c b/arch/powerpc/kvm/powerpc.c
index 27c0fac..24bfe40 100644
--- a/arch/powerpc/kvm/powerpc.c
+++ b/arch/powerpc/kvm/powerpc.c
@@ -807,7 +807,7 @@ int kvmppc_handle_load(struct kvm_run *run, struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 
-	ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+	ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
 			      bytes, &run->mmio.data);
 
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
@@ -880,7 +880,7 @@ int kvmppc_handle_store(struct kvm_run *run, struct kvm_vcpu *vcpu,
 
 	idx = srcu_read_lock(&vcpu->kvm->srcu);
 
-	ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+	ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
 			       bytes, &run->mmio.data);
 
 	srcu_read_unlock(&vcpu->kvm->srcu, idx);
diff --git a/arch/s390/kvm/diag.c b/arch/s390/kvm/diag.c
index 9254aff..329ec75 100644
--- a/arch/s390/kvm/diag.c
+++ b/arch/s390/kvm/diag.c
@@ -213,7 +213,7 @@ static int __diag_virtio_hypercall(struct kvm_vcpu *vcpu)
 	 * - gpr 3 contains the virtqueue index (passed as datamatch)
 	 * - gpr 4 contains the index on the bus (optionally)
 	 */
-	ret = kvm_io_bus_write_cookie(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
+	ret = kvm_io_bus_write_cookie(vcpu, KVM_VIRTIO_CCW_NOTIFY_BUS,
 				      vcpu->run->s.regs.gprs[2] & 0xffffffff,
 				      8, &vcpu->run->s.regs.gprs[3],
 				      vcpu->run->s.regs.gprs[4]);
diff --git a/arch/x86/kvm/i8254.c b/arch/x86/kvm/i8254.c
index 298781d..4dce6f8 100644
--- a/arch/x86/kvm/i8254.c
+++ b/arch/x86/kvm/i8254.c
@@ -443,7 +443,8 @@ static inline int pit_in_range(gpa_t addr)
 		(addr < KVM_PIT_BASE_ADDRESS + KVM_PIT_MEM_LENGTH));
 }
 
-static int pit_ioport_write(struct kvm_io_device *this,
+static int pit_ioport_write(struct kvm_vcpu *vcpu,
+				struct kvm_io_device *this,
 			    gpa_t addr, int len, const void *data)
 {
 	struct kvm_pit *pit = dev_to_pit(this);
@@ -519,7 +520,8 @@ static int pit_ioport_write(struct kvm_io_device *this,
 	return 0;
 }
 
-static int pit_ioport_read(struct kvm_io_device *this,
+static int pit_ioport_read(struct kvm_vcpu *vcpu,
+			   struct kvm_io_device *this,
 			   gpa_t addr, int len, void *data)
 {
 	struct kvm_pit *pit = dev_to_pit(this);
@@ -589,7 +591,8 @@ static int pit_ioport_read(struct kvm_io_device *this,
 	return 0;
 }
 
-static int speaker_ioport_write(struct kvm_io_device *this,
+static int speaker_ioport_write(struct kvm_vcpu *vcpu,
+				struct kvm_io_device *this,
 				gpa_t addr, int len, const void *data)
 {
 	struct kvm_pit *pit = speaker_to_pit(this);
@@ -606,8 +609,9 @@ static int speaker_ioport_write(struct kvm_io_device *this,
 	return 0;
 }
 
-static int speaker_ioport_read(struct kvm_io_device *this,
-			       gpa_t addr, int len, void *data)
+static int speaker_ioport_read(struct kvm_vcpu *vcpu,
+				   struct kvm_io_device *this,
+				   gpa_t addr, int len, void *data)
 {
 	struct kvm_pit *pit = speaker_to_pit(this);
 	struct kvm_kpit_state *pit_state = &pit->pit_state;
diff --git a/arch/x86/kvm/i8259.c b/arch/x86/kvm/i8259.c
index cc31f7c..8ff4eaa 100644
--- a/arch/x86/kvm/i8259.c
+++ b/arch/x86/kvm/i8259.c
@@ -528,42 +528,42 @@ static int picdev_read(struct kvm_pic *s,
 	return 0;
 }
 
-static int picdev_master_write(struct kvm_io_device *dev,
+static int picdev_master_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			       gpa_t addr, int len, const void *val)
 {
 	return picdev_write(container_of(dev, struct kvm_pic, dev_master),
 			    addr, len, val);
 }
 
-static int picdev_master_read(struct kvm_io_device *dev,
+static int picdev_master_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			      gpa_t addr, int len, void *val)
 {
 	return picdev_read(container_of(dev, struct kvm_pic, dev_master),
 			    addr, len, val);
 }
 
-static int picdev_slave_write(struct kvm_io_device *dev,
+static int picdev_slave_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			      gpa_t addr, int len, const void *val)
 {
 	return picdev_write(container_of(dev, struct kvm_pic, dev_slave),
 			    addr, len, val);
 }
 
-static int picdev_slave_read(struct kvm_io_device *dev,
+static int picdev_slave_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			     gpa_t addr, int len, void *val)
 {
 	return picdev_read(container_of(dev, struct kvm_pic, dev_slave),
 			    addr, len, val);
 }
 
-static int picdev_eclr_write(struct kvm_io_device *dev,
+static int picdev_eclr_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			     gpa_t addr, int len, const void *val)
 {
 	return picdev_write(container_of(dev, struct kvm_pic, dev_eclr),
 			    addr, len, val);
 }
 
-static int picdev_eclr_read(struct kvm_io_device *dev,
+static int picdev_eclr_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
 			    gpa_t addr, int len, void *val)
 {
 	return picdev_read(container_of(dev, struct kvm_pic, dev_eclr),
diff --git a/arch/x86/kvm/ioapic.c b/arch/x86/kvm/ioapic.c
index b1947e0..8bf2e49 100644
--- a/arch/x86/kvm/ioapic.c
+++ b/arch/x86/kvm/ioapic.c
@@ -498,8 +498,8 @@ static inline int ioapic_in_range(struct kvm_ioapic *ioapic, gpa_t addr)
 		 (addr < ioapic->base_address + IOAPIC_MEM_LENGTH)));
 }
 
-static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
-			    void *val)
+static int ioapic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+				gpa_t addr, int len, void *val)
 {
 	struct kvm_ioapic *ioapic = to_ioapic(this);
 	u32 result;
@@ -541,8 +541,8 @@ static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
 	return 0;
 }
 
-static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
-			     const void *val)
+static int ioapic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+				 gpa_t addr, int len, const void *val)
 {
 	struct kvm_ioapic *ioapic = to_ioapic(this);
 	u32 data;
diff --git a/arch/x86/kvm/lapic.c b/arch/x86/kvm/lapic.c
index e55b5fc..ba57bb7 100644
--- a/arch/x86/kvm/lapic.c
+++ b/arch/x86/kvm/lapic.c
@@ -1038,7 +1038,7 @@ static int apic_mmio_in_range(struct kvm_lapic *apic, gpa_t addr)
 	    addr < apic->base_address + LAPIC_MMIO_LENGTH;
 }
 
-static int apic_mmio_read(struct kvm_io_device *this,
+static int apic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
 			   gpa_t address, int len, void *data)
 {
 	struct kvm_lapic *apic = to_lapic(this);
@@ -1358,7 +1358,7 @@ static int apic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)
 	return ret;
 }
 
-static int apic_mmio_write(struct kvm_io_device *this,
+static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
 			    gpa_t address, int len, const void *data)
 {
 	struct kvm_lapic *apic = to_lapic(this);
diff --git a/arch/x86/kvm/vmx.c b/arch/x86/kvm/vmx.c
index f7b20b4..317da9b 100644
--- a/arch/x86/kvm/vmx.c
+++ b/arch/x86/kvm/vmx.c
@@ -5822,7 +5822,7 @@ static int handle_ept_misconfig(struct kvm_vcpu *vcpu)
 	gpa_t gpa;
 
 	gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
-	if (!kvm_io_bus_write(vcpu->kvm, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
+	if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
 		skip_emulated_instruction(vcpu);
 		return 1;
 	}
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index bd7a70b..5573d63 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -4115,8 +4115,8 @@ static int vcpu_mmio_write(struct kvm_vcpu *vcpu, gpa_t addr, int len,
 	do {
 		n = min(len, 8);
 		if (!(vcpu->arch.apic &&
-		      !kvm_iodevice_write(&vcpu->arch.apic->dev, addr, n, v))
-		    && kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+		      !kvm_iodevice_write(vcpu, &vcpu->arch.apic->dev, addr, n, v))
+		    && kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, n, v))
 			break;
 		handled += n;
 		addr += n;
@@ -4135,8 +4135,9 @@ static int vcpu_mmio_read(struct kvm_vcpu *vcpu, gpa_t addr, int len, void *v)
 	do {
 		n = min(len, 8);
 		if (!(vcpu->arch.apic &&
-		      !kvm_iodevice_read(&vcpu->arch.apic->dev, addr, n, v))
-		    && kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+		      !kvm_iodevice_read(vcpu, &vcpu->arch.apic->dev,
+					 addr, n, v))
+		    && kvm_io_bus_read(vcpu, KVM_MMIO_BUS, addr, n, v))
 			break;
 		trace_kvm_mmio(KVM_TRACE_MMIO_READ, n, addr, *(u64 *)v);
 		handled += n;
@@ -4630,10 +4631,10 @@ static int kernel_pio(struct kvm_vcpu *vcpu, void *pd)
 	int r;
 
 	if (vcpu->arch.pio.in)
-		r = kvm_io_bus_read(vcpu->kvm, KVM_PIO_BUS, vcpu->arch.pio.port,
+		r = kvm_io_bus_read(vcpu, KVM_PIO_BUS, vcpu->arch.pio.port,
 				    vcpu->arch.pio.size, pd);
 	else
-		r = kvm_io_bus_write(vcpu->kvm, KVM_PIO_BUS,
+		r = kvm_io_bus_write(vcpu, KVM_PIO_BUS,
 				     vcpu->arch.pio.port, vcpu->arch.pio.size,
 				     pd);
 	return r;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index ae9c720..9605e46 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -165,12 +165,12 @@ enum kvm_bus {
 	KVM_NR_BUSES
 };
 
-int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 		     int len, const void *val);
-int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
-			    int len, const void *val, long cookie);
-int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, int len,
-		    void *val);
+int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
+			    gpa_t addr, int len, const void *val, long cookie);
+int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
+		    int len, void *val);
 int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 			    int len, struct kvm_io_device *dev);
 int kvm_io_bus_unregister_dev(struct kvm *kvm, enum kvm_bus bus_idx,
diff --git a/virt/kvm/coalesced_mmio.c b/virt/kvm/coalesced_mmio.c
index 00d8642..c831a40 100644
--- a/virt/kvm/coalesced_mmio.c
+++ b/virt/kvm/coalesced_mmio.c
@@ -60,8 +60,9 @@ static int coalesced_mmio_has_room(struct kvm_coalesced_mmio_dev *dev)
 	return 1;
 }
 
-static int coalesced_mmio_write(struct kvm_io_device *this,
-				gpa_t addr, int len, const void *val)
+static int coalesced_mmio_write(struct kvm_vcpu *vcpu,
+				struct kvm_io_device *this, gpa_t addr,
+				int len, const void *val)
 {
 	struct kvm_coalesced_mmio_dev *dev = to_mmio(this);
 	struct kvm_coalesced_mmio_ring *ring = dev->kvm->coalesced_mmio_ring;
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index fc5f43e..26c72f3 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -715,8 +715,8 @@ ioeventfd_in_range(struct _ioeventfd *p, gpa_t addr, int len, const void *val)
 
 /* MMIO/PIO writes trigger an event if the addr/val match */
 static int
-ioeventfd_write(struct kvm_io_device *this, gpa_t addr, int len,
-		const void *val)
+ioeventfd_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this, gpa_t addr,
+		int len, const void *val)
 {
 	struct _ioeventfd *p = to_ioeventfd(this);
 
diff --git a/virt/kvm/iodev.h b/virt/kvm/iodev.h
index 12fd3ca..9ef709c 100644
--- a/virt/kvm/iodev.h
+++ b/virt/kvm/iodev.h
@@ -20,6 +20,7 @@
 #include <asm/errno.h>
 
 struct kvm_io_device;
+struct kvm_vcpu;
 
 /**
  * kvm_io_device_ops are called under kvm slots_lock.
@@ -27,11 +28,13 @@ struct kvm_io_device;
  * or non-zero to have it passed to the next device.
  **/
 struct kvm_io_device_ops {
-	int (*read)(struct kvm_io_device *this,
+	int (*read)(struct kvm_vcpu *vcpu,
+		    struct kvm_io_device *this,
 		    gpa_t addr,
 		    int len,
 		    void *val);
-	int (*write)(struct kvm_io_device *this,
+	int (*write)(struct kvm_vcpu *vcpu,
+		     struct kvm_io_device *this,
 		     gpa_t addr,
 		     int len,
 		     const void *val);
@@ -49,16 +52,20 @@ static inline void kvm_iodevice_init(struct kvm_io_device *dev,
 	dev->ops = ops;
 }
 
-static inline int kvm_iodevice_read(struct kvm_io_device *dev,
-				    gpa_t addr, int l, void *v)
+static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
+				    struct kvm_io_device *dev, gpa_t addr,
+				    int l, void *v)
 {
-	return dev->ops->read ? dev->ops->read(dev, addr, l, v) : -EOPNOTSUPP;
+	return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
+				: -EOPNOTSUPP;
 }
 
-static inline int kvm_iodevice_write(struct kvm_io_device *dev,
-				     gpa_t addr, int l, const void *v)
+static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
+				     struct kvm_io_device *dev, gpa_t addr,
+				     int l, const void *v)
 {
-	return dev->ops->write ? dev->ops->write(dev, addr, l, v) : -EOPNOTSUPP;
+	return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
+				 : -EOPNOTSUPP;
 }
 
 static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index a109370..ce86493 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -2997,7 +2997,7 @@ static int kvm_io_bus_get_first_dev(struct kvm_io_bus *bus,
 	return off;
 }
 
-static int __kvm_io_bus_write(struct kvm_io_bus *bus,
+static int __kvm_io_bus_write(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
 			      struct kvm_io_range *range, const void *val)
 {
 	int idx;
@@ -3008,7 +3008,7 @@ static int __kvm_io_bus_write(struct kvm_io_bus *bus,
 
 	while (idx < bus->dev_count &&
 		kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
-		if (!kvm_iodevice_write(bus->range[idx].dev, range->addr,
+		if (!kvm_iodevice_write(vcpu, bus->range[idx].dev, range->addr,
 					range->len, val))
 			return idx;
 		idx++;
@@ -3018,7 +3018,7 @@ static int __kvm_io_bus_write(struct kvm_io_bus *bus,
 }
 
 /* kvm_io_bus_write - called under kvm->slots_lock */
-int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 		     int len, const void *val)
 {
 	struct kvm_io_bus *bus;
@@ -3030,14 +3030,14 @@ int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		.len = len,
 	};
 
-	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
-	r = __kvm_io_bus_write(bus, &range, val);
+	bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &kvm->srcu);
+	r = __kvm_io_bus_write(vcpu, bus, &range, val);
 	return r < 0 ? r : 0;
 }
 
 /* kvm_io_bus_write_cookie - called under kvm->slots_lock */
-int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
-			    int len, const void *val, long cookie)
+int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
+			    gpa_t addr, int len, const void *val, long cookie)
 {
 	struct kvm_io_bus *bus;
 	struct kvm_io_range range;
@@ -3047,12 +3047,12 @@ int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		.len = len,
 	};
 
-	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
+	bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &kvm->srcu);
 
 	/* First try the device referenced by cookie. */
 	if ((cookie >= 0) && (cookie < bus->dev_count) &&
 	    (kvm_io_bus_cmp(&range, &bus->range[cookie]) == 0))
-		if (!kvm_iodevice_write(bus->range[cookie].dev, addr, len,
+		if (!kvm_iodevice_write(vcpu, bus->range[cookie].dev, addr, len,
 					val))
 			return cookie;
 
@@ -3060,11 +3060,11 @@ int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 	 * cookie contained garbage; fall back to search and return the
 	 * correct cookie value.
 	 */
-	return __kvm_io_bus_write(bus, &range, val);
+	return __kvm_io_bus_write(vcpu, bus, &range, val);
 }
 
-static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
-			     void *val)
+static int __kvm_io_bus_read(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
+			     struct kvm_io_range *range, void *val)
 {
 	int idx;
 
@@ -3074,7 +3074,7 @@ static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
 
 	while (idx < bus->dev_count &&
 		kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
-		if (!kvm_iodevice_read(bus->range[idx].dev, range->addr,
+		if (!kvm_iodevice_read(vcpu, bus->range[idx].dev, range->addr,
 				       range->len, val))
 			return idx;
 		idx++;
@@ -3085,7 +3085,7 @@ static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
 EXPORT_SYMBOL_GPL(kvm_io_bus_write);
 
 /* kvm_io_bus_read - called under kvm->slots_lock */
-int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
 		    int len, void *val)
 {
 	struct kvm_io_bus *bus;
@@ -3097,8 +3097,8 @@ int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
 		.len = len,
 	};
 
-	bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
-	r = __kvm_io_bus_read(bus, &range, val);
+	bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &kvm->srcu);
+	r = __kvm_io_bus_read(vcpu, bus, &range, val);
 	return r < 0 ? r : 0;
 }
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 02/12] KVM: move iodev.h from virt/kvm/ to include/kvm
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

iodev.h contains definitions for the kvm_io_bus framework. This is
needed both by the generic KVM code in virt/kvm as well as by
architecture specific code under arch/. Putting the header file in
virt/kvm and using local includes in the architecture part seems at
least dodgy to me, so let's move the file into include/kvm, so that a
more natural "#include <kvm/iodev.h>" can be used by all of the code.
This also solves a problem later when using struct kvm_io_device
in arm_vgic.h.
Fixing up the FSF address in the GPL header and a wrong include path
on the way.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/powerpc/kvm/mpic.c   |    2 +-
 arch/x86/kvm/i8254.h      |    2 +-
 arch/x86/kvm/ioapic.h     |    2 +-
 arch/x86/kvm/irq.h        |    2 +-
 arch/x86/kvm/lapic.h      |    2 +-
 include/kvm/iodev.h       |   76 ++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/coalesced_mmio.c |    2 +-
 virt/kvm/eventfd.c        |    2 +-
 virt/kvm/iodev.h          |   77 ---------------------------------------------
 virt/kvm/kvm_main.c       |    2 +-
 10 files changed, 84 insertions(+), 85 deletions(-)
 create mode 100644 include/kvm/iodev.h
 delete mode 100644 virt/kvm/iodev.h

diff --git a/arch/powerpc/kvm/mpic.c b/arch/powerpc/kvm/mpic.c
index 8542f07..4703fad 100644
--- a/arch/powerpc/kvm/mpic.c
+++ b/arch/powerpc/kvm/mpic.c
@@ -34,7 +34,7 @@
 #include <asm/kvm_para.h>
 #include <asm/kvm_host.h>
 #include <asm/kvm_ppc.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #define MAX_CPU     32
 #define MAX_SRC     256
diff --git a/arch/x86/kvm/i8254.h b/arch/x86/kvm/i8254.h
index dd1b16b..c84990b 100644
--- a/arch/x86/kvm/i8254.h
+++ b/arch/x86/kvm/i8254.h
@@ -3,7 +3,7 @@
 
 #include <linux/kthread.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 struct kvm_kpit_channel_state {
 	u32 count; /* can be 65536 */
diff --git a/arch/x86/kvm/ioapic.h b/arch/x86/kvm/ioapic.h
index c2e36d9..d9e02ca 100644
--- a/arch/x86/kvm/ioapic.h
+++ b/arch/x86/kvm/ioapic.h
@@ -3,7 +3,7 @@
 
 #include <linux/kvm_host.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 struct kvm;
 struct kvm_vcpu;
diff --git a/arch/x86/kvm/irq.h b/arch/x86/kvm/irq.h
index 2d03568..ad68c73 100644
--- a/arch/x86/kvm/irq.h
+++ b/arch/x86/kvm/irq.h
@@ -27,7 +27,7 @@
 #include <linux/kvm_host.h>
 #include <linux/spinlock.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 #include "ioapic.h"
 #include "lapic.h"
 
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 0bc6c65..e284c28 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -1,7 +1,7 @@
 #ifndef __KVM_X86_LAPIC_H
 #define __KVM_X86_LAPIC_H
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #include <linux/kvm_host.h>
 
diff --git a/include/kvm/iodev.h b/include/kvm/iodev.h
new file mode 100644
index 0000000..a6d208b
--- /dev/null
+++ b/include/kvm/iodev.h
@@ -0,0 +1,76 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __KVM_IODEV_H__
+#define __KVM_IODEV_H__
+
+#include <linux/kvm_types.h>
+#include <linux/errno.h>
+
+struct kvm_io_device;
+struct kvm_vcpu;
+
+/**
+ * kvm_io_device_ops are called under kvm slots_lock.
+ * read and write handlers return 0 if the transaction has been handled,
+ * or non-zero to have it passed to the next device.
+ **/
+struct kvm_io_device_ops {
+	int (*read)(struct kvm_vcpu *vcpu,
+		    struct kvm_io_device *this,
+		    gpa_t addr,
+		    int len,
+		    void *val);
+	int (*write)(struct kvm_vcpu *vcpu,
+		     struct kvm_io_device *this,
+		     gpa_t addr,
+		     int len,
+		     const void *val);
+	void (*destructor)(struct kvm_io_device *this);
+};
+
+
+struct kvm_io_device {
+	const struct kvm_io_device_ops *ops;
+};
+
+static inline void kvm_iodevice_init(struct kvm_io_device *dev,
+				     const struct kvm_io_device_ops *ops)
+{
+	dev->ops = ops;
+}
+
+static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
+				    struct kvm_io_device *dev, gpa_t addr,
+				    int l, void *v)
+{
+	return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
+				: -EOPNOTSUPP;
+}
+
+static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
+				     struct kvm_io_device *dev, gpa_t addr,
+				     int l, const void *v)
+{
+	return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
+				 : -EOPNOTSUPP;
+}
+
+static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
+{
+	if (dev->ops->destructor)
+		dev->ops->destructor(dev);
+}
+
+#endif /* __KVM_IODEV_H__ */
diff --git a/virt/kvm/coalesced_mmio.c b/virt/kvm/coalesced_mmio.c
index c831a40..571c1ce 100644
--- a/virt/kvm/coalesced_mmio.c
+++ b/virt/kvm/coalesced_mmio.c
@@ -8,7 +8,7 @@
  *
  */
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #include <linux/kvm_host.h>
 #include <linux/slab.h>
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index 26c72f3..9ff4193 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -36,7 +36,7 @@
 #include <linux/seqlock.h>
 #include <trace/events/kvm.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #ifdef CONFIG_HAVE_KVM_IRQFD
 /*
diff --git a/virt/kvm/iodev.h b/virt/kvm/iodev.h
deleted file mode 100644
index 9ef709c..0000000
--- a/virt/kvm/iodev.h
+++ /dev/null
@@ -1,77 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
- */
-
-#ifndef __KVM_IODEV_H__
-#define __KVM_IODEV_H__
-
-#include <linux/kvm_types.h>
-#include <asm/errno.h>
-
-struct kvm_io_device;
-struct kvm_vcpu;
-
-/**
- * kvm_io_device_ops are called under kvm slots_lock.
- * read and write handlers return 0 if the transaction has been handled,
- * or non-zero to have it passed to the next device.
- **/
-struct kvm_io_device_ops {
-	int (*read)(struct kvm_vcpu *vcpu,
-		    struct kvm_io_device *this,
-		    gpa_t addr,
-		    int len,
-		    void *val);
-	int (*write)(struct kvm_vcpu *vcpu,
-		     struct kvm_io_device *this,
-		     gpa_t addr,
-		     int len,
-		     const void *val);
-	void (*destructor)(struct kvm_io_device *this);
-};
-
-
-struct kvm_io_device {
-	const struct kvm_io_device_ops *ops;
-};
-
-static inline void kvm_iodevice_init(struct kvm_io_device *dev,
-				     const struct kvm_io_device_ops *ops)
-{
-	dev->ops = ops;
-}
-
-static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
-				    struct kvm_io_device *dev, gpa_t addr,
-				    int l, void *v)
-{
-	return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
-				: -EOPNOTSUPP;
-}
-
-static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
-				     struct kvm_io_device *dev, gpa_t addr,
-				     int l, const void *v)
-{
-	return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
-				 : -EOPNOTSUPP;
-}
-
-static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
-{
-	if (dev->ops->destructor)
-		dev->ops->destructor(dev);
-}
-
-#endif /* __KVM_IODEV_H__ */
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index ce86493..8c7ab0b 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -16,7 +16,7 @@
  *
  */
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #include <linux/kvm_host.h>
 #include <linux/kvm.h>
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 02/12] KVM: move iodev.h from virt/kvm/ to include/kvm
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

iodev.h contains definitions for the kvm_io_bus framework. This is
needed both by the generic KVM code in virt/kvm as well as by
architecture specific code under arch/. Putting the header file in
virt/kvm and using local includes in the architecture part seems at
least dodgy to me, so let's move the file into include/kvm, so that a
more natural "#include <kvm/iodev.h>" can be used by all of the code.
This also solves a problem later when using struct kvm_io_device
in arm_vgic.h.
Fixing up the FSF address in the GPL header and a wrong include path
on the way.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/powerpc/kvm/mpic.c   |    2 +-
 arch/x86/kvm/i8254.h      |    2 +-
 arch/x86/kvm/ioapic.h     |    2 +-
 arch/x86/kvm/irq.h        |    2 +-
 arch/x86/kvm/lapic.h      |    2 +-
 include/kvm/iodev.h       |   76 ++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/coalesced_mmio.c |    2 +-
 virt/kvm/eventfd.c        |    2 +-
 virt/kvm/iodev.h          |   77 ---------------------------------------------
 virt/kvm/kvm_main.c       |    2 +-
 10 files changed, 84 insertions(+), 85 deletions(-)
 create mode 100644 include/kvm/iodev.h
 delete mode 100644 virt/kvm/iodev.h

diff --git a/arch/powerpc/kvm/mpic.c b/arch/powerpc/kvm/mpic.c
index 8542f07..4703fad 100644
--- a/arch/powerpc/kvm/mpic.c
+++ b/arch/powerpc/kvm/mpic.c
@@ -34,7 +34,7 @@
 #include <asm/kvm_para.h>
 #include <asm/kvm_host.h>
 #include <asm/kvm_ppc.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #define MAX_CPU     32
 #define MAX_SRC     256
diff --git a/arch/x86/kvm/i8254.h b/arch/x86/kvm/i8254.h
index dd1b16b..c84990b 100644
--- a/arch/x86/kvm/i8254.h
+++ b/arch/x86/kvm/i8254.h
@@ -3,7 +3,7 @@
 
 #include <linux/kthread.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 struct kvm_kpit_channel_state {
 	u32 count; /* can be 65536 */
diff --git a/arch/x86/kvm/ioapic.h b/arch/x86/kvm/ioapic.h
index c2e36d9..d9e02ca 100644
--- a/arch/x86/kvm/ioapic.h
+++ b/arch/x86/kvm/ioapic.h
@@ -3,7 +3,7 @@
 
 #include <linux/kvm_host.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 struct kvm;
 struct kvm_vcpu;
diff --git a/arch/x86/kvm/irq.h b/arch/x86/kvm/irq.h
index 2d03568..ad68c73 100644
--- a/arch/x86/kvm/irq.h
+++ b/arch/x86/kvm/irq.h
@@ -27,7 +27,7 @@
 #include <linux/kvm_host.h>
 #include <linux/spinlock.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 #include "ioapic.h"
 #include "lapic.h"
 
diff --git a/arch/x86/kvm/lapic.h b/arch/x86/kvm/lapic.h
index 0bc6c65..e284c28 100644
--- a/arch/x86/kvm/lapic.h
+++ b/arch/x86/kvm/lapic.h
@@ -1,7 +1,7 @@
 #ifndef __KVM_X86_LAPIC_H
 #define __KVM_X86_LAPIC_H
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #include <linux/kvm_host.h>
 
diff --git a/include/kvm/iodev.h b/include/kvm/iodev.h
new file mode 100644
index 0000000..a6d208b
--- /dev/null
+++ b/include/kvm/iodev.h
@@ -0,0 +1,76 @@
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __KVM_IODEV_H__
+#define __KVM_IODEV_H__
+
+#include <linux/kvm_types.h>
+#include <linux/errno.h>
+
+struct kvm_io_device;
+struct kvm_vcpu;
+
+/**
+ * kvm_io_device_ops are called under kvm slots_lock.
+ * read and write handlers return 0 if the transaction has been handled,
+ * or non-zero to have it passed to the next device.
+ **/
+struct kvm_io_device_ops {
+	int (*read)(struct kvm_vcpu *vcpu,
+		    struct kvm_io_device *this,
+		    gpa_t addr,
+		    int len,
+		    void *val);
+	int (*write)(struct kvm_vcpu *vcpu,
+		     struct kvm_io_device *this,
+		     gpa_t addr,
+		     int len,
+		     const void *val);
+	void (*destructor)(struct kvm_io_device *this);
+};
+
+
+struct kvm_io_device {
+	const struct kvm_io_device_ops *ops;
+};
+
+static inline void kvm_iodevice_init(struct kvm_io_device *dev,
+				     const struct kvm_io_device_ops *ops)
+{
+	dev->ops = ops;
+}
+
+static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
+				    struct kvm_io_device *dev, gpa_t addr,
+				    int l, void *v)
+{
+	return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
+				: -EOPNOTSUPP;
+}
+
+static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
+				     struct kvm_io_device *dev, gpa_t addr,
+				     int l, const void *v)
+{
+	return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
+				 : -EOPNOTSUPP;
+}
+
+static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
+{
+	if (dev->ops->destructor)
+		dev->ops->destructor(dev);
+}
+
+#endif /* __KVM_IODEV_H__ */
diff --git a/virt/kvm/coalesced_mmio.c b/virt/kvm/coalesced_mmio.c
index c831a40..571c1ce 100644
--- a/virt/kvm/coalesced_mmio.c
+++ b/virt/kvm/coalesced_mmio.c
@@ -8,7 +8,7 @@
  *
  */
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #include <linux/kvm_host.h>
 #include <linux/slab.h>
diff --git a/virt/kvm/eventfd.c b/virt/kvm/eventfd.c
index 26c72f3..9ff4193 100644
--- a/virt/kvm/eventfd.c
+++ b/virt/kvm/eventfd.c
@@ -36,7 +36,7 @@
 #include <linux/seqlock.h>
 #include <trace/events/kvm.h>
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #ifdef CONFIG_HAVE_KVM_IRQFD
 /*
diff --git a/virt/kvm/iodev.h b/virt/kvm/iodev.h
deleted file mode 100644
index 9ef709c..0000000
--- a/virt/kvm/iodev.h
+++ /dev/null
@@ -1,77 +0,0 @@
-/*
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
- * GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
- */
-
-#ifndef __KVM_IODEV_H__
-#define __KVM_IODEV_H__
-
-#include <linux/kvm_types.h>
-#include <asm/errno.h>
-
-struct kvm_io_device;
-struct kvm_vcpu;
-
-/**
- * kvm_io_device_ops are called under kvm slots_lock.
- * read and write handlers return 0 if the transaction has been handled,
- * or non-zero to have it passed to the next device.
- **/
-struct kvm_io_device_ops {
-	int (*read)(struct kvm_vcpu *vcpu,
-		    struct kvm_io_device *this,
-		    gpa_t addr,
-		    int len,
-		    void *val);
-	int (*write)(struct kvm_vcpu *vcpu,
-		     struct kvm_io_device *this,
-		     gpa_t addr,
-		     int len,
-		     const void *val);
-	void (*destructor)(struct kvm_io_device *this);
-};
-
-
-struct kvm_io_device {
-	const struct kvm_io_device_ops *ops;
-};
-
-static inline void kvm_iodevice_init(struct kvm_io_device *dev,
-				     const struct kvm_io_device_ops *ops)
-{
-	dev->ops = ops;
-}
-
-static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
-				    struct kvm_io_device *dev, gpa_t addr,
-				    int l, void *v)
-{
-	return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
-				: -EOPNOTSUPP;
-}
-
-static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
-				     struct kvm_io_device *dev, gpa_t addr,
-				     int l, const void *v)
-{
-	return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
-				 : -EOPNOTSUPP;
-}
-
-static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
-{
-	if (dev->ops->destructor)
-		dev->ops->destructor(dev);
-}
-
-#endif /* __KVM_IODEV_H__ */
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index ce86493..8c7ab0b 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -16,7 +16,7 @@
  *
  */
 
-#include "iodev.h"
+#include <kvm/iodev.h>
 
 #include <linux/kvm_host.h>
 #include <linux/kvm.h>
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 03/12] KVM: arm/arm64: remove now unneeded include directory from Makefile
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

virt/kvm was never really a good include directory for anything else
than locally included headers.
With the move of iodev.h there is no need anymore to add this
directory the compiler's include path, so remove it from the arm and
arm64 kvm Makefile.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm/kvm/Makefile   |    2 +-
 arch/arm64/kvm/Makefile |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index a093bf1..139e46c 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -7,7 +7,7 @@ ifeq ($(plus_virt),+virt)
 	plus_virt_def := -DREQUIRES_VIRT=1
 endif
 
-ccflags-y += -Ivirt/kvm -Iarch/arm/kvm
+ccflags-y += -Iarch/arm/kvm
 CFLAGS_arm.o := -I. $(plus_virt_def)
 CFLAGS_mmu.o := -I.
 
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index b22c636..d5904f8 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -2,7 +2,7 @@
 # Makefile for Kernel-based Virtual Machine module
 #
 
-ccflags-y += -Ivirt/kvm -Iarch/arm64/kvm
+ccflags-y += -Iarch/arm64/kvm
 CFLAGS_arm.o := -I.
 CFLAGS_mmu.o := -I.
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 03/12] KVM: arm/arm64: remove now unneeded include directory from Makefile
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

virt/kvm was never really a good include directory for anything else
than locally included headers.
With the move of iodev.h there is no need anymore to add this
directory the compiler's include path, so remove it from the arm and
arm64 kvm Makefile.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm/kvm/Makefile   |    2 +-
 arch/arm64/kvm/Makefile |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/arm/kvm/Makefile b/arch/arm/kvm/Makefile
index a093bf1..139e46c 100644
--- a/arch/arm/kvm/Makefile
+++ b/arch/arm/kvm/Makefile
@@ -7,7 +7,7 @@ ifeq ($(plus_virt),+virt)
 	plus_virt_def := -DREQUIRES_VIRT=1
 endif
 
-ccflags-y += -Ivirt/kvm -Iarch/arm/kvm
+ccflags-y += -Iarch/arm/kvm
 CFLAGS_arm.o := -I. $(plus_virt_def)
 CFLAGS_mmu.o := -I.
 
diff --git a/arch/arm64/kvm/Makefile b/arch/arm64/kvm/Makefile
index b22c636..d5904f8 100644
--- a/arch/arm64/kvm/Makefile
+++ b/arch/arm64/kvm/Makefile
@@ -2,7 +2,7 @@
 # Makefile for Kernel-based Virtual Machine module
 #
 
-ccflags-y += -Ivirt/kvm -Iarch/arm64/kvm
+ccflags-y += -Iarch/arm64/kvm
 CFLAGS_arm.o := -I.
 CFLAGS_mmu.o := -I.
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 04/12] KVM: x86: remove now unneeded include directory from Makefile
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

virt/kvm was never really a good include directory for anything else
than locally included headers.
With the move of iodev.h there is no need anymore to add this
directory the compiler's include path, so remove it from the x86 kvm
Makefile.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/x86/kvm/Makefile |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index 08f790d..16e8f96 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -1,5 +1,5 @@
 
-ccflags-y += -Ivirt/kvm -Iarch/x86/kvm
+ccflags-y += -Iarch/x86/kvm
 
 CFLAGS_x86.o := -I.
 CFLAGS_svm.o := -I.
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 04/12] KVM: x86: remove now unneeded include directory from Makefile
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

virt/kvm was never really a good include directory for anything else
than locally included headers.
With the move of iodev.h there is no need anymore to add this
directory the compiler's include path, so remove it from the x86 kvm
Makefile.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/x86/kvm/Makefile |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index 08f790d..16e8f96 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -1,5 +1,5 @@
 
-ccflags-y += -Ivirt/kvm -Iarch/x86/kvm
+ccflags-y += -Iarch/x86/kvm
 
 CFLAGS_x86.o := -I.
 CFLAGS_svm.o := -I.
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 05/12] KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

The name "kvm_mmio_range" is a bit bold, given that it only covers
the VGIC's MMIO ranges. To avoid confusion with kvm_io_range, rename
it to vgic_io_range.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 virt/kvm/arm/vgic-v2-emul.c |    6 +++---
 virt/kvm/arm/vgic-v3-emul.c |    8 ++++----
 virt/kvm/arm/vgic.c         |   18 +++++++++---------
 virt/kvm/arm/vgic.h         |   12 ++++++------
 4 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 19c6210..5002905 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -303,7 +303,7 @@ static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
 		return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
 }
 
-static const struct kvm_mmio_range vgic_dist_ranges[] = {
+static const struct vgic_io_range vgic_dist_ranges[] = {
 	{
 		.base		= GIC_DIST_CTRL,
 		.len		= 12,
@@ -631,7 +631,7 @@ static bool handle_cpu_mmio_ident(struct kvm_vcpu *vcpu,
  * CPU Interface Register accesses - these are not accessed by the VM, but by
  * user space for saving and restoring VGIC state.
  */
-static const struct kvm_mmio_range vgic_cpu_ranges[] = {
+static const struct vgic_io_range vgic_cpu_ranges[] = {
 	{
 		.base		= GIC_CPU_CTRL,
 		.len		= 12,
@@ -658,7 +658,7 @@ static int vgic_attr_regs_access(struct kvm_device *dev,
 				 struct kvm_device_attr *attr,
 				 u32 *reg, bool is_write)
 {
-	const struct kvm_mmio_range *r = NULL, *ranges;
+	const struct vgic_io_range *r = NULL, *ranges;
 	phys_addr_t offset;
 	int ret, cpuid, c;
 	struct kvm_vcpu *vcpu, *tmp_vcpu;
diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
index b3f1546..14943e3 100644
--- a/virt/kvm/arm/vgic-v3-emul.c
+++ b/virt/kvm/arm/vgic-v3-emul.c
@@ -340,7 +340,7 @@ static bool handle_mmio_idregs(struct kvm_vcpu *vcpu,
 	return false;
 }
 
-static const struct kvm_mmio_range vgic_v3_dist_ranges[] = {
+static const struct vgic_io_range vgic_v3_dist_ranges[] = {
 	{
 		.base           = GICD_CTLR,
 		.len            = 0x04,
@@ -570,7 +570,7 @@ static bool handle_mmio_cfg_reg_redist(struct kvm_vcpu *vcpu,
 	return vgic_handle_cfg_reg(reg, mmio, offset);
 }
 
-static const struct kvm_mmio_range vgic_redist_sgi_ranges[] = {
+static const struct vgic_io_range vgic_redist_sgi_ranges[] = {
 	{
 		.base		= GICR_IGROUPR0,
 		.len		= 0x04,
@@ -676,7 +676,7 @@ static bool handle_mmio_typer_redist(struct kvm_vcpu *vcpu,
 	return false;
 }
 
-static const struct kvm_mmio_range vgic_redist_ranges[] = {
+static const struct vgic_io_range vgic_redist_ranges[] = {
 	{
 		.base           = GICR_CTLR,
 		.len            = 0x04,
@@ -726,7 +726,7 @@ static bool vgic_v3_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
 	unsigned long rdbase = dist->vgic_redist_base;
 	int nrcpus = atomic_read(&vcpu->kvm->online_vcpus);
 	int vcpu_id;
-	const struct kvm_mmio_range *mmio_range;
+	const struct vgic_io_range *mmio_range;
 
 	if (is_in_range(mmio->phys_addr, mmio->len, dbase, GIC_V3_DIST_SIZE)) {
 		return vgic_handle_mmio_range(vcpu, run, mmio,
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index c000e97..fe3ac08 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -649,11 +649,11 @@ void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
 }
 
 const
-struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
-				       struct kvm_exit_mmio *mmio,
-				       phys_addr_t offset)
+struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
+				      struct kvm_exit_mmio *mmio,
+				      phys_addr_t offset)
 {
-	const struct kvm_mmio_range *r = ranges;
+	const struct vgic_io_range *r = ranges;
 
 	while (r->len) {
 		if (offset >= r->base &&
@@ -666,7 +666,7 @@ struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
 }
 
 static bool vgic_validate_access(const struct vgic_dist *dist,
-				 const struct kvm_mmio_range *range,
+				 const struct vgic_io_range *range,
 				 unsigned long offset)
 {
 	int irq;
@@ -694,7 +694,7 @@ static bool vgic_validate_access(const struct vgic_dist *dist,
 static bool call_range_handler(struct kvm_vcpu *vcpu,
 			       struct kvm_exit_mmio *mmio,
 			       unsigned long offset,
-			       const struct kvm_mmio_range *range)
+			       const struct vgic_io_range *range)
 {
 	u32 *data32 = (void *)mmio->data;
 	struct kvm_exit_mmio mmio32;
@@ -741,10 +741,10 @@ static bool call_range_handler(struct kvm_vcpu *vcpu,
  */
 bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			    struct kvm_exit_mmio *mmio,
-			    const struct kvm_mmio_range *ranges,
+			    const struct vgic_io_range *ranges,
 			    unsigned long mmio_base)
 {
-	const struct kvm_mmio_range *range;
+	const struct vgic_io_range *range;
 	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
 	bool updated_state;
 	unsigned long offset;
@@ -1837,7 +1837,7 @@ int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
 	return r;
 }
 
-int vgic_has_attr_regs(const struct kvm_mmio_range *ranges, phys_addr_t offset)
+int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset)
 {
 	struct kvm_exit_mmio dev_attr_mmio;
 
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index 1e83bdf..21d062f 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -74,7 +74,7 @@ void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
 	*((u32 *)mmio->data) = cpu_to_le32(value) & mask;
 }
 
-struct kvm_mmio_range {
+struct vgic_io_range {
 	phys_addr_t base;
 	unsigned long len;
 	int bits_per_irq;
@@ -89,13 +89,13 @@ static inline bool is_in_range(phys_addr_t addr, unsigned long len,
 }
 
 const
-struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
-				       struct kvm_exit_mmio *mmio,
-				       phys_addr_t offset);
+struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
+				      struct kvm_exit_mmio *mmio,
+				      phys_addr_t offset);
 
 bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			    struct kvm_exit_mmio *mmio,
-			    const struct kvm_mmio_range *ranges,
+			    const struct vgic_io_range *ranges,
 			    unsigned long mmio_base);
 
 bool vgic_handle_enable_reg(struct kvm *kvm, struct kvm_exit_mmio *mmio,
@@ -112,7 +112,7 @@ bool vgic_handle_cfg_reg(u32 *reg, struct kvm_exit_mmio *mmio,
 
 void vgic_kick_vcpus(struct kvm *kvm);
 
-int vgic_has_attr_regs(const struct kvm_mmio_range *ranges, phys_addr_t offset);
+int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset);
 int vgic_set_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr);
 int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 05/12] KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

The name "kvm_mmio_range" is a bit bold, given that it only covers
the VGIC's MMIO ranges. To avoid confusion with kvm_io_range, rename
it to vgic_io_range.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 virt/kvm/arm/vgic-v2-emul.c |    6 +++---
 virt/kvm/arm/vgic-v3-emul.c |    8 ++++----
 virt/kvm/arm/vgic.c         |   18 +++++++++---------
 virt/kvm/arm/vgic.h         |   12 ++++++------
 4 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 19c6210..5002905 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -303,7 +303,7 @@ static bool handle_mmio_sgi_clear(struct kvm_vcpu *vcpu,
 		return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
 }
 
-static const struct kvm_mmio_range vgic_dist_ranges[] = {
+static const struct vgic_io_range vgic_dist_ranges[] = {
 	{
 		.base		= GIC_DIST_CTRL,
 		.len		= 12,
@@ -631,7 +631,7 @@ static bool handle_cpu_mmio_ident(struct kvm_vcpu *vcpu,
  * CPU Interface Register accesses - these are not accessed by the VM, but by
  * user space for saving and restoring VGIC state.
  */
-static const struct kvm_mmio_range vgic_cpu_ranges[] = {
+static const struct vgic_io_range vgic_cpu_ranges[] = {
 	{
 		.base		= GIC_CPU_CTRL,
 		.len		= 12,
@@ -658,7 +658,7 @@ static int vgic_attr_regs_access(struct kvm_device *dev,
 				 struct kvm_device_attr *attr,
 				 u32 *reg, bool is_write)
 {
-	const struct kvm_mmio_range *r = NULL, *ranges;
+	const struct vgic_io_range *r = NULL, *ranges;
 	phys_addr_t offset;
 	int ret, cpuid, c;
 	struct kvm_vcpu *vcpu, *tmp_vcpu;
diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
index b3f1546..14943e3 100644
--- a/virt/kvm/arm/vgic-v3-emul.c
+++ b/virt/kvm/arm/vgic-v3-emul.c
@@ -340,7 +340,7 @@ static bool handle_mmio_idregs(struct kvm_vcpu *vcpu,
 	return false;
 }
 
-static const struct kvm_mmio_range vgic_v3_dist_ranges[] = {
+static const struct vgic_io_range vgic_v3_dist_ranges[] = {
 	{
 		.base           = GICD_CTLR,
 		.len            = 0x04,
@@ -570,7 +570,7 @@ static bool handle_mmio_cfg_reg_redist(struct kvm_vcpu *vcpu,
 	return vgic_handle_cfg_reg(reg, mmio, offset);
 }
 
-static const struct kvm_mmio_range vgic_redist_sgi_ranges[] = {
+static const struct vgic_io_range vgic_redist_sgi_ranges[] = {
 	{
 		.base		= GICR_IGROUPR0,
 		.len		= 0x04,
@@ -676,7 +676,7 @@ static bool handle_mmio_typer_redist(struct kvm_vcpu *vcpu,
 	return false;
 }
 
-static const struct kvm_mmio_range vgic_redist_ranges[] = {
+static const struct vgic_io_range vgic_redist_ranges[] = {
 	{
 		.base           = GICR_CTLR,
 		.len            = 0x04,
@@ -726,7 +726,7 @@ static bool vgic_v3_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
 	unsigned long rdbase = dist->vgic_redist_base;
 	int nrcpus = atomic_read(&vcpu->kvm->online_vcpus);
 	int vcpu_id;
-	const struct kvm_mmio_range *mmio_range;
+	const struct vgic_io_range *mmio_range;
 
 	if (is_in_range(mmio->phys_addr, mmio->len, dbase, GIC_V3_DIST_SIZE)) {
 		return vgic_handle_mmio_range(vcpu, run, mmio,
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index c000e97..fe3ac08 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -649,11 +649,11 @@ void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
 }
 
 const
-struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
-				       struct kvm_exit_mmio *mmio,
-				       phys_addr_t offset)
+struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
+				      struct kvm_exit_mmio *mmio,
+				      phys_addr_t offset)
 {
-	const struct kvm_mmio_range *r = ranges;
+	const struct vgic_io_range *r = ranges;
 
 	while (r->len) {
 		if (offset >= r->base &&
@@ -666,7 +666,7 @@ struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
 }
 
 static bool vgic_validate_access(const struct vgic_dist *dist,
-				 const struct kvm_mmio_range *range,
+				 const struct vgic_io_range *range,
 				 unsigned long offset)
 {
 	int irq;
@@ -694,7 +694,7 @@ static bool vgic_validate_access(const struct vgic_dist *dist,
 static bool call_range_handler(struct kvm_vcpu *vcpu,
 			       struct kvm_exit_mmio *mmio,
 			       unsigned long offset,
-			       const struct kvm_mmio_range *range)
+			       const struct vgic_io_range *range)
 {
 	u32 *data32 = (void *)mmio->data;
 	struct kvm_exit_mmio mmio32;
@@ -741,10 +741,10 @@ static bool call_range_handler(struct kvm_vcpu *vcpu,
  */
 bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			    struct kvm_exit_mmio *mmio,
-			    const struct kvm_mmio_range *ranges,
+			    const struct vgic_io_range *ranges,
 			    unsigned long mmio_base)
 {
-	const struct kvm_mmio_range *range;
+	const struct vgic_io_range *range;
 	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
 	bool updated_state;
 	unsigned long offset;
@@ -1837,7 +1837,7 @@ int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
 	return r;
 }
 
-int vgic_has_attr_regs(const struct kvm_mmio_range *ranges, phys_addr_t offset)
+int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset)
 {
 	struct kvm_exit_mmio dev_attr_mmio;
 
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index 1e83bdf..21d062f 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -74,7 +74,7 @@ void mmio_data_write(struct kvm_exit_mmio *mmio, u32 mask, u32 value)
 	*((u32 *)mmio->data) = cpu_to_le32(value) & mask;
 }
 
-struct kvm_mmio_range {
+struct vgic_io_range {
 	phys_addr_t base;
 	unsigned long len;
 	int bits_per_irq;
@@ -89,13 +89,13 @@ static inline bool is_in_range(phys_addr_t addr, unsigned long len,
 }
 
 const
-struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
-				       struct kvm_exit_mmio *mmio,
-				       phys_addr_t offset);
+struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
+				      struct kvm_exit_mmio *mmio,
+				      phys_addr_t offset);
 
 bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			    struct kvm_exit_mmio *mmio,
-			    const struct kvm_mmio_range *ranges,
+			    const struct vgic_io_range *ranges,
 			    unsigned long mmio_base);
 
 bool vgic_handle_enable_reg(struct kvm *kvm, struct kvm_exit_mmio *mmio,
@@ -112,7 +112,7 @@ bool vgic_handle_cfg_reg(u32 *reg, struct kvm_exit_mmio *mmio,
 
 void vgic_kick_vcpus(struct kvm *kvm);
 
-int vgic_has_attr_regs(const struct kvm_mmio_range *ranges, phys_addr_t offset);
+int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset);
 int vgic_set_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr);
 int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 06/12] KVM: mark kvm->buses as empty once they were destroyed
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

In kvm_destroy_vm() we call kvm_io_bus_destroy() pretty early,
especially before calling kvm_arch_destroy_vm(). To avoid
unregistering devices from the already destroyed bus, let's mark
the bus with NULL to let other users know it has been destroyed
already.
This avoids a crash on a VM shutdown with the VGIC using the
kvm_io_bus later (the unregistering is in there to be able to roll
back a faulting init).

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 virt/kvm/kvm_main.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 8c7ab0b..6f164eb 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -604,8 +604,10 @@ static void kvm_destroy_vm(struct kvm *kvm)
 	list_del(&kvm->vm_list);
 	spin_unlock(&kvm_lock);
 	kvm_free_irq_routing(kvm);
-	for (i = 0; i < KVM_NR_BUSES; i++)
+	for (i = 0; i < KVM_NR_BUSES; i++) {
 		kvm_io_bus_destroy(kvm->buses[i]);
+		kvm->buses[i] = NULL;
+	}
 	kvm_coalesced_mmio_free(kvm);
 #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
 	mmu_notifier_unregister(&kvm->mmu_notifier, kvm->mm);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 06/12] KVM: mark kvm->buses as empty once they were destroyed
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

In kvm_destroy_vm() we call kvm_io_bus_destroy() pretty early,
especially before calling kvm_arch_destroy_vm(). To avoid
unregistering devices from the already destroyed bus, let's mark
the bus with NULL to let other users know it has been destroyed
already.
This avoids a crash on a VM shutdown with the VGIC using the
kvm_io_bus later (the unregistering is in there to be able to roll
back a faulting init).

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 virt/kvm/kvm_main.c |    4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 8c7ab0b..6f164eb 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -604,8 +604,10 @@ static void kvm_destroy_vm(struct kvm *kvm)
 	list_del(&kvm->vm_list);
 	spin_unlock(&kvm_lock);
 	kvm_free_irq_routing(kvm);
-	for (i = 0; i < KVM_NR_BUSES; i++)
+	for (i = 0; i < KVM_NR_BUSES; i++) {
 		kvm_io_bus_destroy(kvm->buses[i]);
+		kvm->buses[i] = NULL;
+	}
 	kvm_coalesced_mmio_free(kvm);
 #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
 	mmu_notifier_unregister(&kvm->mmu_notifier, kvm->mm);
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 07/12] KVM: arm/arm64: simplify vgic_find_range() and callers
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

The vgic_find_range() function in vgic.c takes a struct kvm_exit_mmio
argument, but actually only used the length field in there. Since we
need to get rid of that structure in that part of the code anyway,
let's rework the function (and it's callers) to pass the length
argument to the function directly.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 virt/kvm/arm/vgic-v2-emul.c |    2 +-
 virt/kvm/arm/vgic.c         |   22 ++++++++--------------
 virt/kvm/arm/vgic.h         |    3 +--
 3 files changed, 10 insertions(+), 17 deletions(-)

diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 5002905..0defac6 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -699,7 +699,7 @@ static int vgic_attr_regs_access(struct kvm_device *dev,
 	default:
 		BUG();
 	}
-	r = vgic_find_range(ranges, &mmio, offset);
+	r = vgic_find_range(ranges, 4, offset);
 
 	if (unlikely(!r || !r->handle_mmio)) {
 		ret = -ENXIO;
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index fe3ac08..7aae19b 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -650,16 +650,13 @@ void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
 
 const
 struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
-				      struct kvm_exit_mmio *mmio,
-				      phys_addr_t offset)
+				      int len, gpa_t offset)
 {
-	const struct vgic_io_range *r = ranges;
-
-	while (r->len) {
-		if (offset >= r->base &&
-		    (offset + mmio->len) <= (r->base + r->len))
-			return r;
-		r++;
+	while (ranges->len) {
+		if (offset >= ranges->base &&
+		    (offset + len) <= (ranges->base + ranges->len))
+			return ranges;
+		ranges++;
 	}
 
 	return NULL;
@@ -750,7 +747,7 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 	unsigned long offset;
 
 	offset = mmio->phys_addr - mmio_base;
-	range = vgic_find_range(ranges, mmio, offset);
+	range = vgic_find_range(ranges, mmio->len, offset);
 	if (unlikely(!range || !range->handle_mmio)) {
 		pr_warn("Unhandled access %d %08llx %d\n",
 			mmio->is_write, mmio->phys_addr, mmio->len);
@@ -1839,10 +1836,7 @@ int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
 
 int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset)
 {
-	struct kvm_exit_mmio dev_attr_mmio;
-
-	dev_attr_mmio.len = 4;
-	if (vgic_find_range(ranges, &dev_attr_mmio, offset))
+	if (vgic_find_range(ranges, 4, offset))
 		return 0;
 	else
 		return -ENXIO;
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index 21d062f..ffafb15 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -90,8 +90,7 @@ static inline bool is_in_range(phys_addr_t addr, unsigned long len,
 
 const
 struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
-				      struct kvm_exit_mmio *mmio,
-				      phys_addr_t offset);
+				      int len, gpa_t offset);
 
 bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			    struct kvm_exit_mmio *mmio,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 07/12] KVM: arm/arm64: simplify vgic_find_range() and callers
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

The vgic_find_range() function in vgic.c takes a struct kvm_exit_mmio
argument, but actually only used the length field in there. Since we
need to get rid of that structure in that part of the code anyway,
let's rework the function (and it's callers) to pass the length
argument to the function directly.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 virt/kvm/arm/vgic-v2-emul.c |    2 +-
 virt/kvm/arm/vgic.c         |   22 ++++++++--------------
 virt/kvm/arm/vgic.h         |    3 +--
 3 files changed, 10 insertions(+), 17 deletions(-)

diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 5002905..0defac6 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -699,7 +699,7 @@ static int vgic_attr_regs_access(struct kvm_device *dev,
 	default:
 		BUG();
 	}
-	r = vgic_find_range(ranges, &mmio, offset);
+	r = vgic_find_range(ranges, 4, offset);
 
 	if (unlikely(!r || !r->handle_mmio)) {
 		ret = -ENXIO;
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index fe3ac08..7aae19b 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -650,16 +650,13 @@ void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
 
 const
 struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
-				      struct kvm_exit_mmio *mmio,
-				      phys_addr_t offset)
+				      int len, gpa_t offset)
 {
-	const struct vgic_io_range *r = ranges;
-
-	while (r->len) {
-		if (offset >= r->base &&
-		    (offset + mmio->len) <= (r->base + r->len))
-			return r;
-		r++;
+	while (ranges->len) {
+		if (offset >= ranges->base &&
+		    (offset + len) <= (ranges->base + ranges->len))
+			return ranges;
+		ranges++;
 	}
 
 	return NULL;
@@ -750,7 +747,7 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 	unsigned long offset;
 
 	offset = mmio->phys_addr - mmio_base;
-	range = vgic_find_range(ranges, mmio, offset);
+	range = vgic_find_range(ranges, mmio->len, offset);
 	if (unlikely(!range || !range->handle_mmio)) {
 		pr_warn("Unhandled access %d %08llx %d\n",
 			mmio->is_write, mmio->phys_addr, mmio->len);
@@ -1839,10 +1836,7 @@ int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr)
 
 int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset)
 {
-	struct kvm_exit_mmio dev_attr_mmio;
-
-	dev_attr_mmio.len = 4;
-	if (vgic_find_range(ranges, &dev_attr_mmio, offset))
+	if (vgic_find_range(ranges, 4, offset))
 		return 0;
 	else
 		return -ENXIO;
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index 21d062f..ffafb15 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -90,8 +90,7 @@ static inline bool is_in_range(phys_addr_t addr, unsigned long len,
 
 const
 struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
-				      struct kvm_exit_mmio *mmio,
-				      phys_addr_t offset);
+				      int len, gpa_t offset);
 
 bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			    struct kvm_exit_mmio *mmio,
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

Currently we use a lot of VGIC specific code to do the MMIO
dispatching.
Use the previous reworks to add kvm_io_bus style MMIO handlers.

Those are not yet called by the MMIO abort handler, also the actual
VGIC emulator function do not make use of it yet, but will be enabled
with the following patches.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h |    9 ++++
 virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.h    |    7 +++
 3 files changed, 127 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index b81630b..4bfc6a3 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -24,6 +24,7 @@
 #include <linux/irqreturn.h>
 #include <linux/spinlock.h>
 #include <linux/types.h>
+#include <kvm/iodev.h>
 
 #define VGIC_NR_IRQS_LEGACY	256
 #define VGIC_NR_SGIS		16
@@ -147,6 +148,14 @@ struct vgic_vm_ops {
 	int	(*map_resources)(struct kvm *, const struct vgic_params *);
 };
 
+struct vgic_io_device {
+	gpa_t addr;
+	int len;
+	const struct vgic_io_range *reg_ranges;
+	struct kvm_vcpu *redist_vcpu;
+	struct kvm_io_device dev;
+};
+
 struct vgic_dist {
 	spinlock_t		lock;
 	bool			in_kernel;
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 7aae19b..71389b8 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -32,6 +32,8 @@
 #include <asm/kvm_arm.h>
 #include <asm/kvm_mmu.h>
 #include <trace/events/kvm.h>
+#include <asm/kvm.h>
+#include <kvm/iodev.h>
 
 /*
  * How the whole thing works (courtesy of Christoffer Dall):
@@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 }
 
 /**
+ * vgic_handle_mmio_access - handle an in-kernel MMIO access
+ * This is called by the read/write KVM IO device wrappers below.
+ * @vcpu:	pointer to the vcpu performing the access
+ * @this:	pointer to the KVM IO device in charge
+ * @addr:	guest physical address of the access
+ * @len:	size of the access
+ * @val:	pointer to the data region
+ * @is_write:	read or write access
+ *
+ * returns true if the MMIO access could be performed
+ */
+static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
+				   struct kvm_io_device *this, gpa_t addr,
+				   int len, void *val, bool is_write)
+{
+	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+	struct vgic_io_device *iodev = container_of(this,
+						    struct vgic_io_device, dev);
+	struct kvm_run *run = vcpu->run;
+	const struct vgic_io_range *range;
+	struct kvm_exit_mmio mmio;
+	bool updated_state;
+	gpa_t offset;
+
+	offset = addr - iodev->addr;
+	range = vgic_find_range(iodev->reg_ranges, len, offset);
+	if (unlikely(!range || !range->handle_mmio)) {
+		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
+		return -ENXIO;
+	}
+
+	mmio.phys_addr = addr;
+	mmio.len = len;
+	mmio.is_write = is_write;
+	if (is_write)
+		memcpy(mmio.data, val, len);
+	mmio.private = iodev->redist_vcpu;
+
+	spin_lock(&dist->lock);
+	offset -= range->base;
+	if (vgic_validate_access(dist, range, offset)) {
+		updated_state = call_range_handler(vcpu, &mmio, offset, range);
+		if (!is_write)
+			memcpy(val, mmio.data, len);
+	} else {
+		if (!is_write)
+			memset(val, 0, len);
+		updated_state = false;
+	}
+	spin_unlock(&dist->lock);
+	kvm_prepare_mmio(run, &mmio);
+	kvm_handle_mmio_return(vcpu, run);
+
+	if (updated_state)
+		vgic_kick_vcpus(vcpu->kvm);
+
+	return 0;
+}
+
+/**
  * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
  * @vcpu:      pointer to the vcpu performing the access
  * @run:       pointer to the kvm_run structure
@@ -797,6 +859,55 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
 	return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
 }
 
+static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
+				 struct kvm_io_device *this,
+				 gpa_t addr, int len, void *val)
+{
+	return vgic_handle_mmio_access(vcpu, this, addr, len, val, false);
+}
+
+static int vgic_handle_mmio_write(struct kvm_vcpu *vcpu,
+				  struct kvm_io_device *this,
+				  gpa_t addr, int len, const void *val)
+{
+	return vgic_handle_mmio_access(vcpu, this, addr, len, (void *)val,
+				       true);
+}
+
+struct kvm_io_device_ops vgic_io_ops = {
+	.read	= vgic_handle_mmio_read,
+	.write	= vgic_handle_mmio_write,
+};
+
+int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
+			     const struct vgic_io_range *ranges,
+			     int redist_id,
+			     struct vgic_io_device *iodev)
+{
+	struct kvm_vcpu *vcpu = NULL;
+	int ret;
+
+	if (redist_id >= 0)
+		vcpu = kvm_get_vcpu(kvm, redist_id);
+
+	iodev->addr		= base;
+	iodev->len		= len;
+	iodev->reg_ranges	= ranges;
+	iodev->redist_vcpu	= vcpu;
+
+	kvm_iodevice_init(&iodev->dev, &vgic_io_ops);
+
+	mutex_lock(&kvm->slots_lock);
+
+	ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, base, len,
+				      &iodev->dev);
+	mutex_unlock(&kvm->slots_lock);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
 static int vgic_nr_shared_irqs(struct vgic_dist *dist)
 {
 	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index ffafb15..f2063a7 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -20,6 +20,8 @@
 #ifndef __KVM_VGIC_H__
 #define __KVM_VGIC_H__
 
+#include <kvm/iodev.h>
+
 #define VGIC_ADDR_UNDEF		(-1)
 #define IS_VGIC_ADDR_UNDEF(_x)  ((_x) == VGIC_ADDR_UNDEF)
 
@@ -82,6 +84,11 @@ struct vgic_io_range {
 			    phys_addr_t offset);
 };
 
+int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
+			     const struct vgic_io_range *ranges,
+			     int redist_id,
+			     struct vgic_io_device *iodev);
+
 static inline bool is_in_range(phys_addr_t addr, unsigned long len,
 			       phys_addr_t baseaddr, unsigned long size)
 {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

Currently we use a lot of VGIC specific code to do the MMIO
dispatching.
Use the previous reworks to add kvm_io_bus style MMIO handlers.

Those are not yet called by the MMIO abort handler, also the actual
VGIC emulator function do not make use of it yet, but will be enabled
with the following patches.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h |    9 ++++
 virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
 virt/kvm/arm/vgic.h    |    7 +++
 3 files changed, 127 insertions(+)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index b81630b..4bfc6a3 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -24,6 +24,7 @@
 #include <linux/irqreturn.h>
 #include <linux/spinlock.h>
 #include <linux/types.h>
+#include <kvm/iodev.h>
 
 #define VGIC_NR_IRQS_LEGACY	256
 #define VGIC_NR_SGIS		16
@@ -147,6 +148,14 @@ struct vgic_vm_ops {
 	int	(*map_resources)(struct kvm *, const struct vgic_params *);
 };
 
+struct vgic_io_device {
+	gpa_t addr;
+	int len;
+	const struct vgic_io_range *reg_ranges;
+	struct kvm_vcpu *redist_vcpu;
+	struct kvm_io_device dev;
+};
+
 struct vgic_dist {
 	spinlock_t		lock;
 	bool			in_kernel;
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 7aae19b..71389b8 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -32,6 +32,8 @@
 #include <asm/kvm_arm.h>
 #include <asm/kvm_mmu.h>
 #include <trace/events/kvm.h>
+#include <asm/kvm.h>
+#include <kvm/iodev.h>
 
 /*
  * How the whole thing works (courtesy of Christoffer Dall):
@@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
 }
 
 /**
+ * vgic_handle_mmio_access - handle an in-kernel MMIO access
+ * This is called by the read/write KVM IO device wrappers below.
+ * @vcpu:	pointer to the vcpu performing the access
+ * @this:	pointer to the KVM IO device in charge
+ * @addr:	guest physical address of the access
+ * @len:	size of the access
+ * @val:	pointer to the data region
+ * @is_write:	read or write access
+ *
+ * returns true if the MMIO access could be performed
+ */
+static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
+				   struct kvm_io_device *this, gpa_t addr,
+				   int len, void *val, bool is_write)
+{
+	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+	struct vgic_io_device *iodev = container_of(this,
+						    struct vgic_io_device, dev);
+	struct kvm_run *run = vcpu->run;
+	const struct vgic_io_range *range;
+	struct kvm_exit_mmio mmio;
+	bool updated_state;
+	gpa_t offset;
+
+	offset = addr - iodev->addr;
+	range = vgic_find_range(iodev->reg_ranges, len, offset);
+	if (unlikely(!range || !range->handle_mmio)) {
+		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
+		return -ENXIO;
+	}
+
+	mmio.phys_addr = addr;
+	mmio.len = len;
+	mmio.is_write = is_write;
+	if (is_write)
+		memcpy(mmio.data, val, len);
+	mmio.private = iodev->redist_vcpu;
+
+	spin_lock(&dist->lock);
+	offset -= range->base;
+	if (vgic_validate_access(dist, range, offset)) {
+		updated_state = call_range_handler(vcpu, &mmio, offset, range);
+		if (!is_write)
+			memcpy(val, mmio.data, len);
+	} else {
+		if (!is_write)
+			memset(val, 0, len);
+		updated_state = false;
+	}
+	spin_unlock(&dist->lock);
+	kvm_prepare_mmio(run, &mmio);
+	kvm_handle_mmio_return(vcpu, run);
+
+	if (updated_state)
+		vgic_kick_vcpus(vcpu->kvm);
+
+	return 0;
+}
+
+/**
  * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
  * @vcpu:      pointer to the vcpu performing the access
  * @run:       pointer to the kvm_run structure
@@ -797,6 +859,55 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
 	return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
 }
 
+static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
+				 struct kvm_io_device *this,
+				 gpa_t addr, int len, void *val)
+{
+	return vgic_handle_mmio_access(vcpu, this, addr, len, val, false);
+}
+
+static int vgic_handle_mmio_write(struct kvm_vcpu *vcpu,
+				  struct kvm_io_device *this,
+				  gpa_t addr, int len, const void *val)
+{
+	return vgic_handle_mmio_access(vcpu, this, addr, len, (void *)val,
+				       true);
+}
+
+struct kvm_io_device_ops vgic_io_ops = {
+	.read	= vgic_handle_mmio_read,
+	.write	= vgic_handle_mmio_write,
+};
+
+int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
+			     const struct vgic_io_range *ranges,
+			     int redist_id,
+			     struct vgic_io_device *iodev)
+{
+	struct kvm_vcpu *vcpu = NULL;
+	int ret;
+
+	if (redist_id >= 0)
+		vcpu = kvm_get_vcpu(kvm, redist_id);
+
+	iodev->addr		= base;
+	iodev->len		= len;
+	iodev->reg_ranges	= ranges;
+	iodev->redist_vcpu	= vcpu;
+
+	kvm_iodevice_init(&iodev->dev, &vgic_io_ops);
+
+	mutex_lock(&kvm->slots_lock);
+
+	ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, base, len,
+				      &iodev->dev);
+	mutex_unlock(&kvm->slots_lock);
+	if (ret < 0)
+		return ret;
+
+	return 0;
+}
+
 static int vgic_nr_shared_irqs(struct vgic_dist *dist)
 {
 	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index ffafb15..f2063a7 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -20,6 +20,8 @@
 #ifndef __KVM_VGIC_H__
 #define __KVM_VGIC_H__
 
+#include <kvm/iodev.h>
+
 #define VGIC_ADDR_UNDEF		(-1)
 #define IS_VGIC_ADDR_UNDEF(_x)  ((_x) == VGIC_ADDR_UNDEF)
 
@@ -82,6 +84,11 @@ struct vgic_io_range {
 			    phys_addr_t offset);
 };
 
+int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
+			     const struct vgic_io_range *ranges,
+			     int redist_id,
+			     struct vgic_io_device *iodev);
+
 static inline bool is_in_range(phys_addr_t addr, unsigned long len,
 			       phys_addr_t baseaddr, unsigned long size)
 {
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

Using the framework provided by the recent vgic.c changes we register
a kvm_io_bus device when initializing the virtual GICv2.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h      |    1 +
 virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
 virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
 3 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4bfc6a3..74a4ac4 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -245,6 +245,7 @@ struct vgic_dist {
 	unsigned long		*irq_pending_on_cpu;
 
 	struct vgic_vm_ops	vm_ops;
+	struct vgic_io_device	dist_iodev;
 };
 
 struct vgic_v2_cpu_if {
diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 0defac6..6f685c9 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
 static int vgic_v2_map_resources(struct kvm *kvm,
 				 const struct vgic_params *params)
 {
+	struct vgic_dist *dist = &kvm->arch.vgic;
 	int ret = 0;
 
 	if (!irqchip_in_kernel(kvm))
@@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
 	if (vgic_ready(kvm))
 		goto out;
 
-	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
-	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
+	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
+	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
 		kvm_err("Need to set vgic cpu and dist addresses first\n");
 		ret = -ENXIO;
 		goto out;
 	}
 
+	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
+				 KVM_VGIC_V2_DIST_SIZE,
+				 vgic_dist_ranges, -1, &dist->dist_iodev);
+
 	/*
 	 * Initialize the vgic if this hasn't already been done on demand by
 	 * accessing the vgic state from userspace.
@@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
 		goto out;
 	}
 
-	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
+	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
 				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
 				    true);
 	if (ret) {
@@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
 		goto out;
 	}
 
-	kvm->arch.vgic.ready = true;
+	dist->ready = true;
 out:
 	if (ret)
 		kvm_vgic_destroy(kvm);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 71389b8..b1dd717 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
 	return 0;
 }
 
+static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+
+	if (!dist || !kvm->buses[KVM_MMIO_BUS])
+		return;
+
+	mutex_lock(&kvm->slots_lock);
+	if (dist->dist_iodev.dev.ops)
+		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
+					  &dist->dist_iodev.dev);
+	mutex_unlock(&kvm->slots_lock);
+}
+
 static int vgic_nr_shared_irqs(struct vgic_dist *dist)
 {
 	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
@@ -1546,6 +1560,8 @@ void kvm_vgic_destroy(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 	int i;
 
+	vgic_unregister_kvm_io_dev(kvm);
+
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_destroy(vcpu);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

Using the framework provided by the recent vgic.c changes we register
a kvm_io_bus device when initializing the virtual GICv2.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h      |    1 +
 virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
 virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
 3 files changed, 26 insertions(+), 4 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 4bfc6a3..74a4ac4 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -245,6 +245,7 @@ struct vgic_dist {
 	unsigned long		*irq_pending_on_cpu;
 
 	struct vgic_vm_ops	vm_ops;
+	struct vgic_io_device	dist_iodev;
 };
 
 struct vgic_v2_cpu_if {
diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 0defac6..6f685c9 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
 static int vgic_v2_map_resources(struct kvm *kvm,
 				 const struct vgic_params *params)
 {
+	struct vgic_dist *dist = &kvm->arch.vgic;
 	int ret = 0;
 
 	if (!irqchip_in_kernel(kvm))
@@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
 	if (vgic_ready(kvm))
 		goto out;
 
-	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
-	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
+	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
+	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
 		kvm_err("Need to set vgic cpu and dist addresses first\n");
 		ret = -ENXIO;
 		goto out;
 	}
 
+	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
+				 KVM_VGIC_V2_DIST_SIZE,
+				 vgic_dist_ranges, -1, &dist->dist_iodev);
+
 	/*
 	 * Initialize the vgic if this hasn't already been done on demand by
 	 * accessing the vgic state from userspace.
@@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
 		goto out;
 	}
 
-	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
+	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
 				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
 				    true);
 	if (ret) {
@@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
 		goto out;
 	}
 
-	kvm->arch.vgic.ready = true;
+	dist->ready = true;
 out:
 	if (ret)
 		kvm_vgic_destroy(kvm);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index 71389b8..b1dd717 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
 	return 0;
 }
 
+static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
+{
+	struct vgic_dist *dist = &kvm->arch.vgic;
+
+	if (!dist || !kvm->buses[KVM_MMIO_BUS])
+		return;
+
+	mutex_lock(&kvm->slots_lock);
+	if (dist->dist_iodev.dev.ops)
+		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
+					  &dist->dist_iodev.dev);
+	mutex_unlock(&kvm->slots_lock);
+}
+
 static int vgic_nr_shared_irqs(struct vgic_dist *dist)
 {
 	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
@@ -1546,6 +1560,8 @@ void kvm_vgic_destroy(struct kvm *kvm)
 	struct kvm_vcpu *vcpu;
 	int i;
 
+	vgic_unregister_kvm_io_dev(kvm);
+
 	kvm_for_each_vcpu(i, vcpu, kvm)
 		kvm_vgic_vcpu_destroy(vcpu);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 10/12] KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO handling
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

Using the framework provided by the recent vgic.c changes, we
register a kvm_io_bus device on mapping the virtual GICv3 resources.
The distributor mapping is pretty straight forward, but the
redistributors need some more love, since they need to be tagged with
the respective redistributor (read: VCPU) they are connected with.
We use the kvm_io_bus framework to register two devices per VCPU, as
each block is handled independently by the VGIC code.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h      |    1 +
 virt/kvm/arm/vgic-v3-emul.c |   32 +++++++++++++++++++++++++++++++-
 virt/kvm/arm/vgic.c         |   14 ++++++++++++++
 3 files changed, 46 insertions(+), 1 deletion(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 74a4ac4..09fd324 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -246,6 +246,7 @@ struct vgic_dist {
 
 	struct vgic_vm_ops	vm_ops;
 	struct vgic_io_device	dist_iodev;
+	struct vgic_io_device	*redist_iodevs;
 };
 
 struct vgic_v2_cpu_if {
diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
index 14943e3..8a81c67 100644
--- a/virt/kvm/arm/vgic-v3-emul.c
+++ b/virt/kvm/arm/vgic-v3-emul.c
@@ -766,6 +766,9 @@ static int vgic_v3_map_resources(struct kvm *kvm,
 {
 	int ret = 0;
 	struct vgic_dist *dist = &kvm->arch.vgic;
+	gpa_t rdbase = dist->vgic_redist_base;
+	struct vgic_io_device *iodevs = NULL;
+	int i;
 
 	if (!irqchip_in_kernel(kvm))
 		return 0;
@@ -791,7 +794,34 @@ static int vgic_v3_map_resources(struct kvm *kvm,
 		goto out;
 	}
 
-	kvm->arch.vgic.ready = true;
+	ret = vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
+				       GIC_V3_DIST_SIZE, vgic_v3_dist_ranges,
+				       -1, &dist->dist_iodev);
+	if (ret)
+		goto out;
+
+	iodevs = kcalloc(dist->nr_cpus * 2, sizeof(iodevs[0]), GFP_KERNEL);
+	if (!iodevs) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	for (i = 0; i < dist->nr_cpus; i++) {
+		ret = vgic_register_kvm_io_dev(kvm, rdbase,
+					       SZ_64K, vgic_redist_ranges,
+					       i, &iodevs[i * 2]);
+		if (ret)
+			goto out;
+		ret = vgic_register_kvm_io_dev(kvm, rdbase + SGI_BASE_OFFSET,
+					       SZ_64K, vgic_redist_sgi_ranges,
+					       i, &iodevs[i * 2 + 1]);
+		if (ret)
+			goto out;
+		rdbase += GIC_V3_REDIST_SIZE;
+	}
+
+	dist->redist_iodevs = iodevs;
+	dist->ready = true;
+
 out:
 	if (ret)
 		kvm_vgic_destroy(kvm);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index b1dd717..f762738 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -911,6 +911,8 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
 static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct vgic_io_device *iodevs;
+	int i;
 
 	if (!dist || !kvm->buses[KVM_MMIO_BUS])
 		return;
@@ -919,6 +921,18 @@ static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
 	if (dist->dist_iodev.dev.ops)
 		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
 					  &dist->dist_iodev.dev);
+
+	iodevs = dist->redist_iodevs;
+	if (iodevs) {
+		for (i = 0; i < dist->nr_cpus * 2; i++) {
+			if (!iodevs[i].dev.ops)
+				continue;
+			kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
+						  &iodevs[i].dev);
+		}
+		kfree(iodevs);
+		dist->redist_iodevs = NULL;
+	}
 	mutex_unlock(&kvm->slots_lock);
 }
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 10/12] KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO handling
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

Using the framework provided by the recent vgic.c changes, we
register a kvm_io_bus device on mapping the virtual GICv3 resources.
The distributor mapping is pretty straight forward, but the
redistributors need some more love, since they need to be tagged with
the respective redistributor (read: VCPU) they are connected with.
We use the kvm_io_bus framework to register two devices per VCPU, as
each block is handled independently by the VGIC code.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h      |    1 +
 virt/kvm/arm/vgic-v3-emul.c |   32 +++++++++++++++++++++++++++++++-
 virt/kvm/arm/vgic.c         |   14 ++++++++++++++
 3 files changed, 46 insertions(+), 1 deletion(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 74a4ac4..09fd324 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -246,6 +246,7 @@ struct vgic_dist {
 
 	struct vgic_vm_ops	vm_ops;
 	struct vgic_io_device	dist_iodev;
+	struct vgic_io_device	*redist_iodevs;
 };
 
 struct vgic_v2_cpu_if {
diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
index 14943e3..8a81c67 100644
--- a/virt/kvm/arm/vgic-v3-emul.c
+++ b/virt/kvm/arm/vgic-v3-emul.c
@@ -766,6 +766,9 @@ static int vgic_v3_map_resources(struct kvm *kvm,
 {
 	int ret = 0;
 	struct vgic_dist *dist = &kvm->arch.vgic;
+	gpa_t rdbase = dist->vgic_redist_base;
+	struct vgic_io_device *iodevs = NULL;
+	int i;
 
 	if (!irqchip_in_kernel(kvm))
 		return 0;
@@ -791,7 +794,34 @@ static int vgic_v3_map_resources(struct kvm *kvm,
 		goto out;
 	}
 
-	kvm->arch.vgic.ready = true;
+	ret = vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
+				       GIC_V3_DIST_SIZE, vgic_v3_dist_ranges,
+				       -1, &dist->dist_iodev);
+	if (ret)
+		goto out;
+
+	iodevs = kcalloc(dist->nr_cpus * 2, sizeof(iodevs[0]), GFP_KERNEL);
+	if (!iodevs) {
+		ret = -ENOMEM;
+		goto out;
+	}
+	for (i = 0; i < dist->nr_cpus; i++) {
+		ret = vgic_register_kvm_io_dev(kvm, rdbase,
+					       SZ_64K, vgic_redist_ranges,
+					       i, &iodevs[i * 2]);
+		if (ret)
+			goto out;
+		ret = vgic_register_kvm_io_dev(kvm, rdbase + SGI_BASE_OFFSET,
+					       SZ_64K, vgic_redist_sgi_ranges,
+					       i, &iodevs[i * 2 + 1]);
+		if (ret)
+			goto out;
+		rdbase += GIC_V3_REDIST_SIZE;
+	}
+
+	dist->redist_iodevs = iodevs;
+	dist->ready = true;
+
 out:
 	if (ret)
 		kvm_vgic_destroy(kvm);
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index b1dd717..f762738 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -911,6 +911,8 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
 static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
+	struct vgic_io_device *iodevs;
+	int i;
 
 	if (!dist || !kvm->buses[KVM_MMIO_BUS])
 		return;
@@ -919,6 +921,18 @@ static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
 	if (dist->dist_iodev.dev.ops)
 		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
 					  &dist->dist_iodev.dev);
+
+	iodevs = dist->redist_iodevs;
+	if (iodevs) {
+		for (i = 0; i < dist->nr_cpus * 2; i++) {
+			if (!iodevs[i].dev.ops)
+				continue;
+			kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
+						  &iodevs[i].dev);
+		}
+		kfree(iodevs);
+		dist->redist_iodevs = NULL;
+	}
 	mutex_unlock(&kvm->slots_lock);
 }
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 11/12] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>

On IO memory abort, try to handle the MMIO access through the KVM
registered read/write callbacks. This is done by invoking the relevant
kvm_io_bus_* API.

[Andre: Since we converted the VGIC already, we can get rid of the
VGIC specific MMIO handler alltogether.]

Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm/kvm/mmio.c    |   32 +++++++++++++++++++++++++++++++-
 include/kvm/arm_vgic.h |    2 --
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index 5d3bfc0..8dc2fde 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	return 0;
 }
 
+/**
+ * handle_kernel_mmio - handle an in-kernel MMIO access
+ * @vcpu:	pointer to the vcpu performing the access
+ * @run:	pointer to the kvm_run structure
+ * @mmio:	pointer to the data describing the access
+ *
+ * returns true if the MMIO access has been performed in kernel space,
+ * and false if it needs to be emulated in user space.
+ */
+static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+		struct kvm_exit_mmio *mmio)
+{
+	int ret;
+
+	if (mmio->is_write) {
+		ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+				mmio->len, &mmio->data);
+
+	} else {
+		ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+				mmio->len, &mmio->data);
+	}
+	if (!ret) {
+		kvm_prepare_mmio(run, mmio);
+		kvm_handle_mmio_return(vcpu, run);
+	}
+
+	return !ret;
+}
+
 int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
 		 phys_addr_t fault_ipa)
 {
@@ -200,7 +230,7 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			       fault_ipa, 0);
 	}
 
-	if (vgic_handle_mmio(vcpu, run, &mmio))
+	if (handle_kernel_mmio(vcpu, run, &mmio))
 		return 1;
 
 	kvm_prepare_mmio(run, &mmio);
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 09fd324..91976c8 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -317,8 +317,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
 			bool level);
 void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
 int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-		      struct kvm_exit_mmio *mmio);
 
 #define irqchip_in_kernel(k)	(!!((k)->arch.vgic.in_kernel))
 #define vgic_initialized(k)	(!!((k)->arch.vgic.nr_cpus))
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 11/12] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>

On IO memory abort, try to handle the MMIO access through the KVM
registered read/write callbacks. This is done by invoking the relevant
kvm_io_bus_* API.

[Andre: Since we converted the VGIC already, we can get rid of the
VGIC specific MMIO handler alltogether.]

Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 arch/arm/kvm/mmio.c    |   32 +++++++++++++++++++++++++++++++-
 include/kvm/arm_vgic.h |    2 --
 2 files changed, 31 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
index 5d3bfc0..8dc2fde 100644
--- a/arch/arm/kvm/mmio.c
+++ b/arch/arm/kvm/mmio.c
@@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	return 0;
 }
 
+/**
+ * handle_kernel_mmio - handle an in-kernel MMIO access
+ * @vcpu:	pointer to the vcpu performing the access
+ * @run:	pointer to the kvm_run structure
+ * @mmio:	pointer to the data describing the access
+ *
+ * returns true if the MMIO access has been performed in kernel space,
+ * and false if it needs to be emulated in user space.
+ */
+static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
+		struct kvm_exit_mmio *mmio)
+{
+	int ret;
+
+	if (mmio->is_write) {
+		ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+				mmio->len, &mmio->data);
+
+	} else {
+		ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
+				mmio->len, &mmio->data);
+	}
+	if (!ret) {
+		kvm_prepare_mmio(run, mmio);
+		kvm_handle_mmio_return(vcpu, run);
+	}
+
+	return !ret;
+}
+
 int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
 		 phys_addr_t fault_ipa)
 {
@@ -200,7 +230,7 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
 			       fault_ipa, 0);
 	}
 
-	if (vgic_handle_mmio(vcpu, run, &mmio))
+	if (handle_kernel_mmio(vcpu, run, &mmio))
 		return 1;
 
 	kvm_prepare_mmio(run, &mmio);
diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 09fd324..91976c8 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -317,8 +317,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
 			bool level);
 void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
 int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-		      struct kvm_exit_mmio *mmio);
 
 #define irqchip_in_kernel(k)	(!!((k)->arch.vgic.in_kernel))
 #define vgic_initialized(k)	(!!((k)->arch.vgic.nr_cpus))
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 12/12] KVM: arm/arm64: remove now obsolete VGIC specific MMIO handling code
  2015-03-13 16:10 ` Andre Przywara
@ 2015-03-13 16:10   ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: marc.zyngier, christoffer.dall, pbonzini; +Cc: linux-arm-kernel, kvm, kvmarm

With all of the virtual GIC emulation code now being registered with
the kvm_io_bus, we can remove all of the old MMIO handling code and
its dispatching functionality.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h      |    2 --
 virt/kvm/arm/vgic-v2-emul.c |   19 ------------
 virt/kvm/arm/vgic-v3-emul.c |   39 ------------------------
 virt/kvm/arm/vgic.c         |   71 -------------------------------------------
 virt/kvm/arm/vgic.h         |    5 ---
 5 files changed, 136 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 91976c8..14d8401 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -140,8 +140,6 @@ struct vgic_params {
 };
 
 struct vgic_vm_ops {
-	bool	(*handle_mmio)(struct kvm_vcpu *, struct kvm_run *,
-			       struct kvm_exit_mmio *);
 	bool	(*queue_sgi)(struct kvm_vcpu *, int irq);
 	void	(*add_sgi_source)(struct kvm_vcpu *, int irq, int source);
 	int	(*init_model)(struct kvm *);
diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 6f685c9..ff264e7 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -388,24 +388,6 @@ static const struct vgic_io_range vgic_dist_ranges[] = {
 	{}
 };
 
-static bool vgic_v2_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-				struct kvm_exit_mmio *mmio)
-{
-	unsigned long base = vcpu->kvm->arch.vgic.vgic_dist_base;
-
-	if (!is_in_range(mmio->phys_addr, mmio->len, base,
-			 KVM_VGIC_V2_DIST_SIZE))
-		return false;
-
-	/* GICv2 does not support accesses wider than 32 bits */
-	if (mmio->len > 4) {
-		kvm_inject_dabt(vcpu, mmio->phys_addr);
-		return true;
-	}
-
-	return vgic_handle_mmio_range(vcpu, run, mmio, vgic_dist_ranges, base);
-}
-
 static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
 {
 	struct kvm *kvm = vcpu->kvm;
@@ -559,7 +541,6 @@ void vgic_v2_init_emulation(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
 
-	dist->vm_ops.handle_mmio = vgic_v2_handle_mmio;
 	dist->vm_ops.queue_sgi = vgic_v2_queue_sgi;
 	dist->vm_ops.add_sgi_source = vgic_v2_add_sgi_source;
 	dist->vm_ops.init_model = vgic_v2_init_model;
diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
index 8a81c67..4957577 100644
--- a/virt/kvm/arm/vgic-v3-emul.c
+++ b/virt/kvm/arm/vgic-v3-emul.c
@@ -710,45 +710,7 @@ static const struct vgic_io_range vgic_redist_ranges[] = {
 	{},
 };
 
-/*
- * This function splits accesses between the distributor and the two
- * redistributor parts (private/SPI). As each redistributor is accessible
- * from any CPU, we have to determine the affected VCPU by taking the faulting
- * address into account. We then pass this VCPU to the handler function via
- * the private parameter.
- */
 #define SGI_BASE_OFFSET SZ_64K
-static bool vgic_v3_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-				struct kvm_exit_mmio *mmio)
-{
-	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
-	unsigned long dbase = dist->vgic_dist_base;
-	unsigned long rdbase = dist->vgic_redist_base;
-	int nrcpus = atomic_read(&vcpu->kvm->online_vcpus);
-	int vcpu_id;
-	const struct vgic_io_range *mmio_range;
-
-	if (is_in_range(mmio->phys_addr, mmio->len, dbase, GIC_V3_DIST_SIZE)) {
-		return vgic_handle_mmio_range(vcpu, run, mmio,
-					      vgic_v3_dist_ranges, dbase);
-	}
-
-	if (!is_in_range(mmio->phys_addr, mmio->len, rdbase,
-	    GIC_V3_REDIST_SIZE * nrcpus))
-		return false;
-
-	vcpu_id = (mmio->phys_addr - rdbase) / GIC_V3_REDIST_SIZE;
-	rdbase += (vcpu_id * GIC_V3_REDIST_SIZE);
-	mmio->private = kvm_get_vcpu(vcpu->kvm, vcpu_id);
-
-	if (mmio->phys_addr >= rdbase + SGI_BASE_OFFSET) {
-		rdbase += SGI_BASE_OFFSET;
-		mmio_range = vgic_redist_sgi_ranges;
-	} else {
-		mmio_range = vgic_redist_ranges;
-	}
-	return vgic_handle_mmio_range(vcpu, run, mmio, mmio_range, rdbase);
-}
 
 static bool vgic_v3_queue_sgi(struct kvm_vcpu *vcpu, int irq)
 {
@@ -862,7 +824,6 @@ void vgic_v3_init_emulation(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
 
-	dist->vm_ops.handle_mmio = vgic_v3_handle_mmio;
 	dist->vm_ops.queue_sgi = vgic_v3_queue_sgi;
 	dist->vm_ops.add_sgi_source = vgic_v3_add_sgi_source;
 	dist->vm_ops.init_model = vgic_v3_init_model;
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index f762738..f8a79d4 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -729,53 +729,6 @@ static bool call_range_handler(struct kvm_vcpu *vcpu,
 }
 
 /**
- * vgic_handle_mmio_range - handle an in-kernel MMIO access
- * @vcpu:	pointer to the vcpu performing the access
- * @run:	pointer to the kvm_run structure
- * @mmio:	pointer to the data describing the access
- * @ranges:	array of MMIO ranges in a given region
- * @mmio_base:	base address of that region
- *
- * returns true if the MMIO access could be performed
- */
-bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
-			    struct kvm_exit_mmio *mmio,
-			    const struct vgic_io_range *ranges,
-			    unsigned long mmio_base)
-{
-	const struct vgic_io_range *range;
-	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
-	bool updated_state;
-	unsigned long offset;
-
-	offset = mmio->phys_addr - mmio_base;
-	range = vgic_find_range(ranges, mmio->len, offset);
-	if (unlikely(!range || !range->handle_mmio)) {
-		pr_warn("Unhandled access %d %08llx %d\n",
-			mmio->is_write, mmio->phys_addr, mmio->len);
-		return false;
-	}
-
-	spin_lock(&vcpu->kvm->arch.vgic.lock);
-	offset -= range->base;
-	if (vgic_validate_access(dist, range, offset)) {
-		updated_state = call_range_handler(vcpu, mmio, offset, range);
-	} else {
-		if (!mmio->is_write)
-			memset(mmio->data, 0, mmio->len);
-		updated_state = false;
-	}
-	spin_unlock(&vcpu->kvm->arch.vgic.lock);
-	kvm_prepare_mmio(run, mmio);
-	kvm_handle_mmio_return(vcpu, run);
-
-	if (updated_state)
-		vgic_kick_vcpus(vcpu->kvm);
-
-	return true;
-}
-
-/**
  * vgic_handle_mmio_access - handle an in-kernel MMIO access
  * This is called by the read/write KVM IO device wrappers below.
  * @vcpu:	pointer to the vcpu performing the access
@@ -835,30 +788,6 @@ static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-/**
- * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
- * @vcpu:      pointer to the vcpu performing the access
- * @run:       pointer to the kvm_run structure
- * @mmio:      pointer to the data describing the access
- *
- * returns true if the MMIO access has been performed in kernel space,
- * and false if it needs to be emulated in user space.
- * Calls the actual handling routine for the selected VGIC model.
- */
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-		      struct kvm_exit_mmio *mmio)
-{
-	if (!irqchip_in_kernel(vcpu->kvm))
-		return false;
-
-	/*
-	 * This will currently call either vgic_v2_handle_mmio() or
-	 * vgic_v3_handle_mmio(), which in turn will call
-	 * vgic_handle_mmio_range() defined above.
-	 */
-	return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
-}
-
 static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
 				 struct kvm_io_device *this,
 				 gpa_t addr, int len, void *val)
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index f2063a7..5dc55f2 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -99,11 +99,6 @@ const
 struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
 				      int len, gpa_t offset);
 
-bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
-			    struct kvm_exit_mmio *mmio,
-			    const struct vgic_io_range *ranges,
-			    unsigned long mmio_base);
-
 bool vgic_handle_enable_reg(struct kvm *kvm, struct kvm_exit_mmio *mmio,
 			    phys_addr_t offset, int vcpu_id, int access);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* [PATCH 12/12] KVM: arm/arm64: remove now obsolete VGIC specific MMIO handling code
@ 2015-03-13 16:10   ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-13 16:10 UTC (permalink / raw)
  To: linux-arm-kernel

With all of the virtual GIC emulation code now being registered with
the kvm_io_bus, we can remove all of the old MMIO handling code and
its dispatching functionality.

Signed-off-by: Andre Przywara <andre.przywara@arm.com>
---
 include/kvm/arm_vgic.h      |    2 --
 virt/kvm/arm/vgic-v2-emul.c |   19 ------------
 virt/kvm/arm/vgic-v3-emul.c |   39 ------------------------
 virt/kvm/arm/vgic.c         |   71 -------------------------------------------
 virt/kvm/arm/vgic.h         |    5 ---
 5 files changed, 136 deletions(-)

diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
index 91976c8..14d8401 100644
--- a/include/kvm/arm_vgic.h
+++ b/include/kvm/arm_vgic.h
@@ -140,8 +140,6 @@ struct vgic_params {
 };
 
 struct vgic_vm_ops {
-	bool	(*handle_mmio)(struct kvm_vcpu *, struct kvm_run *,
-			       struct kvm_exit_mmio *);
 	bool	(*queue_sgi)(struct kvm_vcpu *, int irq);
 	void	(*add_sgi_source)(struct kvm_vcpu *, int irq, int source);
 	int	(*init_model)(struct kvm *);
diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
index 6f685c9..ff264e7 100644
--- a/virt/kvm/arm/vgic-v2-emul.c
+++ b/virt/kvm/arm/vgic-v2-emul.c
@@ -388,24 +388,6 @@ static const struct vgic_io_range vgic_dist_ranges[] = {
 	{}
 };
 
-static bool vgic_v2_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-				struct kvm_exit_mmio *mmio)
-{
-	unsigned long base = vcpu->kvm->arch.vgic.vgic_dist_base;
-
-	if (!is_in_range(mmio->phys_addr, mmio->len, base,
-			 KVM_VGIC_V2_DIST_SIZE))
-		return false;
-
-	/* GICv2 does not support accesses wider than 32 bits */
-	if (mmio->len > 4) {
-		kvm_inject_dabt(vcpu, mmio->phys_addr);
-		return true;
-	}
-
-	return vgic_handle_mmio_range(vcpu, run, mmio, vgic_dist_ranges, base);
-}
-
 static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
 {
 	struct kvm *kvm = vcpu->kvm;
@@ -559,7 +541,6 @@ void vgic_v2_init_emulation(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
 
-	dist->vm_ops.handle_mmio = vgic_v2_handle_mmio;
 	dist->vm_ops.queue_sgi = vgic_v2_queue_sgi;
 	dist->vm_ops.add_sgi_source = vgic_v2_add_sgi_source;
 	dist->vm_ops.init_model = vgic_v2_init_model;
diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
index 8a81c67..4957577 100644
--- a/virt/kvm/arm/vgic-v3-emul.c
+++ b/virt/kvm/arm/vgic-v3-emul.c
@@ -710,45 +710,7 @@ static const struct vgic_io_range vgic_redist_ranges[] = {
 	{},
 };
 
-/*
- * This function splits accesses between the distributor and the two
- * redistributor parts (private/SPI). As each redistributor is accessible
- * from any CPU, we have to determine the affected VCPU by taking the faulting
- * address into account. We then pass this VCPU to the handler function via
- * the private parameter.
- */
 #define SGI_BASE_OFFSET SZ_64K
-static bool vgic_v3_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-				struct kvm_exit_mmio *mmio)
-{
-	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
-	unsigned long dbase = dist->vgic_dist_base;
-	unsigned long rdbase = dist->vgic_redist_base;
-	int nrcpus = atomic_read(&vcpu->kvm->online_vcpus);
-	int vcpu_id;
-	const struct vgic_io_range *mmio_range;
-
-	if (is_in_range(mmio->phys_addr, mmio->len, dbase, GIC_V3_DIST_SIZE)) {
-		return vgic_handle_mmio_range(vcpu, run, mmio,
-					      vgic_v3_dist_ranges, dbase);
-	}
-
-	if (!is_in_range(mmio->phys_addr, mmio->len, rdbase,
-	    GIC_V3_REDIST_SIZE * nrcpus))
-		return false;
-
-	vcpu_id = (mmio->phys_addr - rdbase) / GIC_V3_REDIST_SIZE;
-	rdbase += (vcpu_id * GIC_V3_REDIST_SIZE);
-	mmio->private = kvm_get_vcpu(vcpu->kvm, vcpu_id);
-
-	if (mmio->phys_addr >= rdbase + SGI_BASE_OFFSET) {
-		rdbase += SGI_BASE_OFFSET;
-		mmio_range = vgic_redist_sgi_ranges;
-	} else {
-		mmio_range = vgic_redist_ranges;
-	}
-	return vgic_handle_mmio_range(vcpu, run, mmio, mmio_range, rdbase);
-}
 
 static bool vgic_v3_queue_sgi(struct kvm_vcpu *vcpu, int irq)
 {
@@ -862,7 +824,6 @@ void vgic_v3_init_emulation(struct kvm *kvm)
 {
 	struct vgic_dist *dist = &kvm->arch.vgic;
 
-	dist->vm_ops.handle_mmio = vgic_v3_handle_mmio;
 	dist->vm_ops.queue_sgi = vgic_v3_queue_sgi;
 	dist->vm_ops.add_sgi_source = vgic_v3_add_sgi_source;
 	dist->vm_ops.init_model = vgic_v3_init_model;
diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
index f762738..f8a79d4 100644
--- a/virt/kvm/arm/vgic.c
+++ b/virt/kvm/arm/vgic.c
@@ -729,53 +729,6 @@ static bool call_range_handler(struct kvm_vcpu *vcpu,
 }
 
 /**
- * vgic_handle_mmio_range - handle an in-kernel MMIO access
- * @vcpu:	pointer to the vcpu performing the access
- * @run:	pointer to the kvm_run structure
- * @mmio:	pointer to the data describing the access
- * @ranges:	array of MMIO ranges in a given region
- * @mmio_base:	base address of that region
- *
- * returns true if the MMIO access could be performed
- */
-bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
-			    struct kvm_exit_mmio *mmio,
-			    const struct vgic_io_range *ranges,
-			    unsigned long mmio_base)
-{
-	const struct vgic_io_range *range;
-	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
-	bool updated_state;
-	unsigned long offset;
-
-	offset = mmio->phys_addr - mmio_base;
-	range = vgic_find_range(ranges, mmio->len, offset);
-	if (unlikely(!range || !range->handle_mmio)) {
-		pr_warn("Unhandled access %d %08llx %d\n",
-			mmio->is_write, mmio->phys_addr, mmio->len);
-		return false;
-	}
-
-	spin_lock(&vcpu->kvm->arch.vgic.lock);
-	offset -= range->base;
-	if (vgic_validate_access(dist, range, offset)) {
-		updated_state = call_range_handler(vcpu, mmio, offset, range);
-	} else {
-		if (!mmio->is_write)
-			memset(mmio->data, 0, mmio->len);
-		updated_state = false;
-	}
-	spin_unlock(&vcpu->kvm->arch.vgic.lock);
-	kvm_prepare_mmio(run, mmio);
-	kvm_handle_mmio_return(vcpu, run);
-
-	if (updated_state)
-		vgic_kick_vcpus(vcpu->kvm);
-
-	return true;
-}
-
-/**
  * vgic_handle_mmio_access - handle an in-kernel MMIO access
  * This is called by the read/write KVM IO device wrappers below.
  * @vcpu:	pointer to the vcpu performing the access
@@ -835,30 +788,6 @@ static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
 	return 0;
 }
 
-/**
- * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
- * @vcpu:      pointer to the vcpu performing the access
- * @run:       pointer to the kvm_run structure
- * @mmio:      pointer to the data describing the access
- *
- * returns true if the MMIO access has been performed in kernel space,
- * and false if it needs to be emulated in user space.
- * Calls the actual handling routine for the selected VGIC model.
- */
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
-		      struct kvm_exit_mmio *mmio)
-{
-	if (!irqchip_in_kernel(vcpu->kvm))
-		return false;
-
-	/*
-	 * This will currently call either vgic_v2_handle_mmio() or
-	 * vgic_v3_handle_mmio(), which in turn will call
-	 * vgic_handle_mmio_range() defined above.
-	 */
-	return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
-}
-
 static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
 				 struct kvm_io_device *this,
 				 gpa_t addr, int len, void *val)
diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
index f2063a7..5dc55f2 100644
--- a/virt/kvm/arm/vgic.h
+++ b/virt/kvm/arm/vgic.h
@@ -99,11 +99,6 @@ const
 struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
 				      int len, gpa_t offset);
 
-bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
-			    struct kvm_exit_mmio *mmio,
-			    const struct vgic_io_range *ranges,
-			    unsigned long mmio_base);
-
 bool vgic_handle_enable_reg(struct kvm *kvm, struct kvm_exit_mmio *mmio,
 			    phys_addr_t offset, int vcpu_id, int access);
 
-- 
1.7.9.5

^ permalink raw reply related	[flat|nested] 60+ messages in thread
* Re: [PATCH 04/12] KVM: x86: remove now unneeded include directory from Makefile
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 13:57     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 13:57 UTC (permalink / raw)
  To: Andre Przywara
  Cc: kvm, marc.zyngier, Marcelo Tosatti, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:04PM +0000, Andre Przywara wrote:
> virt/kvm was never really a good include directory for anything else
> than locally included headers.
> With the move of iodev.h there is no need anymore to add this
> directory the compiler's include path, so remove it from the x86 kvm
> Makefile.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Seems fine to me, but you need Paolo/Marcelo's ack here.

-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 04/12] KVM: x86: remove now unneeded include directory from Makefile
@ 2015-03-14 13:57     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 13:57 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:04PM +0000, Andre Przywara wrote:
> virt/kvm was never really a good include directory for anything else
> than locally included headers.
> With the move of iodev.h there is no need anymore to add this
> directory the compiler's include path, so remove it from the x86 kvm
> Makefile.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Seems fine to me, but you need Paolo/Marcelo's ack here.

-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:27     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:27 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:08PM +0000, Andre Przywara wrote:
> Currently we use a lot of VGIC specific code to do the MMIO
> dispatching.
> Use the previous reworks to add kvm_io_bus style MMIO handlers.
> 
> Those are not yet called by the MMIO abort handler, also the actual
> VGIC emulator function do not make use of it yet, but will be enabled
> with the following patches.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  include/kvm/arm_vgic.h |    9 ++++
>  virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic.h    |    7 +++
>  3 files changed, 127 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index b81630b..4bfc6a3 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -24,6 +24,7 @@
>  #include <linux/irqreturn.h>
>  #include <linux/spinlock.h>
>  #include <linux/types.h>
> +#include <kvm/iodev.h>
>  
>  #define VGIC_NR_IRQS_LEGACY	256
>  #define VGIC_NR_SGIS		16
> @@ -147,6 +148,14 @@ struct vgic_vm_ops {
>  	int	(*map_resources)(struct kvm *, const struct vgic_params *);
>  };
>  
> +struct vgic_io_device {
> +	gpa_t addr;
> +	int len;
> +	const struct vgic_io_range *reg_ranges;
> +	struct kvm_vcpu *redist_vcpu;
> +	struct kvm_io_device dev;
> +};
> +
>  struct vgic_dist {
>  	spinlock_t		lock;
>  	bool			in_kernel;
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 7aae19b..71389b8 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -32,6 +32,8 @@
>  #include <asm/kvm_arm.h>
>  #include <asm/kvm_mmu.h>
>  #include <trace/events/kvm.h>
> +#include <asm/kvm.h>
> +#include <kvm/iodev.h>
>  
>  /*
>   * How the whole thing works (courtesy of Christoffer Dall):
> @@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  }
>  
>  /**
> + * vgic_handle_mmio_access - handle an in-kernel MMIO access
> + * This is called by the read/write KVM IO device wrappers below.
> + * @vcpu:	pointer to the vcpu performing the access
> + * @this:	pointer to the KVM IO device in charge
> + * @addr:	guest physical address of the access
> + * @len:	size of the access
> + * @val:	pointer to the data region
> + * @is_write:	read or write access
> + *
> + * returns true if the MMIO access could be performed
> + */
> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
> +				   struct kvm_io_device *this, gpa_t addr,
> +				   int len, void *val, bool is_write)
> +{
> +	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> +	struct vgic_io_device *iodev = container_of(this,
> +						    struct vgic_io_device, dev);
> +	struct kvm_run *run = vcpu->run;
> +	const struct vgic_io_range *range;
> +	struct kvm_exit_mmio mmio;
> +	bool updated_state;
> +	gpa_t offset;
> +
> +	offset = addr - iodev->addr;
> +	range = vgic_find_range(iodev->reg_ranges, len, offset);
> +	if (unlikely(!range || !range->handle_mmio)) {
> +		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
> +		return -ENXIO;
> +	}
> +
> +	mmio.phys_addr = addr;
> +	mmio.len = len;
> +	mmio.is_write = is_write;
> +	if (is_write)
> +		memcpy(mmio.data, val, len);
> +	mmio.private = iodev->redist_vcpu;
> +
> +	spin_lock(&dist->lock);
> +	offset -= range->base;
> +	if (vgic_validate_access(dist, range, offset)) {
> +		updated_state = call_range_handler(vcpu, &mmio, offset, range);
> +		if (!is_write)
> +			memcpy(val, mmio.data, len);
> +	} else {
> +		if (!is_write)
> +			memset(val, 0, len);
> +		updated_state = false;
> +	}
> +	spin_unlock(&dist->lock);
> +	kvm_prepare_mmio(run, &mmio);

we're not the only user of kvm_exit_mmio I believe, so we could rename
this to vgic_io as well and you could change the mmio.data array to be a
void *val pointer, which just gets set to the pointer passed into this
function (which I think points to the kvm_run structs data array) and
you can avoid all these memcopies, right?

> +	kvm_handle_mmio_return(vcpu, run);
> +
> +	if (updated_state)
> +		vgic_kick_vcpus(vcpu->kvm);
> +
> +	return 0;
> +}
> +
> +/**
>   * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
>   * @vcpu:      pointer to the vcpu performing the access
>   * @run:       pointer to the kvm_run structure
> @@ -797,6 +859,55 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  	return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
>  }
>  
> +static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
> +				 struct kvm_io_device *this,
> +				 gpa_t addr, int len, void *val)
> +{
> +	return vgic_handle_mmio_access(vcpu, this, addr, len, val, false);
> +}
> +
> +static int vgic_handle_mmio_write(struct kvm_vcpu *vcpu,
> +				  struct kvm_io_device *this,
> +				  gpa_t addr, int len, const void *val)
> +{
> +	return vgic_handle_mmio_access(vcpu, this, addr, len, (void *)val,
> +				       true);
> +}
> +
> +struct kvm_io_device_ops vgic_io_ops = {
> +	.read	= vgic_handle_mmio_read,
> +	.write	= vgic_handle_mmio_write,
> +};
> +

can you add kdocs to this exported function?

> +int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
> +			     const struct vgic_io_range *ranges,
> +			     int redist_id,

nit: consider renaming to redist_vcpu_id

> +			     struct vgic_io_device *iodev)
> +{
> +	struct kvm_vcpu *vcpu = NULL;
> +	int ret;
> +
> +	if (redist_id >= 0)
> +		vcpu = kvm_get_vcpu(kvm, redist_id);
> +
> +	iodev->addr		= base;
> +	iodev->len		= len;
> +	iodev->reg_ranges	= ranges;
> +	iodev->redist_vcpu	= vcpu;
> +
> +	kvm_iodevice_init(&iodev->dev, &vgic_io_ops);
> +
> +	mutex_lock(&kvm->slots_lock);
> +
> +	ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, base, len,
> +				      &iodev->dev);
> +	mutex_unlock(&kvm->slots_lock);
> +	if (ret < 0)
> +		return ret;
> +
> +	return 0;

kvm_io_bus_register_dev returns either 0 or -ERRNO, so you can just
return ret here.

> +}
> +
>  static int vgic_nr_shared_irqs(struct vgic_dist *dist)
>  {
>  	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
> diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
> index ffafb15..f2063a7 100644
> --- a/virt/kvm/arm/vgic.h
> +++ b/virt/kvm/arm/vgic.h
> @@ -20,6 +20,8 @@
>  #ifndef __KVM_VGIC_H__
>  #define __KVM_VGIC_H__
>  
> +#include <kvm/iodev.h>
> +
>  #define VGIC_ADDR_UNDEF		(-1)
>  #define IS_VGIC_ADDR_UNDEF(_x)  ((_x) == VGIC_ADDR_UNDEF)
>  
> @@ -82,6 +84,11 @@ struct vgic_io_range {
>  			    phys_addr_t offset);
>  };
>  
> +int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
> +			     const struct vgic_io_range *ranges,
> +			     int redist_id,
> +			     struct vgic_io_device *iodev);
> +
>  static inline bool is_in_range(phys_addr_t addr, unsigned long len,
>  			       phys_addr_t baseaddr, unsigned long size)
>  {
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
@ 2015-03-14 14:27     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:27 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:08PM +0000, Andre Przywara wrote:
> Currently we use a lot of VGIC specific code to do the MMIO
> dispatching.
> Use the previous reworks to add kvm_io_bus style MMIO handlers.
> 
> Those are not yet called by the MMIO abort handler, also the actual
> VGIC emulator function do not make use of it yet, but will be enabled
> with the following patches.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  include/kvm/arm_vgic.h |    9 ++++
>  virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
>  virt/kvm/arm/vgic.h    |    7 +++
>  3 files changed, 127 insertions(+)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index b81630b..4bfc6a3 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -24,6 +24,7 @@
>  #include <linux/irqreturn.h>
>  #include <linux/spinlock.h>
>  #include <linux/types.h>
> +#include <kvm/iodev.h>
>  
>  #define VGIC_NR_IRQS_LEGACY	256
>  #define VGIC_NR_SGIS		16
> @@ -147,6 +148,14 @@ struct vgic_vm_ops {
>  	int	(*map_resources)(struct kvm *, const struct vgic_params *);
>  };
>  
> +struct vgic_io_device {
> +	gpa_t addr;
> +	int len;
> +	const struct vgic_io_range *reg_ranges;
> +	struct kvm_vcpu *redist_vcpu;
> +	struct kvm_io_device dev;
> +};
> +
>  struct vgic_dist {
>  	spinlock_t		lock;
>  	bool			in_kernel;
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 7aae19b..71389b8 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -32,6 +32,8 @@
>  #include <asm/kvm_arm.h>
>  #include <asm/kvm_mmu.h>
>  #include <trace/events/kvm.h>
> +#include <asm/kvm.h>
> +#include <kvm/iodev.h>
>  
>  /*
>   * How the whole thing works (courtesy of Christoffer Dall):
> @@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  }
>  
>  /**
> + * vgic_handle_mmio_access - handle an in-kernel MMIO access
> + * This is called by the read/write KVM IO device wrappers below.
> + * @vcpu:	pointer to the vcpu performing the access
> + * @this:	pointer to the KVM IO device in charge
> + * @addr:	guest physical address of the access
> + * @len:	size of the access
> + * @val:	pointer to the data region
> + * @is_write:	read or write access
> + *
> + * returns true if the MMIO access could be performed
> + */
> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
> +				   struct kvm_io_device *this, gpa_t addr,
> +				   int len, void *val, bool is_write)
> +{
> +	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> +	struct vgic_io_device *iodev = container_of(this,
> +						    struct vgic_io_device, dev);
> +	struct kvm_run *run = vcpu->run;
> +	const struct vgic_io_range *range;
> +	struct kvm_exit_mmio mmio;
> +	bool updated_state;
> +	gpa_t offset;
> +
> +	offset = addr - iodev->addr;
> +	range = vgic_find_range(iodev->reg_ranges, len, offset);
> +	if (unlikely(!range || !range->handle_mmio)) {
> +		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
> +		return -ENXIO;
> +	}
> +
> +	mmio.phys_addr = addr;
> +	mmio.len = len;
> +	mmio.is_write = is_write;
> +	if (is_write)
> +		memcpy(mmio.data, val, len);
> +	mmio.private = iodev->redist_vcpu;
> +
> +	spin_lock(&dist->lock);
> +	offset -= range->base;
> +	if (vgic_validate_access(dist, range, offset)) {
> +		updated_state = call_range_handler(vcpu, &mmio, offset, range);
> +		if (!is_write)
> +			memcpy(val, mmio.data, len);
> +	} else {
> +		if (!is_write)
> +			memset(val, 0, len);
> +		updated_state = false;
> +	}
> +	spin_unlock(&dist->lock);
> +	kvm_prepare_mmio(run, &mmio);

we're not the only user of kvm_exit_mmio I believe, so we could rename
this to vgic_io as well and you could change the mmio.data array to be a
void *val pointer, which just gets set to the pointer passed into this
function (which I think points to the kvm_run structs data array) and
you can avoid all these memcopies, right?

> +	kvm_handle_mmio_return(vcpu, run);
> +
> +	if (updated_state)
> +		vgic_kick_vcpus(vcpu->kvm);
> +
> +	return 0;
> +}
> +
> +/**
>   * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
>   * @vcpu:      pointer to the vcpu performing the access
>   * @run:       pointer to the kvm_run structure
> @@ -797,6 +859,55 @@ bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  	return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
>  }
>  
> +static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
> +				 struct kvm_io_device *this,
> +				 gpa_t addr, int len, void *val)
> +{
> +	return vgic_handle_mmio_access(vcpu, this, addr, len, val, false);
> +}
> +
> +static int vgic_handle_mmio_write(struct kvm_vcpu *vcpu,
> +				  struct kvm_io_device *this,
> +				  gpa_t addr, int len, const void *val)
> +{
> +	return vgic_handle_mmio_access(vcpu, this, addr, len, (void *)val,
> +				       true);
> +}
> +
> +struct kvm_io_device_ops vgic_io_ops = {
> +	.read	= vgic_handle_mmio_read,
> +	.write	= vgic_handle_mmio_write,
> +};
> +

can you add kdocs to this exported function?

> +int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
> +			     const struct vgic_io_range *ranges,
> +			     int redist_id,

nit: consider renaming to redist_vcpu_id

> +			     struct vgic_io_device *iodev)
> +{
> +	struct kvm_vcpu *vcpu = NULL;
> +	int ret;
> +
> +	if (redist_id >= 0)
> +		vcpu = kvm_get_vcpu(kvm, redist_id);
> +
> +	iodev->addr		= base;
> +	iodev->len		= len;
> +	iodev->reg_ranges	= ranges;
> +	iodev->redist_vcpu	= vcpu;
> +
> +	kvm_iodevice_init(&iodev->dev, &vgic_io_ops);
> +
> +	mutex_lock(&kvm->slots_lock);
> +
> +	ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, base, len,
> +				      &iodev->dev);
> +	mutex_unlock(&kvm->slots_lock);
> +	if (ret < 0)
> +		return ret;
> +
> +	return 0;

kvm_io_bus_register_dev returns either 0 or -ERRNO, so you can just
return ret here.

> +}
> +
>  static int vgic_nr_shared_irqs(struct vgic_dist *dist)
>  {
>  	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
> diff --git a/virt/kvm/arm/vgic.h b/virt/kvm/arm/vgic.h
> index ffafb15..f2063a7 100644
> --- a/virt/kvm/arm/vgic.h
> +++ b/virt/kvm/arm/vgic.h
> @@ -20,6 +20,8 @@
>  #ifndef __KVM_VGIC_H__
>  #define __KVM_VGIC_H__
>  
> +#include <kvm/iodev.h>
> +
>  #define VGIC_ADDR_UNDEF		(-1)
>  #define IS_VGIC_ADDR_UNDEF(_x)  ((_x) == VGIC_ADDR_UNDEF)
>  
> @@ -82,6 +84,11 @@ struct vgic_io_range {
>  			    phys_addr_t offset);
>  };
>  
> +int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
> +			     const struct vgic_io_range *ranges,
> +			     int redist_id,
> +			     struct vgic_io_device *iodev);
> +
>  static inline bool is_in_range(phys_addr_t addr, unsigned long len,
>  			       phys_addr_t baseaddr, unsigned long size)
>  {
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:30     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:30 UTC (permalink / raw)
  To: Andre Przywara
  Cc: marc.zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:09PM +0000, Andre Przywara wrote:
> Using the framework provided by the recent vgic.c changes we register
> a kvm_io_bus device when initializing the virtual GICv2.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  include/kvm/arm_vgic.h      |    1 +
>  virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
>  virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
>  3 files changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 4bfc6a3..74a4ac4 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -245,6 +245,7 @@ struct vgic_dist {
>  	unsigned long		*irq_pending_on_cpu;
>  
>  	struct vgic_vm_ops	vm_ops;
> +	struct vgic_io_device	dist_iodev;
>  };
>  
>  struct vgic_v2_cpu_if {
> diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
> index 0defac6..6f685c9 100644
> --- a/virt/kvm/arm/vgic-v2-emul.c
> +++ b/virt/kvm/arm/vgic-v2-emul.c
> @@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
>  static int vgic_v2_map_resources(struct kvm *kvm,
>  				 const struct vgic_params *params)
>  {
> +	struct vgic_dist *dist = &kvm->arch.vgic;
>  	int ret = 0;
>  
>  	if (!irqchip_in_kernel(kvm))
> @@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>  	if (vgic_ready(kvm))
>  		goto out;
>  
> -	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
> -	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
> +	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
> +	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
>  		kvm_err("Need to set vgic cpu and dist addresses first\n");
>  		ret = -ENXIO;
>  		goto out;
>  	}
>  
> +	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
> +				 KVM_VGIC_V2_DIST_SIZE,
> +				 vgic_dist_ranges, -1, &dist->dist_iodev);
> +
>  	/*
>  	 * Initialize the vgic if this hasn't already been done on demand by
>  	 * accessing the vgic state from userspace.
> @@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>  		goto out;
>  	}
>  
> -	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
> +	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
>  				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
>  				    true);
>  	if (ret) {
> @@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>  		goto out;
>  	}
>  
> -	kvm->arch.vgic.ready = true;
> +	dist->ready = true;
>  out:
>  	if (ret)
>  		kvm_vgic_destroy(kvm);
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 71389b8..b1dd717 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
>  	return 0;
>  }
>  
> +static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +
> +	if (!dist || !kvm->buses[KVM_MMIO_BUS])
> +		return;
> +
> +	mutex_lock(&kvm->slots_lock);
> +	if (dist->dist_iodev.dev.ops)
> +		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
> +					  &dist->dist_iodev.dev);

why are we only unregisttering the iodev when we have ops?

> +	mutex_unlock(&kvm->slots_lock);
> +}
> +
>  static int vgic_nr_shared_irqs(struct vgic_dist *dist)
>  {
>  	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
> @@ -1546,6 +1560,8 @@ void kvm_vgic_destroy(struct kvm *kvm)
>  	struct kvm_vcpu *vcpu;
>  	int i;
>  
> +	vgic_unregister_kvm_io_dev(kvm);
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_destroy(vcpu);
>  
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
@ 2015-03-14 14:30     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:30 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:09PM +0000, Andre Przywara wrote:
> Using the framework provided by the recent vgic.c changes we register
> a kvm_io_bus device when initializing the virtual GICv2.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  include/kvm/arm_vgic.h      |    1 +
>  virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
>  virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
>  3 files changed, 26 insertions(+), 4 deletions(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 4bfc6a3..74a4ac4 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -245,6 +245,7 @@ struct vgic_dist {
>  	unsigned long		*irq_pending_on_cpu;
>  
>  	struct vgic_vm_ops	vm_ops;
> +	struct vgic_io_device	dist_iodev;
>  };
>  
>  struct vgic_v2_cpu_if {
> diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
> index 0defac6..6f685c9 100644
> --- a/virt/kvm/arm/vgic-v2-emul.c
> +++ b/virt/kvm/arm/vgic-v2-emul.c
> @@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
>  static int vgic_v2_map_resources(struct kvm *kvm,
>  				 const struct vgic_params *params)
>  {
> +	struct vgic_dist *dist = &kvm->arch.vgic;
>  	int ret = 0;
>  
>  	if (!irqchip_in_kernel(kvm))
> @@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>  	if (vgic_ready(kvm))
>  		goto out;
>  
> -	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
> -	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
> +	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
> +	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
>  		kvm_err("Need to set vgic cpu and dist addresses first\n");
>  		ret = -ENXIO;
>  		goto out;
>  	}
>  
> +	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
> +				 KVM_VGIC_V2_DIST_SIZE,
> +				 vgic_dist_ranges, -1, &dist->dist_iodev);
> +
>  	/*
>  	 * Initialize the vgic if this hasn't already been done on demand by
>  	 * accessing the vgic state from userspace.
> @@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>  		goto out;
>  	}
>  
> -	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
> +	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
>  				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
>  				    true);
>  	if (ret) {
> @@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>  		goto out;
>  	}
>  
> -	kvm->arch.vgic.ready = true;
> +	dist->ready = true;
>  out:
>  	if (ret)
>  		kvm_vgic_destroy(kvm);
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index 71389b8..b1dd717 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
>  	return 0;
>  }
>  
> +static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
> +{
> +	struct vgic_dist *dist = &kvm->arch.vgic;
> +
> +	if (!dist || !kvm->buses[KVM_MMIO_BUS])
> +		return;
> +
> +	mutex_lock(&kvm->slots_lock);
> +	if (dist->dist_iodev.dev.ops)
> +		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
> +					  &dist->dist_iodev.dev);

why are we only unregisttering the iodev when we have ops?

> +	mutex_unlock(&kvm->slots_lock);
> +}
> +
>  static int vgic_nr_shared_irqs(struct vgic_dist *dist)
>  {
>  	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
> @@ -1546,6 +1560,8 @@ void kvm_vgic_destroy(struct kvm *kvm)
>  	struct kvm_vcpu *vcpu;
>  	int i;
>  
> +	vgic_unregister_kvm_io_dev(kvm);
> +
>  	kvm_for_each_vcpu(i, vcpu, kvm)
>  		kvm_vgic_vcpu_destroy(vcpu);
>  
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 10/12] KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO handling
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:39     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:39 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:10PM +0000, Andre Przywara wrote:
> Using the framework provided by the recent vgic.c changes, we
> register a kvm_io_bus device on mapping the virtual GICv3 resources.
> The distributor mapping is pretty straight forward, but the
> redistributors need some more love, since they need to be tagged with
> the respective redistributor (read: VCPU) they are connected with.
> We use the kvm_io_bus framework to register two devices per VCPU, as
> each block is handled independently by the VGIC code.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  include/kvm/arm_vgic.h      |    1 +
>  virt/kvm/arm/vgic-v3-emul.c |   32 +++++++++++++++++++++++++++++++-
>  virt/kvm/arm/vgic.c         |   14 ++++++++++++++
>  3 files changed, 46 insertions(+), 1 deletion(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 74a4ac4..09fd324 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -246,6 +246,7 @@ struct vgic_dist {
>  
>  	struct vgic_vm_ops	vm_ops;
>  	struct vgic_io_device	dist_iodev;
> +	struct vgic_io_device	*redist_iodevs;
>  };
>  
>  struct vgic_v2_cpu_if {
> diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
> index 14943e3..8a81c67 100644
> --- a/virt/kvm/arm/vgic-v3-emul.c
> +++ b/virt/kvm/arm/vgic-v3-emul.c
> @@ -766,6 +766,9 @@ static int vgic_v3_map_resources(struct kvm *kvm,
>  {
>  	int ret = 0;
>  	struct vgic_dist *dist = &kvm->arch.vgic;
> +	gpa_t rdbase = dist->vgic_redist_base;
> +	struct vgic_io_device *iodevs = NULL;
> +	int i;
>  
>  	if (!irqchip_in_kernel(kvm))
>  		return 0;
> @@ -791,7 +794,34 @@ static int vgic_v3_map_resources(struct kvm *kvm,
>  		goto out;
>  	}
>  
> -	kvm->arch.vgic.ready = true;
> +	ret = vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
> +				       GIC_V3_DIST_SIZE, vgic_v3_dist_ranges,
> +				       -1, &dist->dist_iodev);
> +	if (ret)
> +		goto out;
> +
> +	iodevs = kcalloc(dist->nr_cpus * 2, sizeof(iodevs[0]), GFP_KERNEL);
> +	if (!iodevs) {
> +		ret = -ENOMEM;
> +		goto out;
> +	}
> +	for (i = 0; i < dist->nr_cpus; i++) {
> +		ret = vgic_register_kvm_io_dev(kvm, rdbase,
> +					       SZ_64K, vgic_redist_ranges,
> +					       i, &iodevs[i * 2]);
> +		if (ret)
> +			goto out;
> +		ret = vgic_register_kvm_io_dev(kvm, rdbase + SGI_BASE_OFFSET,
> +					       SZ_64K, vgic_redist_sgi_ranges,
> +					       i, &iodevs[i * 2 + 1]);
> +		if (ret)
> +			goto out;
> +		rdbase += GIC_V3_REDIST_SIZE;
> +	}
> +
> +	dist->redist_iodevs = iodevs;
> +	dist->ready = true;
> +

May be worth placing a comment in the error path that kvm_vgic_destroy
will clean up all registered kvm iodevs and free the allocated array.

>  out:
>  	if (ret)
>  		kvm_vgic_destroy(kvm);
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index b1dd717..f762738 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -911,6 +911,8 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
>  static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
>  {
>  	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct vgic_io_device *iodevs;
> +	int i;
>  
>  	if (!dist || !kvm->buses[KVM_MMIO_BUS])
>  		return;
> @@ -919,6 +921,18 @@ static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
>  	if (dist->dist_iodev.dev.ops)
>  		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
>  					  &dist->dist_iodev.dev);
> +
> +	iodevs = dist->redist_iodevs;
> +	if (iodevs) {
> +		for (i = 0; i < dist->nr_cpus * 2; i++) {
> +			if (!iodevs[i].dev.ops)
> +				continue;

this looks a bit strange because vgic_register_kvm_io_dev does not clear
the ops pointer if kvm_io_bus_register_dev fails... ?

> +			kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
> +						  &iodevs[i].dev);
> +		}
> +		kfree(iodevs);
> +		dist->redist_iodevs = NULL;
> +	}
>  	mutex_unlock(&kvm->slots_lock);
>  }
>  
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 10/12] KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO handling
@ 2015-03-14 14:39     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:39 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:10PM +0000, Andre Przywara wrote:
> Using the framework provided by the recent vgic.c changes, we
> register a kvm_io_bus device on mapping the virtual GICv3 resources.
> The distributor mapping is pretty straight forward, but the
> redistributors need some more love, since they need to be tagged with
> the respective redistributor (read: VCPU) they are connected with.
> We use the kvm_io_bus framework to register two devices per VCPU, as
> each block is handled independently by the VGIC code.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  include/kvm/arm_vgic.h      |    1 +
>  virt/kvm/arm/vgic-v3-emul.c |   32 +++++++++++++++++++++++++++++++-
>  virt/kvm/arm/vgic.c         |   14 ++++++++++++++
>  3 files changed, 46 insertions(+), 1 deletion(-)
> 
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 74a4ac4..09fd324 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -246,6 +246,7 @@ struct vgic_dist {
>  
>  	struct vgic_vm_ops	vm_ops;
>  	struct vgic_io_device	dist_iodev;
> +	struct vgic_io_device	*redist_iodevs;
>  };
>  
>  struct vgic_v2_cpu_if {
> diff --git a/virt/kvm/arm/vgic-v3-emul.c b/virt/kvm/arm/vgic-v3-emul.c
> index 14943e3..8a81c67 100644
> --- a/virt/kvm/arm/vgic-v3-emul.c
> +++ b/virt/kvm/arm/vgic-v3-emul.c
> @@ -766,6 +766,9 @@ static int vgic_v3_map_resources(struct kvm *kvm,
>  {
>  	int ret = 0;
>  	struct vgic_dist *dist = &kvm->arch.vgic;
> +	gpa_t rdbase = dist->vgic_redist_base;
> +	struct vgic_io_device *iodevs = NULL;
> +	int i;
>  
>  	if (!irqchip_in_kernel(kvm))
>  		return 0;
> @@ -791,7 +794,34 @@ static int vgic_v3_map_resources(struct kvm *kvm,
>  		goto out;
>  	}
>  
> -	kvm->arch.vgic.ready = true;
> +	ret = vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
> +				       GIC_V3_DIST_SIZE, vgic_v3_dist_ranges,
> +				       -1, &dist->dist_iodev);
> +	if (ret)
> +		goto out;
> +
> +	iodevs = kcalloc(dist->nr_cpus * 2, sizeof(iodevs[0]), GFP_KERNEL);
> +	if (!iodevs) {
> +		ret = -ENOMEM;
> +		goto out;
> +	}
> +	for (i = 0; i < dist->nr_cpus; i++) {
> +		ret = vgic_register_kvm_io_dev(kvm, rdbase,
> +					       SZ_64K, vgic_redist_ranges,
> +					       i, &iodevs[i * 2]);
> +		if (ret)
> +			goto out;
> +		ret = vgic_register_kvm_io_dev(kvm, rdbase + SGI_BASE_OFFSET,
> +					       SZ_64K, vgic_redist_sgi_ranges,
> +					       i, &iodevs[i * 2 + 1]);
> +		if (ret)
> +			goto out;
> +		rdbase += GIC_V3_REDIST_SIZE;
> +	}
> +
> +	dist->redist_iodevs = iodevs;
> +	dist->ready = true;
> +

May be worth placing a comment in the error path that kvm_vgic_destroy
will clean up all registered kvm iodevs and free the allocated array.

>  out:
>  	if (ret)
>  		kvm_vgic_destroy(kvm);
> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> index b1dd717..f762738 100644
> --- a/virt/kvm/arm/vgic.c
> +++ b/virt/kvm/arm/vgic.c
> @@ -911,6 +911,8 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
>  static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
>  {
>  	struct vgic_dist *dist = &kvm->arch.vgic;
> +	struct vgic_io_device *iodevs;
> +	int i;
>  
>  	if (!dist || !kvm->buses[KVM_MMIO_BUS])
>  		return;
> @@ -919,6 +921,18 @@ static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
>  	if (dist->dist_iodev.dev.ops)
>  		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
>  					  &dist->dist_iodev.dev);
> +
> +	iodevs = dist->redist_iodevs;
> +	if (iodevs) {
> +		for (i = 0; i < dist->nr_cpus * 2; i++) {
> +			if (!iodevs[i].dev.ops)
> +				continue;

this looks a bit strange because vgic_register_kvm_io_dev does not clear
the ops pointer if kvm_io_bus_register_dev fails... ?

> +			kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
> +						  &iodevs[i].dev);
> +		}
> +		kfree(iodevs);
> +		dist->redist_iodevs = NULL;
> +	}
>  	mutex_unlock(&kvm->slots_lock);
>  }
>  
> -- 
> 1.7.9.5
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 11/12] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:43     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:11PM +0000, Andre Przywara wrote:
> From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> 
> On IO memory abort, try to handle the MMIO access through the KVM
> registered read/write callbacks. This is done by invoking the relevant
> kvm_io_bus_* API.
> 
> [Andre: Since we converted the VGIC already, we can get rid of the
> VGIC specific MMIO handler alltogether.]
> 
> Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  arch/arm/kvm/mmio.c    |   32 +++++++++++++++++++++++++++++++-
>  include/kvm/arm_vgic.h |    2 --
>  2 files changed, 31 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
> index 5d3bfc0..8dc2fde 100644
> --- a/arch/arm/kvm/mmio.c
> +++ b/arch/arm/kvm/mmio.c
> @@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>  	return 0;
>  }
>  
> +/**
> + * handle_kernel_mmio - handle an in-kernel MMIO access
> + * @vcpu:	pointer to the vcpu performing the access
> + * @run:	pointer to the kvm_run structure
> + * @mmio:	pointer to the data describing the access
> + *
> + * returns true if the MMIO access has been performed in kernel space,
> + * and false if it needs to be emulated in user space.
> + */
> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> +		struct kvm_exit_mmio *mmio)
> +{
> +	int ret;
> +
> +	if (mmio->is_write) {
> +		ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
> +				mmio->len, &mmio->data);
> +
> +	} else {
> +		ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
> +				mmio->len, &mmio->data);
> +	}
> +	if (!ret) {
> +		kvm_prepare_mmio(run, mmio);
> +		kvm_handle_mmio_return(vcpu, run);

here I think you can optimize the copying, see my comment on the earlier
patch.

let's be nice to readers here and

return true

> +	}
> +
> +	return !ret;

and replace this with

return false

> +}
> +
>  int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  		 phys_addr_t fault_ipa)
>  {
> @@ -200,7 +230,7 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  			       fault_ipa, 0);
>  	}
>  
> -	if (vgic_handle_mmio(vcpu, run, &mmio))
> +	if (handle_kernel_mmio(vcpu, run, &mmio))
>  		return 1;
>  
>  	kvm_prepare_mmio(run, &mmio);
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 09fd324..91976c8 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -317,8 +317,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
>  			bool level);
>  void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
>  int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> -		      struct kvm_exit_mmio *mmio);
>  
>  #define irqchip_in_kernel(k)	(!!((k)->arch.vgic.in_kernel))
>  #define vgic_initialized(k)	(!!((k)->arch.vgic.nr_cpus))
> -- 
> 1.7.9.5
> 

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 11/12] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus
@ 2015-03-14 14:43     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:11PM +0000, Andre Przywara wrote:
> From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> 
> On IO memory abort, try to handle the MMIO access through the KVM
> registered read/write callbacks. This is done by invoking the relevant
> kvm_io_bus_* API.
> 
> [Andre: Since we converted the VGIC already, we can get rid of the
> VGIC specific MMIO handler alltogether.]
> 
> Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  arch/arm/kvm/mmio.c    |   32 +++++++++++++++++++++++++++++++-
>  include/kvm/arm_vgic.h |    2 --
>  2 files changed, 31 insertions(+), 3 deletions(-)
> 
> diff --git a/arch/arm/kvm/mmio.c b/arch/arm/kvm/mmio.c
> index 5d3bfc0..8dc2fde 100644
> --- a/arch/arm/kvm/mmio.c
> +++ b/arch/arm/kvm/mmio.c
> @@ -162,6 +162,36 @@ static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
>  	return 0;
>  }
>  
> +/**
> + * handle_kernel_mmio - handle an in-kernel MMIO access
> + * @vcpu:	pointer to the vcpu performing the access
> + * @run:	pointer to the kvm_run structure
> + * @mmio:	pointer to the data describing the access
> + *
> + * returns true if the MMIO access has been performed in kernel space,
> + * and false if it needs to be emulated in user space.
> + */
> +static bool handle_kernel_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> +		struct kvm_exit_mmio *mmio)
> +{
> +	int ret;
> +
> +	if (mmio->is_write) {
> +		ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
> +				mmio->len, &mmio->data);
> +
> +	} else {
> +		ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, mmio->phys_addr,
> +				mmio->len, &mmio->data);
> +	}
> +	if (!ret) {
> +		kvm_prepare_mmio(run, mmio);
> +		kvm_handle_mmio_return(vcpu, run);

here I think you can optimize the copying, see my comment on the earlier
patch.

let's be nice to readers here and

return true

> +	}
> +
> +	return !ret;

and replace this with

return false

> +}
> +
>  int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  		 phys_addr_t fault_ipa)
>  {
> @@ -200,7 +230,7 @@ int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
>  			       fault_ipa, 0);
>  	}
>  
> -	if (vgic_handle_mmio(vcpu, run, &mmio))
> +	if (handle_kernel_mmio(vcpu, run, &mmio))
>  		return 1;
>  
>  	kvm_prepare_mmio(run, &mmio);
> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> index 09fd324..91976c8 100644
> --- a/include/kvm/arm_vgic.h
> +++ b/include/kvm/arm_vgic.h
> @@ -317,8 +317,6 @@ int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid, unsigned int irq_num,
>  			bool level);
>  void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
>  int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
> -bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
> -		      struct kvm_exit_mmio *mmio);
>  
>  #define irqchip_in_kernel(k)	(!!((k)->arch.vgic.in_kernel))
>  #define vgic_initialized(k)	(!!((k)->arch.vgic.nr_cpus))
> -- 
> 1.7.9.5
> 

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 02/12] KVM: move iodev.h from virt/kvm/ to include/kvm
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:43     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: Andre Przywara
  Cc: marc.zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:02PM +0000, Andre Przywara wrote:
> iodev.h contains definitions for the kvm_io_bus framework. This is
> needed both by the generic KVM code in virt/kvm as well as by
> architecture specific code under arch/. Putting the header file in
> virt/kvm and using local includes in the architecture part seems at
> least dodgy to me, so let's move the file into include/kvm, so that a
> more natural "#include <kvm/iodev.h>" can be used by all of the code.
> This also solves a problem later when using struct kvm_io_device
> in arm_vgic.h.
> Fixing up the FSF address in the GPL header and a wrong include path
> on the way.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 02/12] KVM: move iodev.h from virt/kvm/ to include/kvm
@ 2015-03-14 14:43     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:02PM +0000, Andre Przywara wrote:
> iodev.h contains definitions for the kvm_io_bus framework. This is
> needed both by the generic KVM code in virt/kvm as well as by
> architecture specific code under arch/. Putting the header file in
> virt/kvm and using local includes in the architecture part seems at
> least dodgy to me, so let's move the file into include/kvm, so that a
> more natural "#include <kvm/iodev.h>" can be used by all of the code.
> This also solves a problem later when using struct kvm_io_device
> in arm_vgic.h.
> Fixing up the FSF address in the GPL header and a wrong include path
> on the way.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 01/12] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks.
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:43     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:01PM +0000, Andre Przywara wrote:
> From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> 
> This is needed in e.g. ARM vGIC emulation, where the MMIO handling
> depends on the VCPU that does the access.
> 
> Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> Acked-by: Paolo Bonzini <pbonzini at redhat.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 01/12] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks.
@ 2015-03-14 14:43     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:01PM +0000, Andre Przywara wrote:
> From: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> 
> This is needed in e.g. ARM vGIC emulation, where the MMIO handling
> depends on the VCPU that does the access.
> 
> Signed-off-by: Nikolay Nikolaev <n.nikolaev@virtualopensystems.com>
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> Acked-by: Paolo Bonzini <pbonzini@redhat.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 03/12] KVM: arm/arm64: remove now unneeded include directory from Makefile
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:43     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:03PM +0000, Andre Przywara wrote:
> virt/kvm was never really a good include directory for anything else
> than locally included headers.
> With the move of iodev.h there is no need anymore to add this
> directory the compiler's include path, so remove it from the arm and
> arm64 kvm Makefile.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 03/12] KVM: arm/arm64: remove now unneeded include directory from Makefile
@ 2015-03-14 14:43     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:03PM +0000, Andre Przywara wrote:
> virt/kvm was never really a good include directory for anything else
> than locally included headers.
> With the move of iodev.h there is no need anymore to add this
> directory the compiler's include path, so remove it from the arm and
> arm64 kvm Makefile.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 05/12] KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:43     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: Andre Przywara
  Cc: marc.zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:05PM +0000, Andre Przywara wrote:
> The name "kvm_mmio_range" is a bit bold, given that it only covers
> the VGIC's MMIO ranges. To avoid confusion with kvm_io_range, rename
> it to vgic_io_range.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 05/12] KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range
@ 2015-03-14 14:43     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:05PM +0000, Andre Przywara wrote:
> The name "kvm_mmio_range" is a bit bold, given that it only covers
> the VGIC's MMIO ranges. To avoid confusion with kvm_io_range, rename
> it to vgic_io_range.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Acked-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 06/12] KVM: mark kvm->buses as empty once they were destroyed
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:43     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:06PM +0000, Andre Przywara wrote:
> In kvm_destroy_vm() we call kvm_io_bus_destroy() pretty early,
> especially before calling kvm_arch_destroy_vm(). To avoid
> unregistering devices from the already destroyed bus, let's mark
> the bus with NULL to let other users know it has been destroyed
> already.
> This avoids a crash on a VM shutdown with the VGIC using the
> kvm_io_bus later (the unregistering is in there to be able to roll
> back a faulting init).
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  virt/kvm/kvm_main.c |    4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 8c7ab0b..6f164eb 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -604,8 +604,10 @@ static void kvm_destroy_vm(struct kvm *kvm)
>  	list_del(&kvm->vm_list);
>  	spin_unlock(&kvm_lock);
>  	kvm_free_irq_routing(kvm);
> -	for (i = 0; i < KVM_NR_BUSES; i++)
> +	for (i = 0; i < KVM_NR_BUSES; i++) {
>  		kvm_io_bus_destroy(kvm->buses[i]);
> +		kvm->buses[i] = NULL;
> +	}
>  	kvm_coalesced_mmio_free(kvm);
>  #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
>  	mmu_notifier_unregister(&kvm->mmu_notifier, kvm->mm);

this smells a bit like papering over an ordering issue with the
teardown, is there no other way to fix it?

-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 06/12] KVM: mark kvm->buses as empty once they were destroyed
@ 2015-03-14 14:43     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:43 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:06PM +0000, Andre Przywara wrote:
> In kvm_destroy_vm() we call kvm_io_bus_destroy() pretty early,
> especially before calling kvm_arch_destroy_vm(). To avoid
> unregistering devices from the already destroyed bus, let's mark
> the bus with NULL to let other users know it has been destroyed
> already.
> This avoids a crash on a VM shutdown with the VGIC using the
> kvm_io_bus later (the unregistering is in there to be able to roll
> back a faulting init).
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> ---
>  virt/kvm/kvm_main.c |    4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
> index 8c7ab0b..6f164eb 100644
> --- a/virt/kvm/kvm_main.c
> +++ b/virt/kvm/kvm_main.c
> @@ -604,8 +604,10 @@ static void kvm_destroy_vm(struct kvm *kvm)
>  	list_del(&kvm->vm_list);
>  	spin_unlock(&kvm_lock);
>  	kvm_free_irq_routing(kvm);
> -	for (i = 0; i < KVM_NR_BUSES; i++)
> +	for (i = 0; i < KVM_NR_BUSES; i++) {
>  		kvm_io_bus_destroy(kvm->buses[i]);
> +		kvm->buses[i] = NULL;
> +	}
>  	kvm_coalesced_mmio_free(kvm);
>  #if defined(CONFIG_MMU_NOTIFIER) && defined(KVM_ARCH_WANT_MMU_NOTIFIER)
>  	mmu_notifier_unregister(&kvm->mmu_notifier, kvm->mm);

this smells a bit like papering over an ordering issue with the
teardown, is there no other way to fix it?

-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 07/12] KVM: arm/arm64: simplify vgic_find_range() and callers
  2015-03-13 16:10   ` Andre Przywara
@ 2015-03-14 14:44     ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:44 UTC (permalink / raw)
  To: Andre Przywara; +Cc: kvm, marc.zyngier, pbonzini, kvmarm, linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:07PM +0000, Andre Przywara wrote:
> The vgic_find_range() function in vgic.c takes a struct kvm_exit_mmio
> argument, but actually only used the length field in there. Since we
> need to get rid of that structure in that part of the code anyway,
> let's rework the function (and it's callers) to pass the length
> argument to the function directly.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 07/12] KVM: arm/arm64: simplify vgic_find_range() and callers
@ 2015-03-14 14:44     ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-14 14:44 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 13, 2015 at 04:10:07PM +0000, Andre Przywara wrote:
> The vgic_find_range() function in vgic.c takes a struct kvm_exit_mmio
> argument, but actually only used the length field in there. Since we
> need to get rid of that structure in that part of the code anyway,
> let's rework the function (and it's callers) to pass the length
> argument to the function directly.
> 
> Signed-off-by: Andre Przywara <andre.przywara@arm.com>

Reviewed-by: Christoffer Dall <christoffer.dall@linaro.org>

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
  2015-03-14 14:30     ` Christoffer Dall
@ 2015-03-17 18:02       ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-17 18:02 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: kvm, Marc Zyngier, pbonzini, kvmarm, linux-arm-kernel

Hej,

On 14/03/15 14:30, Christoffer Dall wrote:
> On Fri, Mar 13, 2015 at 04:10:09PM +0000, Andre Przywara wrote:
>> Using the framework provided by the recent vgic.c changes we register
>> a kvm_io_bus device when initializing the virtual GICv2.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>>  include/kvm/arm_vgic.h      |    1 +
>>  virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
>>  virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
>>  3 files changed, 26 insertions(+), 4 deletions(-)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 4bfc6a3..74a4ac4 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -245,6 +245,7 @@ struct vgic_dist {
>>  	unsigned long		*irq_pending_on_cpu;
>>  
>>  	struct vgic_vm_ops	vm_ops;
>> +	struct vgic_io_device	dist_iodev;
>>  };
>>  
>>  struct vgic_v2_cpu_if {
>> diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
>> index 0defac6..6f685c9 100644
>> --- a/virt/kvm/arm/vgic-v2-emul.c
>> +++ b/virt/kvm/arm/vgic-v2-emul.c
>> @@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
>>  static int vgic_v2_map_resources(struct kvm *kvm,
>>  				 const struct vgic_params *params)
>>  {
>> +	struct vgic_dist *dist = &kvm->arch.vgic;
>>  	int ret = 0;
>>  
>>  	if (!irqchip_in_kernel(kvm))
>> @@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>>  	if (vgic_ready(kvm))
>>  		goto out;
>>  
>> -	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
>> -	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
>> +	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
>> +	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
>>  		kvm_err("Need to set vgic cpu and dist addresses first\n");
>>  		ret = -ENXIO;
>>  		goto out;
>>  	}
>>  
>> +	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
>> +				 KVM_VGIC_V2_DIST_SIZE,
>> +				 vgic_dist_ranges, -1, &dist->dist_iodev);
>> +
>>  	/*
>>  	 * Initialize the vgic if this hasn't already been done on demand by
>>  	 * accessing the vgic state from userspace.
>> @@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>>  		goto out;
>>  	}
>>  
>> -	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
>> +	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
>>  				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
>>  				    true);
>>  	if (ret) {
>> @@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>>  		goto out;
>>  	}
>>  
>> -	kvm->arch.vgic.ready = true;
>> +	dist->ready = true;
>>  out:
>>  	if (ret)
>>  		kvm_vgic_destroy(kvm);
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index 71389b8..b1dd717 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
>>  	return 0;
>>  }
>>  
>> +static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
>> +{
>> +	struct vgic_dist *dist = &kvm->arch.vgic;
>> +
>> +	if (!dist || !kvm->buses[KVM_MMIO_BUS])
>> +		return;
>> +
>> +	mutex_lock(&kvm->slots_lock);
>> +	if (dist->dist_iodev.dev.ops)
>> +		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
>> +					  &dist->dist_iodev.dev);
> 
> why are we only unregisttering the iodev when we have ops?

Because vgic_unregister_kvm_io_dev() is called by kvm_vgic_destroy(),
which we call on some occasions during vgic_init() when we encounter an
error. There is quite a window of cases where the kvm_io_bus devices
haven't been registered yet, so we shouldn't try to unregister them at
this place. Using the .ops parameter seemed like an elegant way to
detect this case.
Does that make sense? Or have I missed something?

Cheers,
Andre.

> 
>> +	mutex_unlock(&kvm->slots_lock);
>> +}
>> +
>>  static int vgic_nr_shared_irqs(struct vgic_dist *dist)
>>  {
>>  	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
>> @@ -1546,6 +1560,8 @@ void kvm_vgic_destroy(struct kvm *kvm)
>>  	struct kvm_vcpu *vcpu;
>>  	int i;
>>  
>> +	vgic_unregister_kvm_io_dev(kvm);
>> +
>>  	kvm_for_each_vcpu(i, vcpu, kvm)
>>  		kvm_vgic_vcpu_destroy(vcpu);
>>  
>> -- 
>> 1.7.9.5
>>
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
@ 2015-03-17 18:02       ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-17 18:02 UTC (permalink / raw)
  To: linux-arm-kernel

Hej,

On 14/03/15 14:30, Christoffer Dall wrote:
> On Fri, Mar 13, 2015 at 04:10:09PM +0000, Andre Przywara wrote:
>> Using the framework provided by the recent vgic.c changes we register
>> a kvm_io_bus device when initializing the virtual GICv2.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>>  include/kvm/arm_vgic.h      |    1 +
>>  virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
>>  virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
>>  3 files changed, 26 insertions(+), 4 deletions(-)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index 4bfc6a3..74a4ac4 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -245,6 +245,7 @@ struct vgic_dist {
>>  	unsigned long		*irq_pending_on_cpu;
>>  
>>  	struct vgic_vm_ops	vm_ops;
>> +	struct vgic_io_device	dist_iodev;
>>  };
>>  
>>  struct vgic_v2_cpu_if {
>> diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
>> index 0defac6..6f685c9 100644
>> --- a/virt/kvm/arm/vgic-v2-emul.c
>> +++ b/virt/kvm/arm/vgic-v2-emul.c
>> @@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
>>  static int vgic_v2_map_resources(struct kvm *kvm,
>>  				 const struct vgic_params *params)
>>  {
>> +	struct vgic_dist *dist = &kvm->arch.vgic;
>>  	int ret = 0;
>>  
>>  	if (!irqchip_in_kernel(kvm))
>> @@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>>  	if (vgic_ready(kvm))
>>  		goto out;
>>  
>> -	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
>> -	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
>> +	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
>> +	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
>>  		kvm_err("Need to set vgic cpu and dist addresses first\n");
>>  		ret = -ENXIO;
>>  		goto out;
>>  	}
>>  
>> +	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
>> +				 KVM_VGIC_V2_DIST_SIZE,
>> +				 vgic_dist_ranges, -1, &dist->dist_iodev);
>> +
>>  	/*
>>  	 * Initialize the vgic if this hasn't already been done on demand by
>>  	 * accessing the vgic state from userspace.
>> @@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>>  		goto out;
>>  	}
>>  
>> -	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
>> +	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
>>  				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
>>  				    true);
>>  	if (ret) {
>> @@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
>>  		goto out;
>>  	}
>>  
>> -	kvm->arch.vgic.ready = true;
>> +	dist->ready = true;
>>  out:
>>  	if (ret)
>>  		kvm_vgic_destroy(kvm);
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index 71389b8..b1dd717 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
>>  	return 0;
>>  }
>>  
>> +static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
>> +{
>> +	struct vgic_dist *dist = &kvm->arch.vgic;
>> +
>> +	if (!dist || !kvm->buses[KVM_MMIO_BUS])
>> +		return;
>> +
>> +	mutex_lock(&kvm->slots_lock);
>> +	if (dist->dist_iodev.dev.ops)
>> +		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
>> +					  &dist->dist_iodev.dev);
> 
> why are we only unregisttering the iodev when we have ops?

Because vgic_unregister_kvm_io_dev() is called by kvm_vgic_destroy(),
which we call on some occasions during vgic_init() when we encounter an
error. There is quite a window of cases where the kvm_io_bus devices
haven't been registered yet, so we shouldn't try to unregister them at
this place. Using the .ops parameter seemed like an elegant way to
detect this case.
Does that make sense? Or have I missed something?

Cheers,
Andre.

> 
>> +	mutex_unlock(&kvm->slots_lock);
>> +}
>> +
>>  static int vgic_nr_shared_irqs(struct vgic_dist *dist)
>>  {
>>  	return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
>> @@ -1546,6 +1560,8 @@ void kvm_vgic_destroy(struct kvm *kvm)
>>  	struct kvm_vcpu *vcpu;
>>  	int i;
>>  
>> +	vgic_unregister_kvm_io_dev(kvm);
>> +
>>  	kvm_for_each_vcpu(i, vcpu, kvm)
>>  		kvm_vgic_vcpu_destroy(vcpu);
>>  
>> -- 
>> 1.7.9.5
>>
> 

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
  2015-03-17 18:02       ` Andre Przywara
@ 2015-03-17 18:51         ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-17 18:51 UTC (permalink / raw)
  To: Andre Przywara
  Cc: Marc Zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

On Tue, Mar 17, 2015 at 06:02:41PM +0000, Andre Przywara wrote:
> Hej,
> 
> On 14/03/15 14:30, Christoffer Dall wrote:
> > On Fri, Mar 13, 2015 at 04:10:09PM +0000, Andre Przywara wrote:
> >> Using the framework provided by the recent vgic.c changes we register
> >> a kvm_io_bus device when initializing the virtual GICv2.
> >>
> >> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h      |    1 +
> >>  virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
> >>  virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
> >>  3 files changed, 26 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 4bfc6a3..74a4ac4 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -245,6 +245,7 @@ struct vgic_dist {
> >>  	unsigned long		*irq_pending_on_cpu;
> >>  
> >>  	struct vgic_vm_ops	vm_ops;
> >> +	struct vgic_io_device	dist_iodev;
> >>  };
> >>  
> >>  struct vgic_v2_cpu_if {
> >> diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
> >> index 0defac6..6f685c9 100644
> >> --- a/virt/kvm/arm/vgic-v2-emul.c
> >> +++ b/virt/kvm/arm/vgic-v2-emul.c
> >> @@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
> >>  static int vgic_v2_map_resources(struct kvm *kvm,
> >>  				 const struct vgic_params *params)
> >>  {
> >> +	struct vgic_dist *dist = &kvm->arch.vgic;
> >>  	int ret = 0;
> >>  
> >>  	if (!irqchip_in_kernel(kvm))
> >> @@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
> >>  	if (vgic_ready(kvm))
> >>  		goto out;
> >>  
> >> -	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
> >> -	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
> >> +	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
> >> +	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
> >>  		kvm_err("Need to set vgic cpu and dist addresses first\n");
> >>  		ret = -ENXIO;
> >>  		goto out;
> >>  	}
> >>  
> >> +	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
> >> +				 KVM_VGIC_V2_DIST_SIZE,
> >> +				 vgic_dist_ranges, -1, &dist->dist_iodev);
> >> +
> >>  	/*
> >>  	 * Initialize the vgic if this hasn't already been done on demand by
> >>  	 * accessing the vgic state from userspace.
> >> @@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
> >>  		goto out;
> >>  	}
> >>  
> >> -	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
> >> +	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
> >>  				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
> >>  				    true);
> >>  	if (ret) {
> >> @@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
> >>  		goto out;
> >>  	}
> >>  
> >> -	kvm->arch.vgic.ready = true;
> >> +	dist->ready = true;
> >>  out:
> >>  	if (ret)
> >>  		kvm_vgic_destroy(kvm);
> >> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >> index 71389b8..b1dd717 100644
> >> --- a/virt/kvm/arm/vgic.c
> >> +++ b/virt/kvm/arm/vgic.c
> >> @@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
> >>  	return 0;
> >>  }
> >>  
> >> +static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
> >> +{
> >> +	struct vgic_dist *dist = &kvm->arch.vgic;
> >> +
> >> +	if (!dist || !kvm->buses[KVM_MMIO_BUS])
> >> +		return;
> >> +
> >> +	mutex_lock(&kvm->slots_lock);
> >> +	if (dist->dist_iodev.dev.ops)
> >> +		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
> >> +					  &dist->dist_iodev.dev);
> > 
> > why are we only unregisttering the iodev when we have ops?
> 
> Because vgic_unregister_kvm_io_dev() is called by kvm_vgic_destroy(),
> which we call on some occasions during vgic_init() when we encounter an
> error. There is quite a window of cases where the kvm_io_bus devices
> haven't been registered yet, so we shouldn't try to unregister them at
> this place. Using the .ops parameter seemed like an elegant way to
> detect this case.
> Does that make sense? Or have I missed something?
> 
That's ok, but it derserves a comment explaining this.

-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus
@ 2015-03-17 18:51         ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-17 18:51 UTC (permalink / raw)
  To: linux-arm-kernel

On Tue, Mar 17, 2015 at 06:02:41PM +0000, Andre Przywara wrote:
> Hej,
> 
> On 14/03/15 14:30, Christoffer Dall wrote:
> > On Fri, Mar 13, 2015 at 04:10:09PM +0000, Andre Przywara wrote:
> >> Using the framework provided by the recent vgic.c changes we register
> >> a kvm_io_bus device when initializing the virtual GICv2.
> >>
> >> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h      |    1 +
> >>  virt/kvm/arm/vgic-v2-emul.c |   13 +++++++++----
> >>  virt/kvm/arm/vgic.c         |   16 ++++++++++++++++
> >>  3 files changed, 26 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index 4bfc6a3..74a4ac4 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -245,6 +245,7 @@ struct vgic_dist {
> >>  	unsigned long		*irq_pending_on_cpu;
> >>  
> >>  	struct vgic_vm_ops	vm_ops;
> >> +	struct vgic_io_device	dist_iodev;
> >>  };
> >>  
> >>  struct vgic_v2_cpu_if {
> >> diff --git a/virt/kvm/arm/vgic-v2-emul.c b/virt/kvm/arm/vgic-v2-emul.c
> >> index 0defac6..6f685c9 100644
> >> --- a/virt/kvm/arm/vgic-v2-emul.c
> >> +++ b/virt/kvm/arm/vgic-v2-emul.c
> >> @@ -490,6 +490,7 @@ static bool vgic_v2_queue_sgi(struct kvm_vcpu *vcpu, int irq)
> >>  static int vgic_v2_map_resources(struct kvm *kvm,
> >>  				 const struct vgic_params *params)
> >>  {
> >> +	struct vgic_dist *dist = &kvm->arch.vgic;
> >>  	int ret = 0;
> >>  
> >>  	if (!irqchip_in_kernel(kvm))
> >> @@ -500,13 +501,17 @@ static int vgic_v2_map_resources(struct kvm *kvm,
> >>  	if (vgic_ready(kvm))
> >>  		goto out;
> >>  
> >> -	if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
> >> -	    IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
> >> +	if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
> >> +	    IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
> >>  		kvm_err("Need to set vgic cpu and dist addresses first\n");
> >>  		ret = -ENXIO;
> >>  		goto out;
> >>  	}
> >>  
> >> +	vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
> >> +				 KVM_VGIC_V2_DIST_SIZE,
> >> +				 vgic_dist_ranges, -1, &dist->dist_iodev);
> >> +
> >>  	/*
> >>  	 * Initialize the vgic if this hasn't already been done on demand by
> >>  	 * accessing the vgic state from userspace.
> >> @@ -517,7 +522,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
> >>  		goto out;
> >>  	}
> >>  
> >> -	ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
> >> +	ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
> >>  				    params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
> >>  				    true);
> >>  	if (ret) {
> >> @@ -525,7 +530,7 @@ static int vgic_v2_map_resources(struct kvm *kvm,
> >>  		goto out;
> >>  	}
> >>  
> >> -	kvm->arch.vgic.ready = true;
> >> +	dist->ready = true;
> >>  out:
> >>  	if (ret)
> >>  		kvm_vgic_destroy(kvm);
> >> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >> index 71389b8..b1dd717 100644
> >> --- a/virt/kvm/arm/vgic.c
> >> +++ b/virt/kvm/arm/vgic.c
> >> @@ -908,6 +908,20 @@ int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
> >>  	return 0;
> >>  }
> >>  
> >> +static void vgic_unregister_kvm_io_dev(struct kvm *kvm)
> >> +{
> >> +	struct vgic_dist *dist = &kvm->arch.vgic;
> >> +
> >> +	if (!dist || !kvm->buses[KVM_MMIO_BUS])
> >> +		return;
> >> +
> >> +	mutex_lock(&kvm->slots_lock);
> >> +	if (dist->dist_iodev.dev.ops)
> >> +		kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
> >> +					  &dist->dist_iodev.dev);
> > 
> > why are we only unregisttering the iodev when we have ops?
> 
> Because vgic_unregister_kvm_io_dev() is called by kvm_vgic_destroy(),
> which we call on some occasions during vgic_init() when we encounter an
> error. There is quite a window of cases where the kvm_io_bus devices
> haven't been registered yet, so we shouldn't try to unregister them at
> this place. Using the .ops parameter seemed like an elegant way to
> detect this case.
> Does that make sense? Or have I missed something?
> 
That's ok, but it derserves a comment explaining this.

-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  2015-03-14 14:27     ` Christoffer Dall
@ 2015-03-19 15:44       ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-19 15:44 UTC (permalink / raw)
  To: Christoffer Dall
  Cc: Marc Zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

Hej Christoffer,

On 14/03/15 14:27, Christoffer Dall wrote:
> On Fri, Mar 13, 2015 at 04:10:08PM +0000, Andre Przywara wrote:
>> Currently we use a lot of VGIC specific code to do the MMIO
>> dispatching.
>> Use the previous reworks to add kvm_io_bus style MMIO handlers.
>>
>> Those are not yet called by the MMIO abort handler, also the actual
>> VGIC emulator function do not make use of it yet, but will be enabled
>> with the following patches.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>>  include/kvm/arm_vgic.h |    9 ++++
>>  virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic.h    |    7 +++
>>  3 files changed, 127 insertions(+)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index b81630b..4bfc6a3 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -24,6 +24,7 @@
>>  #include <linux/irqreturn.h>
>>  #include <linux/spinlock.h>
>>  #include <linux/types.h>
>> +#include <kvm/iodev.h>
>>
>>  #define VGIC_NR_IRQS_LEGACY 256
>>  #define VGIC_NR_SGIS                16
>> @@ -147,6 +148,14 @@ struct vgic_vm_ops {
>>      int     (*map_resources)(struct kvm *, const struct vgic_params *);
>>  };
>>
>> +struct vgic_io_device {
>> +    gpa_t addr;
>> +    int len;
>> +    const struct vgic_io_range *reg_ranges;
>> +    struct kvm_vcpu *redist_vcpu;
>> +    struct kvm_io_device dev;
>> +};
>> +
>>  struct vgic_dist {
>>      spinlock_t              lock;
>>      bool                    in_kernel;
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index 7aae19b..71389b8 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -32,6 +32,8 @@
>>  #include <asm/kvm_arm.h>
>>  #include <asm/kvm_mmu.h>
>>  #include <trace/events/kvm.h>
>> +#include <asm/kvm.h>
>> +#include <kvm/iodev.h>
>>
>>  /*
>>   * How the whole thing works (courtesy of Christoffer Dall):
>> @@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>  }
>>
>>  /**
>> + * vgic_handle_mmio_access - handle an in-kernel MMIO access
>> + * This is called by the read/write KVM IO device wrappers below.
>> + * @vcpu:   pointer to the vcpu performing the access
>> + * @this:   pointer to the KVM IO device in charge
>> + * @addr:   guest physical address of the access
>> + * @len:    size of the access
>> + * @val:    pointer to the data region
>> + * @is_write:       read or write access
>> + *
>> + * returns true if the MMIO access could be performed
>> + */
>> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
>> +                               struct kvm_io_device *this, gpa_t addr,
>> +                               int len, void *val, bool is_write)
>> +{
>> +    struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> +    struct vgic_io_device *iodev = container_of(this,
>> +                                                struct vgic_io_device, dev);
>> +    struct kvm_run *run = vcpu->run;
>> +    const struct vgic_io_range *range;
>> +    struct kvm_exit_mmio mmio;
>> +    bool updated_state;
>> +    gpa_t offset;
>> +
>> +    offset = addr - iodev->addr;
>> +    range = vgic_find_range(iodev->reg_ranges, len, offset);
>> +    if (unlikely(!range || !range->handle_mmio)) {
>> +            pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
>> +            return -ENXIO;
>> +    }
>> +
>> +    mmio.phys_addr = addr;
>> +    mmio.len = len;
>> +    mmio.is_write = is_write;
>> +    if (is_write)
>> +            memcpy(mmio.data, val, len);
>> +    mmio.private = iodev->redist_vcpu;
>> +
>> +    spin_lock(&dist->lock);
>> +    offset -= range->base;
>> +    if (vgic_validate_access(dist, range, offset)) {
>> +            updated_state = call_range_handler(vcpu, &mmio, offset, range);
>> +            if (!is_write)
>> +                    memcpy(val, mmio.data, len);
>> +    } else {
>> +            if (!is_write)
>> +                    memset(val, 0, len);
>> +            updated_state = false;
>> +    }
>> +    spin_unlock(&dist->lock);
>> +    kvm_prepare_mmio(run, &mmio);
>
> we're not the only user of kvm_exit_mmio I believe, so we could rename

(assuming you mean we _are_ the only user here, which I can acknowledge)

> this to vgic_io as well and you could change the mmio.data array to be a
> void *val pointer, which just gets set to the pointer passed into this
> function (which I think points to the kvm_run structs data array) and
> you can avoid all these memcopies, right?

That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
declaration reads:
/*
 * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
 * which is an anonymous type. Use our own type instead.
 */
How I understand this the structure was introduced to _not_ use the same
memory, but use a copy instead. Do you remember any reason for this? And
in how far is this type anonymous? It's even in an uapi header.

Briefly looking at the code we do quite some memcpy on the way.
I am about to go all the way down into that ARM MMIO handling cave now
to check this (Marc, if I am not showing up again after some hours,
please come and rescue me ;-)

Cheers,
Andre.

-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2548782


^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
@ 2015-03-19 15:44       ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-19 15:44 UTC (permalink / raw)
  To: linux-arm-kernel

Hej Christoffer,

On 14/03/15 14:27, Christoffer Dall wrote:
> On Fri, Mar 13, 2015 at 04:10:08PM +0000, Andre Przywara wrote:
>> Currently we use a lot of VGIC specific code to do the MMIO
>> dispatching.
>> Use the previous reworks to add kvm_io_bus style MMIO handlers.
>>
>> Those are not yet called by the MMIO abort handler, also the actual
>> VGIC emulator function do not make use of it yet, but will be enabled
>> with the following patches.
>>
>> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
>> ---
>>  include/kvm/arm_vgic.h |    9 ++++
>>  virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
>>  virt/kvm/arm/vgic.h    |    7 +++
>>  3 files changed, 127 insertions(+)
>>
>> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
>> index b81630b..4bfc6a3 100644
>> --- a/include/kvm/arm_vgic.h
>> +++ b/include/kvm/arm_vgic.h
>> @@ -24,6 +24,7 @@
>>  #include <linux/irqreturn.h>
>>  #include <linux/spinlock.h>
>>  #include <linux/types.h>
>> +#include <kvm/iodev.h>
>>
>>  #define VGIC_NR_IRQS_LEGACY 256
>>  #define VGIC_NR_SGIS                16
>> @@ -147,6 +148,14 @@ struct vgic_vm_ops {
>>      int     (*map_resources)(struct kvm *, const struct vgic_params *);
>>  };
>>
>> +struct vgic_io_device {
>> +    gpa_t addr;
>> +    int len;
>> +    const struct vgic_io_range *reg_ranges;
>> +    struct kvm_vcpu *redist_vcpu;
>> +    struct kvm_io_device dev;
>> +};
>> +
>>  struct vgic_dist {
>>      spinlock_t              lock;
>>      bool                    in_kernel;
>> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
>> index 7aae19b..71389b8 100644
>> --- a/virt/kvm/arm/vgic.c
>> +++ b/virt/kvm/arm/vgic.c
>> @@ -32,6 +32,8 @@
>>  #include <asm/kvm_arm.h>
>>  #include <asm/kvm_mmu.h>
>>  #include <trace/events/kvm.h>
>> +#include <asm/kvm.h>
>> +#include <kvm/iodev.h>
>>
>>  /*
>>   * How the whole thing works (courtesy of Christoffer Dall):
>> @@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
>>  }
>>
>>  /**
>> + * vgic_handle_mmio_access - handle an in-kernel MMIO access
>> + * This is called by the read/write KVM IO device wrappers below.
>> + * @vcpu:   pointer to the vcpu performing the access
>> + * @this:   pointer to the KVM IO device in charge
>> + * @addr:   guest physical address of the access
>> + * @len:    size of the access
>> + * @val:    pointer to the data region
>> + * @is_write:       read or write access
>> + *
>> + * returns true if the MMIO access could be performed
>> + */
>> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
>> +                               struct kvm_io_device *this, gpa_t addr,
>> +                               int len, void *val, bool is_write)
>> +{
>> +    struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>> +    struct vgic_io_device *iodev = container_of(this,
>> +                                                struct vgic_io_device, dev);
>> +    struct kvm_run *run = vcpu->run;
>> +    const struct vgic_io_range *range;
>> +    struct kvm_exit_mmio mmio;
>> +    bool updated_state;
>> +    gpa_t offset;
>> +
>> +    offset = addr - iodev->addr;
>> +    range = vgic_find_range(iodev->reg_ranges, len, offset);
>> +    if (unlikely(!range || !range->handle_mmio)) {
>> +            pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
>> +            return -ENXIO;
>> +    }
>> +
>> +    mmio.phys_addr = addr;
>> +    mmio.len = len;
>> +    mmio.is_write = is_write;
>> +    if (is_write)
>> +            memcpy(mmio.data, val, len);
>> +    mmio.private = iodev->redist_vcpu;
>> +
>> +    spin_lock(&dist->lock);
>> +    offset -= range->base;
>> +    if (vgic_validate_access(dist, range, offset)) {
>> +            updated_state = call_range_handler(vcpu, &mmio, offset, range);
>> +            if (!is_write)
>> +                    memcpy(val, mmio.data, len);
>> +    } else {
>> +            if (!is_write)
>> +                    memset(val, 0, len);
>> +            updated_state = false;
>> +    }
>> +    spin_unlock(&dist->lock);
>> +    kvm_prepare_mmio(run, &mmio);
>
> we're not the only user of kvm_exit_mmio I believe, so we could rename

(assuming you mean we _are_ the only user here, which I can acknowledge)

> this to vgic_io as well and you could change the mmio.data array to be a
> void *val pointer, which just gets set to the pointer passed into this
> function (which I think points to the kvm_run structs data array) and
> you can avoid all these memcopies, right?

That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
declaration reads:
/*
 * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
 * which is an anonymous type. Use our own type instead.
 */
How I understand this the structure was introduced to _not_ use the same
memory, but use a copy instead. Do you remember any reason for this? And
in how far is this type anonymous? It's even in an uapi header.

Briefly looking at the code we do quite some memcpy on the way.
I am about to go all the way down into that ARM MMIO handling cave now
to check this (Marc, if I am not showing up again after some hours,
please come and rescue me ;-)

Cheers,
Andre.

-- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium.  Thank you.

ARM Limited, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2557590
ARM Holdings plc, Registered office 110 Fulbourn Road, Cambridge CB1 9NJ, Registered in England & Wales, Company No:  2548782

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  2015-03-19 15:44       ` Andre Przywara
@ 2015-03-20 12:40         ` Andre Przywara
  -1 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-20 12:40 UTC (permalink / raw)
  To: Christoffer Dall; +Cc: kvm, Marc Zyngier, pbonzini, kvmarm, linux-arm-kernel

On 03/19/2015 03:44 PM, Andre Przywara wrote:
> Hej Christoffer,
> 

[ ... ]

>>> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
>>> +				   struct kvm_io_device *this, gpa_t addr,
>>> +				   int len, void *val, bool is_write)
>>> +{
>>> +	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> +	struct vgic_io_device *iodev = container_of(this,
>>> +						    struct vgic_io_device, dev);
>>> +	struct kvm_run *run = vcpu->run;
>>> +	const struct vgic_io_range *range;
>>> +	struct kvm_exit_mmio mmio;
>>> +	bool updated_state;
>>> +	gpa_t offset;
>>> +
>>> +	offset = addr - iodev->addr;
>>> +	range = vgic_find_range(iodev->reg_ranges, len, offset);
>>> +	if (unlikely(!range || !range->handle_mmio)) {
>>> +		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
>>> +		return -ENXIO;
>>> +	}
>>> +
>>> +	mmio.phys_addr = addr;
>>> +	mmio.len = len;
>>> +	mmio.is_write = is_write;
>>> +	if (is_write)
>>> +		memcpy(mmio.data, val, len);
>>> +	mmio.private = iodev->redist_vcpu;
>>> +
>>> +	spin_lock(&dist->lock);
>>> +	offset -= range->base;
>>> +	if (vgic_validate_access(dist, range, offset)) {
>>> +		updated_state = call_range_handler(vcpu, &mmio, offset, range);
>>> +		if (!is_write)
>>> +			memcpy(val, mmio.data, len);
>>> +	} else {
>>> +		if (!is_write)
>>> +			memset(val, 0, len);
>>> +		updated_state = false;
>>> +	}
>>> +	spin_unlock(&dist->lock);
>>> +	kvm_prepare_mmio(run, &mmio);
>>
>> we're not the only user of kvm_exit_mmio I believe, so we could rename
> 
> (assuming you mean we _are_ the only user here, which I can acknowledge)
> 
>> this to vgic_io as well and you could change the mmio.data array to be a
>> void *val pointer, which just gets set to the pointer passed into this
>> function (which I think points to the kvm_run structs data array) and
>> you can avoid all these memcopies, right?
> 
> That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
> declaration reads:
> /*
>  * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
>  * which is an anonymous type. Use our own type instead.
>  */
> How I understand this the structure was introduced to _not_ use the same
> memory, but use a copy instead. Do you remember any reason for this? And
> in how far is this type anonymous? It's even in an uapi header.
> 
> Briefly looking at the code we do quite some memcpy on the way.
> I am about to go all the way down into that ARM MMIO handling cave now
> to check this (Marc, if I am not showing up again after some hours,
> please come and rescue me ;-)

So, I feel that there is quite some unneeded copying and masking on the
way, but a real fix would be quite invasive and needs quite some testing
and review. I don't feel like rushing this into a v2 of this series.
I quickly did what you proposed (replacing memcpy by pointer
assignment), and that seems to work, but I don't have many chances of
testing this this weekend, since I am on the road. Also I have to dig
out my cross-endian test scripts first. So not sure if you want to take
the risk with this series.
I changed the other minor points you mentioned in the review though, so
do you want to have a "v1.5" or how do we proceed from here?

Cheers,
Andre.

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
@ 2015-03-20 12:40         ` Andre Przywara
  0 siblings, 0 replies; 60+ messages in thread
From: Andre Przywara @ 2015-03-20 12:40 UTC (permalink / raw)
  To: linux-arm-kernel

On 03/19/2015 03:44 PM, Andre Przywara wrote:
> Hej Christoffer,
> 

[ ... ]

>>> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
>>> +				   struct kvm_io_device *this, gpa_t addr,
>>> +				   int len, void *val, bool is_write)
>>> +{
>>> +	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
>>> +	struct vgic_io_device *iodev = container_of(this,
>>> +						    struct vgic_io_device, dev);
>>> +	struct kvm_run *run = vcpu->run;
>>> +	const struct vgic_io_range *range;
>>> +	struct kvm_exit_mmio mmio;
>>> +	bool updated_state;
>>> +	gpa_t offset;
>>> +
>>> +	offset = addr - iodev->addr;
>>> +	range = vgic_find_range(iodev->reg_ranges, len, offset);
>>> +	if (unlikely(!range || !range->handle_mmio)) {
>>> +		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
>>> +		return -ENXIO;
>>> +	}
>>> +
>>> +	mmio.phys_addr = addr;
>>> +	mmio.len = len;
>>> +	mmio.is_write = is_write;
>>> +	if (is_write)
>>> +		memcpy(mmio.data, val, len);
>>> +	mmio.private = iodev->redist_vcpu;
>>> +
>>> +	spin_lock(&dist->lock);
>>> +	offset -= range->base;
>>> +	if (vgic_validate_access(dist, range, offset)) {
>>> +		updated_state = call_range_handler(vcpu, &mmio, offset, range);
>>> +		if (!is_write)
>>> +			memcpy(val, mmio.data, len);
>>> +	} else {
>>> +		if (!is_write)
>>> +			memset(val, 0, len);
>>> +		updated_state = false;
>>> +	}
>>> +	spin_unlock(&dist->lock);
>>> +	kvm_prepare_mmio(run, &mmio);
>>
>> we're not the only user of kvm_exit_mmio I believe, so we could rename
> 
> (assuming you mean we _are_ the only user here, which I can acknowledge)
> 
>> this to vgic_io as well and you could change the mmio.data array to be a
>> void *val pointer, which just gets set to the pointer passed into this
>> function (which I think points to the kvm_run structs data array) and
>> you can avoid all these memcopies, right?
> 
> That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
> declaration reads:
> /*
>  * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
>  * which is an anonymous type. Use our own type instead.
>  */
> How I understand this the structure was introduced to _not_ use the same
> memory, but use a copy instead. Do you remember any reason for this? And
> in how far is this type anonymous? It's even in an uapi header.
> 
> Briefly looking at the code we do quite some memcpy on the way.
> I am about to go all the way down into that ARM MMIO handling cave now
> to check this (Marc, if I am not showing up again after some hours,
> please come and rescue me ;-)

So, I feel that there is quite some unneeded copying and masking on the
way, but a real fix would be quite invasive and needs quite some testing
and review. I don't feel like rushing this into a v2 of this series.
I quickly did what you proposed (replacing memcpy by pointer
assignment), and that seems to work, but I don't have many chances of
testing this this weekend, since I am on the road. Also I have to dig
out my cross-endian test scripts first. So not sure if you want to take
the risk with this series.
I changed the other minor points you mentioned in the review though, so
do you want to have a "v1.5" or how do we proceed from here?

Cheers,
Andre.

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  2015-03-19 15:44       ` Andre Przywara
@ 2015-03-20 14:24         ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-20 14:24 UTC (permalink / raw)
  To: Andre Przywara
  Cc: Marc Zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

On Thu, Mar 19, 2015 at 03:44:51PM +0000, Andre Przywara wrote:
> Hej Christoffer,
> 
> On 14/03/15 14:27, Christoffer Dall wrote:
> > On Fri, Mar 13, 2015 at 04:10:08PM +0000, Andre Przywara wrote:
> >> Currently we use a lot of VGIC specific code to do the MMIO
> >> dispatching.
> >> Use the previous reworks to add kvm_io_bus style MMIO handlers.
> >>
> >> Those are not yet called by the MMIO abort handler, also the actual
> >> VGIC emulator function do not make use of it yet, but will be enabled
> >> with the following patches.
> >>
> >> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h |    9 ++++
> >>  virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
> >>  virt/kvm/arm/vgic.h    |    7 +++
> >>  3 files changed, 127 insertions(+)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index b81630b..4bfc6a3 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -24,6 +24,7 @@
> >>  #include <linux/irqreturn.h>
> >>  #include <linux/spinlock.h>
> >>  #include <linux/types.h>
> >> +#include <kvm/iodev.h>
> >>
> >>  #define VGIC_NR_IRQS_LEGACY 256
> >>  #define VGIC_NR_SGIS                16
> >> @@ -147,6 +148,14 @@ struct vgic_vm_ops {
> >>      int     (*map_resources)(struct kvm *, const struct vgic_params *);
> >>  };
> >>
> >> +struct vgic_io_device {
> >> +    gpa_t addr;
> >> +    int len;
> >> +    const struct vgic_io_range *reg_ranges;
> >> +    struct kvm_vcpu *redist_vcpu;
> >> +    struct kvm_io_device dev;
> >> +};
> >> +
> >>  struct vgic_dist {
> >>      spinlock_t              lock;
> >>      bool                    in_kernel;
> >> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >> index 7aae19b..71389b8 100644
> >> --- a/virt/kvm/arm/vgic.c
> >> +++ b/virt/kvm/arm/vgic.c
> >> @@ -32,6 +32,8 @@
> >>  #include <asm/kvm_arm.h>
> >>  #include <asm/kvm_mmu.h>
> >>  #include <trace/events/kvm.h>
> >> +#include <asm/kvm.h>
> >> +#include <kvm/iodev.h>
> >>
> >>  /*
> >>   * How the whole thing works (courtesy of Christoffer Dall):
> >> @@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >>  }
> >>
> >>  /**
> >> + * vgic_handle_mmio_access - handle an in-kernel MMIO access
> >> + * This is called by the read/write KVM IO device wrappers below.
> >> + * @vcpu:   pointer to the vcpu performing the access
> >> + * @this:   pointer to the KVM IO device in charge
> >> + * @addr:   guest physical address of the access
> >> + * @len:    size of the access
> >> + * @val:    pointer to the data region
> >> + * @is_write:       read or write access
> >> + *
> >> + * returns true if the MMIO access could be performed
> >> + */
> >> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
> >> +                               struct kvm_io_device *this, gpa_t addr,
> >> +                               int len, void *val, bool is_write)
> >> +{
> >> +    struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >> +    struct vgic_io_device *iodev = container_of(this,
> >> +                                                struct vgic_io_device, dev);
> >> +    struct kvm_run *run = vcpu->run;
> >> +    const struct vgic_io_range *range;
> >> +    struct kvm_exit_mmio mmio;
> >> +    bool updated_state;
> >> +    gpa_t offset;
> >> +
> >> +    offset = addr - iodev->addr;
> >> +    range = vgic_find_range(iodev->reg_ranges, len, offset);
> >> +    if (unlikely(!range || !range->handle_mmio)) {
> >> +            pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
> >> +            return -ENXIO;
> >> +    }
> >> +
> >> +    mmio.phys_addr = addr;
> >> +    mmio.len = len;
> >> +    mmio.is_write = is_write;
> >> +    if (is_write)
> >> +            memcpy(mmio.data, val, len);
> >> +    mmio.private = iodev->redist_vcpu;
> >> +
> >> +    spin_lock(&dist->lock);
> >> +    offset -= range->base;
> >> +    if (vgic_validate_access(dist, range, offset)) {
> >> +            updated_state = call_range_handler(vcpu, &mmio, offset, range);
> >> +            if (!is_write)
> >> +                    memcpy(val, mmio.data, len);
> >> +    } else {
> >> +            if (!is_write)
> >> +                    memset(val, 0, len);
> >> +            updated_state = false;
> >> +    }
> >> +    spin_unlock(&dist->lock);
> >> +    kvm_prepare_mmio(run, &mmio);
> >
> > we're not the only user of kvm_exit_mmio I believe, so we could rename
> 
> (assuming you mean we _are_ the only user here, which I can acknowledge)
> 

yes, I think wanted to write now, not not.

> > this to vgic_io as well and you could change the mmio.data array to be a
> > void *val pointer, which just gets set to the pointer passed into this
> > function (which I think points to the kvm_run structs data array) and
> > you can avoid all these memcopies, right?
> 
> That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
> declaration reads:
> /*
>  * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
>  * which is an anonymous type. Use our own type instead.
>  */
> How I understand this the structure was introduced to _not_ use the same
> memory, but use a copy instead. Do you remember any reason for this? And
> in how far is this type anonymous? It's even in an uapi header.

the kvm exit api is designed without regard for how anything in-kernel
works.  Since the work we were doing was leveraging some of the handling
that pointed to the anonymous struct from user-space, we just copied its
definition.

> 
> Briefly looking at the code we do quite some memcpy on the way.
> I am about to go all the way down into that ARM MMIO handling cave now
> to check this (Marc, if I am not showing up again after some hours,
> please come and rescue me ;-)
> 

For the purposes of this series, the struct is just private vgic
parameter passing now.  You just use the struct kvm_exit_mmio pointer
inside the vgic code to call the range handlers.  You don't have to do
that, you can just define your own struct, call it vgic_mmio_params or
whatever, and instead of it having a data array, it now has a data
pointer.

What I think you'll find is that our need for struct kvm_exit_mmio then
goes away, because it is superseeded by what the kvm dev io bus thingy
now uses, and with some clever pattern matching the code should be
fairly trivial, with the worst part being changing the pointer type in
some places.

If it blows up and I missed something, we can leave this for a future
optimization, but it doesn't sound all that scary to me.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
@ 2015-03-20 14:24         ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-20 14:24 UTC (permalink / raw)
  To: linux-arm-kernel

On Thu, Mar 19, 2015 at 03:44:51PM +0000, Andre Przywara wrote:
> Hej Christoffer,
> 
> On 14/03/15 14:27, Christoffer Dall wrote:
> > On Fri, Mar 13, 2015 at 04:10:08PM +0000, Andre Przywara wrote:
> >> Currently we use a lot of VGIC specific code to do the MMIO
> >> dispatching.
> >> Use the previous reworks to add kvm_io_bus style MMIO handlers.
> >>
> >> Those are not yet called by the MMIO abort handler, also the actual
> >> VGIC emulator function do not make use of it yet, but will be enabled
> >> with the following patches.
> >>
> >> Signed-off-by: Andre Przywara <andre.przywara@arm.com>
> >> ---
> >>  include/kvm/arm_vgic.h |    9 ++++
> >>  virt/kvm/arm/vgic.c    |  111 ++++++++++++++++++++++++++++++++++++++++++++++++
> >>  virt/kvm/arm/vgic.h    |    7 +++
> >>  3 files changed, 127 insertions(+)
> >>
> >> diff --git a/include/kvm/arm_vgic.h b/include/kvm/arm_vgic.h
> >> index b81630b..4bfc6a3 100644
> >> --- a/include/kvm/arm_vgic.h
> >> +++ b/include/kvm/arm_vgic.h
> >> @@ -24,6 +24,7 @@
> >>  #include <linux/irqreturn.h>
> >>  #include <linux/spinlock.h>
> >>  #include <linux/types.h>
> >> +#include <kvm/iodev.h>
> >>
> >>  #define VGIC_NR_IRQS_LEGACY 256
> >>  #define VGIC_NR_SGIS                16
> >> @@ -147,6 +148,14 @@ struct vgic_vm_ops {
> >>      int     (*map_resources)(struct kvm *, const struct vgic_params *);
> >>  };
> >>
> >> +struct vgic_io_device {
> >> +    gpa_t addr;
> >> +    int len;
> >> +    const struct vgic_io_range *reg_ranges;
> >> +    struct kvm_vcpu *redist_vcpu;
> >> +    struct kvm_io_device dev;
> >> +};
> >> +
> >>  struct vgic_dist {
> >>      spinlock_t              lock;
> >>      bool                    in_kernel;
> >> diff --git a/virt/kvm/arm/vgic.c b/virt/kvm/arm/vgic.c
> >> index 7aae19b..71389b8 100644
> >> --- a/virt/kvm/arm/vgic.c
> >> +++ b/virt/kvm/arm/vgic.c
> >> @@ -32,6 +32,8 @@
> >>  #include <asm/kvm_arm.h>
> >>  #include <asm/kvm_mmu.h>
> >>  #include <trace/events/kvm.h>
> >> +#include <asm/kvm.h>
> >> +#include <kvm/iodev.h>
> >>
> >>  /*
> >>   * How the whole thing works (courtesy of Christoffer Dall):
> >> @@ -774,6 +776,66 @@ bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
> >>  }
> >>
> >>  /**
> >> + * vgic_handle_mmio_access - handle an in-kernel MMIO access
> >> + * This is called by the read/write KVM IO device wrappers below.
> >> + * @vcpu:   pointer to the vcpu performing the access
> >> + * @this:   pointer to the KVM IO device in charge
> >> + * @addr:   guest physical address of the access
> >> + * @len:    size of the access
> >> + * @val:    pointer to the data region
> >> + * @is_write:       read or write access
> >> + *
> >> + * returns true if the MMIO access could be performed
> >> + */
> >> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
> >> +                               struct kvm_io_device *this, gpa_t addr,
> >> +                               int len, void *val, bool is_write)
> >> +{
> >> +    struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >> +    struct vgic_io_device *iodev = container_of(this,
> >> +                                                struct vgic_io_device, dev);
> >> +    struct kvm_run *run = vcpu->run;
> >> +    const struct vgic_io_range *range;
> >> +    struct kvm_exit_mmio mmio;
> >> +    bool updated_state;
> >> +    gpa_t offset;
> >> +
> >> +    offset = addr - iodev->addr;
> >> +    range = vgic_find_range(iodev->reg_ranges, len, offset);
> >> +    if (unlikely(!range || !range->handle_mmio)) {
> >> +            pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
> >> +            return -ENXIO;
> >> +    }
> >> +
> >> +    mmio.phys_addr = addr;
> >> +    mmio.len = len;
> >> +    mmio.is_write = is_write;
> >> +    if (is_write)
> >> +            memcpy(mmio.data, val, len);
> >> +    mmio.private = iodev->redist_vcpu;
> >> +
> >> +    spin_lock(&dist->lock);
> >> +    offset -= range->base;
> >> +    if (vgic_validate_access(dist, range, offset)) {
> >> +            updated_state = call_range_handler(vcpu, &mmio, offset, range);
> >> +            if (!is_write)
> >> +                    memcpy(val, mmio.data, len);
> >> +    } else {
> >> +            if (!is_write)
> >> +                    memset(val, 0, len);
> >> +            updated_state = false;
> >> +    }
> >> +    spin_unlock(&dist->lock);
> >> +    kvm_prepare_mmio(run, &mmio);
> >
> > we're not the only user of kvm_exit_mmio I believe, so we could rename
> 
> (assuming you mean we _are_ the only user here, which I can acknowledge)
> 

yes, I think wanted to write now, not not.

> > this to vgic_io as well and you could change the mmio.data array to be a
> > void *val pointer, which just gets set to the pointer passed into this
> > function (which I think points to the kvm_run structs data array) and
> > you can avoid all these memcopies, right?
> 
> That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
> declaration reads:
> /*
>  * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
>  * which is an anonymous type. Use our own type instead.
>  */
> How I understand this the structure was introduced to _not_ use the same
> memory, but use a copy instead. Do you remember any reason for this? And
> in how far is this type anonymous? It's even in an uapi header.

the kvm exit api is designed without regard for how anything in-kernel
works.  Since the work we were doing was leveraging some of the handling
that pointed to the anonymous struct from user-space, we just copied its
definition.

> 
> Briefly looking at the code we do quite some memcpy on the way.
> I am about to go all the way down into that ARM MMIO handling cave now
> to check this (Marc, if I am not showing up again after some hours,
> please come and rescue me ;-)
> 

For the purposes of this series, the struct is just private vgic
parameter passing now.  You just use the struct kvm_exit_mmio pointer
inside the vgic code to call the range handlers.  You don't have to do
that, you can just define your own struct, call it vgic_mmio_params or
whatever, and instead of it having a data array, it now has a data
pointer.

What I think you'll find is that our need for struct kvm_exit_mmio then
goes away, because it is superseeded by what the kvm dev io bus thingy
now uses, and with some clever pattern matching the code should be
fairly trivial, with the worst part being changing the pointer type in
some places.

If it blows up and I missed something, we can leave this for a future
optimization, but it doesn't sound all that scary to me.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* Re: [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
  2015-03-20 12:40         ` Andre Przywara
@ 2015-03-20 14:25           ` Christoffer Dall
  -1 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-20 14:25 UTC (permalink / raw)
  To: Andre Przywara
  Cc: Marc Zyngier, pbonzini, n.nikolaev, eric.auger, kvmarm, kvm,
	linux-arm-kernel

On Fri, Mar 20, 2015 at 12:40:02PM +0000, Andre Przywara wrote:
> On 03/19/2015 03:44 PM, Andre Przywara wrote:
> > Hej Christoffer,
> > 
> 
> [ ... ]
> 
> >>> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
> >>> +				   struct kvm_io_device *this, gpa_t addr,
> >>> +				   int len, void *val, bool is_write)
> >>> +{
> >>> +	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >>> +	struct vgic_io_device *iodev = container_of(this,
> >>> +						    struct vgic_io_device, dev);
> >>> +	struct kvm_run *run = vcpu->run;
> >>> +	const struct vgic_io_range *range;
> >>> +	struct kvm_exit_mmio mmio;
> >>> +	bool updated_state;
> >>> +	gpa_t offset;
> >>> +
> >>> +	offset = addr - iodev->addr;
> >>> +	range = vgic_find_range(iodev->reg_ranges, len, offset);
> >>> +	if (unlikely(!range || !range->handle_mmio)) {
> >>> +		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
> >>> +		return -ENXIO;
> >>> +	}
> >>> +
> >>> +	mmio.phys_addr = addr;
> >>> +	mmio.len = len;
> >>> +	mmio.is_write = is_write;
> >>> +	if (is_write)
> >>> +		memcpy(mmio.data, val, len);
> >>> +	mmio.private = iodev->redist_vcpu;
> >>> +
> >>> +	spin_lock(&dist->lock);
> >>> +	offset -= range->base;
> >>> +	if (vgic_validate_access(dist, range, offset)) {
> >>> +		updated_state = call_range_handler(vcpu, &mmio, offset, range);
> >>> +		if (!is_write)
> >>> +			memcpy(val, mmio.data, len);
> >>> +	} else {
> >>> +		if (!is_write)
> >>> +			memset(val, 0, len);
> >>> +		updated_state = false;
> >>> +	}
> >>> +	spin_unlock(&dist->lock);
> >>> +	kvm_prepare_mmio(run, &mmio);
> >>
> >> we're not the only user of kvm_exit_mmio I believe, so we could rename
> > 
> > (assuming you mean we _are_ the only user here, which I can acknowledge)
> > 
> >> this to vgic_io as well and you could change the mmio.data array to be a
> >> void *val pointer, which just gets set to the pointer passed into this
> >> function (which I think points to the kvm_run structs data array) and
> >> you can avoid all these memcopies, right?
> > 
> > That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
> > declaration reads:
> > /*
> >  * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
> >  * which is an anonymous type. Use our own type instead.
> >  */
> > How I understand this the structure was introduced to _not_ use the same
> > memory, but use a copy instead. Do you remember any reason for this? And
> > in how far is this type anonymous? It's even in an uapi header.
> > 
> > Briefly looking at the code we do quite some memcpy on the way.
> > I am about to go all the way down into that ARM MMIO handling cave now
> > to check this (Marc, if I am not showing up again after some hours,
> > please come and rescue me ;-)
> 
> So, I feel that there is quite some unneeded copying and masking on the
> way, but a real fix would be quite invasive and needs quite some testing
> and review. I don't feel like rushing this into a v2 of this series.
> I quickly did what you proposed (replacing memcpy by pointer
> assignment), and that seems to work, but I don't have many chances of
> testing this this weekend, since I am on the road. Also I have to dig
> out my cross-endian test scripts first. So not sure if you want to take
> the risk with this series.
> I changed the other minor points you mentioned in the review though, so
> do you want to have a "v1.5" or how do we proceed from here?
> 
Why is it so invasive?  See my previous mail.  If you can convince me
that we're touching something truly nasty (code snippet?) then ok, we
can take a version without the cleanup and cleanup later.

If you're out of time, send me whatever you have, and we'll see if I fix
it up further or just leave it for the future.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
* [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC
@ 2015-03-20 14:25           ` Christoffer Dall
  0 siblings, 0 replies; 60+ messages in thread
From: Christoffer Dall @ 2015-03-20 14:25 UTC (permalink / raw)
  To: linux-arm-kernel

On Fri, Mar 20, 2015 at 12:40:02PM +0000, Andre Przywara wrote:
> On 03/19/2015 03:44 PM, Andre Przywara wrote:
> > Hej Christoffer,
> > 
> 
> [ ... ]
> 
> >>> +static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
> >>> +				   struct kvm_io_device *this, gpa_t addr,
> >>> +				   int len, void *val, bool is_write)
> >>> +{
> >>> +	struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
> >>> +	struct vgic_io_device *iodev = container_of(this,
> >>> +						    struct vgic_io_device, dev);
> >>> +	struct kvm_run *run = vcpu->run;
> >>> +	const struct vgic_io_range *range;
> >>> +	struct kvm_exit_mmio mmio;
> >>> +	bool updated_state;
> >>> +	gpa_t offset;
> >>> +
> >>> +	offset = addr - iodev->addr;
> >>> +	range = vgic_find_range(iodev->reg_ranges, len, offset);
> >>> +	if (unlikely(!range || !range->handle_mmio)) {
> >>> +		pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
> >>> +		return -ENXIO;
> >>> +	}
> >>> +
> >>> +	mmio.phys_addr = addr;
> >>> +	mmio.len = len;
> >>> +	mmio.is_write = is_write;
> >>> +	if (is_write)
> >>> +		memcpy(mmio.data, val, len);
> >>> +	mmio.private = iodev->redist_vcpu;
> >>> +
> >>> +	spin_lock(&dist->lock);
> >>> +	offset -= range->base;
> >>> +	if (vgic_validate_access(dist, range, offset)) {
> >>> +		updated_state = call_range_handler(vcpu, &mmio, offset, range);
> >>> +		if (!is_write)
> >>> +			memcpy(val, mmio.data, len);
> >>> +	} else {
> >>> +		if (!is_write)
> >>> +			memset(val, 0, len);
> >>> +		updated_state = false;
> >>> +	}
> >>> +	spin_unlock(&dist->lock);
> >>> +	kvm_prepare_mmio(run, &mmio);
> >>
> >> we're not the only user of kvm_exit_mmio I believe, so we could rename
> > 
> > (assuming you mean we _are_ the only user here, which I can acknowledge)
> > 
> >> this to vgic_io as well and you could change the mmio.data array to be a
> >> void *val pointer, which just gets set to the pointer passed into this
> >> function (which I think points to the kvm_run structs data array) and
> >> you can avoid all these memcopies, right?
> > 
> > That sounds indeed tempting, but the comment on the struct kvm_exit_mmio
> > declaration reads:
> > /*
> >  * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
> >  * which is an anonymous type. Use our own type instead.
> >  */
> > How I understand this the structure was introduced to _not_ use the same
> > memory, but use a copy instead. Do you remember any reason for this? And
> > in how far is this type anonymous? It's even in an uapi header.
> > 
> > Briefly looking at the code we do quite some memcpy on the way.
> > I am about to go all the way down into that ARM MMIO handling cave now
> > to check this (Marc, if I am not showing up again after some hours,
> > please come and rescue me ;-)
> 
> So, I feel that there is quite some unneeded copying and masking on the
> way, but a real fix would be quite invasive and needs quite some testing
> and review. I don't feel like rushing this into a v2 of this series.
> I quickly did what you proposed (replacing memcpy by pointer
> assignment), and that seems to work, but I don't have many chances of
> testing this this weekend, since I am on the road. Also I have to dig
> out my cross-endian test scripts first. So not sure if you want to take
> the risk with this series.
> I changed the other minor points you mentioned in the review though, so
> do you want to have a "v1.5" or how do we proceed from here?
> 
Why is it so invasive?  See my previous mail.  If you can convince me
that we're touching something truly nasty (code snippet?) then ok, we
can take a version without the cleanup and cleanup later.

If you're out of time, send me whatever you have, and we'll see if I fix
it up further or just leave it for the future.

Thanks,
-Christoffer

^ permalink raw reply	[flat|nested] 60+ messages in thread
end of thread, other threads:[~2015-03-20 14:25 UTC | newest]

Thread overview: 60+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-13 16:10 [PATCH 00/12] KVM: arm/arm64: move VGIC MMIO to kvm_io_bus Andre Przywara
2015-03-13 16:10 ` Andre Przywara
2015-03-13 16:10 ` [PATCH 01/12] KVM: Redesign kvm_io_bus_ API to pass VCPU structure to the callbacks Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:43   ` Christoffer Dall
2015-03-14 14:43     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 02/12] KVM: move iodev.h from virt/kvm/ to include/kvm Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:43   ` Christoffer Dall
2015-03-14 14:43     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 03/12] KVM: arm/arm64: remove now unneeded include directory from Makefile Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:43   ` Christoffer Dall
2015-03-14 14:43     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 04/12] KVM: x86: " Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 13:57   ` Christoffer Dall
2015-03-14 13:57     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 05/12] KVM: arm/arm64: rename struct kvm_mmio_range to vgic_io_range Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:43   ` Christoffer Dall
2015-03-14 14:43     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 06/12] KVM: mark kvm->buses as empty once they were destroyed Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:43   ` Christoffer Dall
2015-03-14 14:43     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 07/12] KVM: arm/arm64: simplify vgic_find_range() and callers Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:44   ` Christoffer Dall
2015-03-14 14:44     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 08/12] KVM: arm/arm64: implement kvm_io_bus MMIO handling for the VGIC Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:27   ` Christoffer Dall
2015-03-14 14:27     ` Christoffer Dall
2015-03-19 15:44     ` Andre Przywara
2015-03-19 15:44       ` Andre Przywara
2015-03-20 12:40       ` Andre Przywara
2015-03-20 12:40         ` Andre Przywara
2015-03-20 14:25         ` Christoffer Dall
2015-03-20 14:25           ` Christoffer Dall
2015-03-20 14:24       ` Christoffer Dall
2015-03-20 14:24         ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 09/12] KVM: arm/arm64: prepare GICv2 emulation to be handled by kvm_io_bus Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:30   ` Christoffer Dall
2015-03-14 14:30     ` Christoffer Dall
2015-03-17 18:02     ` Andre Przywara
2015-03-17 18:02       ` Andre Przywara
2015-03-17 18:51       ` Christoffer Dall
2015-03-17 18:51         ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 10/12] KVM: arm/arm64: prepare GICv3 emulation to use kvm_io_bus MMIO handling Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:39   ` Christoffer Dall
2015-03-14 14:39     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 11/12] KVM: ARM: on IO mem abort - route the call to KVM MMIO bus Andre Przywara
2015-03-13 16:10   ` Andre Przywara
2015-03-14 14:43   ` Christoffer Dall
2015-03-14 14:43     ` Christoffer Dall
2015-03-13 16:10 ` [PATCH 12/12] KVM: arm/arm64: remove now obsolete VGIC specific MMIO handling code Andre Przywara
2015-03-13 16:10   ` Andre Przywara

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.