[Qemu-devel] the arm cache coherency cluster

[Qemu-devel] the arm cache coherency cluster

[Andrew Jones]

In reply to this message I'll send two series' one for KVM and
one for QEMU. The two series' are their respective component
complements, and attempt to implement cache coherency for arm
guests using emulated devices, where the emulator (qemu) uses
cached memory for the device memory, but the guest uses
uncached - as device memory is generally used. Right now I've
just focused on VGA vram.

This approach starts as the "add a new memslot flag" approach,
and then turns into the "make qemu do some cache maintenance"
approach with the final patch of each series (6/6). It stops
short of the "add syscalls..." approach. Below is a summary of
all the approaches discussed so far, to my knowledge.

"MAIR manipulating"
Posted[1] by Ard. Works. No performance degradation. Potential
issues with device assignment and the guest getting confused.

"add a new memslot flag"
This posting (not counting patches 6/6). Works. Huge performance
degradation.

"make qemu do some cache maintenance"
This posting (patches 6/6). We can only do so much in qemu
without syscalls. This series does what it can. Almost works,
probably could work, after playing 'find the missing flush'.
This approach still requires the new memslot flag, as userspace
can't invalidate the cache, only clean, or clean+invalidate.
No noticeable performance degradation.

"add syscalls to make qemu do all cache maintenance"
Variant 1: implement as kvm ioctls - to avoid trying to get
           syscalls into the general kernel
Variant 2: add real syscalls, or maybe just ARM private SWIs
           like __ARM_NR_cacheflush
This approach should work, and if we add an invalidate syscall,
then we shouldn't need any kvm changes at all, i.e. no need for
the memslot flag. I haven't experimented with this yet, but I'm
starting to like the idea of variant 2, with a private SWI, so
will try to pull something together soon for that.

"describe the problematic memory as cached to the guest"
Not an ideal solution for virt. Could maybe be workable as a
quirk for a specific device though.

re: $SUBJECT; Here 'cluster' is defined by the urban dictionary.

[1] http://thread.gmane.org/gmane.comp.emulators.kvm.arm.devel/34/

the arm cache coherency cluster

[Andrew Jones]

In reply to this message I'll send two series' one for KVM and
one for QEMU. The two series' are their respective component
complements, and attempt to implement cache coherency for arm
guests using emulated devices, where the emulator (qemu) uses
cached memory for the device memory, but the guest uses
uncached - as device memory is generally used. Right now I've
just focused on VGA vram.

This approach starts as the "add a new memslot flag" approach,
and then turns into the "make qemu do some cache maintenance"
approach with the final patch of each series (6/6). It stops
short of the "add syscalls..." approach. Below is a summary of
all the approaches discussed so far, to my knowledge.

"MAIR manipulating"
Posted[1] by Ard. Works. No performance degradation. Potential
issues with device assignment and the guest getting confused.

"add a new memslot flag"
This posting (not counting patches 6/6). Works. Huge performance
degradation.

"make qemu do some cache maintenance"
This posting (patches 6/6). We can only do so much in qemu
without syscalls. This series does what it can. Almost works,
probably could work, after playing 'find the missing flush'.
This approach still requires the new memslot flag, as userspace
can't invalidate the cache, only clean, or clean+invalidate.
No noticeable performance degradation.

"add syscalls to make qemu do all cache maintenance"
Variant 1: implement as kvm ioctls - to avoid trying to get
           syscalls into the general kernel
Variant 2: add real syscalls, or maybe just ARM private SWIs
           like __ARM_NR_cacheflush
This approach should work, and if we add an invalidate syscall,
then we shouldn't need any kvm changes at all, i.e. no need for
the memslot flag. I haven't experimented with this yet, but I'm
starting to like the idea of variant 2, with a private SWI, so
will try to pull something together soon for that.

"describe the problematic memory as cached to the guest"
Not an ideal solution for virt. Could maybe be workable as a
quirk for a specific device though.

re: $SUBJECT; Here 'cluster' is defined by the urban dictionary.

[1] http://thread.gmane.org/gmane.comp.emulators.kvm.arm.devel/34/

[Qemu-devel] [RFC PATCH 0/6] flush/invalidate on entry/exit

[Andrew Jones]

Userspace flags memory regions as incoherent and kvm flushes/
invalidates those regions on entry/exit from userspace.

 Result before patch 6/6: restores coherency, waaaaay toooo sloooooow
 Result with patch 6/6: fast again - well, we removed the code...

Andrew Jones (6):
  kvm: promote KVM_MEMSLOT_INCOHERENT to uapi
  KVM: Introduce incoherent cache maintenance API
  KVM: ARM: change __coherent_cache_guest_page interface
  KVM: ARM: extend __coherent_cache_guest_page
  KVM: ARM: implement kvm_*_incoherent_memory_regions
  KVM: ARM: no need for kvm_arch_flush_incoherent

 arch/arm/include/asm/kvm_mmu.h    | 12 +++----
 arch/arm/include/uapi/asm/kvm.h   |  1 +
 arch/arm/kvm/arm.c                |  4 +++
 arch/arm/kvm/mmu.c                | 72 +++++++++++++++++++++++++++++++++++++--
 arch/arm64/include/asm/kvm_mmu.h  | 12 ++++---
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 include/linux/kvm_host.h          | 15 +++++++-
 include/uapi/linux/kvm.h          |  1 +
 virt/kvm/kvm_main.c               | 45 +++++++++++++++++++++++-
 9 files changed, 147 insertions(+), 16 deletions(-)

-- 
1.8.3.1

[RFC PATCH 0/6] flush/invalidate on entry/exit

[Andrew Jones]

Userspace flags memory regions as incoherent and kvm flushes/
invalidates those regions on entry/exit from userspace.

 Result before patch 6/6: restores coherency, waaaaay toooo sloooooow
 Result with patch 6/6: fast again - well, we removed the code...

Andrew Jones (6):
  kvm: promote KVM_MEMSLOT_INCOHERENT to uapi
  KVM: Introduce incoherent cache maintenance API
  KVM: ARM: change __coherent_cache_guest_page interface
  KVM: ARM: extend __coherent_cache_guest_page
  KVM: ARM: implement kvm_*_incoherent_memory_regions
  KVM: ARM: no need for kvm_arch_flush_incoherent

 arch/arm/include/asm/kvm_mmu.h    | 12 +++----
 arch/arm/include/uapi/asm/kvm.h   |  1 +
 arch/arm/kvm/arm.c                |  4 +++
 arch/arm/kvm/mmu.c                | 72 +++++++++++++++++++++++++++++++++++++--
 arch/arm64/include/asm/kvm_mmu.h  | 12 ++++---
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 include/linux/kvm_host.h          | 15 +++++++-
 include/uapi/linux/kvm.h          |  1 +
 virt/kvm/kvm_main.c               | 45 +++++++++++++++++++++++-
 9 files changed, 147 insertions(+), 16 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 1/6] kvm: promote KVM_MEMSLOT_INCOHERENT to uapi

[Andrew Jones]

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/kvm/mmu.c       | 9 +++++++--
 include/linux/kvm_host.h | 1 -
 include/uapi/linux/kvm.h | 1 +
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index bcc1b3ad2adce..a806e8cecc01b 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1260,7 +1260,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	if (!hugetlb && !force_pte)
 		hugetlb = transparent_hugepage_adjust(&pfn, &fault_ipa);
 
-	fault_ipa_uncached = memslot->flags & KVM_MEMSLOT_INCOHERENT;
+	fault_ipa_uncached = memslot->flags & KVM_MEM_INCOHERENT;
 
 	if (hugetlb) {
 		pmd_t new_pmd = pfn_pmd(pfn, mem_type);
@@ -1784,15 +1784,20 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 			    unsigned long npages)
 {
+#if 1
 	/*
 	 * Readonly memslots are not incoherent with the caches by definition,
 	 * but in practice, they are used mostly to emulate ROMs or NOR flashes
 	 * that the guest may consider devices and hence map as uncached.
 	 * To prevent incoherency issues in these cases, tag all readonly
 	 * regions as incoherent.
+	 *
+	 * This heuristic can be removed after userspace has been updated to
+	 * use KVM_MEM_INCOHERENT on readonly regions when necessary.
 	 */
 	if (slot->flags & KVM_MEM_READONLY)
-		slot->flags |= KVM_MEMSLOT_INCOHERENT;
+		slot->flags |= KVM_MEM_INCOHERENT;
+#endif
 	return 0;
 }
 
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 3b934cc94cc83..9dfb519c51e5b 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -43,7 +43,6 @@
  * include/linux/kvm_h.
  */
 #define KVM_MEMSLOT_INVALID	(1UL << 16)
-#define KVM_MEMSLOT_INCOHERENT	(1UL << 17)
 
 /* Two fragments for cross MMIO pages. */
 #define KVM_MAX_MMIO_FRAGMENTS	2
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index a37fd1224f363..f7f9432bcf485 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {
  */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_INCOHERENT	(1UL << 2)
 
 /* for KVM_IRQ_LINE */
 struct kvm_irq_level {
-- 
1.8.3.1

[RFC PATCH 1/6] kvm: promote KVM_MEMSLOT_INCOHERENT to uapi

[Andrew Jones]

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/kvm/mmu.c       | 9 +++++++--
 include/linux/kvm_host.h | 1 -
 include/uapi/linux/kvm.h | 1 +
 3 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index bcc1b3ad2adce..a806e8cecc01b 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1260,7 +1260,7 @@ static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 	if (!hugetlb && !force_pte)
 		hugetlb = transparent_hugepage_adjust(&pfn, &fault_ipa);
 
-	fault_ipa_uncached = memslot->flags & KVM_MEMSLOT_INCOHERENT;
+	fault_ipa_uncached = memslot->flags & KVM_MEM_INCOHERENT;
 
 	if (hugetlb) {
 		pmd_t new_pmd = pfn_pmd(pfn, mem_type);
@@ -1784,15 +1784,20 @@ void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 			    unsigned long npages)
 {
+#if 1
 	/*
 	 * Readonly memslots are not incoherent with the caches by definition,
 	 * but in practice, they are used mostly to emulate ROMs or NOR flashes
 	 * that the guest may consider devices and hence map as uncached.
 	 * To prevent incoherency issues in these cases, tag all readonly
 	 * regions as incoherent.
+	 *
+	 * This heuristic can be removed after userspace has been updated to
+	 * use KVM_MEM_INCOHERENT on readonly regions when necessary.
 	 */
 	if (slot->flags & KVM_MEM_READONLY)
-		slot->flags |= KVM_MEMSLOT_INCOHERENT;
+		slot->flags |= KVM_MEM_INCOHERENT;
+#endif
 	return 0;
 }
 
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 3b934cc94cc83..9dfb519c51e5b 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -43,7 +43,6 @@
  * include/linux/kvm_h.
  */
 #define KVM_MEMSLOT_INVALID	(1UL << 16)
-#define KVM_MEMSLOT_INCOHERENT	(1UL << 17)
 
 /* Two fragments for cross MMIO pages. */
 #define KVM_MAX_MMIO_FRAGMENTS	2
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index a37fd1224f363..f7f9432bcf485 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {
  */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_INCOHERENT	(1UL << 2)
 
 /* for KVM_IRQ_LINE */
 struct kvm_irq_level {
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 2/6] KVM: Introduce incoherent cache maintenance API

[Andrew Jones]

Add two new memslot functions to the API

  kvm_flush_incoherent_memory_regions
    flush all KVM_MEM_INCOHERENT memslot addresses

  kvm_invalidate_incoherent_memory_regions
    invalidate all KVM_MEM_INCOHERENT memslot addresses

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 include/linux/kvm_host.h | 14 ++++++++++++++
 virt/kvm/kvm_main.c      | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9dfb519c51e5b..2bdbeeb1b2704 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -517,6 +517,8 @@ int kvm_set_memory_region(struct kvm *kvm,
 			  struct kvm_userspace_memory_region *mem);
 int __kvm_set_memory_region(struct kvm *kvm,
 			    struct kvm_userspace_memory_region *mem);
+void kvm_flush_incoherent_memory_regions(struct kvm *kvm);
+void kvm_invalidate_incoherent_memory_regions(struct kvm *kvm);
 void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 			   struct kvm_memory_slot *dont);
 int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
@@ -705,6 +707,18 @@ static inline bool kvm_arch_has_noncoherent_dma(struct kvm *kvm)
 }
 #endif
 
+#ifdef __KVM_HAVE_INCOHERENT_MEM
+void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot);
+void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot);
+#else
+void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+}
+void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+}
+#endif
+
 static inline wait_queue_head_t *kvm_arch_vcpu_wq(struct kvm_vcpu *vcpu)
 {
 #ifdef __KVM_HAVE_ARCH_WQP
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 449c1c9ee68b4..96f44c57b8808 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -716,7 +716,7 @@ static void update_memslots(struct kvm_memslots *slots,
 
 static int check_memory_region_flags(struct kvm_userspace_memory_region *mem)
 {
-	u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES;
+	u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES | KVM_MEM_INCOHERENT;
 
 #ifdef __KVM_HAVE_READONLY_MEM
 	valid_flags |= KVM_MEM_READONLY;
@@ -960,6 +960,49 @@ static int kvm_vm_ioctl_set_memory_region(struct kvm *kvm,
 	return kvm_set_memory_region(kvm, mem);
 }
 
+static inline bool memory_region_is_incoherent(struct kvm_memory_slot *slot)
+{
+	return slot && slot->id < KVM_USER_MEM_SLOTS
+		&& !(slot->flags & KVM_MEMSLOT_INVALID)
+		&& slot->flags & KVM_MEM_INCOHERENT;
+}
+
+void kvm_flush_incoherent_memory_regions(struct kvm *kvm)
+{
+	struct kvm_memslots *slots = kvm_memslots(kvm);
+	struct kvm_memory_slot *slot;
+	int idx;
+
+	idx = srcu_read_lock(&kvm->srcu);
+
+	kvm_for_each_memslot(slot, slots) {
+		if (!memory_region_is_incoherent(slot))
+			continue;
+		kvm_arch_flush_incoherent(kvm, slot);
+	}
+	srcu_read_unlock(&kvm->srcu, idx);
+}
+EXPORT_SYMBOL_GPL(kvm_flush_incoherent_memory_regions);
+
+void kvm_invalidate_incoherent_memory_regions(struct kvm *kvm)
+{
+	struct kvm_memslots *slots = kvm_memslots(kvm);
+	struct kvm_memory_slot *slot;
+	int idx;
+
+	idx = srcu_read_lock(&kvm->srcu);
+
+	kvm_for_each_memslot(slot, slots) {
+		if (slot->flags & KVM_MEM_READONLY)
+			continue;
+		if (!memory_region_is_incoherent(slot))
+			continue;
+		kvm_arch_invalidate_incoherent(kvm, slot);
+	}
+	srcu_read_unlock(&kvm->srcu, idx);
+}
+EXPORT_SYMBOL_GPL(kvm_invalidate_incoherent_memory_regions);
+
 int kvm_get_dirty_log(struct kvm *kvm,
 			struct kvm_dirty_log *log, int *is_dirty)
 {
-- 
1.8.3.1

[RFC PATCH 2/6] KVM: Introduce incoherent cache maintenance API

[Andrew Jones]

Add two new memslot functions to the API

  kvm_flush_incoherent_memory_regions
    flush all KVM_MEM_INCOHERENT memslot addresses

  kvm_invalidate_incoherent_memory_regions
    invalidate all KVM_MEM_INCOHERENT memslot addresses

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 include/linux/kvm_host.h | 14 ++++++++++++++
 virt/kvm/kvm_main.c      | 45 ++++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 58 insertions(+), 1 deletion(-)

diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9dfb519c51e5b..2bdbeeb1b2704 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -517,6 +517,8 @@ int kvm_set_memory_region(struct kvm *kvm,
 			  struct kvm_userspace_memory_region *mem);
 int __kvm_set_memory_region(struct kvm *kvm,
 			    struct kvm_userspace_memory_region *mem);
+void kvm_flush_incoherent_memory_regions(struct kvm *kvm);
+void kvm_invalidate_incoherent_memory_regions(struct kvm *kvm);
 void kvm_arch_free_memslot(struct kvm *kvm, struct kvm_memory_slot *free,
 			   struct kvm_memory_slot *dont);
 int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
@@ -705,6 +707,18 @@ static inline bool kvm_arch_has_noncoherent_dma(struct kvm *kvm)
 }
 #endif
 
+#ifdef __KVM_HAVE_INCOHERENT_MEM
+void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot);
+void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot);
+#else
+void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+}
+void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+}
+#endif
+
 static inline wait_queue_head_t *kvm_arch_vcpu_wq(struct kvm_vcpu *vcpu)
 {
 #ifdef __KVM_HAVE_ARCH_WQP
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 449c1c9ee68b4..96f44c57b8808 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -716,7 +716,7 @@ static void update_memslots(struct kvm_memslots *slots,
 
 static int check_memory_region_flags(struct kvm_userspace_memory_region *mem)
 {
-	u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES;
+	u32 valid_flags = KVM_MEM_LOG_DIRTY_PAGES | KVM_MEM_INCOHERENT;
 
 #ifdef __KVM_HAVE_READONLY_MEM
 	valid_flags |= KVM_MEM_READONLY;
@@ -960,6 +960,49 @@ static int kvm_vm_ioctl_set_memory_region(struct kvm *kvm,
 	return kvm_set_memory_region(kvm, mem);
 }
 
+static inline bool memory_region_is_incoherent(struct kvm_memory_slot *slot)
+{
+	return slot && slot->id < KVM_USER_MEM_SLOTS
+		&& !(slot->flags & KVM_MEMSLOT_INVALID)
+		&& slot->flags & KVM_MEM_INCOHERENT;
+}
+
+void kvm_flush_incoherent_memory_regions(struct kvm *kvm)
+{
+	struct kvm_memslots *slots = kvm_memslots(kvm);
+	struct kvm_memory_slot *slot;
+	int idx;
+
+	idx = srcu_read_lock(&kvm->srcu);
+
+	kvm_for_each_memslot(slot, slots) {
+		if (!memory_region_is_incoherent(slot))
+			continue;
+		kvm_arch_flush_incoherent(kvm, slot);
+	}
+	srcu_read_unlock(&kvm->srcu, idx);
+}
+EXPORT_SYMBOL_GPL(kvm_flush_incoherent_memory_regions);
+
+void kvm_invalidate_incoherent_memory_regions(struct kvm *kvm)
+{
+	struct kvm_memslots *slots = kvm_memslots(kvm);
+	struct kvm_memory_slot *slot;
+	int idx;
+
+	idx = srcu_read_lock(&kvm->srcu);
+
+	kvm_for_each_memslot(slot, slots) {
+		if (slot->flags & KVM_MEM_READONLY)
+			continue;
+		if (!memory_region_is_incoherent(slot))
+			continue;
+		kvm_arch_invalidate_incoherent(kvm, slot);
+	}
+	srcu_read_unlock(&kvm->srcu, idx);
+}
+EXPORT_SYMBOL_GPL(kvm_invalidate_incoherent_memory_regions);
+
 int kvm_get_dirty_log(struct kvm *kvm,
 			struct kvm_dirty_log *log, int *is_dirty)
 {
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 3/6] KVM: ARM: change __coherent_cache_guest_page interface

[Andrew Jones]

Remove the vcpu parameter. We can do this by doing the same
query in the caller of __coherent_cache_guest_page, and then
folding the result into its ipa_uncached parameter, which we
rename to need_flush.

A later patch will add a new caller for __coherent_cache_guest_page
that does not have a vcpu parameter.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/asm/kvm_mmu.h   | 7 ++-----
 arch/arm/kvm/mmu.c               | 3 ++-
 arch/arm64/include/asm/kvm_mmu.h | 7 +++----
 3 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 37ca2a4c6f094..fd801e96fdd3c 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -183,9 +183,8 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 	return (vcpu->arch.cp15[c1_SCTLR] & 0b101) == 0b101;
 }
 
-static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
-					       unsigned long size,
-					       bool ipa_uncached)
+static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
+					       bool need_flush)
 {
 	/*
 	 * If we are going to insert an instruction page and the icache is
@@ -205,8 +204,6 @@ static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 	 * and iterate over the range.
 	 */
 
-	bool need_flush = !vcpu_has_cache_enabled(vcpu) || ipa_uncached;
-
 	VM_BUG_ON(size & PAGE_MASK);
 
 	if (!need_flush && !icache_is_pipt())
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index a806e8cecc01b..781afc712871c 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1157,7 +1157,8 @@ void kvm_arch_mmu_write_protect_pt_masked(struct kvm *kvm,
 static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 				      unsigned long size, bool uncached)
 {
-	__coherent_cache_guest_page(vcpu, pfn, size, uncached);
+	bool need_flush = uncached || !vcpu_has_cache_enabled(vcpu);
+	__coherent_cache_guest_page(pfn, size, need_flush);
 }
 
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6458b53731421..56a976c776bc2 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -264,13 +264,12 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 	return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
 }
 
-static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
-					       unsigned long size,
-					       bool ipa_uncached)
+static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
+					       bool need_flush)
 {
 	void *va = page_address(pfn_to_page(pfn));
 
-	if (!vcpu_has_cache_enabled(vcpu) || ipa_uncached)
+	if (need_flush)
 		kvm_flush_dcache_to_poc(va, size);
 
 	if (!icache_is_aliasing()) {		/* PIPT */
-- 
1.8.3.1

[RFC PATCH 3/6] KVM: ARM: change __coherent_cache_guest_page interface

[Andrew Jones]

Remove the vcpu parameter. We can do this by doing the same
query in the caller of __coherent_cache_guest_page, and then
folding the result into its ipa_uncached parameter, which we
rename to need_flush.

A later patch will add a new caller for __coherent_cache_guest_page
that does not have a vcpu parameter.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/asm/kvm_mmu.h   | 7 ++-----
 arch/arm/kvm/mmu.c               | 3 ++-
 arch/arm64/include/asm/kvm_mmu.h | 7 +++----
 3 files changed, 7 insertions(+), 10 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index 37ca2a4c6f094..fd801e96fdd3c 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -183,9 +183,8 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 	return (vcpu->arch.cp15[c1_SCTLR] & 0b101) == 0b101;
 }
 
-static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
-					       unsigned long size,
-					       bool ipa_uncached)
+static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
+					       bool need_flush)
 {
 	/*
 	 * If we are going to insert an instruction page and the icache is
@@ -205,8 +204,6 @@ static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 	 * and iterate over the range.
 	 */
 
-	bool need_flush = !vcpu_has_cache_enabled(vcpu) || ipa_uncached;
-
 	VM_BUG_ON(size & PAGE_MASK);
 
 	if (!need_flush && !icache_is_pipt())
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index a806e8cecc01b..781afc712871c 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1157,7 +1157,8 @@ void kvm_arch_mmu_write_protect_pt_masked(struct kvm *kvm,
 static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 				      unsigned long size, bool uncached)
 {
-	__coherent_cache_guest_page(vcpu, pfn, size, uncached);
+	bool need_flush = uncached || !vcpu_has_cache_enabled(vcpu);
+	__coherent_cache_guest_page(pfn, size, need_flush);
 }
 
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 6458b53731421..56a976c776bc2 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -264,13 +264,12 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 	return (vcpu_sys_reg(vcpu, SCTLR_EL1) & 0b101) == 0b101;
 }
 
-static inline void __coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
-					       unsigned long size,
-					       bool ipa_uncached)
+static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
+					       bool need_flush)
 {
 	void *va = page_address(pfn_to_page(pfn));
 
-	if (!vcpu_has_cache_enabled(vcpu) || ipa_uncached)
+	if (need_flush)
 		kvm_flush_dcache_to_poc(va, size);
 
 	if (!icache_is_aliasing()) {		/* PIPT */
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 4/6] KVM: ARM: extend __coherent_cache_guest_page

[Andrew Jones]

Also support only invalidating, rather than always invalidate+clear.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/asm/kvm_mmu.h   | 7 +++++--
 arch/arm/kvm/mmu.c               | 2 +-
 arch/arm64/include/asm/kvm_mmu.h | 7 +++++--
 3 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index fd801e96fdd3c..a1c7f554f5de8 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -176,7 +176,8 @@ static inline void *kvm_get_hwpgd(struct kvm *kvm)
 
 struct kvm;
 
-#define kvm_flush_dcache_to_poc(a,l)	__cpuc_flush_dcache_area((a), (l))
+#define kvm_flush_dcache_to_poc(a,l)		__cpuc_flush_dcache_area((a), (l))
+#define kvm_invalidate_cache_to_poc(a,l)	dmac_unmap_area((a), (l), 0)
 
 static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 {
@@ -184,7 +185,7 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 }
 
 static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
-					       bool need_flush)
+					       bool need_flush, bool invalidate)
 {
 	/*
 	 * If we are going to insert an instruction page and the icache is
@@ -214,6 +215,8 @@ static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
 
 		if (need_flush)
 			kvm_flush_dcache_to_poc(va, PAGE_SIZE);
+		if (invalidate)
+			kvm_invalidate_cache_to_poc(va, PAGE_SIZE);
 
 		if (icache_is_pipt())
 			__cpuc_coherent_user_range((unsigned long)va,
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 781afc712871c..2f3a6581b9200 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1158,7 +1158,7 @@ static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 				      unsigned long size, bool uncached)
 {
 	bool need_flush = uncached || !vcpu_has_cache_enabled(vcpu);
-	__coherent_cache_guest_page(pfn, size, need_flush);
+	__coherent_cache_guest_page(pfn, size, need_flush, false);
 }
 
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 56a976c776bc2..e1090ad70133d 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -257,7 +257,8 @@ static inline bool kvm_page_empty(void *ptr)
 
 struct kvm;
 
-#define kvm_flush_dcache_to_poc(a,l)	__flush_dcache_area((a), (l))
+#define kvm_flush_dcache_to_poc(a,l)		__flush_dcache_area((a), (l))
+#define kvm_invalidate_cache_to_poc(a,l)	__dma_unmap_area((a), (l), 0)
 
 static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 {
@@ -265,12 +266,14 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 }
 
 static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
-					       bool need_flush)
+					       bool need_flush, bool invalidate)
 {
 	void *va = page_address(pfn_to_page(pfn));
 
 	if (need_flush)
 		kvm_flush_dcache_to_poc(va, size);
+	if (invalidate)
+		kvm_invalidate_cache_to_poc(va, size);
 
 	if (!icache_is_aliasing()) {		/* PIPT */
 		flush_icache_range((unsigned long)va,
-- 
1.8.3.1

[RFC PATCH 4/6] KVM: ARM: extend __coherent_cache_guest_page

[Andrew Jones]

Also support only invalidating, rather than always invalidate+clear.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/asm/kvm_mmu.h   | 7 +++++--
 arch/arm/kvm/mmu.c               | 2 +-
 arch/arm64/include/asm/kvm_mmu.h | 7 +++++--
 3 files changed, 11 insertions(+), 5 deletions(-)

diff --git a/arch/arm/include/asm/kvm_mmu.h b/arch/arm/include/asm/kvm_mmu.h
index fd801e96fdd3c..a1c7f554f5de8 100644
--- a/arch/arm/include/asm/kvm_mmu.h
+++ b/arch/arm/include/asm/kvm_mmu.h
@@ -176,7 +176,8 @@ static inline void *kvm_get_hwpgd(struct kvm *kvm)
 
 struct kvm;
 
-#define kvm_flush_dcache_to_poc(a,l)	__cpuc_flush_dcache_area((a), (l))
+#define kvm_flush_dcache_to_poc(a,l)		__cpuc_flush_dcache_area((a), (l))
+#define kvm_invalidate_cache_to_poc(a,l)	dmac_unmap_area((a), (l), 0)
 
 static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 {
@@ -184,7 +185,7 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 }
 
 static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
-					       bool need_flush)
+					       bool need_flush, bool invalidate)
 {
 	/*
 	 * If we are going to insert an instruction page and the icache is
@@ -214,6 +215,8 @@ static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
 
 		if (need_flush)
 			kvm_flush_dcache_to_poc(va, PAGE_SIZE);
+		if (invalidate)
+			kvm_invalidate_cache_to_poc(va, PAGE_SIZE);
 
 		if (icache_is_pipt())
 			__cpuc_coherent_user_range((unsigned long)va,
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 781afc712871c..2f3a6581b9200 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1158,7 +1158,7 @@ static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 				      unsigned long size, bool uncached)
 {
 	bool need_flush = uncached || !vcpu_has_cache_enabled(vcpu);
-	__coherent_cache_guest_page(pfn, size, need_flush);
+	__coherent_cache_guest_page(pfn, size, need_flush, false);
 }
 
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
diff --git a/arch/arm64/include/asm/kvm_mmu.h b/arch/arm64/include/asm/kvm_mmu.h
index 56a976c776bc2..e1090ad70133d 100644
--- a/arch/arm64/include/asm/kvm_mmu.h
+++ b/arch/arm64/include/asm/kvm_mmu.h
@@ -257,7 +257,8 @@ static inline bool kvm_page_empty(void *ptr)
 
 struct kvm;
 
-#define kvm_flush_dcache_to_poc(a,l)	__flush_dcache_area((a), (l))
+#define kvm_flush_dcache_to_poc(a,l)		__flush_dcache_area((a), (l))
+#define kvm_invalidate_cache_to_poc(a,l)	__dma_unmap_area((a), (l), 0)
 
 static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 {
@@ -265,12 +266,14 @@ static inline bool vcpu_has_cache_enabled(struct kvm_vcpu *vcpu)
 }
 
 static inline void __coherent_cache_guest_page(pfn_t pfn, unsigned long size,
-					       bool need_flush)
+					       bool need_flush, bool invalidate)
 {
 	void *va = page_address(pfn_to_page(pfn));
 
 	if (need_flush)
 		kvm_flush_dcache_to_poc(va, size);
+	if (invalidate)
+		kvm_invalidate_cache_to_poc(va, size);
 
 	if (!icache_is_aliasing()) {		/* PIPT */
 		flush_icache_range((unsigned long)va,
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 5/6] KVM: ARM: implement kvm_*_incoherent_memory_regions

[Andrew Jones]

Add the kvm_*_incoherent_memory_regions calls to arm's
kvm_arch_vcpu_ioctl_run and implement the corresponding
arch flush/invalidate functions.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/uapi/asm/kvm.h   |  1 +
 arch/arm/kvm/arm.c                |  4 +++
 arch/arm/kvm/mmu.c                | 54 +++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 4 files changed, 60 insertions(+)

diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 09ee408c1a676..cb0898a995c4f 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -26,6 +26,7 @@
 #define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_INCOHERENT_MEM
 
 #define KVM_REG_SIZE(id)						\
 	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index c9e6ef1f7403a..789c03c84e7c0 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -486,6 +486,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 			return ret;
 	}
 
+	kvm_flush_incoherent_memory_regions(vcpu->kvm);
+
 	if (vcpu->sigset_active)
 		sigprocmask(SIG_SETMASK, &vcpu->sigset, &sigsaved);
 
@@ -556,6 +558,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 		ret = handle_exit(vcpu, run, ret);
 	}
 
+	kvm_invalidate_incoherent_memory_regions(vcpu->kvm);
+
 	if (vcpu->sigset_active)
 		sigprocmask(SIG_SETMASK, &sigsaved, NULL);
 	return ret;
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 2f3a6581b9200..2f45db9cd436a 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1161,6 +1161,24 @@ static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 	__coherent_cache_guest_page(pfn, size, need_flush, false);
 }
 
+static void coherent_cache_memslot(struct kvm_memory_slot *slot, bool flush)
+{
+	gfn_t gfn, end = slot->base_gfn + slot->npages;
+	pfn_t pfn;
+
+	for (gfn = slot->base_gfn; gfn < end; ++gfn) {
+		pfn = gfn_to_pfn_memslot(slot, gfn);
+		if (is_error_pfn(pfn)) {
+			pr_err("%s: Bad pfn: gfn=%llx, pfn=%llx, "
+				"userspace_addr=%lx\n", __func__,
+				gfn, pfn, slot->userspace_addr);
+			continue;
+		}
+		__coherent_cache_guest_page(pfn, PAGE_SIZE, flush, !flush);
+		kvm_release_pfn_clean(pfn);
+	}
+}
+
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 			  struct kvm_memory_slot *memslot, unsigned long hva,
 			  unsigned long fault_status)
@@ -1802,6 +1820,42 @@ int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 	return 0;
 }
 
+void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+	if (slot->flags & KVM_MEM_READONLY) {
+		/*
+		 * Readonly memory shouldn't be changing, and we do a
+		 * clean+invalidate for KVM_MEM_INCOHERENT memory when
+		 * faulting it in. So, there's nothing to do now.
+		 */
+		return;
+	}
+
+	/*
+	 * Ideally, we would further filter out all pages not touched by
+	 * userspace on the last exit. No way to know those though, unless
+	 * we force userspace to fault on all pages in the incoherent
+	 * memory regions, but even then, I don't see any sane way for
+	 * do_wp_page to handle the faults without modification. So, sigh...
+	 */
+
+	coherent_cache_memslot(slot, true);
+}
+
+void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+	if (slot->flags & KVM_MEM_LOG_DIRTY_PAGES) {
+		/*
+		 * We fault each write when logging is enabled, and do a
+		 * clean+invalidate on KVM_MEM_INCOHERENT memory while
+		 * handling the fault. So, there's nothing to do now.
+		 */
+		return;
+	}
+
+	coherent_cache_memslot(slot, false);
+}
+
 void kvm_arch_memslots_updated(struct kvm *kvm)
 {
 }
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 8e38878c87c61..29ddf77958c2a 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -38,6 +38,7 @@
 #define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_INCOHERENT_MEM
 
 #define KVM_REG_SIZE(id)						\
 	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
-- 
1.8.3.1

[RFC PATCH 5/6] KVM: ARM: implement kvm_*_incoherent_memory_regions

[Andrew Jones]

Add the kvm_*_incoherent_memory_regions calls to arm's
kvm_arch_vcpu_ioctl_run and implement the corresponding
arch flush/invalidate functions.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/include/uapi/asm/kvm.h   |  1 +
 arch/arm/kvm/arm.c                |  4 +++
 arch/arm/kvm/mmu.c                | 54 +++++++++++++++++++++++++++++++++++++++
 arch/arm64/include/uapi/asm/kvm.h |  1 +
 4 files changed, 60 insertions(+)

diff --git a/arch/arm/include/uapi/asm/kvm.h b/arch/arm/include/uapi/asm/kvm.h
index 09ee408c1a676..cb0898a995c4f 100644
--- a/arch/arm/include/uapi/asm/kvm.h
+++ b/arch/arm/include/uapi/asm/kvm.h
@@ -26,6 +26,7 @@
 #define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_INCOHERENT_MEM
 
 #define KVM_REG_SIZE(id)						\
 	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
diff --git a/arch/arm/kvm/arm.c b/arch/arm/kvm/arm.c
index c9e6ef1f7403a..789c03c84e7c0 100644
--- a/arch/arm/kvm/arm.c
+++ b/arch/arm/kvm/arm.c
@@ -486,6 +486,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 			return ret;
 	}
 
+	kvm_flush_incoherent_memory_regions(vcpu->kvm);
+
 	if (vcpu->sigset_active)
 		sigprocmask(SIG_SETMASK, &vcpu->sigset, &sigsaved);
 
@@ -556,6 +558,8 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu, struct kvm_run *run)
 		ret = handle_exit(vcpu, run, ret);
 	}
 
+	kvm_invalidate_incoherent_memory_regions(vcpu->kvm);
+
 	if (vcpu->sigset_active)
 		sigprocmask(SIG_SETMASK, &sigsaved, NULL);
 	return ret;
diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 2f3a6581b9200..2f45db9cd436a 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1161,6 +1161,24 @@ static void coherent_cache_guest_page(struct kvm_vcpu *vcpu, pfn_t pfn,
 	__coherent_cache_guest_page(pfn, size, need_flush, false);
 }
 
+static void coherent_cache_memslot(struct kvm_memory_slot *slot, bool flush)
+{
+	gfn_t gfn, end = slot->base_gfn + slot->npages;
+	pfn_t pfn;
+
+	for (gfn = slot->base_gfn; gfn < end; ++gfn) {
+		pfn = gfn_to_pfn_memslot(slot, gfn);
+		if (is_error_pfn(pfn)) {
+			pr_err("%s: Bad pfn: gfn=%llx, pfn=%llx, "
+				"userspace_addr=%lx\n", __func__,
+				gfn, pfn, slot->userspace_addr);
+			continue;
+		}
+		__coherent_cache_guest_page(pfn, PAGE_SIZE, flush, !flush);
+		kvm_release_pfn_clean(pfn);
+	}
+}
+
 static int user_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
 			  struct kvm_memory_slot *memslot, unsigned long hva,
 			  unsigned long fault_status)
@@ -1802,6 +1820,42 @@ int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 	return 0;
 }
 
+void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+	if (slot->flags & KVM_MEM_READONLY) {
+		/*
+		 * Readonly memory shouldn't be changing, and we do a
+		 * clean+invalidate for KVM_MEM_INCOHERENT memory when
+		 * faulting it in. So, there's nothing to do now.
+		 */
+		return;
+	}
+
+	/*
+	 * Ideally, we would further filter out all pages not touched by
+	 * userspace on the last exit. No way to know those though, unless
+	 * we force userspace to fault on all pages in the incoherent
+	 * memory regions, but even then, I don't see any sane way for
+	 * do_wp_page to handle the faults without modification. So, sigh...
+	 */
+
+	coherent_cache_memslot(slot, true);
+}
+
+void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
+{
+	if (slot->flags & KVM_MEM_LOG_DIRTY_PAGES) {
+		/*
+		 * We fault each write when logging is enabled, and do a
+		 * clean+invalidate on KVM_MEM_INCOHERENT memory while
+		 * handling the fault. So, there's nothing to do now.
+		 */
+		return;
+	}
+
+	coherent_cache_memslot(slot, false);
+}
+
 void kvm_arch_memslots_updated(struct kvm *kvm)
 {
 }
diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h
index 8e38878c87c61..29ddf77958c2a 100644
--- a/arch/arm64/include/uapi/asm/kvm.h
+++ b/arch/arm64/include/uapi/asm/kvm.h
@@ -38,6 +38,7 @@
 #define __KVM_HAVE_GUEST_DEBUG
 #define __KVM_HAVE_IRQ_LINE
 #define __KVM_HAVE_READONLY_MEM
+#define __KVM_HAVE_INCOHERENT_MEM
 
 #define KVM_REG_SIZE(id)						\
 	(1U << (((id) & KVM_REG_SIZE_MASK) >> KVM_REG_SIZE_SHIFT))
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 6/6] KVM: ARM: no need for kvm_arch_flush_incoherent

[Andrew Jones]

kvm_arch_flush_incoherent makes things too slow, and we don't
need it. Userspace can flush for us, as the necessary cache
maintenance instruction is not (necessarily) privileged.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/kvm/mmu.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 2f45db9cd436a..b2d87587a9d79 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1822,6 +1822,11 @@ int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
 {
+	/*
+	 * We no longer need this function, now that userspace does the
+	 * flushing.
+	 */
+#if 0
 	if (slot->flags & KVM_MEM_READONLY) {
 		/*
 		 * Readonly memory shouldn't be changing, and we do a
@@ -1840,6 +1845,7 @@ void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
 	 */
 
 	coherent_cache_memslot(slot, true);
+#endif
 }
 
 void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
-- 
1.8.3.1

[RFC PATCH 6/6] KVM: ARM: no need for kvm_arch_flush_incoherent

[Andrew Jones]

kvm_arch_flush_incoherent makes things too slow, and we don't
need it. Userspace can flush for us, as the necessary cache
maintenance instruction is not (necessarily) privileged.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 arch/arm/kvm/mmu.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/arch/arm/kvm/mmu.c b/arch/arm/kvm/mmu.c
index 2f45db9cd436a..b2d87587a9d79 100644
--- a/arch/arm/kvm/mmu.c
+++ b/arch/arm/kvm/mmu.c
@@ -1822,6 +1822,11 @@ int kvm_arch_create_memslot(struct kvm *kvm, struct kvm_memory_slot *slot,
 
 void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
 {
+	/*
+	 * We no longer need this function, now that userspace does the
+	 * flushing.
+	 */
+#if 0
 	if (slot->flags & KVM_MEM_READONLY) {
 		/*
 		 * Readonly memory shouldn't be changing, and we do a
@@ -1840,6 +1845,7 @@ void kvm_arch_flush_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
 	 */
 
 	coherent_cache_memslot(slot, true);
+#endif
 }
 
 void kvm_arch_invalidate_incoherent(struct kvm *kvm, struct kvm_memory_slot *slot)
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 0/6] support KVM_MEM_INCOHERENT

[Andrew Jones]

Add support for the new KVM_MEM_INCOHERENT flag, and flag
appropriate memory. (Only flags vram for now.)

Patch 6/6 doesn't appear to be complete. While the VGA output is
99% corruption free, it's not perfect, so it's missing flushes
somewhere... 

Andrew Jones (6):
  memory: add incoherent cache flag
  HACK: linux header update
  kvm-all: put kvm_mem_flags to more work
  kvm-all: set KVM_MEM_INCOHERENT
  vga: flag vram as incoherent
  memory: add clear_cache_to_poc

 exec.c                    | 16 ++++++++++------
 hw/display/vga.c          |  1 +
 include/exec/exec-all.h   | 41 +++++++++++++++++++++++++++++++++++++++++
 include/exec/memory.h     | 23 +++++++++++++++++++++++
 kvm-all.c                 | 28 ++++++++++++++++++----------
 linux-headers/linux/kvm.h |  1 +
 memory.c                  | 15 +++++++++++++++
 7 files changed, 109 insertions(+), 16 deletions(-)

-- 
1.8.3.1

[RFC PATCH 0/6] support KVM_MEM_INCOHERENT

[Andrew Jones]

Add support for the new KVM_MEM_INCOHERENT flag, and flag
appropriate memory. (Only flags vram for now.)

Patch 6/6 doesn't appear to be complete. While the VGA output is
99% corruption free, it's not perfect, so it's missing flushes
somewhere... 

Andrew Jones (6):
  memory: add incoherent cache flag
  HACK: linux header update
  kvm-all: put kvm_mem_flags to more work
  kvm-all: set KVM_MEM_INCOHERENT
  vga: flag vram as incoherent
  memory: add clear_cache_to_poc

 exec.c                    | 16 ++++++++++------
 hw/display/vga.c          |  1 +
 include/exec/exec-all.h   | 41 +++++++++++++++++++++++++++++++++++++++++
 include/exec/memory.h     | 23 +++++++++++++++++++++++
 kvm-all.c                 | 28 ++++++++++++++++++----------
 linux-headers/linux/kvm.h |  1 +
 memory.c                  | 15 +++++++++++++++
 7 files changed, 109 insertions(+), 16 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 1/6] memory: add incoherent cache flag

[Andrew Jones]

Add an incoherent cache flag, which indicates the region
needs explicit cache maintenance.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 include/exec/memory.h | 23 +++++++++++++++++++++++
 memory.c              | 15 +++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 06ffa1d185b93..c947b88b87241 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -158,6 +158,7 @@ struct MemoryRegion {
     bool rom_device;
     bool warning_printed; /* For reservations */
     bool flush_coalesced_mmio;
+    bool incoherent_cache;
     MemoryRegion *alias;
     hwaddr alias_offset;
     int32_t priority;
@@ -778,6 +779,28 @@ void memory_region_set_flush_coalesced(MemoryRegion *mr);
 void memory_region_clear_flush_coalesced(MemoryRegion *mr);
 
 /**
+ * memory_region_set_incoherent_cache: Flag this memory region as needing
+ *                                     explicit cache maintenance.
+ *
+ * @mr: the memory region to be updated.
+ */
+void memory_region_set_incoherent_cache(MemoryRegion *mr);
+
+/**
+ * memory_region_clear_incoherent_cache: Remove the incoherent cache flag.
+ *
+ * @mr: the memory region to be updated.
+ */
+void memory_region_clear_incoherent_cache(MemoryRegion *mr);
+
+/**
+ * memory_region_has_incoherent_cache: Return the incoherent cache flag.
+ *
+ * @mr: the memory region to check.
+ */
+bool memory_region_has_incoherent_cache(MemoryRegion *mr);
+
+/**
  * memory_region_add_eventfd: Request an eventfd to be triggered when a word
  *                            is written to a location.
  *
diff --git a/memory.c b/memory.c
index 20f6d9eeac737..fa74bcb8c1e4c 100644
--- a/memory.c
+++ b/memory.c
@@ -1549,6 +1549,21 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr)
     }
 }
 
+void memory_region_set_incoherent_cache(MemoryRegion *mr)
+{
+    mr->incoherent_cache = true;
+}
+
+void memory_region_clear_incoherent_cache(MemoryRegion *mr)
+{
+    mr->incoherent_cache = false;
+}
+
+bool memory_region_has_incoherent_cache(MemoryRegion *mr)
+{
+    return mr->incoherent_cache;
+}
+
 void memory_region_add_eventfd(MemoryRegion *mr,
                                hwaddr addr,
                                unsigned size,
-- 
1.8.3.1

[RFC PATCH 1/6] memory: add incoherent cache flag

[Andrew Jones]

Add an incoherent cache flag, which indicates the region
needs explicit cache maintenance.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 include/exec/memory.h | 23 +++++++++++++++++++++++
 memory.c              | 15 +++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/include/exec/memory.h b/include/exec/memory.h
index 06ffa1d185b93..c947b88b87241 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -158,6 +158,7 @@ struct MemoryRegion {
     bool rom_device;
     bool warning_printed; /* For reservations */
     bool flush_coalesced_mmio;
+    bool incoherent_cache;
     MemoryRegion *alias;
     hwaddr alias_offset;
     int32_t priority;
@@ -778,6 +779,28 @@ void memory_region_set_flush_coalesced(MemoryRegion *mr);
 void memory_region_clear_flush_coalesced(MemoryRegion *mr);
 
 /**
+ * memory_region_set_incoherent_cache: Flag this memory region as needing
+ *                                     explicit cache maintenance.
+ *
+ * @mr: the memory region to be updated.
+ */
+void memory_region_set_incoherent_cache(MemoryRegion *mr);
+
+/**
+ * memory_region_clear_incoherent_cache: Remove the incoherent cache flag.
+ *
+ * @mr: the memory region to be updated.
+ */
+void memory_region_clear_incoherent_cache(MemoryRegion *mr);
+
+/**
+ * memory_region_has_incoherent_cache: Return the incoherent cache flag.
+ *
+ * @mr: the memory region to check.
+ */
+bool memory_region_has_incoherent_cache(MemoryRegion *mr);
+
+/**
  * memory_region_add_eventfd: Request an eventfd to be triggered when a word
  *                            is written to a location.
  *
diff --git a/memory.c b/memory.c
index 20f6d9eeac737..fa74bcb8c1e4c 100644
--- a/memory.c
+++ b/memory.c
@@ -1549,6 +1549,21 @@ void memory_region_clear_flush_coalesced(MemoryRegion *mr)
     }
 }
 
+void memory_region_set_incoherent_cache(MemoryRegion *mr)
+{
+    mr->incoherent_cache = true;
+}
+
+void memory_region_clear_incoherent_cache(MemoryRegion *mr)
+{
+    mr->incoherent_cache = false;
+}
+
+bool memory_region_has_incoherent_cache(MemoryRegion *mr)
+{
+    return mr->incoherent_cache;
+}
+
 void memory_region_add_eventfd(MemoryRegion *mr,
                                hwaddr addr,
                                unsigned size,
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 2/6] HACK: linux header update

[Andrew Jones]

Should do a proper update-linux-headers.sh update.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 linux-headers/linux/kvm.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 12045a11c036b..d04e2d781c43b 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {
  */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_INCOHERENT	(1UL << 2)
 
 /* for KVM_IRQ_LINE */
 struct kvm_irq_level {
-- 
1.8.3.1

[RFC PATCH 2/6] HACK: linux header update

[Andrew Jones]

Should do a proper update-linux-headers.sh update.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 linux-headers/linux/kvm.h | 1 +
 1 file changed, 1 insertion(+)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index 12045a11c036b..d04e2d781c43b 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -108,6 +108,7 @@ struct kvm_userspace_memory_region {
  */
 #define KVM_MEM_LOG_DIRTY_PAGES	(1UL << 0)
 #define KVM_MEM_READONLY	(1UL << 1)
+#define KVM_MEM_INCOHERENT	(1UL << 2)
 
 /* for KVM_IRQ_LINE */
 struct kvm_irq_level {
-- 
1.8.3.1

[Qemu-devel] [PATCH 3/6] kvm-all: put kvm_mem_flags to more work

[Andrew Jones]

Currently kvm_mem_flags just translates bools to bits, let's
make it also determine the bools first. This avoids its parameter
list growing each time we add a flag.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
Posted this, as it makes sense without this series.
http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01225.html

 kvm-all.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 05a79c20e0bba..507fa7204e062 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -294,10 +294,14 @@ err:
  * dirty pages logging control
  */
 
-static int kvm_mem_flags(KVMState *s, bool log_dirty, bool readonly)
+static int kvm_mem_flags(MemoryRegion *mr)
 {
+    bool readonly = mr->readonly || memory_region_is_romd(mr);
     int flags = 0;
-    flags = log_dirty ? KVM_MEM_LOG_DIRTY_PAGES : 0;
+
+    if (memory_region_is_logging(mr)) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
@@ -312,7 +316,10 @@ static int kvm_slot_dirty_pages_log_change(KVMSlot *mem, bool log_dirty)
 
     old_flags = mem->flags;
 
-    flags = (mem->flags & ~mask) | kvm_mem_flags(s, log_dirty, false);
+    flags = mem->flags & ~mask;
+    if (log_dirty) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
     mem->flags = flags;
 
     /* If nothing changed effectively, no need to issue ioctl */
@@ -642,9 +649,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
     KVMSlot *mem, old;
     int err;
     MemoryRegion *mr = section->mr;
-    bool log_dirty = memory_region_is_logging(mr);
     bool writeable = !mr->readonly && !mr->rom_device;
-    bool readonly_flag = mr->readonly || memory_region_is_romd(mr);
     hwaddr start_addr = section->offset_within_address_space;
     ram_addr_t size = int128_get64(section->size);
     void *ram = NULL;
@@ -688,7 +693,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             (ram - start_addr == mem->ram - mem->start_addr)) {
             /* The new slot fits into the existing one and comes with
              * identical parameters - update flags and done. */
-            kvm_slot_dirty_pages_log_change(mem, log_dirty);
+            kvm_slot_dirty_pages_log_change(mem, memory_region_is_logging(mr));
             return;
         }
 
@@ -721,7 +726,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             mem->memory_size = old.memory_size;
             mem->start_addr = old.start_addr;
             mem->ram = old.ram;
-            mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags = kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -742,7 +747,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             mem->memory_size = start_addr - old.start_addr;
             mem->start_addr = old.start_addr;
             mem->ram = old.ram;
-            mem->flags =  kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags =  kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -766,7 +771,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             size_delta = mem->start_addr - old.start_addr;
             mem->memory_size = old.memory_size - size_delta;
             mem->ram = old.ram + size_delta;
-            mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags = kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -788,7 +793,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
     mem->memory_size = size;
     mem->start_addr = start_addr;
     mem->ram = ram;
-    mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+    mem->flags = kvm_mem_flags(mr);
 
     err = kvm_set_user_memory_region(s, mem);
     if (err) {
-- 
1.8.3.1

[PATCH 3/6] kvm-all: put kvm_mem_flags to more work

[Andrew Jones]

Currently kvm_mem_flags just translates bools to bits, let's
make it also determine the bools first. This avoids its parameter
list growing each time we add a flag.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
Posted this, as it makes sense without this series.
http://lists.gnu.org/archive/html/qemu-devel/2015-03/msg01225.html

 kvm-all.c | 25 +++++++++++++++----------
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/kvm-all.c b/kvm-all.c
index 05a79c20e0bba..507fa7204e062 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -294,10 +294,14 @@ err:
  * dirty pages logging control
  */
 
-static int kvm_mem_flags(KVMState *s, bool log_dirty, bool readonly)
+static int kvm_mem_flags(MemoryRegion *mr)
 {
+    bool readonly = mr->readonly || memory_region_is_romd(mr);
     int flags = 0;
-    flags = log_dirty ? KVM_MEM_LOG_DIRTY_PAGES : 0;
+
+    if (memory_region_is_logging(mr)) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
@@ -312,7 +316,10 @@ static int kvm_slot_dirty_pages_log_change(KVMSlot *mem, bool log_dirty)
 
     old_flags = mem->flags;
 
-    flags = (mem->flags & ~mask) | kvm_mem_flags(s, log_dirty, false);
+    flags = mem->flags & ~mask;
+    if (log_dirty) {
+        flags |= KVM_MEM_LOG_DIRTY_PAGES;
+    }
     mem->flags = flags;
 
     /* If nothing changed effectively, no need to issue ioctl */
@@ -642,9 +649,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
     KVMSlot *mem, old;
     int err;
     MemoryRegion *mr = section->mr;
-    bool log_dirty = memory_region_is_logging(mr);
     bool writeable = !mr->readonly && !mr->rom_device;
-    bool readonly_flag = mr->readonly || memory_region_is_romd(mr);
     hwaddr start_addr = section->offset_within_address_space;
     ram_addr_t size = int128_get64(section->size);
     void *ram = NULL;
@@ -688,7 +693,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             (ram - start_addr == mem->ram - mem->start_addr)) {
             /* The new slot fits into the existing one and comes with
              * identical parameters - update flags and done. */
-            kvm_slot_dirty_pages_log_change(mem, log_dirty);
+            kvm_slot_dirty_pages_log_change(mem, memory_region_is_logging(mr));
             return;
         }
 
@@ -721,7 +726,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             mem->memory_size = old.memory_size;
             mem->start_addr = old.start_addr;
             mem->ram = old.ram;
-            mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags = kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -742,7 +747,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             mem->memory_size = start_addr - old.start_addr;
             mem->start_addr = old.start_addr;
             mem->ram = old.ram;
-            mem->flags =  kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags =  kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -766,7 +771,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
             size_delta = mem->start_addr - old.start_addr;
             mem->memory_size = old.memory_size - size_delta;
             mem->ram = old.ram + size_delta;
-            mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+            mem->flags = kvm_mem_flags(mr);
 
             err = kvm_set_user_memory_region(s, mem);
             if (err) {
@@ -788,7 +793,7 @@ static void kvm_set_phys_mem(MemoryRegionSection *section, bool add)
     mem->memory_size = size;
     mem->start_addr = start_addr;
     mem->ram = ram;
-    mem->flags = kvm_mem_flags(s, log_dirty, readonly_flag);
+    mem->flags = kvm_mem_flags(mr);
 
     err = kvm_set_user_memory_region(s, mem);
     if (err) {
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 4/6] kvm-all: set KVM_MEM_INCOHERENT

[Andrew Jones]

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 kvm-all.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/kvm-all.c b/kvm-all.c
index 507fa7204e062..924b4a0bec21c 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -305,6 +305,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
+    if (memory_region_has_incoherent_cache(mr)) {
+        flags |= KVM_MEM_INCOHERENT;
+    }
     return flags;
 }
 
-- 
1.8.3.1

[RFC PATCH 4/6] kvm-all: set KVM_MEM_INCOHERENT

[Andrew Jones]

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 kvm-all.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/kvm-all.c b/kvm-all.c
index 507fa7204e062..924b4a0bec21c 100644
--- a/kvm-all.c
+++ b/kvm-all.c
@@ -305,6 +305,9 @@ static int kvm_mem_flags(MemoryRegion *mr)
     if (readonly && kvm_readonly_mem_allowed) {
         flags |= KVM_MEM_READONLY;
     }
+    if (memory_region_has_incoherent_cache(mr)) {
+        flags |= KVM_MEM_INCOHERENT;
+    }
     return flags;
 }
 
-- 
1.8.3.1

[Qemu-devel] [RFC PATCH 5/6] vga: flag vram as incoherent

[Andrew Jones]

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 hw/display/vga.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/display/vga.c b/hw/display/vga.c
index c8c49abc6e8ba..2b74eb8e96462 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -2135,6 +2135,7 @@ void vga_common_init(VGACommonState *s, Object *obj, bool global_vmstate)
     s->is_vbe_vmstate = 1;
     memory_region_init_ram(&s->vram, obj, "vga.vram", s->vram_size,
                            &error_abort);
+    memory_region_set_incoherent_cache(&s->vram);
     vmstate_register_ram(&s->vram, global_vmstate ? NULL : DEVICE(obj));
     xen_register_framebuffer(&s->vram);
     s->vram_ptr = memory_region_get_ram_ptr(&s->vram);
-- 
1.8.3.1

[RFC PATCH 5/6] vga: flag vram as incoherent

[Andrew Jones]

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
 hw/display/vga.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/display/vga.c b/hw/display/vga.c
index c8c49abc6e8ba..2b74eb8e96462 100644
--- a/hw/display/vga.c
+++ b/hw/display/vga.c
@@ -2135,6 +2135,7 @@ void vga_common_init(VGACommonState *s, Object *obj, bool global_vmstate)
     s->is_vbe_vmstate = 1;
     memory_region_init_ram(&s->vram, obj, "vga.vram", s->vram_size,
                            &error_abort);
+    memory_region_set_incoherent_cache(&s->vram);
     vmstate_register_ram(&s->vram, global_vmstate ? NULL : DEVICE(obj));
     xen_register_framebuffer(&s->vram);
     s->vram_ptr = memory_region_get_ram_ptr(&s->vram);
-- 
1.8.3.1

[Qemu-devel] [RFC/WIP PATCH 6/6] memory: add clear_cache_to_poc

[Andrew Jones]

Add a function that flushes the cache to PoC. We need a new
function because __builtin___clear_cache only flushes to
PoU. Call this function each time an address in a memory
region that has been flagged as having an incoherent cache
is written. For starters we only implement it for ARM. Most
other architectures don't need it anyway.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
Currently only implemented for aarch64, doesn't completely work yet.

 exec.c                  | 16 ++++++++++------
 include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index c85321a38ba69..68268a5961ff5 100644
--- a/exec.c
+++ b/exec.c
@@ -2261,7 +2261,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
 
 #else
 
-static void invalidate_and_set_dirty(hwaddr addr,
+static void invalidate_and_set_dirty(MemoryRegion *mr, hwaddr addr,
                                      hwaddr length)
 {
     if (cpu_physical_memory_range_includes_clean(addr, length)) {
@@ -2269,6 +2269,10 @@ static void invalidate_and_set_dirty(hwaddr addr,
         cpu_physical_memory_set_dirty_range_nocode(addr, length);
     }
     xen_modified_memory(addr, length);
+    if (memory_region_has_incoherent_cache(mr)) {
+        char *start = qemu_get_ram_ptr(addr);
+        clear_cache_to_poc(start, start + length);
+    }
 }
 
 static int memory_access_size(MemoryRegion *mr, unsigned l, hwaddr addr)
@@ -2348,7 +2352,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,
                 /* RAM case */
                 ptr = qemu_get_ram_ptr(addr1);
                 memcpy(ptr, buf, l);
-                invalidate_and_set_dirty(addr1, l);
+                invalidate_and_set_dirty(mr, addr1, l);
             }
         } else {
             if (!memory_access_is_direct(mr, is_write)) {
@@ -2437,7 +2441,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as,
             switch (type) {
             case WRITE_DATA:
                 memcpy(ptr, buf, l);
-                invalidate_and_set_dirty(addr1, l);
+                invalidate_and_set_dirty(mr, addr1, l);
                 break;
             case FLUSH_CACHE:
                 flush_icache_range((uintptr_t)ptr, (uintptr_t)ptr + l);
@@ -2622,7 +2626,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
         mr = qemu_ram_addr_from_host(buffer, &addr1);
         assert(mr != NULL);
         if (is_write) {
-            invalidate_and_set_dirty(addr1, access_len);
+            invalidate_and_set_dirty(mr, addr1, access_len);
         }
         if (xen_enabled()) {
             xen_invalidate_map_cache_entry(buffer);
@@ -2904,7 +2908,7 @@ static inline void stl_phys_internal(AddressSpace *as,
             stl_p(ptr, val);
             break;
         }
-        invalidate_and_set_dirty(addr1, 4);
+        invalidate_and_set_dirty(mr, addr1, 4);
     }
 }
 
@@ -2967,7 +2971,7 @@ static inline void stw_phys_internal(AddressSpace *as,
             stw_p(ptr, val);
             break;
         }
-        invalidate_and_set_dirty(addr1, 2);
+        invalidate_and_set_dirty(mr, addr1, 2);
     }
 }
 
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 8eb0db3910e86..9bf74e791f357 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -106,6 +106,43 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr,
                   hwaddr paddr, int prot,
                   int mmu_idx, target_ulong size);
 void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr);
+#if defined(__aarch64__)
+static inline void clear_cache_to_poc(char *begin, char *end)
+{
+    /* Unfortunately __builtin___clear_cache only flushes
+     * to PoU, we need to implement this for PoC.
+     */
+    static unsigned long line_sz = 0;
+    unsigned long start, stop, addr;
+
+    if (!line_sz) {
+        unsigned int ctr_el0;
+        asm volatile("mrs %0, ctr_el0" : "=&r" (ctr_el0));
+        line_sz = (1UL << ((ctr_el0 >> 16) & 0xf)) * sizeof(int);
+    }
+
+    start = (unsigned long)begin & ~(line_sz - 1);
+    stop = ((unsigned long)(end + line_sz) & ~(line_sz - 1));
+
+    for (addr = start; addr < stop; addr += line_sz) {
+        asm volatile("dc cvac, %0" : : "r" (addr));
+    }
+
+    /* FIXME: Ideally, we'd also flush the icache now, just in
+     * case this is for an executable region. But, AArch64 can't
+     * flush it to PoC from userspace. We need a syscall.
+     */
+}
+#elif defined(__arm__)
+static inline void clear_cache_to_poc(char *begin, char *end)
+{
+/* TODO */
+}
+#else
+static inline void clear_cache_to_poc(char *begin, char *end)
+{
+}
+#endif
 #else
 static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
 {
@@ -114,6 +151,10 @@ static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
 static inline void tlb_flush(CPUState *cpu, int flush_global)
 {
 }
+
+void clear_cache_to_poc(char *begin, char *end)
+{
+}
 #endif
 
 #define CODE_GEN_ALIGN           16 /* must be >= of the size of a icache line */
-- 
1.8.3.1

[RFC/WIP PATCH 6/6] memory: add clear_cache_to_poc

[Andrew Jones]

Add a function that flushes the cache to PoC. We need a new
function because __builtin___clear_cache only flushes to
PoU. Call this function each time an address in a memory
region that has been flagged as having an incoherent cache
is written. For starters we only implement it for ARM. Most
other architectures don't need it anyway.

Signed-off-by: Andrew Jones <drjones@redhat.com>
---
Currently only implemented for aarch64, doesn't completely work yet.

 exec.c                  | 16 ++++++++++------
 include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 51 insertions(+), 6 deletions(-)

diff --git a/exec.c b/exec.c
index c85321a38ba69..68268a5961ff5 100644
--- a/exec.c
+++ b/exec.c
@@ -2261,7 +2261,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
 
 #else
 
-static void invalidate_and_set_dirty(hwaddr addr,
+static void invalidate_and_set_dirty(MemoryRegion *mr, hwaddr addr,
                                      hwaddr length)
 {
     if (cpu_physical_memory_range_includes_clean(addr, length)) {
@@ -2269,6 +2269,10 @@ static void invalidate_and_set_dirty(hwaddr addr,
         cpu_physical_memory_set_dirty_range_nocode(addr, length);
     }
     xen_modified_memory(addr, length);
+    if (memory_region_has_incoherent_cache(mr)) {
+        char *start = qemu_get_ram_ptr(addr);
+        clear_cache_to_poc(start, start + length);
+    }
 }
 
 static int memory_access_size(MemoryRegion *mr, unsigned l, hwaddr addr)
@@ -2348,7 +2352,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,
                 /* RAM case */
                 ptr = qemu_get_ram_ptr(addr1);
                 memcpy(ptr, buf, l);
-                invalidate_and_set_dirty(addr1, l);
+                invalidate_and_set_dirty(mr, addr1, l);
             }
         } else {
             if (!memory_access_is_direct(mr, is_write)) {
@@ -2437,7 +2441,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as,
             switch (type) {
             case WRITE_DATA:
                 memcpy(ptr, buf, l);
-                invalidate_and_set_dirty(addr1, l);
+                invalidate_and_set_dirty(mr, addr1, l);
                 break;
             case FLUSH_CACHE:
                 flush_icache_range((uintptr_t)ptr, (uintptr_t)ptr + l);
@@ -2622,7 +2626,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
         mr = qemu_ram_addr_from_host(buffer, &addr1);
         assert(mr != NULL);
         if (is_write) {
-            invalidate_and_set_dirty(addr1, access_len);
+            invalidate_and_set_dirty(mr, addr1, access_len);
         }
         if (xen_enabled()) {
             xen_invalidate_map_cache_entry(buffer);
@@ -2904,7 +2908,7 @@ static inline void stl_phys_internal(AddressSpace *as,
             stl_p(ptr, val);
             break;
         }
-        invalidate_and_set_dirty(addr1, 4);
+        invalidate_and_set_dirty(mr, addr1, 4);
     }
 }
 
@@ -2967,7 +2971,7 @@ static inline void stw_phys_internal(AddressSpace *as,
             stw_p(ptr, val);
             break;
         }
-        invalidate_and_set_dirty(addr1, 2);
+        invalidate_and_set_dirty(mr, addr1, 2);
     }
 }
 
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 8eb0db3910e86..9bf74e791f357 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -106,6 +106,43 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr,
                   hwaddr paddr, int prot,
                   int mmu_idx, target_ulong size);
 void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr);
+#if defined(__aarch64__)
+static inline void clear_cache_to_poc(char *begin, char *end)
+{
+    /* Unfortunately __builtin___clear_cache only flushes
+     * to PoU, we need to implement this for PoC.
+     */
+    static unsigned long line_sz = 0;
+    unsigned long start, stop, addr;
+
+    if (!line_sz) {
+        unsigned int ctr_el0;
+        asm volatile("mrs %0, ctr_el0" : "=&r" (ctr_el0));
+        line_sz = (1UL << ((ctr_el0 >> 16) & 0xf)) * sizeof(int);
+    }
+
+    start = (unsigned long)begin & ~(line_sz - 1);
+    stop = ((unsigned long)(end + line_sz) & ~(line_sz - 1));
+
+    for (addr = start; addr < stop; addr += line_sz) {
+        asm volatile("dc cvac, %0" : : "r" (addr));
+    }
+
+    /* FIXME: Ideally, we'd also flush the icache now, just in
+     * case this is for an executable region. But, AArch64 can't
+     * flush it to PoC from userspace. We need a syscall.
+     */
+}
+#elif defined(__arm__)
+static inline void clear_cache_to_poc(char *begin, char *end)
+{
+/* TODO */
+}
+#else
+static inline void clear_cache_to_poc(char *begin, char *end)
+{
+}
+#endif
 #else
 static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
 {
@@ -114,6 +151,10 @@ static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
 static inline void tlb_flush(CPUState *cpu, int flush_global)
 {
 }
+
+void clear_cache_to_poc(char *begin, char *end)
+{
+}
 #endif
 
 #define CODE_GEN_ALIGN           16 /* must be >= of the size of a icache line */
-- 
1.8.3.1

Re: [Qemu-devel] [RFC/WIP PATCH 6/6] memory: add clear_cache_to_poc

[Andrew Jones]

On Fri, Mar 06, 2015 at 01:53:38PM -0500, Andrew Jones wrote:
> Add a function that flushes the cache to PoC. We need a new
> function because __builtin___clear_cache only flushes to
> PoU. Call this function each time an address in a memory
> region that has been flagged as having an incoherent cache
> is written. For starters we only implement it for ARM. Most
> other architectures don't need it anyway.

I started looking for my missing flushes, and see I have stupidity
in this patch. I'm not flushing in the right place at all... My
testing was just [un]lucky, making me think it was on the right
track. I'll send an update to this tomorrow after I remove my head
from a dark hole near my chair.

drew


> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
> Currently only implemented for aarch64, doesn't completely work yet.
> 
>  exec.c                  | 16 ++++++++++------
>  include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 51 insertions(+), 6 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index c85321a38ba69..68268a5961ff5 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -2261,7 +2261,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
>  
>  #else
>  
> -static void invalidate_and_set_dirty(hwaddr addr,
> +static void invalidate_and_set_dirty(MemoryRegion *mr, hwaddr addr,
>                                       hwaddr length)
>  {
>      if (cpu_physical_memory_range_includes_clean(addr, length)) {
> @@ -2269,6 +2269,10 @@ static void invalidate_and_set_dirty(hwaddr addr,
>          cpu_physical_memory_set_dirty_range_nocode(addr, length);
>      }
>      xen_modified_memory(addr, length);
> +    if (memory_region_has_incoherent_cache(mr)) {
> +        char *start = qemu_get_ram_ptr(addr);
> +        clear_cache_to_poc(start, start + length);
> +    }
>  }
>  
>  static int memory_access_size(MemoryRegion *mr, unsigned l, hwaddr addr)
> @@ -2348,7 +2352,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,
>                  /* RAM case */
>                  ptr = qemu_get_ram_ptr(addr1);
>                  memcpy(ptr, buf, l);
> -                invalidate_and_set_dirty(addr1, l);
> +                invalidate_and_set_dirty(mr, addr1, l);
>              }
>          } else {
>              if (!memory_access_is_direct(mr, is_write)) {
> @@ -2437,7 +2441,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as,
>              switch (type) {
>              case WRITE_DATA:
>                  memcpy(ptr, buf, l);
> -                invalidate_and_set_dirty(addr1, l);
> +                invalidate_and_set_dirty(mr, addr1, l);
>                  break;
>              case FLUSH_CACHE:
>                  flush_icache_range((uintptr_t)ptr, (uintptr_t)ptr + l);
> @@ -2622,7 +2626,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
>          mr = qemu_ram_addr_from_host(buffer, &addr1);
>          assert(mr != NULL);
>          if (is_write) {
> -            invalidate_and_set_dirty(addr1, access_len);
> +            invalidate_and_set_dirty(mr, addr1, access_len);
>          }
>          if (xen_enabled()) {
>              xen_invalidate_map_cache_entry(buffer);
> @@ -2904,7 +2908,7 @@ static inline void stl_phys_internal(AddressSpace *as,
>              stl_p(ptr, val);
>              break;
>          }
> -        invalidate_and_set_dirty(addr1, 4);
> +        invalidate_and_set_dirty(mr, addr1, 4);
>      }
>  }
>  
> @@ -2967,7 +2971,7 @@ static inline void stw_phys_internal(AddressSpace *as,
>              stw_p(ptr, val);
>              break;
>          }
> -        invalidate_and_set_dirty(addr1, 2);
> +        invalidate_and_set_dirty(mr, addr1, 2);
>      }
>  }
>  
> diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
> index 8eb0db3910e86..9bf74e791f357 100644
> --- a/include/exec/exec-all.h
> +++ b/include/exec/exec-all.h
> @@ -106,6 +106,43 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr,
>                    hwaddr paddr, int prot,
>                    int mmu_idx, target_ulong size);
>  void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr);
> +#if defined(__aarch64__)
> +static inline void clear_cache_to_poc(char *begin, char *end)
> +{
> +    /* Unfortunately __builtin___clear_cache only flushes
> +     * to PoU, we need to implement this for PoC.
> +     */
> +    static unsigned long line_sz = 0;
> +    unsigned long start, stop, addr;
> +
> +    if (!line_sz) {
> +        unsigned int ctr_el0;
> +        asm volatile("mrs %0, ctr_el0" : "=&r" (ctr_el0));
> +        line_sz = (1UL << ((ctr_el0 >> 16) & 0xf)) * sizeof(int);
> +    }
> +
> +    start = (unsigned long)begin & ~(line_sz - 1);
> +    stop = ((unsigned long)(end + line_sz) & ~(line_sz - 1));
> +
> +    for (addr = start; addr < stop; addr += line_sz) {
> +        asm volatile("dc cvac, %0" : : "r" (addr));
> +    }
> +
> +    /* FIXME: Ideally, we'd also flush the icache now, just in
> +     * case this is for an executable region. But, AArch64 can't
> +     * flush it to PoC from userspace. We need a syscall.
> +     */
> +}
> +#elif defined(__arm__)
> +static inline void clear_cache_to_poc(char *begin, char *end)
> +{
> +/* TODO */
> +}
> +#else
> +static inline void clear_cache_to_poc(char *begin, char *end)
> +{
> +}
> +#endif
>  #else
>  static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
>  {
> @@ -114,6 +151,10 @@ static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
>  static inline void tlb_flush(CPUState *cpu, int flush_global)
>  {
>  }
> +
> +void clear_cache_to_poc(char *begin, char *end)
> +{
> +}
>  #endif
>  
>  #define CODE_GEN_ALIGN           16 /* must be >= of the size of a icache line */
> -- 
> 1.8.3.1
> 
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: [RFC/WIP PATCH 6/6] memory: add clear_cache_to_poc

[Andrew Jones]

On Fri, Mar 06, 2015 at 01:53:38PM -0500, Andrew Jones wrote:
> Add a function that flushes the cache to PoC. We need a new
> function because __builtin___clear_cache only flushes to
> PoU. Call this function each time an address in a memory
> region that has been flagged as having an incoherent cache
> is written. For starters we only implement it for ARM. Most
> other architectures don't need it anyway.

I started looking for my missing flushes, and see I have stupidity
in this patch. I'm not flushing in the right place at all... My
testing was just [un]lucky, making me think it was on the right
track. I'll send an update to this tomorrow after I remove my head
from a dark hole near my chair.

drew


> 
> Signed-off-by: Andrew Jones <drjones@redhat.com>
> ---
> Currently only implemented for aarch64, doesn't completely work yet.
> 
>  exec.c                  | 16 ++++++++++------
>  include/exec/exec-all.h | 41 +++++++++++++++++++++++++++++++++++++++++
>  2 files changed, 51 insertions(+), 6 deletions(-)
> 
> diff --git a/exec.c b/exec.c
> index c85321a38ba69..68268a5961ff5 100644
> --- a/exec.c
> +++ b/exec.c
> @@ -2261,7 +2261,7 @@ int cpu_memory_rw_debug(CPUState *cpu, target_ulong addr,
>  
>  #else
>  
> -static void invalidate_and_set_dirty(hwaddr addr,
> +static void invalidate_and_set_dirty(MemoryRegion *mr, hwaddr addr,
>                                       hwaddr length)
>  {
>      if (cpu_physical_memory_range_includes_clean(addr, length)) {
> @@ -2269,6 +2269,10 @@ static void invalidate_and_set_dirty(hwaddr addr,
>          cpu_physical_memory_set_dirty_range_nocode(addr, length);
>      }
>      xen_modified_memory(addr, length);
> +    if (memory_region_has_incoherent_cache(mr)) {
> +        char *start = qemu_get_ram_ptr(addr);
> +        clear_cache_to_poc(start, start + length);
> +    }
>  }
>  
>  static int memory_access_size(MemoryRegion *mr, unsigned l, hwaddr addr)
> @@ -2348,7 +2352,7 @@ bool address_space_rw(AddressSpace *as, hwaddr addr, uint8_t *buf,
>                  /* RAM case */
>                  ptr = qemu_get_ram_ptr(addr1);
>                  memcpy(ptr, buf, l);
> -                invalidate_and_set_dirty(addr1, l);
> +                invalidate_and_set_dirty(mr, addr1, l);
>              }
>          } else {
>              if (!memory_access_is_direct(mr, is_write)) {
> @@ -2437,7 +2441,7 @@ static inline void cpu_physical_memory_write_rom_internal(AddressSpace *as,
>              switch (type) {
>              case WRITE_DATA:
>                  memcpy(ptr, buf, l);
> -                invalidate_and_set_dirty(addr1, l);
> +                invalidate_and_set_dirty(mr, addr1, l);
>                  break;
>              case FLUSH_CACHE:
>                  flush_icache_range((uintptr_t)ptr, (uintptr_t)ptr + l);
> @@ -2622,7 +2626,7 @@ void address_space_unmap(AddressSpace *as, void *buffer, hwaddr len,
>          mr = qemu_ram_addr_from_host(buffer, &addr1);
>          assert(mr != NULL);
>          if (is_write) {
> -            invalidate_and_set_dirty(addr1, access_len);
> +            invalidate_and_set_dirty(mr, addr1, access_len);
>          }
>          if (xen_enabled()) {
>              xen_invalidate_map_cache_entry(buffer);
> @@ -2904,7 +2908,7 @@ static inline void stl_phys_internal(AddressSpace *as,
>              stl_p(ptr, val);
>              break;
>          }
> -        invalidate_and_set_dirty(addr1, 4);
> +        invalidate_and_set_dirty(mr, addr1, 4);
>      }
>  }
>  
> @@ -2967,7 +2971,7 @@ static inline void stw_phys_internal(AddressSpace *as,
>              stw_p(ptr, val);
>              break;
>          }
> -        invalidate_and_set_dirty(addr1, 2);
> +        invalidate_and_set_dirty(mr, addr1, 2);
>      }
>  }
>  
> diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
> index 8eb0db3910e86..9bf74e791f357 100644
> --- a/include/exec/exec-all.h
> +++ b/include/exec/exec-all.h
> @@ -106,6 +106,43 @@ void tlb_set_page(CPUState *cpu, target_ulong vaddr,
>                    hwaddr paddr, int prot,
>                    int mmu_idx, target_ulong size);
>  void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr);
> +#if defined(__aarch64__)
> +static inline void clear_cache_to_poc(char *begin, char *end)
> +{
> +    /* Unfortunately __builtin___clear_cache only flushes
> +     * to PoU, we need to implement this for PoC.
> +     */
> +    static unsigned long line_sz = 0;
> +    unsigned long start, stop, addr;
> +
> +    if (!line_sz) {
> +        unsigned int ctr_el0;
> +        asm volatile("mrs %0, ctr_el0" : "=&r" (ctr_el0));
> +        line_sz = (1UL << ((ctr_el0 >> 16) & 0xf)) * sizeof(int);
> +    }
> +
> +    start = (unsigned long)begin & ~(line_sz - 1);
> +    stop = ((unsigned long)(end + line_sz) & ~(line_sz - 1));
> +
> +    for (addr = start; addr < stop; addr += line_sz) {
> +        asm volatile("dc cvac, %0" : : "r" (addr));
> +    }
> +
> +    /* FIXME: Ideally, we'd also flush the icache now, just in
> +     * case this is for an executable region. But, AArch64 can't
> +     * flush it to PoC from userspace. We need a syscall.
> +     */
> +}
> +#elif defined(__arm__)
> +static inline void clear_cache_to_poc(char *begin, char *end)
> +{
> +/* TODO */
> +}
> +#else
> +static inline void clear_cache_to_poc(char *begin, char *end)
> +{
> +}
> +#endif
>  #else
>  static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
>  {
> @@ -114,6 +151,10 @@ static inline void tlb_flush_page(CPUState *cpu, target_ulong addr)
>  static inline void tlb_flush(CPUState *cpu, int flush_global)
>  {
>  }
> +
> +void clear_cache_to_poc(char *begin, char *end)
> +{
> +}
>  #endif
>  
>  #define CODE_GEN_ALIGN           16 /* must be >= of the size of a icache line */
> -- 
> 1.8.3.1
> 
> _______________________________________________
> kvmarm mailing list
> kvmarm@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/mailman/listinfo/kvmarm

Re: [Qemu-devel] the arm cache coherency cluster

[Andrew Jones]

On Fri, Mar 06, 2015 at 01:49:40PM -0500, Andrew Jones wrote:
> In reply to this message I'll send two series' one for KVM and
> one for QEMU. The two series' are their respective component
> complements, and attempt to implement cache coherency for arm
> guests using emulated devices, where the emulator (qemu) uses
> cached memory for the device memory, but the guest uses
> uncached - as device memory is generally used. Right now I've
> just focused on VGA vram.
> 
> This approach starts as the "add a new memslot flag" approach,
> and then turns into the "make qemu do some cache maintenance"
> approach with the final patch of each series (6/6). It stops
> short of the "add syscalls..." approach. Below is a summary of
> all the approaches discussed so far, to my knowledge.
> 
> "MAIR manipulating"
> Posted[1] by Ard. Works. No performance degradation. Potential
> issues with device assignment and the guest getting confused.
> 
> "add a new memslot flag"
> This posting (not counting patches 6/6). Works. Huge performance
> degradation.
> 
> "make qemu do some cache maintenance"
> This posting (patches 6/6). We can only do so much in qemu
> without syscalls. This series does what it can. Almost works,
> probably could work, after playing 'find the missing flush'.
> This approach still requires the new memslot flag, as userspace
> can't invalidate the cache, only clean, or clean+invalidate.
> No noticeable performance degradation.
> 
> "add syscalls to make qemu do all cache maintenance"
> Variant 1: implement as kvm ioctls - to avoid trying to get
>            syscalls into the general kernel
> Variant 2: add real syscalls, or maybe just ARM private SWIs
>            like __ARM_NR_cacheflush
> This approach should work, and if we add an invalidate syscall,
> then we shouldn't need any kvm changes at all, i.e. no need for
> the memslot flag. I haven't experimented with this yet, but I'm
> starting to like the idea of variant 2, with a private SWI, so
> will try to pull something together soon for that.
> 
> "describe the problematic memory as cached to the guest"
> Not an ideal solution for virt. Could maybe be workable as a
> quirk for a specific device though.
> 
> re: $SUBJECT; Here 'cluster' is defined by the urban dictionary.
> 
> [1] http://thread.gmane.org/gmane.comp.emulators.kvm.arm.devel/34/

I'm going to send another pair of series'. A "v2", as, IMO, the
new approach supersedes the pair implemented here. After poking
around in qemu, looking for the best places to do cache maintenance,
I decided I didn't really like doing it there at all, and opted to
try another approach, that I'd forgotten to mention in this mail.
That approach is the "MADV_UNCACHED" type that Paolo suggested.
This type of approach could also be described as "make userspace's
memory access type match the expected access type of the guest",
and Mario has suggested using a memory driver, which could have
the same result.

The series I'll send is inspired by both Paolo's and Mario's
suggestions, but it uses a kvm memslot flag, rather than an
madvise flag, and thus for the memory driver, it's just KVM.

drew

Re: the arm cache coherency cluster

[Andrew Jones]

On Fri, Mar 06, 2015 at 01:49:40PM -0500, Andrew Jones wrote:
> In reply to this message I'll send two series' one for KVM and
> one for QEMU. The two series' are their respective component
> complements, and attempt to implement cache coherency for arm
> guests using emulated devices, where the emulator (qemu) uses
> cached memory for the device memory, but the guest uses
> uncached - as device memory is generally used. Right now I've
> just focused on VGA vram.
> 
> This approach starts as the "add a new memslot flag" approach,
> and then turns into the "make qemu do some cache maintenance"
> approach with the final patch of each series (6/6). It stops
> short of the "add syscalls..." approach. Below is a summary of
> all the approaches discussed so far, to my knowledge.
> 
> "MAIR manipulating"
> Posted[1] by Ard. Works. No performance degradation. Potential
> issues with device assignment and the guest getting confused.
> 
> "add a new memslot flag"
> This posting (not counting patches 6/6). Works. Huge performance
> degradation.
> 
> "make qemu do some cache maintenance"
> This posting (patches 6/6). We can only do so much in qemu
> without syscalls. This series does what it can. Almost works,
> probably could work, after playing 'find the missing flush'.
> This approach still requires the new memslot flag, as userspace
> can't invalidate the cache, only clean, or clean+invalidate.
> No noticeable performance degradation.
> 
> "add syscalls to make qemu do all cache maintenance"
> Variant 1: implement as kvm ioctls - to avoid trying to get
>            syscalls into the general kernel
> Variant 2: add real syscalls, or maybe just ARM private SWIs
>            like __ARM_NR_cacheflush
> This approach should work, and if we add an invalidate syscall,
> then we shouldn't need any kvm changes at all, i.e. no need for
> the memslot flag. I haven't experimented with this yet, but I'm
> starting to like the idea of variant 2, with a private SWI, so
> will try to pull something together soon for that.
> 
> "describe the problematic memory as cached to the guest"
> Not an ideal solution for virt. Could maybe be workable as a
> quirk for a specific device though.
> 
> re: $SUBJECT; Here 'cluster' is defined by the urban dictionary.
> 
> [1] http://thread.gmane.org/gmane.comp.emulators.kvm.arm.devel/34/

I'm going to send another pair of series'. A "v2", as, IMO, the
new approach supersedes the pair implemented here. After poking
around in qemu, looking for the best places to do cache maintenance,
I decided I didn't really like doing it there at all, and opted to
try another approach, that I'd forgotten to mention in this mail.
That approach is the "MADV_UNCACHED" type that Paolo suggested.
This type of approach could also be described as "make userspace's
memory access type match the expected access type of the guest",
and Mario has suggested using a memory driver, which could have
the same result.

The series I'll send is inspired by both Paolo's and Mario's
suggestions, but it uses a kvm memslot flag, rather than an
madvise flag, and thus for the memory driver, it's just KVM.

drew