[PATCH v4 00/36] KVM: s390: Add support for protected VMs

[PATCH v4 00/36] KVM: s390: Add support for protected VMs

[Christian Borntraeger]

mm-related patches CCed on linux-mm, the complete list can be found on
the KVM and linux-s390 list. 

Andrew, any chance to take " mm:gup/writeback: add callbacks for
inaccessible pages" for 5.7? I can then carry the s390/kvm part. There
is no build dependency on this patch (just a logical one).  As an
alternative I can take an ack and carry that patch myself. 

This series contains a "pretty small" common code memory management
change that will allow paging, guest backing with files etc almost
just like normal VMs. It should be a no-op for all architectures not
opting in. And it should be usable for others that also try to get
notified on "the pages are in the process of being used for things
like I/O". This time I included error handling and an ACK from Will
Deacon as well as a Reviewed-by: from David Hildenbrand.
This patch will be used by
"[PATCH v4 05/36] s390/mm: provide memory management functions for
protected KVM guests".
We need to call into the "make accessible" architecture function when
the refcount is already increased the writeback bit is set. This will
make sure that we do not call the reverse function (convert to secure)
until the host operation has finished.


Overview
--------
Protected VMs (PVM) are KVM VMs, where KVM can't access the VM's state
like guest memory and guest registers anymore. Instead the PVMs are
mostly managed by a new entity called Ultravisor (UV), which provides
an API, so KVM and the PV can request management actions.

PVMs are encrypted at rest and protected from hypervisor access while
running. They switch from a normal operation into protected mode, so
we can still use the standard boot process to load a encrypted blob
and then move it into protected mode.

Rebooting is only possible by passing through the unprotected/normal
mode and switching to protected again.

All patches are in the protvirtv4 branch of the korg s390 kvm git
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/log/?h=protvirtv6

Claudio presented the technology at his presentation at KVM Forum
2019.

https://static.sched.com/hosted_files/kvmforum2019/3b/ibm_protected_vms_s390x.pdf


v3 -> v4:
general
-------
- copyright updates
- Reviewedby + acked by tags

KVM: s390/interrupt: do not pin adapter interrupt pages
-------------------------------------------------------
- more comments
- get rid of now obsolete adapter parameter

s390/mm: provide memory management functions for protected KVM guests
---------------------------------------------------------------------
- improved patch description

KVM: s390: protvirt: Add initial vm and cpu lifecycle handling
--------------------------------------------------------------
- rework tweak logic to not use an array
- remove _VM_ part of the subfunction names of PV_COMMAND
- merge alloc/create/destroy/dealloc into init/deinit
- handle cmma deallocation on failures
- rework error handling to pass along the first rc/rrc if VCPU or VM CREATE/DESTROY fails
This was tested successfully with error injection and tracing. We do not deallocate on
destroy failure and we pass along the first rc/rrc when vcpu destroy fails.

 KVM: s390: protvirt: Add KVM api documentation
-----------------------------------------------
- mention new MP_STATE
- remove "old" interfaces that are no longer in the previous patch
- move to the end

KVM: s390: protvirt: Secure memory is not mergeable
---------------------------------------------------
- rebase on new lifecycle patch

KVM: s390: protvirt: UV calls in support of diag308 0,1
-------------------------------------------------------
- remove _VM_ part of the subfunction names of PV_COMMAND

KVM: s390: rstify new ioctls in api.rst
---------------------------------------
- removed from this patch queue



v2 -> v3
- rebase against v5.6-rc2
- move some checks into the callers
- typo fixes
- extend UV query size
- do a tlb flush when entering/exiting protected mode
- more comments
- change interface to PV_ENABLE/DISABLE instead of vcpu/vm
  create/destroy
- lockdep checks for *is_protected calls
- locking improments
- move facility 161 to qemu
- checkpatch fixes
- merged error handling in mm patch
- removed vcpu pv commands
- use mp_state for setting the IPL PSW


v1 -> v2
- rebase on top of kvm/master
- pipe through rc and rrc. This might have created some churn here and
  there
- turn off sclp masking when rebooting into "unsecure"
- memory management simplification
- prefix page handling now via intercept 112
- io interrupt intervention request fix (do not use GISA)
- api.txt conversion to rst
- sample patches on top of mm/gup/writeback
- tons of review feedback
- kvm_uv debug feature fixes and unifications
- ultravisor information for /sys/firmware
- 

RFCv2 -> v1 (you can diff the protvirtv2 and the protvirtv3 branch)
- tons of review feedback integrated (see mail thread)
- memory management now complete and working
- Documentation patches merged
- interrupt patches merged
- CONFIG_KVM_S390_PROTECTED_VIRTUALIZATION_HOST removed
- SIDA interface integrated into memop
- for merged patches I removed reviews that were not in all patches



Christian Borntraeger (4):
  KVM: s390/mm: Make pages accessible before destroying the guest
  KVM: s390: protvirt: Add SCLP interrupt handling
  KVM: s390: protvirt: do not inject interrupts after start
  KVM: s390: protvirt: introduce and enable KVM_CAP_S390_PROTECTED

Claudio Imbrenda (3):
  mm/gup/writeback: add callbacks for inaccessible pages
  s390/mm: provide memory management functions for protected KVM guests
  KVM: s390/mm: handle guest unpin events

Janosch Frank (24):
  KVM: s390: protvirt: Add UV debug trace
  KVM: s390: add new variants of UV CALL
  KVM: s390: protvirt: Add initial vm and cpu lifecycle handling
  KVM: s390: protvirt: Secure memory is not mergeable
  KVM: s390: protvirt: Handle SE notification interceptions
  KVM: s390: protvirt: Instruction emulation
  KVM: s390: protvirt: Handle spec exception loops
  KVM: s390: protvirt: Add new gprs location handling
  KVM: S390: protvirt: Introduce instruction data area bounce buffer
  KVM: s390: protvirt: handle secure guest prefix pages
  KVM: s390: protvirt: Write sthyi data to instruction data area
  KVM: s390: protvirt: STSI handling
  KVM: s390: protvirt: disallow one_reg
  KVM: s390: protvirt: Do only reset registers that are accessible
  KVM: s390: protvirt: Only sync fmt4 registers
  KVM: s390: protvirt: Add program exception injection
  KVM: s390: protvirt: UV calls in support of diag308 0, 1
  KVM: s390: protvirt: Report CPU state to Ultravisor
  KVM: s390: protvirt: Support cmd 5 operation state
  KVM: s390: protvirt: Mask PSW interrupt bits for interception 104 and
    112
  KVM: s390: protvirt: Add UV cpu reset calls
  DOCUMENTATION: Protected virtual machine introduction and IPL
  s390: protvirt: Add sysfs firmware interface for Ultravisor
    information
  KVM: s390: protvirt: Add KVM api documentation

Michael Mueller (1):
  KVM: s390: protvirt: Implement interrupt injection

Ulrich Weigand (1):
  KVM: s390/interrupt: do not pin adapter interrupt pages

Vasily Gorbik (3):
  s390/protvirt: introduce host side setup
  s390/protvirt: add ultravisor initialization
  s390/mm: add (non)secure page access exceptions handlers

 .../admin-guide/kernel-parameters.txt         |   5 +
 Documentation/virt/kvm/api.rst                |  61 +-
 Documentation/virt/kvm/devices/s390_flic.rst  |  11 +-
 Documentation/virt/kvm/index.rst              |   2 +
 Documentation/virt/kvm/s390-pv-boot.rst       |  83 +++
 Documentation/virt/kvm/s390-pv.rst            | 116 ++++
 MAINTAINERS                                   |   1 +
 arch/s390/boot/Makefile                       |   2 +-
 arch/s390/boot/uv.c                           |  21 +-
 arch/s390/include/asm/gmap.h                  |   6 +
 arch/s390/include/asm/kvm_host.h              | 113 +++-
 arch/s390/include/asm/mmu.h                   |   2 +
 arch/s390/include/asm/mmu_context.h           |   1 +
 arch/s390/include/asm/page.h                  |   5 +
 arch/s390/include/asm/pgtable.h               |  35 +-
 arch/s390/include/asm/uv.h                    | 252 ++++++++-
 arch/s390/kernel/Makefile                     |   1 +
 arch/s390/kernel/pgm_check.S                  |   4 +-
 arch/s390/kernel/setup.c                      |   9 +-
 arch/s390/kernel/uv.c                         | 413 ++++++++++++++
 arch/s390/kvm/Makefile                        |   2 +-
 arch/s390/kvm/diag.c                          |   6 +-
 arch/s390/kvm/intercept.c                     | 117 +++-
 arch/s390/kvm/interrupt.c                     | 399 +++++++------
 arch/s390/kvm/kvm-s390.c                      | 532 +++++++++++++++---
 arch/s390/kvm/kvm-s390.h                      |  51 +-
 arch/s390/kvm/priv.c                          |  13 +-
 arch/s390/kvm/pv.c                            | 298 ++++++++++
 arch/s390/mm/fault.c                          |  78 +++
 arch/s390/mm/gmap.c                           |  65 ++-
 include/linux/gfp.h                           |   6 +
 include/uapi/linux/kvm.h                      |  43 +-
 mm/gup.c                                      |  15 +-
 mm/page-writeback.c                           |   5 +
 34 files changed, 2461 insertions(+), 312 deletions(-)
 create mode 100644 Documentation/virt/kvm/s390-pv-boot.rst
 create mode 100644 Documentation/virt/kvm/s390-pv.rst
 create mode 100644 arch/s390/kernel/uv.c
 create mode 100644 arch/s390/kvm/pv.c

-- 
2.25.0

[PATCH v4 01/36] mm/gup/writeback: add callbacks for inaccessible pages

[Christian Borntraeger]

From: Claudio Imbrenda <imbrenda@linux.ibm.com>

With the introduction of protected KVM guests on s390 there is now a
concept of inaccessible pages. These pages need to be made accessible
before the host can access them.

While cpu accesses will trigger a fault that can be resolved, I/O
accesses will just fail.  We need to add a callback into architecture
code for places that will do I/O, namely when writeback is started or
when a page reference is taken.

This is not only to enable paging, file backing etc, it is also
necessary to protect the host against a malicious user space. For
example a bad QEMU could simply start direct I/O on such protected
memory.  We do not want userspace to be able to trigger I/O errors and
thus we the logic is "whenever somebody accesses that page (gup) or
doing I/O, make sure that this page can be accessed. When the guest
tries to access that page we will wait in the page fault handler for
writeback to have finished and for the page_ref to be the expected
value.

On s390x the function is not supposed to fail, so it is ok to use a
WARN_ON on failure. If we ever need some more finegrained handling
we can tackle this when we know the details.

Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Acked-by: Will Deacon <will@kernel.org>
Reviewed-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 include/linux/gfp.h |  6 ++++++
 mm/gup.c            | 15 ++++++++++++---
 mm/page-writeback.c |  5 +++++
 3 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/include/linux/gfp.h b/include/linux/gfp.h
index e5b817cb86e7..be2754841369 100644
--- a/include/linux/gfp.h
+++ b/include/linux/gfp.h
@@ -485,6 +485,12 @@ static inline void arch_free_page(struct page *page, int order) { }
 #ifndef HAVE_ARCH_ALLOC_PAGE
 static inline void arch_alloc_page(struct page *page, int order) { }
 #endif
+#ifndef HAVE_ARCH_MAKE_PAGE_ACCESSIBLE
+static inline int arch_make_page_accessible(struct page *page)
+{
+	return 0;
+}
+#endif
 
 struct page *
 __alloc_pages_nodemask(gfp_t gfp_mask, unsigned int order, int preferred_nid,
diff --git a/mm/gup.c b/mm/gup.c
index 1b521e0ac1de..354bcfbd844b 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -193,6 +193,7 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
 	struct page *page;
 	spinlock_t *ptl;
 	pte_t *ptep, pte;
+	int ret;
 
 	/* FOLL_GET and FOLL_PIN are mutually exclusive. */
 	if (WARN_ON_ONCE((flags & (FOLL_PIN | FOLL_GET)) ==
@@ -250,8 +251,6 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
 		if (is_zero_pfn(pte_pfn(pte))) {
 			page = pte_page(pte);
 		} else {
-			int ret;
-
 			ret = follow_pfn_pte(vma, address, ptep, flags);
 			page = ERR_PTR(ret);
 			goto out;
@@ -259,7 +258,6 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
 	}
 
 	if (flags & FOLL_SPLIT && PageTransCompound(page)) {
-		int ret;
 		get_page(page);
 		pte_unmap_unlock(ptep, ptl);
 		lock_page(page);
@@ -276,6 +274,12 @@ static struct page *follow_page_pte(struct vm_area_struct *vma,
 			page = ERR_PTR(-ENOMEM);
 			goto out;
 		}
+		ret = arch_make_page_accessible(page);
+		if (ret) {
+			put_page(page);
+			page = ERR_PTR(ret);
+			goto out;
+		}
 	}
 	if (flags & FOLL_TOUCH) {
 		if ((flags & FOLL_WRITE) &&
@@ -1919,6 +1923,11 @@ static int gup_pte_range(pmd_t pmd, unsigned long addr, unsigned long end,
 
 		VM_BUG_ON_PAGE(compound_head(page) != head, page);
 
+		ret = arch_make_page_accessible(page);
+		if (ret) {
+			put_page(head);
+			goto pte_unmap;
+		}
 		SetPageReferenced(page);
 		pages[*nr] = page;
 		(*nr)++;
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 2caf780a42e7..558d7063c117 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -2807,6 +2807,11 @@ int __test_set_page_writeback(struct page *page, bool keep_write)
 		inc_zone_page_state(page, NR_ZONE_WRITE_PENDING);
 	}
 	unlock_page_memcg(page);
+	/*
+	 * If writeback has been triggered on a page that cannot be made
+	 * accessible, it is too late.
+	 */
+	WARN_ON(arch_make_page_accessible(page));
 	return ret;
 
 }
-- 
2.25.0

[PATCH v4 05/36] s390/mm: provide memory management functions for protected KVM guests

[Christian Borntraeger]

From: Claudio Imbrenda <imbrenda@linux.ibm.com>

This provides the basic ultravisor calls and page table handling to cope
with secure guests:
- provide arch_make_page_accessible
- make pages accessible after unmapping of secure guests
- provide the ultravisor commands convert to/from secure
- provide the ultravisor commands pin/unpin shared
- provide callbacks to make pages secure (inacccessible)
 - we check for the expected pin count to only make pages secure if the
   host is not accessing them
 - we fence hugetlbfs for secure pages
- add missing radix-tree include into gmap.h

The basic idea is that a page can have 3 states: secure, normal or
shared. The hypervisor can call into a firmware function called
ultravisor that allows to change the state of a page: convert from/to
secure. The convert from secure will encrypt the page and make it
available to the host and host I/O. The convert to secure will remove
the host capability to access this page.
The design is that on convert to secure we will wait until writeback and
page refs are indicating no host usage. At the same time the convert
from secure (export to host) will be called in common code when the
refcount or the writeback bit is already set. This avoids races between
convert from and to secure.

Then there is also the concept of shared pages. Those are kind of secure
where the host can still access those pages. We need to be notified when
the guest "unshares" such a page, basically doing a convert to secure by
then. There is a call "pin shared page" that we use instead of convert
from secure when possible.

We do use PG_arch_1 as an optimization to minimize the convert from
secure/pin shared.

Several comments have been added in the code to explain the logic in
the relevant places.

Co-developed-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
Acked-by: David Hildenbrand <david@redhat.com>
Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
[borntraeger@de.ibm.com: patch merging, splitting, fixing]
Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
---
 arch/s390/include/asm/gmap.h        |   4 +
 arch/s390/include/asm/mmu.h         |   2 +
 arch/s390/include/asm/mmu_context.h |   1 +
 arch/s390/include/asm/page.h        |   5 +
 arch/s390/include/asm/pgtable.h     |  35 ++++-
 arch/s390/include/asm/uv.h          |  31 ++++
 arch/s390/kernel/uv.c               | 227 ++++++++++++++++++++++++++++
 7 files changed, 300 insertions(+), 5 deletions(-)

diff --git a/arch/s390/include/asm/gmap.h b/arch/s390/include/asm/gmap.h
index 37f96b6f0e61..3c4926aa78f4 100644
--- a/arch/s390/include/asm/gmap.h
+++ b/arch/s390/include/asm/gmap.h
@@ -9,6 +9,7 @@
 #ifndef _ASM_S390_GMAP_H
 #define _ASM_S390_GMAP_H
 
+#include <linux/radix-tree.h>
 #include <linux/refcount.h>
 
 /* Generic bits for GMAP notification on DAT table entry changes. */
@@ -31,6 +32,7 @@
  * @table: pointer to the page directory
  * @asce: address space control element for gmap page table
  * @pfault_enabled: defines if pfaults are applicable for the guest
+ * @guest_handle: protected virtual machine handle for the ultravisor
  * @host_to_rmap: radix tree with gmap_rmap lists
  * @children: list of shadow gmap structures
  * @pt_list: list of all page tables used in the shadow guest address space
@@ -54,6 +56,8 @@ struct gmap {
 	unsigned long asce_end;
 	void *private;
 	bool pfault_enabled;
+	/* only set for protected virtual machines */
+	unsigned long guest_handle;
 	/* Additional data for shadow guest address spaces */
 	struct radix_tree_root host_to_rmap;
 	struct list_head children;
diff --git a/arch/s390/include/asm/mmu.h b/arch/s390/include/asm/mmu.h
index bcfb6371086f..e21b618ad432 100644
--- a/arch/s390/include/asm/mmu.h
+++ b/arch/s390/include/asm/mmu.h
@@ -16,6 +16,8 @@ typedef struct {
 	unsigned long asce;
 	unsigned long asce_limit;
 	unsigned long vdso_base;
+	/* The mmu context belongs to a secure guest. */
+	atomic_t is_protected;
 	/*
 	 * The following bitfields need a down_write on the mm
 	 * semaphore when they are written to. As they are only
diff --git a/arch/s390/include/asm/mmu_context.h b/arch/s390/include/asm/mmu_context.h
index 8d04e6f3f796..afa836014076 100644
--- a/arch/s390/include/asm/mmu_context.h
+++ b/arch/s390/include/asm/mmu_context.h
@@ -23,6 +23,7 @@ static inline int init_new_context(struct task_struct *tsk,
 	INIT_LIST_HEAD(&mm->context.gmap_list);
 	cpumask_clear(&mm->context.cpu_attach_mask);
 	atomic_set(&mm->context.flush_count, 0);
+	atomic_set(&mm->context.is_protected, 0);
 	mm->context.gmap_asce = 0;
 	mm->context.flush_mm = 0;
 	mm->context.compat_mm = test_thread_flag(TIF_31BIT);
diff --git a/arch/s390/include/asm/page.h b/arch/s390/include/asm/page.h
index 85e944f04c70..4ebcf891ff3c 100644
--- a/arch/s390/include/asm/page.h
+++ b/arch/s390/include/asm/page.h
@@ -153,6 +153,11 @@ static inline int devmem_is_allowed(unsigned long pfn)
 #define HAVE_ARCH_FREE_PAGE
 #define HAVE_ARCH_ALLOC_PAGE
 
+#if IS_ENABLED(CONFIG_PGSTE)
+int arch_make_page_accessible(struct page *page);
+#define HAVE_ARCH_MAKE_PAGE_ACCESSIBLE
+#endif
+
 #endif /* !__ASSEMBLY__ */
 
 #define __PAGE_OFFSET		0x0UL
diff --git a/arch/s390/include/asm/pgtable.h b/arch/s390/include/asm/pgtable.h
index 137a3920ca36..cc7a1adacb94 100644
--- a/arch/s390/include/asm/pgtable.h
+++ b/arch/s390/include/asm/pgtable.h
@@ -19,6 +19,7 @@
 #include <linux/atomic.h>
 #include <asm/bug.h>
 #include <asm/page.h>
+#include <asm/uv.h>
 
 extern pgd_t swapper_pg_dir[];
 extern void paging_init(void);
@@ -520,6 +521,15 @@ static inline int mm_has_pgste(struct mm_struct *mm)
 	return 0;
 }
 
+static inline int mm_is_protected(struct mm_struct *mm)
+{
+#ifdef CONFIG_PGSTE
+	if (unlikely(atomic_read(&mm->context.is_protected)))
+		return 1;
+#endif
+	return 0;
+}
+
 static inline int mm_alloc_pgste(struct mm_struct *mm)
 {
 #ifdef CONFIG_PGSTE
@@ -1061,7 +1071,12 @@ static inline int ptep_clear_flush_young(struct vm_area_struct *vma,
 static inline pte_t ptep_get_and_clear(struct mm_struct *mm,
 				       unsigned long addr, pte_t *ptep)
 {
-	return ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+	pte_t res;
+
+	res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+	if (mm_is_protected(mm) && pte_present(res))
+		uv_convert_from_secure(pte_val(res) & PAGE_MASK);
+	return res;
 }
 
 #define __HAVE_ARCH_PTEP_MODIFY_PROT_TRANSACTION
@@ -1073,7 +1088,12 @@ void ptep_modify_prot_commit(struct vm_area_struct *, unsigned long,
 static inline pte_t ptep_clear_flush(struct vm_area_struct *vma,
 				     unsigned long addr, pte_t *ptep)
 {
-	return ptep_xchg_direct(vma->vm_mm, addr, ptep, __pte(_PAGE_INVALID));
+	pte_t res;
+
+	res = ptep_xchg_direct(vma->vm_mm, addr, ptep, __pte(_PAGE_INVALID));
+	if (mm_is_protected(vma->vm_mm) && pte_present(res))
+		uv_convert_from_secure(pte_val(res) & PAGE_MASK);
+	return res;
 }
 
 /*
@@ -1088,12 +1108,17 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm,
 					    unsigned long addr,
 					    pte_t *ptep, int full)
 {
+	pte_t res;
+
 	if (full) {
-		pte_t pte = *ptep;
+		res = *ptep;
 		*ptep = __pte(_PAGE_INVALID);
-		return pte;
+	} else {
+		res = ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
 	}
-	return ptep_xchg_lazy(mm, addr, ptep, __pte(_PAGE_INVALID));
+	if (mm_is_protected(mm) && pte_present(res))
+		uv_convert_from_secure(pte_val(res) & PAGE_MASK);
+	return res;
 }
 
 #define __HAVE_ARCH_PTEP_SET_WRPROTECT
diff --git a/arch/s390/include/asm/uv.h b/arch/s390/include/asm/uv.h
index cad643b05d19..7956868340c1 100644
--- a/arch/s390/include/asm/uv.h
+++ b/arch/s390/include/asm/uv.h
@@ -15,6 +15,7 @@
 #include <linux/errno.h>
 #include <linux/bug.h>
 #include <asm/page.h>
+#include <asm/gmap.h>
 
 #define UVC_RC_EXECUTED		0x0001
 #define UVC_RC_INV_CMD		0x0002
@@ -24,6 +25,10 @@
 
 #define UVC_CMD_QUI			0x0001
 #define UVC_CMD_INIT_UV			0x000f
+#define UVC_CMD_CONV_TO_SEC_STOR	0x0200
+#define UVC_CMD_CONV_FROM_SEC_STOR	0x0201
+#define UVC_CMD_PIN_PAGE_SHARED		0x0341
+#define UVC_CMD_UNPIN_PAGE_SHARED	0x0342
 #define UVC_CMD_SET_SHARED_ACCESS	0x1000
 #define UVC_CMD_REMOVE_SHARED_ACCESS	0x1001
 
@@ -31,8 +36,12 @@
 enum uv_cmds_inst {
 	BIT_UVC_CMD_QUI = 0,
 	BIT_UVC_CMD_INIT_UV = 1,
+	BIT_UVC_CMD_CONV_TO_SEC_STOR = 6,
+	BIT_UVC_CMD_CONV_FROM_SEC_STOR = 7,
 	BIT_UVC_CMD_SET_SHARED_ACCESS = 8,
 	BIT_UVC_CMD_REMOVE_SHARED_ACCESS = 9,
+	BIT_UVC_CMD_PIN_PAGE_SHARED = 21,
+	BIT_UVC_CMD_UNPIN_PAGE_SHARED = 22,
 };
 
 struct uv_cb_header {
@@ -69,6 +78,19 @@ struct uv_cb_init {
 	u64 reserved28[4];
 } __packed __aligned(8);
 
+struct uv_cb_cts {
+	struct uv_cb_header header;
+	u64 reserved08[2];
+	u64 guest_handle;
+	u64 gaddr;
+} __packed __aligned(8);
+
+struct uv_cb_cfs {
+	struct uv_cb_header header;
+	u64 reserved08[2];
+	u64 paddr;
+} __packed __aligned(8);
+
 struct uv_cb_share {
 	struct uv_cb_header header;
 	u64 reserved08[3];
@@ -171,12 +193,21 @@ static inline int is_prot_virt_host(void)
 	return prot_virt_host;
 }
 
+int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb);
+int uv_convert_from_secure(unsigned long paddr);
+int gmap_convert_to_secure(struct gmap *gmap, unsigned long gaddr);
+
 void setup_uv(void);
 void adjust_to_uv_max(unsigned long *vmax);
 #else
 #define is_prot_virt_host() 0
 static inline void setup_uv(void) {}
 static inline void adjust_to_uv_max(unsigned long *vmax) {}
+
+static inline int uv_convert_from_secure(unsigned long paddr)
+{
+	return 0;
+}
 #endif
 
 #if defined(CONFIG_PROTECTED_VIRTUALIZATION_GUEST) ||                          \
diff --git a/arch/s390/kernel/uv.c b/arch/s390/kernel/uv.c
index 1ddc42154ef6..4539003dac9d 100644
--- a/arch/s390/kernel/uv.c
+++ b/arch/s390/kernel/uv.c
@@ -12,6 +12,8 @@
 #include <linux/sizes.h>
 #include <linux/bitmap.h>
 #include <linux/memblock.h>
+#include <linux/pagemap.h>
+#include <linux/swap.h>
 #include <asm/facility.h>
 #include <asm/sections.h>
 #include <asm/uv.h>
@@ -97,4 +99,229 @@ void adjust_to_uv_max(unsigned long *vmax)
 {
 	*vmax = min_t(unsigned long, *vmax, uv_info.max_sec_stor_addr);
 }
+
+/*
+ * Requests the Ultravisor to pin the page in the shared state. This will
+ * cause an intercept when the guest attempts to unshare the pinned page.
+ */
+static int uv_pin_shared(unsigned long paddr)
+{
+	struct uv_cb_cfs uvcb = {
+		.header.cmd = UVC_CMD_PIN_PAGE_SHARED,
+		.header.len = sizeof(uvcb),
+		.paddr = paddr,
+	};
+
+	if (uv_call(0, (u64)&uvcb))
+		return -EINVAL;
+	return 0;
+}
+
+/*
+ * Requests the Ultravisor to encrypt a guest page and make it
+ * accessible to the host for paging (export).
+ *
+ * @paddr: Absolute host address of page to be exported
+ */
+int uv_convert_from_secure(unsigned long paddr)
+{
+	struct uv_cb_cfs uvcb = {
+		.header.cmd = UVC_CMD_CONV_FROM_SEC_STOR,
+		.header.len = sizeof(uvcb),
+		.paddr = paddr
+	};
+
+	if (uv_call(0, (u64)&uvcb))
+		return -EINVAL;
+	return 0;
+}
+
+/*
+ * Calculate the expected ref_count for a page that would otherwise have no
+ * further pins. This was cribbed from similar functions in other places in
+ * the kernel, but with some slight modifications. We know that a secure
+ * page can not be a huge page for example.
+ */
+static int expected_page_refs(struct page *page)
+{
+	int res;
+
+	res = page_mapcount(page);
+	if (PageSwapCache(page)) {
+		res++;
+	} else if (page_mapping(page)) {
+		res++;
+		if (page_has_private(page))
+			res++;
+	}
+	return res;
+}
+
+static int make_secure_pte(pte_t *ptep, unsigned long addr,
+			   struct page *exp_page, struct uv_cb_header *uvcb)
+{
+	pte_t entry = READ_ONCE(*ptep);
+	struct page *page;
+	int expected, rc = 0;
+
+	if (!pte_present(entry))
+		return -ENXIO;
+	if (pte_val(entry) & _PAGE_INVALID)
+		return -ENXIO;
+
+	page = pte_page(entry);
+	if (page != exp_page)
+		return -ENXIO;
+	if (PageWriteback(page))
+		return -EAGAIN;
+	expected = expected_page_refs(page);
+	if (!page_ref_freeze(page, expected))
+		return -EBUSY;
+	set_bit(PG_arch_1, &page->flags);
+	rc = uv_call(0, (u64)uvcb);
+	page_ref_unfreeze(page, expected);
+	/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
+	if (rc)
+		rc = uvcb->rc == 0x10a ? -ENXIO : -EINVAL;
+	return rc;
+}
+
+/*
+ * Requests the Ultravisor to make a page accessible to a guest.
+ * If it's brought in the first time, it will be cleared. If
+ * it has been exported before, it will be decrypted and integrity
+ * checked.
+ */
+int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb)
+{
+	struct vm_area_struct *vma;
+	bool local_drain = false;
+	spinlock_t *ptelock;
+	unsigned long uaddr;
+	struct page *page;
+	pte_t *ptep;
+	int rc;
+
+again:
+	rc = -EFAULT;
+	down_read(&gmap->mm->mmap_sem);
+
+	uaddr = __gmap_translate(gmap, gaddr);
+	if (IS_ERR_VALUE(uaddr))
+		goto out;
+	vma = find_vma(gmap->mm, uaddr);
+	if (!vma)
+		goto out;
+	/*
+	 * Secure pages cannot be huge and userspace should not combine both.
+	 * In case userspace does it anyway this will result in an -EFAULT for
+	 * the unpack. The guest is thus never reaching secure mode. If
+	 * userspace is playing dirty tricky with mapping huge pages later
+	 * on this will result in a segmentation fault.
+	 */
+	if (is_vm_hugetlb_page(vma))
+		goto out;
+
+	rc = -ENXIO;
+	page = follow_page(vma, uaddr, FOLL_WRITE);
+	if (IS_ERR_OR_NULL(page))
+		goto out;
+
+	lock_page(page);
+	ptep = get_locked_pte(gmap->mm, uaddr, &ptelock);
+	rc = make_secure_pte(ptep, uaddr, page, uvcb);
+	pte_unmap_unlock(ptep, ptelock);
+	unlock_page(page);
+out:
+	up_read(&gmap->mm->mmap_sem);
+
+	if (rc == -EAGAIN) {
+		wait_on_page_writeback(page);
+	} else if (rc == -EBUSY) {
+		/*
+		 * If we have tried a local drain and the page refcount
+		 * still does not match our expected safe value, try with a
+		 * system wide drain. This is needed if the pagevecs holding
+		 * the page are on a different CPU.
+		 */
+		if (local_drain) {
+			lru_add_drain_all();
+			/* We give up here, and let the caller try again */
+			return -EAGAIN;
+		}
+		/*
+		 * We are here if the page refcount does not match the
+		 * expected safe value. The main culprits are usually
+		 * pagevecs. With lru_add_drain() we drain the pagevecs
+		 * on the local CPU so that hopefully the refcount will
+		 * reach the expected safe value.
+		 */
+		lru_add_drain();
+		local_drain = true;
+		/* And now we try again immediately after draining */
+		goto again;
+	} else if (rc == -ENXIO) {
+		if (gmap_fault(gmap, gaddr, FAULT_FLAG_WRITE))
+			return -EFAULT;
+		return -EAGAIN;
+	}
+	return rc;
+}
+EXPORT_SYMBOL_GPL(gmap_make_secure);
+
+int gmap_convert_to_secure(struct gmap *gmap, unsigned long gaddr)
+{
+	struct uv_cb_cts uvcb = {
+		.header.cmd = UVC_CMD_CONV_TO_SEC_STOR,
+		.header.len = sizeof(uvcb),
+		.guest_handle = gmap->guest_handle,
+		.gaddr = gaddr,
+	};
+
+	return gmap_make_secure(gmap, gaddr, &uvcb);
+}
+EXPORT_SYMBOL_GPL(gmap_convert_to_secure);
+
+/*
+ * To be called with the page locked or with an extra reference! This will
+ * prevent gmap_make_secure from touching the page concurrently. Having 2
+ * parallel make_page_accessible is fine, as the UV calls will become a
+ * no-op if the page is already exported.
+ */
+int arch_make_page_accessible(struct page *page)
+{
+	int rc = 0;
+
+	/* Hugepage cannot be protected, so nothing to do */
+	if (PageHuge(page))
+		return 0;
+
+	/*
+	 * PG_arch_1 is used in 3 places:
+	 * 1. for kernel page tables during early boot
+	 * 2. for storage keys of huge pages and KVM
+	 * 3. As an indication that this page might be secure. This can
+	 *    overindicate, e.g. we set the bit before calling
+	 *    convert_to_secure.
+	 * As secure pages are never huge, all 3 variants can co-exists.
+	 */
+	if (!test_bit(PG_arch_1, &page->flags))
+		return 0;
+
+	rc = uv_pin_shared(page_to_phys(page));
+	if (!rc) {
+		clear_bit(PG_arch_1, &page->flags);
+		return 0;
+	}
+
+	rc = uv_convert_from_secure(page_to_phys(page));
+	if (!rc) {
+		clear_bit(PG_arch_1, &page->flags);
+		return 0;
+	}
+
+	return rc;
+}
+EXPORT_SYMBOL_GPL(arch_make_page_accessible);
+
 #endif
-- 
2.25.0

Re: [PATCH v4 05/36] s390/mm: provide memory management functions for protected KVM guests

[Cornelia Huck]

On Mon, 24 Feb 2020 06:40:36 -0500
Christian Borntraeger <borntraeger@de.ibm.com> wrote:

> From: Claudio Imbrenda <imbrenda@linux.ibm.com>
> 
> This provides the basic ultravisor calls and page table handling to cope
> with secure guests:
> - provide arch_make_page_accessible
> - make pages accessible after unmapping of secure guests
> - provide the ultravisor commands convert to/from secure
> - provide the ultravisor commands pin/unpin shared
> - provide callbacks to make pages secure (inacccessible)
>  - we check for the expected pin count to only make pages secure if the
>    host is not accessing them
>  - we fence hugetlbfs for secure pages
> - add missing radix-tree include into gmap.h
> 
> The basic idea is that a page can have 3 states: secure, normal or
> shared. The hypervisor can call into a firmware function called
> ultravisor that allows to change the state of a page: convert from/to
> secure. The convert from secure will encrypt the page and make it
> available to the host and host I/O. The convert to secure will remove
> the host capability to access this page.
> The design is that on convert to secure we will wait until writeback and
> page refs are indicating no host usage. At the same time the convert
> from secure (export to host) will be called in common code when the
> refcount or the writeback bit is already set. This avoids races between
> convert from and to secure.
> 
> Then there is also the concept of shared pages. Those are kind of secure
> where the host can still access those pages. We need to be notified when
> the guest "unshares" such a page, basically doing a convert to secure by
> then. There is a call "pin shared page" that we use instead of convert
> from secure when possible.
> 
> We do use PG_arch_1 as an optimization to minimize the convert from
> secure/pin shared.
> 
> Several comments have been added in the code to explain the logic in
> the relevant places.
> 
> Co-developed-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
> Signed-off-by: Ulrich Weigand <Ulrich.Weigand@de.ibm.com>
> Signed-off-by: Claudio Imbrenda <imbrenda@linux.ibm.com>
> Acked-by: David Hildenbrand <david@redhat.com>
> Reviewed-by: Christian Borntraeger <borntraeger@de.ibm.com>
> [borntraeger@de.ibm.com: patch merging, splitting, fixing]
> Signed-off-by: Christian Borntraeger <borntraeger@de.ibm.com>
> ---
>  arch/s390/include/asm/gmap.h        |   4 +
>  arch/s390/include/asm/mmu.h         |   2 +
>  arch/s390/include/asm/mmu_context.h |   1 +
>  arch/s390/include/asm/page.h        |   5 +
>  arch/s390/include/asm/pgtable.h     |  35 ++++-
>  arch/s390/include/asm/uv.h          |  31 ++++
>  arch/s390/kernel/uv.c               | 227 ++++++++++++++++++++++++++++
>  7 files changed, 300 insertions(+), 5 deletions(-)

My mm-fu is not very strong; but this looks sane to me.

Acked-by: Cornelia Huck <cohuck@redhat.com>

Re: [PATCH v4 00/36] KVM: s390: Add support for protected VMs

[Christian Borntraeger]

The KVM part is almost done with review and I have now pushed this to
kvms390/next to give some exposure:
https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/log/?h=next

We still need a solution for Patch 1,"mm:gup/writeback: add callbacks for
inaccessible pages".
Andrew, I need your guidance here. Take this via s390kvm with an ACK or
take it via your tree?

Christian




On 24.02.20 12:40, Christian Borntraeger wrote:
> mm-related patches CCed on linux-mm, the complete list can be found on
> the KVM and linux-s390 list. 
> 
> Andrew, any chance to take " mm:gup/writeback: add callbacks for
> inaccessible pages" for 5.7? I can then carry the s390/kvm part. There
> is no build dependency on this patch (just a logical one).  As an
> alternative I can take an ack and carry that patch myself. 
> 
> This series contains a "pretty small" common code memory management
> change that will allow paging, guest backing with files etc almost
> just like normal VMs. It should be a no-op for all architectures not
> opting in. And it should be usable for others that also try to get
> notified on "the pages are in the process of being used for things
> like I/O". This time I included error handling and an ACK from Will
> Deacon as well as a Reviewed-by: from David Hildenbrand.
> This patch will be used by
> "[PATCH v4 05/36] s390/mm: provide memory management functions for
> protected KVM guests".
> We need to call into the "make accessible" architecture function when
> the refcount is already increased the writeback bit is set. This will
> make sure that we do not call the reverse function (convert to secure)
> until the host operation has finished.
> 
> 
> Overview
> --------
> Protected VMs (PVM) are KVM VMs, where KVM can't access the VM's state
> like guest memory and guest registers anymore. Instead the PVMs are
> mostly managed by a new entity called Ultravisor (UV), which provides
> an API, so KVM and the PV can request management actions.
> 
> PVMs are encrypted at rest and protected from hypervisor access while
> running. They switch from a normal operation into protected mode, so
> we can still use the standard boot process to load a encrypted blob
> and then move it into protected mode.
> 
> Rebooting is only possible by passing through the unprotected/normal
> mode and switching to protected again.
> 
> All patches are in the protvirtv4 branch of the korg s390 kvm git
> https://git.kernel.org/pub/scm/linux/kernel/git/kvms390/linux.git/log/?h=protvirtv6
> 
> Claudio presented the technology at his presentation at KVM Forum
> 2019.
> 
> https://static.sched.com/hosted_files/kvmforum2019/3b/ibm_protected_vms_s390x.pdf
> 
> 
> v3 -> v4:
> general
> -------
> - copyright updates
> - Reviewedby + acked by tags
> 
> KVM: s390/interrupt: do not pin adapter interrupt pages
> -------------------------------------------------------
> - more comments
> - get rid of now obsolete adapter parameter
> 
> s390/mm: provide memory management functions for protected KVM guests
> ---------------------------------------------------------------------
> - improved patch description
> 
> KVM: s390: protvirt: Add initial vm and cpu lifecycle handling
> --------------------------------------------------------------
> - rework tweak logic to not use an array
> - remove _VM_ part of the subfunction names of PV_COMMAND
> - merge alloc/create/destroy/dealloc into init/deinit
> - handle cmma deallocation on failures
> - rework error handling to pass along the first rc/rrc if VCPU or VM CREATE/DESTROY fails
> This was tested successfully with error injection and tracing. We do not deallocate on
> destroy failure and we pass along the first rc/rrc when vcpu destroy fails.
> 
>  KVM: s390: protvirt: Add KVM api documentation
> -----------------------------------------------
> - mention new MP_STATE
> - remove "old" interfaces that are no longer in the previous patch
> - move to the end
> 
> KVM: s390: protvirt: Secure memory is not mergeable
> ---------------------------------------------------
> - rebase on new lifecycle patch
> 
> KVM: s390: protvirt: UV calls in support of diag308 0,1
> -------------------------------------------------------
> - remove _VM_ part of the subfunction names of PV_COMMAND
> 
> KVM: s390: rstify new ioctls in api.rst
> ---------------------------------------
> - removed from this patch queue
> 
> 
> 
> v2 -> v3
> - rebase against v5.6-rc2
> - move some checks into the callers
> - typo fixes
> - extend UV query size
> - do a tlb flush when entering/exiting protected mode
> - more comments
> - change interface to PV_ENABLE/DISABLE instead of vcpu/vm
>   create/destroy
> - lockdep checks for *is_protected calls
> - locking improments
> - move facility 161 to qemu
> - checkpatch fixes
> - merged error handling in mm patch
> - removed vcpu pv commands
> - use mp_state for setting the IPL PSW
> 
> 
> v1 -> v2
> - rebase on top of kvm/master
> - pipe through rc and rrc. This might have created some churn here and
>   there
> - turn off sclp masking when rebooting into "unsecure"
> - memory management simplification
> - prefix page handling now via intercept 112
> - io interrupt intervention request fix (do not use GISA)
> - api.txt conversion to rst
> - sample patches on top of mm/gup/writeback
> - tons of review feedback
> - kvm_uv debug feature fixes and unifications
> - ultravisor information for /sys/firmware
> - 
> 
> RFCv2 -> v1 (you can diff the protvirtv2 and the protvirtv3 branch)
> - tons of review feedback integrated (see mail thread)
> - memory management now complete and working
> - Documentation patches merged
> - interrupt patches merged
> - CONFIG_KVM_S390_PROTECTED_VIRTUALIZATION_HOST removed
> - SIDA interface integrated into memop
> - for merged patches I removed reviews that were not in all patches
> 
> 
> 
> Christian Borntraeger (4):
>   KVM: s390/mm: Make pages accessible before destroying the guest
>   KVM: s390: protvirt: Add SCLP interrupt handling
>   KVM: s390: protvirt: do not inject interrupts after start
>   KVM: s390: protvirt: introduce and enable KVM_CAP_S390_PROTECTED
> 
> Claudio Imbrenda (3):
>   mm/gup/writeback: add callbacks for inaccessible pages
>   s390/mm: provide memory management functions for protected KVM guests
>   KVM: s390/mm: handle guest unpin events
> 
> Janosch Frank (24):
>   KVM: s390: protvirt: Add UV debug trace
>   KVM: s390: add new variants of UV CALL
>   KVM: s390: protvirt: Add initial vm and cpu lifecycle handling
>   KVM: s390: protvirt: Secure memory is not mergeable
>   KVM: s390: protvirt: Handle SE notification interceptions
>   KVM: s390: protvirt: Instruction emulation
>   KVM: s390: protvirt: Handle spec exception loops
>   KVM: s390: protvirt: Add new gprs location handling
>   KVM: S390: protvirt: Introduce instruction data area bounce buffer
>   KVM: s390: protvirt: handle secure guest prefix pages
>   KVM: s390: protvirt: Write sthyi data to instruction data area
>   KVM: s390: protvirt: STSI handling
>   KVM: s390: protvirt: disallow one_reg
>   KVM: s390: protvirt: Do only reset registers that are accessible
>   KVM: s390: protvirt: Only sync fmt4 registers
>   KVM: s390: protvirt: Add program exception injection
>   KVM: s390: protvirt: UV calls in support of diag308 0, 1
>   KVM: s390: protvirt: Report CPU state to Ultravisor
>   KVM: s390: protvirt: Support cmd 5 operation state
>   KVM: s390: protvirt: Mask PSW interrupt bits for interception 104 and
>     112
>   KVM: s390: protvirt: Add UV cpu reset calls
>   DOCUMENTATION: Protected virtual machine introduction and IPL
>   s390: protvirt: Add sysfs firmware interface for Ultravisor
>     information
>   KVM: s390: protvirt: Add KVM api documentation
> 
> Michael Mueller (1):
>   KVM: s390: protvirt: Implement interrupt injection
> 
> Ulrich Weigand (1):
>   KVM: s390/interrupt: do not pin adapter interrupt pages
> 
> Vasily Gorbik (3):
>   s390/protvirt: introduce host side setup
>   s390/protvirt: add ultravisor initialization
>   s390/mm: add (non)secure page access exceptions handlers
> 
>  .../admin-guide/kernel-parameters.txt         |   5 +
>  Documentation/virt/kvm/api.rst                |  61 +-
>  Documentation/virt/kvm/devices/s390_flic.rst  |  11 +-
>  Documentation/virt/kvm/index.rst              |   2 +
>  Documentation/virt/kvm/s390-pv-boot.rst       |  83 +++
>  Documentation/virt/kvm/s390-pv.rst            | 116 ++++
>  MAINTAINERS                                   |   1 +
>  arch/s390/boot/Makefile                       |   2 +-
>  arch/s390/boot/uv.c                           |  21 +-
>  arch/s390/include/asm/gmap.h                  |   6 +
>  arch/s390/include/asm/kvm_host.h              | 113 +++-
>  arch/s390/include/asm/mmu.h                   |   2 +
>  arch/s390/include/asm/mmu_context.h           |   1 +
>  arch/s390/include/asm/page.h                  |   5 +
>  arch/s390/include/asm/pgtable.h               |  35 +-
>  arch/s390/include/asm/uv.h                    | 252 ++++++++-
>  arch/s390/kernel/Makefile                     |   1 +
>  arch/s390/kernel/pgm_check.S                  |   4 +-
>  arch/s390/kernel/setup.c                      |   9 +-
>  arch/s390/kernel/uv.c                         | 413 ++++++++++++++
>  arch/s390/kvm/Makefile                        |   2 +-
>  arch/s390/kvm/diag.c                          |   6 +-
>  arch/s390/kvm/intercept.c                     | 117 +++-
>  arch/s390/kvm/interrupt.c                     | 399 +++++++------
>  arch/s390/kvm/kvm-s390.c                      | 532 +++++++++++++++---
>  arch/s390/kvm/kvm-s390.h                      |  51 +-
>  arch/s390/kvm/priv.c                          |  13 +-
>  arch/s390/kvm/pv.c                            | 298 ++++++++++
>  arch/s390/mm/fault.c                          |  78 +++
>  arch/s390/mm/gmap.c                           |  65 ++-
>  include/linux/gfp.h                           |   6 +
>  include/uapi/linux/kvm.h                      |  43 +-
>  mm/gup.c                                      |  15 +-
>  mm/page-writeback.c                           |   5 +
>  34 files changed, 2461 insertions(+), 312 deletions(-)
>  create mode 100644 Documentation/virt/kvm/s390-pv-boot.rst
>  create mode 100644 Documentation/virt/kvm/s390-pv.rst
>  create mode 100644 arch/s390/kernel/uv.c
>  create mode 100644 arch/s390/kvm/pv.c
>