[PATCH 0/2] Hypercall page docs and code cleanup

[PATCH 0/2] Hypercall page docs and code cleanup

[Andrew Cooper]

For the rendered result (along with some other in-progress documentation),
see:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/hypercall-abi.html

Andrew Cooper (2):
  x86: init_hypercall_page() cleanup
  docs: Introduce some hypercall page documentation

 docs/guest-guide/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++++++
 docs/guest-guide/index.rst         |  10 +++
 docs/index.rst                     |  13 ++++
 xen/arch/x86/domain.c              |  14 ++++
 xen/arch/x86/domctl.c              |   2 +-
 xen/arch/x86/hvm/hvm.c             |   8 +--
 xen/arch/x86/hvm/svm/svm.c         |  18 +++---
 xen/arch/x86/hvm/vmx/vmx.c         |  18 +++---
 xen/arch/x86/pv/dom0_build.c       |   3 +-
 xen/arch/x86/pv/hypercall.c        |  63 +++++++++---------
 xen/arch/x86/traps.c               |   2 +-
 xen/arch/x86/x86_64/traps.c        |  13 ----
 xen/include/asm-x86/domain.h       |   2 +-
 xen/include/asm-x86/hvm/hvm.h      |   4 +-
 xen/include/asm-x86/hypercall.h    |   4 +-
 15 files changed, 223 insertions(+), 78 deletions(-)
 create mode 100644 docs/guest-guide/hypercall-abi.rst
 create mode 100644 docs/guest-guide/index.rst

-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 0/2] Hypercall page docs and code cleanup

[Andrew Cooper]

For the rendered result (along with some other in-progress documentation),
see:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/hypercall-abi.html

Andrew Cooper (2):
  x86: init_hypercall_page() cleanup
  docs: Introduce some hypercall page documentation

 docs/guest-guide/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++++++
 docs/guest-guide/index.rst         |  10 +++
 docs/index.rst                     |  13 ++++
 xen/arch/x86/domain.c              |  14 ++++
 xen/arch/x86/domctl.c              |   2 +-
 xen/arch/x86/hvm/hvm.c             |   8 +--
 xen/arch/x86/hvm/svm/svm.c         |  18 +++---
 xen/arch/x86/hvm/vmx/vmx.c         |  18 +++---
 xen/arch/x86/pv/dom0_build.c       |   3 +-
 xen/arch/x86/pv/hypercall.c        |  63 +++++++++---------
 xen/arch/x86/traps.c               |   2 +-
 xen/arch/x86/x86_64/traps.c        |  13 ----
 xen/include/asm-x86/domain.h       |   2 +-
 xen/include/asm-x86/hvm/hvm.h      |   4 +-
 xen/include/asm-x86/hypercall.h    |   4 +-
 15 files changed, 223 insertions(+), 78 deletions(-)
 create mode 100644 docs/guest-guide/hypercall-abi.rst
 create mode 100644 docs/guest-guide/index.rst

-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 1/2] x86: init_hypercall_page() cleanup

[Andrew Cooper]

The various pieces of the hypercall page infrastructure have grown
organically over time and ended up in a bit of a mess.

 * Rename all functions to be of the form *_init_hypercall_page().  This
   makes them somewhat shorter, and means they can actually be grepped
   for in one go.
 * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
   terribly appropriate place for it to live.
 * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
   domain parameter from hvm_funcs.init_hypercall_page() as it isn't
   necessary.
 * Rearrange the logic in the each function to avoid needing extra local
   variables, and to write the page in one single pass.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Jun Nakajima <jun.nakajima@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
CC: Boris Ostrovsky <boris.ostrovsky@oracle.com>
CC: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
CC: Brian Woods <brian.woods@amd.com>
---
 xen/arch/x86/domain.c           | 14 +++++++++
 xen/arch/x86/domctl.c           |  2 +-
 xen/arch/x86/hvm/hvm.c          |  8 ++----
 xen/arch/x86/hvm/svm/svm.c      | 18 ++++++------
 xen/arch/x86/hvm/vmx/vmx.c      | 18 ++++++------
 xen/arch/x86/pv/dom0_build.c    |  3 +-
 xen/arch/x86/pv/hypercall.c     | 63 ++++++++++++++++++++---------------------
 xen/arch/x86/traps.c            |  2 +-
 xen/arch/x86/x86_64/traps.c     | 13 ---------
 xen/include/asm-x86/domain.h    |  2 +-
 xen/include/asm-x86/hvm/hvm.h   |  4 +--
 xen/include/asm-x86/hypercall.h |  4 +--
 12 files changed, 73 insertions(+), 78 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ac960dd..9485a17 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -175,6 +175,20 @@ static void noreturn continue_idle_domain(struct vcpu *v)
     reset_stack_and_jump(idle_loop);
 }
 
+void init_hypercall_page(struct domain *d, void *ptr)
+{
+    memset(ptr, 0xcc, PAGE_SIZE);
+
+    if ( is_hvm_domain(d) )
+        hvm_init_hypercall_page(d, ptr);
+    else if ( is_pv_64bit_domain(d) )
+        pv_ring3_init_hypercall_page(ptr);
+    else if ( is_pv_32bit_domain(d) )
+        pv_ring1_init_hypercall_page(ptr);
+    else
+        ASSERT_UNREACHABLE();
+}
+
 void dump_pageframe_info(struct domain *d)
 {
     struct page_info *page;
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 9bf2d08..7c6b809 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -517,7 +517,7 @@ long arch_do_domctl(
         }
 
         hypercall_page = __map_domain_page(page);
-        hypercall_page_initialise(d, hypercall_page);
+        init_hypercall_page(d, hypercall_page);
         unmap_domain_page(hypercall_page);
 
         put_page_and_type(page);
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 8993c2a..5666286 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3801,13 +3801,11 @@ static void hvm_latch_shinfo_size(struct domain *d)
     }
 }
 
-/* Initialise a hypercall transfer page for a VMX domain using
-   paravirtualised drivers. */
-void hvm_hypercall_page_initialise(struct domain *d,
-                                   void *hypercall_page)
+void hvm_init_hypercall_page(struct domain *d, void *ptr)
 {
     hvm_latch_shinfo_size(d);
-    alternative_vcall(hvm_funcs.init_hypercall_page, d, hypercall_page);
+
+    alternative_vcall(hvm_funcs.init_hypercall_page, ptr);
 }
 
 void hvm_vcpu_reset_state(struct vcpu *v, uint16_t cs, uint16_t ip)
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 9f26493..cd6a6b3 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -916,17 +916,20 @@ static unsigned int svm_get_insn_bytes(struct vcpu *v, uint8_t *buf)
     return len;
 }
 
-static void svm_init_hypercall_page(struct domain *d, void *hypercall_page)
+static void svm_init_hypercall_page(void *p)
 {
-    char *p;
-    int i;
+    unsigned int i;
 
-    for ( i = 0; i < (PAGE_SIZE / 32); i++ )
+    for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /* HYPERVISOR_iret isn't supported */
+            *(u16 *)p = 0x0b0f; /* ud2 */
+
             continue;
+        }
 
-        p = (char *)(hypercall_page + (i * 32));
         *(u8  *)(p + 0) = 0xb8; /* mov imm32, %eax */
         *(u32 *)(p + 1) = i;
         *(u8  *)(p + 5) = 0x0f; /* vmmcall */
@@ -934,9 +937,6 @@ static void svm_init_hypercall_page(struct domain *d, void *hypercall_page)
         *(u8  *)(p + 7) = 0xd9;
         *(u8  *)(p + 8) = 0xc3; /* ret */
     }
-
-    /* Don't support HYPERVISOR_iret at the moment */
-    *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
 }
 
 static inline void svm_tsc_ratio_save(struct vcpu *v)
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 7d96678..0060310 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1262,17 +1262,20 @@ static void vmx_set_descriptor_access_exiting(struct vcpu *v, bool enable)
     vmx_vmcs_exit(v);
 }
 
-static void vmx_init_hypercall_page(struct domain *d, void *hypercall_page)
+static void vmx_init_hypercall_page(void *p)
 {
-    char *p;
-    int i;
+    unsigned int i;
 
-    for ( i = 0; i < (PAGE_SIZE / 32); i++ )
+    for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /* HYPERVISOR_iret isn't supported */
+            *(u16 *)p = 0x0b0f; /* ud2 */
+
             continue;
+        }
 
-        p = (char *)(hypercall_page + (i * 32));
         *(u8  *)(p + 0) = 0xb8; /* mov imm32, %eax */
         *(u32 *)(p + 1) = i;
         *(u8  *)(p + 5) = 0x0f; /* vmcall */
@@ -1280,9 +1283,6 @@ static void vmx_init_hypercall_page(struct domain *d, void *hypercall_page)
         *(u8  *)(p + 7) = 0xc1;
         *(u8  *)(p + 8) = 0xc3; /* ret */
     }
-
-    /* Don't support HYPERVISOR_iret at the moment */
-    *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
 }
 
 static unsigned int vmx_get_interrupt_shadow(struct vcpu *v)
diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index cef2d42..d48d014 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -738,8 +738,7 @@ int __init dom0_construct_pv(struct domain *d,
             rc = -1;
             goto out;
         }
-        hypercall_page_initialise(
-            d, (void *)(unsigned long)parms.virt_hypercall);
+        init_hypercall_page(d, _p(parms.virt_hypercall));
     }
 
     /* Free temporary buffers. */
diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
index 5fdb8f9..0c84c0b 100644
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -267,16 +267,28 @@ enum mc_disposition arch_do_multicall_call(struct mc_state *state)
              ? mc_continue : mc_preempt;
 }
 
-void hypercall_page_initialise_ring3_kernel(void *hypercall_page)
+void pv_ring3_init_hypercall_page(void *p)
 {
-    void *p = hypercall_page;
     unsigned int i;
 
-    /* Fill in all the transfer points with template machine code. */
     for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /*
+             * HYPERVISOR_iret is special because it doesn't return and
+             * expects a special stack frame. Guests jump at this transfer
+             * point instead of calling it.
+             */
+            *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
+            *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
+            *(u8  *)(p+ 3) = 0x50;    /* push %rax */
+            *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret, %eax */
+            *(u32 *)(p+ 5) = __HYPERVISOR_iret;
+            *(u16 *)(p+ 9) = 0x050f;  /* syscall */
+
             continue;
+        }
 
         *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
         *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
@@ -287,49 +299,34 @@ void hypercall_page_initialise_ring3_kernel(void *hypercall_page)
         *(u8  *)(p+12) = 0x59;    /* pop  %rcx */
         *(u8  *)(p+13) = 0xc3;    /* ret */
     }
-
-    /*
-     * HYPERVISOR_iret is special because it doesn't return and expects a
-     * special stack frame. Guests jump at this transfer point instead of
-     * calling it.
-     */
-    p = hypercall_page + (__HYPERVISOR_iret * 32);
-    *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
-    *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
-    *(u8  *)(p+ 3) = 0x50;    /* push %rax */
-    *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
-    *(u32 *)(p+ 5) = __HYPERVISOR_iret;
-    *(u16 *)(p+ 9) = 0x050f;  /* syscall */
 }
 
-void hypercall_page_initialise_ring1_kernel(void *hypercall_page)
+void pv_ring1_init_hypercall_page(void *p)
 {
-    void *p = hypercall_page;
     unsigned int i;
 
-    /* Fill in all the transfer points with template machine code. */
-
     for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /*
+             * HYPERVISOR_iret is special because it doesn't return and
+             * expects a special stack frame. Guests jump at this transfer
+             * point instead of calling it.
+             */
+            *(u8  *)(p+ 0) = 0x50;    /* push %eax */
+            *(u8  *)(p+ 1) = 0xb8;    /* mov  $__HYPERVISOR_iret, %eax */
+            *(u32 *)(p+ 2) = __HYPERVISOR_iret;
+            *(u16 *)(p+ 6) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
+
             continue;
+        }
 
         *(u8  *)(p+ 0) = 0xb8;    /* mov  $<i>,%eax */
         *(u32 *)(p+ 1) = i;
         *(u16 *)(p+ 5) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
         *(u8  *)(p+ 7) = 0xc3;    /* ret */
     }
-
-    /*
-     * HYPERVISOR_iret is special because it doesn't return and expects a
-     * special stack frame. Guests jump at this transfer point instead of
-     * calling it.
-     */
-    p = hypercall_page + (__HYPERVISOR_iret * 32);
-    *(u8  *)(p+ 0) = 0x50;    /* push %eax */
-    *(u8  *)(p+ 1) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
-    *(u32 *)(p+ 2) = __HYPERVISOR_iret;
-    *(u16 *)(p+ 6) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
 }
 
 /*
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 05ddc39..ba1053f 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -828,7 +828,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val)
         }
 
         hypercall_page = __map_domain_page(page);
-        hypercall_page_initialise(d, hypercall_page);
+        init_hypercall_page(d, hypercall_page);
         unmap_domain_page(hypercall_page);
 
         put_page_and_type(page);
diff --git a/xen/arch/x86/x86_64/traps.c b/xen/arch/x86/x86_64/traps.c
index cb4bf0a..23d9357 100644
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -360,19 +360,6 @@ void subarch_percpu_traps_init(void)
     wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
 }
 
-void hypercall_page_initialise(struct domain *d, void *hypercall_page)
-{
-    memset(hypercall_page, 0xCC, PAGE_SIZE);
-    if ( is_hvm_domain(d) )
-        hvm_hypercall_page_initialise(d, hypercall_page);
-    else if ( is_pv_64bit_domain(d) )
-        hypercall_page_initialise_ring3_kernel(hypercall_page);
-    else if ( is_pv_32bit_domain(d) )
-        hypercall_page_initialise_ring1_kernel(hypercall_page);
-    else
-        ASSERT_UNREACHABLE();
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 214e44c..72dea80 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -83,7 +83,7 @@ void cpuid_policy_updated(struct vcpu *v);
  * Initialise a hypercall-transfer page. The given pointer must be mapped
  * in Xen virtual address space (accesses are not validated or checked).
  */
-void hypercall_page_initialise(struct domain *d, void *);
+void init_hypercall_page(struct domain *d, void *);
 
 /************************************************/
 /*          shadow paging extension             */
diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
index 1921422..b327bd2 100644
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -152,7 +152,7 @@ struct hvm_function_table {
 
     void (*inject_event)(const struct x86_event *event);
 
-    void (*init_hypercall_page)(struct domain *d, void *hypercall_page);
+    void (*init_hypercall_page)(void *ptr);
 
     bool (*event_pending)(const struct vcpu *v);
     bool (*get_pending_event)(struct vcpu *v, struct x86_event *info);
@@ -272,7 +272,7 @@ int hvm_girq_dest_2_vcpu_id(struct domain *d, uint8_t dest, uint8_t dest_mode);
 enum hvm_intblk
 hvm_interrupt_blocked(struct vcpu *v, struct hvm_intack intack);
 
-void hvm_hypercall_page_initialise(struct domain *d, void *hypercall_page);
+void hvm_init_hypercall_page(struct domain *d, void *ptr);
 
 void hvm_get_segment_register(struct vcpu *v, enum x86_segment seg,
                               struct segment_register *reg);
diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hypercall.h
index 49eb5f1..1cd8046 100644
--- a/xen/include/asm-x86/hypercall.h
+++ b/xen/include/asm-x86/hypercall.h
@@ -30,8 +30,8 @@ extern const hypercall_table_t pv_hypercall_table[];
 void pv_hypercall(struct cpu_user_regs *regs);
 #endif
 
-void hypercall_page_initialise_ring3_kernel(void *hypercall_page);
-void hypercall_page_initialise_ring1_kernel(void *hypercall_page);
+void pv_ring1_init_hypercall_page(void *ptr);
+void pv_ring3_init_hypercall_page(void *ptr);
 
 /*
  * Both do_mmuext_op() and do_mmu_update():
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 1/2] x86: init_hypercall_page() cleanup

[Andrew Cooper]

The various pieces of the hypercall page infrastructure have grown
organically over time and ended up in a bit of a mess.

 * Rename all functions to be of the form *_init_hypercall_page().  This
   makes them somewhat shorter, and means they can actually be grepped
   for in one go.
 * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
   terribly appropriate place for it to live.
 * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
   domain parameter from hvm_funcs.init_hypercall_page() as it isn't
   necessary.
 * Rearrange the logic in the each function to avoid needing extra local
   variables, and to write the page in one single pass.

No functional change.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: Jan Beulich <JBeulich@suse.com>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Roger Pau Monné <roger.pau@citrix.com>
CC: Jun Nakajima <jun.nakajima@intel.com>
CC: Kevin Tian <kevin.tian@intel.com>
CC: Boris Ostrovsky <boris.ostrovsky@oracle.com>
CC: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
CC: Brian Woods <brian.woods@amd.com>
---
 xen/arch/x86/domain.c           | 14 +++++++++
 xen/arch/x86/domctl.c           |  2 +-
 xen/arch/x86/hvm/hvm.c          |  8 ++----
 xen/arch/x86/hvm/svm/svm.c      | 18 ++++++------
 xen/arch/x86/hvm/vmx/vmx.c      | 18 ++++++------
 xen/arch/x86/pv/dom0_build.c    |  3 +-
 xen/arch/x86/pv/hypercall.c     | 63 ++++++++++++++++++++---------------------
 xen/arch/x86/traps.c            |  2 +-
 xen/arch/x86/x86_64/traps.c     | 13 ---------
 xen/include/asm-x86/domain.h    |  2 +-
 xen/include/asm-x86/hvm/hvm.h   |  4 +--
 xen/include/asm-x86/hypercall.h |  4 +--
 12 files changed, 73 insertions(+), 78 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index ac960dd..9485a17 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -175,6 +175,20 @@ static void noreturn continue_idle_domain(struct vcpu *v)
     reset_stack_and_jump(idle_loop);
 }
 
+void init_hypercall_page(struct domain *d, void *ptr)
+{
+    memset(ptr, 0xcc, PAGE_SIZE);
+
+    if ( is_hvm_domain(d) )
+        hvm_init_hypercall_page(d, ptr);
+    else if ( is_pv_64bit_domain(d) )
+        pv_ring3_init_hypercall_page(ptr);
+    else if ( is_pv_32bit_domain(d) )
+        pv_ring1_init_hypercall_page(ptr);
+    else
+        ASSERT_UNREACHABLE();
+}
+
 void dump_pageframe_info(struct domain *d)
 {
     struct page_info *page;
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 9bf2d08..7c6b809 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -517,7 +517,7 @@ long arch_do_domctl(
         }
 
         hypercall_page = __map_domain_page(page);
-        hypercall_page_initialise(d, hypercall_page);
+        init_hypercall_page(d, hypercall_page);
         unmap_domain_page(hypercall_page);
 
         put_page_and_type(page);
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 8993c2a..5666286 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -3801,13 +3801,11 @@ static void hvm_latch_shinfo_size(struct domain *d)
     }
 }
 
-/* Initialise a hypercall transfer page for a VMX domain using
-   paravirtualised drivers. */
-void hvm_hypercall_page_initialise(struct domain *d,
-                                   void *hypercall_page)
+void hvm_init_hypercall_page(struct domain *d, void *ptr)
 {
     hvm_latch_shinfo_size(d);
-    alternative_vcall(hvm_funcs.init_hypercall_page, d, hypercall_page);
+
+    alternative_vcall(hvm_funcs.init_hypercall_page, ptr);
 }
 
 void hvm_vcpu_reset_state(struct vcpu *v, uint16_t cs, uint16_t ip)
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index 9f26493..cd6a6b3 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -916,17 +916,20 @@ static unsigned int svm_get_insn_bytes(struct vcpu *v, uint8_t *buf)
     return len;
 }
 
-static void svm_init_hypercall_page(struct domain *d, void *hypercall_page)
+static void svm_init_hypercall_page(void *p)
 {
-    char *p;
-    int i;
+    unsigned int i;
 
-    for ( i = 0; i < (PAGE_SIZE / 32); i++ )
+    for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /* HYPERVISOR_iret isn't supported */
+            *(u16 *)p = 0x0b0f; /* ud2 */
+
             continue;
+        }
 
-        p = (char *)(hypercall_page + (i * 32));
         *(u8  *)(p + 0) = 0xb8; /* mov imm32, %eax */
         *(u32 *)(p + 1) = i;
         *(u8  *)(p + 5) = 0x0f; /* vmmcall */
@@ -934,9 +937,6 @@ static void svm_init_hypercall_page(struct domain *d, void *hypercall_page)
         *(u8  *)(p + 7) = 0xd9;
         *(u8  *)(p + 8) = 0xc3; /* ret */
     }
-
-    /* Don't support HYPERVISOR_iret at the moment */
-    *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
 }
 
 static inline void svm_tsc_ratio_save(struct vcpu *v)
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 7d96678..0060310 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -1262,17 +1262,20 @@ static void vmx_set_descriptor_access_exiting(struct vcpu *v, bool enable)
     vmx_vmcs_exit(v);
 }
 
-static void vmx_init_hypercall_page(struct domain *d, void *hypercall_page)
+static void vmx_init_hypercall_page(void *p)
 {
-    char *p;
-    int i;
+    unsigned int i;
 
-    for ( i = 0; i < (PAGE_SIZE / 32); i++ )
+    for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /* HYPERVISOR_iret isn't supported */
+            *(u16 *)p = 0x0b0f; /* ud2 */
+
             continue;
+        }
 
-        p = (char *)(hypercall_page + (i * 32));
         *(u8  *)(p + 0) = 0xb8; /* mov imm32, %eax */
         *(u32 *)(p + 1) = i;
         *(u8  *)(p + 5) = 0x0f; /* vmcall */
@@ -1280,9 +1283,6 @@ static void vmx_init_hypercall_page(struct domain *d, void *hypercall_page)
         *(u8  *)(p + 7) = 0xc1;
         *(u8  *)(p + 8) = 0xc3; /* ret */
     }
-
-    /* Don't support HYPERVISOR_iret at the moment */
-    *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
 }
 
 static unsigned int vmx_get_interrupt_shadow(struct vcpu *v)
diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
index cef2d42..d48d014 100644
--- a/xen/arch/x86/pv/dom0_build.c
+++ b/xen/arch/x86/pv/dom0_build.c
@@ -738,8 +738,7 @@ int __init dom0_construct_pv(struct domain *d,
             rc = -1;
             goto out;
         }
-        hypercall_page_initialise(
-            d, (void *)(unsigned long)parms.virt_hypercall);
+        init_hypercall_page(d, _p(parms.virt_hypercall));
     }
 
     /* Free temporary buffers. */
diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
index 5fdb8f9..0c84c0b 100644
--- a/xen/arch/x86/pv/hypercall.c
+++ b/xen/arch/x86/pv/hypercall.c
@@ -267,16 +267,28 @@ enum mc_disposition arch_do_multicall_call(struct mc_state *state)
              ? mc_continue : mc_preempt;
 }
 
-void hypercall_page_initialise_ring3_kernel(void *hypercall_page)
+void pv_ring3_init_hypercall_page(void *p)
 {
-    void *p = hypercall_page;
     unsigned int i;
 
-    /* Fill in all the transfer points with template machine code. */
     for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /*
+             * HYPERVISOR_iret is special because it doesn't return and
+             * expects a special stack frame. Guests jump at this transfer
+             * point instead of calling it.
+             */
+            *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
+            *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
+            *(u8  *)(p+ 3) = 0x50;    /* push %rax */
+            *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret, %eax */
+            *(u32 *)(p+ 5) = __HYPERVISOR_iret;
+            *(u16 *)(p+ 9) = 0x050f;  /* syscall */
+
             continue;
+        }
 
         *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
         *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
@@ -287,49 +299,34 @@ void hypercall_page_initialise_ring3_kernel(void *hypercall_page)
         *(u8  *)(p+12) = 0x59;    /* pop  %rcx */
         *(u8  *)(p+13) = 0xc3;    /* ret */
     }
-
-    /*
-     * HYPERVISOR_iret is special because it doesn't return and expects a
-     * special stack frame. Guests jump at this transfer point instead of
-     * calling it.
-     */
-    p = hypercall_page + (__HYPERVISOR_iret * 32);
-    *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
-    *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
-    *(u8  *)(p+ 3) = 0x50;    /* push %rax */
-    *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
-    *(u32 *)(p+ 5) = __HYPERVISOR_iret;
-    *(u16 *)(p+ 9) = 0x050f;  /* syscall */
 }
 
-void hypercall_page_initialise_ring1_kernel(void *hypercall_page)
+void pv_ring1_init_hypercall_page(void *p)
 {
-    void *p = hypercall_page;
     unsigned int i;
 
-    /* Fill in all the transfer points with template machine code. */
-
     for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
     {
-        if ( i == __HYPERVISOR_iret )
+        if ( unlikely(i == __HYPERVISOR_iret) )
+        {
+            /*
+             * HYPERVISOR_iret is special because it doesn't return and
+             * expects a special stack frame. Guests jump at this transfer
+             * point instead of calling it.
+             */
+            *(u8  *)(p+ 0) = 0x50;    /* push %eax */
+            *(u8  *)(p+ 1) = 0xb8;    /* mov  $__HYPERVISOR_iret, %eax */
+            *(u32 *)(p+ 2) = __HYPERVISOR_iret;
+            *(u16 *)(p+ 6) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
+
             continue;
+        }
 
         *(u8  *)(p+ 0) = 0xb8;    /* mov  $<i>,%eax */
         *(u32 *)(p+ 1) = i;
         *(u16 *)(p+ 5) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
         *(u8  *)(p+ 7) = 0xc3;    /* ret */
     }
-
-    /*
-     * HYPERVISOR_iret is special because it doesn't return and expects a
-     * special stack frame. Guests jump at this transfer point instead of
-     * calling it.
-     */
-    p = hypercall_page + (__HYPERVISOR_iret * 32);
-    *(u8  *)(p+ 0) = 0x50;    /* push %eax */
-    *(u8  *)(p+ 1) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
-    *(u32 *)(p+ 2) = __HYPERVISOR_iret;
-    *(u16 *)(p+ 6) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
 }
 
 /*
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 05ddc39..ba1053f 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -828,7 +828,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val)
         }
 
         hypercall_page = __map_domain_page(page);
-        hypercall_page_initialise(d, hypercall_page);
+        init_hypercall_page(d, hypercall_page);
         unmap_domain_page(hypercall_page);
 
         put_page_and_type(page);
diff --git a/xen/arch/x86/x86_64/traps.c b/xen/arch/x86/x86_64/traps.c
index cb4bf0a..23d9357 100644
--- a/xen/arch/x86/x86_64/traps.c
+++ b/xen/arch/x86/x86_64/traps.c
@@ -360,19 +360,6 @@ void subarch_percpu_traps_init(void)
     wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
 }
 
-void hypercall_page_initialise(struct domain *d, void *hypercall_page)
-{
-    memset(hypercall_page, 0xCC, PAGE_SIZE);
-    if ( is_hvm_domain(d) )
-        hvm_hypercall_page_initialise(d, hypercall_page);
-    else if ( is_pv_64bit_domain(d) )
-        hypercall_page_initialise_ring3_kernel(hypercall_page);
-    else if ( is_pv_32bit_domain(d) )
-        hypercall_page_initialise_ring1_kernel(hypercall_page);
-    else
-        ASSERT_UNREACHABLE();
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 214e44c..72dea80 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -83,7 +83,7 @@ void cpuid_policy_updated(struct vcpu *v);
  * Initialise a hypercall-transfer page. The given pointer must be mapped
  * in Xen virtual address space (accesses are not validated or checked).
  */
-void hypercall_page_initialise(struct domain *d, void *);
+void init_hypercall_page(struct domain *d, void *);
 
 /************************************************/
 /*          shadow paging extension             */
diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
index 1921422..b327bd2 100644
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -152,7 +152,7 @@ struct hvm_function_table {
 
     void (*inject_event)(const struct x86_event *event);
 
-    void (*init_hypercall_page)(struct domain *d, void *hypercall_page);
+    void (*init_hypercall_page)(void *ptr);
 
     bool (*event_pending)(const struct vcpu *v);
     bool (*get_pending_event)(struct vcpu *v, struct x86_event *info);
@@ -272,7 +272,7 @@ int hvm_girq_dest_2_vcpu_id(struct domain *d, uint8_t dest, uint8_t dest_mode);
 enum hvm_intblk
 hvm_interrupt_blocked(struct vcpu *v, struct hvm_intack intack);
 
-void hvm_hypercall_page_initialise(struct domain *d, void *hypercall_page);
+void hvm_init_hypercall_page(struct domain *d, void *ptr);
 
 void hvm_get_segment_register(struct vcpu *v, enum x86_segment seg,
                               struct segment_register *reg);
diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hypercall.h
index 49eb5f1..1cd8046 100644
--- a/xen/include/asm-x86/hypercall.h
+++ b/xen/include/asm-x86/hypercall.h
@@ -30,8 +30,8 @@ extern const hypercall_table_t pv_hypercall_table[];
 void pv_hypercall(struct cpu_user_regs *regs);
 #endif
 
-void hypercall_page_initialise_ring3_kernel(void *hypercall_page);
-void hypercall_page_initialise_ring1_kernel(void *hypercall_page);
+void pv_ring1_init_hypercall_page(void *ptr);
+void pv_ring3_init_hypercall_page(void *ptr);
 
 /*
  * Both do_mmuext_op() and do_mmu_update():
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 1/2] x86: init_hypercall_page() cleanup

[Wei Liu]

On Thu, May 23, 2019 at 11:20:15AM +0100, Andrew Cooper wrote:
> The various pieces of the hypercall page infrastructure have grown
> organically over time and ended up in a bit of a mess.
> 
>  * Rename all functions to be of the form *_init_hypercall_page().  This
>    makes them somewhat shorter, and means they can actually be grepped
>    for in one go.
>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>    terribly appropriate place for it to live.
>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>    necessary.
>  * Rearrange the logic in the each function to avoid needing extra local
>    variables, and to write the page in one single pass.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] x86: init_hypercall_page() cleanup

[Wei Liu]

On Thu, May 23, 2019 at 11:20:15AM +0100, Andrew Cooper wrote:
> The various pieces of the hypercall page infrastructure have grown
> organically over time and ended up in a bit of a mess.
> 
>  * Rename all functions to be of the form *_init_hypercall_page().  This
>    makes them somewhat shorter, and means they can actually be grepped
>    for in one go.
>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>    terribly appropriate place for it to live.
>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>    necessary.
>  * Rearrange the logic in the each function to avoid needing extra local
>    variables, and to write the page in one single pass.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 1/2] x86: init_hypercall_page() cleanup

[Jan Beulich]

>>> On 23.05.19 at 12:23, <wei.liu2@citrix.com> wrote:
> On Thu, May 23, 2019 at 11:20:15AM +0100, Andrew Cooper wrote:
>> The various pieces of the hypercall page infrastructure have grown
>> organically over time and ended up in a bit of a mess.
>> 
>>  * Rename all functions to be of the form *_init_hypercall_page().  This
>>    makes them somewhat shorter, and means they can actually be grepped
>>    for in one go.
>>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>>    terribly appropriate place for it to live.
>>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>>    necessary.
>>  * Rearrange the logic in the each function to avoid needing extra local
>>    variables, and to write the page in one single pass.
>> 
>> No functional change.
>> 
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Reviewed-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] x86: init_hypercall_page() cleanup

[Jan Beulich]

>>> On 23.05.19 at 12:23, <wei.liu2@citrix.com> wrote:
> On Thu, May 23, 2019 at 11:20:15AM +0100, Andrew Cooper wrote:
>> The various pieces of the hypercall page infrastructure have grown
>> organically over time and ended up in a bit of a mess.
>> 
>>  * Rename all functions to be of the form *_init_hypercall_page().  This
>>    makes them somewhat shorter, and means they can actually be grepped
>>    for in one go.
>>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>>    terribly appropriate place for it to live.
>>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>>    necessary.
>>  * Rearrange the logic in the each function to avoid needing extra local
>>    variables, and to write the page in one single pass.
>> 
>> No functional change.
>> 
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Reviewed-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 1/2] x86: init_hypercall_page() cleanup

[Tian, Kevin]

> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com]
> Sent: Thursday, May 23, 2019 6:20 PM
> 
> The various pieces of the hypercall page infrastructure have grown
> organically over time and ended up in a bit of a mess.
> 
>  * Rename all functions to be of the form *_init_hypercall_page().  This
>    makes them somewhat shorter, and means they can actually be grepped
>    for in one go.
>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>    terribly appropriate place for it to live.
>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>    necessary.
>  * Rearrange the logic in the each function to avoid needing extra local
>    variables, and to write the page in one single pass.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] x86: init_hypercall_page() cleanup

[Tian, Kevin]

> From: Andrew Cooper [mailto:andrew.cooper3@citrix.com]
> Sent: Thursday, May 23, 2019 6:20 PM
> 
> The various pieces of the hypercall page infrastructure have grown
> organically over time and ended up in a bit of a mess.
> 
>  * Rename all functions to be of the form *_init_hypercall_page().  This
>    makes them somewhat shorter, and means they can actually be grepped
>    for in one go.
>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>    terribly appropriate place for it to live.
>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>    necessary.
>  * Rearrange the logic in the each function to avoid needing extra local
>    variables, and to write the page in one single pass.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Reviewed-by: Kevin Tian <kevin.tian@intel.com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/2] x86: init_hypercall_page() cleanup

[Woods, Brian]

On Thu, May 23, 2019 at 11:20:15AM +0100, Andy Cooper wrote:
> [CAUTION: External Email]
> 
> The various pieces of the hypercall page infrastructure have grown
> organically over time and ended up in a bit of a mess.
> 
>  * Rename all functions to be of the form *_init_hypercall_page().  This
>    makes them somewhat shorter, and means they can actually be grepped
>    for in one go.
>  * Move init_hypercall_page() to domain.c.  The 64-bit traps.c isn't a
>    terribly appropriate place for it to live.
>  * Drop an obsolete comment from hvm_init_hypercall_page() and drop the
>    domain parameter from hvm_funcs.init_hypercall_page() as it isn't
>    necessary.
>  * Rearrange the logic in the each function to avoid needing extra local
>    variables, and to write the page in one single pass.
> 
> No functional change.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Brian Woods <brian.woods@amd.com>

> ---
> CC: Jan Beulich <JBeulich@suse.com>
> CC: Wei Liu <wei.liu2@citrix.com>
> CC: Roger Pau Monné <roger.pau@citrix.com>
> CC: Jun Nakajima <jun.nakajima@intel.com>
> CC: Kevin Tian <kevin.tian@intel.com>
> CC: Boris Ostrovsky <boris.ostrovsky@oracle.com>
> CC: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> CC: Brian Woods <brian.woods@amd.com>
> ---
>  xen/arch/x86/domain.c           | 14 +++++++++
>  xen/arch/x86/domctl.c           |  2 +-
>  xen/arch/x86/hvm/hvm.c          |  8 ++----
>  xen/arch/x86/hvm/svm/svm.c      | 18 ++++++------
>  xen/arch/x86/hvm/vmx/vmx.c      | 18 ++++++------
>  xen/arch/x86/pv/dom0_build.c    |  3 +-
>  xen/arch/x86/pv/hypercall.c     | 63 ++++++++++++++++++++---------------------
>  xen/arch/x86/traps.c            |  2 +-
>  xen/arch/x86/x86_64/traps.c     | 13 ---------
>  xen/include/asm-x86/domain.h    |  2 +-
>  xen/include/asm-x86/hvm/hvm.h   |  4 +--
>  xen/include/asm-x86/hypercall.h |  4 +--
>  12 files changed, 73 insertions(+), 78 deletions(-)
> 
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index ac960dd..9485a17 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -175,6 +175,20 @@ static void noreturn continue_idle_domain(struct vcpu *v)
>      reset_stack_and_jump(idle_loop);
>  }
> 
> +void init_hypercall_page(struct domain *d, void *ptr)
> +{
> +    memset(ptr, 0xcc, PAGE_SIZE);
> +
> +    if ( is_hvm_domain(d) )
> +        hvm_init_hypercall_page(d, ptr);
> +    else if ( is_pv_64bit_domain(d) )
> +        pv_ring3_init_hypercall_page(ptr);
> +    else if ( is_pv_32bit_domain(d) )
> +        pv_ring1_init_hypercall_page(ptr);
> +    else
> +        ASSERT_UNREACHABLE();
> +}
> +
>  void dump_pageframe_info(struct domain *d)
>  {
>      struct page_info *page;
> diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
> index 9bf2d08..7c6b809 100644
> --- a/xen/arch/x86/domctl.c
> +++ b/xen/arch/x86/domctl.c
> @@ -517,7 +517,7 @@ long arch_do_domctl(
>          }
> 
>          hypercall_page = __map_domain_page(page);
> -        hypercall_page_initialise(d, hypercall_page);
> +        init_hypercall_page(d, hypercall_page);
>          unmap_domain_page(hypercall_page);
> 
>          put_page_and_type(page);
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index 8993c2a..5666286 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -3801,13 +3801,11 @@ static void hvm_latch_shinfo_size(struct domain *d)
>      }
>  }
> 
> -/* Initialise a hypercall transfer page for a VMX domain using
> -   paravirtualised drivers. */
> -void hvm_hypercall_page_initialise(struct domain *d,
> -                                   void *hypercall_page)
> +void hvm_init_hypercall_page(struct domain *d, void *ptr)
>  {
>      hvm_latch_shinfo_size(d);
> -    alternative_vcall(hvm_funcs.init_hypercall_page, d, hypercall_page);
> +
> +    alternative_vcall(hvm_funcs.init_hypercall_page, ptr);
>  }
> 
>  void hvm_vcpu_reset_state(struct vcpu *v, uint16_t cs, uint16_t ip)
> diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
> index 9f26493..cd6a6b3 100644
> --- a/xen/arch/x86/hvm/svm/svm.c
> +++ b/xen/arch/x86/hvm/svm/svm.c
> @@ -916,17 +916,20 @@ static unsigned int svm_get_insn_bytes(struct vcpu *v, uint8_t *buf)
>      return len;
>  }
> 
> -static void svm_init_hypercall_page(struct domain *d, void *hypercall_page)
> +static void svm_init_hypercall_page(void *p)
>  {
> -    char *p;
> -    int i;
> +    unsigned int i;
> 
> -    for ( i = 0; i < (PAGE_SIZE / 32); i++ )
> +    for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
>      {
> -        if ( i == __HYPERVISOR_iret )
> +        if ( unlikely(i == __HYPERVISOR_iret) )
> +        {
> +            /* HYPERVISOR_iret isn't supported */
> +            *(u16 *)p = 0x0b0f; /* ud2 */
> +
>              continue;
> +        }
> 
> -        p = (char *)(hypercall_page + (i * 32));
>          *(u8  *)(p + 0) = 0xb8; /* mov imm32, %eax */
>          *(u32 *)(p + 1) = i;
>          *(u8  *)(p + 5) = 0x0f; /* vmmcall */
> @@ -934,9 +937,6 @@ static void svm_init_hypercall_page(struct domain *d, void *hypercall_page)
>          *(u8  *)(p + 7) = 0xd9;
>          *(u8  *)(p + 8) = 0xc3; /* ret */
>      }
> -
> -    /* Don't support HYPERVISOR_iret at the moment */
> -    *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
>  }
> 
>  static inline void svm_tsc_ratio_save(struct vcpu *v)
> diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
> index 7d96678..0060310 100644
> --- a/xen/arch/x86/hvm/vmx/vmx.c
> +++ b/xen/arch/x86/hvm/vmx/vmx.c
> @@ -1262,17 +1262,20 @@ static void vmx_set_descriptor_access_exiting(struct vcpu *v, bool enable)
>      vmx_vmcs_exit(v);
>  }
> 
> -static void vmx_init_hypercall_page(struct domain *d, void *hypercall_page)
> +static void vmx_init_hypercall_page(void *p)
>  {
> -    char *p;
> -    int i;
> +    unsigned int i;
> 
> -    for ( i = 0; i < (PAGE_SIZE / 32); i++ )
> +    for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
>      {
> -        if ( i == __HYPERVISOR_iret )
> +        if ( unlikely(i == __HYPERVISOR_iret) )
> +        {
> +            /* HYPERVISOR_iret isn't supported */
> +            *(u16 *)p = 0x0b0f; /* ud2 */
> +
>              continue;
> +        }
> 
> -        p = (char *)(hypercall_page + (i * 32));
>          *(u8  *)(p + 0) = 0xb8; /* mov imm32, %eax */
>          *(u32 *)(p + 1) = i;
>          *(u8  *)(p + 5) = 0x0f; /* vmcall */
> @@ -1280,9 +1283,6 @@ static void vmx_init_hypercall_page(struct domain *d, void *hypercall_page)
>          *(u8  *)(p + 7) = 0xc1;
>          *(u8  *)(p + 8) = 0xc3; /* ret */
>      }
> -
> -    /* Don't support HYPERVISOR_iret at the moment */
> -    *(u16 *)(hypercall_page + (__HYPERVISOR_iret * 32)) = 0x0b0f; /* ud2 */
>  }
> 
>  static unsigned int vmx_get_interrupt_shadow(struct vcpu *v)
> diff --git a/xen/arch/x86/pv/dom0_build.c b/xen/arch/x86/pv/dom0_build.c
> index cef2d42..d48d014 100644
> --- a/xen/arch/x86/pv/dom0_build.c
> +++ b/xen/arch/x86/pv/dom0_build.c
> @@ -738,8 +738,7 @@ int __init dom0_construct_pv(struct domain *d,
>              rc = -1;
>              goto out;
>          }
> -        hypercall_page_initialise(
> -            d, (void *)(unsigned long)parms.virt_hypercall);
> +        init_hypercall_page(d, _p(parms.virt_hypercall));
>      }
> 
>      /* Free temporary buffers. */
> diff --git a/xen/arch/x86/pv/hypercall.c b/xen/arch/x86/pv/hypercall.c
> index 5fdb8f9..0c84c0b 100644
> --- a/xen/arch/x86/pv/hypercall.c
> +++ b/xen/arch/x86/pv/hypercall.c
> @@ -267,16 +267,28 @@ enum mc_disposition arch_do_multicall_call(struct mc_state *state)
>               ? mc_continue : mc_preempt;
>  }
> 
> -void hypercall_page_initialise_ring3_kernel(void *hypercall_page)
> +void pv_ring3_init_hypercall_page(void *p)
>  {
> -    void *p = hypercall_page;
>      unsigned int i;
> 
> -    /* Fill in all the transfer points with template machine code. */
>      for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
>      {
> -        if ( i == __HYPERVISOR_iret )
> +        if ( unlikely(i == __HYPERVISOR_iret) )
> +        {
> +            /*
> +             * HYPERVISOR_iret is special because it doesn't return and
> +             * expects a special stack frame. Guests jump at this transfer
> +             * point instead of calling it.
> +             */
> +            *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
> +            *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
> +            *(u8  *)(p+ 3) = 0x50;    /* push %rax */
> +            *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret, %eax */
> +            *(u32 *)(p+ 5) = __HYPERVISOR_iret;
> +            *(u16 *)(p+ 9) = 0x050f;  /* syscall */
> +
>              continue;
> +        }
> 
>          *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
>          *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
> @@ -287,49 +299,34 @@ void hypercall_page_initialise_ring3_kernel(void *hypercall_page)
>          *(u8  *)(p+12) = 0x59;    /* pop  %rcx */
>          *(u8  *)(p+13) = 0xc3;    /* ret */
>      }
> -
> -    /*
> -     * HYPERVISOR_iret is special because it doesn't return and expects a
> -     * special stack frame. Guests jump at this transfer point instead of
> -     * calling it.
> -     */
> -    p = hypercall_page + (__HYPERVISOR_iret * 32);
> -    *(u8  *)(p+ 0) = 0x51;    /* push %rcx */
> -    *(u16 *)(p+ 1) = 0x5341;  /* push %r11 */
> -    *(u8  *)(p+ 3) = 0x50;    /* push %rax */
> -    *(u8  *)(p+ 4) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
> -    *(u32 *)(p+ 5) = __HYPERVISOR_iret;
> -    *(u16 *)(p+ 9) = 0x050f;  /* syscall */
>  }
> 
> -void hypercall_page_initialise_ring1_kernel(void *hypercall_page)
> +void pv_ring1_init_hypercall_page(void *p)
>  {
> -    void *p = hypercall_page;
>      unsigned int i;
> 
> -    /* Fill in all the transfer points with template machine code. */
> -
>      for ( i = 0; i < (PAGE_SIZE / 32); i++, p += 32 )
>      {
> -        if ( i == __HYPERVISOR_iret )
> +        if ( unlikely(i == __HYPERVISOR_iret) )
> +        {
> +            /*
> +             * HYPERVISOR_iret is special because it doesn't return and
> +             * expects a special stack frame. Guests jump at this transfer
> +             * point instead of calling it.
> +             */
> +            *(u8  *)(p+ 0) = 0x50;    /* push %eax */
> +            *(u8  *)(p+ 1) = 0xb8;    /* mov  $__HYPERVISOR_iret, %eax */
> +            *(u32 *)(p+ 2) = __HYPERVISOR_iret;
> +            *(u16 *)(p+ 6) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
> +
>              continue;
> +        }
> 
>          *(u8  *)(p+ 0) = 0xb8;    /* mov  $<i>,%eax */
>          *(u32 *)(p+ 1) = i;
>          *(u16 *)(p+ 5) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
>          *(u8  *)(p+ 7) = 0xc3;    /* ret */
>      }
> -
> -    /*
> -     * HYPERVISOR_iret is special because it doesn't return and expects a
> -     * special stack frame. Guests jump at this transfer point instead of
> -     * calling it.
> -     */
> -    p = hypercall_page + (__HYPERVISOR_iret * 32);
> -    *(u8  *)(p+ 0) = 0x50;    /* push %eax */
> -    *(u8  *)(p+ 1) = 0xb8;    /* mov  $__HYPERVISOR_iret,%eax */
> -    *(u32 *)(p+ 2) = __HYPERVISOR_iret;
> -    *(u16 *)(p+ 6) = (HYPERCALL_VECTOR << 8) | 0xcd; /* int  $xx */
>  }
> 
>  /*
> diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
> index 05ddc39..ba1053f 100644
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -828,7 +828,7 @@ int guest_wrmsr_xen(struct vcpu *v, uint32_t idx, uint64_t val)
>          }
> 
>          hypercall_page = __map_domain_page(page);
> -        hypercall_page_initialise(d, hypercall_page);
> +        init_hypercall_page(d, hypercall_page);
>          unmap_domain_page(hypercall_page);
> 
>          put_page_and_type(page);
> diff --git a/xen/arch/x86/x86_64/traps.c b/xen/arch/x86/x86_64/traps.c
> index cb4bf0a..23d9357 100644
> --- a/xen/arch/x86/x86_64/traps.c
> +++ b/xen/arch/x86/x86_64/traps.c
> @@ -360,19 +360,6 @@ void subarch_percpu_traps_init(void)
>      wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
>  }
> 
> -void hypercall_page_initialise(struct domain *d, void *hypercall_page)
> -{
> -    memset(hypercall_page, 0xCC, PAGE_SIZE);
> -    if ( is_hvm_domain(d) )
> -        hvm_hypercall_page_initialise(d, hypercall_page);
> -    else if ( is_pv_64bit_domain(d) )
> -        hypercall_page_initialise_ring3_kernel(hypercall_page);
> -    else if ( is_pv_32bit_domain(d) )
> -        hypercall_page_initialise_ring1_kernel(hypercall_page);
> -    else
> -        ASSERT_UNREACHABLE();
> -}
> -
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
> index 214e44c..72dea80 100644
> --- a/xen/include/asm-x86/domain.h
> +++ b/xen/include/asm-x86/domain.h
> @@ -83,7 +83,7 @@ void cpuid_policy_updated(struct vcpu *v);
>   * Initialise a hypercall-transfer page. The given pointer must be mapped
>   * in Xen virtual address space (accesses are not validated or checked).
>   */
> -void hypercall_page_initialise(struct domain *d, void *);
> +void init_hypercall_page(struct domain *d, void *);
> 
>  /************************************************/
>  /*          shadow paging extension             */
> diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
> index 1921422..b327bd2 100644
> --- a/xen/include/asm-x86/hvm/hvm.h
> +++ b/xen/include/asm-x86/hvm/hvm.h
> @@ -152,7 +152,7 @@ struct hvm_function_table {
> 
>      void (*inject_event)(const struct x86_event *event);
> 
> -    void (*init_hypercall_page)(struct domain *d, void *hypercall_page);
> +    void (*init_hypercall_page)(void *ptr);
> 
>      bool (*event_pending)(const struct vcpu *v);
>      bool (*get_pending_event)(struct vcpu *v, struct x86_event *info);
> @@ -272,7 +272,7 @@ int hvm_girq_dest_2_vcpu_id(struct domain *d, uint8_t dest, uint8_t dest_mode);
>  enum hvm_intblk
>  hvm_interrupt_blocked(struct vcpu *v, struct hvm_intack intack);
> 
> -void hvm_hypercall_page_initialise(struct domain *d, void *hypercall_page);
> +void hvm_init_hypercall_page(struct domain *d, void *ptr);
> 
>  void hvm_get_segment_register(struct vcpu *v, enum x86_segment seg,
>                                struct segment_register *reg);
> diff --git a/xen/include/asm-x86/hypercall.h b/xen/include/asm-x86/hypercall.h
> index 49eb5f1..1cd8046 100644
> --- a/xen/include/asm-x86/hypercall.h
> +++ b/xen/include/asm-x86/hypercall.h
> @@ -30,8 +30,8 @@ extern const hypercall_table_t pv_hypercall_table[];
>  void pv_hypercall(struct cpu_user_regs *regs);
>  #endif
> 
> -void hypercall_page_initialise_ring3_kernel(void *hypercall_page);
> -void hypercall_page_initialise_ring1_kernel(void *hypercall_page);
> +void pv_ring1_init_hypercall_page(void *ptr);
> +void pv_ring3_init_hypercall_page(void *ptr);
> 
>  /*
>   * Both do_mmuext_op() and do_mmu_update():
> --
> 2.1.4
> 

-- 
Brian Woods

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

This also introduced the top-level Guest Documentation section.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Julien Grall <julien.grall@arm.com>

The rendered version can be viewed at:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/hypercall-abi.html
---
 docs/guest-guide/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++++++
 docs/guest-guide/index.rst         |  10 +++
 docs/index.rst                     |  13 ++++
 3 files changed, 150 insertions(+)
 create mode 100644 docs/guest-guide/hypercall-abi.rst
 create mode 100644 docs/guest-guide/index.rst

diff --git a/docs/guest-guide/hypercall-abi.rst b/docs/guest-guide/hypercall-abi.rst
new file mode 100644
index 0000000..9862b14
--- /dev/null
+++ b/docs/guest-guide/hypercall-abi.rst
@@ -0,0 +1,127 @@
+Hypercall ABI
+=============
+
+Hypercalls are system calls to Xen.  Two modes of guest operation are
+supported, and up to 6 individual parameters are supported.
+
+Hypercalls may only be issued by kernel-level software [1]_.
+
+Registers
+---------
+
+The registers used for hypercalls depends on the operating mode of the guest.
+
+.. list-table::
+  :header-rows: 1
+
+  * - ABI
+    - Hypercall Index
+    - Parameters (1 - 6)
+    - Result
+
+  * - 64bit
+    - %rax
+    - %rdi %rsi %rdx %r10 %r8 %r9
+    - %rax
+
+  * - 32bit
+    - %eax
+    - %ebx %ecx %edx %esi %edi %ebp
+    - %eax
+
+32 and 64bit PV guests have an ABI fixed by their guest type.  The ABI for an
+HVM guest depends on whether the vCPU is operating in a 64bit segment or not
+[2]_.
+
+
+Parameters
+----------
+
+Different hypercalls take a different number of parameters.  Each hypercall
+potentially clobbers each of its parameter registers; a guest may not rely on
+the parameter registers staying the same.  A debug build of Xen checks this by
+deliberately poisoning the parameter registers before returning back to the
+guest.
+
+
+Mode transfer
+-------------
+
+The exact sequence of instructions required to issue a hypercall differs
+between virtualisation mode and hardware vendor.
+
+.. list-table::
+  :header-rows: 1
+
+  * - Guest
+    - Transfer instruction
+
+  * - 32bit PV
+    - INT $0x82
+
+  * - 64bit PV
+    - SYSCALL
+
+  * - Intel HVM
+    - VMCALL
+
+  * - AMD HVM
+    - VMMCALL
+
+To abstract away the details, Xen implements an interface known as the
+Hypercall Page.  This allows a guest to make a hypercall without needing to
+perform mode-specific or vendor-specific setup.
+
+
+Hypercall Page
+==============
+
+The hypercall page is a page of guest RAM into which Xen will write suitable
+transfer stubs.
+
+Creating a hypercall page is an isolated operation from Xen's point of view.
+It is the guests responsibility to ensure that the hypercall page, once
+written by Xen, is mapped with executable permissions so it may be used.
+Multiple hypercall pages may be created by the guest, if it wishes.
+
+The stubs are arranged by hypercall index, and start on 32-byte boundaries.
+To invoke a specific hypercall, ``call`` the relevant stub [3]_:
+
+.. code-block:: none
+
+  call hypercall_page + index * 32
+
+There result is an ABI which is invariant of the exact operating mode or
+hardware vendor.  This is intended to simplify guest kernel interfaces by
+abstracting away the details of how it is currently running.
+
+
+Creating Hypercall Pages
+------------------------
+
+Guests which are started using the PV boot protocol may set set
+``XEN_ELFNOTE_HYPERCALL_PAGE`` to have the nominated page written as a
+hypercall page during construction.  This mechanism is common for PV guests,
+and allows hypercalls to be issued with no additional setup.
+
+Any guest can locate the Xen CPUID leaves and read the *hypercall transfer
+page* information, which specifies an MSR that can be used to create
+additional hypercall pages.  When a guest physical address is written to the
+MSR, Xen writes a hypercall page into the nominated guest page.  This
+mechanism is common for HVM guests which are typically started via legacy
+means.
+
+
+.. rubric:: Footnotes
+
+.. [1] For HVM guests, ``HVMOP_guest_request_vm_event`` may be configured to
+       be usable from userspace, but this behaviour is not default.
+
+.. [2] While it is possible to use compatibility mode segments in a 64bit
+       kernel, hypercalls issues from such a mode will be interpreted with the
+       32bit ABI.  Such a setup is not expected in production scenarios.
+
+.. [3] ``HYPERCALL_iret`` is special.  It is only implemented for PV guests
+       and takes all its parameters on the stack.  This stub should be
+       ``jmp``'d to, rather than ``call``'d.  HVM guests have this stub
+       implemented as ``ud2a`` to prevent accidental use.
diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
new file mode 100644
index 0000000..c657332
--- /dev/null
+++ b/docs/guest-guide/index.rst
@@ -0,0 +1,10 @@
+Guest documentation
+===================
+
+x86
+---
+
+.. toctree::
+  :maxdepth: 2
+
+  hypercall-abi
diff --git a/docs/index.rst b/docs/index.rst
index 9e2e256..31bb892 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,6 +21,19 @@ configure the system.
   admin-guide/index
 
 
+Guest documentation
+-------------------
+
+This documentation concerns the APIs and ABIs available to guests.  It is
+intended for OS developers trying to use a Xen feature, and for Xen developers
+to avoid breaking things.
+
+.. toctree::
+  :maxdepth: 3
+
+  guest-guide/index
+
+
 Hypervisor developer documentation
 ----------------------------------
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

This also introduced the top-level Guest Documentation section.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Julien Grall <julien.grall@arm.com>

The rendered version can be viewed at:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/hypercall-abi.html
---
 docs/guest-guide/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++++++
 docs/guest-guide/index.rst         |  10 +++
 docs/index.rst                     |  13 ++++
 3 files changed, 150 insertions(+)
 create mode 100644 docs/guest-guide/hypercall-abi.rst
 create mode 100644 docs/guest-guide/index.rst

diff --git a/docs/guest-guide/hypercall-abi.rst b/docs/guest-guide/hypercall-abi.rst
new file mode 100644
index 0000000..9862b14
--- /dev/null
+++ b/docs/guest-guide/hypercall-abi.rst
@@ -0,0 +1,127 @@
+Hypercall ABI
+=============
+
+Hypercalls are system calls to Xen.  Two modes of guest operation are
+supported, and up to 6 individual parameters are supported.
+
+Hypercalls may only be issued by kernel-level software [1]_.
+
+Registers
+---------
+
+The registers used for hypercalls depends on the operating mode of the guest.
+
+.. list-table::
+  :header-rows: 1
+
+  * - ABI
+    - Hypercall Index
+    - Parameters (1 - 6)
+    - Result
+
+  * - 64bit
+    - %rax
+    - %rdi %rsi %rdx %r10 %r8 %r9
+    - %rax
+
+  * - 32bit
+    - %eax
+    - %ebx %ecx %edx %esi %edi %ebp
+    - %eax
+
+32 and 64bit PV guests have an ABI fixed by their guest type.  The ABI for an
+HVM guest depends on whether the vCPU is operating in a 64bit segment or not
+[2]_.
+
+
+Parameters
+----------
+
+Different hypercalls take a different number of parameters.  Each hypercall
+potentially clobbers each of its parameter registers; a guest may not rely on
+the parameter registers staying the same.  A debug build of Xen checks this by
+deliberately poisoning the parameter registers before returning back to the
+guest.
+
+
+Mode transfer
+-------------
+
+The exact sequence of instructions required to issue a hypercall differs
+between virtualisation mode and hardware vendor.
+
+.. list-table::
+  :header-rows: 1
+
+  * - Guest
+    - Transfer instruction
+
+  * - 32bit PV
+    - INT $0x82
+
+  * - 64bit PV
+    - SYSCALL
+
+  * - Intel HVM
+    - VMCALL
+
+  * - AMD HVM
+    - VMMCALL
+
+To abstract away the details, Xen implements an interface known as the
+Hypercall Page.  This allows a guest to make a hypercall without needing to
+perform mode-specific or vendor-specific setup.
+
+
+Hypercall Page
+==============
+
+The hypercall page is a page of guest RAM into which Xen will write suitable
+transfer stubs.
+
+Creating a hypercall page is an isolated operation from Xen's point of view.
+It is the guests responsibility to ensure that the hypercall page, once
+written by Xen, is mapped with executable permissions so it may be used.
+Multiple hypercall pages may be created by the guest, if it wishes.
+
+The stubs are arranged by hypercall index, and start on 32-byte boundaries.
+To invoke a specific hypercall, ``call`` the relevant stub [3]_:
+
+.. code-block:: none
+
+  call hypercall_page + index * 32
+
+There result is an ABI which is invariant of the exact operating mode or
+hardware vendor.  This is intended to simplify guest kernel interfaces by
+abstracting away the details of how it is currently running.
+
+
+Creating Hypercall Pages
+------------------------
+
+Guests which are started using the PV boot protocol may set set
+``XEN_ELFNOTE_HYPERCALL_PAGE`` to have the nominated page written as a
+hypercall page during construction.  This mechanism is common for PV guests,
+and allows hypercalls to be issued with no additional setup.
+
+Any guest can locate the Xen CPUID leaves and read the *hypercall transfer
+page* information, which specifies an MSR that can be used to create
+additional hypercall pages.  When a guest physical address is written to the
+MSR, Xen writes a hypercall page into the nominated guest page.  This
+mechanism is common for HVM guests which are typically started via legacy
+means.
+
+
+.. rubric:: Footnotes
+
+.. [1] For HVM guests, ``HVMOP_guest_request_vm_event`` may be configured to
+       be usable from userspace, but this behaviour is not default.
+
+.. [2] While it is possible to use compatibility mode segments in a 64bit
+       kernel, hypercalls issues from such a mode will be interpreted with the
+       32bit ABI.  Such a setup is not expected in production scenarios.
+
+.. [3] ``HYPERCALL_iret`` is special.  It is only implemented for PV guests
+       and takes all its parameters on the stack.  This stub should be
+       ``jmp``'d to, rather than ``call``'d.  HVM guests have this stub
+       implemented as ``ud2a`` to prevent accidental use.
diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
new file mode 100644
index 0000000..c657332
--- /dev/null
+++ b/docs/guest-guide/index.rst
@@ -0,0 +1,10 @@
+Guest documentation
+===================
+
+x86
+---
+
+.. toctree::
+  :maxdepth: 2
+
+  hypercall-abi
diff --git a/docs/index.rst b/docs/index.rst
index 9e2e256..31bb892 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,6 +21,19 @@ configure the system.
   admin-guide/index
 
 
+Guest documentation
+-------------------
+
+This documentation concerns the APIs and ABIs available to guests.  It is
+intended for OS developers trying to use a Xen feature, and for Xen developers
+to avoid breaking things.
+
+.. toctree::
+  :maxdepth: 3
+
+  guest-guide/index
+
+
 Hypervisor developer documentation
 ----------------------------------
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/2] docs: Introduce some hypercall page documentation

[Wei Liu]

On Thu, May 23, 2019 at 11:20:16AM +0100, Andrew Cooper wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Wei Liu]

On Thu, May 23, 2019 at 11:20:16AM +0100, Andrew Cooper wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Large parts of this are entirely x86-centric, yet hypercalls exist
for Arm as well. If this is intentional, then I think you should say
so above.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Large parts of this are entirely x86-centric, yet hypercalls exist
for Arm as well. If this is intentional, then I think you should say
so above.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

On 23/05/2019 11:56, Jan Beulich wrote:
>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>> This also introduced the top-level Guest Documentation section.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Large parts of this are entirely x86-centric, yet hypercalls exist
> for Arm as well. If this is intentional, then I think you should say
> so above.

It is all x86 specific, which is why it is grouped under "x86 guest
documentation".

As for future plans, the actual hypercalls will live in the architecture
neutral guest documentation section.

ARM doesn't actually use anything here, because they have a single
spec-defined instruction for making hypercalls which exists in all
virt-capable hardware.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

On 23/05/2019 11:56, Jan Beulich wrote:
>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>> This also introduced the top-level Guest Documentation section.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Large parts of this are entirely x86-centric, yet hypercalls exist
> for Arm as well. If this is intentional, then I think you should say
> so above.

It is all x86 specific, which is why it is grouped under "x86 guest
documentation".

As for future plans, the actual hypercalls will live in the architecture
neutral guest documentation section.

ARM doesn't actually use anything here, because they have a single
spec-defined instruction for making hypercalls which exists in all
virt-capable hardware.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 23.05.19 at 13:01, <andrew.cooper3@citrix.com> wrote:
> On 23/05/2019 11:56, Jan Beulich wrote:
>>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>>> This also introduced the top-level Guest Documentation section.
>>>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Large parts of this are entirely x86-centric, yet hypercalls exist
>> for Arm as well. If this is intentional, then I think you should say
>> so above.
> 
> It is all x86 specific, which is why it is grouped under "x86 guest
> documentation".

Neither the path nor anything near the top of the added file suggest
this is "x86 guest documentation". How is one to make this
connection? Or are you referring to the sole entry that ends up in
docs/guest-guide/index.rst?

One other remark: Who's the intended audience? People
writing code targeting the hypercall interface, I assume. This
includes people who may not at all be familiar with the AT&T
peculiarities of the assembly language used (mainly for
naming registers). It's easy for the to understand what is
meant nevertheless, but to be honest I'd prefer if the SDM /
PM register names were used instead, i.e. in particular
without the % prefixes (but also omitting the $ on the INT
operand).

> As for future plans, the actual hypercalls will live in the architecture
> neutral guest documentation section.
> 
> ARM doesn't actually use anything here, because they have a single
> spec-defined instruction for making hypercalls which exists in all
> virt-capable hardware.

But register usage would still be relevant to describe, even if
it may just be by stating that it matches a certain ABI defined
elsewhere.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 23.05.19 at 13:01, <andrew.cooper3@citrix.com> wrote:
> On 23/05/2019 11:56, Jan Beulich wrote:
>>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>>> This also introduced the top-level Guest Documentation section.
>>>
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> Large parts of this are entirely x86-centric, yet hypercalls exist
>> for Arm as well. If this is intentional, then I think you should say
>> so above.
> 
> It is all x86 specific, which is why it is grouped under "x86 guest
> documentation".

Neither the path nor anything near the top of the added file suggest
this is "x86 guest documentation". How is one to make this
connection? Or are you referring to the sole entry that ends up in
docs/guest-guide/index.rst?

One other remark: Who's the intended audience? People
writing code targeting the hypercall interface, I assume. This
includes people who may not at all be familiar with the AT&T
peculiarities of the assembly language used (mainly for
naming registers). It's easy for the to understand what is
meant nevertheless, but to be honest I'd prefer if the SDM /
PM register names were used instead, i.e. in particular
without the % prefixes (but also omitting the $ on the INT
operand).

> As for future plans, the actual hypercalls will live in the architecture
> neutral guest documentation section.
> 
> ARM doesn't actually use anything here, because they have a single
> spec-defined instruction for making hypercalls which exists in all
> virt-capable hardware.

But register usage would still be relevant to describe, even if
it may just be by stating that it matches a certain ABI defined
elsewhere.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

On 23/05/2019 12:41, Jan Beulich wrote:
>>>> On 23.05.19 at 13:01, <andrew.cooper3@citrix.com> wrote:
>> On 23/05/2019 11:56, Jan Beulich wrote:
>>>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>>>> This also introduced the top-level Guest Documentation section.
>>>>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Large parts of this are entirely x86-centric, yet hypercalls exist
>>> for Arm as well. If this is intentional, then I think you should say
>>> so above.
>> It is all x86 specific, which is why it is grouped under "x86 guest
>> documentation".
> Neither the path nor anything near the top of the added file suggest
> this is "x86 guest documentation". How is one to make this
> connection? Or are you referring to the sole entry that ends up in
> docs/guest-guide/index.rst?

Yes, and by the way you ask this question, I presume you haven't seen
how sphinx renders it.

Nevertheless, the observation below does warrant some changes here.

>
> One other remark: Who's the intended audience?

Citing the patch:

> This documentation concerns the APIs and ABIs available to guests.  It is
> intended for OS developers trying to use a Xen feature, and for Xen
> developers
> to avoid breaking things.


> People
> writing code targeting the hypercall interface, I assume. This
> includes people who may not at all be familiar with the AT&T
> peculiarities of the assembly language used (mainly for
> naming registers). It's easy for the to understand what is
> meant nevertheless, but to be honest I'd prefer if the SDM /
> PM register names were used instead, i.e. in particular
> without the % prefixes (but also omitting the $ on the INT
> operand).

While this case is, dropping the AT&T-isms is easy, I'm not convinced
that will be the case elsewhere.

Also, I don't want us to be in a case where we develop exclusively with
AT&T, but have our documentation written in Intel syntax.

>
>> As for future plans, the actual hypercalls will live in the architecture
>> neutral guest documentation section.
>>
>> ARM doesn't actually use anything here, because they have a single
>> spec-defined instruction for making hypercalls which exists in all
>> virt-capable hardware.
> But register usage would still be relevant to describe, even if
> it may just be by stating that it matches a certain ABI defined
> elsewhere.

That belongs in ARM's hypercall-abi.rst, not x86's, and it is this
observation which demonstrates that a path distinction is necessary.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

On 23/05/2019 12:41, Jan Beulich wrote:
>>>> On 23.05.19 at 13:01, <andrew.cooper3@citrix.com> wrote:
>> On 23/05/2019 11:56, Jan Beulich wrote:
>>>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>>>> This also introduced the top-level Guest Documentation section.
>>>>
>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> Large parts of this are entirely x86-centric, yet hypercalls exist
>>> for Arm as well. If this is intentional, then I think you should say
>>> so above.
>> It is all x86 specific, which is why it is grouped under "x86 guest
>> documentation".
> Neither the path nor anything near the top of the added file suggest
> this is "x86 guest documentation". How is one to make this
> connection? Or are you referring to the sole entry that ends up in
> docs/guest-guide/index.rst?

Yes, and by the way you ask this question, I presume you haven't seen
how sphinx renders it.

Nevertheless, the observation below does warrant some changes here.

>
> One other remark: Who's the intended audience?

Citing the patch:

> This documentation concerns the APIs and ABIs available to guests.  It is
> intended for OS developers trying to use a Xen feature, and for Xen
> developers
> to avoid breaking things.


> People
> writing code targeting the hypercall interface, I assume. This
> includes people who may not at all be familiar with the AT&T
> peculiarities of the assembly language used (mainly for
> naming registers). It's easy for the to understand what is
> meant nevertheless, but to be honest I'd prefer if the SDM /
> PM register names were used instead, i.e. in particular
> without the % prefixes (but also omitting the $ on the INT
> operand).

While this case is, dropping the AT&T-isms is easy, I'm not convinced
that will be the case elsewhere.

Also, I don't want us to be in a case where we develop exclusively with
AT&T, but have our documentation written in Intel syntax.

>
>> As for future plans, the actual hypercalls will live in the architecture
>> neutral guest documentation section.
>>
>> ARM doesn't actually use anything here, because they have a single
>> spec-defined instruction for making hypercalls which exists in all
>> virt-capable hardware.
> But register usage would still be relevant to describe, even if
> it may just be by stating that it matches a certain ABI defined
> elsewhere.

That belongs in ARM's hypercall-abi.rst, not x86's, and it is this
observation which demonstrates that a path distinction is necessary.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 23.05.19 at 14:02, <andrew.cooper3@citrix.com> wrote:
> On 23/05/2019 12:41, Jan Beulich wrote:
>>>>> On 23.05.19 at 13:01, <andrew.cooper3@citrix.com> wrote:
>>> On 23/05/2019 11:56, Jan Beulich wrote:
>>>>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>>>>> This also introduced the top-level Guest Documentation section.
>>>>>
>>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> Large parts of this are entirely x86-centric, yet hypercalls exist
>>>> for Arm as well. If this is intentional, then I think you should say
>>>> so above.
>>> It is all x86 specific, which is why it is grouped under "x86 guest
>>> documentation".
>> Neither the path nor anything near the top of the added file suggest
>> this is "x86 guest documentation". How is one to make this
>> connection? Or are you referring to the sole entry that ends up in
>> docs/guest-guide/index.rst?
> 
> Yes, and by the way you ask this question, I presume you haven't seen
> how sphinx renders it.

Nor do I think I should need to, for the purpose of reviewing
the change. I'm in particular concerned about further additions
here down the road, when it would then be even less visible
that this is x86-specific documentation.

> Nevertheless, the observation below does warrant some changes here.

Right - hence me mentioning "path" in my original reply.

>> People
>> writing code targeting the hypercall interface, I assume. This
>> includes people who may not at all be familiar with the AT&T
>> peculiarities of the assembly language used (mainly for
>> naming registers). It's easy for the to understand what is
>> meant nevertheless, but to be honest I'd prefer if the SDM /
>> PM register names were used instead, i.e. in particular
>> without the % prefixes (but also omitting the $ on the INT
>> operand).
> 
> While this case is, dropping the AT&T-isms is easy, I'm not convinced
> that will be the case elsewhere.
> 
> Also, I don't want us to be in a case where we develop exclusively with
> AT&T, but have our documentation written in Intel syntax.

For one I don't think documentation and source need to be
aligned in this regard. Aligning our documentation to that
of the CPU vendors seems more important to me. (That
said, in example code I think at least one of the two have
started mixing Intel and AT&T syntax.)

And then - can't we get away without using any assembly
syntax at all in documentation? Surely things can be worded
in a more abstract way (like "INT instruction with an immediate
of 0x82).

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 23.05.19 at 14:02, <andrew.cooper3@citrix.com> wrote:
> On 23/05/2019 12:41, Jan Beulich wrote:
>>>>> On 23.05.19 at 13:01, <andrew.cooper3@citrix.com> wrote:
>>> On 23/05/2019 11:56, Jan Beulich wrote:
>>>>>>> On 23.05.19 at 12:20, <andrew.cooper3@citrix.com> wrote:
>>>>> This also introduced the top-level Guest Documentation section.
>>>>>
>>>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>>> Large parts of this are entirely x86-centric, yet hypercalls exist
>>>> for Arm as well. If this is intentional, then I think you should say
>>>> so above.
>>> It is all x86 specific, which is why it is grouped under "x86 guest
>>> documentation".
>> Neither the path nor anything near the top of the added file suggest
>> this is "x86 guest documentation". How is one to make this
>> connection? Or are you referring to the sole entry that ends up in
>> docs/guest-guide/index.rst?
> 
> Yes, and by the way you ask this question, I presume you haven't seen
> how sphinx renders it.

Nor do I think I should need to, for the purpose of reviewing
the change. I'm in particular concerned about further additions
here down the road, when it would then be even less visible
that this is x86-specific documentation.

> Nevertheless, the observation below does warrant some changes here.

Right - hence me mentioning "path" in my original reply.

>> People
>> writing code targeting the hypercall interface, I assume. This
>> includes people who may not at all be familiar with the AT&T
>> peculiarities of the assembly language used (mainly for
>> naming registers). It's easy for the to understand what is
>> meant nevertheless, but to be honest I'd prefer if the SDM /
>> PM register names were used instead, i.e. in particular
>> without the % prefixes (but also omitting the $ on the INT
>> operand).
> 
> While this case is, dropping the AT&T-isms is easy, I'm not convinced
> that will be the case elsewhere.
> 
> Also, I don't want us to be in a case where we develop exclusively with
> AT&T, but have our documentation written in Intel syntax.

For one I don't think documentation and source need to be
aligned in this regard. Aligning our documentation to that
of the CPU vendors seems more important to me. (That
said, in example code I think at least one of the two have
started mixing Intel and AT&T syntax.)

And then - can't we get away without using any assembly
syntax at all in documentation? Surely things can be worded
in a more abstract way (like "INT instruction with an immediate
of 0x82).

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v2 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

This also introduced the top-level Guest Documentation section.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Julien Grall <julien.grall@arm.com>

v2:
 * Drop AT&T ligatures
 * Move into an x86 directory

The rendered version can be viewed at:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/x86/hypercall-abi.html
---
 docs/guest-guide/index.rst             |   7 ++
 docs/guest-guide/x86/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++
 docs/guest-guide/x86/index.rst         |   7 ++
 docs/index.rst                         |  13 ++++
 4 files changed, 154 insertions(+)
 create mode 100644 docs/guest-guide/index.rst
 create mode 100644 docs/guest-guide/x86/hypercall-abi.rst
 create mode 100644 docs/guest-guide/x86/index.rst

diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
new file mode 100644
index 0000000..108e0b8
--- /dev/null
+++ b/docs/guest-guide/index.rst
@@ -0,0 +1,7 @@
+Guest documentation
+===================
+
+.. toctree::
+  :maxdepth: 2
+
+  x86/index
diff --git a/docs/guest-guide/x86/hypercall-abi.rst b/docs/guest-guide/x86/hypercall-abi.rst
new file mode 100644
index 0000000..dee2585
--- /dev/null
+++ b/docs/guest-guide/x86/hypercall-abi.rst
@@ -0,0 +1,127 @@
+Hypercall ABI
+=============
+
+Hypercalls are system calls to Xen.  Two modes of guest operation are
+supported, and up to 6 individual parameters are supported.
+
+Hypercalls may only be issued by kernel-level software [1]_.
+
+Registers
+---------
+
+The registers used for hypercalls depends on the operating mode of the guest.
+
+.. list-table::
+  :header-rows: 1
+
+  * - ABI
+    - Hypercall Index
+    - Parameters (1 - 6)
+    - Result
+
+  * - 64bit
+    - RAX
+    - RDI RSI RDX R10 R8 R9
+    - RAX
+
+  * - 32bit
+    - EAX
+    - EBX ECX EDX ESI EDI EBP
+    - EAX
+
+32 and 64bit PV guests have an ABI fixed by their guest type.  The ABI for an
+HVM guest depends on whether the vCPU is operating in a 64bit segment or not
+[2]_.
+
+
+Parameters
+----------
+
+Different hypercalls take a different number of parameters.  Each hypercall
+potentially clobbers each of its parameter registers; a guest may not rely on
+the parameter registers staying the same.  A debug build of Xen checks this by
+deliberately poisoning the parameter registers before returning back to the
+guest.
+
+
+Mode transfer
+-------------
+
+The exact sequence of instructions required to issue a hypercall differs
+between virtualisation mode and hardware vendor.
+
+.. list-table::
+  :header-rows: 1
+
+  * - Guest
+    - Transfer instruction
+
+  * - 32bit PV
+    - INT 0x82
+
+  * - 64bit PV
+    - SYSCALL
+
+  * - Intel HVM
+    - VMCALL
+
+  * - AMD HVM
+    - VMMCALL
+
+To abstract away the details, Xen implements an interface known as the
+Hypercall Page.  This allows a guest to make a hypercall without needing to
+perform mode-specific or vendor-specific setup.
+
+
+Hypercall Page
+==============
+
+The hypercall page is a page of guest RAM into which Xen will write suitable
+transfer stubs.
+
+Creating a hypercall page is an isolated operation from Xen's point of view.
+It is the guests responsibility to ensure that the hypercall page, once
+written by Xen, is mapped with executable permissions so it may be used.
+Multiple hypercall pages may be created by the guest, if it wishes.
+
+The stubs are arranged by hypercall index, and start on 32-byte boundaries.
+To invoke a specific hypercall, ``call`` the relevant stub [3]_:
+
+.. code-block:: none
+
+  call hypercall_page + index * 32
+
+There result is an ABI which is invariant of the exact operating mode or
+hardware vendor.  This is intended to simplify guest kernel interfaces by
+abstracting away the details of how it is currently running.
+
+
+Creating Hypercall Pages
+------------------------
+
+Guests which are started using the PV boot protocol may set set
+``XEN_ELFNOTE_HYPERCALL_PAGE`` to have the nominated page written as a
+hypercall page during construction.  This mechanism is common for PV guests,
+and allows hypercalls to be issued with no additional setup.
+
+Any guest can locate the Xen CPUID leaves and read the *hypercall transfer
+page* information, which specifies an MSR that can be used to create
+additional hypercall pages.  When a guest physical address is written to the
+MSR, Xen writes a hypercall page into the nominated guest page.  This
+mechanism is common for HVM guests which are typically started via legacy
+means.
+
+
+.. rubric:: Footnotes
+
+.. [1] For HVM guests, ``HVMOP_guest_request_vm_event`` may be configured to
+       be usable from userspace, but this behaviour is not default.
+
+.. [2] While it is possible to use compatibility mode segments in a 64bit
+       kernel, hypercalls issues from such a mode will be interpreted with the
+       32bit ABI.  Such a setup is not expected in production scenarios.
+
+.. [3] ``HYPERCALL_iret`` is special.  It is only implemented for PV guests
+       and takes all its parameters on the stack.  This stub should be
+       ``jmp``'d to, rather than ``call``'d.  HVM guests have this stub
+       implemented as ``ud2a`` to prevent accidental use.
diff --git a/docs/guest-guide/x86/index.rst b/docs/guest-guide/x86/index.rst
new file mode 100644
index 0000000..a368392
--- /dev/null
+++ b/docs/guest-guide/x86/index.rst
@@ -0,0 +1,7 @@
+x86
+===
+
+.. toctree::
+  :maxdepth: 2
+
+  hypercall-abi
diff --git a/docs/index.rst b/docs/index.rst
index 9e2e256..31bb892 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,6 +21,19 @@ configure the system.
   admin-guide/index
 
 
+Guest documentation
+-------------------
+
+This documentation concerns the APIs and ABIs available to guests.  It is
+intended for OS developers trying to use a Xen feature, and for Xen developers
+to avoid breaking things.
+
+.. toctree::
+  :maxdepth: 3
+
+  guest-guide/index
+
+
 Hypervisor developer documentation
 ----------------------------------
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH v2 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

This also introduced the top-level Guest Documentation section.

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
CC: George Dunlap <George.Dunlap@eu.citrix.com>
CC: Ian Jackson <ian.jackson@citrix.com>
CC: Jan Beulich <JBeulich@suse.com>
CC: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
CC: Stefano Stabellini <sstabellini@kernel.org>
CC: Tim Deegan <tim@xen.org>
CC: Wei Liu <wei.liu2@citrix.com>
CC: Julien Grall <julien.grall@arm.com>

v2:
 * Drop AT&T ligatures
 * Move into an x86 directory

The rendered version can be viewed at:

https://andrewcoop-xen.readthedocs.io/en/docs-devel/guest-guide/x86/hypercall-abi.html
---
 docs/guest-guide/index.rst             |   7 ++
 docs/guest-guide/x86/hypercall-abi.rst | 127 +++++++++++++++++++++++++++++++++
 docs/guest-guide/x86/index.rst         |   7 ++
 docs/index.rst                         |  13 ++++
 4 files changed, 154 insertions(+)
 create mode 100644 docs/guest-guide/index.rst
 create mode 100644 docs/guest-guide/x86/hypercall-abi.rst
 create mode 100644 docs/guest-guide/x86/index.rst

diff --git a/docs/guest-guide/index.rst b/docs/guest-guide/index.rst
new file mode 100644
index 0000000..108e0b8
--- /dev/null
+++ b/docs/guest-guide/index.rst
@@ -0,0 +1,7 @@
+Guest documentation
+===================
+
+.. toctree::
+  :maxdepth: 2
+
+  x86/index
diff --git a/docs/guest-guide/x86/hypercall-abi.rst b/docs/guest-guide/x86/hypercall-abi.rst
new file mode 100644
index 0000000..dee2585
--- /dev/null
+++ b/docs/guest-guide/x86/hypercall-abi.rst
@@ -0,0 +1,127 @@
+Hypercall ABI
+=============
+
+Hypercalls are system calls to Xen.  Two modes of guest operation are
+supported, and up to 6 individual parameters are supported.
+
+Hypercalls may only be issued by kernel-level software [1]_.
+
+Registers
+---------
+
+The registers used for hypercalls depends on the operating mode of the guest.
+
+.. list-table::
+  :header-rows: 1
+
+  * - ABI
+    - Hypercall Index
+    - Parameters (1 - 6)
+    - Result
+
+  * - 64bit
+    - RAX
+    - RDI RSI RDX R10 R8 R9
+    - RAX
+
+  * - 32bit
+    - EAX
+    - EBX ECX EDX ESI EDI EBP
+    - EAX
+
+32 and 64bit PV guests have an ABI fixed by their guest type.  The ABI for an
+HVM guest depends on whether the vCPU is operating in a 64bit segment or not
+[2]_.
+
+
+Parameters
+----------
+
+Different hypercalls take a different number of parameters.  Each hypercall
+potentially clobbers each of its parameter registers; a guest may not rely on
+the parameter registers staying the same.  A debug build of Xen checks this by
+deliberately poisoning the parameter registers before returning back to the
+guest.
+
+
+Mode transfer
+-------------
+
+The exact sequence of instructions required to issue a hypercall differs
+between virtualisation mode and hardware vendor.
+
+.. list-table::
+  :header-rows: 1
+
+  * - Guest
+    - Transfer instruction
+
+  * - 32bit PV
+    - INT 0x82
+
+  * - 64bit PV
+    - SYSCALL
+
+  * - Intel HVM
+    - VMCALL
+
+  * - AMD HVM
+    - VMMCALL
+
+To abstract away the details, Xen implements an interface known as the
+Hypercall Page.  This allows a guest to make a hypercall without needing to
+perform mode-specific or vendor-specific setup.
+
+
+Hypercall Page
+==============
+
+The hypercall page is a page of guest RAM into which Xen will write suitable
+transfer stubs.
+
+Creating a hypercall page is an isolated operation from Xen's point of view.
+It is the guests responsibility to ensure that the hypercall page, once
+written by Xen, is mapped with executable permissions so it may be used.
+Multiple hypercall pages may be created by the guest, if it wishes.
+
+The stubs are arranged by hypercall index, and start on 32-byte boundaries.
+To invoke a specific hypercall, ``call`` the relevant stub [3]_:
+
+.. code-block:: none
+
+  call hypercall_page + index * 32
+
+There result is an ABI which is invariant of the exact operating mode or
+hardware vendor.  This is intended to simplify guest kernel interfaces by
+abstracting away the details of how it is currently running.
+
+
+Creating Hypercall Pages
+------------------------
+
+Guests which are started using the PV boot protocol may set set
+``XEN_ELFNOTE_HYPERCALL_PAGE`` to have the nominated page written as a
+hypercall page during construction.  This mechanism is common for PV guests,
+and allows hypercalls to be issued with no additional setup.
+
+Any guest can locate the Xen CPUID leaves and read the *hypercall transfer
+page* information, which specifies an MSR that can be used to create
+additional hypercall pages.  When a guest physical address is written to the
+MSR, Xen writes a hypercall page into the nominated guest page.  This
+mechanism is common for HVM guests which are typically started via legacy
+means.
+
+
+.. rubric:: Footnotes
+
+.. [1] For HVM guests, ``HVMOP_guest_request_vm_event`` may be configured to
+       be usable from userspace, but this behaviour is not default.
+
+.. [2] While it is possible to use compatibility mode segments in a 64bit
+       kernel, hypercalls issues from such a mode will be interpreted with the
+       32bit ABI.  Such a setup is not expected in production scenarios.
+
+.. [3] ``HYPERCALL_iret`` is special.  It is only implemented for PV guests
+       and takes all its parameters on the stack.  This stub should be
+       ``jmp``'d to, rather than ``call``'d.  HVM guests have this stub
+       implemented as ``ud2a`` to prevent accidental use.
diff --git a/docs/guest-guide/x86/index.rst b/docs/guest-guide/x86/index.rst
new file mode 100644
index 0000000..a368392
--- /dev/null
+++ b/docs/guest-guide/x86/index.rst
@@ -0,0 +1,7 @@
+x86
+===
+
+.. toctree::
+  :maxdepth: 2
+
+  hypercall-abi
diff --git a/docs/index.rst b/docs/index.rst
index 9e2e256..31bb892 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -21,6 +21,19 @@ configure the system.
   admin-guide/index
 
 
+Guest documentation
+-------------------
+
+This documentation concerns the APIs and ABIs available to guests.  It is
+intended for OS developers trying to use a Xen feature, and for Xen developers
+to avoid breaking things.
+
+.. toctree::
+  :maxdepth: 3
+
+  guest-guide/index
+
+
 Hypervisor developer documentation
 ----------------------------------
 
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 29.05.19 at 06:10, <andrew.cooper3@citrix.com> wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with one further remark / question:

> +Hypercall Page
> +==============
> +
> +The hypercall page is a page of guest RAM into which Xen will write suitable
> +transfer stubs.
> +
> +Creating a hypercall page is an isolated operation from Xen's point of view.
> +It is the guests responsibility to ensure that the hypercall page, once
> +written by Xen, is mapped with executable permissions so it may be used.
> +Multiple hypercall pages may be created by the guest, if it wishes.
> +
> +The stubs are arranged by hypercall index, and start on 32-byte boundaries.
> +To invoke a specific hypercall, ``call`` the relevant stub [3]_:
> +
> +.. code-block:: none
> +
> +  call hypercall_page + index * 32
> +
> +There result is an ABI which is invariant of the exact operating mode or
> +hardware vendor.  This is intended to simplify guest kernel interfaces by
> +abstracting away the details of how it is currently running.

Is it worth mentioning here that certain aspects of the hypervisor interface
(shared data structures) are influenced by the mode the guest is in at the
time the most recent hypercall page registration (oddly enough alternatively
the most recent setting of HVM_PARAM_CALLBACK_IRQ) has occurred?

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/2] docs: Introduce some hypercall page documentation

[Jan Beulich]

>>> On 29.05.19 at 06:10, <andrew.cooper3@citrix.com> wrote:
> This also introduced the top-level Guest Documentation section.
> 
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>

Acked-by: Jan Beulich <jbeulich@suse.com>
with one further remark / question:

> +Hypercall Page
> +==============
> +
> +The hypercall page is a page of guest RAM into which Xen will write suitable
> +transfer stubs.
> +
> +Creating a hypercall page is an isolated operation from Xen's point of view.
> +It is the guests responsibility to ensure that the hypercall page, once
> +written by Xen, is mapped with executable permissions so it may be used.
> +Multiple hypercall pages may be created by the guest, if it wishes.
> +
> +The stubs are arranged by hypercall index, and start on 32-byte boundaries.
> +To invoke a specific hypercall, ``call`` the relevant stub [3]_:
> +
> +.. code-block:: none
> +
> +  call hypercall_page + index * 32
> +
> +There result is an ABI which is invariant of the exact operating mode or
> +hardware vendor.  This is intended to simplify guest kernel interfaces by
> +abstracting away the details of how it is currently running.

Is it worth mentioning here that certain aspects of the hypervisor interface
(shared data structures) are influenced by the mode the guest is in at the
time the most recent hypercall page registration (oddly enough alternatively
the most recent setting of HVM_PARAM_CALLBACK_IRQ) has occurred?

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v2 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

On 29/05/2019 09:45, Jan Beulich wrote:
>>>> On 29.05.19 at 06:10, <andrew.cooper3@citrix.com> wrote:
>> This also introduced the top-level Guest Documentation section.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> with one further remark / question:
>
>> +Hypercall Page
>> +==============
>> +
>> +The hypercall page is a page of guest RAM into which Xen will write suitable
>> +transfer stubs.
>> +
>> +Creating a hypercall page is an isolated operation from Xen's point of view.
>> +It is the guests responsibility to ensure that the hypercall page, once
>> +written by Xen, is mapped with executable permissions so it may be used.
>> +Multiple hypercall pages may be created by the guest, if it wishes.
>> +
>> +The stubs are arranged by hypercall index, and start on 32-byte boundaries.
>> +To invoke a specific hypercall, ``call`` the relevant stub [3]_:
>> +
>> +.. code-block:: none
>> +
>> +  call hypercall_page + index * 32
>> +
>> +There result is an ABI which is invariant of the exact operating mode or
>> +hardware vendor.  This is intended to simplify guest kernel interfaces by
>> +abstracting away the details of how it is currently running.
> Is it worth mentioning here that certain aspects of the hypervisor interface
> (shared data structures) are influenced by the mode the guest is in at the
> time the most recent hypercall page registration (oddly enough alternatively
> the most recent setting of HVM_PARAM_CALLBACK_IRQ) has occurred?

That aspect had crossed my mind, but I decided to leave it for now.

Details on the format of the shared_info page should live primarily in
the shared_info documentation, but I do intend to cross reference
appropriate documentation when both sides are in place.  I'd prefer that
any addition to this document happens when the main shared_info
documentation is done, rather than having an unqualified fraction of the
end result.

Note however that I am taking this opportunity to evaluate the current
behaviour of the areas being documented (hence the cleanup patches), and
I make no guarantees that the details of shared_info latching will be
identical to how they currently are, when the documentation is
eventually complete.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH v2 2/2] docs: Introduce some hypercall page documentation

[Andrew Cooper]

On 29/05/2019 09:45, Jan Beulich wrote:
>>>> On 29.05.19 at 06:10, <andrew.cooper3@citrix.com> wrote:
>> This also introduced the top-level Guest Documentation section.
>>
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Acked-by: Jan Beulich <jbeulich@suse.com>

Thanks.

> with one further remark / question:
>
>> +Hypercall Page
>> +==============
>> +
>> +The hypercall page is a page of guest RAM into which Xen will write suitable
>> +transfer stubs.
>> +
>> +Creating a hypercall page is an isolated operation from Xen's point of view.
>> +It is the guests responsibility to ensure that the hypercall page, once
>> +written by Xen, is mapped with executable permissions so it may be used.
>> +Multiple hypercall pages may be created by the guest, if it wishes.
>> +
>> +The stubs are arranged by hypercall index, and start on 32-byte boundaries.
>> +To invoke a specific hypercall, ``call`` the relevant stub [3]_:
>> +
>> +.. code-block:: none
>> +
>> +  call hypercall_page + index * 32
>> +
>> +There result is an ABI which is invariant of the exact operating mode or
>> +hardware vendor.  This is intended to simplify guest kernel interfaces by
>> +abstracting away the details of how it is currently running.
> Is it worth mentioning here that certain aspects of the hypervisor interface
> (shared data structures) are influenced by the mode the guest is in at the
> time the most recent hypercall page registration (oddly enough alternatively
> the most recent setting of HVM_PARAM_CALLBACK_IRQ) has occurred?

That aspect had crossed my mind, but I decided to leave it for now.

Details on the format of the shared_info page should live primarily in
the shared_info documentation, but I do intend to cross reference
appropriate documentation when both sides are in place.  I'd prefer that
any addition to this document happens when the main shared_info
documentation is done, rather than having an unqualified fraction of the
end result.

Note however that I am taking this opportunity to evaluate the current
behaviour of the areas being documented (hence the cleanup patches), and
I make no guarantees that the details of shared_info latching will be
identical to how they currently are, when the documentation is
eventually complete.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel