[PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[Roger Pau Monne]

This series are now rebased on top of Paul Durrant "x86/hvm: I/O emulation 
cleanup and fix" v5.

This series is split in the following order:

 - Patches from 1 to 10 switch HVM domain contruction to use the xc_dom_* 
   family of functions, like they are used to build PV domains. 
 - Patches from 11 to 21 allow disabling the devices emulated inside of Xen.
 - Patches from 22 to 32 introduce the creation of HVM guests without a 
   device model and without the devices emulated inside of Xen.

THe main difference with v2 is that instead of using a 
single XEN_DOMCTL_CDF_noemu flag in order to disable all devices emulated in 
Xen a bitmap is used, which should allow enabling or disabling specific 
emulated devices in the future.

This series has been successfully tested on the following hardware:

 - Intel Xeon W3550.
 - AMD Opteron 4184.

With both hap=0 and hap=1 in the configuration file. I've been able to boot 
a SMP guest in this mode with a virtual hard drive and a virtual network 
card, all working fine AFAICT.

For this round only maintainers of the specific code being modified have 
been Cced on the patches.

The series can also be found in the following git repo:

git://xenbits.xen.org/people/royger/xen.git branch hvm_without_dm_v3

And for the FreeBSD part:

git://xenbits.xen.org/people/royger/freebsd.git branch new_entry_point_v2

In case someone wants to give it a try, I've uploaded a FreeBSD kernel that 
should work when booted into this mode:

https://people.freebsd.org/~royger/kernel_no_dm

The config file that I've used is:

<config>
kernel="/path/to/kernel_no_dm"

builder="hvm"
device_model_version="none"

memory=128
vcpus=2
name = "freebsd"
</config>

Of course if you have a FreeBSD disk already setup it can also be added to 
the configuration file, and the following line can be used to point FreeBSD 
to the disk:

extra="vfs.root.mountfrom=ufs:/dev/ufsid/<disk_id>"

Thanks, Roger.

[PATCH v3 01/32] libxc: split x86 HVM setup_guest into smaller logical functions

[Roger Pau Monne]

This is just a preparatory change to clean up the code in setup_guest.
Should not introduce any functional changes.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/xc_hvm_build_x86.c | 198 ++++++++++++++++++++++++-----------------
 1 file changed, 117 insertions(+), 81 deletions(-)

diff --git a/tools/libxc/xc_hvm_build_x86.c b/tools/libxc/xc_hvm_build_x86.c
index 003ea06..f7616a8 100644
--- a/tools/libxc/xc_hvm_build_x86.c
+++ b/tools/libxc/xc_hvm_build_x86.c
@@ -232,28 +232,20 @@ static int check_mmio_hole(uint64_t start, uint64_t memsize,
         return 1;
 }
 
-static int setup_guest(xc_interface *xch,
-                       uint32_t dom, struct xc_hvm_build_args *args,
-                       char *image, unsigned long image_size)
+static int xc_hvm_populate_memory(xc_interface *xch, uint32_t dom,
+                                  struct xc_hvm_build_args *args,
+                                  xen_pfn_t *page_array)
 {
-    xen_pfn_t *page_array = NULL;
     unsigned long i, vmemid, nr_pages = args->mem_size >> PAGE_SHIFT;
     unsigned long p2m_size;
     unsigned long target_pages = args->mem_target >> PAGE_SHIFT;
-    unsigned long entry_eip, cur_pages, cur_pfn;
-    void *hvm_info_page;
-    uint32_t *ident_pt;
-    struct elf_binary elf;
-    uint64_t v_start, v_end;
-    uint64_t m_start = 0, m_end = 0;
+    unsigned long cur_pages, cur_pfn;
     int rc;
     xen_capabilities_info_t caps;
     unsigned long stat_normal_pages = 0, stat_2mb_pages = 0, 
         stat_1gb_pages = 0;
     unsigned int memflags = 0;
     int claim_enabled = args->claim_enabled;
-    xen_pfn_t special_array[NR_SPECIAL_PAGES];
-    xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
     uint64_t total_pages;
     xen_vmemrange_t dummy_vmemrange[2];
     unsigned int dummy_vnode_to_pnode[1];
@@ -261,19 +253,6 @@ static int setup_guest(xc_interface *xch,
     unsigned int *vnode_to_pnode;
     unsigned int nr_vmemranges, nr_vnodes;
 
-    memset(&elf, 0, sizeof(elf));
-    if ( elf_init(&elf, image, image_size) != 0 )
-    {
-        PERROR("Could not initialise ELF image");
-        goto error_out;
-    }
-
-    xc_elf_set_logfile(xch, &elf, 1);
-
-    elf_parse_binary(&elf);
-    v_start = 0;
-    v_end = args->mem_size;
-
     if ( nr_pages > target_pages )
         memflags |= XENMEMF_populate_on_demand;
 
@@ -346,24 +325,6 @@ static int setup_guest(xc_interface *xch,
         goto error_out;
     }
 
-    if ( modules_init(args, v_end, &elf, &m_start, &m_end) != 0 )
-    {
-        ERROR("Insufficient space to load modules.");
-        goto error_out;
-    }
-
-    DPRINTF("VIRTUAL MEMORY ARRANGEMENT:\n");
-    DPRINTF("  Loader:   %016"PRIx64"->%016"PRIx64"\n", elf.pstart, elf.pend);
-    DPRINTF("  Modules:  %016"PRIx64"->%016"PRIx64"\n", m_start, m_end);
-    DPRINTF("  TOTAL:    %016"PRIx64"->%016"PRIx64"\n", v_start, v_end);
-    DPRINTF("  ENTRY:    %016"PRIx64"\n", elf_uval(&elf, elf.ehdr, e_entry));
-
-    if ( (page_array = malloc(p2m_size * sizeof(xen_pfn_t))) == NULL )
-    {
-        PERROR("Could not allocate memory.");
-        goto error_out;
-    }
-
     for ( i = 0; i < p2m_size; i++ )
         page_array[i] = ((xen_pfn_t)-1);
     for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
@@ -563,7 +524,54 @@ static int setup_guest(xc_interface *xch,
     DPRINTF("  4KB PAGES: 0x%016lx\n", stat_normal_pages);
     DPRINTF("  2MB PAGES: 0x%016lx\n", stat_2mb_pages);
     DPRINTF("  1GB PAGES: 0x%016lx\n", stat_1gb_pages);
-    
+
+    rc = 0;
+    goto out;
+ error_out:
+    rc = -1;
+ out:
+
+    /* ensure no unclaimed pages are left unused */
+    xc_domain_claim_pages(xch, dom, 0 /* cancels the claim */);
+
+    return rc;
+}
+
+static int xc_hvm_load_image(xc_interface *xch,
+                       uint32_t dom, struct xc_hvm_build_args *args,
+                       xen_pfn_t *page_array)
+{
+    unsigned long entry_eip, image_size;
+    struct elf_binary elf;
+    uint64_t v_start, v_end;
+    uint64_t m_start = 0, m_end = 0;
+    char *image;
+    int rc;
+
+    image = xc_read_image(xch, args->image_file_name, &image_size);
+    if ( image == NULL )
+        return -1;
+
+    memset(&elf, 0, sizeof(elf));
+    if ( elf_init(&elf, image, image_size) != 0 )
+        goto error_out;
+
+    xc_elf_set_logfile(xch, &elf, 1);
+
+    elf_parse_binary(&elf);
+    v_start = 0;
+    v_end = args->mem_size;
+
+    if ( modules_init(args, v_end, &elf, &m_start, &m_end) != 0 )
+    {
+        ERROR("Insufficient space to load modules.");
+        goto error_out;
+    }
+
+    DPRINTF("VIRTUAL MEMORY ARRANGEMENT:\n");
+    DPRINTF("  Loader:   %016"PRIx64"->%016"PRIx64"\n", elf.pstart, elf.pend);
+    DPRINTF("  Modules:  %016"PRIx64"->%016"PRIx64"\n", m_start, m_end);
+
     if ( loadelfimage(xch, &elf, dom, page_array) != 0 )
     {
         PERROR("Could not load ELF image");
@@ -576,6 +584,44 @@ static int setup_guest(xc_interface *xch,
         goto error_out;
     }
 
+    /* Insert JMP <rel32> instruction at address 0x0 to reach entry point. */
+    entry_eip = elf_uval(&elf, elf.ehdr, e_entry);
+    if ( entry_eip != 0 )
+    {
+        char *page0 = xc_map_foreign_range(
+            xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, 0);
+        if ( page0 == NULL )
+            goto error_out;
+        page0[0] = 0xe9;
+        *(uint32_t *)&page0[1] = entry_eip - 5;
+        munmap(page0, PAGE_SIZE);
+    }
+
+    rc = 0;
+    goto out;
+ error_out:
+    rc = -1;
+ out:
+    if ( elf_check_broken(&elf) )
+        ERROR("HVM ELF broken: %s", elf_check_broken(&elf));
+    free(image);
+
+    return rc;
+}
+
+static int xc_hvm_populate_params(xc_interface *xch, uint32_t dom,
+                                  struct xc_hvm_build_args *args)
+{
+    unsigned long i;
+    void *hvm_info_page;
+    uint32_t *ident_pt;
+    uint64_t v_end;
+    int rc;
+    xen_pfn_t special_array[NR_SPECIAL_PAGES];
+    xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
+
+    v_end = args->mem_size;
+
     if ( (hvm_info_page = xc_map_foreign_range(
               xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE,
               HVM_INFO_PFN)) == NULL )
@@ -664,34 +710,12 @@ static int setup_guest(xc_interface *xch,
     xc_hvm_param_set(xch, dom, HVM_PARAM_IDENT_PT,
                      special_pfn(SPECIALPAGE_IDENT_PT) << PAGE_SHIFT);
 
-    /* Insert JMP <rel32> instruction at address 0x0 to reach entry point. */
-    entry_eip = elf_uval(&elf, elf.ehdr, e_entry);
-    if ( entry_eip != 0 )
-    {
-        char *page0 = xc_map_foreign_range(
-            xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, 0);
-        if ( page0 == NULL )
-        {
-            PERROR("Could not map page0");
-            goto error_out;
-        }
-        page0[0] = 0xe9;
-        *(uint32_t *)&page0[1] = entry_eip - 5;
-        munmap(page0, PAGE_SIZE);
-    }
-
     rc = 0;
     goto out;
  error_out:
     rc = -1;
  out:
-    if ( elf_check_broken(&elf) )
-        ERROR("HVM ELF broken: %s", elf_check_broken(&elf));
-
-    /* ensure no unclaimed pages are left unused */
-    xc_domain_claim_pages(xch, dom, 0 /* cancels the claim */);
 
-    free(page_array);
     return rc;
 }
 
@@ -702,9 +726,8 @@ int xc_hvm_build(xc_interface *xch, uint32_t domid,
                  struct xc_hvm_build_args *hvm_args)
 {
     struct xc_hvm_build_args args = *hvm_args;
-    void *image;
-    unsigned long image_size;
-    int sts;
+    xen_pfn_t *parray = NULL;
+    int rc;
 
     if ( domid == 0 )
         return -1;
@@ -715,24 +738,37 @@ int xc_hvm_build(xc_interface *xch, uint32_t domid,
     if ( args.mem_size < (2ull << 20) || args.mem_target < (2ull << 20) )
         return -1;
 
-    image = xc_read_image(xch, args.image_file_name, &image_size);
-    if ( image == NULL )
+    parray = malloc((args.mem_size >> PAGE_SHIFT) * sizeof(xen_pfn_t));
+    if ( parray == NULL )
         return -1;
 
-    sts = setup_guest(xch, domid, &args, image, image_size);
-
-    if (!sts)
+    rc = xc_hvm_populate_memory(xch, domid, &args, parray);
+    if ( rc != 0 )
     {
-        /* Return module load addresses to caller */
-        hvm_args->acpi_module.guest_addr_out = 
-            args.acpi_module.guest_addr_out;
-        hvm_args->smbios_module.guest_addr_out = 
-            args.smbios_module.guest_addr_out;
+        PERROR("xc_hvm_populate_memory failed");
+        goto out;
+    }
+    rc = xc_hvm_load_image(xch, domid, &args, parray);
+    if ( rc != 0 )
+    {
+        PERROR("xc_hvm_load_image failed");
+        goto out;
+    }
+    rc = xc_hvm_populate_params(xch, domid, &args);
+    if ( rc != 0 )
+    {
+        PERROR("xc_hvm_populate_params failed");
+        goto out;
     }
 
-    free(image);
+    /* Return module load addresses to caller */
+    hvm_args->acpi_module.guest_addr_out = args.acpi_module.guest_addr_out;
+    hvm_args->smbios_module.guest_addr_out = args.smbios_module.guest_addr_out;
 
-    return sts;
+out:
+    free(parray);
+
+    return rc;
 }
 
 /* xc_hvm_build_target_mem: 
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 01/32] libxc: split x86 HVM setup_guest into smaller logical functions

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> This is just a preparatory change to clean up the code in setup_guest.
> Should not introduce any functional changes.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>, with one note.

> @@ -576,6 +584,44 @@ static int setup_guest(xc_interface *xch,
>          goto error_out;
>      }
>  
> +    /* Insert JMP <rel32> instruction at address 0x0 to reach entry point. */
> +    entry_eip = elf_uval(&elf, elf.ehdr, e_entry);
> +    if ( entry_eip != 0 )
> +    {
> +        char *page0 = xc_map_foreign_range(
> +            xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, 0);
> +        if ( page0 == NULL )
> +            goto error_out;
> +        page0[0] = 0xe9;
> +        *(uint32_t *)&page0[1] = entry_eip - 5;
> +        munmap(page0, PAGE_SIZE);
> +    }

With this series eventually enabling DOMCTL_setvcpucontext properly for
HVM vcpus, this above juggling can be replaced by setting the starting
eip appropriately, which is neater than stuffing a hand-coded
instruction in at 0.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 01/32] libxc: split x86 HVM setup_guest into smaller logical functions

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:39PM +0200, Roger Pau Monne wrote:
> This is just a preparatory change to clean up the code in setup_guest.
> Should not introduce any functional changes.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

[PATCH v3 02/32] libxc: unify xc_dom_p2m_{host/guest}

[Roger Pau Monne]

Unify both functions into xc_dom_p2m. Should not introduce any functional
change.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>
---
 stubdom/grub/kexec.c              |  4 ++--
 tools/libxc/include/xc_dom.h      | 14 ++------------
 tools/libxc/xc_dom_boot.c         | 10 +++++-----
 tools/libxc/xc_dom_compat_linux.c |  4 ++--
 tools/libxc/xc_dom_x86.c          | 32 ++++++++++++++++----------------
 tools/libxl/libxl_dom.c           |  4 ++--
 6 files changed, 29 insertions(+), 39 deletions(-)

diff --git a/stubdom/grub/kexec.c b/stubdom/grub/kexec.c
index 4c33b25..0b2f4f3 100644
--- a/stubdom/grub/kexec.c
+++ b/stubdom/grub/kexec.c
@@ -358,9 +358,9 @@ void kexec(void *kernel, long kernel_size, void *module, long module_size, char
 #ifdef __x86_64__
                 MMUEXT_PIN_L4_TABLE,
 #endif
-                xc_dom_p2m_host(dom, dom->pgtables_seg.pfn),
+                xc_dom_p2m(dom, dom->pgtables_seg.pfn),
                 dom->guest_domid)) != 0 ) {
-        grub_printf("pin_table(%lx) returned %d\n", xc_dom_p2m_host(dom,
+        grub_printf("pin_table(%lx) returned %d\n", xc_dom_p2m(dom,
                     dom->pgtables_seg.pfn), rc);
         errnum = ERR_BOOT_FAILURE;
         goto out_remap;
diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index a7d059a..ec9e293 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -376,19 +376,9 @@ static inline void *xc_dom_vaddr_to_ptr(struct xc_dom_image *dom,
     return ptr + offset;
 }
 
-static inline xen_pfn_t xc_dom_p2m_host(struct xc_dom_image *dom, xen_pfn_t pfn)
+static inline xen_pfn_t xc_dom_p2m(struct xc_dom_image *dom, xen_pfn_t pfn)
 {
-    if (dom->shadow_enabled)
-        return pfn;
-    if (pfn < dom->rambase_pfn || pfn >= dom->rambase_pfn + dom->total_pages)
-        return INVALID_MFN;
-    return dom->p2m_host[pfn - dom->rambase_pfn];
-}
-
-static inline xen_pfn_t xc_dom_p2m_guest(struct xc_dom_image *dom,
-                                         xen_pfn_t pfn)
-{
-    if (xc_dom_feature_translated(dom))
+    if ( dom->shadow_enabled || xc_dom_feature_translated(dom) )
         return pfn;
     if (pfn < dom->rambase_pfn || pfn >= dom->rambase_pfn + dom->total_pages)
         return INVALID_MFN;
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index f82db2d..fda9e52 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -54,7 +54,7 @@ static int setup_hypercall_page(struct xc_dom_image *dom)
                   dom->parms.virt_hypercall, pfn);
     domctl.cmd = XEN_DOMCTL_hypercall_init;
     domctl.domain = dom->guest_domid;
-    domctl.u.hypercall_init.gmfn = xc_dom_p2m_guest(dom, pfn);
+    domctl.u.hypercall_init.gmfn = xc_dom_p2m(dom, pfn);
     rc = do_domctl(dom->xch, &domctl);
     if ( rc != 0 )
         xc_dom_panic(dom->xch, XC_INTERNAL_ERROR,
@@ -84,7 +84,7 @@ static int clear_page(struct xc_dom_image *dom, xen_pfn_t pfn)
     if ( pfn == 0 )
         return 0;
 
-    dst = xc_dom_p2m_host(dom, pfn);
+    dst = xc_dom_p2m(dom, pfn);
     DOMPRINTF("%s: pfn 0x%" PRIpfn ", mfn 0x%" PRIpfn "",
               __FUNCTION__, pfn, dst);
     rc = xc_clear_domain_page(dom->xch, dom->guest_domid, dst);
@@ -178,7 +178,7 @@ void *xc_dom_boot_domU_map(struct xc_dom_image *dom, xen_pfn_t pfn,
     }
 
     for ( i = 0; i < count; i++ )
-        entries[i].mfn = xc_dom_p2m_host(dom, pfn + i);
+        entries[i].mfn = xc_dom_p2m(dom, pfn + i);
 
     ptr = xc_map_foreign_ranges(dom->xch, dom->guest_domid,
                 count << page_shift, PROT_READ | PROT_WRITE, 1 << page_shift,
@@ -435,8 +435,8 @@ int xc_dom_gnttab_init(struct xc_dom_image *dom)
                                       dom->console_domid, dom->xenstore_domid);
     } else {
         return xc_dom_gnttab_seed(dom->xch, dom->guest_domid,
-                                  xc_dom_p2m_host(dom, dom->console_pfn),
-                                  xc_dom_p2m_host(dom, dom->xenstore_pfn),
+                                  xc_dom_p2m(dom, dom->console_pfn),
+                                  xc_dom_p2m(dom, dom->xenstore_pfn),
                                   dom->console_domid, dom->xenstore_domid);
     }
 }
diff --git a/tools/libxc/xc_dom_compat_linux.c b/tools/libxc/xc_dom_compat_linux.c
index 2c14a0f..acccde9 100644
--- a/tools/libxc/xc_dom_compat_linux.c
+++ b/tools/libxc/xc_dom_compat_linux.c
@@ -65,8 +65,8 @@ static int xc_linux_build_internal(struct xc_dom_image *dom,
     if ( (rc = xc_dom_gnttab_init(dom)) != 0)
         goto out;
 
-    *console_mfn = xc_dom_p2m_host(dom, dom->console_pfn);
-    *store_mfn = xc_dom_p2m_host(dom, dom->xenstore_pfn);
+    *console_mfn = xc_dom_p2m(dom, dom->console_pfn);
+    *store_mfn = xc_dom_p2m(dom, dom->xenstore_pfn);
 
  out:
     return rc;
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 920fe42..0d80c18 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -248,7 +248,7 @@ static int setup_pgtables_x86_32_pae(struct xc_dom_image *dom)
     unsigned long l3off, l2off = 0, l1off;
     xen_vaddr_t addr;
     xen_pfn_t pgpfn;
-    xen_pfn_t l3mfn = xc_dom_p2m_guest(dom, l3pfn);
+    xen_pfn_t l3mfn = xc_dom_p2m(dom, l3pfn);
 
     if ( dom->parms.pae == 1 )
     {
@@ -280,7 +280,7 @@ static int setup_pgtables_x86_32_pae(struct xc_dom_image *dom)
                 goto pfn_error;
             l3off = l3_table_offset_pae(addr);
             l3tab[l3off] =
-                pfn_to_paddr(xc_dom_p2m_guest(dom, l2pfn)) | L3_PROT;
+                pfn_to_paddr(xc_dom_p2m(dom, l2pfn)) | L3_PROT;
             l2pfn++;
         }
 
@@ -292,7 +292,7 @@ static int setup_pgtables_x86_32_pae(struct xc_dom_image *dom)
                 goto pfn_error;
             l2off = l2_table_offset_pae(addr);
             l2tab[l2off] =
-                pfn_to_paddr(xc_dom_p2m_guest(dom, l1pfn)) | L2_PROT;
+                pfn_to_paddr(xc_dom_p2m(dom, l1pfn)) | L2_PROT;
             l1pfn++;
         }
 
@@ -300,7 +300,7 @@ static int setup_pgtables_x86_32_pae(struct xc_dom_image *dom)
         l1off = l1_table_offset_pae(addr);
         pgpfn = (addr - dom->parms.virt_base) >> PAGE_SHIFT_X86;
         l1tab[l1off] =
-            pfn_to_paddr(xc_dom_p2m_guest(dom, pgpfn)) | L1_PROT;
+            pfn_to_paddr(xc_dom_p2m(dom, pgpfn)) | L1_PROT;
         if ( (addr >= dom->pgtables_seg.vstart) &&
              (addr < dom->pgtables_seg.vend) )
             l1tab[l1off] &= ~_PAGE_RW; /* page tables are r/o */
@@ -316,7 +316,7 @@ static int setup_pgtables_x86_32_pae(struct xc_dom_image *dom)
     if ( dom->virt_pgtab_end <= 0xc0000000 )
     {
         DOMPRINTF("%s: PAE: extra l2 page table for l3#3", __FUNCTION__);
-        l3tab[3] = pfn_to_paddr(xc_dom_p2m_guest(dom, l2pfn)) | L3_PROT;
+        l3tab[3] = pfn_to_paddr(xc_dom_p2m(dom, l2pfn)) | L3_PROT;
     }
     return 0;
 
@@ -375,7 +375,7 @@ static int setup_pgtables_x86_64(struct xc_dom_image *dom)
                 goto pfn_error;
             l4off = l4_table_offset_x86_64(addr);
             l4tab[l4off] =
-                pfn_to_paddr(xc_dom_p2m_guest(dom, l3pfn)) | L4_PROT;
+                pfn_to_paddr(xc_dom_p2m(dom, l3pfn)) | L4_PROT;
             l3pfn++;
         }
 
@@ -387,7 +387,7 @@ static int setup_pgtables_x86_64(struct xc_dom_image *dom)
                 goto pfn_error;
             l3off = l3_table_offset_x86_64(addr);
             l3tab[l3off] =
-                pfn_to_paddr(xc_dom_p2m_guest(dom, l2pfn)) | L3_PROT;
+                pfn_to_paddr(xc_dom_p2m(dom, l2pfn)) | L3_PROT;
             l2pfn++;
         }
 
@@ -399,7 +399,7 @@ static int setup_pgtables_x86_64(struct xc_dom_image *dom)
                 goto pfn_error;
             l2off = l2_table_offset_x86_64(addr);
             l2tab[l2off] =
-                pfn_to_paddr(xc_dom_p2m_guest(dom, l1pfn)) | L2_PROT;
+                pfn_to_paddr(xc_dom_p2m(dom, l1pfn)) | L2_PROT;
             l1pfn++;
         }
 
@@ -407,7 +407,7 @@ static int setup_pgtables_x86_64(struct xc_dom_image *dom)
         l1off = l1_table_offset_x86_64(addr);
         pgpfn = (addr - dom->parms.virt_base) >> PAGE_SHIFT_X86;
         l1tab[l1off] =
-            pfn_to_paddr(xc_dom_p2m_guest(dom, pgpfn)) | L1_PROT;
+            pfn_to_paddr(xc_dom_p2m(dom, pgpfn)) | L1_PROT;
         if ( (!dom->pvh_enabled)                &&
              (addr >= dom->pgtables_seg.vstart) &&
              (addr < dom->pgtables_seg.vend) )
@@ -490,9 +490,9 @@ static int start_info_x86_32(struct xc_dom_image *dom)
     start_info->mfn_list = dom->p2m_seg.vstart;
 
     start_info->flags = dom->flags;
-    start_info->store_mfn = xc_dom_p2m_guest(dom, dom->xenstore_pfn);
+    start_info->store_mfn = xc_dom_p2m(dom, dom->xenstore_pfn);
     start_info->store_evtchn = dom->xenstore_evtchn;
-    start_info->console.domU.mfn = xc_dom_p2m_guest(dom, dom->console_pfn);
+    start_info->console.domU.mfn = xc_dom_p2m(dom, dom->console_pfn);
     start_info->console.domU.evtchn = dom->console_evtchn;
 
     if ( dom->ramdisk_blob )
@@ -536,9 +536,9 @@ static int start_info_x86_64(struct xc_dom_image *dom)
     start_info->mfn_list = dom->p2m_seg.vstart;
 
     start_info->flags = dom->flags;
-    start_info->store_mfn = xc_dom_p2m_guest(dom, dom->xenstore_pfn);
+    start_info->store_mfn = xc_dom_p2m(dom, dom->xenstore_pfn);
     start_info->store_evtchn = dom->xenstore_evtchn;
-    start_info->console.domU.mfn = xc_dom_p2m_guest(dom, dom->console_pfn);
+    start_info->console.domU.mfn = xc_dom_p2m(dom, dom->console_pfn);
     start_info->console.domU.evtchn = dom->console_evtchn;
 
     if ( dom->ramdisk_blob )
@@ -622,7 +622,7 @@ static int vcpu_x86_32(struct xc_dom_image *dom, void *ptr)
          dom->parms.pae == 3 /* bimodal */ )
         ctxt->vm_assist |= (1UL << VMASST_TYPE_pae_extended_cr3);
 
-    cr3_pfn = xc_dom_p2m_guest(dom, dom->pgtables_seg.pfn);
+    cr3_pfn = xc_dom_p2m(dom, dom->pgtables_seg.pfn);
     ctxt->ctrlreg[3] = xen_pfn_to_cr3_x86_32(cr3_pfn);
     DOMPRINTF("%s: cr3: pfn 0x%" PRIpfn " mfn 0x%" PRIpfn "",
               __FUNCTION__, dom->pgtables_seg.pfn, cr3_pfn);
@@ -648,7 +648,7 @@ static int vcpu_x86_64(struct xc_dom_image *dom, void *ptr)
     ctxt->user_regs.rflags = 1 << 9; /* Interrupt Enable */
 
     ctxt->flags = VGCF_in_kernel_X86_64 | VGCF_online_X86_64;
-    cr3_pfn = xc_dom_p2m_guest(dom, dom->pgtables_seg.pfn);
+    cr3_pfn = xc_dom_p2m(dom, dom->pgtables_seg.pfn);
     ctxt->ctrlreg[3] = xen_pfn_to_cr3_x86_64(cr3_pfn);
     DOMPRINTF("%s: cr3: pfn 0x%" PRIpfn " mfn 0x%" PRIpfn "",
               __FUNCTION__, dom->pgtables_seg.pfn, cr3_pfn);
@@ -1020,7 +1020,7 @@ int arch_setup_bootlate(struct xc_dom_image *dom)
         /* paravirtualized guest */
         xc_dom_unmap_one(dom, dom->pgtables_seg.pfn);
         rc = pin_table(dom->xch, pgd_type,
-                       xc_dom_p2m_host(dom, dom->pgtables_seg.pfn),
+                       xc_dom_p2m(dom, dom->pgtables_seg.pfn),
                        dom->guest_domid);
         if ( rc != 0 )
         {
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 1c9418a..fb2f916 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -745,8 +745,8 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
         state->console_mfn = dom->console_pfn;
         state->store_mfn = dom->xenstore_pfn;
     } else {
-        state->console_mfn = xc_dom_p2m_host(dom, dom->console_pfn);
-        state->store_mfn = xc_dom_p2m_host(dom, dom->xenstore_pfn);
+        state->console_mfn = xc_dom_p2m(dom, dom->console_pfn);
+        state->store_mfn = xc_dom_p2m(dom, dom->xenstore_pfn);
     }
 
     libxl__file_reference_unmap(&state->pv_kernel);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 02/32] libxc: unify xc_dom_p2m_{host/guest}

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> Unify both functions into xc_dom_p2m. Should not introduce any functional
> change.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 02/32] libxc: unify xc_dom_p2m_{host/guest}

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:40PM +0200, Roger Pau Monne wrote:
> Unify both functions into xc_dom_p2m. Should not introduce any functional
> change.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> Cc: Samuel Thibault <samuel.thibault@ens-lyon.org>

Acked-by: Wei Liu <wei.liu2@citrix.com>

[PATCH v3 03/32] libxc: introduce the notion of a container type

[Roger Pau Monne]

Introduce the notion of a container type into xc_dom_image. This will be
needed by later changes that will also use xc_dom_image in order to build
HVM guests.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/include/xc_dom.h | 6 ++++++
 tools/libxc/xc_dom_x86.c     | 4 ++++
 tools/libxl/libxl_dom.c      | 1 +
 3 files changed, 11 insertions(+)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index ec9e293..f7b5f0f 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -180,6 +180,12 @@ struct xc_dom_image {
     struct xc_dom_arch *arch_hooks;
     /* allocate up to virt_alloc_end */
     int (*allocate) (struct xc_dom_image * dom, xen_vaddr_t up_to);
+
+    /* Container type (HVM or PV). */
+    enum {
+        XC_DOM_PV_CONTAINER,
+        XC_DOM_HVM_CONTAINER,
+    } container_type;
 };
 
 /* --- pluggable kernel loader ------------------------------------- */
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 0d80c18..b89f5c2 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -1071,6 +1071,10 @@ int arch_setup_bootlate(struct xc_dom_image *dom)
 
 int xc_dom_feature_translated(struct xc_dom_image *dom)
 {
+    /* Guests running inside HVM containers are always auto-translated. */
+    if ( dom->container_type == XC_DOM_HVM_CONTAINER )
+        return 1;
+
     return elf_xen_feature_get(XENFEAT_auto_translated_physmap, dom->f_active);
 }
 
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index fb2f916..2bae277 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -626,6 +626,7 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
     }
 
     dom->pvh_enabled = state->pvh_enabled;
+    dom->container_type = XC_DOM_PV_CONTAINER;
 
     LOG(DEBUG, "pv kernel mapped %d path %s", state->pv_kernel.mapped, state->pv_kernel.path);
 
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 03/32] libxc: introduce the notion of a container type

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> Introduce the notion of a container type into xc_dom_image. This will be
> needed by later changes that will also use xc_dom_image in order to build
> HVM guests.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 03/32] libxc: introduce the notion of a container type

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:41PM +0200, Roger Pau Monne wrote:
> Introduce the notion of a container type into xc_dom_image. This will be
> needed by later changes that will also use xc_dom_image in order to build
> HVM guests.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

[PATCH v3 04/32] libxc: introduce a domain loader for HVM guest firmware

[Roger Pau Monne]

Introduce a very simple (and dummy) domain loader to be used to load the
firmware (hvmloader) into HVM guests. Since hmvloader is just a 32bit elf
executable the loader is fairly simple.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/Makefile           |   1 +
 tools/libxc/include/xc_dom.h   |   8 ++
 tools/libxc/xc_dom_hvmloader.c | 316 +++++++++++++++++++++++++++++++++++++++++
 xen/include/xen/libelf.h       |   1 +
 4 files changed, 326 insertions(+)
 create mode 100644 tools/libxc/xc_dom_hvmloader.c

diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index 153b79e..48fc473 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -87,6 +87,7 @@ GUEST_SRCS-y                 += xc_dom_core.c xc_dom_boot.c
 GUEST_SRCS-y                 += xc_dom_elfloader.c
 GUEST_SRCS-$(CONFIG_X86)     += xc_dom_bzimageloader.c
 GUEST_SRCS-$(CONFIG_X86)     += xc_dom_decompress_lz4.c
+GUEST_SRCS-$(CONFIG_X86)     += xc_dom_hvmloader.c
 GUEST_SRCS-$(CONFIG_ARM)     += xc_dom_armzimageloader.c
 GUEST_SRCS-y                 += xc_dom_binloader.c
 GUEST_SRCS-y                 += xc_dom_compat_linux.c
diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index f7b5f0f..d61482c 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -15,6 +15,7 @@
  */
 
 #include <xen/libelf/libelf.h>
+#include <xenguest.h>
 
 #define INVALID_P2M_ENTRY   ((xen_pfn_t)-1)
 
@@ -186,6 +187,13 @@ struct xc_dom_image {
         XC_DOM_PV_CONTAINER,
         XC_DOM_HVM_CONTAINER,
     } container_type;
+
+    /* HVM specific fields. */
+    /* Extra ACPI tables passed to HVMLOADER */
+    struct xc_hvm_firmware_module acpi_module;
+
+    /* Extra SMBIOS structures passed to HVMLOADER */
+    struct xc_hvm_firmware_module smbios_module;
 };
 
 /* --- pluggable kernel loader ------------------------------------- */
diff --git a/tools/libxc/xc_dom_hvmloader.c b/tools/libxc/xc_dom_hvmloader.c
new file mode 100644
index 0000000..b6270c9
--- /dev/null
+++ b/tools/libxc/xc_dom_hvmloader.c
@@ -0,0 +1,316 @@
+/*
+ * Xen domain builder -- HVM specific bits.
+ *
+ * Parse and load ELF firmware images for HVM domains.
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation;
+ * version 2.1 of the License.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
+ *
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdarg.h>
+#include <inttypes.h>
+#include <assert.h>
+
+#include "xg_private.h"
+#include "xc_dom.h"
+#include "xc_bitops.h"
+
+/* ------------------------------------------------------------------------ */
+/* parse elf binary                                                         */
+
+static elf_negerrnoval check_elf_kernel(struct xc_dom_image *dom, bool verbose)
+{
+    if ( dom->kernel_blob == NULL )
+    {
+        if ( verbose )
+            xc_dom_panic(dom->xch,
+                         XC_INTERNAL_ERROR, "%s: no kernel image loaded",
+                         __FUNCTION__);
+        return -EINVAL;
+    }
+
+    if ( !elf_is_elfbinary(dom->kernel_blob, dom->kernel_size) )
+    {
+        if ( verbose )
+            xc_dom_panic(dom->xch,
+                         XC_INVALID_KERNEL, "%s: kernel is not an ELF image",
+                         __FUNCTION__);
+        return -EINVAL;
+    }
+    return 0;
+}
+
+static elf_negerrnoval xc_dom_probe_hvm_kernel(struct xc_dom_image *dom)
+{
+    struct elf_binary elf;
+    int rc;
+
+    /* This loader is designed for HVM guest firmware. */
+    if ( dom->container_type != XC_DOM_HVM_CONTAINER )
+        return -EINVAL;
+
+    rc = check_elf_kernel(dom, 0);
+    if ( rc != 0 )
+        return rc;
+
+    rc = elf_init(&elf, dom->kernel_blob, dom->kernel_size);
+    if ( rc != 0 )
+        return rc;
+
+    /*
+     * We need to check that there are no Xen ELFNOTES, or
+     * else we might be trying to load a PV kernel.
+     */
+    elf_parse_binary(&elf);
+    rc = elf_xen_parse(&elf, &dom->parms);
+    if ( rc == 0 )
+        return -EINVAL;
+
+    return 0;
+}
+
+static elf_errorstatus xc_dom_parse_hvm_kernel(struct xc_dom_image *dom)
+    /*
+     * This function sometimes returns -1 for error and sometimes
+     * an errno value.  ?!?!
+     */
+{
+    struct elf_binary *elf;
+    elf_errorstatus rc;
+
+    rc = check_elf_kernel(dom, 1);
+    if ( rc != 0 )
+        return rc;
+
+    elf = xc_dom_malloc(dom, sizeof(*elf));
+    if ( elf == NULL )
+        return -1;
+    dom->private_loader = elf;
+    rc = elf_init(elf, dom->kernel_blob, dom->kernel_size);
+    xc_elf_set_logfile(dom->xch, elf, 1);
+    if ( rc != 0 )
+    {
+        xc_dom_panic(dom->xch, XC_INVALID_KERNEL, "%s: corrupted ELF image",
+                     __FUNCTION__);
+        return rc;
+    }
+
+    if ( !elf_32bit(elf) )
+    {
+        xc_dom_panic(dom->xch, XC_INVALID_KERNEL, "%s: ELF image is not 32bit",
+                     __FUNCTION__);
+        return -EINVAL;
+    }
+
+    /* parse binary and get xen meta info */
+    elf_parse_binary(elf);
+
+    /* find kernel segment */
+    dom->kernel_seg.vstart = elf->pstart;
+    dom->kernel_seg.vend   = elf->pend;
+
+    dom->guest_type = "hvm-3.0-x86_32";
+
+    if ( elf_check_broken(elf) )
+        DOMPRINTF("%s: ELF broken: %s", __FUNCTION__,
+                  elf_check_broken(elf));
+
+    return rc;
+}
+
+static int modules_init(struct xc_dom_image *dom,
+                        uint64_t vend, struct elf_binary *elf,
+                        uint64_t *mstart_out, uint64_t *mend_out)
+{
+#define MODULE_ALIGN 1UL << 7
+#define MB_ALIGN     1UL << 20
+#define MKALIGN(x, a) (((uint64_t)(x) + (a) - 1) & ~(uint64_t)((a) - 1))
+    uint64_t total_len = 0, offset1 = 0;
+
+    if ( (dom->acpi_module.length == 0) && (dom->smbios_module.length == 0) )
+        return 0;
+
+    /* Find the total length for the firmware modules with a reasonable large
+     * alignment size to align each the modules.
+     */
+    total_len = MKALIGN(dom->acpi_module.length, MODULE_ALIGN);
+    offset1 = total_len;
+    total_len += MKALIGN(dom->smbios_module.length, MODULE_ALIGN);
+
+    /* Want to place the modules 1Mb+change behind the loader image. */
+    *mstart_out = MKALIGN(elf->pend, MB_ALIGN) + (MB_ALIGN);
+    *mend_out = *mstart_out + total_len;
+
+    if ( *mend_out > vend )
+        return -1;
+
+    if ( dom->acpi_module.length != 0 )
+        dom->acpi_module.guest_addr_out = *mstart_out;
+    if ( dom->smbios_module.length != 0 )
+        dom->smbios_module.guest_addr_out = *mstart_out + offset1;
+
+    return 0;
+}
+
+static int loadmodules(struct xc_dom_image *dom,
+                       uint64_t mstart, uint64_t mend,
+                       uint32_t domid)
+{
+    privcmd_mmap_entry_t *entries = NULL;
+    unsigned long pfn_start;
+    unsigned long pfn_end;
+    size_t pages;
+    uint32_t i;
+    uint8_t *dest;
+    int rc = -1;
+    xc_interface *xch = dom->xch;
+
+    if ( (mstart == 0)||(mend == 0) )
+        return 0;
+
+    pfn_start = (unsigned long)(mstart >> PAGE_SHIFT);
+    pfn_end = (unsigned long)((mend + PAGE_SIZE - 1) >> PAGE_SHIFT);
+    pages = pfn_end - pfn_start;
+
+    /* Map address space for module list. */
+    entries = calloc(pages, sizeof(privcmd_mmap_entry_t));
+    if ( entries == NULL )
+        goto error_out;
+
+    for ( i = 0; i < pages; i++ )
+        entries[i].mfn = (mstart >> PAGE_SHIFT) + i;
+
+    dest = xc_map_foreign_ranges(
+        xch, domid, pages << PAGE_SHIFT, PROT_READ | PROT_WRITE, 1 << PAGE_SHIFT,
+        entries, pages);
+    if ( dest == NULL )
+        goto error_out;
+
+    /* Zero the range so padding is clear between modules */
+    memset(dest, 0, pages << PAGE_SHIFT);
+
+    /* Load modules into range */
+    if ( dom->acpi_module.length != 0 )
+    {
+        memcpy(dest,
+               dom->acpi_module.data,
+               dom->acpi_module.length);
+    }
+    if ( dom->smbios_module.length != 0 )
+    {
+        memcpy(dest + (dom->smbios_module.guest_addr_out - mstart),
+               dom->smbios_module.data,
+               dom->smbios_module.length);
+    }
+
+    munmap(dest, pages << PAGE_SHIFT);
+    rc = 0;
+
+ error_out:
+    free(entries);
+
+    return rc;
+}
+
+static elf_errorstatus xc_dom_load_hvm_kernel(struct xc_dom_image *dom)
+{
+    struct elf_binary *elf = dom->private_loader;
+    privcmd_mmap_entry_t *entries = NULL;
+    size_t pages = (elf->pend - elf->pstart + PAGE_SIZE - 1) >> PAGE_SHIFT;
+    elf_errorstatus rc;
+    uint64_t m_start = 0, m_end = 0;
+    int i;
+
+    /* Map address space for initial elf image. */
+    entries = calloc(pages, sizeof(privcmd_mmap_entry_t));
+    if ( entries == NULL )
+        return -ENOMEM;
+
+    for ( i = 0; i < pages; i++ )
+        entries[i].mfn = (elf->pstart >> PAGE_SHIFT) + i;
+
+    elf->dest_base = xc_map_foreign_ranges(
+        dom->xch, dom->guest_domid, pages << PAGE_SHIFT,
+        PROT_READ | PROT_WRITE, 1 << PAGE_SHIFT,
+        entries, pages);
+    if ( elf->dest_base == NULL )
+    {
+        DOMPRINTF("%s: unable to map guest memory space", __FUNCTION__);
+        rc = -EFAULT;
+        goto error;
+    }
+
+    elf->dest_size = pages * XC_DOM_PAGE_SIZE(dom);
+
+    rc = elf_load_binary(elf);
+    if ( rc < 0 )
+    {
+        DOMPRINTF("%s: failed to load elf binary", __FUNCTION__);
+        return rc;
+    }
+
+    munmap(elf->dest_base, elf->dest_size);
+
+    rc = modules_init(dom, dom->total_pages << PAGE_SHIFT, elf, &m_start,
+                      &m_end);
+    if ( rc != 0 )
+    {
+        DOMPRINTF("%s: insufficient space to load modules.", __FUNCTION__);
+        return rc;
+    }
+
+    rc = loadmodules(dom, m_start, m_end, dom->guest_domid);
+    if ( rc != 0 )
+    {
+        DOMPRINTF("%s: unable to load modules.", __FUNCTION__);
+        return rc;
+    }
+
+    dom->parms.phys_entry = elf_uval(elf, elf->ehdr, e_entry);
+
+    free(entries);
+    return 0;
+
+error:
+    assert(rc != 0);
+    free(entries);
+    return rc;
+}
+
+/* ------------------------------------------------------------------------ */
+
+struct xc_dom_loader hvm_loader = {
+    .name = "HVM-generic",
+    .probe = xc_dom_probe_hvm_kernel,
+    .parser = xc_dom_parse_hvm_kernel,
+    .loader = xc_dom_load_hvm_kernel,
+};
+
+static void __init register_loader(void)
+{
+    xc_dom_register_loader(&hvm_loader);
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/xen/libelf.h b/xen/include/xen/libelf.h
index 6393040..03d449e 100644
--- a/xen/include/xen/libelf.h
+++ b/xen/include/xen/libelf.h
@@ -424,6 +424,7 @@ struct elf_dom_parms {
     uint64_t elf_paddr_offset;
     uint32_t f_supported[XENFEAT_NR_SUBMAPS];
     uint32_t f_required[XENFEAT_NR_SUBMAPS];
+    uint32_t phys_entry;
 
     /* calculated */
     uint64_t virt_offset;
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 04/32] libxc: introduce a domain loader for HVM guest firmware

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> Introduce a very simple (and dummy) domain loader to be used to load the
> firmware (hvmloader) into HVM guests. Since hmvloader is just a 32bit elf
> executable the loader is fairly simple.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Looks plausible.

I would s/__FUNCTION__/__func__/g throughout, and there are several
coding style issues which could do with fixing.

Otherwise, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 04/32] libxc: introduce a domain loader for HVM guest firmware

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:42PM +0200, Roger Pau Monne wrote:
> Introduce a very simple (and dummy) domain loader to be used to load the
> firmware (hvmloader) into HVM guests. Since hmvloader is just a 32bit elf
> executable the loader is fairly simple.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

[PATCH v3 05/32] libxc: make arch_setup_meminit a xc_dom_arch hook

[Roger Pau Monne]

This allows having different arch_setup_meminit implementations based on the
guest type. It should not introduce any functional changes.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/include/xc_dom.h |  4 ++-
 tools/libxc/xc_dom_arm.c     | 70 +++++++++++++++++++++++--------------------
 tools/libxc/xc_dom_boot.c    |  2 +-
 tools/libxc/xc_dom_x86.c     | 71 +++++++++++++++++++++++---------------------
 4 files changed, 78 insertions(+), 69 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index d61482c..b461f19 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -224,6 +224,9 @@ struct xc_dom_arch {
     int (*shared_info) (struct xc_dom_image * dom, void *shared_info);
     int (*vcpu) (struct xc_dom_image * dom, void *vcpu_ctxt);
 
+    /* arch-specific memory initialization. */
+    int (*meminit) (struct xc_dom_image * dom);
+
     char *guest_type;
     char *native_protocol;
     int page_shift;
@@ -401,7 +404,6 @@ static inline xen_pfn_t xc_dom_p2m(struct xc_dom_image *dom, xen_pfn_t pfn)
 
 /* --- arch bits --------------------------------------------------- */
 
-int arch_setup_meminit(struct xc_dom_image *dom);
 int arch_setup_bootearly(struct xc_dom_image *dom);
 int arch_setup_bootlate(struct xc_dom_image *dom);
 
diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index 065debb..1f00d05 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -195,38 +195,6 @@ static int vcpu_arm64(struct xc_dom_image *dom, void *ptr)
 
 /* ------------------------------------------------------------------------ */
 
-static struct xc_dom_arch xc_dom_32 = {
-    .guest_type = "xen-3.0-armv7l",
-    .native_protocol = XEN_IO_PROTO_ABI_ARM,
-    .page_shift = PAGE_SHIFT_ARM,
-    .sizeof_pfn = 8,
-    .alloc_magic_pages = alloc_magic_pages,
-    .count_pgtables = count_pgtables_arm,
-    .setup_pgtables = setup_pgtables_arm,
-    .start_info = start_info_arm,
-    .shared_info = shared_info_arm,
-    .vcpu = vcpu_arm32,
-};
-
-static struct xc_dom_arch xc_dom_64 = {
-    .guest_type = "xen-3.0-aarch64",
-    .native_protocol = XEN_IO_PROTO_ABI_ARM,
-    .page_shift = PAGE_SHIFT_ARM,
-    .sizeof_pfn = 8,
-    .alloc_magic_pages = alloc_magic_pages,
-    .count_pgtables = count_pgtables_arm,
-    .setup_pgtables = setup_pgtables_arm,
-    .start_info = start_info_arm,
-    .shared_info = shared_info_arm,
-    .vcpu = vcpu_arm64,
-};
-
-static void __init register_arch_hooks(void)
-{
-    xc_dom_register_arch_hooks(&xc_dom_32);
-    xc_dom_register_arch_hooks(&xc_dom_64);
-}
-
 static int set_mode(xc_interface *xch, domid_t domid, char *guest_type)
 {
     static const struct {
@@ -385,7 +353,7 @@ out:
     return rc < 0 ? rc : 0;
 }
 
-int arch_setup_meminit(struct xc_dom_image *dom)
+static int meminit(struct xc_dom_image *dom)
 {
     int i, rc;
     xen_pfn_t pfn;
@@ -522,6 +490,42 @@ int arch_setup_meminit(struct xc_dom_image *dom)
     return 0;
 }
 
+/* ------------------------------------------------------------------------ */
+
+static struct xc_dom_arch xc_dom_32 = {
+    .guest_type = "xen-3.0-armv7l",
+    .native_protocol = XEN_IO_PROTO_ABI_ARM,
+    .page_shift = PAGE_SHIFT_ARM,
+    .sizeof_pfn = 8,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_arm,
+    .setup_pgtables = setup_pgtables_arm,
+    .start_info = start_info_arm,
+    .shared_info = shared_info_arm,
+    .vcpu = vcpu_arm32,
+    .meminit = meminit,
+};
+
+static struct xc_dom_arch xc_dom_64 = {
+    .guest_type = "xen-3.0-aarch64",
+    .native_protocol = XEN_IO_PROTO_ABI_ARM,
+    .page_shift = PAGE_SHIFT_ARM,
+    .sizeof_pfn = 8,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_arm,
+    .setup_pgtables = setup_pgtables_arm,
+    .start_info = start_info_arm,
+    .shared_info = shared_info_arm,
+    .vcpu = vcpu_arm64,
+    .meminit = meminit,
+};
+
+static void __init register_arch_hooks(void)
+{
+    xc_dom_register_arch_hooks(&xc_dom_32);
+    xc_dom_register_arch_hooks(&xc_dom_64);
+}
+
 int arch_setup_bootearly(struct xc_dom_image *dom)
 {
     DOMPRINTF("%s: doing nothing", __FUNCTION__);
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index fda9e52..3ef7e6c 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -147,7 +147,7 @@ int xc_dom_boot_mem_init(struct xc_dom_image *dom)
 
     DOMPRINTF_CALLED(dom->xch);
 
-    rc = arch_setup_meminit(dom);
+    rc = dom->arch_hooks->meminit(dom);
     if ( rc != 0 )
     {
         xc_dom_panic(dom->xch, XC_OUT_OF_MEMORY,
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index b89f5c2..6fb4aee 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -671,38 +671,6 @@ static int vcpu_x86_64(struct xc_dom_image *dom, void *ptr)
 
 /* ------------------------------------------------------------------------ */
 
-static struct xc_dom_arch xc_dom_32_pae = {
-    .guest_type = "xen-3.0-x86_32p",
-    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
-    .page_shift = PAGE_SHIFT_X86,
-    .sizeof_pfn = 4,
-    .alloc_magic_pages = alloc_magic_pages,
-    .count_pgtables = count_pgtables_x86_32_pae,
-    .setup_pgtables = setup_pgtables_x86_32_pae,
-    .start_info = start_info_x86_32,
-    .shared_info = shared_info_x86_32,
-    .vcpu = vcpu_x86_32,
-};
-
-static struct xc_dom_arch xc_dom_64 = {
-    .guest_type = "xen-3.0-x86_64",
-    .native_protocol = XEN_IO_PROTO_ABI_X86_64,
-    .page_shift = PAGE_SHIFT_X86,
-    .sizeof_pfn = 8,
-    .alloc_magic_pages = alloc_magic_pages,
-    .count_pgtables = count_pgtables_x86_64,
-    .setup_pgtables = setup_pgtables_x86_64,
-    .start_info = start_info_x86_64,
-    .shared_info = shared_info_x86_64,
-    .vcpu = vcpu_x86_64,
-};
-
-static void __init register_arch_hooks(void)
-{
-    xc_dom_register_arch_hooks(&xc_dom_32_pae);
-    xc_dom_register_arch_hooks(&xc_dom_64);
-}
-
 static int x86_compat(xc_interface *xch, domid_t domid, char *guest_type)
 {
     static const struct {
@@ -734,7 +702,6 @@ static int x86_compat(xc_interface *xch, domid_t domid, char *guest_type)
     return rc;
 }
 
-
 static int x86_shadow(xc_interface *xch, domid_t domid)
 {
     int rc, mode;
@@ -758,7 +725,7 @@ static int x86_shadow(xc_interface *xch, domid_t domid)
     return rc;
 }
 
-int arch_setup_meminit(struct xc_dom_image *dom)
+static int meminit_pv(struct xc_dom_image *dom)
 {
     int rc;
     xen_pfn_t pfn, allocsz, mfn, total, pfn_base;
@@ -957,6 +924,42 @@ int arch_setup_meminit(struct xc_dom_image *dom)
     return rc;
 }
 
+/* ------------------------------------------------------------------------ */
+
+static struct xc_dom_arch xc_dom_32_pae = {
+    .guest_type = "xen-3.0-x86_32p",
+    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
+    .page_shift = PAGE_SHIFT_X86,
+    .sizeof_pfn = 4,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_x86_32_pae,
+    .setup_pgtables = setup_pgtables_x86_32_pae,
+    .start_info = start_info_x86_32,
+    .shared_info = shared_info_x86_32,
+    .vcpu = vcpu_x86_32,
+    .meminit = meminit_pv,
+};
+
+static struct xc_dom_arch xc_dom_64 = {
+    .guest_type = "xen-3.0-x86_64",
+    .native_protocol = XEN_IO_PROTO_ABI_X86_64,
+    .page_shift = PAGE_SHIFT_X86,
+    .sizeof_pfn = 8,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_x86_64,
+    .setup_pgtables = setup_pgtables_x86_64,
+    .start_info = start_info_x86_64,
+    .shared_info = shared_info_x86_64,
+    .vcpu = vcpu_x86_64,
+    .meminit = meminit_pv,
+};
+
+static void __init register_arch_hooks(void)
+{
+    xc_dom_register_arch_hooks(&xc_dom_32_pae);
+    xc_dom_register_arch_hooks(&xc_dom_64);
+}
+
 int arch_setup_bootearly(struct xc_dom_image *dom)
 {
     DOMPRINTF("%s: doing nothing", __FUNCTION__);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 05/32] libxc: make arch_setup_meminit a xc_dom_arch hook

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> This allows having different arch_setup_meminit implementations based on the
> guest type. It should not introduce any functional changes.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

However, would you mind doing a cleanup patch which changes the use of
__init.  Confusingly given its precedent in hypervisor code, xc_dom.h
has "#define __init __attribute__ ((constructor))".  At a first guess,
'initcall' or even just 'constructor' would be more appropriate names.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 05/32] libxc: make arch_setup_meminit a xc_dom_arch hook

[Roger Pau Monné]

El 06/07/15 a les 14.23, Andrew Cooper ha escrit:
> On 03/07/15 12:34, Roger Pau Monne wrote:
>> This allows having different arch_setup_meminit implementations based on the
>> guest type. It should not introduce any functional changes.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
> 
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> However, would you mind doing a cleanup patch which changes the use of
> __init.  Confusingly given its precedent in hypervisor code, xc_dom.h
> has "#define __init __attribute__ ((constructor))".  At a first guess,
> 'initcall' or even just 'constructor' would be more appropriate names.

Iff you don't mind, I would rather do that after this series. I'm
already carrying a non-trivial amount of patches, and doing something
like this in the middle of the series would be hard to maintain rebase wise.

Roger.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 05/32] libxc: make arch_setup_meminit a xc_dom_arch hook

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:43PM +0200, Roger Pau Monne wrote:
> This allows having different arch_setup_meminit implementations based on the
> guest type. It should not introduce any functional changes.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

[PATCH v3 06/32] libxc: make arch_setup_boot{init/late} xc_dom_arch hooks

[Roger Pau Monne]

This should not introduce any functional change.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/include/xc_dom.h |  7 ++--
 tools/libxc/xc_dom_arm.c     | 38 ++++++++++++---------
 tools/libxc/xc_dom_boot.c    |  4 +--
 tools/libxc/xc_dom_x86.c     | 78 ++++++++++++++++++++++++--------------------
 4 files changed, 68 insertions(+), 59 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index b461f19..42533a5 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -223,6 +223,8 @@ struct xc_dom_arch {
     int (*start_info) (struct xc_dom_image * dom);
     int (*shared_info) (struct xc_dom_image * dom, void *shared_info);
     int (*vcpu) (struct xc_dom_image * dom, void *vcpu_ctxt);
+    int (*bootearly) (struct xc_dom_image * dom);
+    int (*bootlate) (struct xc_dom_image * dom);
 
     /* arch-specific memory initialization. */
     int (*meminit) (struct xc_dom_image * dom);
@@ -402,11 +404,6 @@ static inline xen_pfn_t xc_dom_p2m(struct xc_dom_image *dom, xen_pfn_t pfn)
     return dom->p2m_host[pfn - dom->rambase_pfn];
 }
 
-/* --- arch bits --------------------------------------------------- */
-
-int arch_setup_bootearly(struct xc_dom_image *dom);
-int arch_setup_bootlate(struct xc_dom_image *dom);
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/libxc/xc_dom_arm.c b/tools/libxc/xc_dom_arm.c
index 1f00d05..b0d964c 100644
--- a/tools/libxc/xc_dom_arm.c
+++ b/tools/libxc/xc_dom_arm.c
@@ -492,6 +492,24 @@ static int meminit(struct xc_dom_image *dom)
 
 /* ------------------------------------------------------------------------ */
 
+static int bootearly(struct xc_dom_image *dom)
+{
+    DOMPRINTF("%s: doing nothing", __FUNCTION__);
+    return 0;
+}
+
+static int bootlate(struct xc_dom_image *dom)
+{
+    /* XXX
+     *   map shared info
+     *   map grant tables
+     *   setup shared info
+     */
+    return 0;
+}
+
+/* ------------------------------------------------------------------------ */
+
 static struct xc_dom_arch xc_dom_32 = {
     .guest_type = "xen-3.0-armv7l",
     .native_protocol = XEN_IO_PROTO_ABI_ARM,
@@ -504,6 +522,8 @@ static struct xc_dom_arch xc_dom_32 = {
     .shared_info = shared_info_arm,
     .vcpu = vcpu_arm32,
     .meminit = meminit,
+    .bootearly = bootearly,
+    .bootlate = bootlate,
 };
 
 static struct xc_dom_arch xc_dom_64 = {
@@ -518,6 +538,8 @@ static struct xc_dom_arch xc_dom_64 = {
     .shared_info = shared_info_arm,
     .vcpu = vcpu_arm64,
     .meminit = meminit,
+    .bootearly = bootearly,
+    .bootlate = bootlate,
 };
 
 static void __init register_arch_hooks(void)
@@ -526,22 +548,6 @@ static void __init register_arch_hooks(void)
     xc_dom_register_arch_hooks(&xc_dom_64);
 }
 
-int arch_setup_bootearly(struct xc_dom_image *dom)
-{
-    DOMPRINTF("%s: doing nothing", __FUNCTION__);
-    return 0;
-}
-
-int arch_setup_bootlate(struct xc_dom_image *dom)
-{
-    /* XXX
-     *   map shared info
-     *   map grant tables
-     *   setup shared info
-     */
-    return 0;
-}
-
 int xc_dom_feature_translated(struct xc_dom_image *dom)
 {
     return 1;
diff --git a/tools/libxc/xc_dom_boot.c b/tools/libxc/xc_dom_boot.c
index 3ef7e6c..2f7488d 100644
--- a/tools/libxc/xc_dom_boot.c
+++ b/tools/libxc/xc_dom_boot.c
@@ -209,7 +209,7 @@ int xc_dom_boot_image(struct xc_dom_image *dom)
     DOMPRINTF_CALLED(dom->xch);
 
     /* misc stuff*/
-    if ( (rc = arch_setup_bootearly(dom)) != 0 )
+    if ( (rc = dom->arch_hooks->bootearly(dom)) != 0 )
         return rc;
 
     /* collect some info */
@@ -256,7 +256,7 @@ int xc_dom_boot_image(struct xc_dom_image *dom)
     xc_dom_log_memory_footprint(dom);
 
     /* misc x86 stuff */
-    if ( (rc = arch_setup_bootlate(dom)) != 0 )
+    if ( (rc = dom->arch_hooks->bootlate(dom)) != 0 )
         return rc;
 
     /* let the vm run */
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 6fb4aee..993954e 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -926,41 +926,7 @@ static int meminit_pv(struct xc_dom_image *dom)
 
 /* ------------------------------------------------------------------------ */
 
-static struct xc_dom_arch xc_dom_32_pae = {
-    .guest_type = "xen-3.0-x86_32p",
-    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
-    .page_shift = PAGE_SHIFT_X86,
-    .sizeof_pfn = 4,
-    .alloc_magic_pages = alloc_magic_pages,
-    .count_pgtables = count_pgtables_x86_32_pae,
-    .setup_pgtables = setup_pgtables_x86_32_pae,
-    .start_info = start_info_x86_32,
-    .shared_info = shared_info_x86_32,
-    .vcpu = vcpu_x86_32,
-    .meminit = meminit_pv,
-};
-
-static struct xc_dom_arch xc_dom_64 = {
-    .guest_type = "xen-3.0-x86_64",
-    .native_protocol = XEN_IO_PROTO_ABI_X86_64,
-    .page_shift = PAGE_SHIFT_X86,
-    .sizeof_pfn = 8,
-    .alloc_magic_pages = alloc_magic_pages,
-    .count_pgtables = count_pgtables_x86_64,
-    .setup_pgtables = setup_pgtables_x86_64,
-    .start_info = start_info_x86_64,
-    .shared_info = shared_info_x86_64,
-    .vcpu = vcpu_x86_64,
-    .meminit = meminit_pv,
-};
-
-static void __init register_arch_hooks(void)
-{
-    xc_dom_register_arch_hooks(&xc_dom_32_pae);
-    xc_dom_register_arch_hooks(&xc_dom_64);
-}
-
-int arch_setup_bootearly(struct xc_dom_image *dom)
+static int bootearly(struct xc_dom_image *dom)
 {
     DOMPRINTF("%s: doing nothing", __FUNCTION__);
     return 0;
@@ -999,7 +965,7 @@ static int map_grant_table_frames(struct xc_dom_image *dom)
     return 0;
 }
 
-int arch_setup_bootlate(struct xc_dom_image *dom)
+static int bootlate_pv(struct xc_dom_image *dom)
 {
     static const struct {
         char *guest;
@@ -1072,6 +1038,46 @@ int arch_setup_bootlate(struct xc_dom_image *dom)
     return 0;
 }
 
+/* ------------------------------------------------------------------------ */
+
+static struct xc_dom_arch xc_dom_32_pae = {
+    .guest_type = "xen-3.0-x86_32p",
+    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
+    .page_shift = PAGE_SHIFT_X86,
+    .sizeof_pfn = 4,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_x86_32_pae,
+    .setup_pgtables = setup_pgtables_x86_32_pae,
+    .start_info = start_info_x86_32,
+    .shared_info = shared_info_x86_32,
+    .vcpu = vcpu_x86_32,
+    .meminit = meminit_pv,
+    .bootearly = bootearly,
+    .bootlate = bootlate_pv,
+};
+
+static struct xc_dom_arch xc_dom_64 = {
+    .guest_type = "xen-3.0-x86_64",
+    .native_protocol = XEN_IO_PROTO_ABI_X86_64,
+    .page_shift = PAGE_SHIFT_X86,
+    .sizeof_pfn = 8,
+    .alloc_magic_pages = alloc_magic_pages,
+    .count_pgtables = count_pgtables_x86_64,
+    .setup_pgtables = setup_pgtables_x86_64,
+    .start_info = start_info_x86_64,
+    .shared_info = shared_info_x86_64,
+    .vcpu = vcpu_x86_64,
+    .meminit = meminit_pv,
+    .bootearly = bootearly,
+    .bootlate = bootlate_pv,
+};
+
+static void __init register_arch_hooks(void)
+{
+    xc_dom_register_arch_hooks(&xc_dom_32_pae);
+    xc_dom_register_arch_hooks(&xc_dom_64);
+}
+
 int xc_dom_feature_translated(struct xc_dom_image *dom)
 {
     /* Guests running inside HVM containers are always auto-translated. */
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 06/32] libxc: make arch_setup_boot{init/late} xc_dom_arch hooks

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
> index 6fb4aee..993954e 100644
> --- a/tools/libxc/xc_dom_x86.c
> +++ b/tools/libxc/xc_dom_x86.c
> @@ -926,41 +926,7 @@ static int meminit_pv(struct xc_dom_image *dom)
>  
>  /* ------------------------------------------------------------------------ */
>  
> -static struct xc_dom_arch xc_dom_32_pae = {
> -    .guest_type = "xen-3.0-x86_32p",
> -    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
> -    .page_shift = PAGE_SHIFT_X86,
> -    .sizeof_pfn = 4,
> -    .alloc_magic_pages = alloc_magic_pages,
> -    .count_pgtables = count_pgtables_x86_32_pae,
> -    .setup_pgtables = setup_pgtables_x86_32_pae,
> -    .start_info = start_info_x86_32,
> -    .shared_info = shared_info_x86_32,
> -    .vcpu = vcpu_x86_32,
> -    .meminit = meminit_pv,
> -};
> -
> -static struct xc_dom_arch xc_dom_64 = {
> -    .guest_type = "xen-3.0-x86_64",
> -    .native_protocol = XEN_IO_PROTO_ABI_X86_64,
> -    .page_shift = PAGE_SHIFT_X86,
> -    .sizeof_pfn = 8,
> -    .alloc_magic_pages = alloc_magic_pages,
> -    .count_pgtables = count_pgtables_x86_64,
> -    .setup_pgtables = setup_pgtables_x86_64,
> -    .start_info = start_info_x86_64,
> -    .shared_info = shared_info_x86_64,
> -    .vcpu = vcpu_x86_64,
> -    .meminit = meminit_pv,
> -};
> -
> -static void __init register_arch_hooks(void)
> -{
> -    xc_dom_register_arch_hooks(&xc_dom_32_pae);
> -    xc_dom_register_arch_hooks(&xc_dom_64);
> -}

Instead of moving this lot twice in two consecutive patches, have patch
5 move them to their eventual location, which makes this patch much smaller.

Otherwise, Reviewed-by: Andrew Cooper <andrew.cooper@citrix.com>

Re: [PATCH v3 06/32] libxc: make arch_setup_boot{init/late} xc_dom_arch hooks

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:44PM +0200, Roger Pau Monne wrote:
> This should not introduce any functional change.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Acked-by: Wei Liu <wei.liu2@citrix.com>

> ---

> +/* ------------------------------------------------------------------------ */
> +
> +static struct xc_dom_arch xc_dom_32_pae = {
> +    .guest_type = "xen-3.0-x86_32p",
> +    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
> +    .page_shift = PAGE_SHIFT_X86,
> +    .sizeof_pfn = 4,
> +    .alloc_magic_pages = alloc_magic_pages,
> +    .count_pgtables = count_pgtables_x86_32_pae,
> +    .setup_pgtables = setup_pgtables_x86_32_pae,
> +    .start_info = start_info_x86_32,
> +    .shared_info = shared_info_x86_32,
> +    .vcpu = vcpu_x86_32,
> +    .meminit = meminit_pv,
> +    .bootearly = bootearly,
> +    .bootlate = bootlate_pv,
> +};
> +
> +static struct xc_dom_arch xc_dom_64 = {
> +    .guest_type = "xen-3.0-x86_64",
> +    .native_protocol = XEN_IO_PROTO_ABI_X86_64,
> +    .page_shift = PAGE_SHIFT_X86,
> +    .sizeof_pfn = 8,
> +    .alloc_magic_pages = alloc_magic_pages,
> +    .count_pgtables = count_pgtables_x86_64,
> +    .setup_pgtables = setup_pgtables_x86_64,
> +    .start_info = start_info_x86_64,
> +    .shared_info = shared_info_x86_64,
> +    .vcpu = vcpu_x86_64,
> +    .meminit = meminit_pv,
> +    .bootearly = bootearly,
> +    .bootlate = bootlate_pv,
> +};
> +

One thing I notice in this series is that these hunks are moved
repeatedly in patches. Might be worth moving them right to the end /
only adding new functions before them to reduce patch size.

Just an observation. Not a blocker for this series though. If it turns
out hard to do just ignore that comment.

> +static void __init register_arch_hooks(void)
> +{
> +    xc_dom_register_arch_hooks(&xc_dom_32_pae);
> +    xc_dom_register_arch_hooks(&xc_dom_64);
> +}
> +
>  int xc_dom_feature_translated(struct xc_dom_image *dom)
>  {
>      /* Guests running inside HVM containers are always auto-translated. */
> -- 
> 1.9.5 (Apple Git-50.3)

[PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monne]

Add checks for ignored vcpu fields in HVM mode. HVM vCPUs (BSP and APs) are
always started in 32bit protected mode with paging disabled.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/domain.c | 14 +++++++++++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index a8fe046..a112953 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -795,6 +795,15 @@ int arch_set_info_guest(
               c.nat->fs_base || c.nat->gs_base_user)) )
             return -EINVAL;
     }
+    else if ( is_hvm_domain(d) )
+    {
+        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
+             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
+             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
+             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
+             c(gdt_ents) )
+            return -EINVAL;
+    }
 
     v->fpu_initialised = !!(flags & VGCF_I387_VALID);
 
@@ -1064,15 +1073,14 @@ int arch_set_info_guest(
     if ( v->vcpu_id == 0 )
         update_domain_wallclock_time(d);
 
-    /* Don't redo final setup */
-    v->is_initialised = 1;
-
     if ( paging_mode_enabled(d) )
         paging_update_paging_modes(v);
 
     update_cr3(v);
 
  out:
+    /* Don't redo final setup */
+    v->is_initialised = 1;
     if ( flags & VGCF_online )
         clear_bit(_VPF_down, &v->pause_flags);
     else
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> Add checks for ignored vcpu fields in HVM mode. HVM vCPUs (BSP and APs) are
> always started in 32bit protected mode with paging disabled.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>  xen/arch/x86/domain.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
>
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index a8fe046..a112953 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>                c.nat->fs_base || c.nat->gs_base_user)) )
>              return -EINVAL;
>      }
> +    else if ( is_hvm_domain(d) )
> +    {
> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||

I am not sure that the cr0 and cr4 checks are sensible.  While the new
ABI doesn't need them, there is nothing inherently wrong with setting
them via this interface.

> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
> +             c(gdt_ents) )
> +            return -EINVAL;
> +    }
>  
>      v->fpu_initialised = !!(flags & VGCF_I387_VALID);
>  
> @@ -1064,15 +1073,14 @@ int arch_set_info_guest(
>      if ( v->vcpu_id == 0 )
>          update_domain_wallclock_time(d);
>  
> -    /* Don't redo final setup */
> -    v->is_initialised = 1;
> -

Is this a bugfix, or spurious movement?

~Andrew

>      if ( paging_mode_enabled(d) )
>          paging_update_paging_modes(v);
>  
>      update_cr3(v);
>  
>   out:
> +    /* Don't redo final setup */
> +    v->is_initialised = 1;
>      if ( flags & VGCF_online )
>          clear_bit(_VPF_down, &v->pause_flags);
>      else


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 06/07/15 a les 14.58, Andrew Cooper ha escrit:
> On 03/07/15 12:34, Roger Pau Monne wrote:
>> Add checks for ignored vcpu fields in HVM mode. HVM vCPUs (BSP and APs) are
>> always started in 32bit protected mode with paging disabled.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Jan Beulich <jbeulich@suse.com>
>> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>>  xen/arch/x86/domain.c | 14 +++++++++++---
>>  1 file changed, 11 insertions(+), 3 deletions(-)
>>
>> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
>> index a8fe046..a112953 100644
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>>                c.nat->fs_base || c.nat->gs_base_user)) )
>>              return -EINVAL;
>>      }
>> +    else if ( is_hvm_domain(d) )
>> +    {
>> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
>> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
> 
> I am not sure that the cr0 and cr4 checks are sensible.  While the new
> ABI doesn't need them, there is nothing inherently wrong with setting
> them via this interface.

They will be ignored anyway, so I think the right thing to do is to
notify the caller that setting CR4 and CR0 is not possible.

IMHO this should be as limited as possible, and CR0/CR4 can be easily
setup by the startup AP code in the guest.

>> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
>> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
>> +             c(gdt_ents) )
>> +            return -EINVAL;
>> +    }
>>  
>>      v->fpu_initialised = !!(flags & VGCF_I387_VALID);
>>  
>> @@ -1064,15 +1073,14 @@ int arch_set_info_guest(
>>      if ( v->vcpu_id == 0 )
>>          update_domain_wallclock_time(d);
>>  
>> -    /* Don't redo final setup */
>> -    v->is_initialised = 1;
>> -
> 
> Is this a bugfix, or spurious movement?

Since this hypercall was never used before with HVM guests we need to
move the is_initialized setting to the "out" label, so it's also set for
HVM guests (HVM guests jump to the out label after setting the user and
debug registers).

Roger.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Konrad Rzeszutek Wilk]

On Fri, Jul 03, 2015 at 01:34:45PM +0200, Roger Pau Monne wrote:
> Add checks for ignored vcpu fields in HVM mode. HVM vCPUs (BSP and APs) are
> always started in 32bit protected mode with paging disabled.

Please mention that only the HVM guests which use the VCPU_xx hypercalls
are the ones that are involved in this.

> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>  xen/arch/x86/domain.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index a8fe046..a112953 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>                c.nat->fs_base || c.nat->gs_base_user)) )
>              return -EINVAL;
>      }
> +    else if ( is_hvm_domain(d) )
> +    {
> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
> +             c(gdt_ents) )
> +            return -EINVAL;
> +    }
>  
>      v->fpu_initialised = !!(flags & VGCF_I387_VALID);
>  
> @@ -1064,15 +1073,14 @@ int arch_set_info_guest(
>      if ( v->vcpu_id == 0 )
>          update_domain_wallclock_time(d);
>  
> -    /* Don't redo final setup */
> -    v->is_initialised = 1;
> -
>      if ( paging_mode_enabled(d) )
>          paging_update_paging_modes(v);
>  
>      update_cr3(v);
>  
>   out:
> +    /* Don't redo final setup */
> +    v->is_initialised = 1;
>      if ( flags & VGCF_online )
>          clear_bit(_VPF_down, &v->pause_flags);
>      else
> -- 
> 1.9.5 (Apple Git-50.3)
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Konrad Rzeszutek Wilk]

On Fri, Jul 03, 2015 at 01:34:45PM +0200, Roger Pau Monne wrote:
> Add checks for ignored vcpu fields in HVM mode. HVM vCPUs (BSP and APs) are
> always started in 32bit protected mode with paging disabled.

This is kind of odd in the placement of the series. The next couple of
patches deal with the restructure of the HVM guests and the previous
ones were too.

And it is quite salient on the PVH/HVMLite protocol. I would think
you would want to also update the documentation/misc/pvh as part
of this patch. Perhaps move this further out in the patch series?
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>  xen/arch/x86/domain.c | 14 +++++++++++---
>  1 file changed, 11 insertions(+), 3 deletions(-)
> 
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index a8fe046..a112953 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>                c.nat->fs_base || c.nat->gs_base_user)) )
>              return -EINVAL;
>      }
> +    else if ( is_hvm_domain(d) )
> +    {
> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
> +             c(gdt_ents) )
> +            return -EINVAL;
> +    }
>  
>      v->fpu_initialised = !!(flags & VGCF_I387_VALID);
>  
> @@ -1064,15 +1073,14 @@ int arch_set_info_guest(
>      if ( v->vcpu_id == 0 )
>          update_domain_wallclock_time(d);
>  
> -    /* Don't redo final setup */
> -    v->is_initialised = 1;
> -
>      if ( paging_mode_enabled(d) )
>          paging_update_paging_modes(v);
>  
>      update_cr3(v);
>  
>   out:
> +    /* Don't redo final setup */
> +    v->is_initialised = 1;
>      if ( flags & VGCF_online )
>          clear_bit(_VPF_down, &v->pause_flags);
>      else
> -- 
> 1.9.5 (Apple Git-50.3)
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 10/07/15 a les 20.47, Konrad Rzeszutek Wilk ha escrit:
> On Fri, Jul 03, 2015 at 01:34:45PM +0200, Roger Pau Monne wrote:
>> Add checks for ignored vcpu fields in HVM mode. HVM vCPUs (BSP and APs) are
>> always started in 32bit protected mode with paging disabled.
> 
> This is kind of odd in the placement of the series. The next couple of
> patches deal with the restructure of the HVM guests and the previous
> ones were too.

This patch is placed here because the next patch in this series makes
use of it (see the vcpu_hvm function in "libxc: introduce a xc_dom_arch
for hvm-3.0-x86_32 guests").

> And it is quite salient on the PVH/HVMLite protocol. I would think
> you would want to also update the documentation/misc/pvh as part
> of this patch. Perhaps move this further out in the patch series?

Yes, documentation needs to be updated. I was waiting to have the full
boot protocol nailed down before doing so.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 03.07.15 at 13:34, <roger.pau@citrix.com> wrote:
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>                c.nat->fs_base || c.nat->gs_base_user)) )
>              return -EINVAL;
>      }
> +    else if ( is_hvm_domain(d) )
> +    {
> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
> +             c(gdt_ents) )
> +            return -EINVAL;
> +    }

In addition to what Andrew said - is the use of c() here really correct
considering

    compat = is_pv_32bit_domain(d);

#define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))

near the beginning of the function?

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 13/07/15 a les 16.01, Jan Beulich ha escrit:
>>>> On 03.07.15 at 13:34, <roger.pau@citrix.com> wrote:
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>>                c.nat->fs_base || c.nat->gs_base_user)) )
>>              return -EINVAL;
>>      }
>> +    else if ( is_hvm_domain(d) )
>> +    {
>> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
>> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
>> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
>> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
>> +             c(gdt_ents) )
>> +            return -EINVAL;
>> +    }
> 
> In addition to what Andrew said - is the use of c() here really correct
> considering
> 
>     compat = is_pv_32bit_domain(d);
> 
> #define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))
> 
> near the beginning of the function?

I've been thinking about this. Since HVM vCPUs are always started in
32bit mode, I would ideally like to use the vcpu_guest_context_x86_32_t
struct to set the vCPU context. This means that for HVM guests "compat"
should always be true.

The problem is that inside of the guest there's no notion of
vcpu_guest_context_x86_32_t or vcpu_guest_context_x86_64_t, there's only
vcpu_guest_context, which defaults to the native bitness of the guest.
So if your guest is running in 64bit mode vcpu_guest_context is aliased
to vcpu_guest_context_x86_64_t by default.

TBH I'm not sure about the best way to solve this. Should
vcpu_guest_context_x86_32_t and vcpu_guest_context_x86_64_t be exposed
to the guest like it's done for the tools?

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 23.07.15 at 12:25, <roger.pau@citrix.com> wrote:
> El 13/07/15 a les 16.01, Jan Beulich ha escrit:
>>>>> On 03.07.15 at 13:34, <roger.pau@citrix.com> wrote:
>>> --- a/xen/arch/x86/domain.c
>>> +++ b/xen/arch/x86/domain.c
>>> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>>>                c.nat->fs_base || c.nat->gs_base_user)) )
>>>              return -EINVAL;
>>>      }
>>> +    else if ( is_hvm_domain(d) )
>>> +    {
>>> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) ||
>>> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) ||
>>> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
>>> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
>>> +             c(gdt_ents) )
>>> +            return -EINVAL;
>>> +    }
>> 
>> In addition to what Andrew said - is the use of c() here really correct
>> considering
>> 
>>     compat = is_pv_32bit_domain(d);
>> 
>> #define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))
>> 
>> near the beginning of the function?
> 
> I've been thinking about this. Since HVM vCPUs are always started in
> 32bit mode, I would ideally like to use the vcpu_guest_context_x86_32_t
> struct to set the vCPU context. This means that for HVM guests "compat"
> should always be true.
> 
> The problem is that inside of the guest there's no notion of
> vcpu_guest_context_x86_32_t or vcpu_guest_context_x86_64_t, there's only
> vcpu_guest_context, which defaults to the native bitness of the guest.
> So if your guest is running in 64bit mode vcpu_guest_context is aliased
> to vcpu_guest_context_x86_64_t by default.
> 
> TBH I'm not sure about the best way to solve this. Should
> vcpu_guest_context_x86_32_t and vcpu_guest_context_x86_64_t be exposed
> to the guest like it's done for the tools?

Why? We're in arch_set_info_guest(), which is unreachable for a
HVM guest on itself. Or is this in preparation of allowing HVM
guests to use VCPUOP_initialise? In that case, exposing might be
an option, but remember that the compat mode layout even today
is used only for PV guests. So I'd rather avoid exposing both
layouts to guests, and do translation instead inside the hypervisor.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Ian Campbell]

On Thu, 2015-07-23 at 05:29 -0600, Jan Beulich wrote:
> > 
> > > > On 23.07.15 at 12:25, <roger.pau@citrix.com> wrote:
> > El 13/07/15 a les 16.01, Jan Beulich ha escrit:
> > > > > > On 03.07.15 at 13:34, <roger.pau@citrix.com> wrote:
> > > > --- a/xen/arch/x86/domain.c
> > > > +++ b/xen/arch/x86/domain.c
> > > > @@ -795,6 +795,15 @@ int arch_set_info_guest(
> > > >                c.nat->fs_base || c.nat->gs_base_user)) )
> > > >              return -EINVAL;
> > > >      }
> > > > +    else if ( is_hvm_domain(d) )
> > > > +    {
> > > > +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) 
> > > > ||
> > > > +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) 
> > > > ||
> > > > +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
> > > > +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
> > > > +             c(gdt_ents) )
> > > > +            return -EINVAL;
> > > > +    }
> > > 
> > > In addition to what Andrew said - is the use of c() here really 
> > > correct
> > > considering
> > > 
> > >     compat = is_pv_32bit_domain(d);
> > > 
> > > #define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))
> > > 
> > > near the beginning of the function?
> > 
> > I've been thinking about this. Since HVM vCPUs are always started 
> > in
> > 32bit mode, I would ideally like to use the 
> > vcpu_guest_context_x86_32_t
> > struct to set the vCPU context. This means that for HVM guests 
> > "compat"
> > should always be true.
> > 
> > The problem is that inside of the guest there's no notion of
> > vcpu_guest_context_x86_32_t or vcpu_guest_context_x86_64_t, there's 
> > only
> > vcpu_guest_context, which defaults to the native bitness of the 
> > guest.
> > So if your guest is running in 64bit mode vcpu_guest_context is 
> > aliased
> > to vcpu_guest_context_x86_64_t by default.
> > 
> > TBH I'm not sure about the best way to solve this. Should
> > vcpu_guest_context_x86_32_t and vcpu_guest_context_x86_64_t be 
> > exposed
> > to the guest like it's done for the tools?
> 
> Why? We're in arch_set_info_guest(), which is unreachable for a
> HVM guest on itself. Or is this in preparation of allowing HVM
> guests to use VCPUOP_initialise? In that case, exposing might be
> an option, but remember that the compat mode layout even today
> is used only for PV guests. So I'd rather avoid exposing both
> layouts to guests, and do translation instead inside the hypervisor.

On ARM we deliberately arranged that the vcpu_guest_context was the
same for both arm32 and arm64. Moving to PVH seems like an ideal
opportunity to do something similar for x86, if not by standardising on
the x86_64 layout then by declaring a new struct which can encompass
both and declaring the old ones PV legacy.

Ian.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 23/07/15 a les 13.41, Ian Campbell ha escrit:
> On Thu, 2015-07-23 at 05:29 -0600, Jan Beulich wrote:
>>>
>>>>> On 23.07.15 at 12:25, <roger.pau@citrix.com> wrote:
>>> El 13/07/15 a les 16.01, Jan Beulich ha escrit:
>>>>>>> On 03.07.15 at 13:34, <roger.pau@citrix.com> wrote:
>>>>> --- a/xen/arch/x86/domain.c
>>>>> +++ b/xen/arch/x86/domain.c
>>>>> @@ -795,6 +795,15 @@ int arch_set_info_guest(
>>>>>                c.nat->fs_base || c.nat->gs_base_user)) )
>>>>>              return -EINVAL;
>>>>>      }
>>>>> +    else if ( is_hvm_domain(d) )
>>>>> +    {
>>>>> +        if ( c(ctrlreg[0]) || c(ctrlreg[1]) || c(ctrlreg[2]) 
>>>>> ||
>>>>> +             c(ctrlreg[3]) || c(ctrlreg[4]) || c(ctrlreg[5]) 
>>>>> ||
>>>>> +             c(ctrlreg[6]) || c(ctrlreg[7]) || c(ldt_base) ||
>>>>> +             c(ldt_ents) || c(kernel_ss) || c(kernel_sp) ||
>>>>> +             c(gdt_ents) )
>>>>> +            return -EINVAL;
>>>>> +    }
>>>>
>>>> In addition to what Andrew said - is the use of c() here really 
>>>> correct
>>>> considering
>>>>
>>>>     compat = is_pv_32bit_domain(d);
>>>>
>>>> #define c(fld) (compat ? (c.cmp->fld) : (c.nat->fld))
>>>>
>>>> near the beginning of the function?
>>>
>>> I've been thinking about this. Since HVM vCPUs are always started 
>>> in
>>> 32bit mode, I would ideally like to use the 
>>> vcpu_guest_context_x86_32_t
>>> struct to set the vCPU context. This means that for HVM guests 
>>> "compat"
>>> should always be true.
>>>
>>> The problem is that inside of the guest there's no notion of
>>> vcpu_guest_context_x86_32_t or vcpu_guest_context_x86_64_t, there's 
>>> only
>>> vcpu_guest_context, which defaults to the native bitness of the 
>>> guest.
>>> So if your guest is running in 64bit mode vcpu_guest_context is 
>>> aliased
>>> to vcpu_guest_context_x86_64_t by default.
>>>
>>> TBH I'm not sure about the best way to solve this. Should
>>> vcpu_guest_context_x86_32_t and vcpu_guest_context_x86_64_t be 
>>> exposed
>>> to the guest like it's done for the tools?
>>
>> Why? We're in arch_set_info_guest(), which is unreachable for a
>> HVM guest on itself. Or is this in preparation of allowing HVM
>> guests to use VCPUOP_initialise? In that case, exposing might be
>> an option, but remember that the compat mode layout even today
>> is used only for PV guests. So I'd rather avoid exposing both
>> layouts to guests, and do translation instead inside the hypervisor.
> 
> On ARM we deliberately arranged that the vcpu_guest_context was the
> same for both arm32 and arm64. Moving to PVH seems like an ideal
> opportunity to do something similar for x86, if not by standardising on
> the x86_64 layout then by declaring a new struct which can encompass
> both and declaring the old ones PV legacy.

IMHO introducing a new structure that gets rid of all the PV-only 
fields seems like a good option:

struct vcpu_hvm_context {
#define _VGCF_online                   5
#define VGCF_online                    (1<<_VGCF_online)
    uint32_t flags;                         /* VGCF_* flags                 */
    struct cpu_hvm_user_regs user_regs;     /* User-level CPU registers     */
    /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
    uint32_t ctrlreg[8];                    /* CR0-CR7 (control registers)  */
    uint32_t debugreg[8];                   /* DB0-DB7 (debug registers)    */
};

I'm also seriously considering getting rid of ctrlreg and debugreg.
Since HVM VCPUs will always be started in 32bit flat mode, it doesn't
make sense IMHO to have both the 32 and the 64 version of the 
registers, so cpu_hvm_user_regs is always going to be:

struct cpu_hvm_user_regs {
    uint32_t ebx;
    uint32_t ecx;
    uint32_t edx;
    uint32_t esi;
    uint32_t edi;
    uint32_t ebp;
    uint32_t eax;
    uint32_t eip;
    uint32_t esp;
    uint32_t eflags;
    uint16_t cs;
    uint16_t ss;
    uint16_t es;
    uint16_t ds;
    uint16_t fs;
    uint16_t gs;
};

We could however do something similar to what's done in ARM and have
a union of both the 32 and the 64bit registers in case we want to
start the vCPU in 64bit mode sometime in the future.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
> IMHO introducing a new structure that gets rid of all the PV-only 
> fields seems like a good option:
> 
> struct vcpu_hvm_context {
> #define _VGCF_online                   5
> #define VGCF_online                    (1<<_VGCF_online)
>     uint32_t flags;                         /* VGCF_* flags                 
> */
>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU registers     
> */
>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control registers)  
> */
>     uint32_t debugreg[8];                   /* DB0-DB7 (debug registers)    
> */
> };
> 
> I'm also seriously considering getting rid of ctrlreg and debugreg.
> Since HVM VCPUs will always be started in 32bit flat mode, it doesn't
> make sense IMHO to have both the 32 and the 64 version of the 
> registers, so cpu_hvm_user_regs is always going to be:
> 
> struct cpu_hvm_user_regs {
>     uint32_t ebx;
>     uint32_t ecx;
>     uint32_t edx;
>     uint32_t esi;
>     uint32_t edi;
>     uint32_t ebp;
>     uint32_t eax;
>     uint32_t eip;
>     uint32_t esp;
>     uint32_t eflags;
>     uint16_t cs;
>     uint16_t ss;
>     uint16_t es;
>     uint16_t ds;
>     uint16_t fs;
>     uint16_t gs;
> };
> 
> We could however do something similar to what's done in ARM and have
> a union of both the 32 and the 64bit registers in case we want to
> start the vCPU in 64bit mode sometime in the future.

What you gave above is suitable only for VCPUOP_initialise afaict.
Did you intend this to be the case?

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>> IMHO introducing a new structure that gets rid of all the PV-only 
>> fields seems like a good option:
>>
>> struct vcpu_hvm_context {
>> #define _VGCF_online                   5
>> #define VGCF_online                    (1<<_VGCF_online)
>>     uint32_t flags;                         /* VGCF_* flags                 
>> */
>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU registers     
>> */
>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control registers)  
>> */
>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug registers)    
>> */
>> };
>>
>> I'm also seriously considering getting rid of ctrlreg and debugreg.
>> Since HVM VCPUs will always be started in 32bit flat mode, it doesn't
>> make sense IMHO to have both the 32 and the 64 version of the 
>> registers, so cpu_hvm_user_regs is always going to be:
>>
>> struct cpu_hvm_user_regs {
>>     uint32_t ebx;
>>     uint32_t ecx;
>>     uint32_t edx;
>>     uint32_t esi;
>>     uint32_t edi;
>>     uint32_t ebp;
>>     uint32_t eax;
>>     uint32_t eip;
>>     uint32_t esp;
>>     uint32_t eflags;
>>     uint16_t cs;
>>     uint16_t ss;
>>     uint16_t es;
>>     uint16_t ds;
>>     uint16_t fs;
>>     uint16_t gs;
>> };
>>
>> We could however do something similar to what's done in ARM and have
>> a union of both the 32 and the 64bit registers in case we want to
>> start the vCPU in 64bit mode sometime in the future.
> 
> What you gave above is suitable only for VCPUOP_initialise afaict.
> Did you intend this to be the case?

Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
I'm afraid I'm missing something obvious, but I don't see why this
couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please clarify?

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 23.07.15 at 17:48, <roger.pau@citrix.com> wrote:
> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>> fields seems like a good option:
>>>
>>> struct vcpu_hvm_context {
>>> #define _VGCF_online                   5
>>> #define VGCF_online                    (1<<_VGCF_online)
>>>     uint32_t flags;                         /* VGCF_* flags                 
>>> */
>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU registers     
>>> */
>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control registers)  
>>> */
>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug registers)    
>>> */
>>> };
>>>
>>> I'm also seriously considering getting rid of ctrlreg and debugreg.
>>> Since HVM VCPUs will always be started in 32bit flat mode, it doesn't
>>> make sense IMHO to have both the 32 and the 64 version of the 
>>> registers, so cpu_hvm_user_regs is always going to be:
>>>
>>> struct cpu_hvm_user_regs {
>>>     uint32_t ebx;
>>>     uint32_t ecx;
>>>     uint32_t edx;
>>>     uint32_t esi;
>>>     uint32_t edi;
>>>     uint32_t ebp;
>>>     uint32_t eax;
>>>     uint32_t eip;
>>>     uint32_t esp;
>>>     uint32_t eflags;
>>>     uint16_t cs;
>>>     uint16_t ss;
>>>     uint16_t es;
>>>     uint16_t ds;
>>>     uint16_t fs;
>>>     uint16_t gs;
>>> };
>>>
>>> We could however do something similar to what's done in ARM and have
>>> a union of both the 32 and the 64bit registers in case we want to
>>> start the vCPU in 64bit mode sometime in the future.
>> 
>> What you gave above is suitable only for VCPUOP_initialise afaict.
>> Did you intend this to be the case?
> 
> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
> I'm afraid I'm missing something obvious, but I don't see why this
> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please clarify?

Just like for HVM (and even more so with the plan to allow PVH to
freely switch between 32- and 64-bit modes) the full 64-bit context
should be got and set in a single domctl imo. If the caller was to
first figure out what bitness a to be migrated guest was paused in
(and that may differ between vCPU-s), things would likely become
quite ugly. Not to speak of the need for a 64-bit tool stack to then
have a way to specify which format the structure is in for a "set"
operation.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Ian Campbell]

On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
> > > > > On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
> > > IMHO introducing a new structure that gets rid of all the PV-only 
> > > 
> > > fields seems like a good option:
> > > 
> > > struct vcpu_hvm_context {
> > > #define _VGCF_online                   5
> > > #define VGCF_online                    (1<<_VGCF_online)
> > >     uint32_t flags;                         /* VGCF_* flags      
> > >            
> > > */
> > >     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
> > > registers     
> > > */
> > >     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
> > >     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
> > > registers)  
> > > */
> > >     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
> > > registers)    
> > > */
> > > };
> > > 
> > > I'm also seriously considering getting rid of ctrlreg and 
> > > debugreg.
> > > Since HVM VCPUs will always be started in 32bit flat mode, it 
> > > doesn't
> > > make sense IMHO to have both the 32 and the 64 version of the 
> > > registers, so cpu_hvm_user_regs is always going to be:
> > > 
> > > struct cpu_hvm_user_regs {
> > >     uint32_t ebx;
> > >     uint32_t ecx;
> > >     uint32_t edx;
> > >     uint32_t esi;
> > >     uint32_t edi;
> > >     uint32_t ebp;
> > >     uint32_t eax;
> > >     uint32_t eip;
> > >     uint32_t esp;
> > >     uint32_t eflags;
> > >     uint16_t cs;
> > >     uint16_t ss;
> > >     uint16_t es;
> > >     uint16_t ds;
> > >     uint16_t fs;
> > >     uint16_t gs;
> > > };
> > > 
> > > We could however do something similar to what's done in ARM and 
> > > have
> > > a union of both the 32 and the 64bit registers in case we want to
> > > start the vCPU in 64bit mode sometime in the future.
> > 
> > What you gave above is suitable only for VCPUOP_initialise afaict.
> > Did you intend this to be the case?
> 
> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
> I'm afraid I'm missing something obvious, but I don't see why this
> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
> clarify?

set/getvcpucontext need to pickle all state for save/restore/migration,
not just the start of day state.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 23/07/15 17:00, Ian Campbell wrote:
> On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
>> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>>>
>>>> fields seems like a good option:
>>>>
>>>> struct vcpu_hvm_context {
>>>> #define _VGCF_online                   5
>>>> #define VGCF_online                    (1<<_VGCF_online)
>>>>     uint32_t flags;                         /* VGCF_* flags      
>>>>            
>>>> */
>>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
>>>> registers     
>>>> */
>>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
>>>> registers)  
>>>> */
>>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
>>>> registers)    
>>>> */
>>>> };
>>>>
>>>> I'm also seriously considering getting rid of ctrlreg and 
>>>> debugreg.
>>>> Since HVM VCPUs will always be started in 32bit flat mode, it 
>>>> doesn't
>>>> make sense IMHO to have both the 32 and the 64 version of the 
>>>> registers, so cpu_hvm_user_regs is always going to be:
>>>>
>>>> struct cpu_hvm_user_regs {
>>>>     uint32_t ebx;
>>>>     uint32_t ecx;
>>>>     uint32_t edx;
>>>>     uint32_t esi;
>>>>     uint32_t edi;
>>>>     uint32_t ebp;
>>>>     uint32_t eax;
>>>>     uint32_t eip;
>>>>     uint32_t esp;
>>>>     uint32_t eflags;
>>>>     uint16_t cs;
>>>>     uint16_t ss;
>>>>     uint16_t es;
>>>>     uint16_t ds;
>>>>     uint16_t fs;
>>>>     uint16_t gs;
>>>> };
>>>>
>>>> We could however do something similar to what's done in ARM and 
>>>> have
>>>> a union of both the 32 and the 64bit registers in case we want to
>>>> start the vCPU in 64bit mode sometime in the future.
>>> What you gave above is suitable only for VCPUOP_initialise afaict.
>>> Did you intend this to be the case?
>> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
>> I'm afraid I'm missing something obvious, but I don't see why this
>> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
>> clarify?
> set/getvcpucontext need to pickle all state for save/restore/migration,
> not just the start of day state.

HVM migration doesn't use set/getvcpucontext.

I would expect an HVMLite-based-PVH to follow suit and just use
set/gethvmcontext.

(In fact, while doing migration v2 I was half thinking of making PV
follow HVM suit somewhat, to simplify the toolstack side of things)

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 23.07.15 at 18:15, <andrew.cooper3@citrix.com> wrote:
> On 23/07/15 17:00, Ian Campbell wrote:
>> On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
>>> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>>>>
>>>>> fields seems like a good option:
>>>>>
>>>>> struct vcpu_hvm_context {
>>>>> #define _VGCF_online                   5
>>>>> #define VGCF_online                    (1<<_VGCF_online)
>>>>>     uint32_t flags;                         /* VGCF_* flags      
>>>>>            
>>>>> */
>>>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
>>>>> registers     
>>>>> */
>>>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
>>>>> registers)  
>>>>> */
>>>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
>>>>> registers)    
>>>>> */
>>>>> };
>>>>>
>>>>> I'm also seriously considering getting rid of ctrlreg and 
>>>>> debugreg.
>>>>> Since HVM VCPUs will always be started in 32bit flat mode, it 
>>>>> doesn't
>>>>> make sense IMHO to have both the 32 and the 64 version of the 
>>>>> registers, so cpu_hvm_user_regs is always going to be:
>>>>>
>>>>> struct cpu_hvm_user_regs {
>>>>>     uint32_t ebx;
>>>>>     uint32_t ecx;
>>>>>     uint32_t edx;
>>>>>     uint32_t esi;
>>>>>     uint32_t edi;
>>>>>     uint32_t ebp;
>>>>>     uint32_t eax;
>>>>>     uint32_t eip;
>>>>>     uint32_t esp;
>>>>>     uint32_t eflags;
>>>>>     uint16_t cs;
>>>>>     uint16_t ss;
>>>>>     uint16_t es;
>>>>>     uint16_t ds;
>>>>>     uint16_t fs;
>>>>>     uint16_t gs;
>>>>> };
>>>>>
>>>>> We could however do something similar to what's done in ARM and 
>>>>> have
>>>>> a union of both the 32 and the 64bit registers in case we want to
>>>>> start the vCPU in 64bit mode sometime in the future.
>>>> What you gave above is suitable only for VCPUOP_initialise afaict.
>>>> Did you intend this to be the case?
>>> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
>>> I'm afraid I'm missing something obvious, but I don't see why this
>>> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
>>> clarify?
>> set/getvcpucontext need to pickle all state for save/restore/migration,
>> not just the start of day state.
> 
> HVM migration doesn't use set/getvcpucontext.
> 
> I would expect an HVMLite-based-PVH to follow suit and just use
> set/gethvmcontext.

Oh, right, but that doesn't mean the respective domctl-s should
produce unconsumable data, or have to guess how to interpret
what gets passed in. I.e. at the very minimum the domctl-s then
should be disallowed for PVH if they can#t be made work right.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 23/07/15 17:19, Jan Beulich wrote:
>>>> On 23.07.15 at 18:15, <andrew.cooper3@citrix.com> wrote:
>> On 23/07/15 17:00, Ian Campbell wrote:
>>> On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
>>>> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>>>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>>>>>
>>>>>> fields seems like a good option:
>>>>>>
>>>>>> struct vcpu_hvm_context {
>>>>>> #define _VGCF_online                   5
>>>>>> #define VGCF_online                    (1<<_VGCF_online)
>>>>>>     uint32_t flags;                         /* VGCF_* flags      
>>>>>>            
>>>>>> */
>>>>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
>>>>>> registers     
>>>>>> */
>>>>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
>>>>>> registers)  
>>>>>> */
>>>>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
>>>>>> registers)    
>>>>>> */
>>>>>> };
>>>>>>
>>>>>> I'm also seriously considering getting rid of ctrlreg and 
>>>>>> debugreg.
>>>>>> Since HVM VCPUs will always be started in 32bit flat mode, it 
>>>>>> doesn't
>>>>>> make sense IMHO to have both the 32 and the 64 version of the 
>>>>>> registers, so cpu_hvm_user_regs is always going to be:
>>>>>>
>>>>>> struct cpu_hvm_user_regs {
>>>>>>     uint32_t ebx;
>>>>>>     uint32_t ecx;
>>>>>>     uint32_t edx;
>>>>>>     uint32_t esi;
>>>>>>     uint32_t edi;
>>>>>>     uint32_t ebp;
>>>>>>     uint32_t eax;
>>>>>>     uint32_t eip;
>>>>>>     uint32_t esp;
>>>>>>     uint32_t eflags;
>>>>>>     uint16_t cs;
>>>>>>     uint16_t ss;
>>>>>>     uint16_t es;
>>>>>>     uint16_t ds;
>>>>>>     uint16_t fs;
>>>>>>     uint16_t gs;
>>>>>> };
>>>>>>
>>>>>> We could however do something similar to what's done in ARM and 
>>>>>> have
>>>>>> a union of both the 32 and the 64bit registers in case we want to
>>>>>> start the vCPU in 64bit mode sometime in the future.
>>>>> What you gave above is suitable only for VCPUOP_initialise afaict.
>>>>> Did you intend this to be the case?
>>>> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
>>>> I'm afraid I'm missing something obvious, but I don't see why this
>>>> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
>>>> clarify?
>>> set/getvcpucontext need to pickle all state for save/restore/migration,
>>> not just the start of day state.
>> HVM migration doesn't use set/getvcpucontext.
>>
>> I would expect an HVMLite-based-PVH to follow suit and just use
>> set/gethvmcontext.
> Oh, right, but that doesn't mean the respective domctl-s should
> produce unconsumable data

Indeed.

> , or have to guess how to interpret
> what gets passed in. I.e. at the very minimum the domctl-s then
> should be disallowed for PVH if they can#t be made work right.

They should be able to work with any vcpu.

In particular, it would be nice to start hvmloader with a hypercall to
set eip properly, rather than mapping gfn 0 and writing a jump
instruction into memory.

Having said that, in a virtual environment it would be nice if vcpu 1
could be started by vcpu 0 on straight in long at a given rip, rsp and cr3.

The reason for starting vcpu 0 in 32bit flat mode is to move mode
knowledge out of the domain builder and into the domain, but once the
guest specific loader is running, there is nothing to say it shouldn't
be able to skip 32bit AP trampolines entirely.

Perhaps what we want is a new VCPU_initialise op, rather than attempting
to retrofit an HVM view of the world onto a PV-specific hypercall.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 23/07/15 a les 18.49, Andrew Cooper ha escrit:
> On 23/07/15 17:19, Jan Beulich wrote:
>>>>> On 23.07.15 at 18:15, <andrew.cooper3@citrix.com> wrote:
>>> On 23/07/15 17:00, Ian Campbell wrote:
>>>> On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
>>>>> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>>>>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>>>>>>
>>>>>>> fields seems like a good option:
>>>>>>>
>>>>>>> struct vcpu_hvm_context {
>>>>>>> #define _VGCF_online                   5
>>>>>>> #define VGCF_online                    (1<<_VGCF_online)
>>>>>>>     uint32_t flags;                         /* VGCF_* flags      
>>>>>>>            
>>>>>>> */
>>>>>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
>>>>>>> registers     
>>>>>>> */
>>>>>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>>>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
>>>>>>> registers)  
>>>>>>> */
>>>>>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
>>>>>>> registers)    
>>>>>>> */
>>>>>>> };
>>>>>>>
>>>>>>> I'm also seriously considering getting rid of ctrlreg and 
>>>>>>> debugreg.
>>>>>>> Since HVM VCPUs will always be started in 32bit flat mode, it 
>>>>>>> doesn't
>>>>>>> make sense IMHO to have both the 32 and the 64 version of the 
>>>>>>> registers, so cpu_hvm_user_regs is always going to be:
>>>>>>>
>>>>>>> struct cpu_hvm_user_regs {
>>>>>>>     uint32_t ebx;
>>>>>>>     uint32_t ecx;
>>>>>>>     uint32_t edx;
>>>>>>>     uint32_t esi;
>>>>>>>     uint32_t edi;
>>>>>>>     uint32_t ebp;
>>>>>>>     uint32_t eax;
>>>>>>>     uint32_t eip;
>>>>>>>     uint32_t esp;
>>>>>>>     uint32_t eflags;
>>>>>>>     uint16_t cs;
>>>>>>>     uint16_t ss;
>>>>>>>     uint16_t es;
>>>>>>>     uint16_t ds;
>>>>>>>     uint16_t fs;
>>>>>>>     uint16_t gs;
>>>>>>> };
>>>>>>>
>>>>>>> We could however do something similar to what's done in ARM and 
>>>>>>> have
>>>>>>> a union of both the 32 and the 64bit registers in case we want to
>>>>>>> start the vCPU in 64bit mode sometime in the future.
>>>>>> What you gave above is suitable only for VCPUOP_initialise afaict.
>>>>>> Did you intend this to be the case?
>>>>> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
>>>>> I'm afraid I'm missing something obvious, but I don't see why this
>>>>> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
>>>>> clarify?
>>>> set/getvcpucontext need to pickle all state for save/restore/migration,
>>>> not just the start of day state.
>>> HVM migration doesn't use set/getvcpucontext.
>>>
>>> I would expect an HVMLite-based-PVH to follow suit and just use
>>> set/gethvmcontext.
>> Oh, right, but that doesn't mean the respective domctl-s should
>> produce unconsumable data
> 
> Indeed.
> 
>> , or have to guess how to interpret
>> what gets passed in. I.e. at the very minimum the domctl-s then
>> should be disallowed for PVH if they can#t be made work right.
> 
> They should be able to work with any vcpu.
> 
> In particular, it would be nice to start hvmloader with a hypercall to
> set eip properly, rather than mapping gfn 0 and writing a jump
> instruction into memory.

Yes, this is already done in patch #8, see the vcpu_hvm function.

> Having said that, in a virtual environment it would be nice if vcpu 1
> could be started by vcpu 0 on straight in long at a given rip, rsp and cr3.
> 
> The reason for starting vcpu 0 in 32bit flat mode is to move mode
> knowledge out of the domain builder and into the domain, but once the
> guest specific loader is running, there is nothing to say it shouldn't
> be able to skip 32bit AP trampolines entirely.

In the current version of the series both the BSP and the APs are
started in the same way: 32bit flat mode. We already have code in place
to start the APs with paging enabled and that's what PVH has been using.

> Perhaps what we want is a new VCPU_initialise op, rather than attempting
> to retrofit an HVM view of the world onto a PV-specific hypercall.

IMHO, the problem is not with VCPU_initialise but with the domctls.

Roger.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 23/07/15 a les 18.19, Jan Beulich ha escrit:
>>>> On 23.07.15 at 18:15, <andrew.cooper3@citrix.com> wrote:
>> On 23/07/15 17:00, Ian Campbell wrote:
>>> On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
>>>> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>>>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>>>>>
>>>>>> fields seems like a good option:
>>>>>>
>>>>>> struct vcpu_hvm_context {
>>>>>> #define _VGCF_online                   5
>>>>>> #define VGCF_online                    (1<<_VGCF_online)
>>>>>>     uint32_t flags;                         /* VGCF_* flags      
>>>>>>            
>>>>>> */
>>>>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
>>>>>> registers     
>>>>>> */
>>>>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
>>>>>> registers)  
>>>>>> */
>>>>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
>>>>>> registers)    
>>>>>> */
>>>>>> };
>>>>>>
>>>>>> I'm also seriously considering getting rid of ctrlreg and 
>>>>>> debugreg.
>>>>>> Since HVM VCPUs will always be started in 32bit flat mode, it 
>>>>>> doesn't
>>>>>> make sense IMHO to have both the 32 and the 64 version of the 
>>>>>> registers, so cpu_hvm_user_regs is always going to be:
>>>>>>
>>>>>> struct cpu_hvm_user_regs {
>>>>>>     uint32_t ebx;
>>>>>>     uint32_t ecx;
>>>>>>     uint32_t edx;
>>>>>>     uint32_t esi;
>>>>>>     uint32_t edi;
>>>>>>     uint32_t ebp;
>>>>>>     uint32_t eax;
>>>>>>     uint32_t eip;
>>>>>>     uint32_t esp;
>>>>>>     uint32_t eflags;
>>>>>>     uint16_t cs;
>>>>>>     uint16_t ss;
>>>>>>     uint16_t es;
>>>>>>     uint16_t ds;
>>>>>>     uint16_t fs;
>>>>>>     uint16_t gs;
>>>>>> };
>>>>>>
>>>>>> We could however do something similar to what's done in ARM and 
>>>>>> have
>>>>>> a union of both the 32 and the 64bit registers in case we want to
>>>>>> start the vCPU in 64bit mode sometime in the future.
>>>>> What you gave above is suitable only for VCPUOP_initialise afaict.
>>>>> Did you intend this to be the case?
>>>> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
>>>> I'm afraid I'm missing something obvious, but I don't see why this
>>>> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
>>>> clarify?
>>> set/getvcpucontext need to pickle all state for save/restore/migration,
>>> not just the start of day state.
>>
>> HVM migration doesn't use set/getvcpucontext.
>>
>> I would expect an HVMLite-based-PVH to follow suit and just use
>> set/gethvmcontext.
> 
> Oh, right, but that doesn't mean the respective domctl-s should
> produce unconsumable data, or have to guess how to interpret
> what gets passed in. I.e. at the very minimum the domctl-s then
> should be disallowed for PVH if they can#t be made work right.

Right, the problem is that if I introduce this new vcpu_context
structure plain HVM guests are also going to use it.

This is not a problem with the VCPU_initialize op, because it was not
available to HVM guests. But it's going to be a problem for
XEN_DOMCTL_{get/set}vcpucontext consumers, since the structure
taken/returned by the hypercall is going to change when the target is a
HVM guest, and this is a can of worms I might prefer to avoid opening.

One partial solution would be to allow the usage of this new struct only
for the VCPU_initialize hypercall for HVM guests, and keep using the old
structures for the domctls, aiming at removing them once PV support is
dropped from Xen, but that looks half-baked IMHO.

Roger.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 23/07/15 17:56, Roger Pau Monné wrote:
> El 23/07/15 a les 18.19, Jan Beulich ha escrit:
>>>>> On 23.07.15 at 18:15, <andrew.cooper3@citrix.com> wrote:
>>> On 23/07/15 17:00, Ian Campbell wrote:
>>>> On Thu, 2015-07-23 at 17:48 +0200, Roger Pau Monné wrote:
>>>>> El 23/07/15 a les 17.32, Jan Beulich ha escrit:
>>>>>>>>> On 23.07.15 at 17:10, <roger.pau@citrix.com> wrote:
>>>>>>> IMHO introducing a new structure that gets rid of all the PV-only 
>>>>>>>
>>>>>>> fields seems like a good option:
>>>>>>>
>>>>>>> struct vcpu_hvm_context {
>>>>>>> #define _VGCF_online                   5
>>>>>>> #define VGCF_online                    (1<<_VGCF_online)
>>>>>>>     uint32_t flags;                         /* VGCF_* flags      
>>>>>>>            
>>>>>>> */
>>>>>>>     struct cpu_hvm_user_regs user_regs;     /* User-level CPU 
>>>>>>> registers     
>>>>>>> */
>>>>>>>     /* NB. User pagetable on x86/64 is placed in ctrlreg[1]. */
>>>>>>>     uint32_t ctrlreg[8];                    /* CR0-CR7 (control 
>>>>>>> registers)  
>>>>>>> */
>>>>>>>     uint32_t debugreg[8];                   /* DB0-DB7 (debug 
>>>>>>> registers)    
>>>>>>> */
>>>>>>> };
>>>>>>>
>>>>>>> I'm also seriously considering getting rid of ctrlreg and 
>>>>>>> debugreg.
>>>>>>> Since HVM VCPUs will always be started in 32bit flat mode, it 
>>>>>>> doesn't
>>>>>>> make sense IMHO to have both the 32 and the 64 version of the 
>>>>>>> registers, so cpu_hvm_user_regs is always going to be:
>>>>>>>
>>>>>>> struct cpu_hvm_user_regs {
>>>>>>>     uint32_t ebx;
>>>>>>>     uint32_t ecx;
>>>>>>>     uint32_t edx;
>>>>>>>     uint32_t esi;
>>>>>>>     uint32_t edi;
>>>>>>>     uint32_t ebp;
>>>>>>>     uint32_t eax;
>>>>>>>     uint32_t eip;
>>>>>>>     uint32_t esp;
>>>>>>>     uint32_t eflags;
>>>>>>>     uint16_t cs;
>>>>>>>     uint16_t ss;
>>>>>>>     uint16_t es;
>>>>>>>     uint16_t ds;
>>>>>>>     uint16_t fs;
>>>>>>>     uint16_t gs;
>>>>>>> };
>>>>>>>
>>>>>>> We could however do something similar to what's done in ARM and 
>>>>>>> have
>>>>>>> a union of both the 32 and the 64bit registers in case we want to
>>>>>>> start the vCPU in 64bit mode sometime in the future.
>>>>>> What you gave above is suitable only for VCPUOP_initialise afaict.
>>>>>> Did you intend this to be the case?
>>>>> Certainly not, this should also be used by XEN_DOMCTL_setvcpucontext.
>>>>> I'm afraid I'm missing something obvious, but I don't see why this
>>>>> couldn't be used by XEN_DOMCTL_setvcpucontext TBH, can you please 
>>>>> clarify?
>>>> set/getvcpucontext need to pickle all state for save/restore/migration,
>>>> not just the start of day state.
>>> HVM migration doesn't use set/getvcpucontext.
>>>
>>> I would expect an HVMLite-based-PVH to follow suit and just use
>>> set/gethvmcontext.
>> Oh, right, but that doesn't mean the respective domctl-s should
>> produce unconsumable data, or have to guess how to interpret
>> what gets passed in. I.e. at the very minimum the domctl-s then
>> should be disallowed for PVH if they can#t be made work right.
> Right, the problem is that if I introduce this new vcpu_context
> structure plain HVM guests are also going to use it.
>
> This is not a problem with the VCPU_initialize op, because it was not
> available to HVM guests. But it's going to be a problem for
> XEN_DOMCTL_{get/set}vcpucontext consumers, since the structure
> taken/returned by the hypercall is going to change when the target is a
> HVM guest, and this is a can of worms I might prefer to avoid opening.
>
> One partial solution would be to allow the usage of this new struct only
> for the VCPU_initialize hypercall for HVM guests, and keep using the old
> structures for the domctls, aiming at removing them once PV support is
> dropped from Xen, but that looks half-baked IMHO.

PV and HVM "vcpu state" are fundamentally different because they are
fundamentally different kinds of guest.  Lets not try and mash them
together.

I would go with Jan's suggestion and make
XEN_DOMCTL_{get/set}vcpucontext ineligible for use on HVM guests.

The state of an HVMLite vcpu can already be set with
XEN_DOMCTL_sethvmcontext and a stream containing just a single hvm vcpu
record, and retrieved with XEN_DOMCTL_gethvmcontext_partial.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 23.07.15 at 19:12, <andrew.cooper3@citrix.com> wrote:
> I would go with Jan's suggestion and make
> XEN_DOMCTL_{get/set}vcpucontext ineligible for use on HVM guests.

Plus, as you said earlier, a new VCPUOP_initialize.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 10.22, Jan Beulich ha escrit:
>>>> On 23.07.15 at 19:12, <andrew.cooper3@citrix.com> wrote:
>> I would go with Jan's suggestion and make
>> XEN_DOMCTL_{get/set}vcpucontext ineligible for use on HVM guests.
> 
> Plus, as you said earlier, a new VCPUOP_initialize.

(Adding the Linux maintainers to get their opinion on the interface)

Just to recap and to make sure I got all the points:

 - vCPU initialization from the toolstack (BSP initialization) is going 
to be done using XEN_DOMCTL_sethvmcontext.
 - XEN_DOMCTL_{get/set}vcpucontext is going to return EOPNOTSUPP for 
HVM guests.
 - A new vcpu_guest_contatext (vcpu_hvm_context?) is going to be 
introduced together with a a new VCPUOP_initialize hypercall 
(VCPUOP_hvm_initialize?).

The following is a layout proposal for the new vcpu_context:

struct vcpu_hvm_context {
/* 32bit fields of the structure will be used. */
#define _VCPUHVM_MODE_32B              0
#define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
/* 64bit fields of the structure will be used. */
#define _VCPUHVM_MODE_64B              1
#define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
#define _VCPUHVM_online                2
#define VCPUHVM_online                 (1<<_VCPUHVM_online )
    uint32_t flags;                         /* VCPUHVM_* flags.   */
    struct cpu_hvm_regs user_regs;          /* CPU registers.     */
};

#if defined(__GNUC__) && !defined(__STRICT_ANSI__)
/* Anonymous union includes both 32- and 64-bit names (e.g., ebp/rbp). */
# define __DECL_REG(n64, n32) union {          \
        uint64_t n64;                          \
        uint32_t n32;                          \
    }
#else
/* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
#define __DECL_REG(n64, n32) uint64_t n64
#endif

#define __DECL_GP_REG(n) __DECL_REG(r##n, e##n)

struct cpu_hvm_regs {
    /* General purpose registers. */
    __DECL_GP_REG(bx);
    __DECL_GP_REG(cx);
    __DECL_GP_REG(dx);
    __DECL_GP_REG(si);
    __DECL_GP_REG(di);
    __DECL_GP_REG(bp);
    __DECL_GP_REG(ax);
    __DECL_GP_REG(ip);
    __DECL_GP_REG(sp);
    __DECL_GP_REG(flags);

    /* Control registers.         */
    uint64_t cr[8];
    /* Valid on amd64 only.       */
    uint64_t efer;

    /* Debug registers.           */
    uint64_t db[8];  
};

#undef __DECL_GP_REG
#undef __DECL_REG

Of course the APs will be allowed to start in any mode they wish, 
regardless of the mode the guest is currently running on.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 24.07.15 at 11:59, <roger.pau@citrix.com> wrote:
> Just to recap and to make sure I got all the points:
> 
>  - vCPU initialization from the toolstack (BSP initialization) is going 
> to be done using XEN_DOMCTL_sethvmcontext.
>  - XEN_DOMCTL_{get/set}vcpucontext is going to return EOPNOTSUPP for 
> HVM guests.
>  - A new vcpu_guest_contatext (vcpu_hvm_context?) is going to be 
> introduced together with a a new VCPUOP_initialize hypercall 
> (VCPUOP_hvm_initialize?).

Since the current one is called VCPUOP_initialise, why not just
VCPUOP_initialize? Having "hvm" in the name when it's meant to
also be used for PVH may end up being confusing.

> The following is a layout proposal for the new vcpu_context:
> 
> struct vcpu_hvm_context {
> /* 32bit fields of the structure will be used. */
> #define _VCPUHVM_MODE_32B              0
> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
> /* 64bit fields of the structure will be used. */
> #define _VCPUHVM_MODE_64B              1
> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)

16-bit mode?

> #define _VCPUHVM_online                2
> #define VCPUHVM_online                 (1<<_VCPUHVM_online )

What is this needed for?

>     uint32_t flags;                         /* VCPUHVM_* flags.   */
>     struct cpu_hvm_regs user_regs;          /* CPU registers.     */
> };
> 
> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
> /* Anonymous union includes both 32- and 64-bit names (e.g., ebp/rbp). */
> # define __DECL_REG(n64, n32) union {          \
>         uint64_t n64;                          \
>         uint32_t n32;                          \
>     }
> #else
> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
> #define __DECL_REG(n64, n32) uint64_t n64
> #endif
> 
> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n)
> 
> struct cpu_hvm_regs {
>     /* General purpose registers. */
>     __DECL_GP_REG(bx);
>     __DECL_GP_REG(cx);
>     __DECL_GP_REG(dx);
>     __DECL_GP_REG(si);
>     __DECL_GP_REG(di);
>     __DECL_GP_REG(bp);
>     __DECL_GP_REG(ax);
>     __DECL_GP_REG(ip);
>     __DECL_GP_REG(sp);

Please put GPRs in their natural order - no need to mimic the order
in any existing structure.

>     __DECL_GP_REG(flags);

r8-r11, selector and descriptor registers, pseudo descriptor registers.
Or else for all of them their default state would need to be spelled out.


>     /* Control registers.         */
>     uint64_t cr[8];
>     /* Valid on amd64 only.       */

Fields valid/useful in one mode only should probably be put in
union-ized sub-structures.

Jan

>     uint64_t efer;
> 
>     /* Debug registers.           */
>     uint64_t db[8];  
> };
> 
> #undef __DECL_GP_REG
> #undef __DECL_REG
> 
> Of course the APs will be allowed to start in any mode they wish, 
> regardless of the mode the guest is currently running on.
> 
> Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 12.46, Jan Beulich ha escrit:
>>>> On 24.07.15 at 11:59, <roger.pau@citrix.com> wrote:
>> Just to recap and to make sure I got all the points:
>>
>>  - vCPU initialization from the toolstack (BSP initialization) is going 
>> to be done using XEN_DOMCTL_sethvmcontext.
>>  - XEN_DOMCTL_{get/set}vcpucontext is going to return EOPNOTSUPP for 
>> HVM guests.
>>  - A new vcpu_guest_contatext (vcpu_hvm_context?) is going to be 
>> introduced together with a a new VCPUOP_initialize hypercall 
>> (VCPUOP_hvm_initialize?).
> 
> Since the current one is called VCPUOP_initialise, why not just
> VCPUOP_initialize? Having "hvm" in the name when it's meant to
> also be used for PVH may end up being confusing.

Right.

>> The following is a layout proposal for the new vcpu_context:
>>
>> struct vcpu_hvm_context {
>> /* 32bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_32B              0
>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>> /* 64bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_64B              1
>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
> 
> 16-bit mode?

I wasn't even considering that we would like to start the vCPUs in 16bit
mode, but given that's how APs are started on bare metal I guess it
makes sense to have it.

> 
>> #define _VCPUHVM_online                2
>> #define VCPUHVM_online                 (1<<_VCPUHVM_online )
> 
> What is this needed for?

It clears the VPF_down bit of the vcpu pause_flags, but we can also do
this using the VCPUOP_up hypercall so I'm just going to remove it.

>>     uint32_t flags;                         /* VCPUHVM_* flags.   */
>>     struct cpu_hvm_regs user_regs;          /* CPU registers.     */
>> };
>>
>> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
>> /* Anonymous union includes both 32- and 64-bit names (e.g., ebp/rbp). */
>> # define __DECL_REG(n64, n32) union {          \
>>         uint64_t n64;                          \
>>         uint32_t n32;                          \
>>     }
>> #else
>> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
>> #define __DECL_REG(n64, n32) uint64_t n64
>> #endif
>>
>> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n)
>>
>> struct cpu_hvm_regs {
>>     /* General purpose registers. */
>>     __DECL_GP_REG(bx);
>>     __DECL_GP_REG(cx);
>>     __DECL_GP_REG(dx);
>>     __DECL_GP_REG(si);
>>     __DECL_GP_REG(di);
>>     __DECL_GP_REG(bp);
>>     __DECL_GP_REG(ax);
>>     __DECL_GP_REG(ip);
>>     __DECL_GP_REG(sp);
> 
> Please put GPRs in their natural order - no need to mimic the order
> in any existing structure.

Ack.

>>     __DECL_GP_REG(flags);
> 
> r8-r11, selector and descriptor registers, pseudo descriptor registers.
> Or else for all of them their default state would need to be spelled out.

r8-r15 right?

> 
>>     /* Control registers.         */
>>     uint64_t cr[8];
>>     /* Valid on amd64 only.       */
> 
> Fields valid/useful in one mode only should probably be put in
> union-ized sub-structures.

Do you mean something like:

    union {
        uint64_t efer;
        uint32_t __invalid32;
        uint16_t __invalid16;
    }

It seems kind of pointless IMHO, the reason to have the union is to be
able to access the registers using the native nomenclature, but if a
register doesn't exist in a specific bitness I don't see the point of
adding such "invalid" names.

Or your idea was to put all the bitness specific registers inside of
another separate structure and then unionize them? AFAICT the 16 and
32bit structures are going to be empty.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 24.07.15 at 14:11, <roger.pau@citrix.com> wrote:
> El 24/07/15 a les 12.46, Jan Beulich ha escrit:
>>>>> On 24.07.15 at 11:59, <roger.pau@citrix.com> wrote:
>>>     __DECL_GP_REG(flags);
>> 
>> r8-r11, selector and descriptor registers, pseudo descriptor registers.
>> Or else for all of them their default state would need to be spelled out.
> 
> r8-r15 right?

Oh - of course.

>>>     /* Control registers.         */
>>>     uint64_t cr[8];
>>>     /* Valid on amd64 only.       */
>> 
>> Fields valid/useful in one mode only should probably be put in
>> union-ized sub-structures.
> 
> Do you mean something like:
> 
>     union {
>         uint64_t efer;
>         uint32_t __invalid32;
>         uint16_t __invalid16;
>     }
> 
> It seems kind of pointless IMHO, the reason to have the union is to be
> able to access the registers using the native nomenclature, but if a
> register doesn't exist in a specific bitness I don't see the point of
> adding such "invalid" names.

No - put side by side an item valid in only a subset modes and an
item only valid outside of that subset.

> Or your idea was to put all the bitness specific registers inside of
> another separate structure and then unionize them? AFAICT the 16 and
> 32bit structures are going to be empty.

How that? 64-bit mode e.g. doesn't need full descriptor data for
many of the segment registers.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 14.44, Jan Beulich ha escrit:
>>>> On 24.07.15 at 14:11, <roger.pau@citrix.com> wrote:
>> It seems kind of pointless IMHO, the reason to have the union is to be
>> able to access the registers using the native nomenclature, but if a
>> register doesn't exist in a specific bitness I don't see the point of
>> adding such "invalid" names.
> 
> No - put side by side an item valid in only a subset modes and an
> item only valid outside of that subset.

OK, let me see if I got that right.

>> Or your idea was to put all the bitness specific registers inside of
>> another separate structure and then unionize them? AFAICT the 16 and
>> 32bit structures are going to be empty.
> 
> How that? 64-bit mode e.g. doesn't need full descriptor data for
> many of the segment registers.

Please bear with me, but AFAIK cs, ds, es and ss still need to point to 
a GDT entry with a 0 base and a limit of 2^64.

What about the following layout:

#if defined(__GNUC__) && !defined(__STRICT_ANSI__)
/* Anonymous union includes 16-, 32- and 64-bit names (e.g., bp/ebp/rbp). */
# define __DECL_REG(n64, n32, n16) union {     \
        uint64_t n64;                          \
        uint32_t n32;                          \
        uint16_t n16;                          \
    }
#else
/* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
#define __DECL_REG(n64, n32, n16) uint64_t n64
#endif

#define __DECL_GP_REG(n) __DECL_REG(r##n, e##n, n)

struct cpu_x86_16 {
    /* Control registers. */
    uint32_t cr[8];
    /* Debug registers. */
    uint32_t dr[8];
    /* GDT descriptor address. */
    uint16_t gdtr;
};

struct cpu_x86_32 {
    /* Control registers. */
    uint32_t cr[8];
    /* Debug registers. */
    uint32_t dr[8];
    /* GDT descriptor address. */
    uint32_t gdtr;
};

struct cpu_x86_64 {
    /* Additional amd64 general purpose registers. */
    uint64_t r8, r9, r10, r11, r12, r13, r14, r15;
    /* Control registers. */
    uint64_t cr[9];
    /* Debug registers. */
    uint64_t dr[8];
    /* Extended Feature Enable Register. */
    uint64_t efer;
    /* GDT descriptor address. */
    uint64_t gdtr;
};

struct cpu_hvm_regs {
    /* General purpose registers. */
    __DECL_GP_REG(ax);
    __DECL_GP_REG(bx);
    __DECL_GP_REG(cx);
    __DECL_GP_REG(dx);

    /* Index registers. */
    __DECL_GP_REG(di);
    __DECL_GP_REG(si);

    /* Pointer registers. */
    __DECL_GP_REG(bp);
    __DECL_GP_REG(sp);

    /* Instruction pointer. */
    __DECL_GP_REG(ip);

    /* Status register. */
    __DECL_GP_REG(flags);

    /* Segment registers. */
    uint16_t cs;
    uint16_t ds;
    uint16_t es;
    uint16_t ss;
    uint16_t fs;
    uint16_t gs;

    /* Task state segment. */
    uint16_t ts;

    /* Bitness specific registers. */
    union {
        struct cpu_x86_16 x86_16;
        struct cpu_x86_32 x86_32;
        struct cpu_x86_64 x86_64;
    };
};

struct vcpu_hvm_context {
/* 16bit fields of the strucutre will be used. */
#define _VCPUHVM_MODE_16B              0
#define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
/* 32bit fields of the structure will be used. */
#define _VCPUHVM_MODE_32B              1
#define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
/* 64bit fields of the structure will be used. */
#define _VCPUHVM_MODE_64B              2
#define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
    uint32_t flags;                         /* VCPUHVM_* flags.   */
    struct cpu_hvm_regs user_regs;          /* CPU registers.     */
};

#undef __DECL_GP_REG
#undef __DECL_REG

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 24.07.15 at 17:26, <roger.pau@citrix.com> wrote:
> El 24/07/15 a les 14.44, Jan Beulich ha escrit:
>>>>> On 24.07.15 at 14:11, <roger.pau@citrix.com> wrote:
>>> Or your idea was to put all the bitness specific registers inside of
>>> another separate structure and then unionize them? AFAICT the 16 and
>>> 32bit structures are going to be empty.
>> 
>> How that? 64-bit mode e.g. doesn't need full descriptor data for
>> many of the segment registers.
> 
> Please bear with me, but AFAIK cs, ds, es and ss still need to point to 
> a GDT entry with a 0 base and a limit of 2^64.

No, in 64-bit mode bases are implicitly zero (with the exception of
fs and gs) and limits are implicitly 2^64-1 (which you can't even
express in a descriptor table entry, and with the exception of AMD's
Long Mode Segment Limit extension).

But looking at the code below I see that you're thinking of selector
values only, which would imply that you expect the hypervisor
to actually read descriptors from the descriptor tables. That's
doable, but cumbersome.

> What about the following layout:
> 
> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
> /* Anonymous union includes 16-, 32- and 64-bit names (e.g., bp/ebp/rbp). */
> # define __DECL_REG(n64, n32, n16) union {     \
>         uint64_t n64;                          \
>         uint32_t n32;                          \
>         uint16_t n16;                          \
>     }
> #else
> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
> #define __DECL_REG(n64, n32, n16) uint64_t n64
> #endif

Since you keep repeating this, I finally have to comment on it: This
is a suitable model for where we use it currently. It's imo not
suitable at all here - not the least because even non-gcc users
should be allowed to use just a 32-bit field when they mean one.
But the problem is also that for 16- and 32-bit mode a 64-bit
register is meaningless: Equally well you could expose r8-r15 to
them.

> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n, n)
> 
> struct cpu_x86_16 {
>     /* Control registers. */
>     uint32_t cr[8];
>     /* Debug registers. */
>     uint32_t dr[8];
>     /* GDT descriptor address. */
>     uint16_t gdtr;

Even in 16-bit mode this is a 32-bit base address. And you're
omitting the limit here an below (which is a required part namely
when you expect the hypervisor to access this table).

> };
> 
> struct cpu_x86_32 {
>     /* Control registers. */
>     uint32_t cr[8];
>     /* Debug registers. */
>     uint32_t dr[8];
>     /* GDT descriptor address. */
>     uint32_t gdtr;
> };
> 
> struct cpu_x86_64 {
>     /* Additional amd64 general purpose registers. */
>     uint64_t r8, r9, r10, r11, r12, r13, r14, r15;
>     /* Control registers. */
>     uint64_t cr[9];

[16]

>     /* Debug registers. */
>     uint64_t dr[8];
>     /* Extended Feature Enable Register. */
>     uint64_t efer;
>     /* GDT descriptor address. */
>     uint64_t gdtr;
> };
> 
> struct cpu_hvm_regs {
>     /* General purpose registers. */
>     __DECL_GP_REG(ax);
>     __DECL_GP_REG(bx);
>     __DECL_GP_REG(cx);
>     __DECL_GP_REG(dx);
> 
>     /* Index registers. */
>     __DECL_GP_REG(di);
>     __DECL_GP_REG(si);
> 
>     /* Pointer registers. */
>     __DECL_GP_REG(bp);
>     __DECL_GP_REG(sp);

Now didn't you agree to use the _natural_ order (i.e. according to
the numbers used to encode registers e.g. in instructions)?

> struct vcpu_hvm_context {
> /* 16bit fields of the strucutre will be used. */
> #define _VCPUHVM_MODE_16B              0
> #define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
> /* 32bit fields of the structure will be used. */
> #define _VCPUHVM_MODE_32B              1
> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
> /* 64bit fields of the structure will be used. */
> #define _VCPUHVM_MODE_64B              2
> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)

As soon as we have three values here, a boolean flag approach
doesn't make sense anymore.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 17.49, Jan Beulich ha escrit:
>>>> On 24.07.15 at 17:26, <roger.pau@citrix.com> wrote:
>> El 24/07/15 a les 14.44, Jan Beulich ha escrit:
>>>>>> On 24.07.15 at 14:11, <roger.pau@citrix.com> wrote:
>>>> Or your idea was to put all the bitness specific registers inside of
>>>> another separate structure and then unionize them? AFAICT the 16 and
>>>> 32bit structures are going to be empty.
>>>
>>> How that? 64-bit mode e.g. doesn't need full descriptor data for
>>> many of the segment registers.
>>
>> Please bear with me, but AFAIK cs, ds, es and ss still need to point to 
>> a GDT entry with a 0 base and a limit of 2^64.
> 
> No, in 64-bit mode bases are implicitly zero (with the exception of
> fs and gs) and limits are implicitly 2^64-1 (which you can't even
> express in a descriptor table entry, and with the exception of AMD's
> Long Mode Segment Limit extension).
> 
> But looking at the code below I see that you're thinking of selector
> values only, which would imply that you expect the hypervisor
> to actually read descriptors from the descriptor tables. That's
> doable, but cumbersome.

See the comment in the bottom of the email.

>> What about the following layout:
>>
>> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
>> /* Anonymous union includes 16-, 32- and 64-bit names (e.g., bp/ebp/rbp). */
>> # define __DECL_REG(n64, n32, n16) union {     \
>>         uint64_t n64;                          \
>>         uint32_t n32;                          \
>>         uint16_t n16;                          \
>>     }
>> #else
>> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
>> #define __DECL_REG(n64, n32, n16) uint64_t n64
>> #endif
> 
> Since you keep repeating this, I finally have to comment on it: This
> is a suitable model for where we use it currently. It's imo not
> suitable at all here - not the least because even non-gcc users
> should be allowed to use just a 32-bit field when they mean one.
> But the problem is also that for 16- and 32-bit mode a 64-bit
> register is meaningless: Equally well you could expose r8-r15 to
> them.

Sorry, I didn't get that feeling by reading your previous comments. So
you would prefer to pack everything inside of the cpu_x86_<bitness>
structures?

>> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n, n)
>>
>> struct cpu_x86_16 {
>>     /* Control registers. */
>>     uint32_t cr[8];
>>     /* Debug registers. */
>>     uint32_t dr[8];
>>     /* GDT descriptor address. */
>>     uint16_t gdtr;
> 
> Even in 16-bit mode this is a 32-bit base address. And you're
> omitting the limit here an below (which is a required part namely
> when you expect the hypervisor to access this table).

Well, I had in mind that this would be the memory address where the GDTR
base and limit is stored, just like the one you would use with the lgdt
instruction, but please read below.

>> };
>>
>> struct cpu_x86_32 {
>>     /* Control registers. */
>>     uint32_t cr[8];
>>     /* Debug registers. */
>>     uint32_t dr[8];
>>     /* GDT descriptor address. */
>>     uint32_t gdtr;
>> };
>>
>> struct cpu_x86_64 {
>>     /* Additional amd64 general purpose registers. */
>>     uint64_t r8, r9, r10, r11, r12, r13, r14, r15;
>>     /* Control registers. */
>>     uint64_t cr[9];
> 
> [16]

Why 16? Intel SDM section 2.5 only mentions cr8 as newly added in amd64.
TBH, I'm not sure it makes much sense to be able to set cr8 from this
interface, it can always be set by the guest OS and should not be
required for booting.

>>     /* Debug registers. */
>>     uint64_t dr[8];
>>     /* Extended Feature Enable Register. */
>>     uint64_t efer;
>>     /* GDT descriptor address. */
>>     uint64_t gdtr;
>> };
>>
>> struct cpu_hvm_regs {
>>     /* General purpose registers. */
>>     __DECL_GP_REG(ax);
>>     __DECL_GP_REG(bx);
>>     __DECL_GP_REG(cx);
>>     __DECL_GP_REG(dx);
>>
>>     /* Index registers. */
>>     __DECL_GP_REG(di);
>>     __DECL_GP_REG(si);
>>
>>     /* Pointer registers. */
>>     __DECL_GP_REG(bp);
>>     __DECL_GP_REG(sp);
> 
> Now didn't you agree to use the _natural_ order (i.e. according to
> the numbers used to encode registers e.g. in instructions)?

OK, sorry, I didn't understand it that way. I though you only wanted
them properly sorted (so a, b, c...) and labelled.

>> struct vcpu_hvm_context {
>> /* 16bit fields of the strucutre will be used. */
>> #define _VCPUHVM_MODE_16B              0
>> #define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
>> /* 32bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_32B              1
>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>> /* 64bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_64B              2
>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
> 
> As soon as we have three values here, a boolean flag approach
> doesn't make sense anymore.

Yes, taking into account that only _one_ of them can be used at the same
time.

I have the feeling that we are over engineering this interface. IMHO we
should only allow the user to set the control registers, efer (on amd64)
and the GP registers. Segment selectors would be set by Xen to point to
a flat segment suitable for the mode the guest has selected. I think
that should be enough to get the guest OS into it's entry point, and
then it can do whatever it wants.

If that's not suitable, then my second option would be to allow the
guest to set the base, limit and AR bytes of the selectors, but not load
a GDT.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Konrad Rzeszutek Wilk]

On Fri, Jul 24, 2015 at 06:54:09PM +0200, Roger Pau Monné wrote:
> El 24/07/15 a les 17.49, Jan Beulich ha escrit:
> >>>> On 24.07.15 at 17:26, <roger.pau@citrix.com> wrote:
> >> El 24/07/15 a les 14.44, Jan Beulich ha escrit:
> >>>>>> On 24.07.15 at 14:11, <roger.pau@citrix.com> wrote:
> >>>> Or your idea was to put all the bitness specific registers inside of
> >>>> another separate structure and then unionize them? AFAICT the 16 and
> >>>> 32bit structures are going to be empty.
> >>>
> >>> How that? 64-bit mode e.g. doesn't need full descriptor data for
> >>> many of the segment registers.
> >>
> >> Please bear with me, but AFAIK cs, ds, es and ss still need to point to 
> >> a GDT entry with a 0 base and a limit of 2^64.
> > 
> > No, in 64-bit mode bases are implicitly zero (with the exception of
> > fs and gs) and limits are implicitly 2^64-1 (which you can't even
> > express in a descriptor table entry, and with the exception of AMD's
> > Long Mode Segment Limit extension).
> > 
> > But looking at the code below I see that you're thinking of selector
> > values only, which would imply that you expect the hypervisor
> > to actually read descriptors from the descriptor tables. That's
> > doable, but cumbersome.
> 
> See the comment in the bottom of the email.
> 
> >> What about the following layout:
> >>
> >> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
> >> /* Anonymous union includes 16-, 32- and 64-bit names (e.g., bp/ebp/rbp). */
> >> # define __DECL_REG(n64, n32, n16) union {     \
> >>         uint64_t n64;                          \
> >>         uint32_t n32;                          \
> >>         uint16_t n16;                          \
> >>     }
> >> #else
> >> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
> >> #define __DECL_REG(n64, n32, n16) uint64_t n64
> >> #endif
> > 
> > Since you keep repeating this, I finally have to comment on it: This
> > is a suitable model for where we use it currently. It's imo not
> > suitable at all here - not the least because even non-gcc users
> > should be allowed to use just a 32-bit field when they mean one.
> > But the problem is also that for 16- and 32-bit mode a 64-bit
> > register is meaningless: Equally well you could expose r8-r15 to
> > them.
> 
> Sorry, I didn't get that feeling by reading your previous comments. So
> you would prefer to pack everything inside of the cpu_x86_<bitness>
> structures?
> 
> >> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n, n)
> >>
> >> struct cpu_x86_16 {
> >>     /* Control registers. */
> >>     uint32_t cr[8];
> >>     /* Debug registers. */
> >>     uint32_t dr[8];
> >>     /* GDT descriptor address. */
> >>     uint16_t gdtr;
> > 
> > Even in 16-bit mode this is a 32-bit base address. And you're
> > omitting the limit here an below (which is a required part namely
> > when you expect the hypervisor to access this table).
> 
> Well, I had in mind that this would be the memory address where the GDTR
> base and limit is stored, just like the one you would use with the lgdt
> instruction, but please read below.
> 
> >> };
> >>
> >> struct cpu_x86_32 {
> >>     /* Control registers. */
> >>     uint32_t cr[8];
> >>     /* Debug registers. */
> >>     uint32_t dr[8];
> >>     /* GDT descriptor address. */
> >>     uint32_t gdtr;
> >> };
> >>
> >> struct cpu_x86_64 {
> >>     /* Additional amd64 general purpose registers. */
> >>     uint64_t r8, r9, r10, r11, r12, r13, r14, r15;
> >>     /* Control registers. */
> >>     uint64_t cr[9];
> > 
> > [16]
> 
> Why 16? Intel SDM section 2.5 only mentions cr8 as newly added in amd64.
> TBH, I'm not sure it makes much sense to be able to set cr8 from this
> interface, it can always be set by the guest OS and should not be
> required for booting.
> 
> >>     /* Debug registers. */
> >>     uint64_t dr[8];
> >>     /* Extended Feature Enable Register. */
> >>     uint64_t efer;
> >>     /* GDT descriptor address. */
> >>     uint64_t gdtr;
> >> };
> >>
> >> struct cpu_hvm_regs {
> >>     /* General purpose registers. */
> >>     __DECL_GP_REG(ax);
> >>     __DECL_GP_REG(bx);
> >>     __DECL_GP_REG(cx);
> >>     __DECL_GP_REG(dx);
> >>
> >>     /* Index registers. */
> >>     __DECL_GP_REG(di);
> >>     __DECL_GP_REG(si);
> >>
> >>     /* Pointer registers. */
> >>     __DECL_GP_REG(bp);
> >>     __DECL_GP_REG(sp);
> > 
> > Now didn't you agree to use the _natural_ order (i.e. according to
> > the numbers used to encode registers e.g. in instructions)?
> 
> OK, sorry, I didn't understand it that way. I though you only wanted
> them properly sorted (so a, b, c...) and labelled.
> 
> >> struct vcpu_hvm_context {
> >> /* 16bit fields of the strucutre will be used. */
> >> #define _VCPUHVM_MODE_16B              0
> >> #define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
> >> /* 32bit fields of the structure will be used. */
> >> #define _VCPUHVM_MODE_32B              1
> >> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
> >> /* 64bit fields of the structure will be used. */
> >> #define _VCPUHVM_MODE_64B              2
> >> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
> > 
> > As soon as we have three values here, a boolean flag approach
> > doesn't make sense anymore.
> 
> Yes, taking into account that only _one_ of them can be used at the same
> time.
> 
> I have the feeling that we are over engineering this interface. IMHO we
> should only allow the user to set the control registers, efer (on amd64)
> and the GP registers. Segment selectors would be set by Xen to point to
> a flat segment suitable for the mode the guest has selected. I think
> that should be enough to get the guest OS into it's entry point, and
> then it can do whatever it wants.

If you are doing that why not use the old interface?

> 
> If that's not suitable, then my second option would be to allow the
> guest to set the base, limit and AR bytes of the selectors, but not load
> a GDT.

They should be able to do any of those operations as a normal HVM
guest I would think?

> 
> Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 19.36, Konrad Rzeszutek Wilk ha escrit:
> On Fri, Jul 24, 2015 at 06:54:09PM +0200, Roger Pau Monné wrote:
>> I have the feeling that we are over engineering this interface. IMHO we
>> should only allow the user to set the control registers, efer (on amd64)
>> and the GP registers. Segment selectors would be set by Xen to point to
>> a flat segment suitable for the mode the guest has selected. I think
>> that should be enough to get the guest OS into it's entry point, and
>> then it can do whatever it wants.
> 
> If you are doing that why not use the old interface?

Because the current structure contains quite a lot of PV fields, like
the failsafe_callback, the GDT... Also the old structure is limited in a
way that you can only use the x86-<bitness> format if your kernel is
compiled with <bitness>, and we want to allow the vCPU to start in any
mode, regardless of the mode the kernel is compiled with.

>>
>> If that's not suitable, then my second option would be to allow the
>> guest to set the base, limit and AR bytes of the selectors, but not load
>> a GDT.
> 
> They should be able to do any of those operations as a normal HVM
> guest I would think?

Yes, hence I would like to limit setting anything that's not essential
to boot.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 24/07/15 17:54, Roger Pau Monné wrote:
>
>>> struct vcpu_hvm_context {
>>> /* 16bit fields of the strucutre will be used. */
>>> #define _VCPUHVM_MODE_16B              0
>>> #define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
>>> /* 32bit fields of the structure will be used. */
>>> #define _VCPUHVM_MODE_32B              1
>>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>>> /* 64bit fields of the structure will be used. */
>>> #define _VCPUHVM_MODE_64B              2
>>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
>> As soon as we have three values here, a boolean flag approach
>> doesn't make sense anymore.
> Yes, taking into account that only _one_ of them can be used at the same
> time.
>
> I have the feeling that we are over engineering this interface. IMHO we
> should only allow the user to set the control registers, efer (on amd64)
> and the GP registers. Segment selectors would be set by Xen to point to
> a flat segment suitable for the mode the guest has selected. I think
> that should be enough to get the guest OS into it's entry point, and
> then it can do whatever it wants.
>
> If that's not suitable, then my second option would be to allow the
> guest to set the base, limit and AR bytes of the selectors, but not load
> a GDT.

In an ideal world, it would be nice to pass enough state to be able to
point eip at the idle loop and have everything JustWork.  However, this
makes a lot of assumptions about exactly which state the vcpu wants to
set up, which might include MSRs as well.

Therefore, an appropriate compromise is to be able to point eip at
startup_cpu, running in the correct operating mode to avoid bouncing
through trampolines.

>From that point of view, the minimum quantity of state required is:

CR{0,3,4}, EFER, cs/eip, ds, ss/esp, gdt{base,limit}.  It would be
natural to extend this to all the 8 base GPRs and the user segment
selectors.  (Note that EFER is needed even for 32bit paged entries if NX
is set in the pagetables.)

In addition, a hint to Xen as to the eventual mode so it can widen its
reads to start with rather than attempting to reverse engineer the
operating mode and rereading half the structure later. 

Specifying the gdt{base,limit} will require Xen to read the GDT to set
up the entry attributes for the segment, as simply setting the selector
is insufficient.  A complicating factor is that VT-x and SVM have
different representations for access rights.  (For migration, VT-x's
representation is mutated to match SVM.)

I really undecided between suggesting a gdt{base,limit} and Xen reading
the access rights in an unambiguous fashon, vs specifying the internals
and avoiding any peeking into domain memory.

It would be nice if the new cpus gs base could be set by the caller,
which allows the new cpu straight access to its per-cpu data area (if gs
is in use).  This is not representable using the gdt method and would
involve casing the GSBASE MSR if we were to go down the strict
architectural route.

Therefore, I am leaning slightly towards the specify the internals side
of things, which removes some complexity from the Xen side of the hypercall.

~Andrew

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 03/08/15 a les 18.55, Andrew Cooper ha escrit:
> On 24/07/15 17:54, Roger Pau Monné wrote:
>>
>>>> struct vcpu_hvm_context {
>>>> /* 16bit fields of the strucutre will be used. */
>>>> #define _VCPUHVM_MODE_16B              0
>>>> #define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
>>>> /* 32bit fields of the structure will be used. */
>>>> #define _VCPUHVM_MODE_32B              1
>>>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>>>> /* 64bit fields of the structure will be used. */
>>>> #define _VCPUHVM_MODE_64B              2
>>>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
>>> As soon as we have three values here, a boolean flag approach
>>> doesn't make sense anymore.
>> Yes, taking into account that only _one_ of them can be used at the same
>> time.
>>
>> I have the feeling that we are over engineering this interface. IMHO we
>> should only allow the user to set the control registers, efer (on amd64)
>> and the GP registers. Segment selectors would be set by Xen to point to
>> a flat segment suitable for the mode the guest has selected. I think
>> that should be enough to get the guest OS into it's entry point, and
>> then it can do whatever it wants.
>>
>> If that's not suitable, then my second option would be to allow the
>> guest to set the base, limit and AR bytes of the selectors, but not load
>> a GDT.
> 
> In an ideal world, it would be nice to pass enough state to be able to
> point eip at the idle loop and have everything JustWork.  However, this
> makes a lot of assumptions about exactly which state the vcpu wants to
> set up, which might include MSRs as well.
>
> Therefore, an appropriate compromise is to be able to point eip at
> startup_cpu, running in the correct operating mode to avoid bouncing
> through trampolines.

IMHO, allowing OSes to jump straight into the idle loop is putting a 
lot of burden in the Xen side of things. Also all OSes already contain 
AP startup code, we should be able to hook into this code at a suitable 
location, and that's what we should aim for.

>>From that point of view, the minimum quantity of state required is:
> 
> CR{0,3,4}, EFER, cs/eip, ds, ss/esp, gdt{base,limit}.  It would be
> natural to extend this to all the 8 base GPRs and the user segment
> selectors.  (Note that EFER is needed even for 32bit paged entries if NX
> is set in the pagetables.)

Extending this to the rest of the GPR is needed for Linux to boot. The 
AP startup code of PVH Linux places the cpu id in a GPR for the boot 
stub to find it:

http://lxr.free-electrons.com/source/arch/x86/xen/smp.c#L420

> 
> In addition, a hint to Xen as to the eventual mode so it can widen its
> reads to start with rather than attempting to reverse engineer the
> operating mode and rereading half the structure later. 
> 
> Specifying the gdt{base,limit} will require Xen to read the GDT to set
> up the entry attributes for the segment, as simply setting the selector
> is insufficient.  A complicating factor is that VT-x and SVM have
> different representations for access rights.  (For migration, VT-x's
> representation is mutated to match SVM.)
> 
> I really undecided between suggesting a gdt{base,limit} and Xen reading
> the access rights in an unambiguous fashon, vs specifying the internals
> and avoiding any peeking into domain memory.
> 
> It would be nice if the new cpus gs base could be set by the caller,
> which allows the new cpu straight access to its per-cpu data area (if gs
> is in use).  This is not representable using the gdt method and would
> involve casing the GSBASE MSR if we were to go down the strict
> architectural route.

IMHO, this is out of the scope and I would rather allow the caller to 
set the internal (cached) part of the segment selectors.

> Therefore, I am leaning slightly towards the specify the internals side
> of things, which removes some complexity from the Xen side of the hypercall.

I've updated the proposed structure, so now it looks like:

(I still need to figure out how to order the bits in the access rights 
(_ar) fields.)

struct vcpu_hvm_x86_16 {
    uint16_t ax;
    uint16_t cx;
    uint16_t dx;
    uint16_t bx;
    uint16_t sp;
    uint16_t bp;
    uint16_t si;
    uint16_t di;
    uint16_t ip;

    uint32_t cr[8];

    uint32_t cs_base;
    uint32_t ds_base;
    uint32_t ss_base;
    uint32_t cs_limit;
    uint32_t ds_limit;
    uint32_t ss_limit;
    uint16_t cs_ar;
    uint16_t ds_ar;
    uint16_t ss_ar;
};

struct vcpu_hvm_x86_32 {
    uint32_t eax;
    uint32_t ecx;
    uint32_t edx;
    uint32_t ebx;
    uint32_t esp;
    uint32_t ebp;
    uint32_t esi;
    uint32_t edi;
    uint32_t eip;

    uint32_t cr[8];
    uint64_t efer;

    uint32_t cs_base;
    uint32_t ds_base;
    uint32_t ss_base;
    uint32_t cs_limit;
    uint32_t ds_limit;
    uint32_t ss_limit;
    uint16_t cs_ar;
    uint16_t ds_ar;
    uint16_t ss_ar;
};

struct vcpu_hvm_x86_64 {
    uint64_t rax;
    uint64_t rcx;
    uint64_t rdx;
    uint64_t rbx;
    uint64_t rsp;
    uint64_t rbp;
    uint64_t rsi;
    uint64_t rdi;
    uint64_t r8;
    uint64_t r9;
    uint64_t r10;
    uint64_t r11;
    uint64_t r12;
    uint64_t r13;
    uint64_t r14;
    uint64_t r15;
    uint64_t rip;

    uint64_t cr[8];
    uint64_t efer;
};

typedef enum {
    VCPU_HVM_MODE_16B,       /* 16bit fields of the structure will be used. */
    VCPU_HVM_MODE_32B,       /* 32bit fields of the structure will be used. */
    VCPU_HVM_MODE_64B,       /* 64bit fields of the structure will be used. */
} vcpu_hvm_mode_t;

struct vcpu_hvm_context {
    vcpu_hvm_mode_t mode;

    /* CPU registers. */
    union {
        struct vcpu_hvm_x86_16 x86_16;
        struct vcpu_hvm_x86_32 x86_32;
        struct vcpu_hvm_x86_64 x86_64;
    };
};
typedef struct vcpu_hvm_context vcpu_hvm_context_t;
DEFINE_XEN_GUEST_HANDLE(vcpu_hvm_context_t);

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 03/08/15 18:31, Roger Pau Monné wrote:
> El 03/08/15 a les 18.55, Andrew Cooper ha escrit:
>> On 24/07/15 17:54, Roger Pau Monné wrote:
>>>>> struct vcpu_hvm_context {
>>>>> /* 16bit fields of the strucutre will be used. */
>>>>> #define _VCPUHVM_MODE_16B              0
>>>>> #define VCPUHVM_MODE_16B               (1<<_VCPUHVM_MODE_16B)
>>>>> /* 32bit fields of the structure will be used. */
>>>>> #define _VCPUHVM_MODE_32B              1
>>>>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>>>>> /* 64bit fields of the structure will be used. */
>>>>> #define _VCPUHVM_MODE_64B              2
>>>>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
>>>> As soon as we have three values here, a boolean flag approach
>>>> doesn't make sense anymore.
>>> Yes, taking into account that only _one_ of them can be used at the same
>>> time.
>>>
>>> I have the feeling that we are over engineering this interface. IMHO we
>>> should only allow the user to set the control registers, efer (on amd64)
>>> and the GP registers. Segment selectors would be set by Xen to point to
>>> a flat segment suitable for the mode the guest has selected. I think
>>> that should be enough to get the guest OS into it's entry point, and
>>> then it can do whatever it wants.
>>>
>>> If that's not suitable, then my second option would be to allow the
>>> guest to set the base, limit and AR bytes of the selectors, but not load
>>> a GDT.
>> In an ideal world, it would be nice to pass enough state to be able to
>> point eip at the idle loop and have everything JustWork.  However, this
>> makes a lot of assumptions about exactly which state the vcpu wants to
>> set up, which might include MSRs as well.
>>
>> Therefore, an appropriate compromise is to be able to point eip at
>> startup_cpu, running in the correct operating mode to avoid bouncing
>> through trampolines.
> IMHO, allowing OSes to jump straight into the idle loop is putting a 
> lot of burden in the Xen side of things. Also all OSes already contain 
> AP startup code, we should be able to hook into this code at a suitable 
> location, and that's what we should aim for.
>
>> >From that point of view, the minimum quantity of state required is:
>>
>> CR{0,3,4}, EFER, cs/eip, ds, ss/esp, gdt{base,limit}.  It would be
>> natural to extend this to all the 8 base GPRs and the user segment
>> selectors.  (Note that EFER is needed even for 32bit paged entries if NX
>> is set in the pagetables.)
> Extending this to the rest of the GPR is needed for Linux to boot. The 
> AP startup code of PVH Linux places the cpu id in a GPR for the boot 
> stub to find it:
>
> http://lxr.free-electrons.com/source/arch/x86/xen/smp.c#L420
>
>> In addition, a hint to Xen as to the eventual mode so it can widen its
>> reads to start with rather than attempting to reverse engineer the
>> operating mode and rereading half the structure later. 
>>
>> Specifying the gdt{base,limit} will require Xen to read the GDT to set
>> up the entry attributes for the segment, as simply setting the selector
>> is insufficient.  A complicating factor is that VT-x and SVM have
>> different representations for access rights.  (For migration, VT-x's
>> representation is mutated to match SVM.)
>>
>> I really undecided between suggesting a gdt{base,limit} and Xen reading
>> the access rights in an unambiguous fashon, vs specifying the internals
>> and avoiding any peeking into domain memory.
>>
>> It would be nice if the new cpus gs base could be set by the caller,
>> which allows the new cpu straight access to its per-cpu data area (if gs
>> is in use).  This is not representable using the gdt method and would
>> involve casing the GSBASE MSR if we were to go down the strict
>> architectural route.
> IMHO, this is out of the scope and I would rather allow the caller to 
> set the internal (cached) part of the segment selectors.
>
>> Therefore, I am leaning slightly towards the specify the internals side
>> of things, which removes some complexity from the Xen side of the hypercall.
> I've updated the proposed structure, so now it looks like:
>
> (I still need to figure out how to order the bits in the access rights 
> (_ar) fields.)

All struct's need a flags register.

>
> struct vcpu_hvm_x86_16 {
>     uint16_t ax;
>     uint16_t cx;
>     uint16_t dx;
>     uint16_t bx;
>     uint16_t sp;
>     uint16_t bp;
>     uint16_t si;
>     uint16_t di;
>     uint16_t ip;
>
>     uint32_t cr[8];

Definitely no need for anything other than cr0 and 4 in 16 bit mode.

>
>     uint32_t cs_base;
>     uint32_t ds_base;
>     uint32_t ss_base;
>     uint32_t cs_limit;
>     uint32_t ds_limit;
>     uint32_t ss_limit;
>     uint16_t cs_ar;
>     uint16_t ds_ar;
>     uint16_t ss_ar;
> };
>
> struct vcpu_hvm_x86_32 {
>     uint32_t eax;
>     uint32_t ecx;
>     uint32_t edx;
>     uint32_t ebx;
>     uint32_t esp;
>     uint32_t ebp;
>     uint32_t esi;
>     uint32_t edi;
>     uint32_t eip;
>
>     uint32_t cr[8];

Don't need cr's 5-8.

>     uint64_t efer;
>
>     uint32_t cs_base;
>     uint32_t ds_base;
>     uint32_t ss_base;
>     uint32_t cs_limit;
>     uint32_t ds_limit;
>     uint32_t ss_limit;
>     uint16_t cs_ar;
>     uint16_t ds_ar;
>     uint16_t ss_ar;

You need selector entries for each segment as well.

> };
>
> struct vcpu_hvm_x86_64 {
>     uint64_t rax;
>     uint64_t rcx;
>     uint64_t rdx;
>     uint64_t rbx;
>     uint64_t rsp;
>     uint64_t rbp;
>     uint64_t rsi;
>     uint64_t rdi;
>     uint64_t r8;
>     uint64_t r9;
>     uint64_t r10;
>     uint64_t r11;
>     uint64_t r12;
>     uint64_t r13;
>     uint64_t r14;
>     uint64_t r15;
>     uint64_t rip;
>
>     uint64_t cr[8];
>     uint64_t efer;

What has happened to the segment information?  It cannot be omitted
completely, even in 64bit.

> };
>
> typedef enum {
>     VCPU_HVM_MODE_16B,       /* 16bit fields of the structure will be used. */
>     VCPU_HVM_MODE_32B,       /* 32bit fields of the structure will be used. */
>     VCPU_HVM_MODE_64B,       /* 64bit fields of the structure will be used. */
> } vcpu_hvm_mode_t;
>
> struct vcpu_hvm_context {
>     vcpu_hvm_mode_t mode;

The width of an enum is implementation defined, which makes them
unsuitable in the public interface.

>
>     /* CPU registers. */
>     union {
>         struct vcpu_hvm_x86_16 x86_16;
>         struct vcpu_hvm_x86_32 x86_32;
>         struct vcpu_hvm_x86_64 x86_64;
>     };

We require C89 compatibility for the public interface, so no anonymous
unions.

~Andrew

> };
> typedef struct vcpu_hvm_context vcpu_hvm_context_t;
> DEFINE_XEN_GUEST_HANDLE(vcpu_hvm_context_t);
>
>

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
> On 03/08/15 18:31, Roger Pau Monné wrote:
>>> Therefore, I am leaning slightly towards the specify the internals side
>>> of things, which removes some complexity from the Xen side of the hypercall.
>> I've updated the proposed structure, so now it looks like:
>>
>> (I still need to figure out how to order the bits in the access rights 
>> (_ar) fields.)
> 
> All struct's need a flags register.

I was unsure about adding the {e/r}flags register, I will add it in new
versions.

>>
>> struct vcpu_hvm_x86_16 {
>>     uint16_t ax;
>>     uint16_t cx;
>>     uint16_t dx;
>>     uint16_t bx;
>>     uint16_t sp;
>>     uint16_t bp;
>>     uint16_t si;
>>     uint16_t di;
>>     uint16_t ip;
>>
>>     uint32_t cr[8];
> 
> Definitely no need for anything other than cr0 and 4 in 16 bit mode.

Yes. Would you rather prefer to spell out the exact control registers
that are going to be used instead of using an array?

For 16bit mode this is going to be CR0/CR4, for 32bit or long mode mode
CR0/CR3/CR4.

>>
>>     uint32_t cs_base;
>>     uint32_t ds_base;
>>     uint32_t ss_base;
>>     uint32_t cs_limit;
>>     uint32_t ds_limit;
>>     uint32_t ss_limit;
>>     uint16_t cs_ar;
>>     uint16_t ds_ar;
>>     uint16_t ss_ar;
>> };
>>
>> struct vcpu_hvm_x86_32 {
>>     uint32_t eax;
>>     uint32_t ecx;
>>     uint32_t edx;
>>     uint32_t ebx;
>>     uint32_t esp;
>>     uint32_t ebp;
>>     uint32_t esi;
>>     uint32_t edi;
>>     uint32_t eip;
>>
>>     uint32_t cr[8];
> 
> Don't need cr's 5-8.
> 
>>     uint64_t efer;
>>
>>     uint32_t cs_base;
>>     uint32_t ds_base;
>>     uint32_t ss_base;
>>     uint32_t cs_limit;
>>     uint32_t ds_limit;
>>     uint32_t ss_limit;
>>     uint16_t cs_ar;
>>     uint16_t ds_ar;
>>     uint16_t ss_ar;
> 
> You need selector entries for each segment as well.

Really? What's the point in having the selector if we don't have a GDT,
and we allow loading the cached part, which is the relevant one.

> 
>> };
>>
>> struct vcpu_hvm_x86_64 {
>>     uint64_t rax;
>>     uint64_t rcx;
>>     uint64_t rdx;
>>     uint64_t rbx;
>>     uint64_t rsp;
>>     uint64_t rbp;
>>     uint64_t rsi;
>>     uint64_t rdi;
>>     uint64_t r8;
>>     uint64_t r9;
>>     uint64_t r10;
>>     uint64_t r11;
>>     uint64_t r12;
>>     uint64_t r13;
>>     uint64_t r14;
>>     uint64_t r15;
>>     uint64_t rip;
>>
>>     uint64_t cr[8];
>>     uint64_t efer;
> 
> What has happened to the segment information?  It cannot be omitted
> completely, even in 64bit.

I had in mind to set them to the values required for long mode:

base = 0
limit = 0xffff

and the attributes field for CS to:

ar = 0x29b (L bit set)

And for SS/DS:

ar = 0x093

But maybe it makes sense to allow setting them in case someone wants to
start in compatibility mode.

>> };
>>
>> typedef enum {
>>     VCPU_HVM_MODE_16B,       /* 16bit fields of the structure will be used. */
>>     VCPU_HVM_MODE_32B,       /* 32bit fields of the structure will be used. */
>>     VCPU_HVM_MODE_64B,       /* 64bit fields of the structure will be used. */
>> } vcpu_hvm_mode_t;
>>
>> struct vcpu_hvm_context {
>>     vcpu_hvm_mode_t mode;
> 
> The width of an enum is implementation defined, which makes them
> unsuitable in the public interface.

Right, I'm going to change it to a uint32_t and the modes to defines.

> 
>>
>>     /* CPU registers. */
>>     union {
>>         struct vcpu_hvm_x86_16 x86_16;
>>         struct vcpu_hvm_x86_32 x86_32;
>>         struct vcpu_hvm_x86_64 x86_64;
>>     };
> 
> We require C89 compatibility for the public interface, so no anonymous
> unions.

Ack, thanks for the review.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 05/08/15 10:53, Roger Pau Monné wrote:
> El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
>> On 03/08/15 18:31, Roger Pau Monné wrote:
>>>> Therefore, I am leaning slightly towards the specify the internals side
>>>> of things, which removes some complexity from the Xen side of the hypercall.
>>> I've updated the proposed structure, so now it looks like:
>>>
>>> (I still need to figure out how to order the bits in the access rights 
>>> (_ar) fields.)
>> All struct's need a flags register.
> I was unsure about adding the {e/r}flags register, I will add it in new
> versions.

For first-gen VT-x, CR0.PE in unable to be cleared.   HVMLoader has to
do a dance with vm86 mode, which is where HVM_PARAM_IDENT_PT gets used.

I don't realistically expect this to be a problem for DMlite guests.

>
>>> struct vcpu_hvm_x86_16 {
>>>     uint16_t ax;
>>>     uint16_t cx;
>>>     uint16_t dx;
>>>     uint16_t bx;
>>>     uint16_t sp;
>>>     uint16_t bp;
>>>     uint16_t si;
>>>     uint16_t di;
>>>     uint16_t ip;
>>>
>>>     uint32_t cr[8];
>> Definitely no need for anything other than cr0 and 4 in 16 bit mode.
> Yes. Would you rather prefer to spell out the exact control registers
> that are going to be used instead of using an array?

I would suggest so.

CRs 1,5-7 are unconditionally #UD to attempt to use. CR2 is useless
until you enter the #PF handler. CR8 may (hardware dependent) have the
TPR in it, which is useless until the IDT is set up and interrupts have
been enabled.

>
> For 16bit mode this is going to be CR0/CR4, for 32bit or long mode mode
> CR0/CR3/CR4.

Agreed.

>
>>>     uint32_t cs_base;
>>>     uint32_t ds_base;
>>>     uint32_t ss_base;
>>>     uint32_t cs_limit;
>>>     uint32_t ds_limit;
>>>     uint32_t ss_limit;
>>>     uint16_t cs_ar;
>>>     uint16_t ds_ar;
>>>     uint16_t ss_ar;
>>> };
>>>
>>> struct vcpu_hvm_x86_32 {
>>>     uint32_t eax;
>>>     uint32_t ecx;
>>>     uint32_t edx;
>>>     uint32_t ebx;
>>>     uint32_t esp;
>>>     uint32_t ebp;
>>>     uint32_t esi;
>>>     uint32_t edi;
>>>     uint32_t eip;
>>>
>>>     uint32_t cr[8];
>> Don't need cr's 5-8.
>>
>>>     uint64_t efer;
>>>
>>>     uint32_t cs_base;
>>>     uint32_t ds_base;
>>>     uint32_t ss_base;
>>>     uint32_t cs_limit;
>>>     uint32_t ds_limit;
>>>     uint32_t ss_limit;
>>>     uint16_t cs_ar;
>>>     uint16_t ds_ar;
>>>     uint16_t ss_ar;
>> You need selector entries for each segment as well.
> Really? What's the point in having the selector if we don't have a GDT,
> and we allow loading the cached part, which is the relevant one.

push %cs; push 1f; lret

At all points when segment details are updated, it is the responsibility
of software to ensure that the details match with the GDT/LDT entry. 
See for example the Intel and AMD manuals for syscall/sysenter where
similar "updating segment details behind the scenes" occurs.

>
>>> };
>>>
>>> struct vcpu_hvm_x86_64 {
>>>     uint64_t rax;
>>>     uint64_t rcx;
>>>     uint64_t rdx;
>>>     uint64_t rbx;
>>>     uint64_t rsp;
>>>     uint64_t rbp;
>>>     uint64_t rsi;
>>>     uint64_t rdi;
>>>     uint64_t r8;
>>>     uint64_t r9;
>>>     uint64_t r10;
>>>     uint64_t r11;
>>>     uint64_t r12;
>>>     uint64_t r13;
>>>     uint64_t r14;
>>>     uint64_t r15;
>>>     uint64_t rip;
>>>
>>>     uint64_t cr[8];
>>>     uint64_t efer;
>> What has happened to the segment information?  It cannot be omitted
>> completely, even in 64bit.
> I had in mind to set them to the values required for long mode:
>
> base = 0
> limit = 0xffff
>
> and the attributes field for CS to:
>
> ar = 0x29b (L bit set)
>
> And for SS/DS:
>
> ar = 0x093
>
> But maybe it makes sense to allow setting them in case someone wants to
> start in compatibility mode.

Agreed.

~Andrew

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 05/08/15 a les 17.39, Andrew Cooper ha escrit:
> On 05/08/15 10:53, Roger Pau Monné wrote:
>> El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
>>> On 03/08/15 18:31, Roger Pau Monné wrote:
>>>>     uint32_t cs_base;
>>>>     uint32_t ds_base;
>>>>     uint32_t ss_base;
>>>>     uint32_t cs_limit;
>>>>     uint32_t ds_limit;
>>>>     uint32_t ss_limit;
>>>>     uint16_t cs_ar;
>>>>     uint16_t ds_ar;
>>>>     uint16_t ss_ar;
>>> You need selector entries for each segment as well.
>> Really? What's the point in having the selector if we don't have a GDT,
>> and we allow loading the cached part, which is the relevant one.
> 
> push %cs; push 1f; lret
>
> At all points when segment details are updated, it is the responsibility
> of software to ensure that the details match with the GDT/LDT entry. 
> See for example the Intel and AMD manuals for syscall/sysenter where
> similar "updating segment details behind the scenes" occurs.

I would certainly expect the user to have loaded a proper GDT and
reloaded the segments selectors before attempting to do something like
the above.

I really don't mind adding a cs_sel, ds_sel, ss_sel and tr_sel, I just
think it's quite pointless because it's common practice to load a GDT
and then reload the segment selectors to point to the right entries.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 05/08/15 17:40, Roger Pau Monné wrote:
> El 05/08/15 a les 17.39, Andrew Cooper ha escrit:
>> On 05/08/15 10:53, Roger Pau Monné wrote:
>>> El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
>>>> On 03/08/15 18:31, Roger Pau Monné wrote:
>>>>>     uint32_t cs_base;
>>>>>     uint32_t ds_base;
>>>>>     uint32_t ss_base;
>>>>>     uint32_t cs_limit;
>>>>>     uint32_t ds_limit;
>>>>>     uint32_t ss_limit;
>>>>>     uint16_t cs_ar;
>>>>>     uint16_t ds_ar;
>>>>>     uint16_t ss_ar;
>>>> You need selector entries for each segment as well.
>>> Really? What's the point in having the selector if we don't have a GDT,
>>> and we allow loading the cached part, which is the relevant one.
>> push %cs; push 1f; lret
>>
>> At all points when segment details are updated, it is the responsibility
>> of software to ensure that the details match with the GDT/LDT entry. 
>> See for example the Intel and AMD manuals for syscall/sysenter where
>> similar "updating segment details behind the scenes" occurs.
> I would certainly expect the user to have loaded a proper GDT and
> reloaded the segments selectors before attempting to do something like
> the above.
>
> I really don't mind adding a cs_sel, ds_sel, ss_sel and tr_sel, I just
> think it's quite pointless because it's common practice to load a GDT
> and then reload the segment selectors to point to the right entries.

Consider what would happen if the vcpu hits mmio and bounces into Xen
for emulation before reloading the segment selectors.

The segment selector information needs to be consistent, as it might not
be the guest which is the next entity to observe the vcpu state.

~Andrew

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 05/08/15 a les 18.46, Andrew Cooper ha escrit:
> On 05/08/15 17:40, Roger Pau Monné wrote:
>> El 05/08/15 a les 17.39, Andrew Cooper ha escrit:
>>> On 05/08/15 10:53, Roger Pau Monné wrote:
>>>> El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
>>>>> On 03/08/15 18:31, Roger Pau Monné wrote:
>>>>>>     uint32_t cs_base;
>>>>>>     uint32_t ds_base;
>>>>>>     uint32_t ss_base;
>>>>>>     uint32_t cs_limit;
>>>>>>     uint32_t ds_limit;
>>>>>>     uint32_t ss_limit;
>>>>>>     uint16_t cs_ar;
>>>>>>     uint16_t ds_ar;
>>>>>>     uint16_t ss_ar;
>>>>> You need selector entries for each segment as well.
>>>> Really? What's the point in having the selector if we don't have a GDT,
>>>> and we allow loading the cached part, which is the relevant one.
>>> push %cs; push 1f; lret
>>>
>>> At all points when segment details are updated, it is the responsibility
>>> of software to ensure that the details match with the GDT/LDT entry. 
>>> See for example the Intel and AMD manuals for syscall/sysenter where
>>> similar "updating segment details behind the scenes" occurs.
>> I would certainly expect the user to have loaded a proper GDT and
>> reloaded the segments selectors before attempting to do something like
>> the above.
>>
>> I really don't mind adding a cs_sel, ds_sel, ss_sel and tr_sel, I just
>> think it's quite pointless because it's common practice to load a GDT
>> and then reload the segment selectors to point to the right entries.
> 
> Consider what would happen if the vcpu hits mmio and bounces into Xen
> for emulation before reloading the segment selectors.

I hope Xen is just going to kill the guest.

> The segment selector information needs to be consistent, as it might not
> be the guest which is the next entity to observe the vcpu state.

I guess I'm missing something, but AFAICT we already start the BSP in
such "inconsistent" state, the cached part of the selectors is right but
the selectors themselves don't point to valid GDT entries because
there's no GDT loaded.

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 05/08/15 18:11, Roger Pau Monné wrote:
> El 05/08/15 a les 18.46, Andrew Cooper ha escrit:
>> On 05/08/15 17:40, Roger Pau Monné wrote:
>>> El 05/08/15 a les 17.39, Andrew Cooper ha escrit:
>>>> On 05/08/15 10:53, Roger Pau Monné wrote:
>>>>> El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
>>>>>> On 03/08/15 18:31, Roger Pau Monné wrote:
>>>>>>>     uint32_t cs_base;
>>>>>>>     uint32_t ds_base;
>>>>>>>     uint32_t ss_base;
>>>>>>>     uint32_t cs_limit;
>>>>>>>     uint32_t ds_limit;
>>>>>>>     uint32_t ss_limit;
>>>>>>>     uint16_t cs_ar;
>>>>>>>     uint16_t ds_ar;
>>>>>>>     uint16_t ss_ar;
>>>>>> You need selector entries for each segment as well.
>>>>> Really? What's the point in having the selector if we don't have a GDT,
>>>>> and we allow loading the cached part, which is the relevant one.
>>>> push %cs; push 1f; lret
>>>>
>>>> At all points when segment details are updated, it is the responsibility
>>>> of software to ensure that the details match with the GDT/LDT entry. 
>>>> See for example the Intel and AMD manuals for syscall/sysenter where
>>>> similar "updating segment details behind the scenes" occurs.
>>> I would certainly expect the user to have loaded a proper GDT and
>>> reloaded the segments selectors before attempting to do something like
>>> the above.
>>>
>>> I really don't mind adding a cs_sel, ds_sel, ss_sel and tr_sel, I just
>>> think it's quite pointless because it's common practice to load a GDT
>>> and then reload the segment selectors to point to the right entries.
>> Consider what would happen if the vcpu hits mmio and bounces into Xen
>> for emulation before reloading the segment selectors.
> I hope Xen is just going to kill the guest.

I would hope for it to work properly.

>
>> The segment selector information needs to be consistent, as it might not
>> be the guest which is the next entity to observe the vcpu state.
> I guess I'm missing something, but AFAICT we already start the BSP in
> such "inconsistent" state,

So we do.  (There are a number of inconsistencies in that area.)

> the cached part of the selectors is right but
> the selectors themselves don't point to valid GDT entries because
> there's no GDT loaded.

Hmm yes - you are correct.

On native, the cached part of the segment selectors only get updated
with a segment selector is loaded, at which point the gdt/ldt must be valid.

However there is no requirement or need for the gdt/ldt to stay valid
after the cached part has been loaded.  Unreal mode relies on this.

Considering that the first thing any AP startup function will do is lgdt
and reload the segment registers, we can do without maintaining selector
consistency.

Sorry for the confusion.

~Andrew

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Tim Deegan]

At 19:11 +0200 on 05 Aug (1438801877), Roger Pau Monné wrote:
> El 05/08/15 a les 18.46, Andrew Cooper ha escrit:
> > On 05/08/15 17:40, Roger Pau Monné wrote:
> >> El 05/08/15 a les 17.39, Andrew Cooper ha escrit:
> >>> On 05/08/15 10:53, Roger Pau Monné wrote:
> >>>> El 04/08/15 a les 20.08, Andrew Cooper ha escrit:
> >>>>> On 03/08/15 18:31, Roger Pau Monné wrote:
> >>>>>>     uint32_t cs_base;
> >>>>>>     uint32_t ds_base;
> >>>>>>     uint32_t ss_base;
> >>>>>>     uint32_t cs_limit;
> >>>>>>     uint32_t ds_limit;
> >>>>>>     uint32_t ss_limit;
> >>>>>>     uint16_t cs_ar;
> >>>>>>     uint16_t ds_ar;
> >>>>>>     uint16_t ss_ar;
> >>>>> You need selector entries for each segment as well.
> >>>> Really? What's the point in having the selector if we don't have a GDT,
> >>>> and we allow loading the cached part, which is the relevant one.
> >>> push %cs; push 1f; lret
> >>>
> >>> At all points when segment details are updated, it is the responsibility
> >>> of software to ensure that the details match with the GDT/LDT entry. 
> >>> See for example the Intel and AMD manuals for syscall/sysenter where
> >>> similar "updating segment details behind the scenes" occurs.
> >> I would certainly expect the user to have loaded a proper GDT and
> >> reloaded the segments selectors before attempting to do something like
> >> the above.
> >>
> >> I really don't mind adding a cs_sel, ds_sel, ss_sel and tr_sel, I just
> >> think it's quite pointless because it's common practice to load a GDT
> >> and then reload the segment selectors to point to the right entries.
> > 
> > Consider what would happen if the vcpu hits mmio and bounces into Xen
> > for emulation before reloading the segment selectors.
> 
> I hope Xen is just going to kill the guest.
> 
> > The segment selector information needs to be consistent, as it might not
> > be the guest which is the next entity to observe the vcpu state.
> 
> I guess I'm missing something, but AFAICT we already start the BSP in
> such "inconsistent" state, the cached part of the selectors is right but
> the selectors themselves don't point to valid GDT entries because
> there's no GDT loaded.

Yes, and in extending this call to HVM it should be fine to do the
same.  It would be nice to have a document describing the CPU state
that the HVM guest will find itself in after this call.  (In
particular that the segments will be set up but not the descriptor
tables, and that the selectors may not be relied upon).

Cheers,

Tim.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 04.08.15 at 20:08, <andrew.cooper3@citrix.com> wrote:
> On 03/08/15 18:31, Roger Pau Monné wrote:
>> struct vcpu_hvm_x86_16 {
>>     uint16_t ax;
>>     uint16_t cx;
>>     uint16_t dx;
>>     uint16_t bx;
>>     uint16_t sp;
>>     uint16_t bp;
>>     uint16_t si;
>>     uint16_t di;
>>     uint16_t ip;
>>
>>     uint32_t cr[8];
> 
> Definitely no need for anything other than cr0 and 4 in 16 bit mode.
> 
>>
>>     uint32_t cs_base;
>>     uint32_t ds_base;
>>     uint32_t ss_base;
>>     uint32_t cs_limit;
>>     uint32_t ds_limit;
>>     uint32_t ss_limit;
>>     uint16_t cs_ar;
>>     uint16_t ds_ar;
>>     uint16_t ss_ar;
>> };
>>
>> struct vcpu_hvm_x86_32 {
>>     uint32_t eax;
>>     uint32_t ecx;
>>     uint32_t edx;
>>     uint32_t ebx;
>>     uint32_t esp;
>>     uint32_t ebp;
>>     uint32_t esi;
>>     uint32_t edi;
>>     uint32_t eip;
>>
>>     uint32_t cr[8];
> 
> Don't need cr's 5-8.

I disagree with a number of things discussed so far (like the statement
above), but I guess I'll better comment on v4 than continue this thread.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Andrew Cooper]

On 24/07/15 10:59, Roger Pau Monné wrote:
> El 24/07/15 a les 10.22, Jan Beulich ha escrit:
>>>>> On 23.07.15 at 19:12, <andrew.cooper3@citrix.com> wrote:
>>> I would go with Jan's suggestion and make
>>> XEN_DOMCTL_{get/set}vcpucontext ineligible for use on HVM guests.
>> Plus, as you said earlier, a new VCPUOP_initialize.
> (Adding the Linux maintainers to get their opinion on the interface)
>
> Just to recap and to make sure I got all the points:
>
>   - vCPU initialization from the toolstack (BSP initialization) is going
> to be done using XEN_DOMCTL_sethvmcontext.
>   - XEN_DOMCTL_{get/set}vcpucontext is going to return EOPNOTSUPP for
> HVM guests.
>   - A new vcpu_guest_contatext (vcpu_hvm_context?) is going to be
> introduced together with a a new VCPUOP_initialize hypercall
> (VCPUOP_hvm_initialize?).
>
> The following is a layout proposal for the new vcpu_context:
>
> struct vcpu_hvm_context {
> /* 32bit fields of the structure will be used. */
> #define _VCPUHVM_MODE_32B              0
> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
> /* 64bit fields of the structure will be used. */
> #define _VCPUHVM_MODE_64B              1
> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
> #define _VCPUHVM_online                2
> #define VCPUHVM_online                 (1<<_VCPUHVM_online )
>      uint32_t flags;                         /* VCPUHVM_* flags.   */
>      struct cpu_hvm_regs user_regs;          /* CPU registers.     */
> };

To avoid making the 32bit (and optionally 16bit) massive as a side 
effect of 64bit, can I suggest

uint32_t flags;
union {
     cpu_hvm32_regs;
     cpu_hvm64_regs;
};

Which allows hvm32_regs to be a far smaller structure.

>
> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
> /* Anonymous union includes both 32- and 64-bit names (e.g., ebp/rbp). */
> # define __DECL_REG(n64, n32) union {          \
>          uint64_t n64;                          \
>          uint32_t n32;                          \
>      }
> #else
> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
> #define __DECL_REG(n64, n32) uint64_t n64
> #endif
>
> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n)
>
> struct cpu_hvm_regs {
>      /* General purpose registers. */
>      __DECL_GP_REG(bx);
>      __DECL_GP_REG(cx);
>      __DECL_GP_REG(dx);
>      __DECL_GP_REG(si);
>      __DECL_GP_REG(di);
>      __DECL_GP_REG(bp);
>      __DECL_GP_REG(ax);
>      __DECL_GP_REG(ip);
>      __DECL_GP_REG(sp);
>      __DECL_GP_REG(flags);
>
>      /* Control registers.         */
>      uint64_t cr[8];
>      /* Valid on amd64 only.       */
>      uint64_t efer;
>
>      /* Debug registers.           */
>      uint64_t db[8];
> };
>
> #undef __DECL_GP_REG
> #undef __DECL_REG
>
> Of course the APs will be allowed to start in any mode they wish,
> regardless of the mode the guest is currently running on.

To start straight in 64bit, you need gdtr and cs as a minimum

With that in mind, room for each of the task registers, and segment 
selectors.

~Andrew

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Ian Campbell]

On Fri, 2015-07-24 at 12:11 +0100, Andrew Cooper wrote:
> 
> To avoid making the 32bit (and optionally 16bit) massive as a side 
> effect of 64bit, can I suggest
> 
> uint32_t flags;
> union {
>      cpu_hvm32_regs;
>      cpu_hvm64_regs;
> };
> 
> Which allows hvm32_regs to be a far smaller structure.

Given that you can't really know apriori which mode an HVM vcpu is in
and that having to pick the right half of the union everywhere is faff
for everyone on both sides of the API I'd suggest going with a single
struct as far as possible Perhaps with a union to allow the use of 32-
and 64-bit _names_ for things which are the same underlying register in
different widths (rip/pc, eax/rax etc), but that would be it.

Ian.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 24.07.15 at 13:28, <ian.campbell@citrix.com> wrote:
> On Fri, 2015-07-24 at 12:11 +0100, Andrew Cooper wrote:
>> 
>> To avoid making the 32bit (and optionally 16bit) massive as a side 
>> effect of 64bit, can I suggest
>> 
>> uint32_t flags;
>> union {
>>      cpu_hvm32_regs;
>>      cpu_hvm64_regs;
>> };
>> 
>> Which allows hvm32_regs to be a far smaller structure.
> 
> Given that you can't really know apriori which mode an HVM vcpu is in
> and that having to pick the right half of the union everywhere is faff
> for everyone on both sides of the API I'd suggest going with a single
> struct as far as possible Perhaps with a union to allow the use of 32-
> and 64-bit _names_ for things which are the same underlying register in
> different widths (rip/pc, eax/rax etc), but that would be it.

While I had considered that too (not the least based on what you've
said ARM has), it has its own downsides (the hypervisor needing to
verify that everything that can't be represented is zero), which is
why I wasn't certain whether to suggest that.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 24.07.15 at 13:11, <andrew.cooper3@citrix.com> wrote:

> 
> On 24/07/15 10:59, Roger Pau Monné wrote:
>> El 24/07/15 a les 10.22, Jan Beulich ha escrit:
>>>>>> On 23.07.15 at 19:12, <andrew.cooper3@citrix.com> wrote:
>>>> I would go with Jan's suggestion and make
>>>> XEN_DOMCTL_{get/set}vcpucontext ineligible for use on HVM guests.
>>> Plus, as you said earlier, a new VCPUOP_initialize.
>> (Adding the Linux maintainers to get their opinion on the interface)
>>
>> Just to recap and to make sure I got all the points:
>>
>>   - vCPU initialization from the toolstack (BSP initialization) is going
>> to be done using XEN_DOMCTL_sethvmcontext.
>>   - XEN_DOMCTL_{get/set}vcpucontext is going to return EOPNOTSUPP for
>> HVM guests.
>>   - A new vcpu_guest_contatext (vcpu_hvm_context?) is going to be
>> introduced together with a a new VCPUOP_initialize hypercall
>> (VCPUOP_hvm_initialize?).
>>
>> The following is a layout proposal for the new vcpu_context:
>>
>> struct vcpu_hvm_context {
>> /* 32bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_32B              0
>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>> /* 64bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_64B              1
>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
>> #define _VCPUHVM_online                2
>> #define VCPUHVM_online                 (1<<_VCPUHVM_online )
>>      uint32_t flags;                         /* VCPUHVM_* flags.   */
>>      struct cpu_hvm_regs user_regs;          /* CPU registers.     */
>> };
> 
> To avoid making the 32bit (and optionally 16bit) massive as a side 
> effect of 64bit, can I suggest
> 
> uint32_t flags;
> union {
>      cpu_hvm32_regs;
>      cpu_hvm64_regs;
> };
> 
> Which allows hvm32_regs to be a far smaller structure.

But that wouldn't help the size of the overall structure.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 13.11, Andrew Cooper ha escrit:
> On 24/07/15 10:59, Roger Pau Monné wrote:
>> struct vcpu_hvm_context {
>> /* 32bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_32B              0
>> #define VCPUHVM_MODE_32B               (1<<_VCPUHVM_MODE_32B)
>> /* 64bit fields of the structure will be used. */
>> #define _VCPUHVM_MODE_64B              1
>> #define VCPUHVM_MODE_64B               (1<<_VCPUHVM_MODE_64B)
>> #define _VCPUHVM_online                2
>> #define VCPUHVM_online                 (1<<_VCPUHVM_online )
>>      uint32_t flags;                         /* VCPUHVM_* flags.   */
>>      struct cpu_hvm_regs user_regs;          /* CPU registers.     */
>> };
> 
> To avoid making the 32bit (and optionally 16bit) massive as a side
> effect of 64bit, can I suggest
> 
> uint32_t flags;
> union {
>     cpu_hvm32_regs;
>     cpu_hvm64_regs;
> };
> 
> Which allows hvm32_regs to be a far smaller structure.

But what's the benefit of this? vcpu_hvm_context is going to be equally
large, and that's what we pass as the hypercall argument. This is
mimicking the format of the structure that's already used on ARM, so I
would rather keep it as is.

>>
>> #if defined(__GNUC__) && !defined(__STRICT_ANSI__)
>> /* Anonymous union includes both 32- and 64-bit names (e.g., ebp/rbp). */
>> # define __DECL_REG(n64, n32) union {          \
>>          uint64_t n64;                          \
>>          uint32_t n32;                          \
>>      }
>> #else
>> /* Non-gcc sources must always use the proper 64-bit name (e.g., rbp). */
>> #define __DECL_REG(n64, n32) uint64_t n64
>> #endif
>>
>> #define __DECL_GP_REG(n) __DECL_REG(r##n, e##n)
>>
>> struct cpu_hvm_regs {
>>      /* General purpose registers. */
>>      __DECL_GP_REG(bx);
>>      __DECL_GP_REG(cx);
>>      __DECL_GP_REG(dx);
>>      __DECL_GP_REG(si);
>>      __DECL_GP_REG(di);
>>      __DECL_GP_REG(bp);
>>      __DECL_GP_REG(ax);
>>      __DECL_GP_REG(ip);
>>      __DECL_GP_REG(sp);
>>      __DECL_GP_REG(flags);
>>
>>      /* Control registers.         */
>>      uint64_t cr[8];
>>      /* Valid on amd64 only.       */
>>      uint64_t efer;
>>
>>      /* Debug registers.           */
>>      uint64_t db[8];
>> };
>>
>> #undef __DECL_GP_REG
>> #undef __DECL_REG
>>
>> Of course the APs will be allowed to start in any mode they wish,
>> regardless of the mode the guest is currently running on.
> 
> To start straight in 64bit, you need gdtr and cs as a minimum
> 
> With that in mind, room for each of the task registers, and segment
> selectors.

I was planning to say that on both 32 and 64 bits we start with a flat
GDT (like we do for the BSP). But we can also arrange for this to be set
by the user. So we would need to add the following fields:

     uint{16/32/64}_t gdtr;
     uint16_t ss, es, ds, fs, gs;
     uint16_t tr;

Roger.

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Jan Beulich]

>>> On 24.07.15 at 13:49, <roger.pau@citrix.com> wrote:
> El 24/07/15 a les 13.11, Andrew Cooper ha escrit:
>> To start straight in 64bit, you need gdtr and cs as a minimum
>> 
>> With that in mind, room for each of the task registers, and segment
>> selectors.
> 
> I was planning to say that on both 32 and 64 bits we start with a flat
> GDT (like we do for the BSP).

You'd have to explain what "flat GDT" actually is. There are flat
segments, but I'm unaware of flat descriptor tables.

Jan

Re: [PATCH v3 07/32] xen/x86: fix arch_set_info_guest for HVM guests

[Roger Pau Monné]

El 24/07/15 a les 14.41, Jan Beulich ha escrit:
>>>> On 24.07.15 at 13:49, <roger.pau@citrix.com> wrote:
>> El 24/07/15 a les 13.11, Andrew Cooper ha escrit:
>>> To start straight in 64bit, you need gdtr and cs as a minimum
>>>
>>> With that in mind, room for each of the task registers, and segment
>>> selectors.
>>
>> I was planning to say that on both 32 and 64 bits we start with a flat
>> GDT (like we do for the BSP).
> 
> You'd have to explain what "flat GDT" actually is. There are flat
> segments, but I'm unaware of flat descriptor tables.

Yes, what I meant was that all segment registers would point to a flat
segment.

Roger.

[PATCH v3 08/32] libxc: introduce a xc_dom_arch for hvm-3.0-x86_32 guests

[Roger Pau Monne]

This xc_dom_arch will be used in order to build HVM domains. The code is
based on the existing xc_hvm_populate_memory and xc_hvm_populate_params
functions.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/include/xc_dom.h |   6 +
 tools/libxc/xc_dom_x86.c     | 541 ++++++++++++++++++++++++++++++++++++++++++-
 2 files changed, 536 insertions(+), 11 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 42533a5..68b052c 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -189,6 +189,12 @@ struct xc_dom_image {
     } container_type;
 
     /* HVM specific fields. */
+    xen_pfn_t target_pages;
+    xen_pfn_t mmio_start;
+    xen_pfn_t mmio_size;
+    xen_pfn_t lowmem_end;
+    xen_pfn_t highmem_end;
+
     /* Extra ACPI tables passed to HVMLOADER */
     struct xc_hvm_firmware_module acpi_module;
 
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 993954e..6573b94 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -40,10 +40,29 @@
 
 /* ------------------------------------------------------------------------ */
 
-#define SUPERPAGE_PFN_SHIFT  9
-#define SUPERPAGE_NR_PFNS    (1UL << SUPERPAGE_PFN_SHIFT)
 #define SUPERPAGE_BATCH_SIZE 512
 
+#define SUPERPAGE_2MB_SHIFT   9
+#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT)
+#define SUPERPAGE_1GB_SHIFT   18
+#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT)
+
+#define VGA_HOLE_SIZE (0x20)
+
+#define SPECIALPAGE_PAGING   0
+#define SPECIALPAGE_ACCESS   1
+#define SPECIALPAGE_SHARING  2
+#define SPECIALPAGE_BUFIOREQ 3
+#define SPECIALPAGE_XENSTORE 4
+#define SPECIALPAGE_IOREQ    5
+#define SPECIALPAGE_IDENT_PT 6
+#define SPECIALPAGE_CONSOLE  7
+#define NR_SPECIAL_PAGES     8
+#define special_pfn(x) (0xff000u - NR_SPECIAL_PAGES + (x))
+
+#define NR_IOREQ_SERVER_PAGES 8
+#define ioreq_server_pfn(x) (special_pfn(0) - NR_IOREQ_SERVER_PAGES + (x))
+
 #define bits_to_mask(bits)       (((xen_vaddr_t)1 << (bits))-1)
 #define round_down(addr, mask)   ((addr) & ~(mask))
 #define round_up(addr, mask)     ((addr) | (mask))
@@ -462,6 +481,135 @@ static int alloc_magic_pages(struct xc_dom_image *dom)
     return 0;
 }
 
+static void build_hvm_info(void *hvm_info_page, struct xc_dom_image *dom)
+{
+    struct hvm_info_table *hvm_info = (struct hvm_info_table *)
+        (((unsigned char *)hvm_info_page) + HVM_INFO_OFFSET);
+    uint8_t sum;
+    int i;
+
+    memset(hvm_info_page, 0, PAGE_SIZE);
+
+    /* Fill in the header. */
+    strncpy(hvm_info->signature, "HVM INFO", 8);
+    hvm_info->length = sizeof(struct hvm_info_table);
+
+    /* Sensible defaults: these can be overridden by the caller. */
+    hvm_info->apic_mode = 1;
+    hvm_info->nr_vcpus = 1;
+    memset(hvm_info->vcpu_online, 0xff, sizeof(hvm_info->vcpu_online));
+
+    /* Memory parameters. */
+    hvm_info->low_mem_pgend = dom->lowmem_end >> PAGE_SHIFT;
+    hvm_info->high_mem_pgend = dom->highmem_end >> PAGE_SHIFT;
+    hvm_info->reserved_mem_pgstart = ioreq_server_pfn(0);
+
+    /* Finish with the checksum. */
+    for ( i = 0, sum = 0; i < hvm_info->length; i++ )
+        sum += ((uint8_t *)hvm_info)[i];
+    hvm_info->checksum = -sum;
+}
+
+static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
+{
+    unsigned long i;
+    void *hvm_info_page;
+    uint32_t *ident_pt, domid = dom->guest_domid;
+    int rc;
+    xen_pfn_t special_array[NR_SPECIAL_PAGES];
+    xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
+    xc_interface *xch = dom->xch;
+
+    if ( (hvm_info_page = xc_map_foreign_range(
+              xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
+              HVM_INFO_PFN)) == NULL )
+        goto error_out;
+    build_hvm_info(hvm_info_page, dom);
+    munmap(hvm_info_page, PAGE_SIZE);
+
+    /* Allocate and clear special pages. */
+    for ( i = 0; i < NR_SPECIAL_PAGES; i++ )
+        special_array[i] = special_pfn(i);
+
+    rc = xc_domain_populate_physmap_exact(xch, domid, NR_SPECIAL_PAGES, 0, 0,
+                                          special_array);
+    if ( rc != 0 )
+    {
+        DOMPRINTF("Could not allocate special pages.");
+        goto error_out;
+    }
+
+    if ( xc_clear_domain_pages(xch, domid, special_pfn(0), NR_SPECIAL_PAGES) )
+            goto error_out;
+
+    xc_hvm_param_set(xch, domid, HVM_PARAM_STORE_PFN,
+                     special_pfn(SPECIALPAGE_XENSTORE));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_BUFIOREQ_PFN,
+                     special_pfn(SPECIALPAGE_BUFIOREQ));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_IOREQ_PFN,
+                     special_pfn(SPECIALPAGE_IOREQ));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_CONSOLE_PFN,
+                     special_pfn(SPECIALPAGE_CONSOLE));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_PAGING_RING_PFN,
+                     special_pfn(SPECIALPAGE_PAGING));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_MONITOR_RING_PFN,
+                     special_pfn(SPECIALPAGE_ACCESS));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_SHARING_RING_PFN,
+                     special_pfn(SPECIALPAGE_SHARING));
+
+    /*
+     * Allocate and clear additional ioreq server pages. The default
+     * server will use the IOREQ and BUFIOREQ special pages above.
+     */
+    for ( i = 0; i < NR_IOREQ_SERVER_PAGES; i++ )
+        ioreq_server_array[i] = ioreq_server_pfn(i);
+
+    rc = xc_domain_populate_physmap_exact(xch, domid, NR_IOREQ_SERVER_PAGES, 0,
+                                          0, ioreq_server_array);
+    if ( rc != 0 )
+    {
+        DOMPRINTF("Could not allocate ioreq server pages.");
+        goto error_out;
+    }
+
+    if ( xc_clear_domain_pages(xch, domid, ioreq_server_pfn(0),
+                               NR_IOREQ_SERVER_PAGES) )
+            goto error_out;
+
+    /* Tell the domain where the pages are and how many there are */
+    xc_hvm_param_set(xch, domid, HVM_PARAM_IOREQ_SERVER_PFN,
+                     ioreq_server_pfn(0));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_NR_IOREQ_SERVER_PAGES,
+                     NR_IOREQ_SERVER_PAGES);
+
+    /*
+     * Identity-map page table is required for running with CR0.PG=0 when
+     * using Intel EPT. Create a 32-bit non-PAE page directory of superpages.
+     */
+    if ( (ident_pt = xc_map_foreign_range(
+              xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
+              special_pfn(SPECIALPAGE_IDENT_PT))) == NULL )
+        goto error_out;
+    for ( i = 0; i < PAGE_SIZE / sizeof(*ident_pt); i++ )
+        ident_pt[i] = ((i << 22) | _PAGE_PRESENT | _PAGE_RW | _PAGE_USER |
+                       _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE);
+    munmap(ident_pt, PAGE_SIZE);
+    xc_hvm_param_set(xch, domid, HVM_PARAM_IDENT_PT,
+                     special_pfn(SPECIALPAGE_IDENT_PT) << PAGE_SHIFT);
+
+    dom->console_pfn = special_pfn(SPECIALPAGE_CONSOLE);
+    dom->xenstore_pfn = special_pfn(SPECIALPAGE_XENSTORE);
+    dom->parms.virt_hypercall = -1;
+
+    rc = 0;
+    goto out;
+ error_out:
+    rc = -1;
+ out:
+
+    return rc;
+}
+
 /* ------------------------------------------------------------------------ */
 
 static int start_info_x86_32(struct xc_dom_image *dom)
@@ -669,6 +817,28 @@ static int vcpu_x86_64(struct xc_dom_image *dom, void *ptr)
     return 0;
 }
 
+static int vcpu_hvm(struct xc_dom_image *dom, void *ptr)
+{
+    vcpu_guest_context_x86_64_t *ctxt = ptr;
+
+    DOMPRINTF_CALLED(dom->xch);
+
+    /* clear everything */
+    memset(ctxt, 0, sizeof(*ctxt));
+
+    ctxt->user_regs.ds = FLAT_KERNEL_DS_X86_32;
+    ctxt->user_regs.es = FLAT_KERNEL_DS_X86_32;
+    ctxt->user_regs.fs = FLAT_KERNEL_DS_X86_32;
+    ctxt->user_regs.gs = FLAT_KERNEL_DS_X86_32;
+    ctxt->user_regs.ss = FLAT_KERNEL_SS_X86_32;
+    ctxt->user_regs.cs = FLAT_KERNEL_CS_X86_32;
+    ctxt->user_regs.rip = dom->parms.phys_entry;
+
+    ctxt->flags = VGCF_in_kernel_X86_32 | VGCF_online_X86_32;
+
+    return 0;
+}
+
 /* ------------------------------------------------------------------------ */
 
 static int x86_compat(xc_interface *xch, domid_t domid, char *guest_type)
@@ -749,7 +919,7 @@ static int meminit_pv(struct xc_dom_image *dom)
 
     if ( dom->superpages )
     {
-        int count = dom->total_pages >> SUPERPAGE_PFN_SHIFT;
+        int count = dom->total_pages >> SUPERPAGE_2MB_SHIFT;
         xen_pfn_t extents[count];
 
         dom->p2m_size = dom->total_pages;
@@ -760,9 +930,9 @@ static int meminit_pv(struct xc_dom_image *dom)
 
         DOMPRINTF("Populating memory with %d superpages", count);
         for ( pfn = 0; pfn < count; pfn++ )
-            extents[pfn] = pfn << SUPERPAGE_PFN_SHIFT;
+            extents[pfn] = pfn << SUPERPAGE_2MB_SHIFT;
         rc = xc_domain_populate_physmap_exact(dom->xch, dom->guest_domid,
-                                               count, SUPERPAGE_PFN_SHIFT, 0,
+                                               count, SUPERPAGE_2MB_SHIFT, 0,
                                                extents);
         if ( rc )
             return rc;
@@ -772,7 +942,7 @@ static int meminit_pv(struct xc_dom_image *dom)
         for ( i = 0; i < count; i++ )
         {
             mfn = extents[i];
-            for ( j = 0; j < SUPERPAGE_NR_PFNS; j++, pfn++ )
+            for ( j = 0; j < SUPERPAGE_2MB_NR_PFNS; j++, pfn++ )
                 dom->p2m_host[pfn] = mfn + j;
         }
     }
@@ -848,7 +1018,7 @@ static int meminit_pv(struct xc_dom_image *dom)
             unsigned int memflags;
             uint64_t pages;
             unsigned int pnode = vnode_to_pnode[vmemranges[i].nid];
-            int nr_spages = dom->total_pages >> SUPERPAGE_PFN_SHIFT;
+            int nr_spages = dom->total_pages >> SUPERPAGE_2MB_SHIFT;
             xen_pfn_t extents[SUPERPAGE_BATCH_SIZE];
             xen_pfn_t pfn_base_idx;
 
@@ -869,11 +1039,11 @@ static int meminit_pv(struct xc_dom_image *dom)
                 nr_spages -= count;
 
                 for ( pfn = pfn_base_idx, j = 0;
-                      pfn < pfn_base_idx + (count << SUPERPAGE_PFN_SHIFT);
-                      pfn += SUPERPAGE_NR_PFNS, j++ )
+                      pfn < pfn_base_idx + (count << SUPERPAGE_2MB_SHIFT);
+                      pfn += SUPERPAGE_2MB_NR_PFNS, j++ )
                     extents[j] = dom->p2m_host[pfn];
                 rc = xc_domain_populate_physmap(dom->xch, dom->guest_domid, count,
-                                                SUPERPAGE_PFN_SHIFT, memflags,
+                                                SUPERPAGE_2MB_SHIFT, memflags,
                                                 extents);
                 if ( rc < 0 )
                     return rc;
@@ -883,7 +1053,7 @@ static int meminit_pv(struct xc_dom_image *dom)
                 for ( j = 0; j < rc; j++ )
                 {
                     mfn = extents[j];
-                    for ( k = 0; k < SUPERPAGE_NR_PFNS; k++, pfn++ )
+                    for ( k = 0; k < SUPERPAGE_2MB_NR_PFNS; k++, pfn++ )
                         dom->p2m_host[pfn] = mfn + k;
                 }
                 pfn_base_idx = pfn;
@@ -924,6 +1094,332 @@ static int meminit_pv(struct xc_dom_image *dom)
     return rc;
 }
 
+/*
+ * Check whether there exists mmio hole in the specified memory range.
+ * Returns 1 if exists, else returns 0.
+ */
+static int check_mmio_hole(uint64_t start, uint64_t memsize,
+                           uint64_t mmio_start, uint64_t mmio_size)
+{
+    if ( start + memsize <= mmio_start || start >= mmio_start + mmio_size )
+        return 0;
+    else
+        return 1;
+}
+
+static int meminit_hvm(struct xc_dom_image *dom)
+{
+    unsigned long i, vmemid, nr_pages = dom->total_pages;
+    unsigned long p2m_size;
+    unsigned long target_pages = dom->target_pages;
+    unsigned long cur_pages, cur_pfn;
+    int rc;
+    xen_capabilities_info_t caps;
+    unsigned long stat_normal_pages = 0, stat_2mb_pages = 0, 
+        stat_1gb_pages = 0;
+    unsigned int memflags = 0;
+    int claim_enabled = dom->claim_enabled;
+    uint64_t total_pages;
+    xen_vmemrange_t dummy_vmemrange[2];
+    unsigned int dummy_vnode_to_pnode[1];
+    xen_vmemrange_t *vmemranges;
+    unsigned int *vnode_to_pnode;
+    unsigned int nr_vmemranges, nr_vnodes;
+    xc_interface *xch = dom->xch;
+    uint32_t domid = dom->guest_domid;
+
+    if ( nr_pages > target_pages )
+        memflags |= XENMEMF_populate_on_demand;
+
+    if ( dom->nr_vmemranges == 0 )
+    {
+        /* Build dummy vnode information
+         *
+         * Guest physical address space layout:
+         * [0, hole_start) [hole_start, 4G) [4G, highmem_end)
+         *
+         * Of course if there is no high memory, the second vmemrange
+         * has no effect on the actual result.
+         */
+
+        dummy_vmemrange[0].start = 0;
+        dummy_vmemrange[0].end   = dom->lowmem_end;
+        dummy_vmemrange[0].flags = 0;
+        dummy_vmemrange[0].nid   = 0;
+        nr_vmemranges = 1;
+
+        if ( dom->highmem_end > (1ULL << 32) )
+        {
+            dummy_vmemrange[1].start = 1ULL << 32;
+            dummy_vmemrange[1].end   = dom->highmem_end;
+            dummy_vmemrange[1].flags = 0;
+            dummy_vmemrange[1].nid   = 0;
+
+            nr_vmemranges++;
+        }
+
+        dummy_vnode_to_pnode[0] = XC_NUMA_NO_NODE;
+        nr_vnodes = 1;
+        vmemranges = dummy_vmemrange;
+        vnode_to_pnode = dummy_vnode_to_pnode;
+    }
+    else
+    {
+        if ( nr_pages > target_pages )
+        {
+            DOMPRINTF("Cannot enable vNUMA and PoD at the same time");
+            goto error_out;
+        }
+
+        nr_vmemranges = dom->nr_vmemranges;
+        nr_vnodes = dom->nr_vnodes;
+        vmemranges = dom->vmemranges;
+        vnode_to_pnode = dom->vnode_to_pnode;
+    }
+
+    total_pages = 0;
+    p2m_size = 0;
+    for ( i = 0; i < nr_vmemranges; i++ )
+    {
+        total_pages += ((vmemranges[i].end - vmemranges[i].start)
+                        >> PAGE_SHIFT);
+        p2m_size = p2m_size > (vmemranges[i].end >> PAGE_SHIFT) ?
+            p2m_size : (vmemranges[i].end >> PAGE_SHIFT);
+    }
+
+    if ( total_pages != nr_pages )
+    {
+        DOMPRINTF("vNUMA memory pages mismatch (0x%"PRIx64" != 0x%"PRIx64")",
+               total_pages, nr_pages);
+        goto error_out;
+    }
+
+    if ( xc_version(xch, XENVER_capabilities, &caps) != 0 )
+    {
+        DOMPRINTF("Could not get Xen capabilities");
+        goto error_out;
+    }
+
+    dom->p2m_size = p2m_size;
+    dom->p2m_host = xc_dom_malloc(dom, sizeof(xen_pfn_t) *
+                                      dom->p2m_size);
+    if ( dom->p2m_host == NULL )
+    {
+        DOMPRINTF("Could not allocate p2m");
+        goto error_out;
+    }
+
+    for ( i = 0; i < p2m_size; i++ )
+        dom->p2m_host[i] = ((xen_pfn_t)-1);
+    for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
+    {
+        uint64_t pfn;
+
+        for ( pfn = vmemranges[vmemid].start >> PAGE_SHIFT;
+              pfn < vmemranges[vmemid].end >> PAGE_SHIFT;
+              pfn++ )
+            dom->p2m_host[pfn] = pfn;
+    }
+
+    /*
+     * Try to claim pages for early warning of insufficient memory available.
+     * This should go before xc_domain_set_pod_target, becuase that function
+     * actually allocates memory for the guest. Claiming after memory has been
+     * allocated is pointless.
+     */
+    if ( claim_enabled ) {
+        rc = xc_domain_claim_pages(xch, domid, target_pages - VGA_HOLE_SIZE);
+        if ( rc != 0 )
+        {
+            DOMPRINTF("Could not allocate memory for HVM guest as we cannot claim memory!");
+            goto error_out;
+        }
+    }
+
+    if ( memflags & XENMEMF_populate_on_demand )
+    {
+        /*
+         * Subtract VGA_HOLE_SIZE from target_pages for the VGA
+         * "hole".  Xen will adjust the PoD cache size so that domain
+         * tot_pages will be target_pages - VGA_HOLE_SIZE after
+         * this call.
+         */
+        rc = xc_domain_set_pod_target(xch, domid, target_pages - VGA_HOLE_SIZE,
+                                      NULL, NULL, NULL);
+        if ( rc != 0 )
+        {
+            DOMPRINTF("Could not set PoD target for HVM guest.\n");
+            goto error_out;
+        }
+    }
+
+    /*
+     * Allocate memory for HVM guest, skipping VGA hole 0xA0000-0xC0000.
+     *
+     * We attempt to allocate 1GB pages if possible. It falls back on 2MB
+     * pages if 1GB allocation fails. 4KB pages will be used eventually if
+     * both fail.
+     * 
+     * Under 2MB mode, we allocate pages in batches of no more than 8MB to 
+     * ensure that we can be preempted and hence dom0 remains responsive.
+     */
+    rc = xc_domain_populate_physmap_exact(
+        xch, domid, 0xa0, 0, memflags, &dom->p2m_host[0x00]);
+
+    stat_normal_pages = 0;
+    for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
+    {
+        unsigned int new_memflags = memflags;
+        uint64_t end_pages;
+        unsigned int vnode = vmemranges[vmemid].nid;
+        unsigned int pnode = vnode_to_pnode[vnode];
+
+        if ( pnode != XC_NUMA_NO_NODE )
+            new_memflags |= XENMEMF_exact_node(pnode);
+
+        end_pages = vmemranges[vmemid].end >> PAGE_SHIFT;
+        /*
+         * Consider vga hole belongs to the vmemrange that covers
+         * 0xA0000-0xC0000. Note that 0x00000-0xA0000 is populated just
+         * before this loop.
+         */
+        if ( vmemranges[vmemid].start == 0 )
+        {
+            cur_pages = 0xc0;
+            stat_normal_pages += 0xc0;
+        }
+        else
+            cur_pages = vmemranges[vmemid].start >> PAGE_SHIFT;
+
+        while ( (rc == 0) && (end_pages > cur_pages) )
+        {
+            /* Clip count to maximum 1GB extent. */
+            unsigned long count = end_pages - cur_pages;
+            unsigned long max_pages = SUPERPAGE_1GB_NR_PFNS;
+
+            if ( count > max_pages )
+                count = max_pages;
+
+            cur_pfn = dom->p2m_host[cur_pages];
+
+            /* Take care the corner cases of super page tails */
+            if ( ((cur_pfn & (SUPERPAGE_1GB_NR_PFNS-1)) != 0) &&
+                 (count > (-cur_pfn & (SUPERPAGE_1GB_NR_PFNS-1))) )
+                count = -cur_pfn & (SUPERPAGE_1GB_NR_PFNS-1);
+            else if ( ((count & (SUPERPAGE_1GB_NR_PFNS-1)) != 0) &&
+                      (count > SUPERPAGE_1GB_NR_PFNS) )
+                count &= ~(SUPERPAGE_1GB_NR_PFNS - 1);
+
+            /* Attemp to allocate 1GB super page. Because in each pass
+             * we only allocate at most 1GB, we don't have to clip
+             * super page boundaries.
+             */
+            if ( ((count | cur_pfn) & (SUPERPAGE_1GB_NR_PFNS - 1)) == 0 &&
+                 /* Check if there exists MMIO hole in the 1GB memory
+                  * range */
+                 !check_mmio_hole(cur_pfn << PAGE_SHIFT,
+                                  SUPERPAGE_1GB_NR_PFNS << PAGE_SHIFT,
+                                  dom->mmio_start, dom->mmio_size) )
+            {
+                long done;
+                unsigned long nr_extents = count >> SUPERPAGE_1GB_SHIFT;
+                xen_pfn_t sp_extents[nr_extents];
+
+                for ( i = 0; i < nr_extents; i++ )
+                    sp_extents[i] =
+                        dom->p2m_host[cur_pages+(i<<SUPERPAGE_1GB_SHIFT)];
+
+                done = xc_domain_populate_physmap(xch, domid, nr_extents,
+                                                  SUPERPAGE_1GB_SHIFT,
+                                                  memflags, sp_extents);
+
+                if ( done > 0 )
+                {
+                    stat_1gb_pages += done;
+                    done <<= SUPERPAGE_1GB_SHIFT;
+                    cur_pages += done;
+                    count -= done;
+                }
+            }
+
+            if ( count != 0 )
+            {
+                /* Clip count to maximum 8MB extent. */
+                max_pages = SUPERPAGE_2MB_NR_PFNS * 4;
+                if ( count > max_pages )
+                    count = max_pages;
+
+                /* Clip partial superpage extents to superpage
+                 * boundaries. */
+                if ( ((cur_pfn & (SUPERPAGE_2MB_NR_PFNS-1)) != 0) &&
+                     (count > (-cur_pfn & (SUPERPAGE_2MB_NR_PFNS-1))) )
+                    count = -cur_pfn & (SUPERPAGE_2MB_NR_PFNS-1);
+                else if ( ((count & (SUPERPAGE_2MB_NR_PFNS-1)) != 0) &&
+                          (count > SUPERPAGE_2MB_NR_PFNS) )
+                    count &= ~(SUPERPAGE_2MB_NR_PFNS - 1); /* clip non-s.p. tail */
+
+                /* Attempt to allocate superpage extents. */
+                if ( ((count | cur_pfn) & (SUPERPAGE_2MB_NR_PFNS - 1)) == 0 )
+                {
+                    long done;
+                    unsigned long nr_extents = count >> SUPERPAGE_2MB_SHIFT;
+                    xen_pfn_t sp_extents[nr_extents];
+
+                    for ( i = 0; i < nr_extents; i++ )
+                        sp_extents[i] =
+                            dom->p2m_host[cur_pages+(i<<SUPERPAGE_2MB_SHIFT)];
+
+                    done = xc_domain_populate_physmap(xch, domid, nr_extents,
+                                                      SUPERPAGE_2MB_SHIFT,
+                                                      memflags, sp_extents);
+
+                    if ( done > 0 )
+                    {
+                        stat_2mb_pages += done;
+                        done <<= SUPERPAGE_2MB_SHIFT;
+                        cur_pages += done;
+                        count -= done;
+                    }
+                }
+            }
+
+            /* Fall back to 4kB extents. */
+            if ( count != 0 )
+            {
+                rc = xc_domain_populate_physmap_exact(
+                    xch, domid, count, 0, new_memflags, &dom->p2m_host[cur_pages]);
+                cur_pages += count;
+                stat_normal_pages += count;
+            }
+        }
+
+        if ( rc != 0 )
+            break;
+    }
+
+    if ( rc != 0 )
+    {
+        DOMPRINTF("Could not allocate memory for HVM guest.");
+        goto error_out;
+    }
+
+    DPRINTF("PHYSICAL MEMORY ALLOCATION:\n");
+    DPRINTF("  4KB PAGES: 0x%016lx\n", stat_normal_pages);
+    DPRINTF("  2MB PAGES: 0x%016lx\n", stat_2mb_pages);
+    DPRINTF("  1GB PAGES: 0x%016lx\n", stat_1gb_pages);
+
+    rc = 0;
+    goto out;
+ error_out:
+    rc = -1;
+ out:
+
+    /* ensure no unclaimed pages are left unused */
+    xc_domain_claim_pages(xch, domid, 0 /* cancels the claim */);
+
+    return rc;
+}
+
 /* ------------------------------------------------------------------------ */
 
 static int bootearly(struct xc_dom_image *dom)
@@ -1038,6 +1534,12 @@ static int bootlate_pv(struct xc_dom_image *dom)
     return 0;
 }
 
+static int bootlate_hvm(struct xc_dom_image *dom)
+{
+    DOMPRINTF("%s: doing nothing", __FUNCTION__);
+    return 0;
+}
+
 /* ------------------------------------------------------------------------ */
 
 static struct xc_dom_arch xc_dom_32_pae = {
@@ -1072,10 +1574,27 @@ static struct xc_dom_arch xc_dom_64 = {
     .bootlate = bootlate_pv,
 };
 
+static struct xc_dom_arch xc_hvm_32 = {
+    .guest_type = "hvm-3.0-x86_32",
+    .native_protocol = XEN_IO_PROTO_ABI_X86_32,
+    .page_shift = PAGE_SHIFT_X86,
+    .sizeof_pfn = 4,
+    .alloc_magic_pages = alloc_magic_pages_hvm,
+    .count_pgtables = NULL,
+    .setup_pgtables = NULL,
+    .start_info = NULL,
+    .shared_info = NULL,
+    .vcpu = vcpu_hvm,
+    .meminit = meminit_hvm,
+    .bootearly = bootearly,
+    .bootlate = bootlate_hvm,
+};
+
 static void __init register_arch_hooks(void)
 {
     xc_dom_register_arch_hooks(&xc_dom_32_pae);
     xc_dom_register_arch_hooks(&xc_dom_64);
+    xc_dom_register_arch_hooks(&xc_hvm_32);
 }
 
 int xc_dom_feature_translated(struct xc_dom_image *dom)
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 08/32] libxc: introduce a xc_dom_arch for hvm-3.0-x86_32 guests

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> This xc_dom_arch will be used in order to build HVM domains. The code is
> based on the existing xc_hvm_populate_memory and xc_hvm_populate_params
> functions.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Similar review to patch 5.  One bug - see below.

> ---
>  tools/libxc/include/xc_dom.h |   6 +
>  tools/libxc/xc_dom_x86.c     | 541 ++++++++++++++++++++++++++++++++++++++++++-
>  2 files changed, 536 insertions(+), 11 deletions(-)
>
> diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
> index 42533a5..68b052c 100644
> --- a/tools/libxc/include/xc_dom.h
> +++ b/tools/libxc/include/xc_dom.h
> @@ -189,6 +189,12 @@ struct xc_dom_image {
>      } container_type;
>  
>      /* HVM specific fields. */
> +    xen_pfn_t target_pages;
> +    xen_pfn_t mmio_start;
> +    xen_pfn_t mmio_size;
> +    xen_pfn_t lowmem_end;
> +    xen_pfn_t highmem_end;
> +
>      /* Extra ACPI tables passed to HVMLOADER */
>      struct xc_hvm_firmware_module acpi_module;
>  
> diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
> index 993954e..6573b94 100644
> --- a/tools/libxc/xc_dom_x86.c
> +++ b/tools/libxc/xc_dom_x86.c
> @@ -40,10 +40,29 @@
>  
>  /* ------------------------------------------------------------------------ */
>  
> -#define SUPERPAGE_PFN_SHIFT  9
> -#define SUPERPAGE_NR_PFNS    (1UL << SUPERPAGE_PFN_SHIFT)
>  #define SUPERPAGE_BATCH_SIZE 512
>  
> +#define SUPERPAGE_2MB_SHIFT   9
> +#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT)
> +#define SUPERPAGE_1GB_SHIFT   18
> +#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT)
> +
> +#define VGA_HOLE_SIZE (0x20)
> +
> +#define SPECIALPAGE_PAGING   0
> +#define SPECIALPAGE_ACCESS   1
> +#define SPECIALPAGE_SHARING  2
> +#define SPECIALPAGE_BUFIOREQ 3
> +#define SPECIALPAGE_XENSTORE 4
> +#define SPECIALPAGE_IOREQ    5
> +#define SPECIALPAGE_IDENT_PT 6
> +#define SPECIALPAGE_CONSOLE  7
> +#define NR_SPECIAL_PAGES     8
> +#define special_pfn(x) (0xff000u - NR_SPECIAL_PAGES + (x))
> +
> +#define NR_IOREQ_SERVER_PAGES 8
> +#define ioreq_server_pfn(x) (special_pfn(0) - NR_IOREQ_SERVER_PAGES + (x))
> +
>  #define bits_to_mask(bits)       (((xen_vaddr_t)1 << (bits))-1)
>  #define round_down(addr, mask)   ((addr) & ~(mask))
>  #define round_up(addr, mask)     ((addr) | (mask))
> @@ -462,6 +481,135 @@ static int alloc_magic_pages(struct xc_dom_image *dom)
>      return 0;
>  }
>  
> +static void build_hvm_info(void *hvm_info_page, struct xc_dom_image *dom)
> +{
> +    struct hvm_info_table *hvm_info = (struct hvm_info_table *)
> +        (((unsigned char *)hvm_info_page) + HVM_INFO_OFFSET);
> +    uint8_t sum;
> +    int i;
> +
> +    memset(hvm_info_page, 0, PAGE_SIZE);
> +
> +    /* Fill in the header. */
> +    strncpy(hvm_info->signature, "HVM INFO", 8);

Please apply c/s b9dbe33 to this code.  (Said changeset was since your
submission, but please don't reintroduce the bug).

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 09/32] libxl: switch HVM domain building to use xc_dom_* helpers

[Roger Pau Monne]

Now that we have all the code in place HVM domain building in libxl can be
switched to use the xc_dom_* family of functions, just like they are used in
order to build PV guests.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxl/libxl_dom.c      | 224 +++++++++++++++++++++++++------------------
 tools/libxl/libxl_internal.h |   2 +-
 tools/libxl/libxl_vnuma.c    |  12 ++-
 3 files changed, 139 insertions(+), 99 deletions(-)

diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 2bae277..480b7e7 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -609,6 +609,64 @@ static int set_vnuma_info(libxl__gc *gc, uint32_t domid,
     return rc;
 }
 
+static int libxl__build_dom(libxl__gc *gc, uint32_t domid,
+             libxl_domain_build_info *info, libxl__domain_build_state *state,
+             struct xc_dom_image *dom)
+{
+    libxl_ctx *ctx = libxl__gc_owner(gc);
+    uint64_t mem_kb;
+    int ret;
+
+    if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) {
+        LOGE(ERROR, "xc_dom_boot_xen_init failed");
+        goto out;
+    }
+#ifdef GUEST_RAM_BASE
+    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
+        LOGE(ERROR, "xc_dom_rambase failed");
+        goto out;
+    }
+#endif
+    if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
+        LOGE(ERROR, "xc_dom_parse_image failed");
+        goto out;
+    }
+    if ( (ret = libxl__arch_domain_init_hw_description(gc, info, state, dom)) != 0 ) {
+        LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
+        goto out;
+    }
+
+    mem_kb = dom->container_type == XC_DOM_HVM_CONTAINER ?
+             (info->max_memkb - info->video_memkb) : info->target_memkb;
+    if ( (ret = xc_dom_mem_init(dom, mem_kb / 1024)) != 0 ) {
+        LOGE(ERROR, "xc_dom_mem_init failed");
+        goto out;
+    }
+    if ( (ret = xc_dom_boot_mem_init(dom)) != 0 ) {
+        LOGE(ERROR, "xc_dom_boot_mem_init failed");
+        goto out;
+    }
+    if ( (ret = libxl__arch_domain_finalise_hw_description(gc, info, dom)) != 0 ) {
+        LOGE(ERROR, "libxl__arch_domain_finalise_hw_description failed");
+        goto out;
+    }
+    if ( (ret = xc_dom_build_image(dom)) != 0 ) {
+        LOGE(ERROR, "xc_dom_build_image failed");
+        goto out;
+    }
+    if ( (ret = xc_dom_boot_image(dom)) != 0 ) {
+        LOGE(ERROR, "xc_dom_boot_image failed");
+        goto out;
+    }
+    if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) {
+        LOGE(ERROR, "xc_dom_gnttab_init failed");
+        goto out;
+    }
+
+out:
+    return ret;
+}
+
 int libxl__build_pv(libxl__gc *gc, uint32_t domid,
              libxl_domain_build_info *info, libxl__domain_build_state *state)
 {
@@ -699,48 +757,9 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
             dom->vnode_to_pnode[i] = info->vnuma_nodes[i].pnode;
     }
 
-    if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) {
-        LOGE(ERROR, "xc_dom_boot_xen_init failed");
-        goto out;
-    }
-#ifdef GUEST_RAM_BASE
-    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
-        LOGE(ERROR, "xc_dom_rambase failed");
-        goto out;
-    }
-#endif
-    if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
-        LOGE(ERROR, "xc_dom_parse_image failed");
-        goto out;
-    }
-    if ( (ret = libxl__arch_domain_init_hw_description(gc, info, state, dom)) != 0 ) {
-        LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
-        goto out;
-    }
-    if ( (ret = xc_dom_mem_init(dom, info->target_memkb / 1024)) != 0 ) {
-        LOGE(ERROR, "xc_dom_mem_init failed");
-        goto out;
-    }
-    if ( (ret = xc_dom_boot_mem_init(dom)) != 0 ) {
-        LOGE(ERROR, "xc_dom_boot_mem_init failed");
+    ret = libxl__build_dom(gc, domid, info, state, dom);
+    if (ret != 0)
         goto out;
-    }
-    if ( (ret = libxl__arch_domain_finalise_hw_description(gc, info, dom)) != 0 ) {
-        LOGE(ERROR, "libxl__arch_domain_finalise_hw_description failed");
-        goto out;
-    }
-    if ( (ret = xc_dom_build_image(dom)) != 0 ) {
-        LOGE(ERROR, "xc_dom_build_image failed");
-        goto out;
-    }
-    if ( (ret = xc_dom_boot_image(dom)) != 0 ) {
-        LOGE(ERROR, "xc_dom_boot_image failed");
-        goto out;
-    }
-    if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) {
-        LOGE(ERROR, "xc_dom_gnttab_init failed");
-        goto out;
-    }
 
     if (xc_dom_feature_translated(dom)) {
         state->console_mfn = dom->console_pfn;
@@ -800,39 +819,39 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid,
 
 static int hvm_build_set_xs_values(libxl__gc *gc,
                                    uint32_t domid,
-                                   struct xc_hvm_build_args *args)
+                                   struct xc_dom_image *dom)
 {
     char *path = NULL;
     int ret = 0;
 
-    if (args->smbios_module.guest_addr_out) {
+    if (dom->smbios_module.guest_addr_out) {
         path = GCSPRINTF("/local/domain/%d/"HVM_XS_SMBIOS_PT_ADDRESS, domid);
 
         ret = libxl__xs_write(gc, XBT_NULL, path, "0x%"PRIx64,
-                              args->smbios_module.guest_addr_out);
+                              dom->smbios_module.guest_addr_out);
         if (ret)
             goto err;
 
         path = GCSPRINTF("/local/domain/%d/"HVM_XS_SMBIOS_PT_LENGTH, domid);
 
         ret = libxl__xs_write(gc, XBT_NULL, path, "0x%x",
-                              args->smbios_module.length);
+                              dom->smbios_module.length);
         if (ret)
             goto err;
     }
 
-    if (args->acpi_module.guest_addr_out) {
+    if (dom->acpi_module.guest_addr_out) {
         path = GCSPRINTF("/local/domain/%d/"HVM_XS_ACPI_PT_ADDRESS, domid);
 
         ret = libxl__xs_write(gc, XBT_NULL, path, "0x%"PRIx64,
-                              args->acpi_module.guest_addr_out);
+                              dom->acpi_module.guest_addr_out);
         if (ret)
             goto err;
 
         path = GCSPRINTF("/local/domain/%d/"HVM_XS_ACPI_PT_LENGTH, domid);
 
         ret = libxl__xs_write(gc, XBT_NULL, path, "0x%x",
-                              args->acpi_module.length);
+                              dom->acpi_module.length);
         if (ret)
             goto err;
     }
@@ -846,7 +865,7 @@ err:
 
 static int libxl__domain_firmware(libxl__gc *gc,
                                   libxl_domain_build_info *info,
-                                  struct xc_hvm_build_args *args)
+                                  struct xc_dom_image *dom)
 {
     libxl_ctx *ctx = libxl__gc_owner(gc);
     const char *firmware;
@@ -872,8 +891,13 @@ static int libxl__domain_firmware(libxl__gc *gc,
             break;
         }
     }
-    args->image_file_name = libxl__abs_path(gc, firmware,
-                                            libxl__xenfirmwaredir_path());
+
+    rc = xc_dom_kernel_file(dom, libxl__abs_path(gc, firmware,
+                                                 libxl__xenfirmwaredir_path()));
+    if (rc != 0) {
+        LOGE(ERROR, "xc_dom_kernel_file failed");
+        goto out;
+    }
 
     if (info->u.hvm.smbios_firmware) {
         data = NULL;
@@ -887,8 +911,8 @@ static int libxl__domain_firmware(libxl__gc *gc,
         libxl__ptr_add(gc, data);
         if (datalen) {
             /* Only accept non-empty files */
-            args->smbios_module.data = data;
-            args->smbios_module.length = (uint32_t)datalen;
+            dom->smbios_module.data = data;
+            dom->smbios_module.length = (uint32_t)datalen;
         }
     }
 
@@ -904,8 +928,8 @@ static int libxl__domain_firmware(libxl__gc *gc,
         libxl__ptr_add(gc, data);
         if (datalen) {
             /* Only accept non-empty files */
-            args->acpi_module.data = data;
-            args->acpi_module.length = (uint32_t)datalen;
+            dom->acpi_module.data = data;
+            dom->acpi_module.length = (uint32_t)datalen;
         }
     }
 
@@ -919,50 +943,60 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
               libxl__domain_build_state *state)
 {
     libxl_ctx *ctx = libxl__gc_owner(gc);
-    struct xc_hvm_build_args args = {};
-    int ret, rc = ERROR_FAIL;
-    uint64_t mmio_start, lowmem_end, highmem_end;
+    int ret;
+    uint64_t mmio_start, lowmem_end, highmem_end, mem_size;
+    struct xc_dom_image *dom = NULL;
+
+    xc_dom_loginit(ctx->xch);
+
+    dom = xc_dom_allocate(ctx->xch, NULL, NULL);
+    if (!dom) {
+        LOGE(ERROR, "xc_dom_allocate failed");
+        goto out;
+    }
+
+    dom->container_type = XC_DOM_HVM_CONTAINER;
 
-    memset(&args, 0, sizeof(struct xc_hvm_build_args));
     /* The params from the configuration file are in Mb, which are then
      * multiplied by 1 Kb. This was then divided off when calling
      * the old xc_hvm_build_target_mem() which then turned them to bytes.
      * Do all this in one step here...
      */
-    args.mem_size = (uint64_t)(info->max_memkb - info->video_memkb) << 10;
-    args.mem_target = (uint64_t)(info->target_memkb - info->video_memkb) << 10;
-    args.claim_enabled = libxl_defbool_val(info->claim_mode);
+    mem_size = (uint64_t)(info->max_memkb - info->video_memkb) << 10;
+    dom->target_pages = (uint64_t)(info->target_memkb - info->video_memkb) >> 2;
+    dom->claim_enabled = libxl_defbool_val(info->claim_mode);
     if (info->u.hvm.mmio_hole_memkb) {
         uint64_t max_ram_below_4g = (1ULL << 32) -
             (info->u.hvm.mmio_hole_memkb << 10);
 
         if (max_ram_below_4g < HVM_BELOW_4G_MMIO_START)
-            args.mmio_size = info->u.hvm.mmio_hole_memkb << 10;
+            dom->mmio_size = info->u.hvm.mmio_hole_memkb << 10;
     }
-    if (libxl__domain_firmware(gc, info, &args)) {
+    if (libxl__domain_firmware(gc, info, dom)) {
         LOG(ERROR, "initializing domain firmware failed");
         goto out;
     }
-    if (args.mem_target == 0)
-        args.mem_target = args.mem_size;
-    if (args.mmio_size == 0)
-        args.mmio_size = HVM_BELOW_4G_MMIO_LENGTH;
-    lowmem_end = args.mem_size;
+
+    if (dom->target_pages == 0)
+        dom->target_pages = mem_size >> XC_PAGE_SHIFT;
+    if (dom->mmio_size == 0)
+        dom->mmio_size = HVM_BELOW_4G_MMIO_LENGTH;
+    lowmem_end = mem_size;
     highmem_end = 0;
-    mmio_start = (1ull << 32) - args.mmio_size;
+    mmio_start = (1ull << 32) - dom->mmio_size;
     if (lowmem_end > mmio_start)
     {
         highmem_end = (1ull << 32) + (lowmem_end - mmio_start);
         lowmem_end = mmio_start;
     }
-    args.lowmem_end = lowmem_end;
-    args.highmem_end = highmem_end;
-    args.mmio_start = mmio_start;
+    dom->lowmem_end = lowmem_end;
+    dom->highmem_end = highmem_end;
+    dom->mmio_start = mmio_start;
 
     if (info->num_vnuma_nodes != 0) {
         int i;
 
-        ret = libxl__vnuma_build_vmemrange_hvm(gc, domid, info, state, &args);
+        ret = libxl__vnuma_build_vmemrange_hvm(gc, domid, info, state, dom);
         if (ret) {
             LOGEV(ERROR, ret, "hvm build vmemranges failed");
             goto out;
@@ -972,30 +1006,34 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
         ret = set_vnuma_info(gc, domid, info, state);
         if (ret) goto out;
 
-        args.nr_vmemranges = state->num_vmemranges;
-        args.vmemranges = libxl__malloc(gc, sizeof(*args.vmemranges) *
-                                        args.nr_vmemranges);
+        dom->nr_vmemranges = state->num_vmemranges;
+        dom->vmemranges = libxl__malloc(gc, sizeof(*dom->vmemranges) *
+                                        dom->nr_vmemranges);
 
-        for (i = 0; i < args.nr_vmemranges; i++) {
-            args.vmemranges[i].start = state->vmemranges[i].start;
-            args.vmemranges[i].end   = state->vmemranges[i].end;
-            args.vmemranges[i].flags = state->vmemranges[i].flags;
-            args.vmemranges[i].nid   = state->vmemranges[i].nid;
+        for (i = 0; i < dom->nr_vmemranges; i++) {
+            dom->vmemranges[i].start = state->vmemranges[i].start;
+            dom->vmemranges[i].end   = state->vmemranges[i].end;
+            dom->vmemranges[i].flags = state->vmemranges[i].flags;
+            dom->vmemranges[i].nid   = state->vmemranges[i].nid;
         }
 
-        args.nr_vnodes = info->num_vnuma_nodes;
-        args.vnode_to_pnode = libxl__malloc(gc, sizeof(*args.vnode_to_pnode) *
-                                            args.nr_vnodes);
-        for (i = 0; i < args.nr_vnodes; i++)
-            args.vnode_to_pnode[i] = info->vnuma_nodes[i].pnode;
+        dom->nr_vnodes = info->num_vnuma_nodes;
+        dom->vnode_to_pnode = libxl__malloc(gc, sizeof(*dom->vnode_to_pnode) *
+                                            dom->nr_vnodes);
+        for (i = 0; i < dom->nr_vnodes; i++)
+            dom->vnode_to_pnode[i] = info->vnuma_nodes[i].pnode;
     }
 
-    ret = xc_hvm_build(ctx->xch, domid, &args);
-    if (ret) {
-        LOGEV(ERROR, ret, "hvm building failed");
+    ret = xc_dom_boot_xen_init(dom, ctx->xch, domid);
+    if (ret != 0) {
+        LOGE(ERROR, "xc_dom_boot_xen_init failed");
         goto out;
     }
 
+    ret = libxl__build_dom(gc, domid, info, state, dom);
+    if (ret != 0)
+        goto out;
+
     ret = hvm_build_set_params(ctx->xch, domid, info, state->store_port,
                                &state->store_mfn, state->console_port,
                                &state->console_mfn, state->store_domid,
@@ -1005,15 +1043,15 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
         goto out;
     }
 
-    ret = hvm_build_set_xs_values(gc, domid, &args);
+    ret = hvm_build_set_xs_values(gc, domid, dom);
     if (ret) {
         LOG(ERROR, "hvm build set xenstore values failed (ret=%d)", ret);
         goto out;
     }
 
-    return 0;
 out:
-    return rc;
+    if (dom != NULL) xc_dom_release(dom);
+    return ret == 0 ? 0 : ERROR_FAIL;
 }
 
 int libxl__qemu_traditional_cmd(libxl__gc *gc, uint32_t domid,
diff --git a/tools/libxl/libxl_internal.h b/tools/libxl/libxl_internal.h
index d52589e..f4314e6 100644
--- a/tools/libxl/libxl_internal.h
+++ b/tools/libxl/libxl_internal.h
@@ -3540,7 +3540,7 @@ int libxl__vnuma_build_vmemrange_hvm(libxl__gc *gc,
                                      uint32_t domid,
                                      libxl_domain_build_info *b_info,
                                      libxl__domain_build_state *state,
-                                     struct xc_hvm_build_args *args);
+                                     struct xc_dom_image *dom);
 bool libxl__vnuma_configured(const libxl_domain_build_info *b_info);
 
 _hidden int libxl__ms_vm_genid_set(libxl__gc *gc, uint32_t domid,
diff --git a/tools/libxl/libxl_vnuma.c b/tools/libxl/libxl_vnuma.c
index 56856d2..db22799 100644
--- a/tools/libxl/libxl_vnuma.c
+++ b/tools/libxl/libxl_vnuma.c
@@ -17,6 +17,8 @@
 #include "libxl_arch.h"
 #include <stdlib.h>
 
+#include <xc_dom.h>
+
 bool libxl__vnuma_configured(const libxl_domain_build_info *b_info)
 {
     return b_info->num_vnuma_nodes != 0;
@@ -252,7 +254,7 @@ int libxl__vnuma_build_vmemrange_hvm(libxl__gc *gc,
                                      uint32_t domid,
                                      libxl_domain_build_info *b_info,
                                      libxl__domain_build_state *state,
-                                     struct xc_hvm_build_args *args)
+                                     struct xc_dom_image *dom)
 {
     uint64_t hole_start, hole_end, next;
     int nid, nr_vmemrange;
@@ -264,10 +266,10 @@ int libxl__vnuma_build_vmemrange_hvm(libxl__gc *gc,
      * Guest physical address space layout:
      * [0, hole_start) [hole_start, hole_end) [hole_end, highmem_end)
      */
-    hole_start = args->lowmem_end < args->mmio_start ?
-        args->lowmem_end : args->mmio_start;
-    hole_end = (args->mmio_start + args->mmio_size) > (1ULL << 32) ?
-        (args->mmio_start + args->mmio_size) : (1ULL << 32);
+    hole_start = dom->lowmem_end < dom->mmio_start ?
+        dom->lowmem_end : dom->mmio_start;
+    hole_end = (dom->mmio_start + dom->mmio_size) > (1ULL << 32) ?
+        (dom->mmio_start + dom->mmio_size) : (1ULL << 32);
 
     assert(state->vmemranges == NULL);
 
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 09/32] libxl: switch HVM domain building to use xc_dom_* helpers

[Wei Liu]

On Fri, Jul 03, 2015 at 01:34:47PM +0200, Roger Pau Monne wrote:
> Now that we have all the code in place HVM domain building in libxl can be
> switched to use the xc_dom_* family of functions, just like they are used in
> order to build PV guests.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

Mostly looks good. Some nits below.

> ---
>  tools/libxl/libxl_dom.c      | 224 +++++++++++++++++++++++++------------------
>  tools/libxl/libxl_internal.h |   2 +-
>  tools/libxl/libxl_vnuma.c    |  12 ++-
>  3 files changed, 139 insertions(+), 99 deletions(-)
> 
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 2bae277..480b7e7 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -609,6 +609,64 @@ static int set_vnuma_info(libxl__gc *gc, uint32_t domid,
>      return rc;
>  }
>  
> +static int libxl__build_dom(libxl__gc *gc, uint32_t domid,
> +             libxl_domain_build_info *info, libxl__domain_build_state *state,
> +             struct xc_dom_image *dom)
> +{
> +    libxl_ctx *ctx = libxl__gc_owner(gc);

No need to have this line.

> +    uint64_t mem_kb;
> +    int ret;
> +
> +    if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_boot_xen_init failed");
> +        goto out;
> +    }
> +#ifdef GUEST_RAM_BASE
> +    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_rambase failed");
> +        goto out;
> +    }
> +#endif
> +    if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_parse_image failed");
> +        goto out;
> +    }
> +    if ( (ret = libxl__arch_domain_init_hw_description(gc, info, state, dom)) != 0 ) {
> +        LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
> +        goto out;
> +    }
> +
> +    mem_kb = dom->container_type == XC_DOM_HVM_CONTAINER ?
> +             (info->max_memkb - info->video_memkb) : info->target_memkb;
> +    if ( (ret = xc_dom_mem_init(dom, mem_kb / 1024)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_mem_init failed");
> +        goto out;
> +    }
> +    if ( (ret = xc_dom_boot_mem_init(dom)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_boot_mem_init failed");
> +        goto out;
> +    }
> +    if ( (ret = libxl__arch_domain_finalise_hw_description(gc, info, dom)) != 0 ) {
> +        LOGE(ERROR, "libxl__arch_domain_finalise_hw_description failed");
> +        goto out;
> +    }
> +    if ( (ret = xc_dom_build_image(dom)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_build_image failed");
> +        goto out;
> +    }
> +    if ( (ret = xc_dom_boot_image(dom)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_boot_image failed");
> +        goto out;
> +    }
> +    if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) {
> +        LOGE(ERROR, "xc_dom_gnttab_init failed");
> +        goto out;
> +    }
> +
> +out:
> +    return ret;

Please translate libxc error code to libxl error code.

> +}
> +
>  int libxl__build_pv(libxl__gc *gc, uint32_t domid,
>               libxl_domain_build_info *info, libxl__domain_build_state *state)
>  {
> @@ -699,48 +757,9 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
>              dom->vnode_to_pnode[i] = info->vnuma_nodes[i].pnode;
>      }
>  
[...]
>      }
>  
> @@ -919,50 +943,60 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
>                libxl__domain_build_state *state)
>  {
>      libxl_ctx *ctx = libxl__gc_owner(gc);
> -    struct xc_hvm_build_args args = {};
> -    int ret, rc = ERROR_FAIL;
> -    uint64_t mmio_start, lowmem_end, highmem_end;
> +    int ret;

This should be rc.

Wei.

Re: [PATCH v3 09/32] libxl: switch HVM domain building to use xc_dom_* helpers

[Roger Pau Monné]

El 28/07/15 a les 13.22, Wei Liu ha escrit:
> On Fri, Jul 03, 2015 at 01:34:47PM +0200, Roger Pau Monne wrote:
>> Now that we have all the code in place HVM domain building in libxl can be
>> switched to use the xc_dom_* family of functions, just like they are used in
>> order to build PV guests.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
> 
> Mostly looks good. Some nits below.

Thanks for the review.

>> ---
>>  tools/libxl/libxl_dom.c      | 224 +++++++++++++++++++++++++------------------
>>  tools/libxl/libxl_internal.h |   2 +-
>>  tools/libxl/libxl_vnuma.c    |  12 ++-
>>  3 files changed, 139 insertions(+), 99 deletions(-)
>>
>> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
>> index 2bae277..480b7e7 100644
>> --- a/tools/libxl/libxl_dom.c
>> +++ b/tools/libxl/libxl_dom.c
>> @@ -609,6 +609,64 @@ static int set_vnuma_info(libxl__gc *gc, uint32_t domid,
>>      return rc;
>>  }
>>  
>> +static int libxl__build_dom(libxl__gc *gc, uint32_t domid,
>> +             libxl_domain_build_info *info, libxl__domain_build_state *state,
>> +             struct xc_dom_image *dom)
>> +{
>> +    libxl_ctx *ctx = libxl__gc_owner(gc);
> 
> No need to have this line.

But ctx is needed...

>> +    uint64_t mem_kb;
>> +    int ret;
>> +
>> +    if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) {
                                              ^ here.
>> +        LOGE(ERROR, "xc_dom_boot_xen_init failed");
>> +        goto out;
>> +    }
>> +#ifdef GUEST_RAM_BASE
>> +    if ( (ret = xc_dom_rambase_init(dom, GUEST_RAM_BASE)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_rambase failed");
>> +        goto out;
>> +    }
>> +#endif
>> +    if ( (ret = xc_dom_parse_image(dom)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_parse_image failed");
>> +        goto out;
>> +    }
>> +    if ( (ret = libxl__arch_domain_init_hw_description(gc, info, state, dom)) != 0 ) {
>> +        LOGE(ERROR, "libxl__arch_domain_init_hw_description failed");
>> +        goto out;
>> +    }
>> +
>> +    mem_kb = dom->container_type == XC_DOM_HVM_CONTAINER ?
>> +             (info->max_memkb - info->video_memkb) : info->target_memkb;
>> +    if ( (ret = xc_dom_mem_init(dom, mem_kb / 1024)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_mem_init failed");
>> +        goto out;
>> +    }
>> +    if ( (ret = xc_dom_boot_mem_init(dom)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_boot_mem_init failed");
>> +        goto out;
>> +    }
>> +    if ( (ret = libxl__arch_domain_finalise_hw_description(gc, info, dom)) != 0 ) {
>> +        LOGE(ERROR, "libxl__arch_domain_finalise_hw_description failed");
>> +        goto out;
>> +    }
>> +    if ( (ret = xc_dom_build_image(dom)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_build_image failed");
>> +        goto out;
>> +    }
>> +    if ( (ret = xc_dom_boot_image(dom)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_boot_image failed");
>> +        goto out;
>> +    }
>> +    if ( (ret = xc_dom_gnttab_init(dom)) != 0 ) {
>> +        LOGE(ERROR, "xc_dom_gnttab_init failed");
>> +        goto out;
>> +    }
>> +
>> +out:
>> +    return ret;
> 
> Please translate libxc error code to libxl error code.

Done, I have however just done:

return ret != 0 ? ERROR_FAIL : 0;

Not sure if we need/want a more complex error code translation here.

>> +}
>> +
>>  int libxl__build_pv(libxl__gc *gc, uint32_t domid,
>>               libxl_domain_build_info *info, libxl__domain_build_state *state)
>>  {
>> @@ -699,48 +757,9 @@ int libxl__build_pv(libxl__gc *gc, uint32_t domid,
>>              dom->vnode_to_pnode[i] = info->vnuma_nodes[i].pnode;
>>      }
>>  
> [...]
>>      }
>>  
>> @@ -919,50 +943,60 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
>>                libxl__domain_build_state *state)
>>  {
>>      libxl_ctx *ctx = libxl__gc_owner(gc);
>> -    struct xc_hvm_build_args args = {};
>> -    int ret, rc = ERROR_FAIL;
>> -    uint64_t mmio_start, lowmem_end, highmem_end;
>> +    int ret;
> 
> This should be rc.

Right, we don't call any libxc functions directly any longer, so only rc
should be used.

Roger.

Re: [PATCH v3 09/32] libxl: switch HVM domain building to use xc_dom_* helpers

[Wei Liu]

On Tue, Jul 28, 2015 at 04:26:15PM +0200, Roger Pau Monné wrote:
> El 28/07/15 a les 13.22, Wei Liu ha escrit:
> > On Fri, Jul 03, 2015 at 01:34:47PM +0200, Roger Pau Monne wrote:
> >> Now that we have all the code in place HVM domain building in libxl can be
> >> switched to use the xc_dom_* family of functions, just like they are used in
> >> order to build PV guests.
> >>
> >> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> >> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> >> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> >> Cc: Ian Campbell <ian.campbell@citrix.com>
> >> Cc: Wei Liu <wei.liu2@citrix.com>
> > 
> > Mostly looks good. Some nits below.
> 
> Thanks for the review.
> 
> >> ---
> >>  tools/libxl/libxl_dom.c      | 224 +++++++++++++++++++++++++------------------
> >>  tools/libxl/libxl_internal.h |   2 +-
> >>  tools/libxl/libxl_vnuma.c    |  12 ++-
> >>  3 files changed, 139 insertions(+), 99 deletions(-)
> >>
> >> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> >> index 2bae277..480b7e7 100644
> >> --- a/tools/libxl/libxl_dom.c
> >> +++ b/tools/libxl/libxl_dom.c
> >> @@ -609,6 +609,64 @@ static int set_vnuma_info(libxl__gc *gc, uint32_t domid,
> >>      return rc;
> >>  }
> >>  
> >> +static int libxl__build_dom(libxl__gc *gc, uint32_t domid,
> >> +             libxl_domain_build_info *info, libxl__domain_build_state *state,
> >> +             struct xc_dom_image *dom)
> >> +{
> >> +    libxl_ctx *ctx = libxl__gc_owner(gc);
> > 
> > No need to have this line.
> 
> But ctx is needed...
> 
> >> +    uint64_t mem_kb;
> >> +    int ret;
> >> +
> >> +    if ( (ret = xc_dom_boot_xen_init(dom, ctx->xch, domid)) != 0 ) {
>                                               ^ here.

Use CTX macro here. Sorry I missed this.

Wei.

[PATCH v3 10/32] libxc: remove dead HVM building code

[Roger Pau Monne]

Remove xc_hvm_build_x86.c and xc_hvm_build_arm.c since xc_hvm_build is not
longer used in order to create HVM guests.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/Makefile              |   2 -
 tools/libxc/include/xenguest.h    |  44 ---
 tools/libxc/xc_hvm_build_arm.c    |  49 ---
 tools/libxc/xc_hvm_build_x86.c    | 805 --------------------------------------
 tools/libxc/xg_private.c          |   9 -
 tools/python/xen/lowlevel/xc/xc.c |  81 ----
 6 files changed, 990 deletions(-)
 delete mode 100644 tools/libxc/xc_hvm_build_arm.c
 delete mode 100644 tools/libxc/xc_hvm_build_x86.c

diff --git a/tools/libxc/Makefile b/tools/libxc/Makefile
index 48fc473..a143c0f 100644
--- a/tools/libxc/Makefile
+++ b/tools/libxc/Makefile
@@ -94,9 +94,7 @@ GUEST_SRCS-y                 += xc_dom_compat_linux.c
 
 GUEST_SRCS-$(CONFIG_X86)     += xc_dom_x86.c
 GUEST_SRCS-$(CONFIG_X86)     += xc_cpuid_x86.c
-GUEST_SRCS-$(CONFIG_X86)     += xc_hvm_build_x86.c
 GUEST_SRCS-$(CONFIG_ARM)     += xc_dom_arm.c
-GUEST_SRCS-$(CONFIG_ARM)     += xc_hvm_build_arm.c
 
 ifeq ($(CONFIG_LIBXC_MINIOS),y)
 GUEST_SRCS-y                 += xc_dom_decompress_unsafe.c
diff --git a/tools/libxc/include/xenguest.h b/tools/libxc/include/xenguest.h
index 7581263..d96bf7d 100644
--- a/tools/libxc/include/xenguest.h
+++ b/tools/libxc/include/xenguest.h
@@ -233,50 +233,6 @@ struct xc_hvm_firmware_module {
     uint64_t  guest_addr_out;
 };
 
-struct xc_hvm_build_args {
-    uint64_t mem_size;           /* Memory size in bytes. */
-    uint64_t mem_target;         /* Memory target in bytes. */
-    uint64_t mmio_size;          /* Size of the MMIO hole in bytes. */
-    const char *image_file_name; /* File name of the image to load. */
-
-    /* Extra ACPI tables passed to HVMLOADER */
-    struct xc_hvm_firmware_module acpi_module;
-
-    /* Extra SMBIOS structures passed to HVMLOADER */
-    struct xc_hvm_firmware_module smbios_module;
-    /* Whether to use claim hypercall (1 - enable, 0 - disable). */
-    int claim_enabled;
-
-    /* vNUMA information*/
-    xen_vmemrange_t *vmemranges;
-    unsigned int nr_vmemranges;
-    unsigned int *vnode_to_pnode;
-    unsigned int nr_vnodes;
-
-    /* Out parameters  */
-    uint64_t lowmem_end;
-    uint64_t highmem_end;
-    uint64_t mmio_start;
-};
-
-/**
- * Build a HVM domain.
- * @parm xch      libxc context handle.
- * @parm domid    domain ID for the new domain.
- * @parm hvm_args parameters for the new domain.
- *
- * The memory size and image file parameters are required, the rest
- * are optional.
- */
-int xc_hvm_build(xc_interface *xch, uint32_t domid,
-                 struct xc_hvm_build_args *hvm_args);
-
-int xc_hvm_build_target_mem(xc_interface *xch,
-                            uint32_t domid,
-                            int memsize,
-                            int target,
-                            const char *image_name);
-
 /*
  * Sets *lockfd to -1.
  * Has deallocated everything even on error.
diff --git a/tools/libxc/xc_hvm_build_arm.c b/tools/libxc/xc_hvm_build_arm.c
deleted file mode 100644
index ff66689..0000000
--- a/tools/libxc/xc_hvm_build_arm.c
+++ /dev/null
@@ -1,49 +0,0 @@
-/******************************************************************************
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation;
- * version 2.1 of the License.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- *
- * You should have received a copy of the GNU Lesser General Public
- * License along with this library; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
- *
- * Copyright (c) 2011, Citrix Systems
- */
-
-#include <inttypes.h>
-#include <errno.h>
-#include <xenctrl.h>
-#include <xenguest.h>
-
-int xc_hvm_build(xc_interface *xch, uint32_t domid,
-                 struct xc_hvm_build_args *hvm_args)
-{
-    errno = ENOSYS;
-    return -1;
-}
-
-int xc_hvm_build_target_mem(xc_interface *xch,
-                           uint32_t domid,
-                           int memsize,
-                           int target,
-                           const char *image_name)
-{
-    errno = ENOSYS;
-    return -1;
-}
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * tab-width: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/tools/libxc/xc_hvm_build_x86.c b/tools/libxc/xc_hvm_build_x86.c
deleted file mode 100644
index f7616a8..0000000
--- a/tools/libxc/xc_hvm_build_x86.c
+++ /dev/null
@@ -1,805 +0,0 @@
-/******************************************************************************
- * xc_hvm_build.c
- *
- * This library is free software; you can redistribute it and/or
- * modify it under the terms of the GNU Lesser General Public
- * License as published by the Free Software Foundation;
- * version 2.1 of the License.
- *
- * This library is distributed in the hope that it will be useful,
- * but WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
- * Lesser General Public License for more details.
- *
- * You should have received a copy of the GNU Lesser General Public
- * License along with this library; if not, write to the Free Software
- * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA
- */
-
-#include <stddef.h>
-#include <inttypes.h>
-#include <stdlib.h>
-#include <unistd.h>
-#include <zlib.h>
-
-#include "xg_private.h"
-#include "xc_private.h"
-
-#include <xen/foreign/x86_32.h>
-#include <xen/foreign/x86_64.h>
-#include <xen/hvm/hvm_info_table.h>
-#include <xen/hvm/params.h>
-#include <xen/hvm/e820.h>
-
-#include <xen/libelf/libelf.h>
-
-#define SUPERPAGE_2MB_SHIFT   9
-#define SUPERPAGE_2MB_NR_PFNS (1UL << SUPERPAGE_2MB_SHIFT)
-#define SUPERPAGE_1GB_SHIFT   18
-#define SUPERPAGE_1GB_NR_PFNS (1UL << SUPERPAGE_1GB_SHIFT)
-
-#define SPECIALPAGE_PAGING   0
-#define SPECIALPAGE_ACCESS   1
-#define SPECIALPAGE_SHARING  2
-#define SPECIALPAGE_BUFIOREQ 3
-#define SPECIALPAGE_XENSTORE 4
-#define SPECIALPAGE_IOREQ    5
-#define SPECIALPAGE_IDENT_PT 6
-#define SPECIALPAGE_CONSOLE  7
-#define NR_SPECIAL_PAGES     8
-#define special_pfn(x) (0xff000u - NR_SPECIAL_PAGES + (x))
-
-#define NR_IOREQ_SERVER_PAGES 8
-#define ioreq_server_pfn(x) (special_pfn(0) - NR_IOREQ_SERVER_PAGES + (x))
-
-#define VGA_HOLE_SIZE (0x20)
-
-static int modules_init(struct xc_hvm_build_args *args,
-                        uint64_t vend, struct elf_binary *elf,
-                        uint64_t *mstart_out, uint64_t *mend_out)
-{
-#define MODULE_ALIGN 1UL << 7
-#define MB_ALIGN     1UL << 20
-#define MKALIGN(x, a) (((uint64_t)(x) + (a) - 1) & ~(uint64_t)((a) - 1))
-    uint64_t total_len = 0, offset1 = 0;
-
-    if ( (args->acpi_module.length == 0)&&(args->smbios_module.length == 0) )
-        return 0;
-
-    /* Find the total length for the firmware modules with a reasonable large
-     * alignment size to align each the modules.
-     */
-    total_len = MKALIGN(args->acpi_module.length, MODULE_ALIGN);
-    offset1 = total_len;
-    total_len += MKALIGN(args->smbios_module.length, MODULE_ALIGN);
-
-    /* Want to place the modules 1Mb+change behind the loader image. */
-    *mstart_out = MKALIGN(elf->pend, MB_ALIGN) + (MB_ALIGN);
-    *mend_out = *mstart_out + total_len;
-
-    if ( *mend_out > vend )    
-        return -1;
-
-    if ( args->acpi_module.length != 0 )
-        args->acpi_module.guest_addr_out = *mstart_out;
-    if ( args->smbios_module.length != 0 )
-        args->smbios_module.guest_addr_out = *mstart_out + offset1;
-
-    return 0;
-}
-
-static void build_hvm_info(void *hvm_info_page,
-                           struct xc_hvm_build_args *args)
-{
-    struct hvm_info_table *hvm_info = (struct hvm_info_table *)
-        (((unsigned char *)hvm_info_page) + HVM_INFO_OFFSET);
-    uint8_t sum;
-    int i;
-
-    memset(hvm_info_page, 0, PAGE_SIZE);
-
-    /* Fill in the header. */
-    strncpy(hvm_info->signature, "HVM INFO", 8);
-    hvm_info->length = sizeof(struct hvm_info_table);
-
-    /* Sensible defaults: these can be overridden by the caller. */
-    hvm_info->apic_mode = 1;
-    hvm_info->nr_vcpus = 1;
-    memset(hvm_info->vcpu_online, 0xff, sizeof(hvm_info->vcpu_online));
-
-    /* Memory parameters. */
-    hvm_info->low_mem_pgend = args->lowmem_end >> PAGE_SHIFT;
-    hvm_info->high_mem_pgend = args->highmem_end >> PAGE_SHIFT;
-    hvm_info->reserved_mem_pgstart = ioreq_server_pfn(0);
-
-    /* Finish with the checksum. */
-    for ( i = 0, sum = 0; i < hvm_info->length; i++ )
-        sum += ((uint8_t *)hvm_info)[i];
-    hvm_info->checksum = -sum;
-}
-
-static int loadelfimage(xc_interface *xch, struct elf_binary *elf,
-                        uint32_t dom, unsigned long *parray)
-{
-    privcmd_mmap_entry_t *entries = NULL;
-    unsigned long pfn_start = elf->pstart >> PAGE_SHIFT;
-    unsigned long pfn_end = (elf->pend + PAGE_SIZE - 1) >> PAGE_SHIFT;
-    size_t pages = pfn_end - pfn_start;
-    int i, rc = -1;
-
-    /* Map address space for initial elf image. */
-    entries = calloc(pages, sizeof(privcmd_mmap_entry_t));
-    if ( entries == NULL )
-        goto err;
-
-    for ( i = 0; i < pages; i++ )
-        entries[i].mfn = parray[(elf->pstart >> PAGE_SHIFT) + i];
-
-    elf->dest_base = xc_map_foreign_ranges(
-        xch, dom, pages << PAGE_SHIFT, PROT_READ | PROT_WRITE, 1 << PAGE_SHIFT,
-        entries, pages);
-    if ( elf->dest_base == NULL )
-        goto err;
-    elf->dest_size = pages * PAGE_SIZE;
-
-    ELF_ADVANCE_DEST(elf, elf->pstart & (PAGE_SIZE - 1));
-
-    /* Load the initial elf image. */
-    rc = elf_load_binary(elf);
-    if ( rc < 0 )
-        PERROR("Failed to load elf binary\n");
-
-    munmap(elf->dest_base, pages << PAGE_SHIFT);
-    elf->dest_base = NULL;
-    elf->dest_size = 0;
-
- err:
-    free(entries);
-
-    return rc;
-}
-
-static int loadmodules(xc_interface *xch,
-                       struct xc_hvm_build_args *args,
-                       uint64_t mstart, uint64_t mend,
-                       uint32_t dom, unsigned long *parray)
-{
-    privcmd_mmap_entry_t *entries = NULL;
-    unsigned long pfn_start;
-    unsigned long pfn_end;
-    size_t pages;
-    uint32_t i;
-    uint8_t *dest;
-    int rc = -1;
-
-    if ( (mstart == 0)||(mend == 0) )
-        return 0;
-
-    pfn_start = (unsigned long)(mstart >> PAGE_SHIFT);
-    pfn_end = (unsigned long)((mend + PAGE_SIZE - 1) >> PAGE_SHIFT);
-    pages = pfn_end - pfn_start;
-
-    /* Map address space for module list. */
-    entries = calloc(pages, sizeof(privcmd_mmap_entry_t));
-    if ( entries == NULL )
-        goto error_out;
-
-    for ( i = 0; i < pages; i++ )
-        entries[i].mfn = parray[(mstart >> PAGE_SHIFT) + i];
-
-    dest = xc_map_foreign_ranges(
-        xch, dom, pages << PAGE_SHIFT, PROT_READ | PROT_WRITE, 1 << PAGE_SHIFT,
-        entries, pages);
-    if ( dest == NULL )
-        goto error_out;
-
-    /* Zero the range so padding is clear between modules */
-    memset(dest, 0, pages << PAGE_SHIFT);
-
-    /* Load modules into range */    
-    if ( args->acpi_module.length != 0 )
-    {
-        memcpy(dest,
-               args->acpi_module.data,
-               args->acpi_module.length);
-    }
-    if ( args->smbios_module.length != 0 )
-    {
-        memcpy(dest + (args->smbios_module.guest_addr_out - mstart),
-               args->smbios_module.data,
-               args->smbios_module.length);
-    }
-
-    munmap(dest, pages << PAGE_SHIFT);
-    rc = 0;
-
- error_out:
-    free(entries);
-
-    return rc;
-}
-
-/*
- * Check whether there exists mmio hole in the specified memory range.
- * Returns 1 if exists, else returns 0.
- */
-static int check_mmio_hole(uint64_t start, uint64_t memsize,
-                           uint64_t mmio_start, uint64_t mmio_size)
-{
-    if ( start + memsize <= mmio_start || start >= mmio_start + mmio_size )
-        return 0;
-    else
-        return 1;
-}
-
-static int xc_hvm_populate_memory(xc_interface *xch, uint32_t dom,
-                                  struct xc_hvm_build_args *args,
-                                  xen_pfn_t *page_array)
-{
-    unsigned long i, vmemid, nr_pages = args->mem_size >> PAGE_SHIFT;
-    unsigned long p2m_size;
-    unsigned long target_pages = args->mem_target >> PAGE_SHIFT;
-    unsigned long cur_pages, cur_pfn;
-    int rc;
-    xen_capabilities_info_t caps;
-    unsigned long stat_normal_pages = 0, stat_2mb_pages = 0, 
-        stat_1gb_pages = 0;
-    unsigned int memflags = 0;
-    int claim_enabled = args->claim_enabled;
-    uint64_t total_pages;
-    xen_vmemrange_t dummy_vmemrange[2];
-    unsigned int dummy_vnode_to_pnode[1];
-    xen_vmemrange_t *vmemranges;
-    unsigned int *vnode_to_pnode;
-    unsigned int nr_vmemranges, nr_vnodes;
-
-    if ( nr_pages > target_pages )
-        memflags |= XENMEMF_populate_on_demand;
-
-    if ( args->nr_vmemranges == 0 )
-    {
-        /* Build dummy vnode information
-         *
-         * Guest physical address space layout:
-         * [0, hole_start) [hole_start, 4G) [4G, highmem_end)
-         *
-         * Of course if there is no high memory, the second vmemrange
-         * has no effect on the actual result.
-         */
-
-        dummy_vmemrange[0].start = 0;
-        dummy_vmemrange[0].end   = args->lowmem_end;
-        dummy_vmemrange[0].flags = 0;
-        dummy_vmemrange[0].nid   = 0;
-        nr_vmemranges = 1;
-
-        if ( args->highmem_end > (1ULL << 32) )
-        {
-            dummy_vmemrange[1].start = 1ULL << 32;
-            dummy_vmemrange[1].end   = args->highmem_end;
-            dummy_vmemrange[1].flags = 0;
-            dummy_vmemrange[1].nid   = 0;
-
-            nr_vmemranges++;
-        }
-
-        dummy_vnode_to_pnode[0] = XC_NUMA_NO_NODE;
-        nr_vnodes = 1;
-        vmemranges = dummy_vmemrange;
-        vnode_to_pnode = dummy_vnode_to_pnode;
-    }
-    else
-    {
-        if ( nr_pages > target_pages )
-        {
-            PERROR("Cannot enable vNUMA and PoD at the same time");
-            goto error_out;
-        }
-
-        nr_vmemranges = args->nr_vmemranges;
-        nr_vnodes = args->nr_vnodes;
-        vmemranges = args->vmemranges;
-        vnode_to_pnode = args->vnode_to_pnode;
-    }
-
-    total_pages = 0;
-    p2m_size = 0;
-    for ( i = 0; i < nr_vmemranges; i++ )
-    {
-        total_pages += ((vmemranges[i].end - vmemranges[i].start)
-                        >> PAGE_SHIFT);
-        p2m_size = p2m_size > (vmemranges[i].end >> PAGE_SHIFT) ?
-            p2m_size : (vmemranges[i].end >> PAGE_SHIFT);
-    }
-
-    if ( total_pages != (args->mem_size >> PAGE_SHIFT) )
-    {
-        PERROR("vNUMA memory pages mismatch (0x%"PRIx64" != 0x%"PRIx64")",
-               total_pages, args->mem_size >> PAGE_SHIFT);
-        goto error_out;
-    }
-
-    if ( xc_version(xch, XENVER_capabilities, &caps) != 0 )
-    {
-        PERROR("Could not get Xen capabilities");
-        goto error_out;
-    }
-
-    for ( i = 0; i < p2m_size; i++ )
-        page_array[i] = ((xen_pfn_t)-1);
-    for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
-    {
-        uint64_t pfn;
-
-        for ( pfn = vmemranges[vmemid].start >> PAGE_SHIFT;
-              pfn < vmemranges[vmemid].end >> PAGE_SHIFT;
-              pfn++ )
-            page_array[pfn] = pfn;
-    }
-
-    /*
-     * Try to claim pages for early warning of insufficient memory available.
-     * This should go before xc_domain_set_pod_target, becuase that function
-     * actually allocates memory for the guest. Claiming after memory has been
-     * allocated is pointless.
-     */
-    if ( claim_enabled ) {
-        rc = xc_domain_claim_pages(xch, dom, target_pages - VGA_HOLE_SIZE);
-        if ( rc != 0 )
-        {
-            PERROR("Could not allocate memory for HVM guest as we cannot claim memory!");
-            goto error_out;
-        }
-    }
-
-    if ( memflags & XENMEMF_populate_on_demand )
-    {
-        /*
-         * Subtract VGA_HOLE_SIZE from target_pages for the VGA
-         * "hole".  Xen will adjust the PoD cache size so that domain
-         * tot_pages will be target_pages - VGA_HOLE_SIZE after
-         * this call.
-         */
-        rc = xc_domain_set_pod_target(xch, dom,
-                                      target_pages - VGA_HOLE_SIZE,
-                                      NULL, NULL, NULL);
-        if ( rc != 0 )
-        {
-            PERROR("Could not set PoD target for HVM guest.\n");
-            goto error_out;
-        }
-    }
-
-    /*
-     * Allocate memory for HVM guest, skipping VGA hole 0xA0000-0xC0000.
-     *
-     * We attempt to allocate 1GB pages if possible. It falls back on 2MB
-     * pages if 1GB allocation fails. 4KB pages will be used eventually if
-     * both fail.
-     * 
-     * Under 2MB mode, we allocate pages in batches of no more than 8MB to 
-     * ensure that we can be preempted and hence dom0 remains responsive.
-     */
-    rc = xc_domain_populate_physmap_exact(
-        xch, dom, 0xa0, 0, memflags, &page_array[0x00]);
-
-    stat_normal_pages = 0;
-    for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
-    {
-        unsigned int new_memflags = memflags;
-        uint64_t end_pages;
-        unsigned int vnode = vmemranges[vmemid].nid;
-        unsigned int pnode = vnode_to_pnode[vnode];
-
-        if ( pnode != XC_NUMA_NO_NODE )
-            new_memflags |= XENMEMF_exact_node(pnode);
-
-        end_pages = vmemranges[vmemid].end >> PAGE_SHIFT;
-        /*
-         * Consider vga hole belongs to the vmemrange that covers
-         * 0xA0000-0xC0000. Note that 0x00000-0xA0000 is populated just
-         * before this loop.
-         */
-        if ( vmemranges[vmemid].start == 0 )
-        {
-            cur_pages = 0xc0;
-            stat_normal_pages += 0xc0;
-        }
-        else
-            cur_pages = vmemranges[vmemid].start >> PAGE_SHIFT;
-
-        while ( (rc == 0) && (end_pages > cur_pages) )
-        {
-            /* Clip count to maximum 1GB extent. */
-            unsigned long count = end_pages - cur_pages;
-            unsigned long max_pages = SUPERPAGE_1GB_NR_PFNS;
-
-            if ( count > max_pages )
-                count = max_pages;
-
-            cur_pfn = page_array[cur_pages];
-
-            /* Take care the corner cases of super page tails */
-            if ( ((cur_pfn & (SUPERPAGE_1GB_NR_PFNS-1)) != 0) &&
-                 (count > (-cur_pfn & (SUPERPAGE_1GB_NR_PFNS-1))) )
-                count = -cur_pfn & (SUPERPAGE_1GB_NR_PFNS-1);
-            else if ( ((count & (SUPERPAGE_1GB_NR_PFNS-1)) != 0) &&
-                      (count > SUPERPAGE_1GB_NR_PFNS) )
-                count &= ~(SUPERPAGE_1GB_NR_PFNS - 1);
-
-            /* Attemp to allocate 1GB super page. Because in each pass
-             * we only allocate at most 1GB, we don't have to clip
-             * super page boundaries.
-             */
-            if ( ((count | cur_pfn) & (SUPERPAGE_1GB_NR_PFNS - 1)) == 0 &&
-                 /* Check if there exists MMIO hole in the 1GB memory
-                  * range */
-                 !check_mmio_hole(cur_pfn << PAGE_SHIFT,
-                                  SUPERPAGE_1GB_NR_PFNS << PAGE_SHIFT,
-                                  args->mmio_start, args->mmio_size) )
-            {
-                long done;
-                unsigned long nr_extents = count >> SUPERPAGE_1GB_SHIFT;
-                xen_pfn_t sp_extents[nr_extents];
-
-                for ( i = 0; i < nr_extents; i++ )
-                    sp_extents[i] =
-                        page_array[cur_pages+(i<<SUPERPAGE_1GB_SHIFT)];
-
-                done = xc_domain_populate_physmap(xch, dom, nr_extents,
-                                                  SUPERPAGE_1GB_SHIFT,
-                                                  memflags, sp_extents);
-
-                if ( done > 0 )
-                {
-                    stat_1gb_pages += done;
-                    done <<= SUPERPAGE_1GB_SHIFT;
-                    cur_pages += done;
-                    count -= done;
-                }
-            }
-
-            if ( count != 0 )
-            {
-                /* Clip count to maximum 8MB extent. */
-                max_pages = SUPERPAGE_2MB_NR_PFNS * 4;
-                if ( count > max_pages )
-                    count = max_pages;
-
-                /* Clip partial superpage extents to superpage
-                 * boundaries. */
-                if ( ((cur_pfn & (SUPERPAGE_2MB_NR_PFNS-1)) != 0) &&
-                     (count > (-cur_pfn & (SUPERPAGE_2MB_NR_PFNS-1))) )
-                    count = -cur_pfn & (SUPERPAGE_2MB_NR_PFNS-1);
-                else if ( ((count & (SUPERPAGE_2MB_NR_PFNS-1)) != 0) &&
-                          (count > SUPERPAGE_2MB_NR_PFNS) )
-                    count &= ~(SUPERPAGE_2MB_NR_PFNS - 1); /* clip non-s.p. tail */
-
-                /* Attempt to allocate superpage extents. */
-                if ( ((count | cur_pfn) & (SUPERPAGE_2MB_NR_PFNS - 1)) == 0 )
-                {
-                    long done;
-                    unsigned long nr_extents = count >> SUPERPAGE_2MB_SHIFT;
-                    xen_pfn_t sp_extents[nr_extents];
-
-                    for ( i = 0; i < nr_extents; i++ )
-                        sp_extents[i] =
-                            page_array[cur_pages+(i<<SUPERPAGE_2MB_SHIFT)];
-
-                    done = xc_domain_populate_physmap(xch, dom, nr_extents,
-                                                      SUPERPAGE_2MB_SHIFT,
-                                                      memflags, sp_extents);
-
-                    if ( done > 0 )
-                    {
-                        stat_2mb_pages += done;
-                        done <<= SUPERPAGE_2MB_SHIFT;
-                        cur_pages += done;
-                        count -= done;
-                    }
-                }
-            }
-
-            /* Fall back to 4kB extents. */
-            if ( count != 0 )
-            {
-                rc = xc_domain_populate_physmap_exact(
-                    xch, dom, count, 0, new_memflags, &page_array[cur_pages]);
-                cur_pages += count;
-                stat_normal_pages += count;
-            }
-        }
-
-        if ( rc != 0 )
-            break;
-    }
-
-    if ( rc != 0 )
-    {
-        PERROR("Could not allocate memory for HVM guest.");
-        goto error_out;
-    }
-
-    DPRINTF("PHYSICAL MEMORY ALLOCATION:\n");
-    DPRINTF("  4KB PAGES: 0x%016lx\n", stat_normal_pages);
-    DPRINTF("  2MB PAGES: 0x%016lx\n", stat_2mb_pages);
-    DPRINTF("  1GB PAGES: 0x%016lx\n", stat_1gb_pages);
-
-    rc = 0;
-    goto out;
- error_out:
-    rc = -1;
- out:
-
-    /* ensure no unclaimed pages are left unused */
-    xc_domain_claim_pages(xch, dom, 0 /* cancels the claim */);
-
-    return rc;
-}
-
-static int xc_hvm_load_image(xc_interface *xch,
-                       uint32_t dom, struct xc_hvm_build_args *args,
-                       xen_pfn_t *page_array)
-{
-    unsigned long entry_eip, image_size;
-    struct elf_binary elf;
-    uint64_t v_start, v_end;
-    uint64_t m_start = 0, m_end = 0;
-    char *image;
-    int rc;
-
-    image = xc_read_image(xch, args->image_file_name, &image_size);
-    if ( image == NULL )
-        return -1;
-
-    memset(&elf, 0, sizeof(elf));
-    if ( elf_init(&elf, image, image_size) != 0 )
-        goto error_out;
-
-    xc_elf_set_logfile(xch, &elf, 1);
-
-    elf_parse_binary(&elf);
-    v_start = 0;
-    v_end = args->mem_size;
-
-    if ( modules_init(args, v_end, &elf, &m_start, &m_end) != 0 )
-    {
-        ERROR("Insufficient space to load modules.");
-        goto error_out;
-    }
-
-    DPRINTF("VIRTUAL MEMORY ARRANGEMENT:\n");
-    DPRINTF("  Loader:   %016"PRIx64"->%016"PRIx64"\n", elf.pstart, elf.pend);
-    DPRINTF("  Modules:  %016"PRIx64"->%016"PRIx64"\n", m_start, m_end);
-
-    if ( loadelfimage(xch, &elf, dom, page_array) != 0 )
-    {
-        PERROR("Could not load ELF image");
-        goto error_out;
-    }
-
-    if ( loadmodules(xch, args, m_start, m_end, dom, page_array) != 0 )
-    {
-        PERROR("Could not load ACPI modules");
-        goto error_out;
-    }
-
-    /* Insert JMP <rel32> instruction at address 0x0 to reach entry point. */
-    entry_eip = elf_uval(&elf, elf.ehdr, e_entry);
-    if ( entry_eip != 0 )
-    {
-        char *page0 = xc_map_foreign_range(
-            xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE, 0);
-        if ( page0 == NULL )
-            goto error_out;
-        page0[0] = 0xe9;
-        *(uint32_t *)&page0[1] = entry_eip - 5;
-        munmap(page0, PAGE_SIZE);
-    }
-
-    rc = 0;
-    goto out;
- error_out:
-    rc = -1;
- out:
-    if ( elf_check_broken(&elf) )
-        ERROR("HVM ELF broken: %s", elf_check_broken(&elf));
-    free(image);
-
-    return rc;
-}
-
-static int xc_hvm_populate_params(xc_interface *xch, uint32_t dom,
-                                  struct xc_hvm_build_args *args)
-{
-    unsigned long i;
-    void *hvm_info_page;
-    uint32_t *ident_pt;
-    uint64_t v_end;
-    int rc;
-    xen_pfn_t special_array[NR_SPECIAL_PAGES];
-    xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
-
-    v_end = args->mem_size;
-
-    if ( (hvm_info_page = xc_map_foreign_range(
-              xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE,
-              HVM_INFO_PFN)) == NULL )
-    {
-        PERROR("Could not map hvm info page");
-        goto error_out;
-    }
-    build_hvm_info(hvm_info_page, args);
-    munmap(hvm_info_page, PAGE_SIZE);
-
-    /* Allocate and clear special pages. */
-    for ( i = 0; i < NR_SPECIAL_PAGES; i++ )
-        special_array[i] = special_pfn(i);
-
-    rc = xc_domain_populate_physmap_exact(xch, dom, NR_SPECIAL_PAGES, 0, 0,
-                                          special_array);
-    if ( rc != 0 )
-    {
-        PERROR("Could not allocate special pages.");
-        goto error_out;
-    }
-
-    if ( xc_clear_domain_pages(xch, dom, special_pfn(0), NR_SPECIAL_PAGES) )
-    {
-        PERROR("Could not clear special pages");
-        goto error_out;
-    }
-
-    xc_hvm_param_set(xch, dom, HVM_PARAM_STORE_PFN,
-                     special_pfn(SPECIALPAGE_XENSTORE));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_BUFIOREQ_PFN,
-                     special_pfn(SPECIALPAGE_BUFIOREQ));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_IOREQ_PFN,
-                     special_pfn(SPECIALPAGE_IOREQ));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_CONSOLE_PFN,
-                     special_pfn(SPECIALPAGE_CONSOLE));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_PAGING_RING_PFN,
-                     special_pfn(SPECIALPAGE_PAGING));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_MONITOR_RING_PFN,
-                     special_pfn(SPECIALPAGE_ACCESS));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_SHARING_RING_PFN,
-                     special_pfn(SPECIALPAGE_SHARING));
-
-    /*
-     * Allocate and clear additional ioreq server pages. The default
-     * server will use the IOREQ and BUFIOREQ special pages above.
-     */
-    for ( i = 0; i < NR_IOREQ_SERVER_PAGES; i++ )
-        ioreq_server_array[i] = ioreq_server_pfn(i);
-
-    rc = xc_domain_populate_physmap_exact(xch, dom, NR_IOREQ_SERVER_PAGES, 0, 0,
-                                          ioreq_server_array);
-    if ( rc != 0 )
-    {
-        PERROR("Could not allocate ioreq server pages.");
-        goto error_out;
-    }
-
-    if ( xc_clear_domain_pages(xch, dom, ioreq_server_pfn(0), NR_IOREQ_SERVER_PAGES) )
-    {
-        PERROR("Could not clear ioreq page");
-        goto error_out;
-    }
-
-    /* Tell the domain where the pages are and how many there are */
-    xc_hvm_param_set(xch, dom, HVM_PARAM_IOREQ_SERVER_PFN,
-                     ioreq_server_pfn(0));
-    xc_hvm_param_set(xch, dom, HVM_PARAM_NR_IOREQ_SERVER_PAGES,
-                     NR_IOREQ_SERVER_PAGES);
-
-    /*
-     * Identity-map page table is required for running with CR0.PG=0 when
-     * using Intel EPT. Create a 32-bit non-PAE page directory of superpages.
-     */
-    if ( (ident_pt = xc_map_foreign_range(
-              xch, dom, PAGE_SIZE, PROT_READ | PROT_WRITE,
-              special_pfn(SPECIALPAGE_IDENT_PT))) == NULL )
-    {
-        PERROR("Could not map special page ident_pt");
-        goto error_out;
-    }
-    for ( i = 0; i < PAGE_SIZE / sizeof(*ident_pt); i++ )
-        ident_pt[i] = ((i << 22) | _PAGE_PRESENT | _PAGE_RW | _PAGE_USER |
-                       _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE);
-    munmap(ident_pt, PAGE_SIZE);
-    xc_hvm_param_set(xch, dom, HVM_PARAM_IDENT_PT,
-                     special_pfn(SPECIALPAGE_IDENT_PT) << PAGE_SHIFT);
-
-    rc = 0;
-    goto out;
- error_out:
-    rc = -1;
- out:
-
-    return rc;
-}
-
-/* xc_hvm_build:
- * Create a domain for a virtualized Linux, using files/filenames.
- */
-int xc_hvm_build(xc_interface *xch, uint32_t domid,
-                 struct xc_hvm_build_args *hvm_args)
-{
-    struct xc_hvm_build_args args = *hvm_args;
-    xen_pfn_t *parray = NULL;
-    int rc;
-
-    if ( domid == 0 )
-        return -1;
-    if ( args.image_file_name == NULL )
-        return -1;
-
-    /* An HVM guest must be initialised with at least 2MB memory. */
-    if ( args.mem_size < (2ull << 20) || args.mem_target < (2ull << 20) )
-        return -1;
-
-    parray = malloc((args.mem_size >> PAGE_SHIFT) * sizeof(xen_pfn_t));
-    if ( parray == NULL )
-        return -1;
-
-    rc = xc_hvm_populate_memory(xch, domid, &args, parray);
-    if ( rc != 0 )
-    {
-        PERROR("xc_hvm_populate_memory failed");
-        goto out;
-    }
-    rc = xc_hvm_load_image(xch, domid, &args, parray);
-    if ( rc != 0 )
-    {
-        PERROR("xc_hvm_load_image failed");
-        goto out;
-    }
-    rc = xc_hvm_populate_params(xch, domid, &args);
-    if ( rc != 0 )
-    {
-        PERROR("xc_hvm_populate_params failed");
-        goto out;
-    }
-
-    /* Return module load addresses to caller */
-    hvm_args->acpi_module.guest_addr_out = args.acpi_module.guest_addr_out;
-    hvm_args->smbios_module.guest_addr_out = args.smbios_module.guest_addr_out;
-
-out:
-    free(parray);
-
-    return rc;
-}
-
-/* xc_hvm_build_target_mem: 
- * Create a domain for a pre-ballooned virtualized Linux, using
- * files/filenames.  If target < memsize, domain is created with
- * memsize pages marked populate-on-demand, 
- * calculating pod cache size based on target.
- * If target == memsize, pages are populated normally.
- */
-int xc_hvm_build_target_mem(xc_interface *xch,
-                           uint32_t domid,
-                           int memsize,
-                           int target,
-                           const char *image_name)
-{
-    struct xc_hvm_build_args args = {};
-
-    memset(&args, 0, sizeof(struct xc_hvm_build_args));
-    args.mem_size = (uint64_t)memsize << 20;
-    args.mem_target = (uint64_t)target << 20;
-    args.image_file_name = image_name;
-
-    return xc_hvm_build(xch, domid, &args);
-}
-
-/*
- * Local variables:
- * mode: C
- * c-file-style: "BSD"
- * c-basic-offset: 4
- * tab-width: 4
- * indent-tabs-mode: nil
- * End:
- */
diff --git a/tools/libxc/xg_private.c b/tools/libxc/xg_private.c
index c52cb44..98f33b2 100644
--- a/tools/libxc/xg_private.c
+++ b/tools/libxc/xg_private.c
@@ -188,15 +188,6 @@ unsigned long csum_page(void *page)
     return sum ^ (sum>>32);
 }
 
-__attribute__((weak)) 
-    int xc_hvm_build(xc_interface *xch,
-                     uint32_t domid,
-                     struct xc_hvm_build_args *hvm_args)
-{
-    errno = ENOSYS;
-    return -1;
-}
-
 /*
  * Local variables:
  * mode: C
diff --git a/tools/python/xen/lowlevel/xc/xc.c b/tools/python/xen/lowlevel/xc/xc.c
index c77e15b..690780c 100644
--- a/tools/python/xen/lowlevel/xc/xc.c
+++ b/tools/python/xen/lowlevel/xc/xc.c
@@ -910,77 +910,6 @@ static PyObject *pyxc_dom_suppress_spurious_page_faults(XcObject *self,
 }
 #endif /* __i386__ || __x86_64__ */
 
-static PyObject *pyxc_hvm_build(XcObject *self,
-                                PyObject *args,
-                                PyObject *kwds)
-{
-    uint32_t dom;
-    struct hvm_info_table *va_hvm;
-    uint8_t *va_map, sum;
-    int i;
-    char *image;
-    int memsize, target=-1, vcpus = 1, acpi = 0, apic = 1;
-    PyObject *vcpu_avail_handle = NULL;
-    uint8_t vcpu_avail[(HVM_MAX_VCPUS + 7)/8];
-
-    static char *kwd_list[] = { "domid",
-                                "memsize", "image", "target", "vcpus", 
-                                "vcpu_avail", "acpi", "apic", NULL };
-    if ( !PyArg_ParseTupleAndKeywords(args, kwds, "iis|iiOii", kwd_list,
-                                      &dom, &memsize, &image, &target, &vcpus,
-                                      &vcpu_avail_handle, &acpi, &apic) )
-        return NULL;
-
-    memset(vcpu_avail, 0, sizeof(vcpu_avail));
-    vcpu_avail[0] = 1;
-    if ( vcpu_avail_handle != NULL )
-    {
-        if ( PyInt_Check(vcpu_avail_handle) )
-        {
-            unsigned long v = PyInt_AsLong(vcpu_avail_handle);
-            for ( i = 0; i < sizeof(long); i++ )
-                vcpu_avail[i] = (uint8_t)(v>>(i*8));
-        }
-        else if ( PyLong_Check(vcpu_avail_handle) )
-        {
-            if ( _PyLong_AsByteArray((PyLongObject *)vcpu_avail_handle,
-                                     (unsigned char *)vcpu_avail,
-                                     sizeof(vcpu_avail), 1, 0) )
-                return NULL;
-        }
-        else
-        {
-            errno = EINVAL;
-            PyErr_SetFromErrno(xc_error_obj);
-            return NULL;
-        }
-    }
-
-    if ( target == -1 )
-        target = memsize;
-
-    if ( xc_hvm_build_target_mem(self->xc_handle, dom, memsize,
-                                 target, image) != 0 )
-        return pyxc_error_to_exception(self->xc_handle);
-
-    /* Fix up the HVM info table. */
-    va_map = xc_map_foreign_range(self->xc_handle, dom, XC_PAGE_SIZE,
-                                  PROT_READ | PROT_WRITE,
-                                  HVM_INFO_PFN);
-    if ( va_map == NULL )
-        return PyErr_SetFromErrno(xc_error_obj);
-    va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
-    va_hvm->apic_mode    = apic;
-    va_hvm->nr_vcpus     = vcpus;
-    memcpy(va_hvm->vcpu_online, vcpu_avail, sizeof(vcpu_avail));
-    for ( i = 0, sum = 0; i < va_hvm->length; i++ )
-        sum += ((uint8_t *)va_hvm)[i];
-    va_hvm->checksum -= sum;
-    munmap(va_map, XC_PAGE_SIZE);
-
-    return Py_BuildValue("{}");
-}
-
 static PyObject *pyxc_gnttab_hvm_seed(XcObject *self,
 				      PyObject *args,
 				      PyObject *kwds)
@@ -2411,16 +2340,6 @@ static PyMethodDef pyxc_methods[] = {
       " image   [str]:      Name of kernel image file. May be gzipped.\n"
       " cmdline [str, n/a]: Kernel parameters, if any.\n\n"},
 
-    { "hvm_build", 
-      (PyCFunction)pyxc_hvm_build, 
-      METH_VARARGS | METH_KEYWORDS, "\n"
-      "Build a new HVM guest OS.\n"
-      " dom     [int]:      Identifier of domain to build into.\n"
-      " image   [str]:      Name of HVM loader image file.\n"
-      " vcpus   [int, 1]:   Number of Virtual CPUS in domain.\n\n"
-      " vcpu_avail [long, 1]: Which Virtual CPUS available.\n\n"
-      "Returns: [int] 0 on success; -1 on error.\n" },
-
     { "gnttab_hvm_seed",
       (PyCFunction)pyxc_gnttab_hvm_seed,
       METH_KEYWORDS, "\n"
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 10/32] libxc: remove dead HVM building code

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> Remove xc_hvm_build_x86.c and xc_hvm_build_arm.c since xc_hvm_build is not
> longer used in order to create HVM guests.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>

In principle, very much +1

However, I suspect this wants functional testing from the ARM folks.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 10/32] libxc: remove dead HVM building code

[Roger Pau Monné]

El 06/07/15 a les 15.46, Andrew Cooper ha escrit:
> On 03/07/15 12:34, Roger Pau Monne wrote:
>> Remove xc_hvm_build_x86.c and xc_hvm_build_arm.c since xc_hvm_build is not
>> longer used in order to create HVM guests.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
> 
> In principle, very much +1
> 
> However, I suspect this wants functional testing from the ARM folks.

Yes, although xc_hvm_build_arm.c just contained dummy stubs to fill the
interface.

Roger.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 11/32] xen/x86: add bitmap of enabled emulated devices

[Roger Pau Monne]

Introduce a bitmap in x86 xen_arch_domainconfig that allows enabling or
disabling specific devices emulated inside of Xen for HVM guests.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxl/libxl_x86.c           |  6 ++++--
 xen/arch/x86/domain.c             | 15 +++++++++++++++
 xen/include/asm-x86/domain.h      | 13 +++++++++++++
 xen/include/public/arch-x86/xen.h | 20 +++++++++++++++++++-
 4 files changed, 51 insertions(+), 3 deletions(-)

diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
index ed2bd38..f04433a 100644
--- a/tools/libxl/libxl_x86.c
+++ b/tools/libxl/libxl_x86.c
@@ -5,8 +5,10 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
                                       libxl_domain_config *d_config,
                                       xc_domain_configuration_t *xc_config)
 {
-    /* No specific configuration right now */
-
+    if (d_config->c_info.type == LIBXL_DOMAIN_TYPE_HVM)
+        xc_config->emulation_flags = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER |
+                                      EMU_RTC | EMU_IOAPIC | EMU_PIC |
+                                      EMU_PMU | EMU_VGA | EMU_IOMMU);
     return 0;
 }
 
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index a112953..d684f49 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -528,6 +528,7 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
 {
     int i, paging_initialised = 0;
     int rc = -ENOMEM;
+    uint32_t emulation_mask;
 
     d->arch.s3_integrity = !!(domcr_flags & DOMCRF_s3_integrity);
 
@@ -550,6 +551,20 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
                d->domain_id);
     }
 
+    if ( is_hvm_domain(d) )
+    {
+        emulation_mask = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER | EMU_RTC |
+                          EMU_IOAPIC | EMU_PIC | EMU_PMU | EMU_VGA | EMU_IOMMU);
+        if ( (config->emulation_flags & emulation_mask) != emulation_mask )
+        {
+            printk(XENLOG_G_ERR "Xen does not allow HVM creation with the "
+                   "current selection of emulators: 0x%x.\n",
+                   config->emulation_flags);
+            return -EPERM;
+        }
+        d->arch.emulation_flags = config->emulation_flags;
+    }
+
     if ( has_hvm_container_domain(d) )
     {
         d->arch.hvm_domain.hap_enabled =
diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
index 96bde65..502379e 100644
--- a/xen/include/asm-x86/domain.h
+++ b/xen/include/asm-x86/domain.h
@@ -356,8 +356,21 @@ struct arch_domain
 
     /* Mem_access emulation control */
     bool_t mem_access_emulate_enabled;
+
+    /* Emulated devices enabled bitmap. */
+    uint32_t emulation_flags;
 } __cacheline_aligned;
 
+#define has_vlapic(d)       (d->arch.emulation_flags & EMU_LAPIC)
+#define has_vhpet(d)        (d->arch.emulation_flags & EMU_HPET)
+#define has_vpmtimer(d)     (d->arch.emulation_flags & EMU_PMTIMER)
+#define has_vrtc(d)         (d->arch.emulation_flags & EMU_RTC)
+#define has_vioapic(d)      (d->arch.emulation_flags & EMU_IOAPIC)
+#define has_vpic(d)         (d->arch.emulation_flags & EMU_PIC)
+#define has_vpmu(d)         (d->arch.emulation_flags & EMU_PMU)
+#define has_vvga(d)         (d->arch.emulation_flags & EMU_VGA)
+#define has_viommu(d)       (d->arch.emulation_flags & EMU_IOMMU)
+
 #define has_arch_pdevs(d)    (!list_empty(&(d)->arch.pdev_list))
 
 #define gdt_ldt_pt_idx(v) \
diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
index 2ecc9c9..6b387a3 100644
--- a/xen/include/public/arch-x86/xen.h
+++ b/xen/include/public/arch-x86/xen.h
@@ -268,7 +268,25 @@ typedef struct arch_shared_info arch_shared_info_t;
  * XEN_DOMCTL_INTERFACE_VERSION.
  */
 struct xen_arch_domainconfig {
-    char dummy;
+#define _EMU_LAPIC                  0
+#define EMU_LAPIC                   (1U<<_EMU_LAPIC)
+#define _EMU_HPET                   1
+#define EMU_HPET                    (1U<<_EMU_HPET)
+#define _EMU_PMTIMER                2
+#define EMU_PMTIMER                 (1U<<_EMU_PMTIMER)
+#define _EMU_RTC                    3
+#define EMU_RTC                     (1U<<_EMU_RTC)
+#define _EMU_IOAPIC                 4
+#define EMU_IOAPIC                  (1U<<_EMU_IOAPIC)
+#define _EMU_PIC                    5
+#define EMU_PIC                     (1U<<_EMU_PIC)
+#define _EMU_PMU                    6
+#define EMU_PMU                     (1U<<_EMU_PMU)
+#define _EMU_VGA                    7
+#define EMU_VGA                     (1U<<_EMU_VGA)
+#define _EMU_IOMMU                  8
+#define EMU_IOMMU                   (1U<<_EMU_IOMMU)
+    uint32_t emulation_flags;
 };
 #endif
 
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 11/32] xen/x86: add bitmap of enabled emulated devices

[Andrew Cooper]

On 03/07/15 12:34, Roger Pau Monne wrote:
> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
> index a112953..d684f49 100644
> --- a/xen/arch/x86/domain.c
> +++ b/xen/arch/x86/domain.c
> @@ -528,6 +528,7 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
>  {
>      int i, paging_initialised = 0;
>      int rc = -ENOMEM;
> +    uint32_t emulation_mask;
>  
>      d->arch.s3_integrity = !!(domcr_flags & DOMCRF_s3_integrity);
>  
> @@ -550,6 +551,20 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
>                 d->domain_id);
>      }
>  
> +    if ( is_hvm_domain(d) )
> +    {
> +        emulation_mask = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER | EMU_RTC |
> +                          EMU_IOAPIC | EMU_PIC | EMU_PMU | EMU_VGA | EMU_IOMMU);
> +        if ( (config->emulation_flags & emulation_mask) != emulation_mask )
> +        {
> +            printk(XENLOG_G_ERR "Xen does not allow HVM creation with the "
> +                   "current selection of emulators: 0x%x.\n",
> +                   config->emulation_flags);

Please put "d%d" in the error message to identify which domain has the
bad configuration.

> +            return -EPERM;

EPERM feels wrong here.  It is not a lack of permissions causing the
issue.  EINVAL/EOPNOTSUPP perhaps?

> +        }
> +        d->arch.emulation_flags = config->emulation_flags;
> +    }
> +
>      if ( has_hvm_container_domain(d) )
>      {
>          d->arch.hvm_domain.hap_enabled =
> diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
> index 96bde65..502379e 100644
> --- a/xen/include/asm-x86/domain.h
> +++ b/xen/include/asm-x86/domain.h
> @@ -356,8 +356,21 @@ struct arch_domain
>  
>      /* Mem_access emulation control */
>      bool_t mem_access_emulate_enabled;
> +
> +    /* Emulated devices enabled bitmap. */
> +    uint32_t emulation_flags;
>  } __cacheline_aligned;
>  
> +#define has_vlapic(d)       (d->arch.emulation_flags & EMU_LAPIC)
> +#define has_vhpet(d)        (d->arch.emulation_flags & EMU_HPET)
> +#define has_vpmtimer(d)     (d->arch.emulation_flags & EMU_PMTIMER)
> +#define has_vrtc(d)         (d->arch.emulation_flags & EMU_RTC)
> +#define has_vioapic(d)      (d->arch.emulation_flags & EMU_IOAPIC)
> +#define has_vpic(d)         (d->arch.emulation_flags & EMU_PIC)
> +#define has_vpmu(d)         (d->arch.emulation_flags & EMU_PMU)
> +#define has_vvga(d)         (d->arch.emulation_flags & EMU_VGA)
> +#define has_viommu(d)       (d->arch.emulation_flags & EMU_IOMMU)

(d) should be bracketed in the expansion.

> +
>  #define has_arch_pdevs(d)    (!list_empty(&(d)->arch.pdev_list))
>  
>  #define gdt_ldt_pt_idx(v) \
> diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
> index 2ecc9c9..6b387a3 100644
> --- a/xen/include/public/arch-x86/xen.h
> +++ b/xen/include/public/arch-x86/xen.h
> @@ -268,7 +268,25 @@ typedef struct arch_shared_info arch_shared_info_t;
>   * XEN_DOMCTL_INTERFACE_VERSION.
>   */
>  struct xen_arch_domainconfig {
> -    char dummy;
> +#define _EMU_LAPIC                  0
> +#define EMU_LAPIC                   (1U<<_EMU_LAPIC)
> +#define _EMU_HPET                   1
> +#define EMU_HPET                    (1U<<_EMU_HPET)
> +#define _EMU_PMTIMER                2
> +#define EMU_PMTIMER                 (1U<<_EMU_PMTIMER)
> +#define _EMU_RTC                    3
> +#define EMU_RTC                     (1U<<_EMU_RTC)
> +#define _EMU_IOAPIC                 4
> +#define EMU_IOAPIC                  (1U<<_EMU_IOAPIC)
> +#define _EMU_PIC                    5
> +#define EMU_PIC                     (1U<<_EMU_PIC)
> +#define _EMU_PMU                    6
> +#define EMU_PMU                     (1U<<_EMU_PMU)
> +#define _EMU_VGA                    7
> +#define EMU_VGA                     (1U<<_EMU_VGA)
> +#define _EMU_IOMMU                  8
> +#define EMU_IOMMU                   (1U<<_EMU_IOMMU)

These are all in the public API, so absolutely needs a XEN_ prefix, and
likely an X86 one as well.

> +    uint32_t emulation_flags;

Either state that this is ignored for domain types other than HVM, or
have arch_domain_create() confirm that it is 0.

~Andrew

Re: [PATCH v3 11/32] xen/x86: add bitmap of enabled emulated devices

[Jan Beulich]

>>> On 06.07.15 at 16:10, <andrew.cooper3@citrix.com> wrote:
> On 03/07/15 12:34, Roger Pau Monne wrote:
>> @@ -550,6 +551,20 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
>>                 d->domain_id);
>>      }
>>  
>> +    if ( is_hvm_domain(d) )
>> +    {
>> +        emulation_mask = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER | EMU_RTC |
>> +                          EMU_IOAPIC | EMU_PIC | EMU_PMU | EMU_VGA | EMU_IOMMU);
>> +        if ( (config->emulation_flags & emulation_mask) != emulation_mask )
>> +        {
>> +            printk(XENLOG_G_ERR "Xen does not allow HVM creation with the "
>> +                   "current selection of emulators: 0x%x.\n",
>> +                   config->emulation_flags);
> 
> Please put "d%d" in the error message to identify which domain has the
> bad configuration.

Plus please use %#x instead of 0x%x.

>> +            return -EPERM;
> 
> EPERM feels wrong here.  It is not a lack of permissions causing the
> issue.  EINVAL/EOPNOTSUPP perhaps?

Indeed, -EOPNOTSUPP.

Jan

Re: [PATCH v3 11/32] xen/x86: add bitmap of enabled emulated devices

[Roger Pau Monné]

El 06/07/15 a les 16.10, Andrew Cooper ha escrit:
> On 03/07/15 12:34, Roger Pau Monne wrote:
>> diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
>> index a112953..d684f49 100644
>> --- a/xen/arch/x86/domain.c
>> +++ b/xen/arch/x86/domain.c
>> @@ -528,6 +528,7 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
>>  {
>>      int i, paging_initialised = 0;
>>      int rc = -ENOMEM;
>> +    uint32_t emulation_mask;
>>  
>>      d->arch.s3_integrity = !!(domcr_flags & DOMCRF_s3_integrity);
>>  
>> @@ -550,6 +551,20 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
>>                 d->domain_id);
>>      }
>>  
>> +    if ( is_hvm_domain(d) )
>> +    {
>> +        emulation_mask = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER | EMU_RTC |
>> +                          EMU_IOAPIC | EMU_PIC | EMU_PMU | EMU_VGA | EMU_IOMMU);
>> +        if ( (config->emulation_flags & emulation_mask) != emulation_mask )
>> +        {
>> +            printk(XENLOG_G_ERR "Xen does not allow HVM creation with the "
>> +                   "current selection of emulators: 0x%x.\n",
>> +                   config->emulation_flags);
> 
> Please put "d%d" in the error message to identify which domain has the
> bad configuration.
> 
>> +            return -EPERM;
> 
> EPERM feels wrong here.  It is not a lack of permissions causing the
> issue.  EINVAL/EOPNOTSUPP perhaps?

EOPNOTSUPP looks better indeed.

>> +        }
>> +        d->arch.emulation_flags = config->emulation_flags;
>> +    }
>> +
>>      if ( has_hvm_container_domain(d) )
>>      {
>>          d->arch.hvm_domain.hap_enabled =
>> diff --git a/xen/include/asm-x86/domain.h b/xen/include/asm-x86/domain.h
>> index 96bde65..502379e 100644
>> --- a/xen/include/asm-x86/domain.h
>> +++ b/xen/include/asm-x86/domain.h
>> @@ -356,8 +356,21 @@ struct arch_domain
>>  
>>      /* Mem_access emulation control */
>>      bool_t mem_access_emulate_enabled;
>> +
>> +    /* Emulated devices enabled bitmap. */
>> +    uint32_t emulation_flags;
>>  } __cacheline_aligned;
>>  
>> +#define has_vlapic(d)       (d->arch.emulation_flags & EMU_LAPIC)
>> +#define has_vhpet(d)        (d->arch.emulation_flags & EMU_HPET)
>> +#define has_vpmtimer(d)     (d->arch.emulation_flags & EMU_PMTIMER)
>> +#define has_vrtc(d)         (d->arch.emulation_flags & EMU_RTC)
>> +#define has_vioapic(d)      (d->arch.emulation_flags & EMU_IOAPIC)
>> +#define has_vpic(d)         (d->arch.emulation_flags & EMU_PIC)
>> +#define has_vpmu(d)         (d->arch.emulation_flags & EMU_PMU)
>> +#define has_vvga(d)         (d->arch.emulation_flags & EMU_VGA)
>> +#define has_viommu(d)       (d->arch.emulation_flags & EMU_IOMMU)
> 
> (d) should be bracketed in the expansion.
> 
>> +
>>  #define has_arch_pdevs(d)    (!list_empty(&(d)->arch.pdev_list))
>>  
>>  #define gdt_ldt_pt_idx(v) \
>> diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h
>> index 2ecc9c9..6b387a3 100644
>> --- a/xen/include/public/arch-x86/xen.h
>> +++ b/xen/include/public/arch-x86/xen.h
>> @@ -268,7 +268,25 @@ typedef struct arch_shared_info arch_shared_info_t;
>>   * XEN_DOMCTL_INTERFACE_VERSION.
>>   */
>>  struct xen_arch_domainconfig {
>> -    char dummy;
>> +#define _EMU_LAPIC                  0
>> +#define EMU_LAPIC                   (1U<<_EMU_LAPIC)
>> +#define _EMU_HPET                   1
>> +#define EMU_HPET                    (1U<<_EMU_HPET)
>> +#define _EMU_PMTIMER                2
>> +#define EMU_PMTIMER                 (1U<<_EMU_PMTIMER)
>> +#define _EMU_RTC                    3
>> +#define EMU_RTC                     (1U<<_EMU_RTC)
>> +#define _EMU_IOAPIC                 4
>> +#define EMU_IOAPIC                  (1U<<_EMU_IOAPIC)
>> +#define _EMU_PIC                    5
>> +#define EMU_PIC                     (1U<<_EMU_PIC)
>> +#define _EMU_PMU                    6
>> +#define EMU_PMU                     (1U<<_EMU_PMU)
>> +#define _EMU_VGA                    7
>> +#define EMU_VGA                     (1U<<_EMU_VGA)
>> +#define _EMU_IOMMU                  8
>> +#define EMU_IOMMU                   (1U<<_EMU_IOMMU)
> 
> These are all in the public API, so absolutely needs a XEN_ prefix, and
> likely an X86 one as well.
> 
>> +    uint32_t emulation_flags;
> 
> Either state that this is ignored for domain types other than HVM, or
> have arch_domain_create() confirm that it is 0.

IMHO adding a check to arch_domain_create is TRTTD.

Roger.

[PATCH v3 12/32] xen/x86: allow disabling the emulated local apic

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>
Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@amd.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: Jun Nakajima <jun.nakajima@intel.com>
Cc: Eddie Dong <eddie.dong@intel.com>
Cc: Kevin Tian <kevin.tian@intel.com>
---
 xen/arch/x86/hvm/svm/svm.c  | 16 +++++++++-------
 xen/arch/x86/hvm/vlapic.c   | 30 +++++++++++++++++++++++++-----
 xen/arch/x86/hvm/vmsi.c     |  6 ++++++
 xen/arch/x86/hvm/vmx/vmcs.c | 14 ++++++++++++++
 xen/arch/x86/hvm/vmx/vmx.c  |  9 ++++++++-
 5 files changed, 62 insertions(+), 13 deletions(-)

diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index a02f983..46062dd 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -1036,6 +1036,7 @@ static void noreturn svm_do_resume(struct vcpu *v)
     struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
     bool_t debug_state = v->domain->debugger_attached;
     bool_t vcpu_guestmode = 0;
+    struct vlapic *vlapic = vcpu_vlapic(v);
 
     if ( nestedhvm_enabled(v->domain) && nestedhvm_vcpu_in_guestmode(v) )
         vcpu_guestmode = 1;
@@ -1059,14 +1060,14 @@ static void noreturn svm_do_resume(struct vcpu *v)
         hvm_asid_flush_vcpu(v);
     }
 
-    if ( !vcpu_guestmode )
+    if ( !vcpu_guestmode && !vlapic_hw_disabled(vlapic) )
     {
         vintr_t intr;
 
         /* Reflect the vlapic's TPR in the hardware vtpr */
         intr = vmcb_get_vintr(vmcb);
         intr.fields.tpr =
-            (vlapic_get_reg(vcpu_vlapic(v), APIC_TASKPRI) & 0xFF) >> 4;
+            (vlapic_get_reg(vlapic, APIC_TASKPRI) & 0xFF) >> 4;
         vmcb_set_vintr(vmcb, intr);
     }
 
@@ -2295,6 +2296,7 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     int inst_len, rc;
     vintr_t intr;
     bool_t vcpu_guestmode = 0;
+    struct vlapic *vlapic = vcpu_vlapic(v);
 
     hvm_invalidate_regs_fields(regs);
 
@@ -2312,11 +2314,11 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
      * NB. We need to preserve the low bits of the TPR to make checked builds
      * of Windows work, even though they don't actually do anything.
      */
-    if ( !vcpu_guestmode ) {
+    if ( !vcpu_guestmode && !vlapic_hw_disabled(vlapic) ) {
         intr = vmcb_get_vintr(vmcb);
-        vlapic_set_reg(vcpu_vlapic(v), APIC_TASKPRI,
+        vlapic_set_reg(vlapic, APIC_TASKPRI,
                    ((intr.fields.tpr & 0x0F) << 4) |
-                   (vlapic_get_reg(vcpu_vlapic(v), APIC_TASKPRI) & 0x0F));
+                   (vlapic_get_reg(vlapic, APIC_TASKPRI) & 0x0F));
     }
 
     exit_reason = vmcb->exitcode;
@@ -2698,14 +2700,14 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
     }
 
   out:
-    if ( vcpu_guestmode )
+    if ( vcpu_guestmode || vlapic_hw_disabled(vlapic) )
         /* Don't clobber TPR of the nested guest. */
         return;
 
     /* The exit may have updated the TPR: reflect this in the hardware vtpr */
     intr = vmcb_get_vintr(vmcb);
     intr.fields.tpr =
-        (vlapic_get_reg(vcpu_vlapic(v), APIC_TASKPRI) & 0xFF) >> 4;
+        (vlapic_get_reg(vlapic, APIC_TASKPRI) & 0xFF) >> 4;
     vmcb_set_vintr(vmcb, intr);
 }
 
diff --git a/xen/arch/x86/hvm/vlapic.c b/xen/arch/x86/hvm/vlapic.c
index 5472f79..ff648b9 100644
--- a/xen/arch/x86/hvm/vlapic.c
+++ b/xen/arch/x86/hvm/vlapic.c
@@ -994,6 +994,9 @@ static void set_x2apic_id(struct vlapic *vlapic)
 
 bool_t vlapic_msr_set(struct vlapic *vlapic, uint64_t value)
 {
+    if ( !has_vlapic(vlapic_domain(vlapic)) )
+        return 0;
+
     if ( (vlapic->hw.apic_base_msr ^ value) & MSR_IA32_APICBASE_ENABLE )
     {
         if ( unlikely(value & MSR_IA32_APICBASE_EXTD) )
@@ -1043,8 +1046,7 @@ void vlapic_tdt_msr_set(struct vlapic *vlapic, uint64_t value)
     uint64_t guest_tsc;
     struct vcpu *v = vlapic_vcpu(vlapic);
 
-    /* may need to exclude some other conditions like vlapic->hw.disabled */
-    if ( !vlapic_lvtt_tdt(vlapic) )
+    if ( !vlapic_lvtt_tdt(vlapic) || vlapic_hw_disabled(vlapic) )
     {
         HVM_DBG_LOG(DBG_LEVEL_VLAPIC_TIMER, "ignore tsc deadline msr write");
         return;
@@ -1119,6 +1121,9 @@ static int __vlapic_accept_pic_intr(struct vcpu *v)
 
 int vlapic_accept_pic_intr(struct vcpu *v)
 {
+    if ( vlapic_hw_disabled(vcpu_vlapic(v)) )
+        return 0;
+
     TRACE_2D(TRC_HVM_EMUL_LAPIC_PIC_INTR,
              (v == v->domain->arch.hvm_domain.i8259_target),
              v ? __vlapic_accept_pic_intr(v) : -1);
@@ -1266,6 +1271,9 @@ static int lapic_save_hidden(struct domain *d, hvm_domain_context_t *h)
     struct vlapic *s;
     int rc = 0;
 
+    if ( !has_vlapic(d) )
+        return 0;
+
     for_each_vcpu ( d, v )
     {
         s = vcpu_vlapic(v);
@@ -1282,6 +1290,9 @@ static int lapic_save_regs(struct domain *d, hvm_domain_context_t *h)
     struct vlapic *s;
     int rc = 0;
 
+    if ( !has_vlapic(d) )
+        return 0;
+
     for_each_vcpu ( d, v )
     {
         if ( hvm_funcs.sync_pir_to_irr )
@@ -1329,7 +1340,10 @@ static int lapic_load_hidden(struct domain *d, hvm_domain_context_t *h)
     uint16_t vcpuid;
     struct vcpu *v;
     struct vlapic *s;
-    
+
+    if ( !has_vlapic(d) )
+        return 0;
+
     /* Which vlapic to load? */
     vcpuid = hvm_load_instance(h); 
     if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
@@ -1361,7 +1375,10 @@ static int lapic_load_regs(struct domain *d, hvm_domain_context_t *h)
     uint16_t vcpuid;
     struct vcpu *v;
     struct vlapic *s;
-    
+
+    if ( !has_vlapic(d) )
+        return 0;
+
     /* Which vlapic to load? */
     vcpuid = hvm_load_instance(h); 
     if ( vcpuid >= d->max_vcpus || (v = d->vcpu[vcpuid]) == NULL )
@@ -1400,7 +1417,7 @@ int vlapic_init(struct vcpu *v)
 
     HVM_DBG_LOG(DBG_LEVEL_VLAPIC, "%d", v->vcpu_id);
 
-    if ( is_pvh_vcpu(v) )
+    if ( is_pvh_vcpu(v) || !has_vlapic(v->domain) )
     {
         vlapic->hw.disabled = VLAPIC_HW_DISABLED;
         return 0;
@@ -1453,6 +1470,9 @@ void vlapic_destroy(struct vcpu *v)
 {
     struct vlapic *vlapic = vcpu_vlapic(v);
 
+    if ( !has_vlapic(vlapic_domain(vlapic)) )
+        return;
+
     tasklet_kill(&vlapic->init_sipi.tasklet);
     TRACE_0D(TRC_HVM_EMUL_LAPIC_STOP_TIMER);
     destroy_periodic_time(&vlapic->pt);
diff --git a/xen/arch/x86/hvm/vmsi.c b/xen/arch/x86/hvm/vmsi.c
index bf6090e..264e54d 100644
--- a/xen/arch/x86/hvm/vmsi.c
+++ b/xen/arch/x86/hvm/vmsi.c
@@ -483,6 +483,9 @@ found:
 
 void msixtbl_init(struct domain *d)
 {
+    if ( !has_vlapic(d) )
+        return;
+
     register_mmio_handler(d, &msixtbl_mmio_ops);
 }
 
@@ -491,6 +494,9 @@ void msixtbl_pt_cleanup(struct domain *d)
     struct msixtbl_entry *entry, *temp;
     unsigned long flags;
 
+    if ( !has_vlapic(d) )
+        return;
+
     /* msixtbl_list_lock must be acquired with irq_disabled for check_lock() */
     local_irq_save(flags); 
     spin_lock(&d->arch.hvm_domain.msixtbl_list_lock);
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 4c5ceb5..8df6b71 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -992,6 +992,20 @@ static int construct_vmcs(struct vcpu *v)
         ASSERT(!(v->arch.hvm_vmx.exec_control & CPU_BASED_RDTSC_EXITING));
     }
 
+    if ( !has_vlapic(d) )
+    {
+        /* Disable virtual apics, TPR */
+        v->arch.hvm_vmx.secondary_exec_control &=
+            ~(SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES
+              | SECONDARY_EXEC_APIC_REGISTER_VIRT
+              | SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY);
+        v->arch.hvm_vmx.exec_control &= ~CPU_BASED_TPR_SHADOW;
+
+        /* In turn, disable posted interrupts. */
+        __vmwrite(PIN_BASED_VM_EXEC_CONTROL,
+                  vmx_pin_based_exec_control & ~PIN_BASED_POSTED_INTERRUPT);
+    }
+
     vmx_update_cpu_exec_control(v);
 
     __vmwrite(VM_EXIT_CONTROLS, vmexit_ctl);
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index fc29b89..1e1aca3 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -88,6 +88,9 @@ static int vmx_domain_initialise(struct domain *d)
 {
     int rc;
 
+    if ( !has_vlapic(d) )
+        return 0;
+
     if ( (rc = vmx_alloc_vlapic_mapping(d)) != 0 )
         return rc;
 
@@ -96,6 +99,9 @@ static int vmx_domain_initialise(struct domain *d)
 
 static void vmx_domain_destroy(struct domain *d)
 {
+    if ( !has_vlapic(d) )
+        return;
+
     vmx_free_vlapic_mapping(d);
 }
 
@@ -2240,7 +2246,8 @@ static void vmx_install_vlapic_mapping(struct vcpu *v)
 {
     paddr_t virt_page_ma, apic_page_ma;
 
-    if ( !cpu_has_vmx_virtualize_apic_accesses )
+    if ( !cpu_has_vmx_virtualize_apic_accesses ||
+         v->domain->arch.hvm_domain.vmx.apic_access_mfn == 0 )
         return;
 
     virt_page_ma = page_to_maddr(vcpu_vlapic(v)->regs_page);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 13/32] xen/x86: allow disabling the emulated HPET

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/hpet.c | 13 +++++++++++++
 xen/arch/x86/hvm/hvm.c  |  1 -
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/hpet.c b/xen/arch/x86/hvm/hpet.c
index 732504a..9773034 100644
--- a/xen/arch/x86/hvm/hpet.c
+++ b/xen/arch/x86/hvm/hpet.c
@@ -517,6 +517,9 @@ static int hpet_save(struct domain *d, hvm_domain_context_t *h)
     int rc;
     uint64_t guest_time;
 
+    if ( !has_vhpet(d) )
+        return 0;
+
     write_lock(&hp->lock);
     guest_time = guest_time_hpet(hp);
 
@@ -576,6 +579,9 @@ static int hpet_load(struct domain *d, hvm_domain_context_t *h)
     uint64_t guest_time;
     int i;
 
+    if ( !has_vhpet(d) )
+        return 0;
+
     write_lock(&hp->lock);
 
     /* Reload the HPET registers */
@@ -634,6 +640,9 @@ void hpet_init(struct domain *d)
     HPETState *h = domain_vhpet(d);
     int i;
 
+    if ( !has_vhpet(d) )
+        return;
+
     memset(h, 0, sizeof(HPETState));
 
     rwlock_init(&h->lock);
@@ -661,6 +670,7 @@ void hpet_init(struct domain *d)
     }
 
     register_mmio_handler(d, &hpet_mmio_ops);
+    d->arch.hvm_domain.params[HVM_PARAM_HPET_ENABLED] = 1;
 }
 
 void hpet_deinit(struct domain *d)
@@ -668,6 +678,9 @@ void hpet_deinit(struct domain *d)
     int i;
     HPETState *h = domain_vhpet(d);
 
+    if ( !has_vhpet(d) )
+        return;
+
     write_lock(&h->lock);
 
     if ( hpet_enabled(h) )
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 901f9a8..9c6e7fd 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -1547,7 +1547,6 @@ int hvm_domain_initialise(struct domain *d)
 
     hvm_init_guest_time(d);
 
-    d->arch.hvm_domain.params[HVM_PARAM_HPET_ENABLED] = 1;
     d->arch.hvm_domain.params[HVM_PARAM_TRIPLE_FAULT_REASON] = SHUTDOWN_reboot;
 
     vpic_init(d);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 14/32] xen/x86: allow disabling the pmtimer

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/pmtimer.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/xen/arch/x86/hvm/pmtimer.c b/xen/arch/x86/hvm/pmtimer.c
index 8b531d9..335a296 100644
--- a/xen/arch/x86/hvm/pmtimer.c
+++ b/xen/arch/x86/hvm/pmtimer.c
@@ -248,6 +248,9 @@ static int pmtimer_save(struct domain *d, hvm_domain_context_t *h)
     uint32_t x, msb = s->pm.tmr_val & TMR_VAL_MSB;
     int rc;
 
+    if ( !has_vpmtimer(d) )
+        return 0;
+
     spin_lock(&s->lock);
 
     /* Update the counter to the guest's current time.  We always save
@@ -272,6 +275,9 @@ static int pmtimer_load(struct domain *d, hvm_domain_context_t *h)
 {
     PMTState *s = &d->arch.hvm_domain.pl_time.vpmt;
 
+    if ( !has_vpmtimer(d) )
+        return 0;
+
     spin_lock(&s->lock);
 
     /* Reload the registers */
@@ -329,6 +335,9 @@ void pmtimer_init(struct vcpu *v)
 {
     PMTState *s = &v->domain->arch.hvm_domain.pl_time.vpmt;
 
+    if ( !has_vpmtimer(v->domain) )
+        return;
+
     spin_lock_init(&s->lock);
 
     s->scale = ((uint64_t)FREQUENCE_PMTIMER << 32) / SYSTEM_TIME_HZ;
@@ -349,6 +358,10 @@ void pmtimer_init(struct vcpu *v)
 void pmtimer_deinit(struct domain *d)
 {
     PMTState *s = &d->arch.hvm_domain.pl_time.vpmt;
+
+    if ( !has_vpmtimer(d) )
+        return;
+
     kill_timer(&s->timer);
 }
 
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 15/32] xen/x86: allow disabling the emulated RTC

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/rtc.c | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)

diff --git a/xen/arch/x86/hvm/rtc.c b/xen/arch/x86/hvm/rtc.c
index fde120b..d6d24c8 100644
--- a/xen/arch/x86/hvm/rtc.c
+++ b/xen/arch/x86/hvm/rtc.c
@@ -726,6 +726,9 @@ void rtc_migrate_timers(struct vcpu *v)
 {
     RTCState *s = vcpu_vrtc(v);
 
+    if ( !has_vrtc(v->domain) )
+        return;
+
     if ( v->vcpu_id == 0 )
     {
         migrate_timer(&s->update_timer, v->processor);;
@@ -739,6 +742,10 @@ static int rtc_save(struct domain *d, hvm_domain_context_t *h)
 {
     RTCState *s = domain_vrtc(d);
     int rc;
+
+    if ( !has_vrtc(d) )
+        return 0;
+
     spin_lock(&s->lock);
     rc = hvm_save_entry(RTC, 0, h, &s->hw);
     spin_unlock(&s->lock);
@@ -750,6 +757,9 @@ static int rtc_load(struct domain *d, hvm_domain_context_t *h)
 {
     RTCState *s = domain_vrtc(d);
 
+    if ( !has_vrtc(d) )
+        return 0;
+
     spin_lock(&s->lock);
 
     /* Restore the registers */
@@ -790,6 +800,9 @@ void rtc_init(struct domain *d)
 {
     RTCState *s = domain_vrtc(d);
 
+    if ( !has_vrtc(d) )
+        return;
+
     spin_lock_init(&s->lock);
 
     init_timer(&s->update_timer, rtc_update_timer, s, smp_processor_id());
@@ -820,6 +833,9 @@ void rtc_deinit(struct domain *d)
 {
     RTCState *s = domain_vrtc(d);
 
+    if ( !has_vrtc(d) )
+        return;
+
     spin_barrier(&s->lock);
 
     TRACE_0D(TRC_HVM_EMUL_RTC_STOP_TIMER);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 16/32] xen/x86: allow disabling the emulated IO APIC

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/vioapic.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 9de2ff3..4cfc704 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -425,12 +425,20 @@ void vioapic_update_EOI(struct domain *d, u8 vector)
 static int ioapic_save(struct domain *d, hvm_domain_context_t *h)
 {
     struct hvm_hw_vioapic *s = domain_vioapic(d);
+
+    if ( !has_vioapic(d) )
+        return 0;
+
     return hvm_save_entry(IOAPIC, 0, h, s);
 }
 
 static int ioapic_load(struct domain *d, hvm_domain_context_t *h)
 {
     struct hvm_hw_vioapic *s = domain_vioapic(d);
+
+    if ( !has_vioapic(d) )
+        return 0;
+
     return hvm_load_entry(IOAPIC, h, s);
 }
 
@@ -449,6 +457,9 @@ void vioapic_reset(struct domain *d)
 
 int vioapic_init(struct domain *d)
 {
+    if ( !has_vioapic(d) )
+        return 0;
+
     if ( (d->arch.hvm_domain.vioapic == NULL) &&
          ((d->arch.hvm_domain.vioapic = xmalloc(struct hvm_vioapic)) == NULL) )
         return -ENOMEM;
@@ -463,6 +474,9 @@ int vioapic_init(struct domain *d)
 
 void vioapic_deinit(struct domain *d)
 {
+    if ( !has_vioapic(d) )
+        return;
+
     xfree(d->arch.hvm_domain.vioapic);
     d->arch.hvm_domain.vioapic = NULL;
 }
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 17/32] xen/x86: allow disabling the emulated PIC

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/vpic.c | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vpic.c b/xen/arch/x86/hvm/vpic.c
index 909b9d0..2eb162a 100644
--- a/xen/arch/x86/hvm/vpic.c
+++ b/xen/arch/x86/hvm/vpic.c
@@ -377,6 +377,9 @@ static int vpic_save(struct domain *d, hvm_domain_context_t *h)
     struct hvm_hw_vpic *s;
     int i;
 
+    if ( !has_vpic(d) )
+        return 0;
+
     /* Save the state of both PICs */
     for ( i = 0; i < 2 ; i++ )
     {
@@ -392,7 +395,10 @@ static int vpic_load(struct domain *d, hvm_domain_context_t *h)
 {
     struct hvm_hw_vpic *s;
     uint16_t inst;
-    
+
+    if ( !has_vpic(d) )
+        return 0;
+
     /* Which PIC is this? */
     inst = hvm_load_instance(h);
     if ( inst > 1 )
@@ -425,6 +431,9 @@ void vpic_reset(struct domain *d)
 
 void vpic_init(struct domain *d)
 {
+    if ( !has_vpic(d) )
+        return;
+
     vpic_reset(d);
 
     register_portio_handler(d, 0x20, 2, vpic_intercept_pic_io);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 18/32] xen/x86: allow disabling the emulated pmu

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/vpmu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/arch/x86/hvm/vpmu.c b/xen/arch/x86/hvm/vpmu.c
index c3273ee..776661d 100644
--- a/xen/arch/x86/hvm/vpmu.c
+++ b/xen/arch/x86/hvm/vpmu.c
@@ -233,7 +233,7 @@ void vpmu_initialise(struct vcpu *v)
     uint8_t vendor = current_cpu_data.x86_vendor;
     int ret;
 
-    if ( is_pvh_vcpu(v) )
+    if ( is_pvh_vcpu(v) || !has_vpmu(v->domain) )
         return;
 
     ASSERT(!vpmu->flags && !vpmu->context);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 19/32] xen/x86: allow disabling the emulated VGA

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/stdvga.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/xen/arch/x86/hvm/stdvga.c b/xen/arch/x86/hvm/stdvga.c
index f13983e..87d0725 100644
--- a/xen/arch/x86/hvm/stdvga.c
+++ b/xen/arch/x86/hvm/stdvga.c
@@ -546,6 +546,9 @@ void stdvga_init(struct domain *d)
     void *p;
     int i;
 
+    if ( !has_vvga(d) )
+        return;
+
     memset(s, 0, sizeof(*s));
     spin_lock_init(&s->lock);
     
@@ -581,6 +584,9 @@ void stdvga_deinit(struct domain *d)
     struct hvm_hw_stdvga *s = &d->arch.hvm_domain.stdvga;
     int i;
 
+    if ( !has_vvga(d) )
+        return;
+
     for ( i = 0; i != ARRAY_SIZE(s->vram_page); i++ )
     {
         if ( s->vram_page[i] == NULL )
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 20/32] xen/x86: allow disabling the emulated IOMMU

[Roger Pau Monne]

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: Aravind Gopalakrishnan <Aravind.Gopalakrishnan@amd.com>
---
 xen/drivers/passthrough/amd/iommu_guest.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/drivers/passthrough/amd/iommu_guest.c b/xen/drivers/passthrough/amd/iommu_guest.c
index 5908bf3..0831908 100644
--- a/xen/drivers/passthrough/amd/iommu_guest.c
+++ b/xen/drivers/passthrough/amd/iommu_guest.c
@@ -888,7 +888,8 @@ int guest_iommu_init(struct domain* d)
     struct guest_iommu *iommu;
     struct hvm_iommu *hd  = domain_hvm_iommu(d);
 
-    if ( !is_hvm_domain(d) || !iommu_enabled || !iommuv2_enabled )
+    if ( !is_hvm_domain(d) || !iommu_enabled || !iommuv2_enabled ||
+         !has_viommu(d) )
         return 0;
 
     iommu = xzalloc(struct guest_iommu);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 21/32] xen/x86: allow disabling all emulated devices inside of Xen

[Roger Pau Monne]

Only allow enabling or disabling all the emulated devices inside of Xen,
right now Xen doesn't support enabling specific emulated devices only.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/domain.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index d684f49..bdaf646 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -555,7 +555,8 @@ int arch_domain_create(struct domain *d, unsigned int domcr_flags,
     {
         emulation_mask = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER | EMU_RTC |
                           EMU_IOAPIC | EMU_PIC | EMU_PMU | EMU_VGA | EMU_IOMMU);
-        if ( (config->emulation_flags & emulation_mask) != emulation_mask )
+        if ( (config->emulation_flags & emulation_mask) != emulation_mask &&
+             (config->emulation_flags & emulation_mask) != 0 )
         {
             printk(XENLOG_G_ERR "Xen does not allow HVM creation with the "
                    "current selection of emulators: 0x%x.\n",
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 22/32] elfnotes: intorduce a new PHYS_ENTRY elfnote

[Roger Pau Monne]

This new elfnote contains the 32bit entry point into the kernel. Xen will
use this entry point in order to launch the guest kernel in 32bit protected
mode with paging disabled.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/xcutils/readnotes.c          |  3 +++
 xen/common/libelf/libelf-dominfo.c |  4 ++++
 xen/include/public/elfnote.h       | 11 ++++++++++-
 3 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/tools/xcutils/readnotes.c b/tools/xcutils/readnotes.c
index 5fa445e..e682dd1 100644
--- a/tools/xcutils/readnotes.c
+++ b/tools/xcutils/readnotes.c
@@ -159,6 +159,9 @@ static unsigned print_notes(struct elf_binary *elf, ELF_HANDLE_DECL(elf_note) st
 		case XEN_ELFNOTE_L1_MFN_VALID:
 			print_l1_mfn_valid_note("L1_MFN_VALID", elf , note);
 			break;
+		case XEN_ELFNOTE_PHYS32_ENTRY:
+			print_numeric_note("PHYS32_ENTRY", elf , note);
+			break;
 		default:
 			printf("unknown note type %#x\n",
 			       (unsigned)elf_uval(elf, note, type));
diff --git a/xen/common/libelf/libelf-dominfo.c b/xen/common/libelf/libelf-dominfo.c
index 0771323..78d75ad 100644
--- a/xen/common/libelf/libelf-dominfo.c
+++ b/xen/common/libelf/libelf-dominfo.c
@@ -120,6 +120,7 @@ elf_errorstatus elf_xen_parse_note(struct elf_binary *elf,
         [XEN_ELFNOTE_BSD_SYMTAB] = { "BSD_SYMTAB", 1},
         [XEN_ELFNOTE_SUSPEND_CANCEL] = { "SUSPEND_CANCEL", 0 },
         [XEN_ELFNOTE_MOD_START_PFN] = { "MOD_START_PFN", 0 },
+        [XEN_ELFNOTE_PHYS32_ENTRY] = { "PHYS32_ENTRY", 0 },
     };
 /* *INDENT-ON* */
 
@@ -213,6 +214,9 @@ elf_errorstatus elf_xen_parse_note(struct elf_binary *elf,
                 elf, note, sizeof(*parms->f_supported), i);
         break;
 
+    case XEN_ELFNOTE_PHYS32_ENTRY:
+        parms->phys_entry = val;
+        break;
     }
     return 0;
 }
diff --git a/xen/include/public/elfnote.h b/xen/include/public/elfnote.h
index 3824a94..e6fc596 100644
--- a/xen/include/public/elfnote.h
+++ b/xen/include/public/elfnote.h
@@ -200,9 +200,18 @@
 #define XEN_ELFNOTE_SUPPORTED_FEATURES 17
 
 /*
+ * Physical entry point into the kernel.
+ *
+ * 32bit entry point into the kernel. Xen will use this entry point
+ * in order to launch the guest kernel in 32bit protected mode
+ * with paging disabled.
+ */
+#define XEN_ELFNOTE_PHYS32_ENTRY 18
+
+/*
  * The number of the highest elfnote defined.
  */
-#define XEN_ELFNOTE_MAX XEN_ELFNOTE_SUPPORTED_FEATURES
+#define XEN_ELFNOTE_MAX XEN_ELFNOTE_PHYS32_ENTRY
 
 /*
  * System information exported through crash notes.
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 23/32] libxc: allow creating domains without emulated devices.

[Roger Pau Monne]

Introduce a new flag in xc_dom_image that turns on and off the emulated
devices. This prevents creating the VGA hole, the hvm_info page and the
ioreq server pages. libxl unconditionally sets it to true for all HVM
domains at the moment.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/include/xc_dom.h |  1 +
 tools/libxc/xc_dom_x86.c     | 71 +++++++++++++++++++++++++-------------------
 tools/libxl/libxl_dom.c      |  1 +
 3 files changed, 42 insertions(+), 31 deletions(-)

diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
index 68b052c..7086f6e 100644
--- a/tools/libxc/include/xc_dom.h
+++ b/tools/libxc/include/xc_dom.h
@@ -194,6 +194,7 @@ struct xc_dom_image {
     xen_pfn_t mmio_size;
     xen_pfn_t lowmem_end;
     xen_pfn_t highmem_end;
+    bool emulation;
 
     /* Extra ACPI tables passed to HVMLOADER */
     struct xc_hvm_firmware_module acpi_module;
diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 6573b94..336eab4 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -520,12 +520,15 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
     xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
     xc_interface *xch = dom->xch;
 
-    if ( (hvm_info_page = xc_map_foreign_range(
-              xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
-              HVM_INFO_PFN)) == NULL )
-        goto error_out;
-    build_hvm_info(hvm_info_page, dom);
-    munmap(hvm_info_page, PAGE_SIZE);
+    if ( dom->emulation )
+    {
+        if ( (hvm_info_page = xc_map_foreign_range(
+                  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
+                  HVM_INFO_PFN)) == NULL )
+            goto error_out;
+        build_hvm_info(hvm_info_page, dom);
+        munmap(hvm_info_page, PAGE_SIZE);
+    }
 
     /* Allocate and clear special pages. */
     for ( i = 0; i < NR_SPECIAL_PAGES; i++ )
@@ -557,30 +560,33 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
     xc_hvm_param_set(xch, domid, HVM_PARAM_SHARING_RING_PFN,
                      special_pfn(SPECIALPAGE_SHARING));
 
-    /*
-     * Allocate and clear additional ioreq server pages. The default
-     * server will use the IOREQ and BUFIOREQ special pages above.
-     */
-    for ( i = 0; i < NR_IOREQ_SERVER_PAGES; i++ )
-        ioreq_server_array[i] = ioreq_server_pfn(i);
-
-    rc = xc_domain_populate_physmap_exact(xch, domid, NR_IOREQ_SERVER_PAGES, 0,
-                                          0, ioreq_server_array);
-    if ( rc != 0 )
+    if ( dom->emulation )
     {
-        DOMPRINTF("Could not allocate ioreq server pages.");
-        goto error_out;
-    }
+        /*
+         * Allocate and clear additional ioreq server pages. The default
+         * server will use the IOREQ and BUFIOREQ special pages above.
+         */
+        for ( i = 0; i < NR_IOREQ_SERVER_PAGES; i++ )
+            ioreq_server_array[i] = ioreq_server_pfn(i);
 
-    if ( xc_clear_domain_pages(xch, domid, ioreq_server_pfn(0),
-                               NR_IOREQ_SERVER_PAGES) )
+        rc = xc_domain_populate_physmap_exact(xch, domid, NR_IOREQ_SERVER_PAGES, 0,
+                                              0, ioreq_server_array);
+        if ( rc != 0 )
+        {
+            DOMPRINTF("Could not allocate ioreq server pages.");
             goto error_out;
+        }
 
-    /* Tell the domain where the pages are and how many there are */
-    xc_hvm_param_set(xch, domid, HVM_PARAM_IOREQ_SERVER_PFN,
-                     ioreq_server_pfn(0));
-    xc_hvm_param_set(xch, domid, HVM_PARAM_NR_IOREQ_SERVER_PAGES,
-                     NR_IOREQ_SERVER_PAGES);
+        if ( xc_clear_domain_pages(xch, domid, ioreq_server_pfn(0),
+                                   NR_IOREQ_SERVER_PAGES) )
+                goto error_out;
+
+        /* Tell the domain where the pages are and how many there are */
+        xc_hvm_param_set(xch, domid, HVM_PARAM_IOREQ_SERVER_PFN,
+                         ioreq_server_pfn(0));
+        xc_hvm_param_set(xch, domid, HVM_PARAM_NR_IOREQ_SERVER_PAGES,
+                         NR_IOREQ_SERVER_PAGES);
+    }
 
     /*
      * Identity-map page table is required for running with CR0.PG=0 when
@@ -1228,7 +1234,8 @@ static int meminit_hvm(struct xc_dom_image *dom)
      * allocated is pointless.
      */
     if ( claim_enabled ) {
-        rc = xc_domain_claim_pages(xch, domid, target_pages - VGA_HOLE_SIZE);
+        rc = xc_domain_claim_pages(xch, domid, target_pages -
+                                   dom->emulation ? VGA_HOLE_SIZE : 0);
         if ( rc != 0 )
         {
             DOMPRINTF("Could not allocate memory for HVM guest as we cannot claim memory!");
@@ -1244,7 +1251,8 @@ static int meminit_hvm(struct xc_dom_image *dom)
          * tot_pages will be target_pages - VGA_HOLE_SIZE after
          * this call.
          */
-        rc = xc_domain_set_pod_target(xch, domid, target_pages - VGA_HOLE_SIZE,
+        rc = xc_domain_set_pod_target(xch, domid, target_pages -
+                                      dom->emulation ? VGA_HOLE_SIZE : 0,
                                       NULL, NULL, NULL);
         if ( rc != 0 )
         {
@@ -1263,8 +1271,9 @@ static int meminit_hvm(struct xc_dom_image *dom)
      * Under 2MB mode, we allocate pages in batches of no more than 8MB to 
      * ensure that we can be preempted and hence dom0 remains responsive.
      */
-    rc = xc_domain_populate_physmap_exact(
-        xch, domid, 0xa0, 0, memflags, &dom->p2m_host[0x00]);
+    if ( dom->emulation )
+        rc = xc_domain_populate_physmap_exact(
+            xch, domid, 0xa0, 0, memflags, &dom->p2m_host[0x00]);
 
     stat_normal_pages = 0;
     for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
@@ -1283,7 +1292,7 @@ static int meminit_hvm(struct xc_dom_image *dom)
          * 0xA0000-0xC0000. Note that 0x00000-0xA0000 is populated just
          * before this loop.
          */
-        if ( vmemranges[vmemid].start == 0 )
+        if ( vmemranges[vmemid].start == 0 && dom->emulation )
         {
             cur_pages = 0xc0;
             stat_normal_pages += 0xc0;
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 480b7e7..6ce4115 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -992,6 +992,7 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
     dom->lowmem_end = lowmem_end;
     dom->highmem_end = highmem_end;
     dom->mmio_start = mmio_start;
+    dom->emulation = true;
 
     if (info->num_vnuma_nodes != 0) {
         int i;
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 23/32] libxc: allow creating domains without emulated devices.

[Wei Liu]

On Fri, Jul 03, 2015 at 01:35:01PM +0200, Roger Pau Monne wrote:
> Introduce a new flag in xc_dom_image that turns on and off the emulated
> devices. This prevents creating the VGA hole, the hvm_info page and the
> ioreq server pages. libxl unconditionally sets it to true for all HVM
> domains at the moment.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> ---
>  tools/libxc/include/xc_dom.h |  1 +
>  tools/libxc/xc_dom_x86.c     | 71 +++++++++++++++++++++++++-------------------
>  tools/libxl/libxl_dom.c      |  1 +
>  3 files changed, 42 insertions(+), 31 deletions(-)
> 
> diff --git a/tools/libxc/include/xc_dom.h b/tools/libxc/include/xc_dom.h
> index 68b052c..7086f6e 100644
> --- a/tools/libxc/include/xc_dom.h
> +++ b/tools/libxc/include/xc_dom.h
> @@ -194,6 +194,7 @@ struct xc_dom_image {
>      xen_pfn_t mmio_size;
>      xen_pfn_t lowmem_end;
>      xen_pfn_t highmem_end;
> +    bool emulation;
>  

Can you document what this flag enables / disables?

Wei.

[PATCH v3 24/32] xen: allow HVM guests to use XENMEM_memory_map

[Roger Pau Monne]

Enable this hypercall for HVM guests in order to fetch the e820 memory
map in the absence of an emulated BIOS. The memory map is populated and
notified to Xen in arch_setup_meminit_hvm.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_dom_x86.c | 29 ++++++++++++++++++++++++++++-
 xen/arch/x86/hvm/hvm.c   |  2 --
 xen/arch/x86/mm.c        |  6 ------
 3 files changed, 28 insertions(+), 9 deletions(-)

diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 336eab4..45b1f03 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -1113,6 +1113,7 @@ static int check_mmio_hole(uint64_t start, uint64_t memsize,
         return 1;
 }
 
+#define MAX_E820_ENTRIES    128
 static int meminit_hvm(struct xc_dom_image *dom)
 {
     unsigned long i, vmemid, nr_pages = dom->total_pages;
@@ -1133,6 +1134,8 @@ static int meminit_hvm(struct xc_dom_image *dom)
     unsigned int nr_vmemranges, nr_vnodes;
     xc_interface *xch = dom->xch;
     uint32_t domid = dom->guest_domid;
+    struct e820entry entries[MAX_E820_ENTRIES];
+    int e820_index = 0;
 
     if ( nr_pages > target_pages )
         memflags |= XENMEMF_populate_on_demand;
@@ -1183,6 +1186,13 @@ static int meminit_hvm(struct xc_dom_image *dom)
         vnode_to_pnode = dom->vnode_to_pnode;
     }
 
+    /* Add one additional memory range to account for the VGA hole */
+    if ( (nr_vmemranges + (dom->emulation ? 1 : 0)) > MAX_E820_ENTRIES )
+    {
+        DOMPRINTF("Too many memory ranges");
+        goto error_out;
+    }
+
     total_pages = 0;
     p2m_size = 0;
     for ( i = 0; i < nr_vmemranges; i++ )
@@ -1271,9 +1281,13 @@ static int meminit_hvm(struct xc_dom_image *dom)
      * Under 2MB mode, we allocate pages in batches of no more than 8MB to 
      * ensure that we can be preempted and hence dom0 remains responsive.
      */
-    if ( dom->emulation )
+    if ( dom->emulation ) {
         rc = xc_domain_populate_physmap_exact(
             xch, domid, 0xa0, 0, memflags, &dom->p2m_host[0x00]);
+        entries[e820_index].addr = 0;
+        entries[e820_index].size = 0xa0 << PAGE_SHIFT;
+        entries[e820_index++].type = E820_RAM;
+    }
 
     stat_normal_pages = 0;
     for ( vmemid = 0; vmemid < nr_vmemranges; vmemid++ )
@@ -1300,6 +1314,12 @@ static int meminit_hvm(struct xc_dom_image *dom)
         else
             cur_pages = vmemranges[vmemid].start >> PAGE_SHIFT;
 
+                /* Build an e820 map. */
+        entries[e820_index].addr = cur_pages << PAGE_SHIFT;
+        entries[e820_index].size = vmemranges[vmemid].end -
+                                   entries[e820_index].addr;
+        entries[e820_index++].type = E820_RAM;
+
         while ( (rc == 0) && (end_pages > cur_pages) )
         {
             /* Clip count to maximum 1GB extent. */
@@ -1417,6 +1437,13 @@ static int meminit_hvm(struct xc_dom_image *dom)
     DPRINTF("  2MB PAGES: 0x%016lx\n", stat_2mb_pages);
     DPRINTF("  1GB PAGES: 0x%016lx\n", stat_1gb_pages);
 
+    rc = xc_domain_set_memory_map(xch, domid, entries, e820_index);
+    if ( rc != 0 )
+    {
+        DOMPRINTF("unable to set memory map.");
+        goto error_out;
+    }
+
     rc = 0;
     goto out;
  error_out:
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 9c6e7fd..75a5566 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4765,7 +4765,6 @@ static long hvm_memory_op(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 
     switch ( cmd & MEMOP_CMD_MASK )
     {
-    case XENMEM_memory_map:
     case XENMEM_machine_memory_map:
     case XENMEM_machphys_mapping:
         return -ENOSYS;
@@ -4841,7 +4840,6 @@ static long hvm_memory_op_compat32(int cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
 
     switch ( cmd & MEMOP_CMD_MASK )
     {
-    case XENMEM_memory_map:
     case XENMEM_machine_memory_map:
     case XENMEM_machphys_mapping:
         return -ENOSYS;
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index fd151c6..92eccd0 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -4717,12 +4717,6 @@ long arch_memory_op(unsigned long cmd, XEN_GUEST_HANDLE_PARAM(void) arg)
             return rc;
         }
 
-        if ( is_hvm_domain(d) )
-        {
-            rcu_unlock_domain(d);
-            return -EPERM;
-        }
-
         e820 = xmalloc_array(e820entry_t, fmap.map.nr_entries);
         if ( e820 == NULL )
         {
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 25/32] xen/x86: allow HVM guests to use hypercalls to bring up vCPUs

[Roger Pau Monne]

Allow the usage of the VCPUOP_initialise, VCPUOP_up, VCPUOP_down and
VCPUOP_is_up hypercalls from HVM guests.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/hvm.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 75a5566..350e409 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4808,6 +4808,10 @@ static long hvm_vcpu_op(
     case VCPUOP_stop_singleshot_timer:
     case VCPUOP_register_vcpu_info:
     case VCPUOP_register_vcpu_time_memory_area:
+    case VCPUOP_initialise:
+    case VCPUOP_up:
+    case VCPUOP_down:
+    case VCPUOP_is_up:
         rc = do_vcpu_op(cmd, vcpuid, arg);
         break;
     default:
@@ -4866,6 +4870,10 @@ static long hvm_vcpu_op_compat32(
     case VCPUOP_stop_singleshot_timer:
     case VCPUOP_register_vcpu_info:
     case VCPUOP_register_vcpu_time_memory_area:
+    case VCPUOP_initialise:
+    case VCPUOP_up:
+    case VCPUOP_down:
+    case VCPUOP_is_up:
         rc = compat_vcpu_op(cmd, vcpuid, arg);
         break;
     default:
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 26/32] xenconsole: try to attach to PV console if HVM fails

[Roger Pau Monne]

HVM guests have always used the emulated serial console by default, but if
the emulated serial pty cannot be fetched from xenstore try to use the PV
console instead.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/console/client/main.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/tools/console/client/main.c b/tools/console/client/main.c
index f4c783b..c92553e 100644
--- a/tools/console/client/main.c
+++ b/tools/console/client/main.c
@@ -279,7 +279,7 @@ int main(int argc, char **argv)
 		{ 0 },
 
 	};
-	char *dom_path = NULL, *path = NULL;
+	char *dom_path = NULL, *path = NULL, *test = NULL;
 	int spty, xsfd;
 	struct xs_handle *xs;
 	char *end;
@@ -357,9 +357,14 @@ int main(int argc, char **argv)
 	path = malloc(strlen(dom_path) + strlen("/device/console/0/tty") + 5);
 	if (path == NULL)
 		err(ENOMEM, "malloc");
-	if (type == CONSOLE_SERIAL)
+	if (type == CONSOLE_SERIAL) {
 		snprintf(path, strlen(dom_path) + strlen("/serial/0/tty") + 5, "%s/serial/%d/tty", dom_path, num);
-	else {
+		test = xs_read(xs, XBT_NULL, path, NULL);
+		free(test);
+		if (test == NULL)
+			goto pv_console;
+	} else {
+pv_console:
 		if (num == 0)
 			snprintf(path, strlen(dom_path) + strlen("/console/tty") + 1, "%s/console/tty", dom_path);
 		else
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 26/32] xenconsole: try to attach to PV console if HVM fails

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1806 bytes --]

On Fri, 3 Jul 2015, Roger Pau Monne wrote:
> HVM guests have always used the emulated serial console by default, but if
> the emulated serial pty cannot be fetched from xenstore try to use the PV
> console instead.
>
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> ---
>  tools/console/client/main.c | 11 ++++++++---
>  1 file changed, 8 insertions(+), 3 deletions(-)
>
> diff --git a/tools/console/client/main.c b/tools/console/client/main.c
> index f4c783b..c92553e 100644
> --- a/tools/console/client/main.c
> +++ b/tools/console/client/main.c
> @@ -279,7 +279,7 @@ int main(int argc, char **argv)
>  		{ 0 },
>
>  	};
> -	char *dom_path = NULL, *path = NULL;
> +	char *dom_path = NULL, *path = NULL, *test = NULL;
>  	int spty, xsfd;
>  	struct xs_handle *xs;
>  	char *end;
> @@ -357,9 +357,14 @@ int main(int argc, char **argv)
>  	path = malloc(strlen(dom_path) + strlen("/device/console/0/tty") + 5);
>  	if (path == NULL)
>  		err(ENOMEM, "malloc");
> -	if (type == CONSOLE_SERIAL)
> +	if (type == CONSOLE_SERIAL) {
>  		snprintf(path, strlen(dom_path) + strlen("/serial/0/tty") + 5, "%s/serial/%d/tty", dom_path, num);
> -	else {
> +		test = xs_read(xs, XBT_NULL, path, NULL);
> +		free(test);
> +		if (test == NULL)
> +			goto pv_console;

instead of using labels I would probably do:

        if (test == NULL)
            type = CONSOLE_PV;
    }
    if (type == CONSOLE_PV) {
        ...

> +	} else {
> +pv_console:
>  		if (num == 0)
>  			snprintf(path, strlen(dom_path) + strlen("/console/tty") + 1, "%s/console/tty", dom_path);
>  		else

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 26/32] xenconsole: try to attach to PV console if HVM fails

[Roger Pau Monné]

El 14/07/15 a les 15.46, Stefano Stabellini ha escrit:
> On Fri, 3 Jul 2015, Roger Pau Monne wrote:
>> HVM guests have always used the emulated serial console by default, but if
>> the emulated serial pty cannot be fetched from xenstore try to use the PV
>> console instead.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
>> ---
>>  tools/console/client/main.c | 11 ++++++++---
>>  1 file changed, 8 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/console/client/main.c b/tools/console/client/main.c
>> index f4c783b..c92553e 100644
>> --- a/tools/console/client/main.c
>> +++ b/tools/console/client/main.c
>> @@ -279,7 +279,7 @@ int main(int argc, char **argv)
>>  		{ 0 },
>>  
>>  	};
>> -	char *dom_path = NULL, *path = NULL;
>> +	char *dom_path = NULL, *path = NULL, *test = NULL;
>>  	int spty, xsfd;
>>  	struct xs_handle *xs;
>>  	char *end;
>> @@ -357,9 +357,14 @@ int main(int argc, char **argv)
>>  	path = malloc(strlen(dom_path) + strlen("/device/console/0/tty") + 5);
>>  	if (path == NULL)
>>  		err(ENOMEM, "malloc");
>> -	if (type == CONSOLE_SERIAL)
>> +	if (type == CONSOLE_SERIAL) {
>>  		snprintf(path, strlen(dom_path) + strlen("/serial/0/tty") + 5, "%s/serial/%d/tty", dom_path, num);
>> -	else {
>> +		test = xs_read(xs, XBT_NULL, path, NULL);
>> +		free(test);
>> +		if (test == NULL)
>> +			goto pv_console;
> 
> instead of using labels I would probably do:
> 
>         if (test == NULL)
>             type = CONSOLE_PV;
>     }
>     if (type == CONSOLE_PV) {
>         ...

Yes thanks, that's much more clearer than using labels.

Roger.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 27/32] libxc: change the position of the special pages

[Roger Pau Monne]

Change the physical memory address of the special pages when there are no
emulated devices. On HVM guests the special pages have always been reserved
so that they end at the 0xff000 pfn, but there are some problems with this
approach when used without emulated devices:

 - If we want to allow HVMlite to be used for Dom0 those special pages
   cannot be placed inside of the MMIO hole or else they may clash with MMIO
   regions of physical devices passed-through to Dom0.
 - If there's no emulated devices the guests needs to access the console
   page and the command line very early during boot. This is a problem for
   FreeBSD at least, since the early page tables only map memory up to 1GiB.

So instead append the special pages after the kernel and ramdisk. The guest
must make sure it figures out the position of those pages very early during
boot, since the region is not marked as reserved in the memory map.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 tools/libxc/xc_dom_x86.c | 50 ++++++++++++++++++++++++++++++------------------
 1 file changed, 31 insertions(+), 19 deletions(-)

diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 45b1f03..1195400 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -58,15 +58,27 @@
 #define SPECIALPAGE_IDENT_PT 6
 #define SPECIALPAGE_CONSOLE  7
 #define NR_SPECIAL_PAGES     8
-#define special_pfn(x) (0xff000u - NR_SPECIAL_PAGES + (x))
+#define LAST_SPECIAL_PFN     0xff000u
 
 #define NR_IOREQ_SERVER_PAGES 8
-#define ioreq_server_pfn(x) (special_pfn(0) - NR_IOREQ_SERVER_PAGES + (x))
+#define ioreq_server_pfn(x, dom) (special_pfn(0, dom) - NR_IOREQ_SERVER_PAGES + (x))
 
 #define bits_to_mask(bits)       (((xen_vaddr_t)1 << (bits))-1)
 #define round_down(addr, mask)   ((addr) & ~(mask))
 #define round_up(addr, mask)     ((addr) | (mask))
 
+static unsigned long special_pfn(int index, struct xc_dom_image *dom)
+{
+    unsigned long start_pfn;
+
+    if ( dom->emulation )
+        return (LAST_SPECIAL_PFN - NR_SPECIAL_PAGES + index);
+
+    start_pfn = (dom->virt_alloc_end - dom->parms.virt_base) >>
+                XC_DOM_PAGE_SHIFT(dom);
+    return (start_pfn + index);
+}
+
 /* get guest IO ABI protocol */
 const char *xc_domain_get_native_protocol(xc_interface *xch,
                                           uint32_t domid)
@@ -502,7 +514,7 @@ static void build_hvm_info(void *hvm_info_page, struct xc_dom_image *dom)
     /* Memory parameters. */
     hvm_info->low_mem_pgend = dom->lowmem_end >> PAGE_SHIFT;
     hvm_info->high_mem_pgend = dom->highmem_end >> PAGE_SHIFT;
-    hvm_info->reserved_mem_pgstart = ioreq_server_pfn(0);
+    hvm_info->reserved_mem_pgstart = ioreq_server_pfn(0, dom);
 
     /* Finish with the checksum. */
     for ( i = 0, sum = 0; i < hvm_info->length; i++ )
@@ -532,7 +544,7 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
 
     /* Allocate and clear special pages. */
     for ( i = 0; i < NR_SPECIAL_PAGES; i++ )
-        special_array[i] = special_pfn(i);
+        special_array[i] = special_pfn(i, dom);
 
     rc = xc_domain_populate_physmap_exact(xch, domid, NR_SPECIAL_PAGES, 0, 0,
                                           special_array);
@@ -542,23 +554,23 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
         goto error_out;
     }
 
-    if ( xc_clear_domain_pages(xch, domid, special_pfn(0), NR_SPECIAL_PAGES) )
+    if ( xc_clear_domain_pages(xch, domid, special_pfn(0, dom), NR_SPECIAL_PAGES) )
             goto error_out;
 
     xc_hvm_param_set(xch, domid, HVM_PARAM_STORE_PFN,
-                     special_pfn(SPECIALPAGE_XENSTORE));
+                     special_pfn(SPECIALPAGE_XENSTORE, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_BUFIOREQ_PFN,
-                     special_pfn(SPECIALPAGE_BUFIOREQ));
+                     special_pfn(SPECIALPAGE_BUFIOREQ, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_IOREQ_PFN,
-                     special_pfn(SPECIALPAGE_IOREQ));
+                     special_pfn(SPECIALPAGE_IOREQ, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_CONSOLE_PFN,
-                     special_pfn(SPECIALPAGE_CONSOLE));
+                     special_pfn(SPECIALPAGE_CONSOLE, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_PAGING_RING_PFN,
-                     special_pfn(SPECIALPAGE_PAGING));
+                     special_pfn(SPECIALPAGE_PAGING, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_MONITOR_RING_PFN,
-                     special_pfn(SPECIALPAGE_ACCESS));
+                     special_pfn(SPECIALPAGE_ACCESS, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_SHARING_RING_PFN,
-                     special_pfn(SPECIALPAGE_SHARING));
+                     special_pfn(SPECIALPAGE_SHARING, dom));
 
     if ( dom->emulation )
     {
@@ -567,7 +579,7 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
          * server will use the IOREQ and BUFIOREQ special pages above.
          */
         for ( i = 0; i < NR_IOREQ_SERVER_PAGES; i++ )
-            ioreq_server_array[i] = ioreq_server_pfn(i);
+            ioreq_server_array[i] = ioreq_server_pfn(i, dom);
 
         rc = xc_domain_populate_physmap_exact(xch, domid, NR_IOREQ_SERVER_PAGES, 0,
                                               0, ioreq_server_array);
@@ -577,13 +589,13 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
             goto error_out;
         }
 
-        if ( xc_clear_domain_pages(xch, domid, ioreq_server_pfn(0),
+        if ( xc_clear_domain_pages(xch, domid, ioreq_server_pfn(0, dom),
                                    NR_IOREQ_SERVER_PAGES) )
                 goto error_out;
 
         /* Tell the domain where the pages are and how many there are */
         xc_hvm_param_set(xch, domid, HVM_PARAM_IOREQ_SERVER_PFN,
-                         ioreq_server_pfn(0));
+                         ioreq_server_pfn(0, dom));
         xc_hvm_param_set(xch, domid, HVM_PARAM_NR_IOREQ_SERVER_PAGES,
                          NR_IOREQ_SERVER_PAGES);
     }
@@ -594,17 +606,17 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
      */
     if ( (ident_pt = xc_map_foreign_range(
               xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
-              special_pfn(SPECIALPAGE_IDENT_PT))) == NULL )
+              special_pfn(SPECIALPAGE_IDENT_PT, dom))) == NULL )
         goto error_out;
     for ( i = 0; i < PAGE_SIZE / sizeof(*ident_pt); i++ )
         ident_pt[i] = ((i << 22) | _PAGE_PRESENT | _PAGE_RW | _PAGE_USER |
                        _PAGE_ACCESSED | _PAGE_DIRTY | _PAGE_PSE);
     munmap(ident_pt, PAGE_SIZE);
     xc_hvm_param_set(xch, domid, HVM_PARAM_IDENT_PT,
-                     special_pfn(SPECIALPAGE_IDENT_PT) << PAGE_SHIFT);
+                     special_pfn(SPECIALPAGE_IDENT_PT, dom) << PAGE_SHIFT);
 
-    dom->console_pfn = special_pfn(SPECIALPAGE_CONSOLE);
-    dom->xenstore_pfn = special_pfn(SPECIALPAGE_XENSTORE);
+    dom->console_pfn = special_pfn(SPECIALPAGE_CONSOLE, dom);
+    dom->xenstore_pfn = special_pfn(SPECIALPAGE_XENSTORE, dom);
     dom->parms.virt_hypercall = -1;
 
     rc = 0;
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 27/32] libxc: change the position of the special pages

[Roger Pau Monné]

El 03/07/15 a les 13.35, Roger Pau Monne ha escrit:
> Change the physical memory address of the special pages when there are no
> emulated devices. On HVM guests the special pages have always been reserved
> so that they end at the 0xff000 pfn, but there are some problems with this
> approach when used without emulated devices:
> 
>  - If we want to allow HVMlite to be used for Dom0 those special pages
>    cannot be placed inside of the MMIO hole or else they may clash with MMIO
>    regions of physical devices passed-through to Dom0.
>  - If there's no emulated devices the guests needs to access the console
>    page and the command line very early during boot. This is a problem for
>    FreeBSD at least, since the early page tables only map memory up to 1GiB.
> 
> So instead append the special pages after the kernel and ramdisk. The guest
> must make sure it figures out the position of those pages very early during
> boot, since the region is not marked as reserved in the memory map.

After thinking a little bit about this, I don't think this is the right
solution. Placing all the special pages after the kernel/ramdisk will
make it quite hard to use bootloaders like pvgrub inside of HVMlite
guests. I will see if I can come up with a better way to solve this,
comments welcome.

Roger.

Re: [PATCH v3 27/32] libxc: change the position of the special pages

[Konrad Rzeszutek Wilk]

On Fri, Jul 03, 2015 at 05:36:15PM +0200, Roger Pau Monné wrote:
> El 03/07/15 a les 13.35, Roger Pau Monne ha escrit:
> > Change the physical memory address of the special pages when there are no
> > emulated devices. On HVM guests the special pages have always been reserved
> > so that they end at the 0xff000 pfn, but there are some problems with this
> > approach when used without emulated devices:
> > 
> >  - If we want to allow HVMlite to be used for Dom0 those special pages
> >    cannot be placed inside of the MMIO hole or else they may clash with MMIO
> >    regions of physical devices passed-through to Dom0.
> >  - If there's no emulated devices the guests needs to access the console
> >    page and the command line very early during boot. This is a problem for
> >    FreeBSD at least, since the early page tables only map memory up to 1GiB.
> > 
> > So instead append the special pages after the kernel and ramdisk. The guest
> > must make sure it figures out the position of those pages very early during
> > boot, since the region is not marked as reserved in the memory map.
> 
> After thinking a little bit about this, I don't think this is the right
> solution. Placing all the special pages after the kernel/ramdisk will
> make it quite hard to use bootloaders like pvgrub inside of HVMlite
> guests. I will see if I can come up with a better way to solve this,
> comments welcome.

Could stick at the start of memory, mark it as E820_RESV or such. Or
at 16MB or such?
> 
> Roger.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v3 27/32] libxc: change the position of the special pages

[Roger Pau Monné]

El 10/07/15 a les 21.06, Konrad Rzeszutek Wilk ha escrit:
> On Fri, Jul 03, 2015 at 05:36:15PM +0200, Roger Pau Monné wrote:
>> El 03/07/15 a les 13.35, Roger Pau Monne ha escrit:
>>> Change the physical memory address of the special pages when there are no
>>> emulated devices. On HVM guests the special pages have always been reserved
>>> so that they end at the 0xff000 pfn, but there are some problems with this
>>> approach when used without emulated devices:
>>>
>>>  - If we want to allow HVMlite to be used for Dom0 those special pages
>>>    cannot be placed inside of the MMIO hole or else they may clash with MMIO
>>>    regions of physical devices passed-through to Dom0.
>>>  - If there's no emulated devices the guests needs to access the console
>>>    page and the command line very early during boot. This is a problem for
>>>    FreeBSD at least, since the early page tables only map memory up to 1GiB.
>>>
>>> So instead append the special pages after the kernel and ramdisk. The guest
>>> must make sure it figures out the position of those pages very early during
>>> boot, since the region is not marked as reserved in the memory map.
>>
>> After thinking a little bit about this, I don't think this is the right
>> solution. Placing all the special pages after the kernel/ramdisk will
>> make it quite hard to use bootloaders like pvgrub inside of HVMlite
>> guests. I will see if I can come up with a better way to solve this,
>> comments welcome.
> 
> Could stick at the start of memory, mark it as E820_RESV or such. Or
> at 16MB or such?

I was thinking about leaving the already existing special pages as they
are now (just below the 4GiB), and placing the command line and module
info just after the kernel and initramfs, so the memory layout looks like:

                      Command line string
                      +
                      |  Module list
                      |  +
0GiB                  |  |                                 4GiB
+---+--------+-------+v-+v-+----------------------------+--+
|   |        |       |  |  |                            |  |
|   |        |       |  |  |                            |  |
|   | Kernel |Modules|  |  |                            |  |
|   |        |       |  |  |                            |  |
+---+--------+-------+--+--+----------------------------+--+
                                                         Console page,
                                                         Xenstore page,
                                                         ...

Roger.

[PATCH v3 28/32] libxc/xen: introduce HVM_PARAM_CMDLINE_PFN

[Roger Pau Monne]

This HVM parameter returns a PFN that contains the address of the memory
page where the guest command line has been placed.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_dom_x86.c        | 21 ++++++++++++++++++++-
 xen/arch/x86/hvm/hvm.c          |  1 +
 xen/include/public/hvm/params.h |  5 ++++-
 3 files changed, 25 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 1195400..41ff7a4 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -57,7 +57,8 @@
 #define SPECIALPAGE_IOREQ    5
 #define SPECIALPAGE_IDENT_PT 6
 #define SPECIALPAGE_CONSOLE  7
-#define NR_SPECIAL_PAGES     8
+#define SPECIALPAGE_CMDLINE  8
+#define NR_SPECIAL_PAGES     9
 #define LAST_SPECIAL_PFN     0xff000u
 
 #define NR_IOREQ_SERVER_PAGES 8
@@ -531,6 +532,7 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
     xen_pfn_t special_array[NR_SPECIAL_PAGES];
     xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
     xc_interface *xch = dom->xch;
+    char *cmdline;
 
     if ( dom->emulation )
     {
@@ -572,6 +574,23 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
     xc_hvm_param_set(xch, domid, HVM_PARAM_SHARING_RING_PFN,
                      special_pfn(SPECIALPAGE_SHARING, dom));
 
+    if ( dom->cmdline )
+    {
+        cmdline = xc_map_foreign_range(
+                  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
+                  special_pfn(SPECIALPAGE_CMDLINE, dom));
+        if ( cmdline == NULL ) {
+            DOMPRINTF("Unable to map command line page");
+            goto error_out;
+        }
+
+        strncpy(cmdline, dom->cmdline, MAX_GUEST_CMDLINE);
+        cmdline[MAX_GUEST_CMDLINE - 1] = '\0';
+        munmap(cmdline, PAGE_SIZE);
+        xc_hvm_param_set(xch, domid, HVM_PARAM_CMDLINE_PFN,
+                     special_pfn(SPECIALPAGE_CMDLINE, dom));
+    }
+
     if ( dom->emulation )
     {
         /*
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 350e409..efe7232 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5917,6 +5917,7 @@ static int hvm_allow_get_param(struct domain *d,
     case HVM_PARAM_STORE_EVTCHN:
     case HVM_PARAM_CONSOLE_PFN:
     case HVM_PARAM_CONSOLE_EVTCHN:
+    case HVM_PARAM_CMDLINE_PFN:
         break;
     /*
      * The following parameters must not be read by the guest
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index 7c73089..b7f8839 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -187,6 +187,9 @@
 /* Location of the VM Generation ID in guest physical address space. */
 #define HVM_PARAM_VM_GENERATION_ID_ADDR 34
 
-#define HVM_NR_PARAMS          35
+/* PFN of the command line. */
+#define HVM_PARAM_CMDLINE_PFN 35
+
+#define HVM_NR_PARAMS          36
 
 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 29/32] libxc/xen: introduce HVM_PARAM_FIRST_FREE_PFN

[Roger Pau Monne]

This HVM parameter returns the first free pfn after all the special pages.
It can be used by guests to figure out the first free memory address after
the kernel, ramdisk and special pages. This is interesting for compatibility
reasons in case more special pages are later added, older guests can still
use this parameter to figure out the first free address, ignoring newly
added special pages.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_dom_x86.c        | 2 ++
 xen/arch/x86/hvm/hvm.c          | 1 +
 xen/include/public/hvm/params.h | 5 ++++-
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 41ff7a4..74c3819 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -573,6 +573,8 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
                      special_pfn(SPECIALPAGE_ACCESS, dom));
     xc_hvm_param_set(xch, domid, HVM_PARAM_SHARING_RING_PFN,
                      special_pfn(SPECIALPAGE_SHARING, dom));
+    xc_hvm_param_set(xch, domid, HVM_PARAM_FIRST_FREE_PFN,
+                     special_pfn(NR_SPECIAL_PAGES, dom));
 
     if ( dom->cmdline )
     {
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index efe7232..2b190e9 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5918,6 +5918,7 @@ static int hvm_allow_get_param(struct domain *d,
     case HVM_PARAM_CONSOLE_PFN:
     case HVM_PARAM_CONSOLE_EVTCHN:
     case HVM_PARAM_CMDLINE_PFN:
+    case HVM_PARAM_FIRST_FREE_PFN:
         break;
     /*
      * The following parameters must not be read by the guest
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index b7f8839..47e38a8 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -190,6 +190,9 @@
 /* PFN of the command line. */
 #define HVM_PARAM_CMDLINE_PFN 35
 
-#define HVM_NR_PARAMS          36
+/* First free PFN after the special pages. */
+#define HVM_PARAM_FIRST_FREE_PFN 36
+
+#define HVM_NR_PARAMS          37
 
 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 29/32] libxc/xen: introduce HVM_PARAM_FIRST_FREE_PFN

[Konrad Rzeszutek Wilk]

On Fri, Jul 03, 2015 at 01:35:07PM +0200, Roger Pau Monne wrote:
> This HVM parameter returns the first free pfn after all the special pages.

Could you describe how the layout of PFNs is now for the HVM guests?

Is there some document that outlines it? I know that for PV we do 
have it as part of the start_info structure but not sure about HVM?

> It can be used by guests to figure out the first free memory address after
> the kernel, ramdisk and special pages. This is interesting for compatibility

So you mention it here, perhaps there should be an document
mentioning this. Or the header for the PFNs?

> reasons in case more special pages are later added, older guests can still
> use this parameter to figure out the first free address, ignoring newly
> added special pages.

And presumarily figure out the 'special PFN's and mark them
as reserved? Are those PFNs in the E820 as RESERVED?

We should also put some restrictions in the code then to make sure
that there will never be any HVM_PARAM_*PFN _after_ this PFN.

And document that somewhere.

> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>  tools/libxc/xc_dom_x86.c        | 2 ++
>  xen/arch/x86/hvm/hvm.c          | 1 +
>  xen/include/public/hvm/params.h | 5 ++++-
>  3 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
> index 41ff7a4..74c3819 100644
> --- a/tools/libxc/xc_dom_x86.c
> +++ b/tools/libxc/xc_dom_x86.c
> @@ -573,6 +573,8 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
>                       special_pfn(SPECIALPAGE_ACCESS, dom));
>      xc_hvm_param_set(xch, domid, HVM_PARAM_SHARING_RING_PFN,
>                       special_pfn(SPECIALPAGE_SHARING, dom));
> +    xc_hvm_param_set(xch, domid, HVM_PARAM_FIRST_FREE_PFN,
> +                     special_pfn(NR_SPECIAL_PAGES, dom));
>  
>      if ( dom->cmdline )
>      {
> diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
> index efe7232..2b190e9 100644
> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -5918,6 +5918,7 @@ static int hvm_allow_get_param(struct domain *d,
>      case HVM_PARAM_CONSOLE_PFN:
>      case HVM_PARAM_CONSOLE_EVTCHN:
>      case HVM_PARAM_CMDLINE_PFN:
> +    case HVM_PARAM_FIRST_FREE_PFN:
>          break;
>      /*
>       * The following parameters must not be read by the guest
> diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
> index b7f8839..47e38a8 100644
> --- a/xen/include/public/hvm/params.h
> +++ b/xen/include/public/hvm/params.h
> @@ -190,6 +190,9 @@
>  /* PFN of the command line. */
>  #define HVM_PARAM_CMDLINE_PFN 35
>  
> -#define HVM_NR_PARAMS          36
> +/* First free PFN after the special pages. */
> +#define HVM_PARAM_FIRST_FREE_PFN 36
> +
> +#define HVM_NR_PARAMS          37
>  
>  #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
> -- 
> 1.9.5 (Apple Git-50.3)
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v3 29/32] libxc/xen: introduce HVM_PARAM_FIRST_FREE_PFN

[Roger Pau Monné]

El 10/07/15 a les 21.05, Konrad Rzeszutek Wilk ha escrit:
> On Fri, Jul 03, 2015 at 01:35:07PM +0200, Roger Pau Monne wrote:
>> This HVM parameter returns the first free pfn after all the special pages.
> 
> Could you describe how the layout of PFNs is now for the HVM guests?
> 
> Is there some document that outlines it? I know that for PV we do 
> have it as part of the start_info structure but not sure about HVM?
> 
>> It can be used by guests to figure out the first free memory address after
>> the kernel, ramdisk and special pages. This is interesting for compatibility
> 
> So you mention it here, perhaps there should be an document
> mentioning this. Or the header for the PFNs?
> 
>> reasons in case more special pages are later added, older guests can still
>> use this parameter to figure out the first free address, ignoring newly
>> added special pages.
> 
> And presumarily figure out the 'special PFN's and mark them
> as reserved? Are those PFNs in the E820 as RESERVED?
> 
> We should also put some restrictions in the code then to make sure
> that there will never be any HVM_PARAM_*PFN _after_ this PFN.

I don't think we need this any longer, with the new approach that I've
described in the previous email (see the reply to "libxc: change the
position of the special pages") there's no need to introduce this new
HVMPARAM, and the special pages will be marked as "reserved" in the
memory map, with the exception of the command line and the module list
special pages, that are appended after the kernel/initramfs.

Roger.

[PATCH v3 30/32] libxc/xen: introduce HVM_PARAM_MODLIST_PFN

[Roger Pau Monne]

This HVM parameter is used to pass a list of loaded modules to the guest.
Right now the number of loaded modules is limited to 1 by the current
implementation, but this interface allows passing more than one module.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Keir Fraser <keir@xen.org>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 tools/libxc/xc_dom_x86.c        | 24 +++++++++++++++++++++++-
 xen/arch/x86/hvm/hvm.c          |  1 +
 xen/include/public/hvm/params.h | 23 ++++++++++++++++++++++-
 3 files changed, 46 insertions(+), 2 deletions(-)

diff --git a/tools/libxc/xc_dom_x86.c b/tools/libxc/xc_dom_x86.c
index 74c3819..71b353e 100644
--- a/tools/libxc/xc_dom_x86.c
+++ b/tools/libxc/xc_dom_x86.c
@@ -58,7 +58,8 @@
 #define SPECIALPAGE_IDENT_PT 6
 #define SPECIALPAGE_CONSOLE  7
 #define SPECIALPAGE_CMDLINE  8
-#define NR_SPECIAL_PAGES     9
+#define SPECIALPAGE_MODLIST  9
+#define NR_SPECIAL_PAGES     10
 #define LAST_SPECIAL_PFN     0xff000u
 
 #define NR_IOREQ_SERVER_PAGES 8
@@ -533,6 +534,7 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
     xen_pfn_t ioreq_server_array[NR_IOREQ_SERVER_PAGES];
     xc_interface *xch = dom->xch;
     char *cmdline;
+    uint64_t *modlist;
 
     if ( dom->emulation )
     {
@@ -621,6 +623,26 @@ static int alloc_magic_pages_hvm(struct xc_dom_image *dom)
                          NR_IOREQ_SERVER_PAGES);
     }
 
+    if ( dom->ramdisk_blob )
+    {
+        modlist = xc_map_foreign_range(
+                  xch, domid, PAGE_SIZE, PROT_READ | PROT_WRITE,
+                  special_pfn(SPECIALPAGE_MODLIST, dom));
+        if ( modlist == NULL ) {
+            DOMPRINTF("Unable to map module list page");
+            goto error_out;
+        }
+
+        /* This is currently limited to only one module. */
+        modlist[0] = dom->ramdisk_seg.vstart - dom->parms.virt_base;
+        modlist[1] = dom->ramdisk_seg.vend - dom->ramdisk_seg.vstart;
+        modlist[2] = 0;
+        modlist[3] = 0;
+        munmap(modlist, PAGE_SIZE);
+        xc_hvm_param_set(xch, domid, HVM_PARAM_MODLIST_PFN,
+                     special_pfn(SPECIALPAGE_MODLIST, dom));
+    }
+
     /*
      * Identity-map page table is required for running with CR0.PG=0 when
      * using Intel EPT. Create a 32-bit non-PAE page directory of superpages.
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 2b190e9..95f12c0 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -5919,6 +5919,7 @@ static int hvm_allow_get_param(struct domain *d,
     case HVM_PARAM_CONSOLE_EVTCHN:
     case HVM_PARAM_CMDLINE_PFN:
     case HVM_PARAM_FIRST_FREE_PFN:
+    case HVM_PARAM_MODLIST_PFN:
         break;
     /*
      * The following parameters must not be read by the guest
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index 47e38a8..86cc15b 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -193,6 +193,27 @@
 /* First free PFN after the special pages. */
 #define HVM_PARAM_FIRST_FREE_PFN 36
 
-#define HVM_NR_PARAMS          37
+/*
+ * List of modules passed to the kernel.
+ *
+ * The PFN returned by this HVM_PARAM points to a page that contains an
+ * array of unsigned 64bit integers encoded in little endian.
+ *
+ * The first integer contains the address where the module has been loaded,
+ * while the second contains the size of the module in bytes. The last element
+ * in the array is a module with address 0 and length 0:
+ *
+ * module[0] = <address of 1st module>
+ * module[1] = <size of 1st module>
+ * [...]
+ * module[N/2] = <address of module N>
+ * module[N/2+1] = <size of module N>
+ * [...]
+ * module[M] = 0
+ * module[M+1] = 0
+ */
+#define HVM_PARAM_MODLIST_PFN 37
+
+#define HVM_NR_PARAMS          38
 
 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3 31/32] libxc: switch xc_dom_elfloader to be used with HVMlite domains

[Roger Pau Monne]

Allow xc_dom_elfloader to report a guest type as hvm-3.0-x86_32 if it's
running inside of a HVM container and has the PHYS32_ENTRY elfnote set.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
Only xc_dom_elfloader has been switched to support HVMlite, other loaders
should also be switched once we have a HVMlite compatible kernel that uses
them.
---
 tools/libxc/xc_dom_elfloader.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c
index 6ce1062..2f05015 100644
--- a/tools/libxc/xc_dom_elfloader.c
+++ b/tools/libxc/xc_dom_elfloader.c
@@ -57,6 +57,10 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom,
 {
     uint64_t machine = elf_uval(elf, elf->ehdr, e_machine);
 
+    if ( dom->container_type == XC_DOM_HVM_CONTAINER &&
+         dom->parms.phys_entry != UNSET_ADDR )
+        return "hvm-3.0-x86_32";
+
     switch ( machine )
     {
     case EM_386:
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 31/32] libxc: switch xc_dom_elfloader to be used with HVMlite domains

[Konrad Rzeszutek Wilk]

On Fri, Jul 03, 2015 at 01:35:09PM +0200, Roger Pau Monne wrote:
> Allow xc_dom_elfloader to report a guest type as hvm-3.0-x86_32 if it's
> running inside of a HVM container and has the PHYS32_ENTRY elfnote set.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> ---
> Only xc_dom_elfloader has been switched to support HVMlite, other loaders
> should also be switched once we have a HVMlite compatible kernel that uses
> them.
> ---
>  tools/libxc/xc_dom_elfloader.c | 4 ++++
>  1 file changed, 4 insertions(+)
> 
> diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c
> index 6ce1062..2f05015 100644
> --- a/tools/libxc/xc_dom_elfloader.c
> +++ b/tools/libxc/xc_dom_elfloader.c
> @@ -57,6 +57,10 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom,
>  {
>      uint64_t machine = elf_uval(elf, elf->ehdr, e_machine);
>  
> +    if ( dom->container_type == XC_DOM_HVM_CONTAINER &&
> +         dom->parms.phys_entry != UNSET_ADDR )
> +        return "hvm-3.0-x86_32";

why not 'hvm-lite-1.0-x86_64' since it is new protocol?
> +
>      switch ( machine )
>      {
>      case EM_386:
> -- 
> 1.9.5 (Apple Git-50.3)
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v3 31/32] libxc: switch xc_dom_elfloader to be used with HVMlite domains

[Roger Pau Monné]

El 10/07/15 a les 21.07, Konrad Rzeszutek Wilk ha escrit:
> On Fri, Jul 03, 2015 at 01:35:09PM +0200, Roger Pau Monne wrote:
>> Allow xc_dom_elfloader to report a guest type as hvm-3.0-x86_32 if it's
>> running inside of a HVM container and has the PHYS32_ENTRY elfnote set.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
>> ---
>> Only xc_dom_elfloader has been switched to support HVMlite, other loaders
>> should also be switched once we have a HVMlite compatible kernel that uses
>> them.
>> ---
>>  tools/libxc/xc_dom_elfloader.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/tools/libxc/xc_dom_elfloader.c b/tools/libxc/xc_dom_elfloader.c
>> index 6ce1062..2f05015 100644
>> --- a/tools/libxc/xc_dom_elfloader.c
>> +++ b/tools/libxc/xc_dom_elfloader.c
>> @@ -57,6 +57,10 @@ static char *xc_dom_guest_type(struct xc_dom_image *dom,
>>  {
>>      uint64_t machine = elf_uval(elf, elf->ehdr, e_machine);
>>  
>> +    if ( dom->container_type == XC_DOM_HVM_CONTAINER &&
>> +         dom->parms.phys_entry != UNSET_ADDR )
>> +        return "hvm-3.0-x86_32";
> 
> why not 'hvm-lite-1.0-x86_64' since it is new protocol?

Because it shares the code paths with the HVM builder. We could
introduce a new type, but I don't see any value in doing it, it's just
going to be a copy of hvm-3.0-x86_32.

Roger.

[PATCH v3 32/32] libxl: allow the creation of HVM domains without a device model.

[Roger Pau Monne]

Replace the firmware loaded into HVM guests with an OS kernel. Since the HVM
builder now uses the PV xc_dom_* set of functions this kernel will be parsed
and loaded inside the guest like on PV, but the container is a pure HVM
guest.

Also, if device_model_version is set to none unconditinally set the nic type
to LIBXL_NIC_TYPE_VIF.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Cc: Ian Campbell <ian.campbell@citrix.com>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 docs/man/xl.cfg.pod.5       |  5 ++++
 tools/libxl/libxl.c         |  7 +++---
 tools/libxl/libxl_create.c  | 18 ++++++++++++++
 tools/libxl/libxl_dom.c     | 60 ++++++++++++++++++++++++++++++---------------
 tools/libxl/libxl_types.idl |  1 +
 tools/libxl/libxl_x86.c     |  4 ++-
 tools/libxl/xl_cmdimpl.c    |  2 ++
 7 files changed, 72 insertions(+), 25 deletions(-)

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index a3e0e2e..1270605 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1631,6 +1631,11 @@ This device-model is the default for Linux dom0.
 Use the device-model based upon the historical Xen fork of Qemu.
 This device-model is still the default for NetBSD dom0.
 
+=item B<none>
+
+Don't use any device model. This requires a kernel capable of booting
+in this mode.
+
 =back
 
 It is recommended to accept the default value for new guests.  If
diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
index e9a2d26..a6690cf 100644
--- a/tools/libxl/libxl.c
+++ b/tools/libxl/libxl.c
@@ -1590,11 +1590,10 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
 
     switch (libxl__domain_type(gc, domid)) {
     case LIBXL_DOMAIN_TYPE_HVM:
-        if (!libxl_get_stubdom_id(CTX, domid))
-            dm_present = 1;
-        else
+        if (libxl_get_stubdom_id(CTX, domid)) {
             dm_present = 0;
-        break;
+            break;
+        }
     case LIBXL_DOMAIN_TYPE_PV:
         pid = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/image/device-model-pid", domid));
         dm_present = (pid != NULL);
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index f366a09..1444e40 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -164,6 +164,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
                 b_info->u.hvm.bios = LIBXL_BIOS_TYPE_ROMBIOS; break;
             case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
                 b_info->u.hvm.bios = LIBXL_BIOS_TYPE_SEABIOS; break;
+            case LIBXL_DEVICE_MODEL_VERSION_NONE:
+                break;
             default:return ERROR_INVAL;
             }
 
@@ -177,6 +179,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
             if (b_info->u.hvm.bios == LIBXL_BIOS_TYPE_ROMBIOS)
                 return ERROR_INVAL;
             break;
+        case LIBXL_DEVICE_MODEL_VERSION_NONE:
+            break;
         default:abort();
         }
 
@@ -278,6 +282,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
                 break;
             }
             break;
+        case LIBXL_DEVICE_MODEL_VERSION_NONE:
+            b_info->video_memkb = 0;
+            break;
         case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
         default:
             switch (b_info->u.hvm.vga.kind) {
@@ -945,6 +952,11 @@ static void initiate_domain_create(libxl__egc *egc,
         ret = libxl__device_nic_setdefault(gc, &d_config->nics[i], domid);
         if (ret) goto error_out;
 
+        /* HVM guests without a device model only have PV nics. */
+        if (d_config->b_info.device_model_version ==
+            LIBXL_DEVICE_MODEL_VERSION_NONE)
+            d_config->nics[i].nictype = LIBXL_NIC_TYPE_VIF;
+
         if (d_config->nics[i].devid > last_devid)
             last_devid = d_config->nics[i].devid;
     }
@@ -1293,6 +1305,12 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
         libxl__device_console_add(gc, domid, &console, state, &device);
         libxl__device_console_dispose(&console);
 
+        if (d_config->b_info.device_model_version ==
+            LIBXL_DEVICE_MODEL_VERSION_NONE) {
+            domcreate_devmodel_started(egc, &dcs->dmss.dm, 0);
+            return;
+        }
+
         libxl_device_vkb_init(&vkb);
         libxl__device_vkb_add(gc, domid, &vkb);
         libxl_device_vkb_dispose(&vkb);
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index 6ce4115..59854a1 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -789,21 +789,23 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid,
     uint64_t str_mfn, cons_mfn;
     int i;
 
-    va_map = xc_map_foreign_range(handle, domid,
-                                  XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
-                                  HVM_INFO_PFN);
-    if (va_map == NULL)
-        return -1;
-
-    va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
-    va_hvm->apic_mode = libxl_defbool_val(info->u.hvm.apic);
-    va_hvm->nr_vcpus = info->max_vcpus;
-    memset(va_hvm->vcpu_online, 0, sizeof(va_hvm->vcpu_online));
-    memcpy(va_hvm->vcpu_online, info->avail_vcpus.map, info->avail_vcpus.size);
-    for (i = 0, sum = 0; i < va_hvm->length; i++)
-        sum += ((uint8_t *) va_hvm)[i];
-    va_hvm->checksum -= sum;
-    munmap(va_map, XC_PAGE_SIZE);
+    if ( info->device_model_version != LIBXL_DEVICE_MODEL_VERSION_NONE) {
+        va_map = xc_map_foreign_range(handle, domid,
+                                      XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
+                                      HVM_INFO_PFN);
+        if (va_map == NULL)
+            return -1;
+
+        va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
+        va_hvm->apic_mode = libxl_defbool_val(info->u.hvm.apic);
+        va_hvm->nr_vcpus = info->max_vcpus;
+        memset(va_hvm->vcpu_online, 0, sizeof(va_hvm->vcpu_online));
+        memcpy(va_hvm->vcpu_online, info->avail_vcpus.map, info->avail_vcpus.size);
+        for (i = 0, sum = 0; i < va_hvm->length; i++)
+            sum += ((uint8_t *) va_hvm)[i];
+        va_hvm->checksum -= sum;
+        munmap(va_map, XC_PAGE_SIZE);
+    }
 
     xc_hvm_param_get(handle, domid, HVM_PARAM_STORE_PFN, &str_mfn);
     xc_hvm_param_get(handle, domid, HVM_PARAM_CONSOLE_PFN, &cons_mfn);
@@ -884,6 +886,12 @@ static int libxl__domain_firmware(libxl__gc *gc,
         case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
             firmware = "hvmloader";
             break;
+        case LIBXL_DEVICE_MODEL_VERSION_NONE:
+            if (info->kernel == NULL) {
+                LOG(ERROR, "no device model requested without a kernel");
+                return ERROR_FAIL;
+            }
+            break;
         default:
             LOG(ERROR, "invalid device model version %d",
                 info->device_model_version);
@@ -892,10 +900,19 @@ static int libxl__domain_firmware(libxl__gc *gc,
         }
     }
 
-    rc = xc_dom_kernel_file(dom, libxl__abs_path(gc, firmware,
+    if (info->kernel != NULL &&
+        info->device_model_version == LIBXL_DEVICE_MODEL_VERSION_NONE) {
+        /* Try to load a kernel instead of the firmware. */
+        rc = xc_dom_kernel_file(dom, info->kernel);
+        if (rc == 0 && info->ramdisk != NULL)
+            rc = xc_dom_ramdisk_file(dom, info->ramdisk);
+    } else {
+        rc = xc_dom_kernel_file(dom, libxl__abs_path(gc, firmware,
                                                  libxl__xenfirmwaredir_path()));
+    }
+
     if (rc != 0) {
-        LOGE(ERROR, "xc_dom_kernel_file failed");
+        LOGE(ERROR, "xc_dom_{kernel_file/ramdisk_file} failed");
         goto out;
     }
 
@@ -946,10 +963,13 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
     int ret;
     uint64_t mmio_start, lowmem_end, highmem_end, mem_size;
     struct xc_dom_image *dom = NULL;
+    bool emulated_devices =
+        info->device_model_version != LIBXL_DEVICE_MODEL_VERSION_NONE ?
+        true : false;
 
     xc_dom_loginit(ctx->xch);
 
-    dom = xc_dom_allocate(ctx->xch, NULL, NULL);
+    dom = xc_dom_allocate(ctx->xch, info->cmdline, NULL);
     if (!dom) {
         LOGE(ERROR, "xc_dom_allocate failed");
         goto out;
@@ -979,7 +999,7 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
 
     if (dom->target_pages == 0)
         dom->target_pages = mem_size >> XC_PAGE_SHIFT;
-    if (dom->mmio_size == 0)
+    if (dom->mmio_size == 0 && emulated_devices)
         dom->mmio_size = HVM_BELOW_4G_MMIO_LENGTH;
     lowmem_end = mem_size;
     highmem_end = 0;
@@ -992,7 +1012,7 @@ int libxl__build_hvm(libxl__gc *gc, uint32_t domid,
     dom->lowmem_end = lowmem_end;
     dom->highmem_end = highmem_end;
     dom->mmio_start = mmio_start;
-    dom->emulation = true;
+    dom->emulation = emulated_devices;
 
     if (info->num_vnuma_nodes != 0) {
         int i;
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index e1632fa..0283197 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -86,6 +86,7 @@ libxl_device_model_version = Enumeration("device_model_version", [
     (0, "UNKNOWN"),
     (1, "QEMU_XEN_TRADITIONAL"), # Historical qemu-xen device model (qemu-dm)
     (2, "QEMU_XEN"),             # Upstream based qemu-xen device model
+    (3, "NONE"),                 # No device model
     ])
 
 libxl_console_type = Enumeration("console_type", [
diff --git a/tools/libxl/libxl_x86.c b/tools/libxl/libxl_x86.c
index f04433a..435290b 100644
--- a/tools/libxl/libxl_x86.c
+++ b/tools/libxl/libxl_x86.c
@@ -5,7 +5,9 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc,
                                       libxl_domain_config *d_config,
                                       xc_domain_configuration_t *xc_config)
 {
-    if (d_config->c_info.type == LIBXL_DOMAIN_TYPE_HVM)
+    if (d_config->c_info.type == LIBXL_DOMAIN_TYPE_HVM &&
+        d_config->b_info.device_model_version !=
+        LIBXL_DEVICE_MODEL_VERSION_NONE)
         xc_config->emulation_flags = (EMU_LAPIC | EMU_HPET | EMU_PMTIMER |
                                       EMU_RTC | EMU_IOAPIC | EMU_PIC |
                                       EMU_PMU | EMU_VGA | EMU_IOMMU);
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index c858068..3d9b3d4 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -2067,6 +2067,8 @@ skip_vfb:
         } else if (!strcmp(buf, "qemu-xen")) {
             b_info->device_model_version
                 = LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN;
+        } else if (!strcmp(buf, "none")) {
+            b_info->device_model_version = LIBXL_DEVICE_MODEL_VERSION_NONE;
         } else {
             fprintf(stderr,
                     "Unknown device_model_version \"%s\" specified\n", buf);
-- 
1.9.5 (Apple Git-50.3)


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v3 32/32] libxl: allow the creation of HVM domains without a device model.

[Wei Liu]

On Fri, Jul 03, 2015 at 01:35:10PM +0200, Roger Pau Monne wrote:
> Replace the firmware loaded into HVM guests with an OS kernel. Since the HVM
> builder now uses the PV xc_dom_* set of functions this kernel will be parsed
> and loaded inside the guest like on PV, but the container is a pure HVM
> guest.
> 
> Also, if device_model_version is set to none unconditinally set the nic type
> to LIBXL_NIC_TYPE_VIF.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
> Cc: Ian Campbell <ian.campbell@citrix.com>
> Cc: Wei Liu <wei.liu2@citrix.com>
> ---
>  docs/man/xl.cfg.pod.5       |  5 ++++
>  tools/libxl/libxl.c         |  7 +++---
>  tools/libxl/libxl_create.c  | 18 ++++++++++++++
>  tools/libxl/libxl_dom.c     | 60 ++++++++++++++++++++++++++++++---------------
>  tools/libxl/libxl_types.idl |  1 +
>  tools/libxl/libxl_x86.c     |  4 ++-
>  tools/libxl/xl_cmdimpl.c    |  2 ++
>  7 files changed, 72 insertions(+), 25 deletions(-)
> 
> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index a3e0e2e..1270605 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -1631,6 +1631,11 @@ This device-model is the default for Linux dom0.
>  Use the device-model based upon the historical Xen fork of Qemu.
>  This device-model is still the default for NetBSD dom0.
>  
> +=item B<none>
> +
> +Don't use any device model. This requires a kernel capable of booting
> +in this mode.
> +
>  =back
>  
>  It is recommended to accept the default value for new guests.  If
> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> index e9a2d26..a6690cf 100644
> --- a/tools/libxl/libxl.c
> +++ b/tools/libxl/libxl.c
> @@ -1590,11 +1590,10 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
>  
>      switch (libxl__domain_type(gc, domid)) {
>      case LIBXL_DOMAIN_TYPE_HVM:
> -        if (!libxl_get_stubdom_id(CTX, domid))
> -            dm_present = 1;
> -        else
> +        if (libxl_get_stubdom_id(CTX, domid)) {
>              dm_present = 0;
> -        break;
> +            break;
> +        }

There is a path that dm_present contains garbage.

>      case LIBXL_DOMAIN_TYPE_PV:
>          pid = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/image/device-model-pid", domid));
>          dm_present = (pid != NULL);
> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> index f366a09..1444e40 100644
> --- a/tools/libxl/libxl_create.c
> +++ b/tools/libxl/libxl_create.c
> @@ -164,6 +164,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>                  b_info->u.hvm.bios = LIBXL_BIOS_TYPE_ROMBIOS; break;
>              case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
>                  b_info->u.hvm.bios = LIBXL_BIOS_TYPE_SEABIOS; break;
> +            case LIBXL_DEVICE_MODEL_VERSION_NONE:
> +                break;
>              default:return ERROR_INVAL;
>              }
>  
> @@ -177,6 +179,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>              if (b_info->u.hvm.bios == LIBXL_BIOS_TYPE_ROMBIOS)
>                  return ERROR_INVAL;
>              break;
> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
> +            break;
>          default:abort();
>          }
>  
> @@ -278,6 +282,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>                  break;
>              }
>              break;
> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
> +            b_info->video_memkb = 0;
> +            break;
>          case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
>          default:
>              switch (b_info->u.hvm.vga.kind) {
> @@ -945,6 +952,11 @@ static void initiate_domain_create(libxl__egc *egc,
>          ret = libxl__device_nic_setdefault(gc, &d_config->nics[i], domid);
>          if (ret) goto error_out;
>  
> +        /* HVM guests without a device model only have PV nics. */
> +        if (d_config->b_info.device_model_version ==
> +            LIBXL_DEVICE_MODEL_VERSION_NONE)
> +            d_config->nics[i].nictype = LIBXL_NIC_TYPE_VIF;
> +

Better to check if users have asked for emulated NIC and bail.

>          if (d_config->nics[i].devid > last_devid)
>              last_devid = d_config->nics[i].devid;
>      }
> @@ -1293,6 +1305,12 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
>          libxl__device_console_add(gc, domid, &console, state, &device);
>          libxl__device_console_dispose(&console);
>  
> +        if (d_config->b_info.device_model_version ==
> +            LIBXL_DEVICE_MODEL_VERSION_NONE) {
> +            domcreate_devmodel_started(egc, &dcs->dmss.dm, 0);
> +            return;
> +        }
> +
>          libxl_device_vkb_init(&vkb);
>          libxl__device_vkb_add(gc, domid, &vkb);
>          libxl_device_vkb_dispose(&vkb);
> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
> index 6ce4115..59854a1 100644
> --- a/tools/libxl/libxl_dom.c
> +++ b/tools/libxl/libxl_dom.c
> @@ -789,21 +789,23 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid,
>      uint64_t str_mfn, cons_mfn;
>      int i;
>  
> -    va_map = xc_map_foreign_range(handle, domid,
> -                                  XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
> -                                  HVM_INFO_PFN);
> -    if (va_map == NULL)
> -        return -1;
> -
> -    va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
> -    va_hvm->apic_mode = libxl_defbool_val(info->u.hvm.apic);
> -    va_hvm->nr_vcpus = info->max_vcpus;
> -    memset(va_hvm->vcpu_online, 0, sizeof(va_hvm->vcpu_online));
> -    memcpy(va_hvm->vcpu_online, info->avail_vcpus.map, info->avail_vcpus.size);
> -    for (i = 0, sum = 0; i < va_hvm->length; i++)
> -        sum += ((uint8_t *) va_hvm)[i];
> -    va_hvm->checksum -= sum;
> -    munmap(va_map, XC_PAGE_SIZE);
> +    if ( info->device_model_version != LIBXL_DEVICE_MODEL_VERSION_NONE) {

Stray space.

> +        va_map = xc_map_foreign_range(handle, domid,
> +                                      XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
> +                                      HVM_INFO_PFN);
> +        if (va_map == NULL)
> +            return -1;
> +
> +        va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
> +        va_hvm->apic_mode = libxl_defbool_val(info->u.hvm.apic);
> +        va_hvm->nr_vcpus = info->max_vcpus;
> +        memset(va_hvm->vcpu_online, 0, sizeof(va_hvm->vcpu_online));
> +        memcpy(va_hvm->vcpu_online, info->avail_vcpus.map, info->avail_vcpus.size);
> +        for (i = 0, sum = 0; i < va_hvm->length; i++)
> +            sum += ((uint8_t *) va_hvm)[i];
> +        va_hvm->checksum -= sum;
> +        munmap(va_map, XC_PAGE_SIZE);
> +    }
>  
>      xc_hvm_param_get(handle, domid, HVM_PARAM_STORE_PFN, &str_mfn);
>      xc_hvm_param_get(handle, domid, HVM_PARAM_CONSOLE_PFN, &cons_mfn);
> @@ -884,6 +886,12 @@ static int libxl__domain_firmware(libxl__gc *gc,
>          case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
>              firmware = "hvmloader";
>              break;
> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
> +            if (info->kernel == NULL) {
> +                LOG(ERROR, "no device model requested without a kernel");
> +                return ERROR_FAIL;

goto out

Maybe fix the other return as well...

Wei.

Re: [PATCH v3 32/32] libxl: allow the creation of HVM domains without a device model.

[Roger Pau Monné]

El 28/07/15 a les 13.22, Wei Liu ha escrit:
> On Fri, Jul 03, 2015 at 01:35:10PM +0200, Roger Pau Monne wrote:
>> Replace the firmware loaded into HVM guests with an OS kernel. Since the HVM
>> builder now uses the PV xc_dom_* set of functions this kernel will be parsed
>> and loaded inside the guest like on PV, but the container is a pure HVM
>> guest.
>>
>> Also, if device_model_version is set to none unconditinally set the nic type
>> to LIBXL_NIC_TYPE_VIF.
>>
>> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
>> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
>> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
>> Cc: Ian Campbell <ian.campbell@citrix.com>
>> Cc: Wei Liu <wei.liu2@citrix.com>
>> ---
>>  docs/man/xl.cfg.pod.5       |  5 ++++
>>  tools/libxl/libxl.c         |  7 +++---
>>  tools/libxl/libxl_create.c  | 18 ++++++++++++++
>>  tools/libxl/libxl_dom.c     | 60 ++++++++++++++++++++++++++++++---------------
>>  tools/libxl/libxl_types.idl |  1 +
>>  tools/libxl/libxl_x86.c     |  4 ++-
>>  tools/libxl/xl_cmdimpl.c    |  2 ++
>>  7 files changed, 72 insertions(+), 25 deletions(-)
>>
>> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
>> index a3e0e2e..1270605 100644
>> --- a/docs/man/xl.cfg.pod.5
>> +++ b/docs/man/xl.cfg.pod.5
>> @@ -1631,6 +1631,11 @@ This device-model is the default for Linux dom0.
>>  Use the device-model based upon the historical Xen fork of Qemu.
>>  This device-model is still the default for NetBSD dom0.
>>  
>> +=item B<none>
>> +
>> +Don't use any device model. This requires a kernel capable of booting
>> +in this mode.
>> +
>>  =back
>>  
>>  It is recommended to accept the default value for new guests.  If
>> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
>> index e9a2d26..a6690cf 100644
>> --- a/tools/libxl/libxl.c
>> +++ b/tools/libxl/libxl.c
>> @@ -1590,11 +1590,10 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
>>  
>>      switch (libxl__domain_type(gc, domid)) {
>>      case LIBXL_DOMAIN_TYPE_HVM:
>> -        if (!libxl_get_stubdom_id(CTX, domid))
>> -            dm_present = 1;
>> -        else
>> +        if (libxl_get_stubdom_id(CTX, domid)) {
>>              dm_present = 0;
>> -        break;
>> +            break;
>> +        }
> 
> There is a path that dm_present contains garbage.

Please take into account that I've removed the last break in
LIBXL_DOMAIN_TYPE_HVM, so now the LIBXL_DOMAIN_TYPE_HVM case expands
into the LIBXL_DOMAIN_TYPE_PV case.

>>      case LIBXL_DOMAIN_TYPE_PV:
>>          pid = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/image/device-model-pid", domid));
>>          dm_present = (pid != NULL);

dm_present is set here for both PV and HVM guests without a stubdom.

>> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
>> index f366a09..1444e40 100644
>> --- a/tools/libxl/libxl_create.c
>> +++ b/tools/libxl/libxl_create.c
>> @@ -164,6 +164,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>>                  b_info->u.hvm.bios = LIBXL_BIOS_TYPE_ROMBIOS; break;
>>              case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
>>                  b_info->u.hvm.bios = LIBXL_BIOS_TYPE_SEABIOS; break;
>> +            case LIBXL_DEVICE_MODEL_VERSION_NONE:
>> +                break;
>>              default:return ERROR_INVAL;
>>              }
>>  
>> @@ -177,6 +179,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>>              if (b_info->u.hvm.bios == LIBXL_BIOS_TYPE_ROMBIOS)
>>                  return ERROR_INVAL;
>>              break;
>> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
>> +            break;
>>          default:abort();
>>          }
>>  
>> @@ -278,6 +282,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
>>                  break;
>>              }
>>              break;
>> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
>> +            b_info->video_memkb = 0;
>> +            break;
>>          case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
>>          default:
>>              switch (b_info->u.hvm.vga.kind) {
>> @@ -945,6 +952,11 @@ static void initiate_domain_create(libxl__egc *egc,
>>          ret = libxl__device_nic_setdefault(gc, &d_config->nics[i], domid);
>>          if (ret) goto error_out;
>>  
>> +        /* HVM guests without a device model only have PV nics. */
>> +        if (d_config->b_info.device_model_version ==
>> +            LIBXL_DEVICE_MODEL_VERSION_NONE)
>> +            d_config->nics[i].nictype = LIBXL_NIC_TYPE_VIF;
>> +
> 
> Better to check if users have asked for emulated NIC and bail.

IMHO, I think it would be better to move this to
libxl__device_nic_setdefault and I will do so in the next version.

>>          if (d_config->nics[i].devid > last_devid)
>>              last_devid = d_config->nics[i].devid;
>>      }
>> @@ -1293,6 +1305,12 @@ static void domcreate_launch_dm(libxl__egc *egc, libxl__multidev *multidev,
>>          libxl__device_console_add(gc, domid, &console, state, &device);
>>          libxl__device_console_dispose(&console);
>>  
>> +        if (d_config->b_info.device_model_version ==
>> +            LIBXL_DEVICE_MODEL_VERSION_NONE) {
>> +            domcreate_devmodel_started(egc, &dcs->dmss.dm, 0);
>> +            return;
>> +        }
>> +
>>          libxl_device_vkb_init(&vkb);
>>          libxl__device_vkb_add(gc, domid, &vkb);
>>          libxl_device_vkb_dispose(&vkb);
>> diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
>> index 6ce4115..59854a1 100644
>> --- a/tools/libxl/libxl_dom.c
>> +++ b/tools/libxl/libxl_dom.c
>> @@ -789,21 +789,23 @@ static int hvm_build_set_params(xc_interface *handle, uint32_t domid,
>>      uint64_t str_mfn, cons_mfn;
>>      int i;
>>  
>> -    va_map = xc_map_foreign_range(handle, domid,
>> -                                  XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
>> -                                  HVM_INFO_PFN);
>> -    if (va_map == NULL)
>> -        return -1;
>> -
>> -    va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
>> -    va_hvm->apic_mode = libxl_defbool_val(info->u.hvm.apic);
>> -    va_hvm->nr_vcpus = info->max_vcpus;
>> -    memset(va_hvm->vcpu_online, 0, sizeof(va_hvm->vcpu_online));
>> -    memcpy(va_hvm->vcpu_online, info->avail_vcpus.map, info->avail_vcpus.size);
>> -    for (i = 0, sum = 0; i < va_hvm->length; i++)
>> -        sum += ((uint8_t *) va_hvm)[i];
>> -    va_hvm->checksum -= sum;
>> -    munmap(va_map, XC_PAGE_SIZE);
>> +    if ( info->device_model_version != LIBXL_DEVICE_MODEL_VERSION_NONE) {
> 
> Stray space.
> 
>> +        va_map = xc_map_foreign_range(handle, domid,
>> +                                      XC_PAGE_SIZE, PROT_READ | PROT_WRITE,
>> +                                      HVM_INFO_PFN);
>> +        if (va_map == NULL)
>> +            return -1;
>> +
>> +        va_hvm = (struct hvm_info_table *)(va_map + HVM_INFO_OFFSET);
>> +        va_hvm->apic_mode = libxl_defbool_val(info->u.hvm.apic);
>> +        va_hvm->nr_vcpus = info->max_vcpus;
>> +        memset(va_hvm->vcpu_online, 0, sizeof(va_hvm->vcpu_online));
>> +        memcpy(va_hvm->vcpu_online, info->avail_vcpus.map, info->avail_vcpus.size);
>> +        for (i = 0, sum = 0; i < va_hvm->length; i++)
>> +            sum += ((uint8_t *) va_hvm)[i];
>> +        va_hvm->checksum -= sum;
>> +        munmap(va_map, XC_PAGE_SIZE);
>> +    }
>>  
>>      xc_hvm_param_get(handle, domid, HVM_PARAM_STORE_PFN, &str_mfn);
>>      xc_hvm_param_get(handle, domid, HVM_PARAM_CONSOLE_PFN, &cons_mfn);
>> @@ -884,6 +886,12 @@ static int libxl__domain_firmware(libxl__gc *gc,
>>          case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
>>              firmware = "hvmloader";
>>              break;
>> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
>> +            if (info->kernel == NULL) {
>> +                LOG(ERROR, "no device model requested without a kernel");
>> +                return ERROR_FAIL;
> 
> goto out
> 
> Maybe fix the other return as well...

While there I've done a fixup of all the error paths in the function,
because AFAICT some of them were doing goto out without rc set.


Thanks, Roger.

Re: [PATCH v3 32/32] libxl: allow the creation of HVM domains without a device model.

[Wei Liu]

On Wed, Jul 29, 2015 at 04:43:17PM +0200, Roger Pau Monné wrote:
[...]
> >>  
> >>  It is recommended to accept the default value for new guests.  If
> >> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c
> >> index e9a2d26..a6690cf 100644
> >> --- a/tools/libxl/libxl.c
> >> +++ b/tools/libxl/libxl.c
> >> @@ -1590,11 +1590,10 @@ void libxl__destroy_domid(libxl__egc *egc, libxl__destroy_domid_state *dis)
> >>  
> >>      switch (libxl__domain_type(gc, domid)) {
> >>      case LIBXL_DOMAIN_TYPE_HVM:
> >> -        if (!libxl_get_stubdom_id(CTX, domid))
> >> -            dm_present = 1;
> >> -        else
> >> +        if (libxl_get_stubdom_id(CTX, domid)) {
> >>              dm_present = 0;
> >> -        break;
> >> +            break;
> >> +        }
> > 
> > There is a path that dm_present contains garbage.
> 
> Please take into account that I've removed the last break in
> LIBXL_DOMAIN_TYPE_HVM, so now the LIBXL_DOMAIN_TYPE_HVM case expands
> into the LIBXL_DOMAIN_TYPE_PV case.
> 

Oh yes.

Could use /* fall through */ to explicitly state this behaviour to avoid
confusion and make Coverity happy.

> >>      case LIBXL_DOMAIN_TYPE_PV:
> >>          pid = libxl__xs_read(gc, XBT_NULL, libxl__sprintf(gc, "/local/domain/%d/image/device-model-pid", domid));
> >>          dm_present = (pid != NULL);
> 
> dm_present is set here for both PV and HVM guests without a stubdom.
> 
> >> diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
> >> index f366a09..1444e40 100644
> >> --- a/tools/libxl/libxl_create.c
> >> +++ b/tools/libxl/libxl_create.c
> >> @@ -164,6 +164,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
> >>                  b_info->u.hvm.bios = LIBXL_BIOS_TYPE_ROMBIOS; break;
> >>              case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
> >>                  b_info->u.hvm.bios = LIBXL_BIOS_TYPE_SEABIOS; break;
> >> +            case LIBXL_DEVICE_MODEL_VERSION_NONE:
> >> +                break;
> >>              default:return ERROR_INVAL;
> >>              }
> >>  
> >> @@ -177,6 +179,8 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
> >>              if (b_info->u.hvm.bios == LIBXL_BIOS_TYPE_ROMBIOS)
> >>                  return ERROR_INVAL;
> >>              break;
> >> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
> >> +            break;
> >>          default:abort();
> >>          }
> >>  
> >> @@ -278,6 +282,9 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc,
> >>                  break;
> >>              }
> >>              break;
> >> +        case LIBXL_DEVICE_MODEL_VERSION_NONE:
> >> +            b_info->video_memkb = 0;
> >> +            break;
> >>          case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN:
> >>          default:
> >>              switch (b_info->u.hvm.vga.kind) {
> >> @@ -945,6 +952,11 @@ static void initiate_domain_create(libxl__egc *egc,
> >>          ret = libxl__device_nic_setdefault(gc, &d_config->nics[i], domid);
> >>          if (ret) goto error_out;
> >>  
> >> +        /* HVM guests without a device model only have PV nics. */
> >> +        if (d_config->b_info.device_model_version ==
> >> +            LIBXL_DEVICE_MODEL_VERSION_NONE)
> >> +            d_config->nics[i].nictype = LIBXL_NIC_TYPE_VIF;
> >> +
> > 
> > Better to check if users have asked for emulated NIC and bail.
> 
> IMHO, I think it would be better to move this to
> libxl__device_nic_setdefault and I will do so in the next version.

Yes, that's better.

Wei.

Re: [PATCH v3 32/32] libxl: allow the creation of HVM domains without a device model.

[Andrew Cooper]

On 29/07/15 15:43, Roger Pau Monné wrote:
>> There is a path that dm_present contains garbage.
> Please take into account that I've removed the last break in
> LIBXL_DOMAIN_TYPE_HVM, so now the LIBXL_DOMAIN_TYPE_HVM case expands
> into the LIBXL_DOMAIN_TYPE_PV case.

As a general note, please mark deliberate fallthough cases with a
comment, even if it is just /* fallthrough */

This is an aid to Coverity spotting accidental fallthrough.

~Andrew

Re: [PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[David Vrabel]

On 03/07/15 12:34, Roger Pau Monne wrote:
> 
> And for the FreeBSD part:

Have you thought at all about what the Linux support for this mode would
look like?

I started briefly looking today but don't really have time to look into
it properly.  I do think we want to use as much of the native/HVM code
as possible and avoid the PV path (the current Linux PVH code uses too
much of the PV entry point, IMO).

David

Re: [PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[Roger Pau Monné]

El 03/07/15 a les 18.19, David Vrabel ha escrit:
> On 03/07/15 12:34, Roger Pau Monne wrote:
>>
>> And for the FreeBSD part:
> 
> Have you thought at all about what the Linux support for this mode would
> look like?
> 
> I started briefly looking today but don't really have time to look into
> it properly.  I do think we want to use as much of the native/HVM code
> as possible and avoid the PV path (the current Linux PVH code uses too
> much of the PV entry point, IMO).

I'm afraid I'm not that familiar with the native Linux entry point in
order to know how much of that can be reused. Does the amd64 kernel
contain a 32bit entry point or the kernel is expected to be launched in
long mode with some page tables already setup?

Iff Linux on i386/amd64 already contains a 32bit protected mode without
paging entry point then I guess most of the code can be reused.

Note that FreeBSD/amd64 didn't have any of this (the kernel is launched
in long mode with some initial page tables setup by the loader), so the
code needed for Linux should be similar to the one on FreeBSD or even
less if there's an entry point that we can reuse:

http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=blob;f=sys/amd64/amd64/xen-locore32.S;h=fd73d150009ab21059d4df219e6a62cc40d5994e;hb=refs/heads/new_entry_point_v2

Roger.

Re: [PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[Konrad Rzeszutek Wilk]

On Fri, Jul 03, 2015 at 06:36:37PM +0200, Roger Pau Monné wrote:
> El 03/07/15 a les 18.19, David Vrabel ha escrit:
> > On 03/07/15 12:34, Roger Pau Monne wrote:
> >>
> >> And for the FreeBSD part:
> > 
> > Have you thought at all about what the Linux support for this mode would
> > look like?
> > 
> > I started briefly looking today but don't really have time to look into
> > it properly.  I do think we want to use as much of the native/HVM code
> > as possible and avoid the PV path (the current Linux PVH code uses too
> > much of the PV entry point, IMO).
> 
> I'm afraid I'm not that familiar with the native Linux entry point in
> order to know how much of that can be reused. Does the amd64 kernel
> contain a 32bit entry point or the kernel is expected to be launched in
> long mode with some page tables already setup?

It does : arch/x86/kernel/head_64[32].S:startup_32

The problem is that it depends to have the boot_params (a Linux structure)
filled out.

Which then calls 'default_entry' [32-bit] or 'startup_64', and
then it goes off somewhere (there is a lot of EFI code in there).

Anyhow it eventually ends up:

 x86_64_start_kernel -> x86_64_start_reservations
 
But the problem is that during this execution none of the pvops get
set to the Xen ones (which we need to tell the kernel that there is
almost no hardware).

One could change initial_code to some xen_x86_64_start_kernel that
would be similar to xen_startup - in that it would set up the
basic pvops structures and then call x86_64_start_kernel.

In essence something like:
 startup_32
   \-> startup_64 [in compressed/head_64.S]
      \-> startup_64 [ head_64.S]
         \-> secondary_startup_64
            \-> jmpq *initial_code [which is by default x86_64_start_kernel]

         
[if I got the assmebler call chain correctly figured out] 
> Iff Linux on i386/amd64 already contains a 32bit protected mode without
> paging entry point then I guess most of the code can be reused.

Yes except that we would have to have before then to fill out the
boot_params to get use going, setup the initial stack.

The during the bootup we would need to swap over to an Xen entry
point so that it can neuter most of the pvops calls.

> 
> Note that FreeBSD/amd64 didn't have any of this (the kernel is launched
> in long mode with some initial page tables setup by the loader), so the
> code needed for Linux should be similar to the one on FreeBSD or even
> less if there's an entry point that we can reuse:a


The Linux kernel can also start in 32/64-bit mode with page tables
(thanks to EFI)
> 
> http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=blob;f=sys/amd64/amd64/xen-locore32.S;h=fd73d150009ab21059d4df219e6a62cc40d5994e;hb=refs/heads/new_entry_point_v2
> 
> Roger.
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v3 00/32] Introduce HVM without dm and new boot ABI

[Konrad Rzeszutek Wilk]

> > > I started briefly looking today but don't really have time to look into
> > > it properly.  I do think we want to use as much of the native/HVM code
> > > as possible and avoid the PV path (the current Linux PVH code uses too
> > > much of the PV entry point, IMO).
> > 
> > I'm afraid I'm not that familiar with the native Linux entry point in
> > order to know how much of that can be reused. Does the amd64 kernel
> > contain a 32bit entry point or the kernel is expected to be launched in
> > long mode with some page tables already setup?
> 
> It does : arch/x86/kernel/head_64[32].S:startup_32
> 
> The problem is that it depends to have the boot_params (a Linux structure)
> filled out.
> 
> Which then calls 'default_entry' [32-bit] or 'startup_64', and
> then it goes off somewhere (there is a lot of EFI code in there).
> 
> Anyhow it eventually ends up:
> 
>  x86_64_start_kernel -> x86_64_start_reservations
>  
> But the problem is that during this execution none of the pvops get
> set to the Xen ones (which we need to tell the kernel that there is
> almost no hardware).
> 
> One could change initial_code to some xen_x86_64_start_kernel that
> would be similar to xen_startup - in that it would set up the
> basic pvops structures and then call x86_64_start_kernel.
> 
> In essence something like:
>  startup_32
>    \-> startup_64 [in compressed/head_64.S]
>       \-> startup_64 [ head_64.S]
>          \-> secondary_startup_64
>             \-> jmpq *initial_code [which is by default x86_64_start_kernel]
                   \-> x86_64_start_reservations

> 
>          
> [if I got the assmebler call chain correctly figured out] 

Just to give an comparison, the PV entry point is:

 xen_start_kernel [copies appropiate things to boot_parms - such as command line, makes
     tons of hypercalls to set appropiate tweaks in boot_params]
   \-> x86_64_start_reservations

If we were to hook up to the above one, we would need:

xen_no_paging_start_kernel [copy minimum amount of things we need on the bootparams
    to keep on going, sets initial_code to our xen_secondary_start]
   \- startup_32
	\-> startup_64
            \-> secondary_startup_64
		 -> jmpq *initial_code
			\-> xen_secondary_start [set pvops, copies rest of bootparams,
                                                 etc]
				\-> x86_64_start_kernel
					\-> x86_64_start_reservation

If we were to borrow from what Roger has [http://xenbits.xen.org/gitweb/?p=people/royger/freebsd.git;a=blob;f=sys/amd64/amd64/xen-locore32.S;h=fd73d150009ab21059d4df219e6a62cc40d5994e;hb=refs/heads/new_entry_point_v2]
we would do:

xen_no_paging_start_kernel [xen_start32 copy of FreeBSD]
  \-> xen_secondary_start [sets pvops, copies rest of params, etc]
    \-> x86_64_start_reservation

But that to me looks quite similar to the PV path, except more code, and
also x86 folks asking us why we aren't using the baremetal boot paths.

We could omit some of this if the toolstack/hyprecall knew the Linux
boot_params similar to how GRUB2 knows it. And just populate that.
But then if we are doing that, why not just go the simple route
and use the Xen PV entry point.