[PATCH v2 0/3] Xen VMware tools support

[PATCH v2 0/3] Xen VMware tools support

[Don Slutz]

This is a small part of the changes needed to allow running Linux
and windows (and others) guests that were built on VMware and run
run them unchanged on Xen.

This small part is the start of Xen support of VMware backdoor I/O
port which is how VMware tools (a standard addition installed on a
guest) communicates to the hypervisor.

I picked this subset to start with because it only has changes in
Xen.

Some of this code is already in QEMU and so KVM has some of this
already.  QEMU supported backdoor commands include VMware mouse
support.  A later patch set exists that links these changes, new
code and Xen changes to QEMU to provide VMware mouse support under
Xen.  The important part is that VMware mouse is an absolute
position mouse and so network delays do not effect usage of the
virtual mouse.

For example from the guest:

[root@C63-min-tools ~]# vmtoolsd --cmd "info-get guestinfo.joejoel"
No value found
[root@C63-min-tools ~]# vmtoolsd --cmd "info-set guestinfo.joejoel short"

[root@C63-min-tools ~]# vmtoolsd --cmd "info-get guestinfo.joejoel"
short
[root@C63-min-tools ~]# vmtoolsd --cmd "info-set guestinfo.joejoel long222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222222200000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000joel"

[root@C63-min-tools ~]# vmtoolsd --cmd "info-get guestinfo.key1"
data1
[root@C63-min-tools ~]# vmtoolsd --cmd "info-get guestinfo.key2"
No value found
[root@C63-min-tools ~]# vmtoolsd --cmd "info-get guestinfo.key2"
data2
[root@C63-min-tools ~]# 


Most of this code has been reverse engineered by looking at
source code for Linux and open VMware tools.

http://open-vm-tools.sourceforge.net


changes RFC to v2:

Jan Beulich:
  Add xen/arch/x86/hvm/vmware.c for cpuid_vmware_leaves
  Fewer patches

Andrew Cooper:
  use the proper constant for apic_khz
  Follow 839b966e3f587bbb1a0d954230fb3904330dccb6 style changes.
  Changed HVM_PARAM_VMWARE_HW to write once (make is_vmware_domain()
    more static).
  Dropped vmport status stuff.
  Added checks for xzalloc() having failed.
  You should include backdoor_def.h ...
     Every thing I tried did not work better.  So I did not
     change VMPORT_PORT and BDOOR_PORT being the same value.
     I did not try and adjust VMware's include file backdoor_def.h
     to working in other xen source files.
  Switching to s_time_t is not valid. get_sec() is defined:
    unsigned long get_sec(void);
  and so my uses of it should be using unsigned long.  However
  since that is not a fixed width type, I used the uint64_t
  data type which is almost the same, but does allow the 32 bit
  build of libxc, libxl to do the correct thing.


Konrad Rzeszutek Wilk:
  Please don't include the address. It should be, etc
      about the Vmware provided include files.
    I went with no changes to these files.  Even if the files should
    be changed to match xen coding style, etc I still feel that the
    original ones should be added via a patch, and then adjusted in a
    2nd patch.
  Can you use XenBus?
    I would say no.  XenBus (and XenStore) is about domain to domain
    communication.  This is about VMware's hyper-call and providing
    access to VMware's guest info very low speed access.

Olaf Hering:
   Dropped changing of bios-strings.  Still needs some documentation
   about this may be needed to do in a tool stack or set of commands.


Boris Ostrovsky:
  Use svm_nextrip_insn_length()
    Looks like __get_instruction_length() does this, so switched to
    __get_instruction_length().
 
RFC:

See

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009458

for info on detecting VMware.

Linux does not follow this exactly.  It checks for CPUID 1st.  If
that fails, it checks for SMBIOS containing "VMware" (not VMware- or
VMW).

So this patch set provides:

        SMBIOS -- Add string VMware-
        CPUID -- Add VMware's CPUID (Note: currently HyperV (viridian support) breaks this check.)
        Add the magic VMware port
            Allow VMware tools poweroff and reboot
            Enable access to VMware's guest info
            Provide the VMware tools build number


Don Slutz (3):
  Add vmware_hw to xl.cfg
  vmport: Add VMware provided include files.
  Add limited support of VMware's hyper-call

 docs/man/xl.cfg.pod.5                        |    6 +
 tools/firmware/hvmloader/pci.c               |   10 +-
 tools/libxc/xc_domain.c                      |  115 +++
 tools/libxc/xc_domain_restore.c              |   14 +
 tools/libxc/xc_domain_save.c                 |   11 +
 tools/libxc/xenctrl.h                        |   24 +
 tools/libxc/xg_save_restore.h                |    2 +
 tools/libxl/libxl_create.c                   |    4 +-
 tools/libxl/libxl_dm.c                       |   33 +-
 tools/libxl/libxl_dom.c                      |    2 +
 tools/libxl/libxl_types.idl                  |    1 +
 tools/libxl/xl_cmdimpl.c                     |    2 +
 tools/misc/xen-hvmctx.c                      |  229 ++++
 tools/xentrace/formats                       |    8 +
 xen/arch/x86/domctl.c                        |   34 +
 xen/arch/x86/hvm/Makefile                    |    4 +-
 xen/arch/x86/hvm/hvm.c                       |  499 ++++++++-
 xen/arch/x86/hvm/io.c                        |    1 +
 xen/arch/x86/hvm/svm/emulate.c               |    3 +
 xen/arch/x86/hvm/svm/svm.c                   |   79 ++
 xen/arch/x86/hvm/svm/vmcb.c                  |    1 +
 xen/arch/x86/hvm/vmport/Makefile             |    1 +
 xen/arch/x86/hvm/vmport/backdoor_def.h       |  167 +++
 xen/arch/x86/hvm/vmport/guest_msg_def.h      |   87 ++
 xen/arch/x86/hvm/vmport/includeCheck.h       |   17 +
 xen/arch/x86/hvm/vmport/vmport.c             | 1436 ++++++++++++++++++++++++++
 xen/arch/x86/hvm/vmware.c                    |   85 ++
 xen/arch/x86/hvm/vmx/vmcs.c                  |    1 +
 xen/arch/x86/hvm/vmx/vmx.c                   |   90 ++
 xen/arch/x86/hvm/vmx/vvmx.c                  |   14 +
 xen/arch/x86/traps.c                         |    7 +-
 xen/include/asm-x86/hvm/domain.h             |    5 +
 xen/include/asm-x86/hvm/hvm.h                |    3 +
 xen/include/asm-x86/hvm/svm/emulate.h        |    1 +
 xen/include/asm-x86/hvm/trace.h              |   31 +
 xen/include/asm-x86/hvm/vmport.h             |   90 ++
 xen/include/asm-x86/hvm/vmware.h             |   32 +
 xen/include/public/arch-x86/hvm/save.h       |   39 +-
 xen/include/public/arch-x86/hvm/vmporttype.h |  105 ++
 xen/include/public/domctl.h                  |    3 +
 xen/include/public/hvm/hvm_op.h              |   18 +
 xen/include/public/hvm/params.h              |    8 +-
 xen/include/public/trace.h                   |    7 +
 43 files changed, 3309 insertions(+), 20 deletions(-)
 create mode 100644 xen/arch/x86/hvm/vmport/Makefile
 create mode 100644 xen/arch/x86/hvm/vmport/backdoor_def.h
 create mode 100644 xen/arch/x86/hvm/vmport/guest_msg_def.h
 create mode 100644 xen/arch/x86/hvm/vmport/includeCheck.h
 create mode 100644 xen/arch/x86/hvm/vmport/vmport.c
 create mode 100644 xen/arch/x86/hvm/vmware.c
 create mode 100644 xen/include/asm-x86/hvm/vmport.h
 create mode 100644 xen/include/asm-x86/hvm/vmware.h
 create mode 100644 xen/include/public/arch-x86/hvm/vmporttype.h

-- 
1.8.4

[PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

If non-zero then
  Return VMware's cpuid leaves.
  Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
  Force use of VMware's VGA in QEMU.

The support of hypervisor cpuid leaves has not been agreed to.

MicroSoft Hyper-V (AKA viridian) currently must be at 0x40000000.

VMware currently must be at 0x40000000.

KVM currently must be at 0x40000000 (from Seabios).

Seabios will find xen at 0x40000000, 0x40000100, 0x40000200 ..
0x40010000.

http://download.microsoft.com/download/F/B/0/FB0D01A3-8E3A-4F5F-AA59-08C8026D3B8A/requirements-for-implementing-microsoft-hypervisor-interface.docx

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009458

http://lwn.net/Articles/301888/
  Attempted to get this cleaned up.

So based on this, I picked the order:

0x40000000 is viridian, vmware or xen
0x40000100 is vmware or xen
0x40000200 is xen

Signed-off-by: Don Slutz <dslutz@verizon.com>
---
 docs/man/xl.cfg.pod.5            |  6 +++
 tools/firmware/hvmloader/pci.c   | 10 ++++-
 tools/libxc/xc_domain_restore.c  | 14 +++++++
 tools/libxc/xc_domain_save.c     | 11 ++++++
 tools/libxc/xg_save_restore.h    |  2 +
 tools/libxl/libxl_create.c       |  4 +-
 tools/libxl/libxl_dm.c           | 33 ++++++++++------
 tools/libxl/libxl_dom.c          |  2 +
 tools/libxl/libxl_types.idl      |  1 +
 tools/libxl/xl_cmdimpl.c         |  2 +
 xen/arch/x86/hvm/Makefile        |  3 +-
 xen/arch/x86/hvm/hvm.c           | 19 +++++++++
 xen/arch/x86/hvm/vmware.c        | 85 ++++++++++++++++++++++++++++++++++++++++
 xen/arch/x86/traps.c             |  7 +++-
 xen/include/asm-x86/hvm/domain.h |  1 +
 xen/include/asm-x86/hvm/hvm.h    |  3 ++
 xen/include/asm-x86/hvm/vmware.h | 32 +++++++++++++++
 xen/include/public/hvm/params.h  |  5 ++-
 18 files changed, 222 insertions(+), 18 deletions(-)
 create mode 100644 xen/arch/x86/hvm/vmware.c
 create mode 100644 xen/include/asm-x86/hvm/vmware.h

diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
index f1fc906..34fb021 100644
--- a/docs/man/xl.cfg.pod.5
+++ b/docs/man/xl.cfg.pod.5
@@ -1139,6 +1139,12 @@ some other Operating Systems and in some circumstance can prevent
 Xen's own paravirtualisation interfaces for HVM guests from being
 used.
 
+=item B<vmware_hw=NUMBER>
+
+Turns on or off the exposure of VMware cpuid.  The number is the
+VMware's hardware version number, where 0 is off.  If on it also
+forces the use of VMware's VGA in QEMU.
+
 =back
 
 =head3 Emulated VGA Graphics Device
diff --git a/tools/firmware/hvmloader/pci.c b/tools/firmware/hvmloader/pci.c
index 3712988..f6eca89 100644
--- a/tools/firmware/hvmloader/pci.c
+++ b/tools/firmware/hvmloader/pci.c
@@ -79,12 +79,15 @@ void pci_setup(void)
      * option that will have the least impact.
      */
     bool allow_memory_relocate = 1;
+    int vmware_hw;
 
     s = xenstore_read(HVM_XS_ALLOW_MEMORY_RELOCATE, NULL);
     if ( s )
         allow_memory_relocate = strtoll(s, NULL, 0);
     printf("Relocating guest memory for lowmem MMIO space %s\n",
            allow_memory_relocate?"enabled":"disabled");
+    s = xenstore_read("platform/vmware_hw", "0");
+    vmware_hw = strtoll(s, NULL, 0);
 
     /* Program PCI-ISA bridge with appropriate link routes. */
     isa_irq = 0;
@@ -149,8 +152,11 @@ void pci_setup(void)
             pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
             pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
             pci_writew(devfn, 0x22, 0x0000);
-            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
-            pci_writew(devfn, 0x3d, 0x0001);
+            if ( !vmware_hw )
+            {
+                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
+                pci_writew(devfn, 0x3d, 0x0001);
+            }
             pci_writel(devfn, 0x40, ACPI_PM1A_EVT_BLK_ADDRESS_V1 | 1);
             pci_writeb(devfn, 0x80, 0x01); /* enable PM io space */
             break;
diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
index b9a56d5..bc5cd57 100644
--- a/tools/libxc/xc_domain_restore.c
+++ b/tools/libxc/xc_domain_restore.c
@@ -743,6 +743,7 @@ typedef struct {
     uint64_t vm_generationid_addr;
     uint64_t ioreq_server_pfn;
     uint64_t nr_ioreq_server_pages;
+    uint64_t vmware_hw;
 
     struct toolstack_data_t tdata;
 } pagebuf_t;
@@ -927,6 +928,16 @@ static int pagebuf_get_one(xc_interface *xch, struct restore_ctx *ctx,
         }
         return pagebuf_get_one(xch, ctx, buf, fd, dom);
 
+    case XC_SAVE_ID_HVM_VMWARE_HW:
+        /* Skip padding 4 bytes then read the vmware hw version. */
+        if ( RDEXACT(fd, &buf->vmware_hw, sizeof(uint32_t)) ||
+             RDEXACT(fd, &buf->vmware_hw, sizeof(uint64_t)) )
+        {
+            PERROR("error read the vmware_hw value");
+            return -1;
+        }
+        return pagebuf_get_one(xch, ctx, buf, fd, dom);
+
     case XC_SAVE_ID_TOOLSTACK:
         {
             if ( RDEXACT(fd, &buf->tdata.len, sizeof(buf->tdata.len)) )
@@ -1760,6 +1771,9 @@ int xc_domain_restore(xc_interface *xch, int io_fd, uint32_t dom,
         }
     }
 
+    if (pagebuf.vmware_hw != 0)
+        xc_set_hvm_param(xch, dom, HVM_PARAM_VMWARE_HW, pagebuf.vmware_hw);
+
     if (pagebuf.acpi_ioport_location == 1) {
         DBGPRINTF("Use new firmware ioport from the checkpoint\n");
         xc_hvm_param_set(xch, dom, HVM_PARAM_ACPI_IOPORTS_LOCATION, 1);
diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
index 254fdb3..76dc307 100644
--- a/tools/libxc/xc_domain_save.c
+++ b/tools/libxc/xc_domain_save.c
@@ -1750,6 +1750,17 @@ int xc_domain_save(xc_interface *xch, int io_fd, uint32_t dom, uint32_t max_iter
             PERROR("Error when writing the ioreq server gmfn count");
             goto out;
         }
+
+        chunk.id = XC_SAVE_ID_HVM_VMWARE_HW;
+        chunk.data = 0;
+        xc_hvm_param_get(xch, dom, HVM_PARAM_VMWARE_HW, &chunk.data);
+
+        if ( (chunk.data != 0) &&
+             wrexact(io_fd, &chunk, sizeof(chunk)) )
+        {
+            PERROR("Error when writing the vmware_hw value");
+            goto out;
+        }
     }
 
     if ( callbacks != NULL && callbacks->toolstack_save != NULL )
diff --git a/tools/libxc/xg_save_restore.h b/tools/libxc/xg_save_restore.h
index bdd9009..d185ba9 100644
--- a/tools/libxc/xg_save_restore.h
+++ b/tools/libxc/xg_save_restore.h
@@ -262,6 +262,8 @@
 /* These are a pair; it is an error for one to exist without the other */
 #define XC_SAVE_ID_HVM_IOREQ_SERVER_PFN -19
 #define XC_SAVE_ID_HVM_NR_IOREQ_SERVER_PAGES -20
+/* VMware data */
+#define XC_SAVE_ID_HVM_VMWARE_HW      -21
 
 /*
 ** We process save/restore/migrate in batches of pages; the below
diff --git a/tools/libxl/libxl_create.c b/tools/libxl/libxl_create.c
index fc332ef..238fa24 100644
--- a/tools/libxl/libxl_create.c
+++ b/tools/libxl/libxl_create.c
@@ -424,13 +424,15 @@ int libxl__domain_build(libxl__gc *gc,
         vments[4] = "start_time";
         vments[5] = libxl__sprintf(gc, "%lu.%02d", start_time.tv_sec,(int)start_time.tv_usec/10000);
 
-        localents = libxl__calloc(gc, 7, sizeof(char *));
+        localents = libxl__calloc(gc, 9, sizeof(char *));
         localents[0] = "platform/acpi";
         localents[1] = libxl_defbool_val(info->u.hvm.acpi) ? "1" : "0";
         localents[2] = "platform/acpi_s3";
         localents[3] = libxl_defbool_val(info->u.hvm.acpi_s3) ? "1" : "0";
         localents[4] = "platform/acpi_s4";
         localents[5] = libxl_defbool_val(info->u.hvm.acpi_s4) ? "1" : "0";
+        localents[6] = "platform/vmware_hw";
+        localents[7] = libxl__sprintf(gc, "%"PRId64, info->u.hvm.vmware_hw);
 
         break;
     case LIBXL_DOMAIN_TYPE_PV:
diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
index 103cbca..c79274b 100644
--- a/tools/libxl/libxl_dm.c
+++ b/tools/libxl/libxl_dm.c
@@ -542,19 +542,28 @@ static char ** libxl__build_device_model_args_new(libxl__gc *gc,
             }
         }
 
-        switch (b_info->u.hvm.vga.kind) {
-        case LIBXL_VGA_INTERFACE_TYPE_STD:
-            flexarray_append_pair(dm_args, "-device",
-                GCSPRINTF("VGA,vgamem_mb=%d",
-                libxl__sizekb_to_mb(b_info->video_memkb)));
-            break;
-        case LIBXL_VGA_INTERFACE_TYPE_CIRRUS:
+        if (b_info->u.hvm.vmware_hw) {
             flexarray_append_pair(dm_args, "-device",
-                GCSPRINTF("cirrus-vga,vgamem_mb=%d",
-                libxl__sizekb_to_mb(b_info->video_memkb)));
-            break;
-        case LIBXL_VGA_INTERFACE_TYPE_NONE:
-            break;
+                                  GCSPRINTF("vmware-svga,vgamem_mb=%d",
+                                            libxl__sizekb_to_mb(
+                                                b_info->video_memkb)));
+        } else {
+            switch (b_info->u.hvm.vga.kind) {
+            case LIBXL_VGA_INTERFACE_TYPE_STD:
+                flexarray_append_pair(dm_args, "-device",
+                                      GCSPRINTF("VGA,vgamem_mb=%d",
+                                                libxl__sizekb_to_mb(
+                                                    b_info->video_memkb)));
+                break;
+            case LIBXL_VGA_INTERFACE_TYPE_CIRRUS:
+                flexarray_append_pair(dm_args, "-device",
+                                      GCSPRINTF("cirrus-vga,vgamem_mb=%d",
+                                                libxl__sizekb_to_mb(
+                                                    b_info->video_memkb)));
+                break;
+            case LIBXL_VGA_INTERFACE_TYPE_NONE:
+                break;
+            }
         }
 
         if (b_info->u.hvm.boot) {
diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c
index c944804..63ae4c5 100644
--- a/tools/libxl/libxl_dom.c
+++ b/tools/libxl/libxl_dom.c
@@ -219,6 +219,8 @@ static void hvm_set_conf_params(xc_interface *handle, uint32_t domid,
                     libxl_defbool_val(info->u.hvm.viridian));
     xc_hvm_param_set(handle, domid, HVM_PARAM_HPET_ENABLED,
                     libxl_defbool_val(info->u.hvm.hpet));
+    xc_set_hvm_param(handle, domid, HVM_PARAM_VMWARE_HW,
+                     info->u.hvm.vmware_hw);
 #endif
     xc_hvm_param_set(handle, domid, HVM_PARAM_TIMER_MODE, timer_mode(info));
     xc_hvm_param_set(handle, domid, HVM_PARAM_VPT_ALIGN,
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 08a7927..e03f46d 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -371,6 +371,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
                                        ("timeoffset",       string),
                                        ("hpet",             libxl_defbool),
                                        ("vpt_align",        libxl_defbool),
+                                       ("vmware_hw",        UInt(64, init_val = 0)),
                                        ("timer_mode",       libxl_timer_mode),
                                        ("nested_hvm",       libxl_defbool),
                                        ("smbios_firmware",  string),
diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c
index 409a795..9b1fd2f 100644
--- a/tools/libxl/xl_cmdimpl.c
+++ b/tools/libxl/xl_cmdimpl.c
@@ -1033,6 +1033,8 @@ static void parse_config_data(const char *config_source,
         xlu_cfg_get_defbool(config, "acpi_s4", &b_info->u.hvm.acpi_s4, 0);
         xlu_cfg_get_defbool(config, "nx", &b_info->u.hvm.nx, 0);
         xlu_cfg_get_defbool(config, "viridian", &b_info->u.hvm.viridian, 0);
+        if (!xlu_cfg_get_long(config, "vmware_hw",  &l, 1))
+            b_info->u.hvm.vmware_hw = l;
         xlu_cfg_get_defbool(config, "hpet", &b_info->u.hvm.hpet, 0);
         xlu_cfg_get_defbool(config, "vpt_align", &b_info->u.hvm.vpt_align, 0);
 
diff --git a/xen/arch/x86/hvm/Makefile b/xen/arch/x86/hvm/Makefile
index eea5555..3474a65 100644
--- a/xen/arch/x86/hvm/Makefile
+++ b/xen/arch/x86/hvm/Makefile
@@ -18,8 +18,9 @@ obj-y += save.o
 obj-y += stdvga.o
 obj-y += vioapic.o
 obj-y += viridian.o
+obj-y += vmware.o
 obj-y += vlapic.o
 obj-y += vmsi.o
 obj-y += vpic.o
 obj-y += vpt.o
-obj-y += vpmu.o
\ No newline at end of file
+obj-y += vpmu.o
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 83e6fae..eded446 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4228,6 +4228,9 @@ void hvm_cpuid(unsigned int input, unsigned int *eax, unsigned int *ebx,
     if ( cpuid_viridian_leaves(input, eax, ebx, ecx, edx) )
         return;
 
+    if ( cpuid_vmware_leaves(input, count, eax, ebx, ecx, edx) )
+        return;
+
     if ( cpuid_hypervisor_leaves(input, count, eax, ebx, ecx, edx) )
         return;
 
@@ -5692,6 +5695,22 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
 
                 break;
             }
+            case HVM_PARAM_VMWARE_HW:
+                /*
+                 * This should only ever be set non-zero one time by
+                 * the tools and is read only by the guest.
+                 */
+                if ( d == current->domain )
+                {
+                    rc = -EPERM;
+                    break;
+                }
+                if ( d->arch.hvm_domain.params[HVM_PARAM_VMWARE_HW] )
+                {
+                    rc = -EEXIST;
+                    break;
+                }
+                break;
             }
 
             if ( rc == 0 ) 
diff --git a/xen/arch/x86/hvm/vmware.c b/xen/arch/x86/hvm/vmware.c
new file mode 100644
index 0000000..9ca41fa
--- /dev/null
+++ b/xen/arch/x86/hvm/vmware.c
@@ -0,0 +1,85 @@
+/*
+ * arch/x86/hvm/vmware.c
+ *
+ * Copyright (C) 2012 Verizon Corporation
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License Version 2 (GPLv2)
+ * as published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details. <http://www.gnu.org/licenses/>.
+ */
+
+#include <xen/sched.h>
+#include <xen/version.h>
+#include <xen/perfc.h>
+#include <xen/hypercall.h>
+#include <xen/domain_page.h>
+#include <asm/paging.h>
+#include <asm/p2m.h>
+#include <asm/apic.h>
+#include <asm/hvm/support.h>
+
+#include <public/xen.h>
+#include <public/sched.h>
+#include <public/hvm/params.h>
+
+#include <asm/hvm/hvm.h>
+#include <asm/hvm/vlapic.h>
+#include <asm/hvm/vmware.h>
+
+int cpuid_vmware_leaves(uint32_t idx, uint32_t sub_idx,
+                        uint32_t *eax, uint32_t *ebx,
+                        uint32_t *ecx, uint32_t *edx)
+{
+    struct domain *d = current->domain;
+    /* Optionally shift out of the way of Viridian architectural leaves. */
+    uint32_t base = is_viridian_domain(d) ? 0x40000100 : 0x40000000;
+
+    if ( !is_vmware_domain(d) )
+        return 0;
+
+    switch ( idx - base )
+    {
+    case 0x0:
+        *eax = base + 0x10; /* Largest leaf */
+        *ebx = 0x61774d56;  /* "VMwa" */
+        *ecx = 0x4d566572;  /* "reVM" */
+        *edx = 0x65726177;  /* "ware" */
+        break;
+
+    case 0x1 ... 0xf:
+        *eax = 0;          /* Reserved */
+        *ebx = 0;          /* Reserved */
+        *ecx = 0;          /* Reserved */
+        *edx = 0;          /* Reserved */
+        break;
+
+    case 0x10:
+        /* (Virtual) TSC frequency in kHz. */
+        *eax =  d->arch.tsc_khz;
+        /* (Virtual) Bus (local apic timer) frequency in kHz. */
+        *ebx = 1000000000ull / APIC_BUS_CYCLE_NS / 1000ull;
+        *ecx = 0;          /* Reserved */
+        *edx = 0;          /* Reserved */
+        break;
+
+    default:
+        return 0;
+    }
+
+    return 1;
+}
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 7f5306f..3c18fe1 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -685,9 +685,14 @@ int cpuid_hypervisor_leaves( uint32_t idx, uint32_t sub_idx,
 {
     struct domain *d = current->domain;
     /* Optionally shift out of the way of Viridian architectural leaves. */
-    uint32_t base = is_viridian_domain(d) ? 0x40000100 : 0x40000000;
+    uint32_t base = 0x40000000;
     uint32_t limit, dummy;
 
+    if ( is_viridian_domain(d) )
+        base += 0x100;
+    if ( is_vmware_domain(d) )
+        base += 0x100;
+
     idx -= base;
     if ( idx > XEN_CPUID_MAX_NUM_LEAVES )
         return 0; /* Avoid unnecessary pass through domain_cpuid() */
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 291a2e0..526b5da 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -29,6 +29,7 @@
 #include <asm/hvm/io.h>
 #include <xen/hvm/iommu.h>
 #include <asm/hvm/viridian.h>
+#include <asm/hvm/vmware.h>
 #include <asm/hvm/vmx/vmcs.h>
 #include <asm/hvm/svm/vmcb.h>
 #include <public/grant_table.h>
diff --git a/xen/include/asm-x86/hvm/hvm.h b/xen/include/asm-x86/hvm/hvm.h
index 1123857..546210a 100644
--- a/xen/include/asm-x86/hvm/hvm.h
+++ b/xen/include/asm-x86/hvm/hvm.h
@@ -347,6 +347,9 @@ static inline unsigned long hvm_get_shadow_gs_base(struct vcpu *v)
 #define is_viridian_domain(_d)                                             \
  (is_hvm_domain(_d) && ((_d)->arch.hvm_domain.params[HVM_PARAM_VIRIDIAN]))
 
+#define is_vmware_domain(_d)                                             \
+ (is_hvm_domain(_d) && ((_d)->arch.hvm_domain.params[HVM_PARAM_VMWARE_HW]))
+
 void hvm_hypervisor_cpuid_leaf(uint32_t sub_idx,
                                uint32_t *eax, uint32_t *ebx,
                                uint32_t *ecx, uint32_t *edx);
diff --git a/xen/include/asm-x86/hvm/vmware.h b/xen/include/asm-x86/hvm/vmware.h
new file mode 100644
index 0000000..a214349
--- /dev/null
+++ b/xen/include/asm-x86/hvm/vmware.h
@@ -0,0 +1,32 @@
+/*
+ * asm-x86/hvm/vmware.h
+ *
+ * Copyright (C) 2012 Verizon Corporation
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License Version 2 (GPLv2)
+ * as published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details. <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_X86_HVM_VMWARE_H__
+#define __ASM_X86_HVM_VMWARE_H__
+
+int cpuid_vmware_leaves(uint32_t idx, uint32_t sub_idx,
+                        uint32_t *eax, uint32_t *ebx,
+                        uint32_t *ecx, uint32_t *edx);
+
+#endif /* __ASM_X86_HVM_VMWARE_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index 614ff5f..dee6d68 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -151,6 +151,9 @@
 /* Location of the VM Generation ID in guest physical address space. */
 #define HVM_PARAM_VM_GENERATION_ID_ADDR 34
 
-#define HVM_NR_PARAMS          35
+/* Params for VMware */
+#define HVM_PARAM_VMWARE_HW                 35
+
+#define HVM_NR_PARAMS          36
 
 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
-- 
1.8.4

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Jan Beulich]

>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> If non-zero then
>   Return VMware's cpuid leaves.
>   Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
>   Force use of VMware's VGA in QEMU.
> 
> The support of hypervisor cpuid leaves has not been agreed to.
> 
> MicroSoft Hyper-V (AKA viridian) currently must be at 0x40000000.
> 
> VMware currently must be at 0x40000000.
> 
> KVM currently must be at 0x40000000 (from Seabios).
> 
> Seabios will find xen at 0x40000000, 0x40000100, 0x40000200 ..
> 0x40010000.
> 
> http://download.microsoft.com/download/F/B/0/FB0D01A3-8E3A-4F5F-AA59-08C8026D3B8 
> A/requirements-for-implementing-microsoft-hypervisor-interface.docx
> 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=disp 
> layKC&externalId=1009458
> 
> http://lwn.net/Articles/301888/ 
>   Attempted to get this cleaned up.
> 
> So based on this, I picked the order:
> 
> 0x40000000 is viridian, vmware or xen
> 0x40000100 is vmware or xen
> 0x40000200 is xen

Is there really a point in enabling both Viridian and VMware extensions
at the same time?

> @@ -149,8 +152,11 @@ void pci_setup(void)
>              pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
>              pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
>              pci_writew(devfn, 0x22, 0x0000);
> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
> -            pci_writew(devfn, 0x3d, 0x0001);
> +            if ( !vmware_hw )
> +            {
> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
> +                pci_writew(devfn, 0x3d, 0x0001);
> +            }

This needs an explanation (it is merely being mentioned in the
description).

> @@ -5692,6 +5695,22 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
>  
>                  break;
>              }
> +            case HVM_PARAM_VMWARE_HW:
> +                /*
> +                 * This should only ever be set non-zero one time by
> +                 * the tools and is read only by the guest.
> +                 */
> +                if ( d == current->domain )
> +                {
> +                    rc = -EPERM;
> +                    break;
> +                }
> +                if ( d->arch.hvm_domain.params[HVM_PARAM_VMWARE_HW] )
> +                {
> +                    rc = -EEXIST;
> +                    break;
> +                }
> +                break;

Can you make this similar to Viridian, returning success when setting
the value to what it already is.

> --- /dev/null
> +++ b/xen/arch/x86/hvm/vmware.c
> @@ -0,0 +1,85 @@
> +/*
> + * arch/x86/hvm/vmware.c
> + *
> + * Copyright (C) 2012 Verizon Corporation
> + *
> + * This file is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License Version 2 (GPLv2)
> + * as published by the Free Software Foundation.
> + *
> + * This file is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> + * General Public License for more details. <http://www.gnu.org/licenses/>.
> + */
> +
> +#include <xen/sched.h>
> +#include <xen/version.h>
> +#include <xen/perfc.h>
> +#include <xen/hypercall.h>
> +#include <xen/domain_page.h>
> +#include <asm/paging.h>
> +#include <asm/p2m.h>
> +#include <asm/apic.h>
> +#include <asm/hvm/support.h>
> +
> +#include <public/xen.h>
> +#include <public/sched.h>
> +#include <public/hvm/params.h>
> +
> +#include <asm/hvm/hvm.h>
> +#include <asm/hvm/vlapic.h>
> +#include <asm/hvm/vmware.h>

Please keep all asm/ includes together. Also, are all above includes
really needed? The list looks more like having got copied from
somewhere else - e.g. I can't see a need for xen/perfc.h.

> +int cpuid_vmware_leaves(uint32_t idx, uint32_t sub_idx,
> +                        uint32_t *eax, uint32_t *ebx,
> +                        uint32_t *ecx, uint32_t *edx)
> +{

You don't seem to be using sub_idx: Is this a mistake, or are you
passing the value in just in case of future extensions (in which
case I'd be slightly in favor of not passing it for now)?

> --- a/xen/include/asm-x86/hvm/domain.h
> +++ b/xen/include/asm-x86/hvm/domain.h
> @@ -29,6 +29,7 @@
>  #include <asm/hvm/io.h>
>  #include <xen/hvm/iommu.h>
>  #include <asm/hvm/viridian.h>
> +#include <asm/hvm/vmware.h>

This being th only change to this file - why?

Jan

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/02/14 03:28, Jan Beulich wrote:
>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>> If non-zero then
>>    Return VMware's cpuid leaves.
>>    Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
>>    Force use of VMware's VGA in QEMU.
>>
>> The support of hypervisor cpuid leaves has not been agreed to.
>>
>> MicroSoft Hyper-V (AKA viridian) currently must be at 0x40000000.
>>
>> VMware currently must be at 0x40000000.
>>
>> KVM currently must be at 0x40000000 (from Seabios).
>>
>> Seabios will find xen at 0x40000000, 0x40000100, 0x40000200 ..
>> 0x40010000.
>>
>> http://download.microsoft.com/download/F/B/0/FB0D01A3-8E3A-4F5F-AA59-08C8026D3B8
>> A/requirements-for-implementing-microsoft-hypervisor-interface.docx
>>
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=disp
>> layKC&externalId=1009458
>>
>> http://lwn.net/Articles/301888/
>>    Attempted to get this cleaned up.
>>
>> So based on this, I picked the order:
>>
>> 0x40000000 is viridian, vmware or xen
>> 0x40000100 is vmware or xen
>> 0x40000200 is xen
> Is there really a point in enabling both Viridian and VMware extensions
> at the same time?

Not that I know of (and I do not want to say there there is no code
out there that can work with both).  Instead of an error or warning
I went with what xen is currently doing and that seabios was happy
to find xen at 0x40000200.

If the consensus is to ignore, or report an error or warning I will go that
way.  For now I am not planning on changing.

>> @@ -149,8 +152,11 @@ void pci_setup(void)
>>               pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
>>               pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
>>               pci_writew(devfn, 0x22, 0x0000);
>> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>> -            pci_writew(devfn, 0x3d, 0x0001);
>> +            if ( !vmware_hw )
>> +            {
>> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>> +                pci_writew(devfn, 0x3d, 0x0001);
>> +            }
> This needs an explanation (it is merely being mentioned in the
> description).

Ok, how does this comment sound:

/*
  * When looking more like VMware, let the guest pick the
  * PCI_INTERRUPT_LINE (0x3c) and the PCI_INTERRUPT_PIN(3d)
  * instead of them being hardcoded.  This allows for example
  * lscpci in the guest to match for "PIIX4 ACPI PM" what
  * one gets on VMware.
  */


I also plan to add this to the comment message:

The "not doing the hardcoded IRQ9 on PIIX4 ACPI PM" is only one of
many places where VMware's pci config space is different.  There are
many that also need this kind of change in QEMU.  That part is
outside the scope of this patch.



>> @@ -5692,6 +5695,22 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
>>   
>>                   break;
>>               }
>> +            case HVM_PARAM_VMWARE_HW:
>> +                /*
>> +                 * This should only ever be set non-zero one time by
>> +                 * the tools and is read only by the guest.
>> +                 */
>> +                if ( d == current->domain )
>> +                {
>> +                    rc = -EPERM;
>> +                    break;
>> +                }
>> +                if ( d->arch.hvm_domain.params[HVM_PARAM_VMWARE_HW] )
>> +                {
>> +                    rc = -EEXIST;
>> +                    break;
>> +                }
>> +                break;
> Can you make this similar to Viridian, returning success when setting
> the value to what it already is.

Yes  (but that patch is not there yet).

>> --- /dev/null
>> +++ b/xen/arch/x86/hvm/vmware.c
>> @@ -0,0 +1,85 @@
>> +/*
>> + * arch/x86/hvm/vmware.c
>> + *
>> + * Copyright (C) 2012 Verizon Corporation
>> + *
>> + * This file is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License Version 2 (GPLv2)
>> + * as published by the Free Software Foundation.
>> + *
>> + * This file is distributed in the hope that it will be useful,
>> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
>> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
>> + * General Public License for more details. <http://www.gnu.org/licenses/>.
>> + */
>> +
>> +#include <xen/sched.h>
>> +#include <xen/version.h>
>> +#include <xen/perfc.h>
>> +#include <xen/hypercall.h>
>> +#include <xen/domain_page.h>
>> +#include <asm/paging.h>
>> +#include <asm/p2m.h>
>> +#include <asm/apic.h>
>> +#include <asm/hvm/support.h>
>> +
>> +#include <public/xen.h>
>> +#include <public/sched.h>
>> +#include <public/hvm/params.h>
>> +
>> +#include <asm/hvm/hvm.h>
>> +#include <asm/hvm/vlapic.h>
>> +#include <asm/hvm/vmware.h>
> Please keep all asm/ includes together. Also, are all above includes
> really needed? The list looks more like having got copied from
> somewhere else - e.g. I can't see a need for xen/perfc.h.

Yes they were copied.  Will work out a smaller set.

>> +int cpuid_vmware_leaves(uint32_t idx, uint32_t sub_idx,
>> +                        uint32_t *eax, uint32_t *ebx,
>> +                        uint32_t *ecx, uint32_t *edx)
>> +{
> You don't seem to be using sub_idx: Is this a mistake, or are you
> passing the value in just in case of future extensions (in which
> case I'd be slightly in favor of not passing it for now)?

I was just copying cpuid_hypervisor_leaves() and missed that
it was not needed.  Will remove.

>> --- a/xen/include/asm-x86/hvm/domain.h
>> +++ b/xen/include/asm-x86/hvm/domain.h
>> @@ -29,6 +29,7 @@
>>   #include <asm/hvm/io.h>
>>   #include <xen/hvm/iommu.h>
>>   #include <asm/hvm/viridian.h>
>> +#include <asm/hvm/vmware.h>
> This being th only change to this file - why?

I was just following how viridian.h is done.

When I had it in just hvm.c, I remember other files not building. This
is no longer the case and so, this file is no longer changed.

    -Don Slutz


> Jan

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Jan Beulich]

>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
> On 09/02/14 03:28, Jan Beulich wrote:
>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>> So based on this, I picked the order:
>>>
>>> 0x40000000 is viridian, vmware or xen
>>> 0x40000100 is vmware or xen
>>> 0x40000200 is xen
>> Is there really a point in enabling both Viridian and VMware extensions
>> at the same time?
> 
> Not that I know of (and I do not want to say there there is no code
> out there that can work with both).  Instead of an error or warning
> I went with what xen is currently doing and that seabios was happy
> to find xen at 0x40000200.
> 
> If the consensus is to ignore, or report an error or warning I will go that
> way.  For now I am not planning on changing.

My personal take on this is that the hypervisor (or perhaps already
the tools) should reject enabling both at the same time.

>>> @@ -149,8 +152,11 @@ void pci_setup(void)
>>>               pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
>>>               pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
>>>               pci_writew(devfn, 0x22, 0x0000);
>>> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>>> -            pci_writew(devfn, 0x3d, 0x0001);
>>> +            if ( !vmware_hw )
>>> +            {
>>> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>>> +                pci_writew(devfn, 0x3d, 0x0001);
>>> +            }
>> This needs an explanation (it is merely being mentioned in the
>> description).
> 
> Ok, how does this comment sound:
> 
> /*
>   * When looking more like VMware, let the guest pick the
>   * PCI_INTERRUPT_LINE (0x3c) and the PCI_INTERRUPT_PIN(3d)
>   * instead of them being hardcoded.  This allows for example
>   * lscpci in the guest to match for "PIIX4 ACPI PM" what
>   * one gets on VMware.
>   */

The first sentence is just stating verbally what the code does, i.e.
pretty pointless. The second sentence - at least to me - doesn't
explain anything (to a large part perhaps because having lspci
produce identical output with real VMware is rather secondary a
goal imo).

Jan

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/03/14 03:45, Jan Beulich wrote:
>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>> @@ -149,8 +152,11 @@ void pci_setup(void)
>>>>                pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
>>>>                pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
>>>>                pci_writew(devfn, 0x22, 0x0000);
>>>> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>>>> -            pci_writew(devfn, 0x3d, 0x0001);
>>>> +            if ( !vmware_hw )
>>>> +            {
>>>> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>>>> +                pci_writew(devfn, 0x3d, 0x0001);
>>>> +            }
>>> This needs an explanation (it is merely being mentioned in the
>>> description).
>> Ok, how does this comment sound:
>>
>> /*
>>    * When looking more like VMware, let the guest pick the
>>    * PCI_INTERRUPT_LINE (0x3c) and the PCI_INTERRUPT_PIN(3d)
>>    * instead of them being hardcoded.  This allows for example
>>    * lscpci in the guest to match for "PIIX4 ACPI PM" what
>>    * one gets on VMware.
>>    */
> The first sentence is just stating verbally what the code does, i.e.
> pretty pointless. The second sentence - at least to me - doesn't
> explain anything (to a large part perhaps because having lspci
> produce identical output with real VMware is rather secondary a
> goal imo).

I can agree on it being a secondary goal, and so will drop it.

I just remembered that the better statement:

Attempt to reduce windows reactivations by making the hardware
look as much like VMware's.

I am fine with delaying this to 4.6 time frame.
    -Don Slutz


> Jan
>

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Jan Beulich]

>>> On 03.09.14 at 12:59, <dslutz@verizon.com> wrote:
> Attempt to reduce windows reactivations by making the hardware
> look as much like VMware's.

I continue having the impression that you imply prior knowledge by
the reader. To me "reduce windows reactivations" is at best
ambiguous, which may then be the reason why I'm not seeing the
connection to looking as much like VMware as possible is a desirable
goal.

Jan

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/03/14 08:33, Jan Beulich wrote:
>>>> On 03.09.14 at 12:59, <dslutz@verizon.com> wrote:
>> Attempt to reduce windows reactivations by making the hardware
>> look as much like VMware's.
> I continue having the impression that you imply prior knowledge by
> the reader. To me "reduce windows reactivations" is at best
> ambiguous, which may then be the reason why I'm not seeing the
> connection to looking as much like VMware as possible is a desirable
> goal.

That is a problem I have of lots of prior knowledge and not stating
all the things needed by the reader.  Please continue to ask me when
it is not clear.

    -Don Slutz

> Jan
>

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Wed, 2014-09-03 at 06:59 -0400, Don Slutz wrote:
> On 09/03/14 03:45, Jan Beulich wrote:
> >>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
> >> On 09/02/14 03:28, Jan Beulich wrote:
> >>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> >>>> @@ -149,8 +152,11 @@ void pci_setup(void)
> >>>>                pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
> >>>>                pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
> >>>>                pci_writew(devfn, 0x22, 0x0000);
> >>>> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
> >>>> -            pci_writew(devfn, 0x3d, 0x0001);
> >>>> +            if ( !vmware_hw )
> >>>> +            {
> >>>> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
> >>>> +                pci_writew(devfn, 0x3d, 0x0001);
> >>>> +            }
> >>> This needs an explanation (it is merely being mentioned in the
> >>> description).
> >> Ok, how does this comment sound:
> >>
> >> /*
> >>    * When looking more like VMware, let the guest pick the
> >>    * PCI_INTERRUPT_LINE (0x3c) and the PCI_INTERRUPT_PIN(3d)
> >>    * instead of them being hardcoded.  This allows for example
> >>    * lscpci in the guest to match for "PIIX4 ACPI PM" what
> >>    * one gets on VMware.
> >>    */
> > The first sentence is just stating verbally what the code does, i.e.
> > pretty pointless. The second sentence - at least to me - doesn't
> > explain anything (to a large part perhaps because having lspci
> > produce identical output with real VMware is rather secondary a
> > goal imo).
> 
> I can agree on it being a secondary goal, and so will drop it.
> 
> I just remembered that the better statement:
> 
> Attempt to reduce windows reactivations by making the hardware
> look as much like VMware's.

On the contrary avoiding the need to reactivate Windows seems like a
pretty good primary reason for this change, isn't it? (or maybe I don't
quite get what a reactivation is and how costly it is for the admin...)

Ian.

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 09:21, Ian Campbell wrote:
> On Wed, 2014-09-03 at 06:59 -0400, Don Slutz wrote:
>> On 09/03/14 03:45, Jan Beulich wrote:
>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>> @@ -149,8 +152,11 @@ void pci_setup(void)
>>>>>>                 pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
>>>>>>                 pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
>>>>>>                 pci_writew(devfn, 0x22, 0x0000);
>>>>>> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>>>>>> -            pci_writew(devfn, 0x3d, 0x0001);
>>>>>> +            if ( !vmware_hw )
>>>>>> +            {
>>>>>> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
>>>>>> +                pci_writew(devfn, 0x3d, 0x0001);
>>>>>> +            }
>>>>> This needs an explanation (it is merely being mentioned in the
>>>>> description).
>>>> Ok, how does this comment sound:
>>>>
>>>> /*
>>>>     * When looking more like VMware, let the guest pick the
>>>>     * PCI_INTERRUPT_LINE (0x3c) and the PCI_INTERRUPT_PIN(3d)
>>>>     * instead of them being hardcoded.  This allows for example
>>>>     * lscpci in the guest to match for "PIIX4 ACPI PM" what
>>>>     * one gets on VMware.
>>>>     */
>>> The first sentence is just stating verbally what the code does, i.e.
>>> pretty pointless. The second sentence - at least to me - doesn't
>>> explain anything (to a large part perhaps because having lspci
>>> produce identical output with real VMware is rather secondary a
>>> goal imo).
>> I can agree on it being a secondary goal, and so will drop it.
>>
>> I just remembered that the better statement:
>>
>> Attempt to reduce windows reactivations by making the hardware
>> look as much like VMware's.
> On the contrary avoiding the need to reactivate Windows seems like a
> pretty good primary reason for this change, isn't it? (or maybe I don't
> quite get what a reactivation is and how costly it is for the admin...)

I agree, but since this is 1 of about 20 similar changes (all the rest are
in QEMU), I am happy to defer this to 4.6 time frame.

Windows reactivation can at some times be expensive because Microsoft says you
have activated too many times with this license, you need a new one...

And it more effects smaller shops that have not paid the big bucks for a site
license.


     -Don Slutz


> Ian.
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Mon, 2014-09-08 at 09:47 -0400, Don Slutz wrote:
> On 09/08/14 09:21, Ian Campbell wrote:
> > On Wed, 2014-09-03 at 06:59 -0400, Don Slutz wrote:
> >> On 09/03/14 03:45, Jan Beulich wrote:
> >>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
> >>>> On 09/02/14 03:28, Jan Beulich wrote:
> >>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> >>>>>> @@ -149,8 +152,11 @@ void pci_setup(void)
> >>>>>>                 pci_writew(devfn, 0x20, 0x0000); /* No smb bus IO enable */
> >>>>>>                 pci_writew(devfn, 0xd2, 0x0000); /* No smb bus IO enable */
> >>>>>>                 pci_writew(devfn, 0x22, 0x0000);
> >>>>>> -            pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
> >>>>>> -            pci_writew(devfn, 0x3d, 0x0001);
> >>>>>> +            if ( !vmware_hw )
> >>>>>> +            {
> >>>>>> +                pci_writew(devfn, 0x3c, 0x0009); /* Hardcoded IRQ9 */
> >>>>>> +                pci_writew(devfn, 0x3d, 0x0001);
> >>>>>> +            }
> >>>>> This needs an explanation (it is merely being mentioned in the
> >>>>> description).
> >>>> Ok, how does this comment sound:
> >>>>
> >>>> /*
> >>>>     * When looking more like VMware, let the guest pick the
> >>>>     * PCI_INTERRUPT_LINE (0x3c) and the PCI_INTERRUPT_PIN(3d)
> >>>>     * instead of them being hardcoded.  This allows for example
> >>>>     * lscpci in the guest to match for "PIIX4 ACPI PM" what
> >>>>     * one gets on VMware.
> >>>>     */
> >>> The first sentence is just stating verbally what the code does, i.e.
> >>> pretty pointless. The second sentence - at least to me - doesn't
> >>> explain anything (to a large part perhaps because having lspci
> >>> produce identical output with real VMware is rather secondary a
> >>> goal imo).
> >> I can agree on it being a secondary goal, and so will drop it.
> >>
> >> I just remembered that the better statement:
> >>
> >> Attempt to reduce windows reactivations by making the hardware
> >> look as much like VMware's.
> > On the contrary avoiding the need to reactivate Windows seems like a
> > pretty good primary reason for this change, isn't it? (or maybe I don't
> > quite get what a reactivation is and how costly it is for the admin...)
> 
> I agree, but since this is 1 of about 20 similar changes (all the rest are
> in QEMU), I am happy to defer this to 4.6 time frame.

OK makes sense.

> Windows reactivation can at some times be expensive because Microsoft says you
> have activated too many times with this license, you need a new one...
> 
> And it more effects smaller shops that have not paid the big bucks for a site
> license.
> 
> 
>      -Don Slutz
> 
> 
> > Ian.
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
> >>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
> > On 09/02/14 03:28, Jan Beulich wrote:
> >>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> >>> So based on this, I picked the order:
> >>>
> >>> 0x40000000 is viridian, vmware or xen
> >>> 0x40000100 is vmware or xen
> >>> 0x40000200 is xen
> >> Is there really a point in enabling both Viridian and VMware extensions
> >> at the same time?
> > 
> > Not that I know of (and I do not want to say there there is no code
> > out there that can work with both).  Instead of an error or warning
> > I went with what xen is currently doing and that seabios was happy
> > to find xen at 0x40000200.
> > 
> > If the consensus is to ignore, or report an error or warning I will go that
> > way.  For now I am not planning on changing.
> 
> My personal take on this is that the hypervisor (or perhaps already
> the tools) should reject enabling both at the same time.

That sounds sensible to me.

Generally we seem to have the hypervisor check these things as a
backstop, to stop broken tools, but also check in the tools so we can
give a better error message.

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 09:20, Ian Campbell wrote:
> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>> So based on this, I picked the order:
>>>>>
>>>>> 0x40000000 is viridian, vmware or xen
>>>>> 0x40000100 is vmware or xen
>>>>> 0x40000200 is xen
>>>> Is there really a point in enabling both Viridian and VMware extensions
>>>> at the same time?
>>> Not that I know of (and I do not want to say there there is no code
>>> out there that can work with both).  Instead of an error or warning
>>> I went with what xen is currently doing and that seabios was happy
>>> to find xen at 0x40000200.
>>>
>>> If the consensus is to ignore, or report an error or warning I will go that
>>> way.  For now I am not planning on changing.
>> My personal take on this is that the hypervisor (or perhaps already
>> the tools) should reject enabling both at the same time.
> That sounds sensible to me.
>
> Generally we seem to have the hypervisor check these things as a
> backstop, to stop broken tools, but also check in the tools so we can
> give a better error message.
>

Ok, with 2 votes this way how about (for v4) I will drop the change to
xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change

cpuid_vmware_leaves to return 0 if is_viridian_domain().

And add some logic in the and doc in the tools patch to do this error
message.

    -Don Slutz

>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Andrew Cooper]

On 08/09/14 14:56, Don Slutz wrote:
> On 09/08/14 09:20, Ian Campbell wrote:
>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>> So based on this, I picked the order:
>>>>>>
>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>> 0x40000100 is vmware or xen
>>>>>> 0x40000200 is xen
>>>>> Is there really a point in enabling both Viridian and VMware
>>>>> extensions
>>>>> at the same time?
>>>> Not that I know of (and I do not want to say there there is no code
>>>> out there that can work with both).  Instead of an error or warning
>>>> I went with what xen is currently doing and that seabios was happy
>>>> to find xen at 0x40000200.
>>>>
>>>> If the consensus is to ignore, or report an error or warning I will
>>>> go that
>>>> way.  For now I am not planning on changing.
>>> My personal take on this is that the hypervisor (or perhaps already
>>> the tools) should reject enabling both at the same time.
>> That sounds sensible to me.
>>
>> Generally we seem to have the hypervisor check these things as a
>> backstop, to stop broken tools, but also check in the tools so we can
>> give a better error message.
>>
>
> Ok, with 2 votes this way how about (for v4) I will drop the change to
> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>
> cpuid_vmware_leaves to return 0 if is_viridian_domain().
>
> And add some logic in the and doc in the tools patch to do this error
> message.
>
>    -Don Slutz

I expect that Vmware will expose viridian to windows domains, as it is
the only supported Microsoft way of doing doing virt for windows. 
Therefore it is entirely plausible that both could need to be active at
once.  (Although this does depend on whether the vmware leaf supports
being somewhere other than 0x40000000, as the viridian leaf certainly
doesn't.)

Either way, the current 0x4000xxxx leaf handling is somewhat special in
Xen, as the viridian support was hacked in after the Xen leafs were
already present.  It is one area I was planning to fix up as part of my
cpuid levelling work for 4.6

~Andrew

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 10:07, Andrew Cooper wrote:
> On 08/09/14 14:56, Don Slutz wrote:
>> On 09/08/14 09:20, Ian Campbell wrote:
>>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>>> So based on this, I picked the order:
>>>>>>>
>>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>>> 0x40000100 is vmware or xen
>>>>>>> 0x40000200 is xen
>>>>>> Is there really a point in enabling both Viridian and VMware
>>>>>> extensions
>>>>>> at the same time?
>>>>> Not that I know of (and I do not want to say there there is no code
>>>>> out there that can work with both).  Instead of an error or warning
>>>>> I went with what xen is currently doing and that seabios was happy
>>>>> to find xen at 0x40000200.
>>>>>
>>>>> If the consensus is to ignore, or report an error or warning I will
>>>>> go that
>>>>> way.  For now I am not planning on changing.
>>>> My personal take on this is that the hypervisor (or perhaps already
>>>> the tools) should reject enabling both at the same time.
>>> That sounds sensible to me.
>>>
>>> Generally we seem to have the hypervisor check these things as a
>>> backstop, to stop broken tools, but also check in the tools so we can
>>> give a better error message.
>>>
>> Ok, with 2 votes this way how about (for v4) I will drop the change to
>> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>>
>> cpuid_vmware_leaves to return 0 if is_viridian_domain().
>>
>> And add some logic in the and doc in the tools patch to do this error
>> message.
>>
>>     -Don Slutz
> I expect that Vmware will expose viridian to windows domains, as it is
> the only supported Microsoft way of doing doing virt for windows.
> Therefore it is entirely plausible that both could need to be active at
> once.  (Although this does depend on whether the vmware leaf supports
> being somewhere other than 0x40000000, as the viridian leaf certainly
> doesn't.)

As far as I can tell, VMware does not expose viridian to windows
domains.  As I understand it they adjust the time that guest sees
so that windows does not BSOD 101.  They also adjust more things
so windows "works".  Going with viridian is much better.

I only see VMware on ESXi 4.1.0, they may have added this in newer
versions.  And (from the commit message's url):

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009458

Which says "Updated Jul 29, 2014", it also does not support being
somewhere other than 0x40000000.

Xen is the only one that I know of that current supports being
somewhere other than 0x40000000.

So I am happy to go with "only one of viridian or vmware_hw can
be non-zero".  Just need to know which way to go.

    -Don Slutz


> Either way, the current 0x4000xxxx leaf handling is somewhat special in
> Xen, as the viridian support was hacked in after the Xen leafs were
> already present.  It is one area I was planning to fix up as part of my
> cpuid levelling work for 4.6
>
> ~Andrew
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 10:07, Andrew Cooper wrote:
> On 08/09/14 14:56, Don Slutz wrote:
>> On 09/08/14 09:20, Ian Campbell wrote:
>>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>>> On 02.09.14 at 20:24,<dslutz@verizon.com>  wrote:
>>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>>> On 01.09.14 at 17:33,<dslutz@verizon.com>  wrote:
>>>>>>> So based on this, I picked the order:
>>>>>>>
>>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>>> 0x40000100 is vmware or xen
>>>>>>> 0x40000200 is xen
>>>>>> Is there really a point in enabling both Viridian and VMware
>>>>>> extensions
>>>>>> at the same time?
>>>>> Not that I know of (and I do not want to say there there is no code
>>>>> out there that can work with both).  Instead of an error or warning
>>>>> I went with what xen is currently doing and that seabios was happy
>>>>> to find xen at 0x40000200.
>>>>>
>>>>> If the consensus is to ignore, or report an error or warning I will
>>>>> go that
>>>>> way.  For now I am not planning on changing.
>>>> My personal take on this is that the hypervisor (or perhaps already
>>>> the tools) should reject enabling both at the same time.
>>> That sounds sensible to me.
>>>
>>> Generally we seem to have the hypervisor check these things as a
>>> backstop, to stop broken tools, but also check in the tools so we can
>>> give a better error message.
>>>
>> Ok, with 2 votes this way how about (for v4) I will drop the change to
>> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>>
>> cpuid_vmware_leaves to return 0 if is_viridian_domain().
>>
>> And add some logic in the and doc in the tools patch to do this error
>> message.
>>
>>     -Don Slutz
> I expect that Vmware will expose viridian to windows domains, as it is
> the only supported Microsoft way of doing doing virt for windows.
> Therefore it is entirely plausible that both could need to be active at
> once.  (Although this does depend on whether the vmware leaf supports
> being somewhere other than 0x40000000, as the viridian leaf certainly
> doesn't.)

As far as I can tell, VMware does not expose viridian to windows
domains.  As I understand it they adjust the time that guest sees
so that windows does not BSOD 101.

I only see VMware on ESXi 4.1.0, they may have added this in newer
versions.  And (from the commit message which the following url:)


http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009458

Which says "Updated Jul 29, 2014", it also does not support being
somewhere other than 0x40000000 .


> Either way, the current 0x4000xxxx leaf handling is somewhat special in
> Xen, as the viridian support was hacked in after the Xen leafs were
> already present.  It is one area I was planning to fix up as part of my
> cpuid levelling work for 4.6

Ok.  I just need to know what to provide for 4.5 -- which seems to be
allow only viridian or vmware_hw but not both.

     -Don Slutz

> ~Andrew
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Andrew Cooper]

On 08/09/2014 23:11, Don Slutz wrote:
> On 09/08/14 10:07, Andrew Cooper wrote:
>> On 08/09/14 14:56, Don Slutz wrote:
>>> On 09/08/14 09:20, Ian Campbell wrote:
>>>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>>>> On 02.09.14 at 20:24,<dslutz@verizon.com>  wrote:
>>>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>>>> On 01.09.14 at 17:33,<dslutz@verizon.com>  wrote:
>>>>>>>> So based on this, I picked the order:
>>>>>>>>
>>>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>>>> 0x40000100 is vmware or xen
>>>>>>>> 0x40000200 is xen
>>>>>>> Is there really a point in enabling both Viridian and VMware
>>>>>>> extensions
>>>>>>> at the same time?
>>>>>> Not that I know of (and I do not want to say there there is no code
>>>>>> out there that can work with both).  Instead of an error or warning
>>>>>> I went with what xen is currently doing and that seabios was happy
>>>>>> to find xen at 0x40000200.
>>>>>>
>>>>>> If the consensus is to ignore, or report an error or warning I will
>>>>>> go that
>>>>>> way.  For now I am not planning on changing.
>>>>> My personal take on this is that the hypervisor (or perhaps already
>>>>> the tools) should reject enabling both at the same time.
>>>> That sounds sensible to me.
>>>>
>>>> Generally we seem to have the hypervisor check these things as a
>>>> backstop, to stop broken tools, but also check in the tools so we can
>>>> give a better error message.
>>>>
>>> Ok, with 2 votes this way how about (for v4) I will drop the change to
>>> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>>>
>>> cpuid_vmware_leaves to return 0 if is_viridian_domain().
>>>
>>> And add some logic in the and doc in the tools patch to do this error
>>> message.
>>>
>>>     -Don Slutz
>> I expect that Vmware will expose viridian to windows domains, as it is
>> the only supported Microsoft way of doing doing virt for windows.
>> Therefore it is entirely plausible that both could need to be active at
>> once.  (Although this does depend on whether the vmware leaf supports
>> being somewhere other than 0x40000000, as the viridian leaf certainly
>> doesn't.)
>
> As far as I can tell, VMware does not expose viridian to windows
> domains.  As I understand it they adjust the time that guest sees
> so that windows does not BSOD 101.
>
> I only see VMware on ESXi 4.1.0, they may have added this in newer
> versions.  And (from the commit message which the following url:)
>
>
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009458
>
>
> Which says "Updated Jul 29, 2014", it also does not support being
> somewhere other than 0x40000000 .

Hmm - that's interesting, and does simplify matters.

In which case the current set of allowable leaves are:

Xen at 0x40000000 or
Viridian or Vmware at 0x40000000 and Xen at 0x40000100

Perhaps a doc in docs/misc/hypervisor-cpuid.{markdown/pandoc} explaining
this? (or more appropriate name if applicable; pandoc being dependent on
my migration v2 docs changes, although trivial to backport.)

>
>
>> Either way, the current 0x4000xxxx leaf handling is somewhat special in
>> Xen, as the viridian support was hacked in after the Xen leafs were
>> already present.  It is one area I was planning to fix up as part of my
>> cpuid levelling work for 4.6
>
> Ok.  I just need to know what to provide for 4.5 -- which seems to be
> allow only viridian or vmware_hw but not both.
>
>     -Don Slutz

Seems reasonable to me at this juncture.

~Andrew

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Jan Beulich]

>>> On 08.09.14 at 15:56, <dslutz@verizon.com> wrote:
> On 09/08/14 09:20, Ian Campbell wrote:
>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>> So based on this, I picked the order:
>>>>>>
>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>> 0x40000100 is vmware or xen
>>>>>> 0x40000200 is xen
>>>>> Is there really a point in enabling both Viridian and VMware extensions
>>>>> at the same time?
>>>> Not that I know of (and I do not want to say there there is no code
>>>> out there that can work with both).  Instead of an error or warning
>>>> I went with what xen is currently doing and that seabios was happy
>>>> to find xen at 0x40000200.
>>>>
>>>> If the consensus is to ignore, or report an error or warning I will go that
>>>> way.  For now I am not planning on changing.
>>> My personal take on this is that the hypervisor (or perhaps already
>>> the tools) should reject enabling both at the same time.
>> That sounds sensible to me.
>>
>> Generally we seem to have the hypervisor check these things as a
>> backstop, to stop broken tools, but also check in the tools so we can
>> give a better error message.
>>
> 
> Ok, with 2 votes this way how about (for v4) I will drop the change to
> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
> 
> cpuid_vmware_leaves to return 0 if is_viridian_domain().

Not exactly - the conclusion rather is to not allow both to become
true at the same time.

Jan

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Boris Ostrovsky]

On 09/08/2014 10:21 AM, Jan Beulich wrote:
>>>> On 08.09.14 at 15:56, <dslutz@verizon.com> wrote:
>> On 09/08/14 09:20, Ian Campbell wrote:
>>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>>> So based on this, I picked the order:
>>>>>>>
>>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>>> 0x40000100 is vmware or xen
>>>>>>> 0x40000200 is xen
>>>>>> Is there really a point in enabling both Viridian and VMware extensions
>>>>>> at the same time?
>>>>> Not that I know of (and I do not want to say there there is no code
>>>>> out there that can work with both).  Instead of an error or warning
>>>>> I went with what xen is currently doing and that seabios was happy
>>>>> to find xen at 0x40000200.
>>>>>
>>>>> If the consensus is to ignore, or report an error or warning I will go that
>>>>> way.  For now I am not planning on changing.
>>>> My personal take on this is that the hypervisor (or perhaps already
>>>> the tools) should reject enabling both at the same time.
>>> That sounds sensible to me.
>>>
>>> Generally we seem to have the hypervisor check these things as a
>>> backstop, to stop broken tools, but also check in the tools so we can
>>> give a better error message.
>>>
>> Ok, with 2 votes this way how about (for v4) I will drop the change to
>> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>>
>> cpuid_vmware_leaves to return 0 if is_viridian_domain().
> Not exactly - the conclusion rather is to not allow both to become
> true at the same time.

I have vague recollection of some Windows products (newer Microsoft 
Server releases?) expecting to run on hypervisor, i.e. Viridian. Would 
such restriction break these?

Or is this orthogonal to this discussion (assuming I am right about MS 
in the first place)?

-boris

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Jan Beulich]

>>> On 08.09.14 at 17:16, <boris.ostrovsky@oracle.com> wrote:
> On 09/08/2014 10:21 AM, Jan Beulich wrote:
>>>>> On 08.09.14 at 15:56, <dslutz@verizon.com> wrote:
>>> On 09/08/14 09:20, Ian Campbell wrote:
>>>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>>>> So based on this, I picked the order:
>>>>>>>>
>>>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>>>> 0x40000100 is vmware or xen
>>>>>>>> 0x40000200 is xen
>>>>>>> Is there really a point in enabling both Viridian and VMware extensions
>>>>>>> at the same time?
>>>>>> Not that I know of (and I do not want to say there there is no code
>>>>>> out there that can work with both).  Instead of an error or warning
>>>>>> I went with what xen is currently doing and that seabios was happy
>>>>>> to find xen at 0x40000200.
>>>>>>
>>>>>> If the consensus is to ignore, or report an error or warning I will go that
>>>>>> way.  For now I am not planning on changing.
>>>>> My personal take on this is that the hypervisor (or perhaps already
>>>>> the tools) should reject enabling both at the same time.
>>>> That sounds sensible to me.
>>>>
>>>> Generally we seem to have the hypervisor check these things as a
>>>> backstop, to stop broken tools, but also check in the tools so we can
>>>> give a better error message.
>>>>
>>> Ok, with 2 votes this way how about (for v4) I will drop the change to
>>> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>>>
>>> cpuid_vmware_leaves to return 0 if is_viridian_domain().
>> Not exactly - the conclusion rather is to not allow both to become
>> true at the same time.
> 
> I have vague recollection of some Windows products (newer Microsoft 
> Server releases?) expecting to run on hypervisor, i.e. Viridian. Would 
> such restriction break these?
> 
> Or is this orthogonal to this discussion (assuming I am right about MS 
> in the first place)?

The question is whether Windows, when run on VMware, makes use
of the VMware extensions _and_ after being migrated to Xen would
be capable of making use of the Viridian ones. I doubt that, i.e. I'd
assume that until rebooted they'd continue to use VMware's (with
the wild assumption that such a "live" migration is actually possible in
the first place), and they'd prefer using Viridian's after reboot. The
only dependency might be on devices that shouldn't disappear, but
that's independent of the hypervisor side foreign VMM emulation
afaict.

Jan

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 11:27, Jan Beulich wrote:
>>>> On 08.09.14 at 17:16, <boris.ostrovsky@oracle.com> wrote:
>> On 09/08/2014 10:21 AM, Jan Beulich wrote:
>>>>>> On 08.09.14 at 15:56, <dslutz@verizon.com> wrote:
>>>> On 09/08/14 09:20, Ian Campbell wrote:
>>>>> On Wed, 2014-09-03 at 08:45 +0100, Jan Beulich wrote:
>>>>>>>>> On 02.09.14 at 20:24, <dslutz@verizon.com> wrote:
>>>>>>> On 09/02/14 03:28, Jan Beulich wrote:
>>>>>>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>>>>>>> So based on this, I picked the order:
>>>>>>>>>
>>>>>>>>> 0x40000000 is viridian, vmware or xen
>>>>>>>>> 0x40000100 is vmware or xen
>>>>>>>>> 0x40000200 is xen
>>>>>>>> Is there really a point in enabling both Viridian and VMware extensions
>>>>>>>> at the same time?
>>>>>>> Not that I know of (and I do not want to say there there is no code
>>>>>>> out there that can work with both).  Instead of an error or warning
>>>>>>> I went with what xen is currently doing and that seabios was happy
>>>>>>> to find xen at 0x40000200.
>>>>>>>
>>>>>>> If the consensus is to ignore, or report an error or warning I will go that
>>>>>>> way.  For now I am not planning on changing.
>>>>>> My personal take on this is that the hypervisor (or perhaps already
>>>>>> the tools) should reject enabling both at the same time.
>>>>> That sounds sensible to me.
>>>>>
>>>>> Generally we seem to have the hypervisor check these things as a
>>>>> backstop, to stop broken tools, but also check in the tools so we can
>>>>> give a better error message.
>>>>>
>>>> Ok, with 2 votes this way how about (for v4) I will drop the change to
>>>> xen/arch/x86/traps.c (I.E. 0x40000100 will be xen)  And change
>>>>
>>>> cpuid_vmware_leaves to return 0 if is_viridian_domain().
>>> Not exactly - the conclusion rather is to not allow both to become
>>> true at the same time.
>> I have vague recollection of some Windows products (newer Microsoft
>> Server releases?) expecting to run on hypervisor, i.e. Viridian. Would
>> such restriction break these?

Not directly, newer windows and multiple cpus need viridian to run on
xen.  We have no plans to adjust xen so that windows without viridian
and multiple cpus would work.

VMware does all a fall back mode:


    Testing the virtual BIOS DMI information and the hypervisor port

Apart from the CPUID-based method for VMware virtual machine detection, 
VMware also provides a fallback mechanism for the following reasons:

  * This CPUID-based technique will not work for guest code running at
    CPL3 when VT/AMD-V is not available or not enabled.
  * The hypervisor present bit and hypervisor information leaf are only
    defined for products based on VMware hardware version 7.




Which does require getting data into smbios xenstore entries before

running the guest.  But does allow use of vmware tools even when
cpuid data says viridian.




>> Or is this orthogonal to this discussion (assuming I am right about MS
>> in the first place)?
> The question is whether Windows, when run on VMware, makes use
> of the VMware extensions _and_ after being migrated to Xen would
> be capable of making use of the Viridian ones. I doubt that, i.e. I'd
> assume that until rebooted they'd continue to use VMware's (with
> the wild assumption that such a "live" migration is actually possible in
> the first place), and they'd prefer using Viridian's after reboot. The
> only dependency might be on devices that shouldn't disappear, but
> that's independent of the hypervisor side foreign VMM emulation
> afaict.

Such a "live" migration would need a lot of work, and not clear
this topic would apply.  Most OSes like to determine what the hardware
looks like at boot, and not have it change while they are running.
  As far as I know for standard windows images the VMware support is not
enough with out the guest time faking that VMware does to get windows to
work.

So for 4.5 I see no issues with saying only viridian or vmware_hw, not both.


    -Don Slutz

> Jan
>

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
> If non-zero then
>   Return VMware's cpuid leaves.
>   Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.

Please can you say a words about why this is and what the implications
are?

>   Force use of VMware's VGA in QEMU.
> 
> The support of hypervisor cpuid leaves has not been agreed to.

Who needs to agree to this? Just us or do we need to be seeking
consensus with other hypervisors?

> So based on this, I picked the order:
> 
> 0x40000000 is viridian, vmware or xen
> 0x40000100 is vmware or xen
> 0x40000200 is xen

Which is another way of saying that the enabled options will be
presented in the order viridian, vmware, xen.

> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> index f1fc906..34fb021 100644
> --- a/docs/man/xl.cfg.pod.5
> +++ b/docs/man/xl.cfg.pod.5
> @@ -1139,6 +1139,12 @@ some other Operating Systems and in some circumstance can prevent
>  Xen's own paravirtualisation interfaces for HVM guests from being
>  used.
>  
> +=item B<vmware_hw=NUMBER>
> +
> +Turns on or off the exposure of VMware cpuid.  The number is the
> +VMware's hardware version number, where 0 is off.  If on it also
> +forces the use of VMware's VGA in QEMU.

Do you have a reference of the non-zero values of this field? How can a
user determine what the correct number to use is?

Other than parroting this value back to the guest in a cpuid leaf does
this value control anything else? If so then we may want to consider
something like an enum to allow us to advertise more precisely which
versions of vmware we are prepared to ape, but at the least we need to
range check this input somewhere along the way.

> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
> index b9a56d5..bc5cd57 100644
> --- a/tools/libxc/xc_domain_restore.c
> +++ b/tools/libxc/xc_domain_restore.c
> diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
> index 254fdb3..76dc307 100644
> --- a/tools/libxc/xc_domain_save.c
> +++ b/tools/libxc/xc_domain_save.c

There will be a race condition between this series and the new migration
stuff. Copying Andy for his opinion on how to proceed.

> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
> index 103cbca..c79274b 100644
> --- a/tools/libxl/libxl_dm.c
> +++ b/tools/libxl/libxl_dm.c
> @@ -542,19 +542,28 @@ static char ** libxl__build_device_model_args_new(libxl__gc *gc,
>              }
>          }
>  
> -        switch (b_info->u.hvm.vga.kind) {
> -        case LIBXL_VGA_INTERFACE_TYPE_STD:
> -            flexarray_append_pair(dm_args, "-device",
> -                GCSPRINTF("VGA,vgamem_mb=%d",
> -                libxl__sizekb_to_mb(b_info->video_memkb)));
> -            break;
> -        case LIBXL_VGA_INTERFACE_TYPE_CIRRUS:
> +        if (b_info->u.hvm.vmware_hw) {

Should we have LIBXL_VGA_INTERFACE_TYPE_VMWARE, which is the default for
hvm.vga.kind if vmware_hw is set?

Apart from letting this code become cleaner this would allow user to
enable vmware_hw and override the vga to something else (if they so
wished for some reason).

> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
> index 08a7927..e03f46d 100644
> --- a/tools/libxl/libxl_types.idl
> +++ b/tools/libxl/libxl_types.idl
> @@ -371,6 +371,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>                                         ("timeoffset",       string),
>                                         ("hpet",             libxl_defbool),
>                                         ("vpt_align",        libxl_defbool),
> +                                       ("vmware_hw",        UInt(64, init_val = 0)),

Everything defaults to 0 by default, so you don't need the initval. Also
you can use "uint64" rather than open coding UInt(64).

You need to add a LIBXL_HAVE_FOO to libxl.h so that applications know
they can use this. There are a bunch of existing examples.

Ian.

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Andrew Cooper]

On 08/09/14 14:17, Ian Campbell wrote:
>> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
>> index b9a56d5..bc5cd57 100644
>> --- a/tools/libxc/xc_domain_restore.c
>> +++ b/tools/libxc/xc_domain_restore.c
>> diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
>> index 254fdb3..76dc307 100644
>> --- a/tools/libxc/xc_domain_save.c
>> +++ b/tools/libxc/xc_domain_save.c
> There will be a race condition between this series and the new migration
> stuff. Copying Andy for his opinion on how to proceed.

So it will.  This is a single HVM_PARAM being sent, so can trivially be
added with all of 1 line addition in the new v2 code.

~Andrew

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Mon, 2014-09-08 at 14:27 +0100, Andrew Cooper wrote:
> On 08/09/14 14:17, Ian Campbell wrote:
> >> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
> >> index b9a56d5..bc5cd57 100644
> >> --- a/tools/libxc/xc_domain_restore.c
> >> +++ b/tools/libxc/xc_domain_restore.c
> >> diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
> >> index 254fdb3..76dc307 100644
> >> --- a/tools/libxc/xc_domain_save.c
> >> +++ b/tools/libxc/xc_domain_save.c
> > There will be a race condition between this series and the new migration
> > stuff. Copying Andy for his opinion on how to proceed.
> 
> So it will.  This is a single HVM_PARAM being sent, so can trivially be
> added with all of 1 line addition in the new v2 code.

Ah, yes, I was thinking you'd need to spec a new record etc, but no.
Perfect (I mean that in the positive sense).

Ian.

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 09:41, Ian Campbell wrote:
> On Mon, 2014-09-08 at 14:27 +0100, Andrew Cooper wrote:
>> On 08/09/14 14:17, Ian Campbell wrote:
>>>> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
>>>> index b9a56d5..bc5cd57 100644
>>>> --- a/tools/libxc/xc_domain_restore.c
>>>> +++ b/tools/libxc/xc_domain_restore.c
>>>> diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
>>>> index 254fdb3..76dc307 100644
>>>> --- a/tools/libxc/xc_domain_save.c
>>>> +++ b/tools/libxc/xc_domain_save.c
>>> There will be a race condition between this series and the new migration
>>> stuff. Copying Andy for his opinion on how to proceed.
>> So it will.  This is a single HVM_PARAM being sent, so can trivially be
>> added with all of 1 line addition in the new v2 code.
> Ah, yes, I was thinking you'd need to spec a new record etc, but no.
> Perfect (I mean that in the positive sense).

Well, v3 now has 2 of these.   And this is good news.

     -Don Slutz

> Ian.
>

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 09:17, Ian Campbell wrote:
> On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
>> If non-zero then
>>    Return VMware's cpuid leaves.
>>    Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
> Please can you say a words about why this is and what the implications
> are?

Duplicate -- windows activation.

>>    Force use of VMware's VGA in QEMU.
>>
>> The support of hypervisor cpuid leaves has not been agreed to.
> Who needs to agree to this? Just us or do we need to be seeking
> consensus with other hypervisors?

Possible consensus with other hypervisors.    The 2 that are an issue
if MicroSoft (Hyper-V, viridian) and VMware.  Xen is not the issue.


>> So based on this, I picked the order:
>>
>> 0x40000000 is viridian, vmware or xen
>> 0x40000100 is vmware or xen
>> 0x40000200 is xen
> Which is another way of saying that the enabled options will be
> presented in the order viridian, vmware, xen.
>

Yes.

>> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
>> index f1fc906..34fb021 100644
>> --- a/docs/man/xl.cfg.pod.5
>> +++ b/docs/man/xl.cfg.pod.5
>> @@ -1139,6 +1139,12 @@ some other Operating Systems and in some circumstance can prevent
>>   Xen's own paravirtualisation interfaces for HVM guests from being
>>   used.
>>   
>> +=item B<vmware_hw=NUMBER>
>> +
>> +Turns on or off the exposure of VMware cpuid.  The number is the
>> +VMware's hardware version number, where 0 is off.  If on it also
>> +forces the use of VMware's VGA in QEMU.
> Do you have a reference of the non-zero values of this field? How can a
> user determine what the correct number to use is?

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003746

For most uses, any non-zero is good enough.  Where it matters is how
QEMU presents emulated hardware.  VMware changes pci config space
based on this number which is stored in a .vmx file.

Is vmware_hw too short?  Should I use vmware_virtual_hardware_version?

or just adjust the xl.cfg.pod.5 description?




> Other than parroting this value back to the guest in a cpuid leaf does
> this value control anything else? If so then we may want to consider
> something like an enum to allow us to advertise more precisely which
> versions of vmware we are prepared to ape, but at the least we need to
> range check this input somewhere along the way.

See above, mostly just QEMU.


>> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
>> index b9a56d5..bc5cd57 100644
>> --- a/tools/libxc/xc_domain_restore.c
>> +++ b/tools/libxc/xc_domain_restore.c
>> diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
>> index 254fdb3..76dc307 100644
>> --- a/tools/libxc/xc_domain_save.c
>> +++ b/tools/libxc/xc_domain_save.c
> There will be a race condition between this series and the new migration
> stuff. Copying Andy for his opinion on how to proceed.
>
>> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
>> index 103cbca..c79274b 100644
>> --- a/tools/libxl/libxl_dm.c
>> +++ b/tools/libxl/libxl_dm.c
>> @@ -542,19 +542,28 @@ static char ** libxl__build_device_model_args_new(libxl__gc *gc,
>>               }
>>           }
>>   
>> -        switch (b_info->u.hvm.vga.kind) {
>> -        case LIBXL_VGA_INTERFACE_TYPE_STD:
>> -            flexarray_append_pair(dm_args, "-device",
>> -                GCSPRINTF("VGA,vgamem_mb=%d",
>> -                libxl__sizekb_to_mb(b_info->video_memkb)));
>> -            break;
>> -        case LIBXL_VGA_INTERFACE_TYPE_CIRRUS:
>> +        if (b_info->u.hvm.vmware_hw) {
> Should we have LIBXL_VGA_INTERFACE_TYPE_VMWARE, which is the default for
> hvm.vga.kind if vmware_hw is set?
>
> Apart from letting this code become cleaner this would allow user to
> enable vmware_hw and override the vga to something else (if they so
> wished for some reason).
>

Ok, I will look to adding this.


>> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
>> index 08a7927..e03f46d 100644
>> --- a/tools/libxl/libxl_types.idl
>> +++ b/tools/libxl/libxl_types.idl
>> @@ -371,6 +371,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>>                                          ("timeoffset",       string),
>>                                          ("hpet",             libxl_defbool),
>>                                          ("vpt_align",        libxl_defbool),
>> +                                       ("vmware_hw",        UInt(64, init_val = 0)),
> Everything defaults to 0 by default, so you don't need the initval. Also
> you can use "uint64" rather than open coding UInt(64).
>
> You need to add a LIBXL_HAVE_FOO to libxl.h so that applications know
> they can use this. There are a bunch of existing examples.

Will do.

    -Don Slutz


> Ian.
>

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Mon, 2014-09-08 at 15:16 -0400, Don Slutz wrote:
> On 09/08/14 09:17, Ian Campbell wrote:
> > On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
> >> If non-zero then
> >>    Return VMware's cpuid leaves.
> >>    Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
> > Please can you say a words about why this is and what the implications
> > are?
> 
> Duplicate -- windows activation.
> 
> >>    Force use of VMware's VGA in QEMU.
> >>
> >> The support of hypervisor cpuid leaves has not been agreed to.
> > Who needs to agree to this? Just us or do we need to be seeking
> > consensus with other hypervisors?
> 
> Possible consensus with other hypervisors.    The 2 that are an issue
> if MicroSoft (Hyper-V, viridian) and VMware.  Xen is not the issue.
> 
> 
> >> So based on this, I picked the order:
> >>
> >> 0x40000000 is viridian, vmware or xen
> >> 0x40000100 is vmware or xen
> >> 0x40000200 is xen
> > Which is another way of saying that the enabled options will be
> > presented in the order viridian, vmware, xen.
> >
> 
> Yes.
> 
> >> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
> >> index f1fc906..34fb021 100644
> >> --- a/docs/man/xl.cfg.pod.5
> >> +++ b/docs/man/xl.cfg.pod.5
> >> @@ -1139,6 +1139,12 @@ some other Operating Systems and in some circumstance can prevent
> >>   Xen's own paravirtualisation interfaces for HVM guests from being
> >>   used.
> >>   
> >> +=item B<vmware_hw=NUMBER>
> >> +
> >> +Turns on or off the exposure of VMware cpuid.  The number is the
> >> +VMware's hardware version number, where 0 is off.  If on it also
> >> +forces the use of VMware's VGA in QEMU.
> > Do you have a reference of the non-zero values of this field? How can a
> > user determine what the correct number to use is?
> 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003746
> 
> For most uses, any non-zero is good enough.  Where it matters is how
> QEMU presents emulated hardware.  VMware changes pci config space
> based on this number which is stored in a .vmx file.
> 
> Is vmware_hw too short?  Should I use vmware_virtual_hardware_version?
> 
> or just adjust the xl.cfg.pod.5 description?

Just updating the description to give users some clue as to what number
they should use would be enough.

I think you are implying that there are occasions where a user would
care about which specific value is used, so turning this into a boolean
isn't good enough. But an enum might be appropriate. (see below)

> > Other than parroting this value back to the guest in a cpuid leaf does
> > this value control anything else? If so then we may want to consider
> > something like an enum to allow us to advertise more precisely which
> > versions of vmware we are prepared to ape, but at the least we need to
> > range check this input somewhere along the way.
> 
> See above, mostly just QEMU.

What does qemu do with a number which it doesn't understand, perhaps
corresponding to a newer vmware version which it hasn't learnt about
yet?

This sort of issue is why I was proposing an enum, or at least some sort
of range checking.

Ian.

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/09/14 05:39, Ian Campbell wrote:
> On Mon, 2014-09-08 at 15:16 -0400, Don Slutz wrote:
>> On 09/08/14 09:17, Ian Campbell wrote:
>>> On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
>>>> If non-zero then
>>>>     Return VMware's cpuid leaves.
>>>>     Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
>>> Please can you say a words about why this is and what the implications
>>> are?
>> Duplicate -- windows activation.
>>
>>>>     Force use of VMware's VGA in QEMU.
>>>>
>>>> The support of hypervisor cpuid leaves has not been agreed to.
>>> Who needs to agree to this? Just us or do we need to be seeking
>>> consensus with other hypervisors?
>> Possible consensus with other hypervisors.    The 2 that are an issue
>> if MicroSoft (Hyper-V, viridian) and VMware.  Xen is not the issue.
>>
>>
>>>> So based on this, I picked the order:
>>>>
>>>> 0x40000000 is viridian, vmware or xen
>>>> 0x40000100 is vmware or xen
>>>> 0x40000200 is xen
>>> Which is another way of saying that the enabled options will be
>>> presented in the order viridian, vmware, xen.
>>>
>> Yes.
>>
>>>> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
>>>> index f1fc906..34fb021 100644
>>>> --- a/docs/man/xl.cfg.pod.5
>>>> +++ b/docs/man/xl.cfg.pod.5
>>>> @@ -1139,6 +1139,12 @@ some other Operating Systems and in some circumstance can prevent
>>>>    Xen's own paravirtualisation interfaces for HVM guests from being
>>>>    used.
>>>>    
>>>> +=item B<vmware_hw=NUMBER>
>>>> +
>>>> +Turns on or off the exposure of VMware cpuid.  The number is the
>>>> +VMware's hardware version number, where 0 is off.  If on it also
>>>> +forces the use of VMware's VGA in QEMU.
>>> Do you have a reference of the non-zero values of this field? How can a
>>> user determine what the correct number to use is?
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003746
>>
>> For most uses, any non-zero is good enough.  Where it matters is how
>> QEMU presents emulated hardware.  VMware changes pci config space
>> based on this number which is stored in a .vmx file.
>>
>> Is vmware_hw too short?  Should I use vmware_virtual_hardware_version?
>>
>> or just adjust the xl.cfg.pod.5 description?
> Just updating the description to give users some clue as to what number
> they should use would be enough.

How does the following look:

   vmware_hw numbers come from VMware config files.

   In a .vmx it is virtualHW.version

   In a .ovf it is part of the value of vssd:VirtualSystemType

   for vssd:VirtualSystemType == vmx-07, vmware_hw = 7

Should I refer them to the vmware web site?

> I think you are implying that there are occasions where a user would
> care about which specific value is used, so turning this into a boolean
> isn't good enough. But an enum might be appropriate. (see below)

Yes, a boolean is not enough.

>>> Other than parroting this value back to the guest in a cpuid leaf does
>>> this value control anything else? If so then we may want to consider
>>> something like an enum to allow us to advertise more precisely which
>>> versions of vmware we are prepared to ape, but at the least we need to
>>> range check this input somewhere along the way.
>> See above, mostly just QEMU.
> What does qemu do with a number which it doesn't understand, perhaps
> corresponding to a newer vmware version which it hasn't learnt about
> yet?

My version currently checks for various ranges.  Like != 0, >= 4,
 >= 4 && < 7, >= 7.  I do not expect that I can upstream it with this,
but it should be similar.


> This sort of issue is why I was proposing an enum, or at least some sort
> of range checking.

Since most of the use is in QEMU, I see no need for an enum in xen.
All xen uses I know of are == 0 or != 0.  I can add some range checking
but think a warning might be better so that a newer QEMU with support
for a given value could be used with an older xen without change to
xen.

     -Don Slutz


> Ian.
>

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Ian Campbell]

On Tue, 2014-09-09 at 13:02 -0400, Don Slutz wrote:
> On 09/09/14 05:39, Ian Campbell wrote:
> > Just updating the description to give users some clue as to what number
> > they should use would be enough.
> 
> How does the following look:
> 
>    vmware_hw numbers come from VMware config files.
> 
>    In a .vmx it is virtualHW.version
> 
>    In a .ovf it is part of the value of vssd:VirtualSystemType
> 
>    for vssd:VirtualSystemType == vmx-07, vmware_hw = 7
> 
> Should I refer them to the vmware web site?

Probably. Will the above make sense to a normal vmware user? Since it is
essentially Greek to me. The includes how to dig into a .vmx or .ovf to
find these values. e..g are they plain text files? Are specific tools
needed?

> >>> Other than parroting this value back to the guest in a cpuid leaf does
> >>> this value control anything else? If so then we may want to consider
> >>> something like an enum to allow us to advertise more precisely which
> >>> versions of vmware we are prepared to ape, but at the least we need to
> >>> range check this input somewhere along the way.
> >> See above, mostly just QEMU.
> > What does qemu do with a number which it doesn't understand, perhaps
> > corresponding to a newer vmware version which it hasn't learnt about
> > yet?
> 
> My version currently checks for various ranges.  Like != 0, >= 4,
>  >= 4 && < 7, >= 7.  I do not expect that I can upstream it with this,
> but it should be similar.
> 
> 
> > This sort of issue is why I was proposing an enum, or at least some sort
> > of range checking.
> 
> Since most of the use is in QEMU, I see no need for an enum in xen.
> All xen uses I know of are == 0 or != 0.  I can add some range checking
> but think a warning might be better so that a newer QEMU with support
> for a given value could be used with an older xen without change to
> xen.

Will qemu error out in an obvious way if an unsupported number is used?

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/10/14 05:30, Ian Campbell wrote:
> On Tue, 2014-09-09 at 13:02 -0400, Don Slutz wrote:
>> On 09/09/14 05:39, Ian Campbell wrote:
>>> Just updating the description to give users some clue as to what number
>>> they should use would be enough.
>> How does the following look:
>>
>>     vmware_hw numbers come from VMware config files.
>>
>>     In a .vmx it is virtualHW.version
>>
>>     In a .ovf it is part of the value of vssd:VirtualSystemType
>>
>>     for vssd:VirtualSystemType == vmx-07, vmware_hw = 7
>>
>> Should I refer them to the vmware web site?
> Probably. Will the above make sense to a normal vmware user? Since it is
> essentially Greek to me. The includes how to dig into a .vmx or .ovf to
> find these values. e..g are they plain text files? Are specific tools
> needed?
>

Yes, I would expect a normal vmware user to understand this.

Both files are "text".  .vmx is a list of "key = value" lines.
.ovf is a xml file.  Any text editor is enough (or more or less).

    -Don Slutz


>>>>> Other than parroting this value back to the guest in a cpuid leaf does
>>>>> this value control anything else? If so then we may want to consider
>>>>> something like an enum to allow us to advertise more precisely which
>>>>> versions of vmware we are prepared to ape, but at the least we need to
>>>>> range check this input somewhere along the way.
>>>> See above, mostly just QEMU.
>>> What does qemu do with a number which it doesn't understand, perhaps
>>> corresponding to a newer vmware version which it hasn't learnt about
>>> yet?
>> My version currently checks for various ranges.  Like != 0, >= 4,
>>   >= 4 && < 7, >= 7.  I do not expect that I can upstream it with this,
>> but it should be similar.
>>
>>
>>> This sort of issue is why I was proposing an enum, or at least some sort
>>> of range checking.
>> Since most of the use is in QEMU, I see no need for an enum in xen.
>> All xen uses I know of are == 0 or != 0.  I can add some range checking
>> but think a warning might be better so that a newer QEMU with support
>> for a given value could be used with an older xen without change to
>> xen.
> Will qemu error out in an obvious way if an unsupported number is used?
>

My version does not.  However, that is the kind of change I expect to make
when I get there.   Since this is not yet done, not sure it matters.

Today the valid values are 0,3-4,6-11.  When I first coded this it was
0,3-4,6-7.   At that time all my testing showed only 0, 4 and 7 as the key
points. 6 was the same as 4.  VMware does have differences
between 6 and 4 but they were all around things like 4 does not support
device pvscsi, 6 and 7 do.   Trying and embedding this knowledge in Xen
to me is not the right way to go.  I would expect some one using this to
know the "right" value, or to just use 3.

    -Don Slutz

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Slutz, Donald Christopher]

On 09/10/14 05:30, Ian Campbell wrote:
> On Tue, 2014-09-09 at 13:02 -0400, Don Slutz wrote:
>> On 09/09/14 05:39, Ian Campbell wrote:
>>> Just updating the description to give users some clue as to what number
>>> they should use would be enough.
>> How does the following look:
>>
>>     vmware_hw numbers come from VMware config files.
>>
>>     In a .vmx it is virtualHW.version
>>
>>     In a .ovf it is part of the value of vssd:VirtualSystemType
>>
>>     for vssd:VirtualSystemType == vmx-07, vmware_hw = 7
>>
>> Should I refer them to the vmware web site?
> Probably. Will the above make sense to a normal vmware user? Since it is
> essentially Greek to me. The includes how to dig into a .vmx or .ovf to
> find these values. e..g are they plain text files? Are specific tools
> needed?
>

Yes, I would expect a normal vmware user to understand this.

Both files are "text".  .vmx is a list of "key = value" lines.
.ovf is a xml file.  Any text editor is enough (or more or less).

    -Don Slutz


>>>>> Other than parroting this value back to the guest in a cpuid leaf does
>>>>> this value control anything else? If so then we may want to consider
>>>>> something like an enum to allow us to advertise more precisely which
>>>>> versions of vmware we are prepared to ape, but at the least we need to
>>>>> range check this input somewhere along the way.
>>>> See above, mostly just QEMU.
>>> What does qemu do with a number which it doesn't understand, perhaps
>>> corresponding to a newer vmware version which it hasn't learnt about
>>> yet?
>> My version currently checks for various ranges.  Like != 0, >= 4,
>>   >= 4 && < 7, >= 7.  I do not expect that I can upstream it with this,
>> but it should be similar.
>>
>>
>>> This sort of issue is why I was proposing an enum, or at least some sort
>>> of range checking.
>> Since most of the use is in QEMU, I see no need for an enum in xen.
>> All xen uses I know of are == 0 or != 0.  I can add some range checking
>> but think a warning might be better so that a newer QEMU with support
>> for a given value could be used with an older xen without change to
>> xen.
> Will qemu error out in an obvious way if an unsupported number is used?
>

My version does not.  However, that is the kind of change I expect to make
when I get there.   Since this is not yet done, not sure it matters.

Today the valid values are 0,3-4,6-11.  When I first coded this it was
0,3-4,6-7.   At that time all my testing showed only 0, 4 and 7 as the key
points. 6 was the same as 4.  VMware does have differences
between 6 and 4 but they were all around things like 4 does not support
device pvscsi, 6 and 7 do.   Trying and embedding this knowledge in Xen
to me is not the right way to go.  I would expect some one using this to
know the "right" value,

Re: [PATCH v2 1/3] Add vmware_hw to xl.cfg

[Don Slutz]

On 09/08/14 09:17, Ian Campbell wrote:
> On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
>> If non-zero then
>>    Return VMware's cpuid leaves.
>>    Not doing the hardcoded IRQ9 on PIIX4 ACPI PM.
> Please can you say a words about why this is and what the implications
> are?

Duplicate -- windows activation.

>>    Force use of VMware's VGA in QEMU.
>>
>> The support of hypervisor cpuid leaves has not been agreed to.
> Who needs to agree to this? Just us or do we need to be seeking
> consensus with other hypervisors?

Possible consensus with other hypervisors.    The 2 that are an issue
if MicroSoft (Hyper-V, viridian) and VMware.  Xen is not the issue.


>> So based on this, I picked the order:
>>
>> 0x40000000 is viridian, vmware or xen
>> 0x40000100 is vmware or xen
>> 0x40000200 is xen
> Which is another way of saying that the enabled options will be
> presented in the order viridian, vmware, xen.
>

Yes.

>> diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5
>> index f1fc906..34fb021 100644
>> --- a/docs/man/xl.cfg.pod.5
>> +++ b/docs/man/xl.cfg.pod.5
>> @@ -1139,6 +1139,12 @@ some other Operating Systems and in some circumstance can prevent
>>   Xen's own paravirtualisation interfaces for HVM guests from being
>>   used.
>>
>> +=item B<vmware_hw=NUMBER>
>> +
>> +Turns on or off the exposure of VMware cpuid.  The number is the
>> +VMware's hardware version number, where 0 is off.  If on it also
>> +forces the use of VMware's VGA in QEMU.
> Do you have a reference of the non-zero values of this field? How can a
> user determine what the correct number to use is?

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1003746

For most uses, any non-zero is good enough.  Where it matters is how
QEMU presents emulated hardware.  VMware changes pci config space
based on this number which is stored in a .vmx file.

Is vmware_hw too short?  Should I use vmware_virtual_hardware_version?

or just adjust the xl.cfg.pod.5 description?




> Other than parroting this value back to the guest in a cpuid leaf does
> this value control anything else? If so then we may want to consider
> something like an enum to allow us to advertise more precisely which
> versions of vmware we are prepared to ape, but at the least we need to
> range check this input somewhere along the way.

See above, mostly just QEMU.


>> diff --git a/tools/libxc/xc_domain_restore.c b/tools/libxc/xc_domain_restore.c
>> index b9a56d5..bc5cd57 100644
>> --- a/tools/libxc/xc_domain_restore.c
>> +++ b/tools/libxc/xc_domain_restore.c
>> diff --git a/tools/libxc/xc_domain_save.c b/tools/libxc/xc_domain_save.c
>> index 254fdb3..76dc307 100644
>> --- a/tools/libxc/xc_domain_save.c
>> +++ b/tools/libxc/xc_domain_save.c
> There will be a race condition between this series and the new migration
> stuff. Copying Andy for his opinion on how to proceed.
>
>> diff --git a/tools/libxl/libxl_dm.c b/tools/libxl/libxl_dm.c
>> index 103cbca..c79274b 100644
>> --- a/tools/libxl/libxl_dm.c
>> +++ b/tools/libxl/libxl_dm.c
>> @@ -542,19 +542,28 @@ static char ** libxl__build_device_model_args_new(libxl__gc *gc,
>>               }
>>           }
>>
>> -        switch (b_info->u.hvm.vga.kind) {
>> -        case LIBXL_VGA_INTERFACE_TYPE_STD:
>> -            flexarray_append_pair(dm_args, "-device",
>> -                GCSPRINTF("VGA,vgamem_mb=%d",
>> -                libxl__sizekb_to_mb(b_info->video_memkb)));
>> -            break;
>> -        case LIBXL_VGA_INTERFACE_TYPE_CIRRUS:
>> +        if (b_info->u.hvm.vmware_hw) {
> Should we have LIBXL_VGA_INTERFACE_TYPE_VMWARE, which is the default for
> hvm.vga.kind if vmware_hw is set?
>
> Apart from letting this code become cleaner this would allow user to
> enable vmware_hw and override the vga to something else (if they so
> wished for some reason).
>

Ok, I will look to adding this.


>> diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
>> index 08a7927..e03f46d 100644
>> --- a/tools/libxl/libxl_types.idl
>> +++ b/tools/libxl/libxl_types.idl
>> @@ -371,6 +371,7 @@ libxl_domain_build_info = Struct("domain_build_info",[
>>                                          ("timeoffset",       string),
>>                                          ("hpet",             libxl_defbool),
>>                                          ("vpt_align",        libxl_defbool),
>> +                                       ("vmware_hw",        UInt(64, init_val = 0)),
> Everything defaults to 0 by default, so you don't need the initval. Also
> you can use "uint64" rather than open coding UInt(64).
>
> You need to add a LIBXL_HAVE_FOO to libxl.h so that applications know
> they can use this. There are a bunch of existing examples.

Will do.

    -Don Slutz


> Ian.
>

[PATCH v2 2/3] vmport: Add VMware provided include files.

[Don Slutz]

These 2 files: backdoor_def.h and guest_msg_def.h come from:

http://packages.vmware.com/tools/esx/3.5latest/rhel4/SRPMS/index.html
 open-vm-tools-kmod-7.4.8-396269.423167.src.rpm
  open-vm-tools-kmod-7.4.8.tar.gz
   vmhgfs/backdoor_def.h
   vmhgfs/guest_msg_def.h

and are unchanged.

Signed-off-by: Don Slutz <dslutz@verizon.com>
---
 xen/arch/x86/hvm/vmport/backdoor_def.h  | 167 ++++++++++++++++++++++++++++++++
 xen/arch/x86/hvm/vmport/guest_msg_def.h |  87 +++++++++++++++++
 2 files changed, 254 insertions(+)
 create mode 100644 xen/arch/x86/hvm/vmport/backdoor_def.h
 create mode 100644 xen/arch/x86/hvm/vmport/guest_msg_def.h

diff --git a/xen/arch/x86/hvm/vmport/backdoor_def.h b/xen/arch/x86/hvm/vmport/backdoor_def.h
new file mode 100644
index 0000000..e76795f
--- /dev/null
+++ b/xen/arch/x86/hvm/vmport/backdoor_def.h
@@ -0,0 +1,167 @@
+/* **********************************************************
+ * Copyright 1998 VMware, Inc.  All rights reserved. 
+ * **********************************************************
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation version 2 and no later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
+ */
+
+/*
+ * backdoor_def.h --
+ *
+ * This contains backdoor defines that can be included from
+ * an assembly language file.
+ */
+
+
+
+#ifndef _BACKDOOR_DEF_H_
+#define _BACKDOOR_DEF_H_
+
+#define INCLUDE_ALLOW_MODULE
+#define INCLUDE_ALLOW_USERLEVEL
+#define INCLUDE_ALLOW_VMMEXT
+#define INCLUDE_ALLOW_VMCORE
+#define INCLUDE_ALLOW_VMKERNEL
+#include "includeCheck.h"
+
+/*
+ * If you want to add a new low-level backdoor call for a guest userland
+ * application, please consider using the GuestRpc mechanism instead. --hpreg
+ */
+
+#define BDOOR_MAGIC 0x564D5868
+
+/* Low-bandwidth backdoor port. --hpreg */
+
+#define BDOOR_PORT 0x5658
+
+#define BDOOR_CMD_GETMHZ      		   1
+/*
+ * BDOOR_CMD_APMFUNCTION is used by:
+ *
+ * o The FrobOS code, which instead should either program the virtual chipset
+ *   (like the new BIOS code does, matthias offered to implement that), or not
+ *   use any VM-specific code (which requires that we correctly implement
+ *   "power off on CLI HLT" for SMP VMs, boris offered to implement that)
+ *
+ * o The old BIOS code, which will soon be jettisoned
+ *
+ *  --hpreg
+ */
+#define BDOOR_CMD_APMFUNCTION 		   2
+#define BDOOR_CMD_GETDISKGEO  		   3
+#define BDOOR_CMD_GETPTRLOCATION	      4
+#define BDOOR_CMD_SETPTRLOCATION	      5
+#define BDOOR_CMD_GETSELLENGTH		   6
+#define BDOOR_CMD_GETNEXTPIECE		   7
+#define BDOOR_CMD_SETSELLENGTH		   8
+#define BDOOR_CMD_SETNEXTPIECE		   9
+#define BDOOR_CMD_GETVERSION		      10
+#define BDOOR_CMD_GETDEVICELISTELEMENT	11
+#define BDOOR_CMD_TOGGLEDEVICE		   12
+#define BDOOR_CMD_GETGUIOPTIONS		   13
+#define BDOOR_CMD_SETGUIOPTIONS		   14
+#define BDOOR_CMD_GETSCREENSIZE		   15
+#define BDOOR_CMD_MONITOR_CONTROL       16
+#define BDOOR_CMD_GETHWVERSION          17
+#define BDOOR_CMD_OSNOTFOUND            18
+#define BDOOR_CMD_GETUUID               19
+#define BDOOR_CMD_GETMEMSIZE            20
+#define BDOOR_CMD_HOSTCOPY              21 /* Devel only */
+/* BDOOR_CMD_GETOS2INTCURSOR, 22, is very old and defunct. Reuse. */
+#define BDOOR_CMD_GETTIME               23 /* Deprecated. Use GETTIMEFULL. */
+#define BDOOR_CMD_STOPCATCHUP           24
+#define BDOOR_CMD_PUTCHR	        25 /* Devel only */
+#define BDOOR_CMD_ENABLE_MSG	        26 /* Devel only */
+#define BDOOR_CMD_GOTO_TCL	        27 /* Devel only */
+#define BDOOR_CMD_INITPCIOPROM		28
+#define BDOOR_CMD_INT13			29
+#define BDOOR_CMD_MESSAGE               30
+#define BDOOR_CMD_RSVD0                 31
+#define BDOOR_CMD_RSVD1                 32
+#define BDOOR_CMD_RSVD2                 33
+#define BDOOR_CMD_ISACPIDISABLED	34
+#define BDOOR_CMD_TOE			35 /* Not in use */
+/* BDOOR_CMD_INITLSIOPROM, 36, was merged with 28. Reuse. */
+#define BDOOR_CMD_PATCH_SMBIOS_STRUCTS  37
+#define BDOOR_CMD_MAPMEM                38 /* Devel only */
+#define BDOOR_CMD_ABSPOINTER_DATA	39
+#define BDOOR_CMD_ABSPOINTER_STATUS	40
+#define BDOOR_CMD_ABSPOINTER_COMMAND	41
+#define BDOOR_CMD_TIMER_SPONGE          42
+#define BDOOR_CMD_PATCH_ACPI_TABLES	43
+/* Catch-all to allow synchronous tests */
+#define BDOOR_CMD_DEVEL_FAKEHARDWARE	44 /* Debug only - needed in beta */
+#define BDOOR_CMD_GETHZ      		45
+#define BDOOR_CMD_GETTIMEFULL           46
+#define BDOOR_CMD_STATELOGGER           47
+#define BDOOR_CMD_CHECKFORCEBIOSSETUP	48
+#define BDOOR_CMD_LAZYTIMEREMULATION    49
+#define BDOOR_CMD_BIOSBBS               50
+#define BDOOR_CMD_MAX                   51
+
+/* 
+ * IMPORTANT NOTE: When modifying the behavior of an existing backdoor command,
+ * you must adhere to the semantics expected by the oldest Tools who use that
+ * command. Specifically, do not alter the way in which the command modifies 
+ * the registers. Otherwise backwards compatibility will suffer.
+ */
+
+/* High-bandwidth backdoor port. --hpreg */
+
+#define BDOORHB_PORT 0x5659
+
+#define BDOORHB_CMD_MESSAGE 0
+#define BDOORHB_CMD_MAX 1
+
+/*
+ * There is another backdoor which allows access to certain TSC-related
+ * values using otherwise illegal PMC indices when the pseudo_perfctr
+ * control flag is set.
+ */
+
+#define BDOOR_PMC_HW_TSC      0x10000
+#define BDOOR_PMC_REAL_NS     0x10001
+#define BDOOR_PMC_APPARENT_NS 0x10002
+
+#define IS_BDOOR_PMC(index)  (((index) | 3) == 0x10003)
+#define BDOOR_CMD(ecx)       ((ecx) & 0xffff)
+
+
+#ifdef VMM
+/*
+ *----------------------------------------------------------------------
+ *
+ * Backdoor_CmdRequiresFullyValidVCPU --
+ *
+ *    A few backdoor commands require the full VCPU to be valid
+ *    (including GDTR, IDTR, TR and LDTR). The rest get read/write
+ *    access to GPRs and read access to Segment registers (selectors).
+ *
+ * Result:
+ *    True iff VECX contains a command that require the full VCPU to
+ *    be valid.
+ *
+ *----------------------------------------------------------------------
+ */
+static INLINE Bool
+Backdoor_CmdRequiresFullyValidVCPU(unsigned cmd)
+{
+   return cmd == BDOOR_CMD_RSVD0 ||
+          cmd == BDOOR_CMD_RSVD1 ||
+          cmd == BDOOR_CMD_RSVD2;
+}
+#endif
+
+#endif
diff --git a/xen/arch/x86/hvm/vmport/guest_msg_def.h b/xen/arch/x86/hvm/vmport/guest_msg_def.h
new file mode 100644
index 0000000..44ae0fa
--- /dev/null
+++ b/xen/arch/x86/hvm/vmport/guest_msg_def.h
@@ -0,0 +1,87 @@
+/* **********************************************************
+ * Copyright 1998 VMware, Inc.  All rights reserved. 
+ * **********************************************************
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation version 2 and no later version.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
+ * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
+ * for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
+ */
+
+/*
+ * guest_msg_def.h --
+ *
+ *    Second layer of the internal communication channel between guest
+ *    applications and vmware
+ *
+ */
+
+#ifndef _GUEST_MSG_DEF_H_
+#define _GUEST_MSG_DEF_H_
+
+#define INCLUDE_ALLOW_MODULE
+#define INCLUDE_ALLOW_USERLEVEL
+#define INCLUDE_ALLOW_VMMEXT
+#include "includeCheck.h"
+
+
+/* Basic request types */
+typedef enum {
+   MESSAGE_TYPE_OPEN,
+   MESSAGE_TYPE_SENDSIZE,
+   MESSAGE_TYPE_SENDPAYLOAD,
+   MESSAGE_TYPE_RECVSIZE,
+   MESSAGE_TYPE_RECVPAYLOAD,
+   MESSAGE_TYPE_RECVSTATUS,
+   MESSAGE_TYPE_CLOSE,
+} MessageType;
+
+
+/* Reply statuses */
+/*  The basic request succeeded */
+#define MESSAGE_STATUS_SUCCESS  0x0001
+/*  vmware has a message available for its party */
+#define MESSAGE_STATUS_DORECV   0x0002
+/*  The channel has been closed */
+#define MESSAGE_STATUS_CLOSED   0x0004
+/*  vmware removed the message before the party fetched it */
+#define MESSAGE_STATUS_UNSENT   0x0008
+/*  A checkpoint occurred */
+#define MESSAGE_STATUS_CPT      0x0010
+/*  An underlying device is powering off */
+#define MESSAGE_STATUS_POWEROFF 0x0020
+/*  vmware has detected a timeout on the channel */
+#define MESSAGE_STATUS_TIMEOUT  0x0040
+/*  vmware supports high-bandwidth for sending and receiving the payload */
+#define MESSAGE_STATUS_HB       0x0080
+
+/*
+ * This mask defines the status bits that the guest is allowed to set;
+ * we use this to mask out all other bits when receiving the status
+ * from the guest. Otherwise, the guest can manipulate VMX state by
+ * setting status bits that are only supposed to be changed by the
+ * VMX. See bug 45385.
+ */
+#define MESSAGE_STATUS_GUEST_MASK    MESSAGE_STATUS_SUCCESS
+
+/*
+ * Max number of channels.
+ * Unfortunately this has to be public because the monitor part
+ * of the backdoor needs it for its trivial-case optimization. [greg]
+ */
+#define GUESTMSG_MAX_CHANNEL 8
+
+/* Flags to open a channel. --hpreg */
+#define GUESTMSG_FLAG_COOKIE 0x80000000
+#define GUESTMSG_FLAG_ALL GUESTMSG_FLAG_COOKIE
+
+
+#endif /* _GUEST_MSG_DEF_H_ */
-- 
1.8.4

Re: [PATCH v2 2/3] vmport: Add VMware provided include files.

[Jan Beulich]

>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> These 2 files: backdoor_def.h and guest_msg_def.h come from:
> 
> http://packages.vmware.com/tools/esx/3.5latest/rhel4/SRPMS/index.html 
>  open-vm-tools-kmod-7.4.8-396269.423167.src.rpm
>   open-vm-tools-kmod-7.4.8.tar.gz
>    vmhgfs/backdoor_def.h
>    vmhgfs/guest_msg_def.h
> 
> and are unchanged.

Okay, I can the value in keeping them as is even if they're in
conflict with some of our coding style rules. But ...

> --- /dev/null
> +++ b/xen/arch/x86/hvm/vmport/backdoor_def.h
> @@ -0,0 +1,167 @@
> +/* **********************************************************
> + * Copyright 1998 VMware, Inc.  All rights reserved. 
> + * **********************************************************
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of the GNU General Public License as published by the
> + * Free Software Foundation version 2 and no later version.
> + *
> + * This program is distributed in the hope that it will be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
> + * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
> + * for more details.
> + *
> + * You should have received a copy of the GNU General Public License along
> + * with this program; if not, write to the Free Software Foundation, Inc.,
> + * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
> + */
> +
> +/*
> + * backdoor_def.h --
> + *
> + * This contains backdoor defines that can be included from
> + * an assembly language file.
> + */
> +
> +
> +
> +#ifndef _BACKDOOR_DEF_H_
> +#define _BACKDOOR_DEF_H_

... these guards have the (theoretical at this point) risk of clashing
(apart from violating the C standard's naming rules) and ...

> +#define INCLUDE_ALLOW_MODULE
> +#define INCLUDE_ALLOW_USERLEVEL
> +#define INCLUDE_ALLOW_VMMEXT
> +#define INCLUDE_ALLOW_VMCORE
> +#define INCLUDE_ALLOW_VMKERNEL
> +#include "includeCheck.h"

... the patch is obviously incomplete without this header, the name
of which is certainly not in line with our way of naming files.

Jan

Re: [PATCH v2 2/3] vmport: Add VMware provided include files.

[Don Slutz]

On 09/02/14 03:34, Jan Beulich wrote:
>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>> These 2 files: backdoor_def.h and guest_msg_def.h come from:
>>
>> http://packages.vmware.com/tools/esx/3.5latest/rhel4/SRPMS/index.html
>>   open-vm-tools-kmod-7.4.8-396269.423167.src.rpm
>>    open-vm-tools-kmod-7.4.8.tar.gz
>>     vmhgfs/backdoor_def.h
>>     vmhgfs/guest_msg_def.h
>>
>> and are unchanged.
> Okay, I can the value in keeping them as is even if they're in
> conflict with some of our coding style rules. But ...
>
>> --- /dev/null
>> +++ b/xen/arch/x86/hvm/vmport/backdoor_def.h
>> @@ -0,0 +1,167 @@
>> +/* **********************************************************
>> + * Copyright 1998 VMware, Inc.  All rights reserved.
>> + * **********************************************************
>> + *
>> + * This program is free software; you can redistribute it and/or modify it
>> + * under the terms of the GNU General Public License as published by the
>> + * Free Software Foundation version 2 and no later version.
>> + *
>> + * This program is distributed in the hope that it will be useful, but
>> + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
>> + * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>> + * for more details.
>> + *
>> + * You should have received a copy of the GNU General Public License along
>> + * with this program; if not, write to the Free Software Foundation, Inc.,
>> + * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
>> + */
>> +
>> +/*
>> + * backdoor_def.h --
>> + *
>> + * This contains backdoor defines that can be included from
>> + * an assembly language file.
>> + */
>> +
>> +
>> +
>> +#ifndef _BACKDOOR_DEF_H_
>> +#define _BACKDOOR_DEF_H_
> ... these guards have the (theoretical at this point) risk of clashing
> (apart from violating the C standard's naming rules) and ...
>

Yes. That is why I had the comment:


/* Note: VMPORT_PORT and VMPORT_MAGIC is also defined as BDOOR_PORT
* and BDOOR_MAGIC in backdoor_def.h Defined in vmport.h also.
*/

because I did not want the reset of xen to be including this file.

If I have found the right "C standard's naming rule", this file is not
alone in violating it:

The following is the C standard's specification for naming the 
identifiers (C11 draft 
<http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf>):

*7.1.3 Reserved identifiers*

Each header declares or defines all identifiers listed in its associated 
subclause, and optionally declares or defines identifiers listed in its 
associated future library directions subclause and identifiers which are 
always reserved either for any use or for use as file scope identifiers.

— All identifiers that begin with an underscore and either an uppercase 
letter or another underscore are always reserved for any use.



For example:


xen/arch/x86/boot/video.h:#ifndef __BOOT_VIDEO_H__
xen/arch/arm/vtimer.h:#ifndef __ARCH_ARM_VTIMER_H__
xen/arch/x86/cpu/mcheck/barrier.h:#ifndef _MCHECK_BARRIER_H
tools/xenstore/talloc.h:#ifndef _TALLOC_H_




>> +#define INCLUDE_ALLOW_MODULE
>> +#define INCLUDE_ALLOW_USERLEVEL
>> +#define INCLUDE_ALLOW_VMMEXT
>> +#define INCLUDE_ALLOW_VMCORE
>> +#define INCLUDE_ALLOW_VMKERNEL
>> +#include "includeCheck.h"
> ... the patch is obviously incomplete without this header, the name
> of which is certainly not in line with our way of naming files.

Yes. And an empty file of that name is added in the next patch.


So I am getting the feeling is that it would be better to have a patch
adjusting these include files to better fit in xen, and then move them
out of the new xen/arch/x86/hvm/vmware directory.

In any case I would continue with this patch to provide the history of
where the include file came from.

-Don Slutz
> Jan
>

Re: [PATCH v2 2/3] vmport: Add VMware provided include files.

[Jan Beulich]

>>> On 02.09.14 at 20:46, <dslutz@verizon.com> wrote:
> On 09/02/14 03:34, Jan Beulich wrote:
>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>> These 2 files: backdoor_def.h and guest_msg_def.h come from:
>>>
>>> http://packages.vmware.com/tools/esx/3.5latest/rhel4/SRPMS/index.html 
>>>   open-vm-tools-kmod-7.4.8-396269.423167.src.rpm
>>>    open-vm-tools-kmod-7.4.8.tar.gz
>>>     vmhgfs/backdoor_def.h
>>>     vmhgfs/guest_msg_def.h
>>>
>>> and are unchanged.
>> Okay, I can the value in keeping them as is even if they're in
>> conflict with some of our coding style rules. But ...
>>
>>> --- /dev/null
>>> +++ b/xen/arch/x86/hvm/vmport/backdoor_def.h
>>> @@ -0,0 +1,167 @@
>>> +/* **********************************************************
>>> + * Copyright 1998 VMware, Inc.  All rights reserved.
>>> + * **********************************************************
>>> + *
>>> + * This program is free software; you can redistribute it and/or modify it
>>> + * under the terms of the GNU General Public License as published by the
>>> + * Free Software Foundation version 2 and no later version.
>>> + *
>>> + * This program is distributed in the hope that it will be useful, but
>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
>>> + * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>>> + * for more details.
>>> + *
>>> + * You should have received a copy of the GNU General Public License along
>>> + * with this program; if not, write to the Free Software Foundation, Inc.,
>>> + * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
>>> + */
>>> +
>>> +/*
>>> + * backdoor_def.h --
>>> + *
>>> + * This contains backdoor defines that can be included from
>>> + * an assembly language file.
>>> + */
>>> +
>>> +
>>> +
>>> +#ifndef _BACKDOOR_DEF_H_
>>> +#define _BACKDOOR_DEF_H_
>> ... these guards have the (theoretical at this point) risk of clashing
>> (apart from violating the C standard's naming rules) and ...
>>
> 
> Yes. That is why I had the comment:
> 
> 
> /* Note: VMPORT_PORT and VMPORT_MAGIC is also defined as BDOOR_PORT
> * and BDOOR_MAGIC in backdoor_def.h Defined in vmport.h also.
> */
> 
> because I did not want the reset of xen to be including this file.

I can't connect your response to my comment.

> If I have found the right "C standard's naming rule", this file is not
> alone in violating it:

Yes, that's the one. And no, bad examples elsewhere aren't an
excuse to introduce more bad code.

>>> +#define INCLUDE_ALLOW_MODULE
>>> +#define INCLUDE_ALLOW_USERLEVEL
>>> +#define INCLUDE_ALLOW_VMMEXT
>>> +#define INCLUDE_ALLOW_VMCORE
>>> +#define INCLUDE_ALLOW_VMKERNEL
>>> +#include "includeCheck.h"
>> ... the patch is obviously incomplete without this header, the name
>> of which is certainly not in line with our way of naming files.
> 
> Yes. And an empty file of that name is added in the next patch.

Question is - why not here (to make the set self consistent)?

> So I am getting the feeling is that it would be better to have a patch
> adjusting these include files to better fit in xen, and then move them
> out of the new xen/arch/x86/hvm/vmware directory.

Not sure what makes you think so - as said earlier on, I can see
the value of having the file mostly (if not entirely) identical to
what is coming from VMware. That way, eventual future updates
are going to be easier. All we need to avoid are eventual future
naming clashes. Of course, if the headers are so badly written
that the majority of their defines has a risk of clashing, then of
course it might indeed be preferable to redo them in Xen style.

Jan

Re: [PATCH v2 2/3] vmport: Add VMware provided include files.

[Don Slutz]

On 09/03/14 03:51, Jan Beulich wrote:
>>>> On 02.09.14 at 20:46, <dslutz@verizon.com> wrote:
>> On 09/02/14 03:34, Jan Beulich wrote:
>>>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>>>> These 2 files: backdoor_def.h and guest_msg_def.h come from:
>>>>
>>>> http://packages.vmware.com/tools/esx/3.5latest/rhel4/SRPMS/index.html
>>>>    open-vm-tools-kmod-7.4.8-396269.423167.src.rpm
>>>>     open-vm-tools-kmod-7.4.8.tar.gz
>>>>      vmhgfs/backdoor_def.h
>>>>      vmhgfs/guest_msg_def.h
>>>>
>>>> and are unchanged.
>>> Okay, I can the value in keeping them as is even if they're in
>>> conflict with some of our coding style rules. But ...
>>>
>>>> --- /dev/null
>>>> +++ b/xen/arch/x86/hvm/vmport/backdoor_def.h
>>>> @@ -0,0 +1,167 @@
>>>> +/* **********************************************************
>>>> + * Copyright 1998 VMware, Inc.  All rights reserved.
>>>> + * **********************************************************
>>>> + *
>>>> + * This program is free software; you can redistribute it and/or modify it
>>>> + * under the terms of the GNU General Public License as published by the
>>>> + * Free Software Foundation version 2 and no later version.
>>>> + *
>>>> + * This program is distributed in the hope that it will be useful, but
>>>> + * WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
>>>> + * or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License
>>>> + * for more details.
>>>> + *
>>>> + * You should have received a copy of the GNU General Public License along
>>>> + * with this program; if not, write to the Free Software Foundation, Inc.,
>>>> + * 51 Franklin St, Fifth Floor, Boston, MA  02110-1301 USA
>>>> + */
>>>> +
>>>> +/*
>>>> + * backdoor_def.h --
>>>> + *
>>>> + * This contains backdoor defines that can be included from
>>>> + * an assembly language file.
>>>> + */
>>>> +
>>>> +
>>>> +
>>>> +#ifndef _BACKDOOR_DEF_H_
>>>> +#define _BACKDOOR_DEF_H_
>>> ... these guards have the (theoretical at this point) risk of clashing
>>> (apart from violating the C standard's naming rules) and ...
>>>
>> Yes. That is why I had the comment:
>>
>>
>> /* Note: VMPORT_PORT and VMPORT_MAGIC is also defined as BDOOR_PORT
>> * and BDOOR_MAGIC in backdoor_def.h Defined in vmport.h also.
>> */
>>
>> because I did not want the reset of xen to be including this file.
> I can't connect your response to my comment.

Reading it a again, I agree.  s/reset/rest/ helps a little.  Hope this
is better:


I put these files in the new vmware directory in order to prevent other
parts of xen from attempting to use these include files.  This is a work
around for any possible issue with them.  In v2 I had defined in a different
file the 2 numbers that are used else where.  It now looks like in v3 I
can fix this by having all uses contained in this new directory.

>> If I have found the right "C standard's naming rule", this file is not
>> alone in violating it:
> Yes, that's the one. And no, bad examples elsewhere aren't an
> excuse to introduce more bad code.

Fine with me.  I was also attempting to understand how it would need
to change.

>>>> +#define INCLUDE_ALLOW_MODULE
>>>> +#define INCLUDE_ALLOW_USERLEVEL
>>>> +#define INCLUDE_ALLOW_VMMEXT
>>>> +#define INCLUDE_ALLOW_VMCORE
>>>> +#define INCLUDE_ALLOW_VMKERNEL
>>>> +#include "includeCheck.h"
>>> ... the patch is obviously incomplete without this header, the name
>>> of which is certainly not in line with our way of naming files.
>> Yes. And an empty file of that name is added in the next patch.
> Question is - why not here (to make the set self consistent)?

I see no reason not to.  So I have added it here.

>> So I am getting the feeling is that it would be better to have a patch
>> adjusting these include files to better fit in xen, and then move them
>> out of the new xen/arch/x86/hvm/vmware directory.
> Not sure what makes you think so - as said earlier on, I can see
> the value of having the file mostly (if not entirely) identical to
> what is coming from VMware. That way, eventual future updates
> are going to be easier. All we need to avoid are eventual future
> naming clashes. Of course, if the headers are so badly written
> that the majority of their defines has a risk of clashing, then of
> course it might indeed be preferable to redo them in Xen style.

You are the 2nd person to point out issues with these files.  So I guess
I was jumping the gun in saying to change them and move them.  In any
case if I was to do the smallest change to make them better that would
be a 2nd patch.

Thinking more about it, it makes more sense to leave them in the vmware
directory.

     -Don Slutz


> Jan
>

[PATCH v2 3/3] Add limited support of VMware's hyper-call

[Don Slutz]

This is both a more complete support then in currently provided by
QEMU and/or KVM and less.  The missing part requires QEMU changes
and has been left out until the QEMU patches are accepted upstream.

VMware's hyper-call is also known as VMware Backdoor I/O Port.

Note: this support does not depend on vmware_hw being non-zero.

Summary is that VMware treats "IN EAX, DX" (or "OUT DX, EAX"; or
"inl %dx, %eax" in AT&T syntax ) to port 0x5658 specially.  Note:
since many operations return data in EAX, "OUT DX, EAX" does not
work for them on VMware, I did not support the "OUT DX, EAX", but it
would not be hard to add.

Also this instruction is allowed to be used from ring 3.  To
support this the vmexit for GP needs to be enabled.  I have not
fully tested that nested HVM is doing the right thing for this.

An open source example of using this is:

http://open-vm-tools.sourceforge.net/

Which only uses "IN EAX, DX".  Also

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1009458

Lists just "inl (%%dx)" (I assume this is AT&T syntax and is the
same as "inl %dx, %eax").

The support included is enough to allow VMware tools to install in a
HVM domU and provide guestinfo support.  guestinfo support is
provide by what is known as VMware RPC support.  This guestinfo
support is provided via libxc.  libxl support has not be written.

Note: VMware RPC support is only available on HVM domU.

This interface is an extension of __HYPERVISOR_HVM_op.  It was
picked because xc_get_hvm_param() also uses it and VMware guest
info is a lot like a hvm param.

The HVMOP_get_vmport_guest_info is used by two libxc functions,
xc_get_vmport_guest_info and xc_fetch_vmport_guest_info.
xc_fetch_vmport_guest_info is designed to be used to fetch all
currently set guestinfo values.

To save on hypervisor heap memory, the guestinfo support in done in
two sizes, normal and jumbo.  Normal is used to handle up to 128
byte values and jumbo is used to handle up to 4096 byte values.

Since all this is work is done when the guest is doing a single
instruction; it was designed to not use the hypervisor heap to
allocate the memory at this time.  Instead a few are allocated at
the create domain time and during the xen's hyper-call to get or set
them.  This was picked in that if a tool stack is using the VMware
guest info support, it should be using either of both of the get and
set.  And so in this case the guest should only see an out of memory
error when the compile max amount of hypervisor heap memory is in
use.

Doing it this way does lead to a lot of pointer use and many
sub structures.

If the domU is running VMware tools, then the "build version" of
the tools is also available via xc_get_HVM_param().  This also
enables the use of new triggers that will use the VMware hyper-call
to do some limited control of the domU.  The most useful are
poweroff and reboot.  Since a guest process needs to be running
for these to work, a tool stack should check that the build version
is non zero before assuming these will work.

The 2 hvm param's HVM_PARAM_VMPORT_BUILD_NUMBER_TIME and
HVM_PARAM_VMPORT_BUILD_NUMBER_VALUE are how "build version" is
accessed.  These 2 params are only allowed to be set to zero.  The
HVM_PARAM_VMPORT_BUILD_NUMBER_TIME can be used to track the last
time the VMware tools in the guest responded.  One such use would
be the health of the tools in the guest.  The hvm param
HVM_PARAM_VMPORT_RESET_TIME controls how often to request them in
seconds minus 1.  The minus 1 is to handle to 0 case.  I.E. the
fastest that can be selected is every second.  The default is 4
times a minute.

The VMware RPC support includes the notion of channels that are
opened, active and closed.  All RPC messages sent via a channel
starts with normal ASCII text.  The message some times does include
binary data.

Currently there are 2 protocols defined for VMware RPC.  They
determine the direction for data flow, domU to tool stack or
tool stack to domU.

There is no provided interrupt for VMware RPC.

The VMware's hyper-call state is included in live migration and
save/restore.  Because the max size of the VMware guestinfo is
large, then data is compressed and expanded in the
vmport_save_domain_ctxt and vmport_load_domain_ctxt.

For a debug=y build there is a new command line option
vmport_debug=.  It enabled output to the console of various
stages of handling the "IN EAX, DX" instruction.  Most uses
are the summary ones that show complete RPC actions.

Signed-off-by: Don Slutz <dslutz@verizon.com>
---
 tools/libxc/xc_domain.c                      |  115 +++
 tools/libxc/xenctrl.h                        |   24 +
 tools/misc/xen-hvmctx.c                      |  229 ++++
 tools/xentrace/formats                       |    8 +
 xen/arch/x86/domctl.c                        |   34 +
 xen/arch/x86/hvm/Makefile                    |    1 +
 xen/arch/x86/hvm/hvm.c                       |  480 ++++++++-
 xen/arch/x86/hvm/io.c                        |    1 +
 xen/arch/x86/hvm/svm/emulate.c               |    3 +
 xen/arch/x86/hvm/svm/svm.c                   |   79 ++
 xen/arch/x86/hvm/svm/vmcb.c                  |    1 +
 xen/arch/x86/hvm/vmport/Makefile             |    1 +
 xen/arch/x86/hvm/vmport/includeCheck.h       |   17 +
 xen/arch/x86/hvm/vmport/vmport.c             | 1436 ++++++++++++++++++++++++++
 xen/arch/x86/hvm/vmx/vmcs.c                  |    1 +
 xen/arch/x86/hvm/vmx/vmx.c                   |   90 ++
 xen/arch/x86/hvm/vmx/vvmx.c                  |   14 +
 xen/include/asm-x86/hvm/domain.h             |    4 +
 xen/include/asm-x86/hvm/svm/emulate.h        |    1 +
 xen/include/asm-x86/hvm/trace.h              |   31 +
 xen/include/asm-x86/hvm/vmport.h             |   90 ++
 xen/include/public/arch-x86/hvm/save.h       |   39 +-
 xen/include/public/arch-x86/hvm/vmporttype.h |  105 ++
 xen/include/public/domctl.h                  |    3 +
 xen/include/public/hvm/hvm_op.h              |   18 +
 xen/include/public/hvm/params.h              |    5 +-
 xen/include/public/trace.h                   |    7 +
 27 files changed, 2834 insertions(+), 3 deletions(-)
 create mode 100644 xen/arch/x86/hvm/vmport/Makefile
 create mode 100644 xen/arch/x86/hvm/vmport/includeCheck.h
 create mode 100644 xen/arch/x86/hvm/vmport/vmport.c
 create mode 100644 xen/include/asm-x86/hvm/vmport.h
 create mode 100644 xen/include/public/arch-x86/hvm/vmporttype.h

diff --git a/tools/libxc/xc_domain.c b/tools/libxc/xc_domain.c
index c67ac9a..24a72e9 100644
--- a/tools/libxc/xc_domain.c
+++ b/tools/libxc/xc_domain.c
@@ -1577,6 +1577,121 @@ int xc_hvm_set_ioreq_server_state(xc_interface *xch,
     return rc;
 }
 
+int xc_set_vmport_guest_info(xc_interface *handle,
+                             domid_t dom,
+                             unsigned int key_len,
+                             char *key,
+                             unsigned int val_len,
+                             char *val)
+{
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_vmport_guest_info_t, arg);
+    int rc;
+
+    if ( (key_len < 1) ||
+        (key_len > VMPORT_GUEST_INFO_KEY_MAX) ||
+        (val_len > VMPORT_GUEST_INFO_VAL_MAX) )
+        return -1;
+
+    arg = xc_hypercall_buffer_alloc(handle, arg, sizeof(*arg));
+    if ( arg == NULL )
+        return -1;
+
+    hypercall.op     = __HYPERVISOR_hvm_op;
+    hypercall.arg[0] = HVMOP_set_vmport_guest_info;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+    arg->domid = dom;
+    arg->key_length = key_len;
+    arg->value_length = val_len;
+    memcpy(arg->data, key, key_len);
+    memcpy(&arg->data[key_len], val, val_len);
+    rc = do_xen_hypercall(handle, &hypercall);
+    xc_hypercall_buffer_free(handle, arg);
+    return rc;
+}
+
+int xc_get_vmport_guest_info(xc_interface *handle,
+                             domid_t dom,
+                             unsigned int key_len,
+                             char *key,
+                             unsigned int val_max,
+                             unsigned int *val_len,
+                             char *val)
+{
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_vmport_guest_info_t, arg);
+    int rc;
+
+    if ( (key_len < 1) ||
+        (key_len > VMPORT_GUEST_INFO_KEY_MAX) )
+        return -1;
+
+    arg = xc_hypercall_buffer_alloc(handle, arg, sizeof(*arg));
+
+    hypercall.op     = __HYPERVISOR_hvm_op;
+    hypercall.arg[0] = HVMOP_get_vmport_guest_info;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+    arg->domid = dom;
+    arg->key_length = key_len;
+    arg->value_length = 0;
+    *val_len = 0;
+    memcpy(arg->data, key, key_len);
+    rc = do_xen_hypercall(handle, &hypercall);
+    if ( rc == 0 )
+    {
+        *val_len = arg->value_length;
+        if ( arg->value_length > val_max )
+            arg->value_length = val_max;
+        memcpy(val, &arg->data[key_len], arg->value_length);
+    }
+    xc_hypercall_buffer_free(handle, arg);
+    return rc;
+}
+
+int xc_fetch_vmport_guest_info(xc_interface *handle,
+                               domid_t dom,
+                               unsigned int idx,
+                               unsigned int key_max,
+                               unsigned int *key_len,
+                               char *key,
+                               unsigned int val_max,
+                               unsigned int *val_len,
+                               char *val)
+{
+    DECLARE_HYPERCALL;
+    DECLARE_HYPERCALL_BUFFER(xen_hvm_vmport_guest_info_t, arg);
+    int rc;
+
+    arg = xc_hypercall_buffer_alloc(handle, arg, sizeof(*arg));
+
+    hypercall.op     = __HYPERVISOR_hvm_op;
+    hypercall.arg[0] = HVMOP_get_vmport_guest_info;
+    hypercall.arg[1] = HYPERCALL_BUFFER_AS_ARG(arg);
+    arg->domid = dom;
+    arg->key_length = 0;
+    arg->value_length = idx;
+    *key_len = 0;
+    *val_len = 0;
+    rc = do_xen_hypercall(handle, &hypercall);
+    if ( rc == 0 )
+    {
+        uint16_t val_off = arg->key_length;
+
+        *key_len = arg->key_length;
+        if ( arg->key_length > key_max )
+            arg->key_length = key_max;
+        memcpy(key, arg->data, arg->key_length);
+        *val_len = arg->value_length;
+        if ( arg->value_length > val_max )
+            arg->value_length = val_max;
+        memcpy(val,
+               &arg->data[val_off],
+               arg->value_length);
+    }
+    xc_hypercall_buffer_free(handle, arg);
+    return rc;
+}
+
 int xc_domain_setdebugging(xc_interface *xch,
                            uint32_t domid,
                            unsigned int enable)
diff --git a/tools/libxc/xenctrl.h b/tools/libxc/xenctrl.h
index 1c5d0db..6af3fd2 100644
--- a/tools/libxc/xenctrl.h
+++ b/tools/libxc/xenctrl.h
@@ -2009,6 +2009,30 @@ int xc_hvm_destroy_ioreq_server(xc_interface *xch,
                                 domid_t domid,
                                 ioservid_t id);
 
+int xc_set_vmport_guest_info(xc_interface *handle,
+                             domid_t dom,
+                             unsigned int key_len,
+                             char *key,
+                             unsigned int val_len,
+                             char *val);
+int xc_get_vmport_guest_info(xc_interface *handle,
+                             domid_t dom,
+                             unsigned int key_len,
+                             char *key,
+                             unsigned int val_max,
+                             unsigned int *val_len,
+                             char *val);
+int xc_fetch_vmport_guest_info(xc_interface *handle,
+                               domid_t dom,
+                               unsigned int idx,
+                               unsigned int key_max,
+                               unsigned int *key_len,
+                               char *key,
+                               unsigned int val_max,
+                               unsigned int *val_len,
+                               char *val);
+
+
 /* HVM guest pass-through */
 int xc_assign_device(xc_interface *xch,
                      uint32_t domid,
diff --git a/tools/misc/xen-hvmctx.c b/tools/misc/xen-hvmctx.c
index 5a69245..1abd2be 100644
--- a/tools/misc/xen-hvmctx.c
+++ b/tools/misc/xen-hvmctx.c
@@ -399,6 +399,225 @@ static void dump_tsc_adjust(void)
     printf("    TSC_ADJUST: tsc_adjust %" PRIx64 "\n", p.tsc_adjust);
 }
 
+static void dump_vmport(int vmport_size)
+{
+    int i;
+    HVM_SAVE_TYPE(VMPORT) *vp;
+    int64_t vmport_guestsize;
+    uint32_t vmport_used_guestinfo;
+    uint32_t vmport_used_guestinfo_jumbo;
+    uint8_t pb[vmport_size];
+    char *p;
+    int chans_size;
+
+    READ(pb);
+    vp = (void *)&pb;
+
+    p = vp->u.packed.packed_data;
+
+    vmport_guestsize = vp->used_guestsize;
+    chans_size = vmport_size - vmport_guestsize - (p - (char *)pb);
+    if ( chans_size < 0 )
+    {
+        fprintf(stderr, "*** VMPORT: bogus chans_size=%d should be >= 0\n"
+                "   vmport_size=%d vmport_guestsize=%d fixed_size=%ld\n",
+                chans_size, vmport_size, (int)vmport_guestsize,
+                p - (char *)pb);
+        chans_size = 0;
+    }
+
+    printf("    VMPORT: ping_time %" PRIu64 "\n", vp->ping_time);
+    printf("    VMPORT: open_cookie %" PRIx32 "\n", vp->open_cookie);
+    for (i = 0; i < VMPORT_MAX_CHANS; i++)
+    {
+        int j;
+        vmport_channel_control_t *vc = &vp->u.packed.chan_ctl[i].chan;
+        vmport_bucket_control_t *jb =  &vp->u.packed.chan_ctl[i].jumbo;
+
+        printf("    VMPORT: chan[%d] chan_id %d\n", i, vc->chan_id);
+        printf("    VMPORT: chan[%d] active_time %" PRIx64 "\n",
+               i, vc->active_time);
+        printf("    VMPORT: chan[%d] proto_num %" PRIx32 "\n",
+               i, vc->proto_num);
+        printf("    VMPORT: chan[%d] recv_read %d\n", i, vc->recv_read);
+        printf("    VMPORT: chan[%d] recv_write %d\n", i, vc->recv_write);
+        printf("    VMPORT: chan[%d] jumbo %d\n", i, vc->jumbo);
+        printf("    VMPORT: chan[%d] send_len %d\n", i, vc->send_len);
+        if ( vc->send_len > VMPORT_MAX_SEND_BUF * 4 )
+        {
+            printf("--- VMPORT: trucated send_len=%d > %d\n",
+                   vc->send_len, VMPORT_MAX_SEND_BUF * 4);
+            vc->send_len = VMPORT_MAX_SEND_BUF * 4;
+        }
+        if ( vc->send_len > chans_size )
+        {
+            fprintf(stderr, "*** VMPORT: bogus send_len=%d > %d\n",
+                    vc->send_len, chans_size);
+            if ( chans_size >= 0 )
+                vc->send_len = chans_size;
+            else
+                vc->send_len = 0;
+        }
+        p += vc->send_len;
+        chans_size -= vc->send_len;
+        for (j = 0; j < VMPORT_MAX_BKTS; j++)
+        {
+            vmport_bucket_control_t *b = &vp->u.packed.chan_ctl[i].recv[j];
+
+            printf("    VMPORT: chan[%d] bucket[%d] recv_len %d\n",
+                   i, j, b->recv_len);
+            if ( b->recv_len > VMPORT_MAX_RECV_BUF * 4 )
+            {
+                printf("--- VMPORT: trucated recv_len=%d > %d\n",
+                       b->recv_len, VMPORT_MAX_RECV_BUF * 4);
+                b->recv_len = VMPORT_MAX_RECV_BUF * 4;
+            }
+            if ( b->recv_len > chans_size )
+            {
+                fprintf(stderr, "*** VMPORT: bogus recv_len=%d > %d\n",
+                        b->recv_len, chans_size);
+                if ( chans_size >= 0 )
+                    b->recv_len = chans_size;
+                else
+                    b->recv_len = 0;
+            }
+            p += b->recv_len;
+            chans_size -= b->recv_len;
+        }
+        printf("    VMPORT: chan[%d] jumbo_bkt recv_len %d\n", i, jb->recv_len);
+        if ( jb->recv_len > VMPORT_MAX_RECV_JUMBO_BUF * 4 )
+        {
+            printf("--- VMPORT: trucated recv_len=%d > %d\n",
+                   jb->recv_len, VMPORT_MAX_RECV_JUMBO_BUF * 4);
+            jb->recv_len = VMPORT_MAX_RECV_JUMBO_BUF * 4;
+        }
+        if ( jb->recv_len > chans_size )
+        {
+            fprintf(stderr, "*** VMPORT: bogus recv_len=%d > %d\n",
+                    jb->recv_len, chans_size);
+            if ( chans_size >= 0 )
+                jb->recv_len = chans_size;
+            else
+                jb->recv_len = 0;
+        }
+        p += jb->recv_len;
+        chans_size -= jb->recv_len;
+    }
+
+    if ( chans_size != 0 )
+        fprintf(stderr, "*** VMPORT: bogus chans_size=%d should be 0\n",
+                chans_size);
+
+    vmport_used_guestinfo = vp->used_guestinfo;
+    vmport_used_guestinfo_jumbo = vp->used_guestinfo_jumbo;
+
+    if ( vmport_used_guestinfo == 0 )
+        printf("    VMPORT: no small data\n");
+    for (i = 0; i < vmport_used_guestinfo; i++)
+    {
+        if ( vmport_guestsize > 0 )
+        {
+            uint8_t key_len = (uint8_t)(*p++);
+            uint8_t val_len = (uint8_t)(*p++);
+            if ( key_len )
+            {
+                char key[VMPORT_MAX_KEY_LEN + 1];
+                char val[VMPORT_MAX_VAL_LEN + 1];
+
+                if ( key_len > VMPORT_MAX_KEY_LEN )
+                {
+                    fprintf(stderr,
+                            "*** VMPORT: bogus key_len=%d > %d for guestinfo[%d]\n",
+                            key_len, VMPORT_MAX_KEY_LEN, i);
+                    key_len = VMPORT_MAX_KEY_LEN;
+                }
+                memcpy(key, p, key_len);
+                p += key_len;
+                key[key_len] = '\0';
+                if ( val_len > VMPORT_MAX_VAL_LEN )
+                {
+                    fprintf(stderr,
+                            "*** VMPORT: bogus val_len=%d > %d for guestinfo[%d]\n",
+                            val_len, VMPORT_MAX_VAL_LEN, i);
+                    val_len = VMPORT_MAX_VAL_LEN;
+                }
+                memcpy(val, p, val_len);
+                p += val_len;
+                val[val_len] = '\0';
+                vmport_guestsize -= 2 + key_len + val_len;
+                printf("    VMPORT: guestinfo[%d](%s) = \"%s\"\n",
+                       i, key, val);
+            }
+            else
+            {
+                fprintf(stderr,
+                        "*** VMPORT: bogus len for guestinfo[%d]\n",
+                        i);
+                vmport_guestsize -= 2;
+            }
+            if ( vmport_guestsize < 0 )
+                printf("    VMPORT: data length skew at guestinfo[%d]\n"
+                       "         remaining datasize=%ld\n",
+                       i, vmport_guestsize);
+        }
+    }
+
+    if ( vmport_guestsize == 0 )
+        printf("    VMPORT: no jumbo data\n");
+    for (i = 0; i < vmport_used_guestinfo_jumbo; i++)
+    {
+        if ( vmport_guestsize > 0 )
+        {
+            uint8_t key_len = (uint8_t)(*p++);
+            uint16_t val_len;
+
+            memcpy(&val_len, p, 2);
+            p += 2;
+            if ( key_len )
+            {
+                char key[VMPORT_MAX_KEY_LEN + 1];
+                char val[VMPORT_MAX_VAL_JUMBO_LEN + 1];
+
+                if ( key_len > VMPORT_MAX_KEY_LEN )
+                {
+                    fprintf(stderr,
+                            "*** VMPORT: bogus key_len=%d > %d for guestinfo[%d]\n",
+                            key_len, VMPORT_MAX_KEY_LEN, i);
+                    key_len = VMPORT_MAX_KEY_LEN;
+                }
+                memcpy(key, p, key_len);
+                p += key_len;
+                key[key_len] = '\0';
+                if ( val_len > VMPORT_MAX_VAL_JUMBO_LEN )
+                {
+                    fprintf(stderr,
+                            "*** VMPORT: bogus val_len=%d > %d for guestinfo[%d]\n",
+                            val_len, VMPORT_MAX_VAL_JUMBO_LEN, i);
+                    val_len = VMPORT_MAX_VAL_JUMBO_LEN;
+                }
+                memcpy(val, p, val_len);
+                p += val_len;
+                val[val_len] = '\0';
+                vmport_guestsize -= 2 + key_len + val_len;
+                printf("    VMPORT: guestinfo_jumbo[%d](%s) = \"%s\"\n",
+                       i, key, val);
+            }
+            else
+            {
+                printf("    VMPORT: bogus len for guestinfo_jumbo[%d]\n", i);
+                vmport_guestsize -= 2;
+            }
+            if ( vmport_guestsize < 0 )
+                printf("    VMPORT: data length skew at guestinfo_jumbo[%d]\n"
+                       "         remaining datasize=%ld\n", i,
+                       vmport_guestsize);
+        }
+    }
+
+    if ( !vmport_guestsize )
+        printf("    VMPORT: %ld bytes leftover data\n", vmport_guestsize);
+}
+
 int main(int argc, char **argv)
 {
     int entry, domid;
@@ -467,6 +686,7 @@ int main(int argc, char **argv)
         case HVM_SAVE_CODE(VIRIDIAN_VCPU): dump_viridian_vcpu(); break;
         case HVM_SAVE_CODE(VMCE_VCPU): dump_vmce_vcpu(); break;
         case HVM_SAVE_CODE(TSC_ADJUST): dump_tsc_adjust(); break;
+        case HVM_SAVE_CODE(VMPORT): dump_vmport(desc.length); break;
         case HVM_SAVE_CODE(END): break;
         default:
             printf(" ** Don't understand type %u: skipping\n",
@@ -477,3 +697,12 @@ int main(int argc, char **argv)
 
     return 0;
 } 
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/tools/xentrace/formats b/tools/xentrace/formats
index da658bf..75a5d34 100644
--- a/tools/xentrace/formats
+++ b/tools/xentrace/formats
@@ -79,6 +79,14 @@
 0x00082020  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  INTR_WINDOW [ value = 0x%(1)08x ]
 0x00082021  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  NPF         [ gpa = 0x%(2)08x%(1)08x mfn = 0x%(4)08x%(3)08x qual = 0x%(5)04x p2mt = 0x%(6)04x ]
 0x00082023  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  TRAP        [ vector = 0x%(1)02x ]
+0x00082024  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  TRAP_DEBUG  [ exit_qualification = 0x%(1)08x ]
+0x00082025  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  VLAPIC
+0x00082026  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  TRAP_GP     [ vector = 0x%(1)02x inst_len = %(2)d exitinfo1 = %(3)d exitinfo2 = 0x%(4)08x ]
+0x00082126  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  TRAP_GP     [ vector = 0x%(1)02x inst_len = %(2)d exitinfo1 = %(3)d exitinfo2 = 0x%(5)08x%(4)08x ]
+0x00082027  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  VMPORT_READ1  [ eax = 0x%(1)08x ebx = 0x%(2)08x ecx = 0x%(3)08x edx = 0x%(4)08x esi = 0x%(5)08x edi = 0x%(6)08x ]
+0x00082227  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  VMPORT_WRITE1 [ eax = 0x%(1)08x ebx = 0x%(2)08x ecx = 0x%(3)08x edx = 0x%(4)08x esi = 0x%(5)08x edi = 0x%(6)08x ]
+0x00082028  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  VMPORT_READ2  [ eax = 0x%(1)08x ebx = 0x%(2)08x ecx = 0x%(3)08x edx = 0x%(4)08x esi = 0x%(5)08x edi = 0x%(6)08x ]
+0x00082228  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  VMPORT_WRITE2 [ eax = 0x%(1)08x ebx = 0x%(2)08x ecx = 0x%(3)08x edx = 0x%(4)08x esi = 0x%(5)08x edi = 0x%(6)08x ]
 
 0x0010f001  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_map      [ domid = %(1)d ]
 0x0010f002  CPU%(cpu)d  %(tsc)d (+%(reltsc)8d)  page_grant_unmap    [ domid = %(1)d ]
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index d1517c4..bee2b30 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -23,6 +23,7 @@
 #include <xen/paging.h>
 #include <asm/irq.h>
 #include <asm/hvm/hvm.h>
+#include <asm/hvm/vmport.h>
 #include <asm/hvm/support.h>
 #include <asm/hvm/cacheattr.h>
 #include <asm/processor.h>
@@ -579,6 +580,39 @@ long arch_do_domctl(
         }
         break;
 
+        case XEN_DOMCTL_SENDTRIGGER_VTPOWER:
+        {
+            ret = -EINVAL;
+            if ( is_hvm_domain(d) )
+            {
+                ret = 0;
+                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Halt");
+            }
+        }
+        break;
+
+        case XEN_DOMCTL_SENDTRIGGER_VTREBOOT:
+        {
+            ret = -EINVAL;
+            if ( is_hvm_domain(d) )
+            {
+                ret = 0;
+                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Reboot");
+            }
+        }
+        break;
+
+        case XEN_DOMCTL_SENDTRIGGER_VTPING:
+        {
+            ret = -EINVAL;
+            if ( is_hvm_domain(d) )
+            {
+                ret = 0;
+                vmport_ctrl_send(&d->arch.hvm_domain, "ping");
+            }
+        }
+        break;
+
         default:
             ret = -ENOSYS;
         }
diff --git a/xen/arch/x86/hvm/Makefile b/xen/arch/x86/hvm/Makefile
index 3474a65..8e630db 100644
--- a/xen/arch/x86/hvm/Makefile
+++ b/xen/arch/x86/hvm/Makefile
@@ -1,5 +1,6 @@
 subdir-y += svm
 subdir-y += vmx
+subdir-y += vmport
 
 obj-y += asid.o
 obj-y += emulate.o
diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index eded446..3e5ac67 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -57,6 +57,7 @@
 #include <asm/hvm/cacheattr.h>
 #include <asm/hvm/trace.h>
 #include <asm/hvm/nestedhvm.h>
+#include <asm/hvm/vmport.h>
 #include <asm/mtrr.h>
 #include <asm/apic.h>
 #include <public/sched.h>
@@ -1480,6 +1481,7 @@ int hvm_domain_initialise(struct domain *d)
     INIT_LIST_HEAD(&d->arch.hvm_domain.ioreq_server.list);
     spin_lock_init(&d->arch.hvm_domain.irq_lock);
     spin_lock_init(&d->arch.hvm_domain.uc_lock);
+    spin_lock_init(&d->arch.hvm_domain.vmport_lock);
 
     INIT_LIST_HEAD(&d->arch.hvm_domain.msixtbl_list);
     spin_lock_init(&d->arch.hvm_domain.msixtbl_list_lock);
@@ -1492,11 +1494,36 @@ int hvm_domain_initialise(struct domain *d)
 
     d->arch.hvm_domain.params = xzalloc_array(uint64_t, HVM_NR_PARAMS);
     d->arch.hvm_domain.io_handler = xmalloc(struct hvm_io_handler);
+    d->arch.hvm_domain.vmport_data = xzalloc(struct vmport_state);
     rc = -ENOMEM;
-    if ( !d->arch.hvm_domain.params || !d->arch.hvm_domain.io_handler )
+    if ( !d->arch.hvm_domain.params || !d->arch.hvm_domain.io_handler ||
+         !d->arch.hvm_domain.vmport_data )
         goto fail1;
     d->arch.hvm_domain.io_handler->num_slot = 0;
 
+    /*
+     * Any value is fine here. In fact a random number may better.
+     * It is used to help validate that a both sides are talking
+     * about the same channel.
+     */
+    d->arch.hvm_domain.vmport_data->open_cookie = 435;
+
+    d->arch.hvm_domain.vmport_data->used_guestinfo = 10;
+    for (rc = 0;
+         rc < d->arch.hvm_domain.vmport_data->used_guestinfo;
+         rc++)
+        d->arch.hvm_domain.vmport_data->guestinfo[rc] =
+            xzalloc(vmport_guestinfo_t);
+
+    d->arch.hvm_domain.vmport_data->used_guestinfo_jumbo = 2;
+    for (rc = 0;
+         rc < d->arch.hvm_domain.vmport_data->used_guestinfo_jumbo;
+         rc++)
+        d->arch.hvm_domain.vmport_data->guestinfo_jumbo[rc] =
+            xzalloc(vmport_guestinfo_jumbo_t);
+
+    vmport_flush(&d->arch.hvm_domain);
+
     if ( is_pvh_domain(d) )
     {
         register_portio_handler(d, 0, 0x10003, handle_pvh_io);
@@ -1507,6 +1534,7 @@ int hvm_domain_initialise(struct domain *d)
 
     d->arch.hvm_domain.params[HVM_PARAM_HPET_ENABLED] = 1;
     d->arch.hvm_domain.params[HVM_PARAM_TRIPLE_FAULT_REASON] = SHUTDOWN_reboot;
+    d->arch.hvm_domain.params[HVM_PARAM_VMPORT_RESET_TIME] = 14;
 
     vpic_init(d);
 
@@ -1520,6 +1548,7 @@ int hvm_domain_initialise(struct domain *d)
 
     register_portio_handler(d, 0xe9, 1, hvm_print_line);
     register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
+    register_portio_handler(d, VMPORT_PORT, 4, vmport_ioport);
 
     rc = hvm_funcs.domain_initialise(d);
     if ( rc != 0 )
@@ -1532,6 +1561,17 @@ int hvm_domain_initialise(struct domain *d)
     stdvga_deinit(d);
     vioapic_deinit(d);
  fail1:
+    if ( d->arch.hvm_domain.vmport_data )
+    {
+        struct vmport_state *vs = d->arch.hvm_domain.vmport_data;
+        int idx;
+
+        for (idx = 0; idx < vs->used_guestinfo; idx++)
+            xfree(vs->guestinfo[idx]);
+        for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+            xfree(vs->guestinfo_jumbo[idx]);
+    }
+    xfree(d->arch.hvm_domain.vmport_data);
     xfree(d->arch.hvm_domain.io_handler);
     xfree(d->arch.hvm_domain.params);
  fail0:
@@ -1541,6 +1581,15 @@ int hvm_domain_initialise(struct domain *d)
 
 void hvm_domain_relinquish_resources(struct domain *d)
 {
+    if ( d->arch.hvm_domain.vmport_data )
+    {
+        struct vmport_state *vs = d->arch.hvm_domain.vmport_data;
+        int idx;
+
+        for (idx = 0; idx < vs->used_guestinfo; idx++)
+            xfree(vs->guestinfo[idx]);
+        xfree(d->arch.hvm_domain.vmport_data);
+    }
     xfree(d->arch.hvm_domain.io_handler);
     xfree(d->arch.hvm_domain.params);
 
@@ -5473,6 +5522,257 @@ static int hvmop_destroy_ioreq_server(
     return rc;
 }
 
+static void hvm_del_guestinfo_jumbo(struct vmport_state *vs, char *key,
+                                    uint8_t len)
+{
+    int idx;
+
+    for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+    {
+        if ( vs->guestinfo[idx] &&
+             (vs->guestinfo_jumbo[idx]->key_len == len) &&
+             (memcmp(key, vs->guestinfo_jumbo[idx]->key_data, len) == 0) )
+        {
+            vs->guestinfo_jumbo[idx]->val_data[0] = '\0';
+            vs->guestinfo_jumbo[idx]->key_len = 0;
+            break;
+        }
+    }
+}
+
+static void hvm_del_guestinfo(struct vmport_state *vs, char *key, uint8_t len)
+{
+    int idx;
+
+    for (idx = 0; idx < vs->used_guestinfo; idx++)
+    {
+        if ( vs->guestinfo[idx] &&
+             (vs->guestinfo[idx]->key_len == len) &&
+             (memcmp(key, vs->guestinfo[idx]->key_data, len) == 0) )
+        {
+            vs->guestinfo[idx]->val_data[0] = '\0';
+            vs->guestinfo[idx]->key_len = 0;
+            break;
+        }
+    }
+}
+
+static long hvm_set_guestinfo(struct vmport_state *vs,
+                              struct xen_hvm_vmport_guest_info *a,
+                              char *key, char *value)
+{
+    int idx;
+    int free_idx = -1;
+    long rc = 0;
+
+    for (idx = 0; idx < vs->used_guestinfo; idx++)
+    {
+        if ( !vs->guestinfo[idx] )
+        {
+#ifndef NDEBUG
+            gdprintk(XENLOG_WARNING,
+                     "idx=%d not allocated, but used_guestinfo=%d\n",
+                     idx, vs->used_guestinfo);
+#endif
+        }
+        else if ( (vs->guestinfo[idx]->key_len == a->key_length) &&
+                  (memcmp(key,
+                          vs->guestinfo[idx]->key_data,
+                          vs->guestinfo[idx]->key_len) == 0) )
+        {
+            vs->guestinfo[idx]->val_len = a->value_length;
+            memcpy(vs->guestinfo[idx]->val_data, value, a->value_length);
+            break;
+
+        }
+        else if ( (vs->guestinfo[idx]->key_len == 0) &&
+                  (free_idx == -1) )
+            free_idx = idx;
+    }
+
+    if ( idx >= vs->used_guestinfo )
+    {
+        if ( free_idx == -1 )
+            rc = -EBUSY;
+        else
+        {
+            vs->guestinfo[free_idx]->key_len = a->key_length;
+            memcpy(vs->guestinfo[free_idx]->key_data, key, a->key_length);
+            vs->guestinfo[free_idx]->val_len = a->value_length;
+            memcpy(vs->guestinfo[free_idx]->val_data, value, a->value_length);
+        }
+    }
+
+    /* Delete any duplicate entry */
+    if ( rc == 0 )
+        hvm_del_guestinfo_jumbo(vs, key, a->key_length);
+
+    return rc;
+}
+
+static long hvm_set_guestinfo_jumbo(struct vmport_state *vs,
+                                    struct xen_hvm_vmport_guest_info *a,
+                                    char *key, char *value)
+{
+    int idx;
+    int free_idx = -1;
+    long rc = 0;
+
+    for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+    {
+
+        if ( !vs->guestinfo_jumbo[idx] )
+        {
+#ifndef NDEBUG
+            gdprintk(XENLOG_WARNING,
+                     "idx=%d not allocated, but used_guestinfo_jumbo=%d\n",
+                     idx, vs->used_guestinfo_jumbo);
+#endif
+        }
+        else if ( (vs->guestinfo_jumbo[idx]->key_len == a->key_length) &&
+                  (memcmp(key, vs->guestinfo_jumbo[idx]->key_data,
+                          vs->guestinfo_jumbo[idx]->key_len) == 0) )
+        {
+            vs->guestinfo_jumbo[idx]->val_len = a->value_length;
+            memcpy(vs->guestinfo_jumbo[idx]->val_data, value, a->value_length);
+            break;
+
+        }
+        else if ( (vs->guestinfo_jumbo[idx]->key_len == 0) &&
+                  (free_idx == -1) )
+            free_idx = idx;
+    }
+
+    if ( idx >= vs->used_guestinfo_jumbo )
+    {
+        if ( free_idx == -1 )
+            rc = -EBUSY;
+        else
+        {
+            vs->guestinfo_jumbo[free_idx]->key_len = a->key_length;
+            memcpy(vs->guestinfo_jumbo[free_idx]->key_data,
+                   key, a->key_length);
+            vs->guestinfo_jumbo[free_idx]->val_len = a->value_length;
+            memcpy(vs->guestinfo_jumbo[free_idx]->val_data,
+                   value, a->value_length);
+        }
+    }
+
+    /* Delete any duplicate entry */
+    if ( rc == 0 )
+        hvm_del_guestinfo(vs, key, a->key_length);
+
+    return rc;
+}
+
+static long hvm_get_guestinfo(struct vmport_state *vs,
+                              struct xen_hvm_vmport_guest_info *a,
+                              char *key, char *value,
+                              XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    int idx;
+    long rc = 0;
+
+    if ( a->key_length == 0 )
+    {
+        /*
+         * Here we are iterating on getting all guestinfo entries
+         * using index
+         */
+        idx = a->value_length;
+        if ( idx >= vs->used_guestinfo ||
+             !vs->guestinfo[idx] )
+            rc = -ENOENT;
+        else
+        {
+            a->key_length = vs->guestinfo[idx]->key_len;
+            memcpy(a->data, vs->guestinfo[idx]->key_data, a->key_length);
+            a->value_length = vs->guestinfo[idx]->val_len;
+            memcpy(&a->data[a->key_length], vs->guestinfo[idx]->val_data,
+                   a->value_length);
+            rc = copy_to_guest(arg, a, 1) ? -EFAULT : 0;
+        }
+    }
+    else
+    {
+        for (idx = 0; idx < vs->used_guestinfo; idx++)
+        {
+            if ( vs->guestinfo[idx] &&
+                 (vs->guestinfo[idx]->key_len == a->key_length) &&
+                 (memcmp(key, vs->guestinfo[idx]->key_data,
+                         vs->guestinfo[idx]->key_len) == 0) )
+            {
+                a->value_length = vs->guestinfo[idx]->val_len;
+                memcpy(value, vs->guestinfo[idx]->val_data,
+                       a->value_length);
+                rc = copy_to_guest(arg, a, 1) ? -EFAULT : 0;
+                break;
+            }
+        }
+        if ( idx >= vs->used_guestinfo )
+            rc = -ENOENT;
+    }
+    return rc;
+}
+
+static long hvm_get_guestinfo_jumbo(struct vmport_state *vs,
+                                    struct xen_hvm_vmport_guest_info *a,
+                                    char *key, char *value,
+                                    XEN_GUEST_HANDLE_PARAM(void) arg)
+{
+    int idx, total_entries;
+    long rc = 0;
+
+    if ( a->key_length == 0 )
+    {
+        /*
+         * Here we are iterating on getting all guestinfo entries
+         * using index
+         */
+        total_entries = vs->used_guestinfo + vs->used_guestinfo_jumbo;
+
+        /* Input index is in a->value_length */
+        if ( (a->value_length < 0) || (a->value_length >= total_entries) )
+        {
+            rc = -ENOENT;
+            return rc;
+        }
+        idx = a->value_length - vs->used_guestinfo;
+        if ( idx >= vs->used_guestinfo_jumbo ||
+             !vs->guestinfo_jumbo[idx] )
+            rc = -ENOENT;
+        else
+        {
+            a->key_length = vs->guestinfo_jumbo[idx]->key_len;
+            memcpy(a->data, vs->guestinfo_jumbo[idx]->key_data, a->key_length);
+            a->value_length = vs->guestinfo_jumbo[idx]->val_len;
+            memcpy(&a->data[a->key_length],
+                   vs->guestinfo_jumbo[idx]->val_data, a->value_length);
+            rc = copy_to_guest(arg, a, 1) ? -EFAULT : 0;
+        }
+    }
+    else
+    {
+        for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+        {
+            if ( vs->guestinfo_jumbo[idx] &&
+                 (vs->guestinfo_jumbo[idx]->key_len == a->key_length) &&
+                 (memcmp(key, vs->guestinfo_jumbo[idx]->key_data,
+                         vs->guestinfo_jumbo[idx]->key_len) == 0) )
+            {
+                a->value_length = vs->guestinfo_jumbo[idx]->val_len;
+                memcpy(value, vs->guestinfo_jumbo[idx]->val_data,
+                       a->value_length);
+                rc = copy_to_guest(arg, a, 1) ? -EFAULT : 0;
+                break;
+            }
+        }
+        if ( idx >= vs->used_guestinfo_jumbo )
+            rc = -ENOENT;
+    }
+    return rc;
+}
+
 #define HVMOP_op_mask 0xff
 
 long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
@@ -5711,6 +6011,18 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
                     break;
                 }
                 break;
+            case HVM_PARAM_VMPORT_BUILD_NUMBER_TIME:
+            case HVM_PARAM_VMPORT_BUILD_NUMBER_VALUE:
+                /*
+                 * These should only ever be set to zero by tools.
+                 * So disallow anything else.
+                 */
+                if ( a.value )
+                {
+                    rc = -EINVAL;
+                    break;
+                }
+                break;
             }
 
             if ( rc == 0 ) 
@@ -6125,6 +6437,172 @@ long do_hvm_op(unsigned long op, XEN_GUEST_HANDLE_PARAM(void) arg)
         break;
     }
 
+    case HVMOP_get_vmport_guest_info:
+    case HVMOP_set_vmport_guest_info:
+    {
+        struct xen_hvm_vmport_guest_info a;
+        struct domain *d;
+        char *key = NULL;
+        char *value = NULL;
+        struct vmport_state *vs;
+        int idx, total_entries;
+        vmport_guestinfo_t *add_slots[5];
+        vmport_guestinfo_jumbo_t *add_slots_jumbo[2];
+        int num_slots = 0, num_slots_jumbo = 0, num_free_slots = 0;
+
+        if ( copy_from_guest(&a, arg, 1) )
+            return -EFAULT;
+
+        ASSERT(strlen("guestinfo.") == 10);
+
+#if VMPORT_MAX_KEY_LEN + 10 != VMPORT_GUEST_INFO_KEY_MAX
+#error Need to adjust VMPORT_MAX_KEY_LEN & VMPORT_GUEST_INFO_KEY_MAX
+#endif
+
+#if VMPORT_MAX_VAL_JUMBO_LEN != VMPORT_GUEST_INFO_VAL_MAX
+#error Need to adjust VMPORT_MAX_VAL_JUMBO_LEN & VMPORT_GUEST_INFO_VAL_MAX
+#endif
+
+        if ( a.key_length > strlen("guestinfo.") )
+        {
+            if ( (size_t)a.key_length + (size_t)a.value_length >
+                 sizeof(a.data) )
+                return -EINVAL;
+
+            if ( memcmp(a.data, "guestinfo.", strlen("guestinfo.")) == 0 )
+            {
+                key = &a.data[strlen("guestinfo.")];
+                a.key_length -= strlen("guestinfo.");
+            }
+            else
+                key = &a.data[0];
+            if ( a.key_length > VMPORT_MAX_KEY_LEN )
+            {
+                gdprintk(XENLOG_ERR, "bad key len %d\n", a.key_length);
+                return -EINVAL;
+            }
+            if ( a.value_length > VMPORT_MAX_VAL_JUMBO_LEN )
+            {
+                gdprintk(XENLOG_ERR, "bad val len %d\n", a.value_length);
+                return -EINVAL;
+            }
+            value = key + a.key_length;
+
+        }
+        else if (a.key_length > 0)
+        {
+            if ( (size_t)a.key_length + (size_t)a.value_length >
+                 sizeof(a.data) )
+                return -EINVAL;
+            key = &a.data[0];
+            if ( a.key_length > VMPORT_MAX_KEY_LEN )
+            {
+                gdprintk(XENLOG_ERR, "bad key len %d", a.key_length);
+                return -EINVAL;
+            }
+            if ( a.value_length > VMPORT_MAX_VAL_JUMBO_LEN )
+            {
+                gdprintk(XENLOG_ERR, "bad val len %d\n", a.value_length);
+                return -EINVAL;
+            }
+            value = key + a.key_length;
+
+        }
+        else if ( (a.key_length == 0) && (op == HVMOP_set_vmport_guest_info) )
+            return -EINVAL;
+
+        d = rcu_lock_domain_by_any_id(a.domid);
+        if ( d == NULL )
+            return rc;
+
+        rc = -EINVAL;
+        if ( !is_hvm_domain(d) )
+            goto param_fail9;
+
+        rc = xsm_hvm_param(XSM_TARGET, d, op);
+        if ( rc )
+            goto param_fail9;
+
+        vs = d->arch.hvm_domain.vmport_data;
+        total_entries = vs->used_guestinfo + vs->used_guestinfo_jumbo;
+
+        if ( (a.key_length == 0) && (a.value_length >= total_entries) )
+        {
+            /*
+             * When key length is zero, we are interating on
+             * get-guest-info hypercalls to retrieve all guestinfo
+             * entries using index passed in a.value_length
+             */
+            rc = -E2BIG;
+            goto param_fail9;
+        }
+
+        num_free_slots = 0;
+        for (idx = 0; idx < vs->used_guestinfo; idx++)
+        {
+            if ( vs->guestinfo[idx] &&
+                (vs->guestinfo[idx]->key_len == 0) )
+                num_free_slots++;
+        }
+        if ( num_free_slots < 5 )
+        {
+            num_slots = 5 - num_free_slots;
+            if ( vs->used_guestinfo + num_slots > VMPORT_MAX_NUM_KEY )
+                num_slots = VMPORT_MAX_NUM_KEY - vs->used_guestinfo;
+            for (idx = 0; idx < num_slots; idx++)
+                add_slots[idx] = xzalloc(vmport_guestinfo_t);
+        }
+
+        num_free_slots = 0;
+        for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+        {
+            if ( vs->guestinfo_jumbo[idx] &&
+                 (vs->guestinfo_jumbo[idx]->key_len == 0) )
+                num_free_slots++;
+        }
+        if ( num_free_slots < 1 )
+        {
+            num_slots_jumbo = 1 - num_free_slots;
+            if ( vs->used_guestinfo_jumbo + num_slots_jumbo >
+                 VMPORT_MAX_NUM_JUMBO_KEY )
+                num_slots_jumbo = VMPORT_MAX_NUM_JUMBO_KEY -
+                    vs->used_guestinfo_jumbo;
+            for (idx = 0; idx < num_slots_jumbo; idx++)
+                add_slots_jumbo[idx] = xzalloc(vmport_guestinfo_jumbo_t);
+        }
+
+        spin_lock(&d->arch.hvm_domain.vmport_lock);
+
+        for (idx = 0; idx < num_slots; idx++)
+            vs->guestinfo[vs->used_guestinfo + idx] = add_slots[idx];
+        vs->used_guestinfo += num_slots;
+
+        for (idx = 0; idx < num_slots_jumbo; idx++)
+            vs->guestinfo_jumbo[vs->used_guestinfo_jumbo + idx] =
+                add_slots_jumbo[idx];
+        vs->used_guestinfo_jumbo += num_slots_jumbo;
+
+        if ( op == HVMOP_set_vmport_guest_info )
+        {
+            if ( a.value_length > VMPORT_MAX_VAL_LEN )
+                rc = hvm_set_guestinfo_jumbo(vs, &a, key, value);
+            else
+                rc = hvm_set_guestinfo(vs, &a, key, value);
+        }
+        else
+        {
+            /* Get Guest Info */
+            rc = hvm_get_guestinfo(vs, &a, key, value, arg);
+            if ( rc != 0 )
+                rc = hvm_get_guestinfo_jumbo(vs, &a, key, value, arg);
+        }
+        spin_unlock(&d->arch.hvm_domain.vmport_lock);
+
+    param_fail9:
+        rcu_unlock_domain(d);
+        break;
+    }
+
     default:
     {
         gdprintk(XENLOG_DEBUG, "Bad HVM op %ld.\n", op);
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index 9f565d6..94e08eb 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -42,6 +42,7 @@
 #include <asm/hvm/vlapic.h>
 #include <asm/hvm/trace.h>
 #include <asm/hvm/emulate.h>
+#include <asm/hvm/vmport.h>
 #include <public/sched.h>
 #include <xen/iocap.h>
 #include <public/hvm/ioreq.h>
diff --git a/xen/arch/x86/hvm/svm/emulate.c b/xen/arch/x86/hvm/svm/emulate.c
index 37a1ece..61002fb 100644
--- a/xen/arch/x86/hvm/svm/emulate.c
+++ b/xen/arch/x86/hvm/svm/emulate.c
@@ -78,6 +78,7 @@ static unsigned long svm_nextrip_insn_length(struct vcpu *v)
         /* ...and the rest of the #VMEXITs */
     case VMEXIT_CR0_SEL_WRITE:
     case VMEXIT_EXCEPTION_BP:
+    case VMEXIT_EXCEPTION_GP:
         break;
     default:
         BUG();
@@ -106,6 +107,7 @@ MAKE_INSTR(VMSAVE, 3, 0x0f, 0x01, 0xdb);
 MAKE_INSTR(STGI,   3, 0x0f, 0x01, 0xdc);
 MAKE_INSTR(CLGI,   3, 0x0f, 0x01, 0xdd);
 MAKE_INSTR(INVLPGA,3, 0x0f, 0x01, 0xdf);
+MAKE_INSTR(IN,     1, 0xed);
 
 static const u8 *const opc_bytes[INSTR_MAX_COUNT] =
 {
@@ -126,6 +128,7 @@ static const u8 *const opc_bytes[INSTR_MAX_COUNT] =
     [INSTR_STGI]   = OPCODE_STGI,
     [INSTR_CLGI]   = OPCODE_CLGI,
     [INSTR_INVLPGA] = OPCODE_INVLPGA,
+    [INSTR_IN] = OPCODE_IN,
 };
 
 static int fetch(struct vcpu *v, u8 *buf, unsigned long addr, int len)
diff --git a/xen/arch/x86/hvm/svm/svm.c b/xen/arch/x86/hvm/svm/svm.c
index b5188e6..f61bd02 100644
--- a/xen/arch/x86/hvm/svm/svm.c
+++ b/xen/arch/x86/hvm/svm/svm.c
@@ -56,6 +56,7 @@
 #include <asm/hvm/svm/nestedsvm.h>
 #include <asm/hvm/nestedhvm.h>
 #include <asm/x86_emulate.h>
+#include <asm/hvm/vmport.h>
 #include <public/sched.h>
 #include <asm/hvm/vpt.h>
 #include <asm/hvm/trace.h>
@@ -2065,6 +2066,80 @@ svm_vmexit_do_vmsave(struct vmcb_struct *vmcb,
     return;
 }
 
+static void svm_vmexit_gp_intercept(struct cpu_user_regs *regs, struct vcpu *v)
+{
+    struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
+    unsigned long inst_len = __get_instruction_length(v, INSTR_IN);
+
+    regs->error_code = vmcb->exitinfo1;
+    if ( hvm_long_mode_enabled(v) )
+        HVMTRACE_LONG_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
+                          regs->error_code,
+                          TRC_PAR_LONG(vmcb->exitinfo2));
+    else
+        HVMTRACE_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
+                     regs->error_code, vmcb->exitinfo2);
+
+    if ( inst_len <= 1 && (regs->rdx & 0xffff) == VMPORT_PORT &&
+         vmcb->exitinfo2 == 0 && regs->error_code == 0 )
+    {
+        uint32_t magic = regs->rax;
+
+        if ( magic == VMPORT_MAGIC )
+        {
+            unsigned char bytes[1] = { 0 };
+            int frc = hvm_fetch_from_guest_virt_nofault(bytes, regs->rip,
+                                                        1, PFEC_page_present);
+
+            if ( !frc && bytes[0] == 0xed ) /* in (%dx),%eax */
+            {
+                uint32_t val;
+
+                __update_guest_eip(regs, 1);
+                vmport_ioport(IOREQ_READ, VMPORT_PORT, 4, &val);
+                VMPORT_DBG_LOG(VMPORT_LOG_VMWARE_AFTER,
+                               "gp: VMware ip=%"PRIx64" ax=%"PRIx64
+                               " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                               " si=%"PRIx64" di=%"PRIx64,
+                               regs->rip, regs->rax, regs->rbx, regs->rcx,
+                               regs->rdx, regs->rsi, regs->rdi);
+                return;
+            }
+            else
+            {
+                VMPORT_DBG_LOG(VMPORT_LOG_GP_FAIL_RD_INST,
+                               "gp: VMware? ip=%"PRIx64"=>0x%x(%ld,%d) ax=%"PRIx64
+                               " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                               " si=%"PRIx64" di=%"PRIx64,
+                               regs->rip, bytes[0], inst_len, frc, regs->rax, regs->rbx,
+                               regs->rcx, regs->rdx, regs->rsi, regs->rdi);
+            }
+        }
+        else
+        {
+            VMPORT_DBG_LOG(VMPORT_LOG_GP_NOT_VMWARE,
+                           "gp: ip=%"PRIx64" ax=%"PRIx64" bx=%"PRIx64
+                           " cx=%"PRIx64 " dx=%"PRIx64" si=%"PRIx64
+                           " di=%"PRIx64,
+                           regs->rip, regs->rax, regs->rbx, regs->rcx,
+                           regs->rdx, regs->rsi, regs->rdi);
+            hvm_inject_hw_exception(TRAP_gp_fault, regs->error_code);
+        }
+    }
+    else
+    {
+        VMPORT_DBG_LOG(VMPORT_LOG_GP_UNKNOWN,
+                       "gp: e2=%lx ec=%lx ip=%"PRIx64" (%ld) ax=%"PRIx64
+                       " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64" si=%"PRIx64
+                       " di=%"PRIx64,
+                       (unsigned long)vmcb->exitinfo2,
+                       (unsigned long)regs->error_code,
+                       regs->rip, inst_len, regs->rax, regs->rbx, regs->rcx,
+                       regs->rdx, regs->rsi, regs->rdi);
+        hvm_inject_hw_exception(TRAP_gp_fault, regs->error_code);
+    }
+}
+
 static void svm_vmexit_ud_intercept(struct cpu_user_regs *regs)
 {
     struct hvm_emulate_ctxt ctxt;
@@ -2412,6 +2487,10 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)
         break;
     }
 
+    case VMEXIT_EXCEPTION_GP:
+        svm_vmexit_gp_intercept(regs, v);
+        break;
+
     case VMEXIT_EXCEPTION_UD:
         svm_vmexit_ud_intercept(regs);
         break;
diff --git a/xen/arch/x86/hvm/svm/vmcb.c b/xen/arch/x86/hvm/svm/vmcb.c
index 21292bb..791c045 100644
--- a/xen/arch/x86/hvm/svm/vmcb.c
+++ b/xen/arch/x86/hvm/svm/vmcb.c
@@ -193,6 +193,7 @@ static int construct_vmcb(struct vcpu *v)
 
     vmcb->_exception_intercepts =
         HVM_TRAP_MASK
+        | (1U << TRAP_gp_fault)
         | (1U << TRAP_no_device);
 
     if ( paging_mode_hap(v->domain) )
diff --git a/xen/arch/x86/hvm/vmport/Makefile b/xen/arch/x86/hvm/vmport/Makefile
new file mode 100644
index 0000000..2648fae
--- /dev/null
+++ b/xen/arch/x86/hvm/vmport/Makefile
@@ -0,0 +1 @@
+obj-y += vmport.o
diff --git a/xen/arch/x86/hvm/vmport/includeCheck.h b/xen/arch/x86/hvm/vmport/includeCheck.h
new file mode 100644
index 0000000..26e0d59
--- /dev/null
+++ b/xen/arch/x86/hvm/vmport/includeCheck.h
@@ -0,0 +1,17 @@
+/*
+ * includeCheck.h
+ *
+ * Copyright (C) 2012 Verizon Corporation
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License Version 2 (GPLv2)
+ * as published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details. <http://www.gnu.org/licenses/>.
+ */
+/*
+ * Nothing here.  Just to use backdoor_def.h without change.
+ */
diff --git a/xen/arch/x86/hvm/vmport/vmport.c b/xen/arch/x86/hvm/vmport/vmport.c
new file mode 100644
index 0000000..64609f2
--- /dev/null
+++ b/xen/arch/x86/hvm/vmport/vmport.c
@@ -0,0 +1,1436 @@
+/*
+ * HVM VMPORT emulation
+ *
+ * Copyright (C) 2012 Verizon Corporation
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License Version 2 (GPLv2)
+ * as published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details. <http://www.gnu.org/licenses/>.
+ */
+
+/*
+ * VMware Tools running in a DOMU will do "info-get" and "info-set"
+ * guestinfo commands to get and set keys and values. Inside the VM,
+ * vmtools at its lower level will feed the command string 4 bytes
+ * at a time into the VMWARE magic port using the IN
+ * instruction. Each 4 byte mini-rpc will get handled
+ * vmport_io()-->vmport_rpc()-->vmport_process_packet()-->
+ * vmport_process_send_payload()-->vmport_send() and the command
+ * string will get accumulated into a channels send_buffer.  When
+ * the full length of the string has been accumulated, then this
+ * code copies the send_buffer into a free
+ * vmport_state->channel-->receive_bucket.buffer
+ * VMware tools then does RECVSIZE and RECVPAYLOAD messages, the
+ * latter then reads 4 bytes at a time using the IN instruction (for
+ * the info-get case).  Then a final RECVSTATUS message is sent to
+ * finish up
+ */
+
+#include <xen/config.h>
+#include <xen/init.h>
+#include <xen/mm.h>
+#include <xen/lib.h>
+#include <xen/errno.h>
+#include <xen/trace.h>
+#include <xen/event.h>
+#include <xen/hypercall.h>
+#include <asm/current.h>
+#include <asm/cpufeature.h>
+#include <asm/processor.h>
+#include <asm/hvm/hvm.h>
+#include <asm/hvm/support.h>
+#include <asm/hvm/trace.h>
+#include <asm/hvm/vmport.h>
+
+#include "backdoor_def.h"
+#include "guest_msg_def.h"
+#include <asm/hvm/support.h>
+#include <asm/hvm/vlapic.h>
+#include <public/hvm/save.h>
+#include <public/arch-x86/hvm/save.h>
+#include <xen/cper.h>
+
+#define VMWARE_PROTO_TO_GUEST        0x4f4c4354
+#define VMWARE_PROTO_FROM_GUEST      0x49435052
+
+/* Some more VMware defines */
+#define VMWARE_GUI_AUTO_GRAB              0x001
+#define VMWARE_GUI_AUTO_UNGRAB            0x002
+#define VMWARE_GUI_AUTO_SCROLL            0x004
+#define VMWARE_GUI_AUTO_RAISE             0x008
+#define VMWARE_GUI_EXCHANGE_SELECTIONS    0x010
+#define VMWARE_GUI_WARP_CURSOR_ON_UNGRAB  0x020
+#define VMWARE_GUI_FULL_SCREEN            0x040
+
+#define VMWARE_GUI_TO_FULL_SCREEN         0x080
+#define VMWARE_GUI_TO_WINDOW              0x100
+
+#define VMWARE_GUI_AUTO_RAISE_DISABLED    0x200
+
+#define VMWARE_GUI_SYNC_TIME              0x400
+
+/* When set, toolboxes should not show the cursor options page. */
+#define VMWARE_DISABLE_CURSOR_OPTIONS     0x800
+
+/* End of more VMware defines */
+
+#define GUESTINFO_NOTFOUND      500
+#define GUESTINFO_VALTOOLONG    1
+#define GUESTINFO_KEYTOOLONG    2
+#define GUESTINFO_TOOMANYKEYS   3
+
+
+#ifndef NDEBUG
+unsigned int opt_vmport_debug __read_mostly;
+integer_param("vmport_debug", opt_vmport_debug);
+#endif
+
+/* Note: VMPORT_PORT and VMPORT_MAGIC is also defined as BDOOR_PORT
+ * and BDOOR_MAGIC in backdoor_def.h Defined in vmport.h also.
+ */
+
+inline uint16_t get_low_bits(uint32_t bits)
+{
+    return bits & 0xffff;
+}
+
+inline uint16_t get_high_bits(uint32_t bits)
+{
+    return bits >> 16;
+}
+
+inline uint32_t set_high_bits(uint32_t b, uint32_t val)
+{
+    return (val << 16) | get_low_bits(b);
+}
+
+inline void set_status(struct cpu_user_regs *ur, uint16_t val)
+{
+    /* VMware defines this to be only 32 bits */
+    ur->rcx = (val << 16) | (ur->rcx & 0xffff);
+}
+
+#ifndef NDEBUG
+void vmport_safe_print(char *prefix, int len, const char *msg)
+{
+    unsigned char c;
+    unsigned int end = len;
+    unsigned int i, k;
+    char out[4 * (VMPORT_MAX_SEND_BUF + 1) * 3 + 6];
+
+    if ( end > (sizeof(out) / 3 - 6) )
+        end = sizeof(out) / 3 - 6;
+    out[0] = '<';
+    k = 1;
+    for (i = 0; i < end; i++)
+    {
+        c = msg[i];
+        if ( (c == '^') || (c == '\\') || (c == '>') )
+        {
+            out[k++] = '\\';
+            out[k++] = c;
+        }
+        else if ( (c >= ' ') && (c <= '~') )
+            out[k++] = c;
+        else if ( c < ' ' )
+        {
+            out[k++] = '^';
+            out[k++] = c ^ 0x40;
+        }
+        else
+        {
+            snprintf(&out[k], sizeof(out) - k, "\\%02x", c);
+            k += 3;
+        }
+    }
+    out[k++] = '>';
+    if ( len > end )
+    {
+        out[k++] = '.';
+        out[k++] = '.';
+        out[k++] = '.';
+    }
+    out[k++] = 0;
+    gdprintk(XENLOG_DEBUG, "%s%d(%d,%d,%zu)%s\n", prefix, end, len, k,
+             sizeof(out), out);
+}
+#endif
+
+/*
+ * Copy message into a jumbo bucket buffer which vmtools will use to
+ * read from 4 bytes at a time until done with it
+ */
+void vmport_send_jumbo(struct hvm_domain *hd, vmport_channel_t *c,
+                       const char *msg)
+{
+    unsigned int cur_recv_len = strlen(msg) + 1;
+    vmport_jumbo_bucket_t *b = &(c->jumbo_recv_bkt);
+
+    b->ctl.recv_len = cur_recv_len;
+    b->ctl.recv_idx = 0;
+
+    memset(b->recv_buf, 0, sizeof(b->recv_buf));
+
+    if ( cur_recv_len >= (sizeof(b->recv_buf) - 1) )
+    {
+        VMPORT_DBG_LOG(VMPORT_LOG_ERROR,
+                       "VMware jumbo recv_len=%d >= %ld",
+                       cur_recv_len, sizeof(b->recv_buf) - 1);
+        cur_recv_len = sizeof(b->recv_buf) - 1;
+    }
+
+    memcpy(b->recv_buf, msg, cur_recv_len);
+
+    c->ctl.jumbo = 1;
+}
+
+/*
+ * Copy message into a free receive bucket buffer which vmtools will use to
+ * read from 4 bytes at a time until done with it
+ */
+void vmport_send_normal(struct hvm_domain *hd, vmport_channel_t *c,
+                        const char *msg)
+{
+    unsigned int cur_recv_len = strlen(msg) + 1;
+    unsigned int my_bkt = c->ctl.recv_write;
+    unsigned int next_bkt = my_bkt + 1;
+    vmport_bucket_t *b;
+
+    if ( next_bkt >= VMPORT_MAX_BKTS )
+        next_bkt = 0;
+
+    if ( next_bkt == c->ctl.recv_read )
+    {
+#ifndef NDEBUG
+        if ( opt_vmport_debug & VMPORT_LOG_SKIP_SEND )
+        {
+            char prefix[30];
+
+            snprintf(prefix, sizeof(prefix),
+                     "VMware _send skipped %d (%d, %d) ",
+                     c->ctl.chan_id, my_bkt, c->ctl.recv_read);
+            prefix[sizeof(prefix) - 1] = 0;
+            vmport_safe_print(prefix, cur_recv_len, msg);
+        }
+#endif
+        return;
+    }
+
+    c->ctl.recv_write = next_bkt;
+    b = &c->recv_bkt[my_bkt];
+#ifndef NDEBUG
+    if ( opt_vmport_debug & VMPORT_LOG_SEND )
+    {
+        char prefix[30];
+
+        snprintf(prefix, sizeof(prefix), "VMware _send %d (%d) ",
+                 c->ctl.chan_id, my_bkt);
+        prefix[sizeof(prefix) - 1] = 0;
+        vmport_safe_print(prefix, cur_recv_len, msg);
+    }
+#endif
+
+    b->ctl.recv_len = cur_recv_len;
+    b->ctl.recv_idx = 0;
+    memset(b->recv_buf, 0, sizeof(b->recv_buf));
+    if ( cur_recv_len >= (sizeof(b->recv_buf) - 1) )
+    {
+        VMPORT_DBG_LOG(VMPORT_LOG_ERROR, "VMware recv_len=%d >= %zd",
+                       cur_recv_len, sizeof(b->recv_buf) - 1);
+        cur_recv_len = sizeof(b->recv_buf) - 1;
+    }
+    memcpy(b->recv_buf, msg, cur_recv_len);
+}
+
+void vmport_send(struct hvm_domain *hd, vmport_channel_t *c,
+                 const char *msg)
+{
+    unsigned int cur_recv_len = strlen(msg) + 1;
+
+    if ( cur_recv_len > VMPORT_MAX_VAL_LEN )
+        vmport_send_jumbo(hd, c, msg);
+    else
+        vmport_send_normal(hd, c, msg);
+}
+
+void vmport_ctrl_send(struct hvm_domain *hd, char *msg)
+{
+    struct vmport_state *vs = hd->vmport_data;
+    unsigned int i;
+
+    hd->vmport_data->ping_time = get_sec();
+    spin_lock(&hd->vmport_lock);
+    for (i = 0; i < VMPORT_MAX_CHANS; i++)
+    {
+        if ( vs->chans[i].ctl.proto_num == VMWARE_PROTO_TO_GUEST )
+            vmport_send(hd, &vs->chans[i], msg);
+    }
+    spin_unlock(&hd->vmport_lock);
+}
+
+void vmport_flush(struct hvm_domain *hd)
+{
+    VMPORT_DBG_LOG(VMPORT_LOG_FLUSH, "VMware flush");
+    spin_lock(&hd->vmport_lock);
+    memset(&hd->vmport_data->chans, 0, sizeof(hd->vmport_data->chans));
+    spin_unlock(&hd->vmport_lock);
+}
+
+void vmport_sweep(struct hvm_domain *hd, unsigned long now_time)
+{
+    struct vmport_state *vs = hd->vmport_data;
+    unsigned int i;
+
+    for (i = 0; i < VMPORT_MAX_CHANS; i++)
+    {
+        if ( vs->chans[i].ctl.proto_num )
+        {
+            vmport_channel_t *c = &vs->chans[i];
+            long delta = now_time - c->ctl.active_time;
+
+            if ( delta >= 80 )
+            {
+                VMPORT_DBG_LOG(VMPORT_LOG_SWEEP, "VMware flush %d. delta=%ld",
+                               c->ctl.chan_id, delta);
+                /* Return channel to free pool */
+                c->ctl.proto_num = 0;
+            }
+        }
+    }
+}
+
+vmport_channel_t *vmport_new_chan(struct vmport_state *vs,
+                                  unsigned long now_time)
+{
+    unsigned int i;
+
+    for (i = 0; i < VMPORT_MAX_CHANS; i++)
+    {
+        if ( !vs->chans[i].ctl.proto_num )
+        {
+            vmport_channel_t *c = &vs->chans[i];
+
+            c->ctl.chan_id = i;
+            c->ctl.cookie = vs->open_cookie++;
+            c->ctl.active_time = now_time;
+            c->ctl.send_len = 0;
+            c->ctl.send_idx = 0;
+            c->ctl.recv_read = 0;
+            c->ctl.recv_write = 0;
+            return c;
+        }
+    }
+    return NULL;
+}
+
+void vmport_process_send_size(struct hvm_domain *hd, vmport_channel_t *c,
+                              struct cpu_user_regs *ur)
+{
+    /* vmware tools often send a 0 byte request size. */
+    c->ctl.send_len = ur->rbx;
+    c->ctl.send_idx = 0;
+
+    set_status(ur, MESSAGE_STATUS_SUCCESS);
+#ifndef NDEBUG
+    if ( (opt_vmport_debug & VMPORT_LOG_SEND_SIZE_ALL) ||
+         ((opt_vmport_debug & VMPORT_LOG_SEND_SIZE) && (c->ctl.send_len)) )
+        gdprintk(XENLOG_DEBUG, "VMware SENDSIZE %d is %d\n",
+                 c->ctl.chan_id, c->ctl.send_len);
+#endif
+}
+
+/* ret_buffer is in/out param */
+int vmport_get_guestinfo(struct hvm_domain *hd, struct vmport_state *vs,
+                         char *a_info_key, unsigned int a_key_len,
+                         char *ret_buffer, unsigned int ret_buffer_len)
+{
+    unsigned int idx;
+
+    for (idx = 0; idx < vs->used_guestinfo; idx++)
+    {
+        if ( vs->guestinfo[idx] &&
+             (vs->guestinfo[idx]->key_len == a_key_len) &&
+             (memcmp(a_info_key, vs->guestinfo[idx]->key_data,
+                     vs->guestinfo[idx]->key_len) == 0) )
+        {
+#ifndef NDEBUG
+            if ( opt_vmport_debug & VMPORT_LOG_INFO_GET )
+            {
+                char prefix[30];
+
+                snprintf(prefix, sizeof(prefix),
+                         "VMware info-get val:");
+                vmport_safe_print(prefix, vs->guestinfo[idx]->val_len,
+                                  vs->guestinfo[idx]->val_data);
+            }
+#endif
+            snprintf(ret_buffer, ret_buffer_len - 1, "1 %.*s",
+                     (int)vs->guestinfo[idx]->val_len,
+                     vs->guestinfo[idx]->val_data);
+            return idx;
+        }
+    }
+
+    for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+    {
+        if ( vs->guestinfo_jumbo[idx] &&
+             (vs->guestinfo_jumbo[idx]->key_len == a_key_len) &&
+             (memcmp(a_info_key, vs->guestinfo_jumbo[idx]->key_data,
+                     vs->guestinfo_jumbo[idx]->key_len) == 0) )
+        {
+#ifndef NDEBUG
+            if ( opt_vmport_debug & VMPORT_LOG_INFO_GET )
+            {
+                char prefix[30];
+
+                snprintf(prefix, sizeof(prefix),
+                         "VMware info-get val:");
+                vmport_safe_print(prefix, vs->guestinfo_jumbo[idx]->val_len,
+                                  vs->guestinfo_jumbo[idx]->val_data);
+            }
+#endif
+            snprintf(ret_buffer, ret_buffer_len - 1, "1 %.*s",
+                     (int)vs->guestinfo_jumbo[idx]->val_len,
+                     vs->guestinfo_jumbo[idx]->val_data);
+            return idx;
+        }
+    }
+    return GUESTINFO_NOTFOUND;
+}
+
+int vmport_set_guestinfo(struct vmport_state *vs, int a_key_len,
+                         unsigned int a_val_len, char *a_info_key, char *val)
+{
+    unsigned int idx;
+    int free_idx = -1, rc = 0;
+
+#ifndef NDEBUG
+    gdprintk(XENLOG_WARNING, "vmport_set_guestinfo a_val_len=%d\n", a_val_len);
+#endif
+
+    if ( a_key_len <= VMPORT_MAX_KEY_LEN )
+    {
+        if ( a_val_len <= VMPORT_MAX_VAL_LEN )
+        {
+            for (idx = 0; idx < vs->used_guestinfo; idx++)
+            {
+                if ( !vs->guestinfo[idx] )
+                {
+#ifndef NDEBUG
+                    gdprintk(XENLOG_WARNING,
+                             "idx=%d not allocated, but used_guestinfo=%d\n",
+                             idx, vs->used_guestinfo);
+#endif
+                }
+                else if ( (vs->guestinfo[idx]->key_len == a_key_len) &&
+                          (memcmp(a_info_key, vs->guestinfo[idx]->key_data,
+                                  vs->guestinfo[idx]->key_len) == 0) )
+                {
+                    vs->guestinfo[idx]->val_len = a_val_len;
+                    memcpy(vs->guestinfo[idx]->val_data, val, a_val_len);
+                    break;
+                }
+                else if ( (vs->guestinfo[idx]->key_len == 0) &&
+                          (free_idx == -1) )
+                    free_idx = idx;
+            }
+            if ( idx >= vs->used_guestinfo )
+            {
+                if ( free_idx == -1 )
+                    rc = GUESTINFO_TOOMANYKEYS;
+                else
+                {
+                    vs->guestinfo[free_idx]->key_len = a_key_len;
+                    memcpy(vs->guestinfo[free_idx]->key_data,
+                           a_info_key, a_key_len);
+                    vs->guestinfo[free_idx]->val_len = a_val_len;
+                    memcpy(vs->guestinfo[free_idx]->val_data,
+                           val, a_val_len);
+                }
+            }
+        }
+        else
+            rc = GUESTINFO_VALTOOLONG;
+    }
+    else
+        rc = GUESTINFO_KEYTOOLONG;
+    if ( !rc )
+    {
+        for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+        {
+            if ( !vs->guestinfo_jumbo[idx] )
+            {
+#ifndef NDEBUG
+                gdprintk(XENLOG_WARNING,
+                         "idx=%d not allocated used_guestinfo_jumbo=%d\n",
+                         idx, vs->used_guestinfo_jumbo);
+#endif
+            }
+            else if ( (vs->guestinfo_jumbo[idx]->key_len == a_key_len) &&
+                      (memcmp(a_info_key,
+                              vs->guestinfo_jumbo[idx]->key_data,
+                              vs->guestinfo_jumbo[idx]->key_len) == 0) )
+            {
+                vs->guestinfo_jumbo[idx]->key_len = 0;
+                vs->guestinfo_jumbo[idx]->val_len = 0;
+                break;
+            }
+        }
+    }
+    return rc;
+}
+
+int vmport_set_guestinfo_jumbo(struct vmport_state *vs, int a_key_len,
+                               int a_val_len, char *a_info_key, char *val)
+{
+    unsigned int idx;
+    int free_idx = -1, rc = 0;
+
+#ifndef NDEBUG
+    gdprintk(XENLOG_WARNING, "vmport_set_guestinfo_jumbo a_val_len=%d\n",
+             a_val_len);
+#endif
+
+    if ( a_key_len <= VMPORT_MAX_KEY_LEN )
+    {
+        if ( a_val_len <= VMPORT_MAX_VAL_JUMBO_LEN )
+        {
+            for (idx = 0; idx < vs->used_guestinfo_jumbo; idx++)
+            {
+                if ( !vs->guestinfo_jumbo[idx] )
+                {
+#ifndef NDEBUG
+                    gdprintk(XENLOG_WARNING,
+                             "idx=%d not allocated; used_guestinfo_jumbo=%d\n",
+                             idx, vs->used_guestinfo_jumbo);
+#endif
+                }
+                else if ( (vs->guestinfo_jumbo[idx]->key_len == a_key_len) &&
+                          (memcmp(a_info_key,
+                                  vs->guestinfo_jumbo[idx]->key_data,
+                                  vs->guestinfo_jumbo[idx]->key_len) == 0) )
+                {
+
+                    vs->guestinfo_jumbo[idx]->val_len = a_val_len;
+                    memcpy(vs->guestinfo_jumbo[idx]->val_data, val, a_val_len);
+                    break;
+                }
+                else if ( (vs->guestinfo_jumbo[idx]->key_len == 0) &&
+                          (free_idx == -1) )
+                    free_idx = idx;
+            }
+            if ( idx >= vs->used_guestinfo_jumbo )
+            {
+                if ( free_idx == -1 )
+                    rc = GUESTINFO_TOOMANYKEYS;
+                else
+                {
+                    vs->guestinfo_jumbo[free_idx]->key_len = a_key_len;
+                    memcpy(vs->guestinfo_jumbo[free_idx]->key_data,
+                           a_info_key, a_key_len);
+                    vs->guestinfo_jumbo[free_idx]->val_len = a_val_len;
+                    memcpy(vs->guestinfo_jumbo[free_idx]->val_data,
+                           val, a_val_len);
+                }
+            }
+        }
+        else
+            rc = GUESTINFO_VALTOOLONG;
+    }
+    else
+        rc = GUESTINFO_KEYTOOLONG;
+    if ( !rc )
+    {
+        for (idx = 0; idx < vs->used_guestinfo; idx++)
+        {
+            if ( !vs->guestinfo[idx] )
+            {
+#ifndef NDEBUG
+                gdprintk(XENLOG_WARNING,
+                         "idx=%d not allocated, but used_guestinfo=%d\n",
+                         idx, vs->used_guestinfo);
+#endif
+            }
+            else if ( (vs->guestinfo[idx]->key_len == a_key_len) &&
+                      (memcmp(a_info_key,
+                              vs->guestinfo[idx]->key_data,
+                              vs->guestinfo[idx]->key_len) == 0) )
+            {
+                vs->guestinfo[idx]->key_len = 0;
+                vs->guestinfo[idx]->val_len = 0;
+                break;
+            }
+        }
+    }
+    return rc;
+}
+
+void vmport_process_send_payload(struct hvm_domain *hd, vmport_channel_t *c,
+                                 struct cpu_user_regs *ur,
+                                 unsigned long now_time)
+{
+    /* Accumulate 4 bytes of paload into send_buf using offset */
+    if ( c->ctl.send_idx < VMPORT_MAX_SEND_BUF )
+        c->send_buf[c->ctl.send_idx] = ur->rbx;
+
+    c->ctl.send_idx++;
+    set_status(ur, MESSAGE_STATUS_SUCCESS);
+
+    if ( c->ctl.send_idx * 4 >= c->ctl.send_len )
+    {
+
+        /* We are done accumulating so handle the command */
+
+        if ( c->ctl.send_idx < VMPORT_MAX_SEND_BUF )
+            ((char *)c->send_buf)[c->ctl.send_len] = 0;
+#ifndef NDEBUG
+        if ( opt_vmport_debug & VMPORT_LOG_RPC )
+        {
+            char prefix[30];
+
+            snprintf(prefix, sizeof(prefix),
+                     "VMware RPC %d (%d) ", c->ctl.chan_id, c->ctl.recv_read);
+            prefix[sizeof(prefix) - 1] = 0;
+            vmport_safe_print(prefix, c->ctl.send_len, (char *)c->send_buf);
+        }
+#endif
+        if ( c->ctl.proto_num == VMWARE_PROTO_FROM_GUEST )
+        {
+            /*
+             * Eaxmples of messages:
+             *
+             *   log toolbox: Version: build-341836
+             *   SetGuestInfo  4 build-341836
+             *   info-get guestinfo.ip
+             *   info-set guestinfo.ip joe
+             *
+             */
+
+            char *build = NULL;
+            char *info_key = NULL;
+            char *ret_msg = "1 ";
+            char ret_buffer[2 + VMPORT_MAX_VAL_JUMBO_LEN + 2];
+
+            if ( strncmp((char *)c->send_buf, "log toolbox: Version: build-",
+                         strlen("log toolbox: Version: build-")) == 0 )
+
+                build = (char *)c->send_buf +
+                    strlen("log toolbox: Version: build-");
+
+            else if ( strncmp((char *)c->send_buf, "SetGuestInfo  4 build-",
+                              strlen("SetGuestInfo  4 build-")) == 0 )
+
+                build = (char *)c->send_buf + strlen("SetGuestInfo  4 build-");
+
+            else if ( strncmp((char *)c->send_buf, "info-get guestinfo.",
+                              strlen("info-get guestinfo.")) == 0 )
+            {
+
+                unsigned int a_key_len = c->ctl.send_len -
+                    strlen("info-get guestinfo.");
+                int rc;
+                struct vmport_state *vs = hd->vmport_data;
+
+                info_key = (char *)c->send_buf + strlen("info-get guestinfo.");
+#ifndef NDEBUG
+                if ( opt_vmport_debug & VMPORT_LOG_INFO_GET )
+                {
+                    char prefix[30];
+
+                    snprintf(prefix, sizeof(prefix),
+                             "VMware info-get key:");
+                    vmport_safe_print(prefix, a_key_len, info_key);
+                }
+#endif
+                if ( a_key_len <= VMPORT_MAX_KEY_LEN )
+                {
+
+                    rc = vmport_get_guestinfo(hd, vs, info_key, a_key_len,
+                                              ret_buffer, sizeof(ret_buffer));
+                    if ( rc == GUESTINFO_NOTFOUND )
+                        ret_msg = "0 No value found";
+                    else
+                        ret_msg = ret_buffer;
+                }
+                else
+                    ret_msg = "0 Key is too long";
+
+            }
+            else if ( strncmp((char *)c->send_buf, "info-set guestinfo.",
+                              strlen("info-set guestinfo.")) == 0 )
+            {
+                char *val;
+                unsigned int rest_len = c->ctl.send_len -
+                    strlen("info-set guestinfo.");
+
+                info_key = (char *)c->send_buf + strlen("info-set guestinfo.");
+                val = strstr(info_key, " ");
+                if ( val )
+                {
+                    unsigned int a_key_len = val - info_key;
+                    unsigned int a_val_len = rest_len - a_key_len - 1;
+                    int rc;
+                    struct vmport_state *vs = hd->vmport_data;
+
+                    val++;
+#ifndef NDEBUG
+                    if ( opt_vmport_debug & VMPORT_LOG_INFO_SET )
+                    {
+                        char prefix[30];
+
+                        snprintf(prefix, sizeof(prefix),
+                                 "VMware info-set key:");
+                        vmport_safe_print(prefix, a_key_len, info_key);
+                        snprintf(prefix, sizeof(prefix),
+                                 "VMware info-set val:");
+                        vmport_safe_print(prefix, a_val_len, val);
+                    }
+#endif
+                    if ( a_val_len > VMPORT_MAX_VAL_LEN )
+                        rc = vmport_set_guestinfo_jumbo(vs, a_key_len,
+                                                        a_val_len,
+                                                        info_key, val);
+                    else
+                        rc = vmport_set_guestinfo(vs, a_key_len, a_val_len,
+                                                  info_key, val);
+                    if ( rc == 0 )
+                        ret_msg = "1 ";
+                    if ( rc == GUESTINFO_VALTOOLONG )
+                        ret_msg = "0 Value too long";
+                    if ( rc == GUESTINFO_KEYTOOLONG )
+                        ret_msg = "0 Key is too long";
+                    if ( rc == GUESTINFO_TOOMANYKEYS )
+                        ret_msg = "0 Too many keys";
+
+
+                }
+                else
+                {
+#ifndef NDEBUG
+                    if ( opt_vmport_debug & VMPORT_LOG_INFO_SET )
+                    {
+                        char prefix[30];
+
+                        snprintf(prefix, sizeof(prefix),
+                                 "VMware info-set missing val; key:");
+                        vmport_safe_print(prefix, rest_len, info_key);
+                    }
+#endif
+                    ret_msg = "0 Two and exactly two arguments expected";
+                }
+            }
+
+            vmport_send(hd, c, ret_msg);
+            if ( build )
+            {
+                long val = 0;
+                char *p = build;
+
+                while ( *p )
+                {
+                    if ( *p < '0' || *p > '9' )
+                        break;
+                    val = val * 10 + *p - '0';
+                    p++;
+                };
+
+                hd->params[HVM_PARAM_VMPORT_BUILD_NUMBER_VALUE] = val;
+                hd->params[HVM_PARAM_VMPORT_BUILD_NUMBER_TIME] = now_time;
+#ifndef NDEBUG
+                if ( opt_vmport_debug & VMPORT_LOG_BUILD )
+                {
+                    char prefix[30];
+
+                    snprintf(prefix, sizeof(prefix),
+                             "VMware build %ld ", val);
+                    vmport_safe_print(prefix, p - build, build);
+                }
+#endif
+            }
+        }
+        else
+        {
+            unsigned int my_bkt = c->ctl.recv_read - 1;
+            vmport_bucket_t *b;
+
+            if ( my_bkt >= VMPORT_MAX_BKTS )
+                my_bkt = VMPORT_MAX_BKTS - 1;
+            b = &c->recv_bkt[my_bkt];
+            b->ctl.recv_len = 0;
+        }
+    }
+}
+
+void vmport_process_recv_size(struct hvm_domain *hd, vmport_channel_t *c,
+                              struct cpu_user_regs *ur)
+{
+    vmport_bucket_t *b;
+    vmport_jumbo_bucket_t *jb;
+    int16_t recv_len;
+
+    if ( c->ctl.jumbo )
+    {
+        jb = &c->jumbo_recv_bkt;
+        recv_len = jb->ctl.recv_len;
+    }
+    else
+    {
+        b = &c->recv_bkt[c->ctl.recv_read];
+        recv_len = b->ctl.recv_len;
+    }
+    if ( recv_len )
+    {
+        set_status(ur, MESSAGE_STATUS_DORECV | MESSAGE_STATUS_SUCCESS);
+        ur->rdx = set_high_bits(ur->rdx, MESSAGE_TYPE_SENDSIZE);
+        ur->rbx = recv_len;
+    }
+    else
+        set_status(ur, MESSAGE_STATUS_SUCCESS);
+#ifndef NDEBUG
+    if ( (opt_vmport_debug & VMPORT_LOG_RECV_SIZE_ALL) ||
+         ((opt_vmport_debug & VMPORT_LOG_RECV_SIZE) && (recv_len)) )
+        gdprintk(XENLOG_DEBUG, "VMware RECVSIZE %d is %d.\n",
+                 c->ctl.chan_id, recv_len);
+#endif
+}
+
+void vmport_process_recv_payload(struct hvm_domain *hd, vmport_channel_t *c,
+                                 struct cpu_user_regs *ur)
+{
+    vmport_bucket_t *b;
+    vmport_jumbo_bucket_t *jb;
+
+    if ( c->ctl.jumbo )
+    {
+        jb = &c->jumbo_recv_bkt;
+        ur->rbx = jb->recv_buf[jb->ctl.recv_idx++];
+    }
+    else
+    {
+        b = &c->recv_bkt[c->ctl.recv_read];
+        if ( b->ctl.recv_idx < VMPORT_MAX_RECV_BUF )
+            ur->rbx = b->recv_buf[b->ctl.recv_idx++];
+        else
+            ur->rbx = 0;
+    }
+
+    set_status(ur, MESSAGE_STATUS_SUCCESS);
+    ur->rdx = set_high_bits(ur->rdx, MESSAGE_TYPE_SENDPAYLOAD);
+}
+
+void vmport_process_recv_status(struct hvm_domain *hd, vmport_channel_t *c,
+                                struct cpu_user_regs *ur)
+{
+    vmport_bucket_t *b;
+    vmport_jumbo_bucket_t *jb;
+
+    set_status(ur, MESSAGE_STATUS_SUCCESS);
+
+    if ( c->ctl.jumbo )
+    {
+        c->ctl.jumbo = 0;
+        /* add debug here */
+        jb = &c->jumbo_recv_bkt;
+        return;
+    }
+
+    b = &c->recv_bkt[c->ctl.recv_read];
+
+#ifndef NDEBUG
+    if ( unlikely(opt_vmport_debug & VMPORT_LOG_RECV_STATUS) )
+    {
+        char prefix[30];
+
+        snprintf(prefix, sizeof(prefix),
+                 "VMware RECVSTATUS %d (%d) ",
+                 c->ctl.chan_id, c->ctl.recv_read);
+        prefix[sizeof(prefix) - 1] = 0;
+        vmport_safe_print(prefix, b->ctl.recv_len, (char *)b->recv_buf);
+    }
+#endif
+    c->ctl.recv_read++;
+    if ( c->ctl.recv_read >= VMPORT_MAX_BKTS )
+        c->ctl.recv_read = 0;
+}
+
+void vmport_process_close(struct hvm_domain *hd, vmport_channel_t *c,
+                          struct cpu_user_regs *ur)
+{
+    /* Return channel to free pool */
+    c->ctl.proto_num = 0;
+    set_status(ur, MESSAGE_STATUS_SUCCESS);
+    VMPORT_DBG_LOG(VMPORT_LOG_CLOSE, "VMware CLOSE %d",
+                   c->ctl.chan_id);
+}
+
+void vmport_process_packet(struct hvm_domain *hd, vmport_channel_t *c,
+                           struct cpu_user_regs *ur, unsigned int sub_cmd,
+                           unsigned long now_time)
+{
+    c->ctl.active_time = now_time;
+
+    switch (sub_cmd)
+    {
+    case MESSAGE_TYPE_SENDSIZE:
+        vmport_process_send_size(hd, c, ur);
+        break;
+
+    case MESSAGE_TYPE_SENDPAYLOAD:
+        vmport_process_send_payload(hd, c, ur, now_time);
+        break;
+
+    case MESSAGE_TYPE_RECVSIZE:
+        vmport_process_recv_size(hd, c, ur);
+        break;
+
+    case MESSAGE_TYPE_RECVPAYLOAD:
+        vmport_process_recv_payload(hd, c, ur);
+        break;
+
+    case MESSAGE_TYPE_RECVSTATUS:
+        vmport_process_recv_status(hd, c, ur);
+        break;
+
+    case MESSAGE_TYPE_CLOSE:
+        vmport_process_close(hd, c, ur);
+        break;
+
+    default:
+        ur->rcx = 0;
+        break;
+    }
+}
+
+void vmport_rpc(struct hvm_domain *hd, struct cpu_user_regs *ur)
+{
+    unsigned int sub_cmd = get_high_bits(ur->rcx);
+    vmport_channel_t *c = NULL;
+    uint16_t msg_id;
+    uint32_t msg_cookie;
+    unsigned long now_time = get_sec();
+    long delta = now_time - hd->vmport_data->ping_time;
+
+    if ( delta > hd->params[HVM_PARAM_VMPORT_RESET_TIME] )
+    {
+        VMPORT_DBG_LOG(VMPORT_LOG_PING, "VMware ping. delta=%ld",
+                       delta);
+        vmport_ctrl_send(hd, "reset");
+    }
+    spin_lock(&hd->vmport_lock);
+    vmport_sweep(hd, now_time);
+    do {
+        /* Check to see if a new open request is happening... */
+        if ( MESSAGE_TYPE_OPEN == sub_cmd )
+        {
+            c = vmport_new_chan(hd->vmport_data, now_time);
+            if ( NULL == c )
+            {
+                VMPORT_DBG_LOG(VMPORT_LOG_ERROR,
+                               "VMware failed to find a free channel");
+                break;
+            }
+
+            /* Attach the apropriate protocol the the channel */
+            c->ctl.proto_num = ur->rbx & ~GUESTMSG_FLAG_COOKIE;
+            set_status(ur, MESSAGE_STATUS_SUCCESS);
+            ur->rdx = set_high_bits(ur->rdx, c->ctl.chan_id);
+            ur->rdi = get_low_bits(c->ctl.cookie);
+            ur->rsi = get_high_bits(c->ctl.cookie);
+            VMPORT_DBG_LOG(VMPORT_LOG_OPEN, "VMware OPEN %d p=%x",
+                           c->ctl.chan_id, c->ctl.proto_num);
+            if ( c->ctl.proto_num == VMWARE_PROTO_TO_GUEST )
+                vmport_send(hd, c, "reset");
+            break;
+        }
+
+        msg_id = get_high_bits(ur->rdx);
+        msg_cookie = set_high_bits(ur->rdi, ur->rsi);
+        if ( msg_id >= VMPORT_MAX_CHANS )
+        {
+            VMPORT_DBG_LOG(VMPORT_LOG_ERROR, "VMware chan id err %d >= %d",
+                           msg_id, VMPORT_MAX_CHANS);
+            break;
+        }
+        c = &hd->vmport_data->chans[msg_id];
+        if ( !c->ctl.proto_num )
+        {
+            VMPORT_DBG_LOG(VMPORT_LOG_ERROR, "VMware chan %d not open",
+                           msg_id);
+            break;
+        }
+
+        /* We check the cookie here since it's possible that the
+         * connection timed out on us and another channel was opened
+         * if this happens, return error and the um tool will
+         * need to reopen the connection
+         */
+        if ( msg_cookie != c->ctl.cookie )
+        {
+            VMPORT_DBG_LOG(VMPORT_LOG_ERROR, "VMware ctl.cookie err %x vs %x",
+                           msg_cookie, c->ctl.cookie);
+            break;
+        }
+        vmport_process_packet(hd, c, ur, sub_cmd, now_time);
+    } while ( 0 );
+
+    if ( NULL == c )
+        set_status(ur, 0);
+
+    spin_unlock(&hd->vmport_lock);
+}
+
+int vmport_ioport(int dir, uint32_t port, uint32_t bytes, uint32_t *val)
+{
+    struct cpu_user_regs *regs = guest_cpu_user_regs();
+    uint32_t cmd = get_low_bits(regs->rcx);
+    uint32_t magic = regs->rax;
+    int rc = X86EMUL_OKAY;
+
+    if ( magic == BDOOR_MAGIC )
+    {
+        uint64_t value;
+
+        if (dir == IOREQ_READ)
+            HVMTRACE_ND(VMPORT_READ1, 0, 1/*cycles*/, 6,
+                        regs->rax, regs->rbx, regs->rcx,
+                        regs->rdx, regs->rsi, regs->rdi);
+        else
+            HVMTRACE_ND(VMPORT_WRITE1, 0, 1/*cycles*/, 6,
+                        regs->rax, regs->rbx, regs->rcx,
+                        regs->rdx, regs->rsi, regs->rdi);
+
+#ifndef NDEBUG
+        if ( opt_vmport_debug & VMPORT_LOG_TRACE &&
+             (!(opt_vmport_debug & VMPORT_LOG_SKIP_MESSAGE) ||
+              (opt_vmport_debug & VMPORT_LOG_SKIP_MESSAGE &&
+               cmd != BDOOR_CMD_MESSAGE)) )
+            gdprintk(XENLOG_DEBUG,
+                     "VMware trace ip=%"PRIx64" cmd=%d ax=%"PRIx64" bx=%"PRIx64
+                     " cx=%"PRIx64" dx=%"PRIx64" si=%"PRIx64" di=%"PRIx64"\n",
+                     regs->rip, cmd, regs->rax, regs->rbx, regs->rcx, regs->rdx,
+                     regs->rsi, regs->rdi);
+#endif
+        switch (cmd)
+        {
+        case BDOOR_CMD_GETMHZ:
+            /* ... */
+            regs->rbx = BDOOR_MAGIC;
+            regs->rax = (uint32_t)(current->domain->arch.tsc_khz / 1000);
+            break;
+        case BDOOR_CMD_GETVERSION:
+            /* ... */
+            regs->rbx = BDOOR_MAGIC;
+            /* VERSION_MAGIC */
+            regs->rax = 6;
+            /* Claim we are an ESX. VMX_TYPE_SCALABLE_SERVER */
+            regs->rcx = 2;
+            break;
+        case BDOOR_CMD_GETHWVERSION:
+            /* ... */
+            regs->rbx = BDOOR_MAGIC;
+            /* ?? */
+            regs->rax = 0x4;
+            break;
+        case BDOOR_CMD_GETHZ:
+            value = current->domain->arch.tsc_khz * 1000;
+            /* apic-frequency (bus speed) */
+            regs->rcx = (uint32_t)(1000000000ULL / APIC_BUS_CYCLE_NS);
+            /* High part of tsc-frequency */
+            regs->rbx = (uint32_t)(value >> 32);
+            /* Low part of tsc-frequency */
+            regs->rax = (uint32_t)value;
+            break;
+        case BDOOR_CMD_GETTIME:
+            value = get_localtime_us(current->domain);
+            /* hostUsecs */
+            regs->rbx = (uint32_t)(value % 1000000UL);
+            /* hostSecs */
+            regs->rax = (uint32_t)(value / 1000000ULL);
+            /* maxTimeLag */
+            regs->rcx = 0;
+            break;
+        case BDOOR_CMD_GETTIMEFULL:
+            value = get_localtime_us(current->domain);
+            /* ... */
+            regs->rax = BDOOR_MAGIC;
+            /* hostUsecs */
+            regs->rbx = (uint32_t)(value % 1000000UL);
+            /* High part of hostSecs */
+            regs->rsi = (uint32_t)((value / 1000000ULL) >> 32);
+            /* Low part of hostSecs */
+            regs->rdx = (uint32_t)(value / 1000000ULL);
+            /* maxTimeLag */
+            regs->rcx = 0;
+            break;
+        case BDOOR_CMD_MESSAGE:
+            if ( !is_pv_vcpu(current) )
+            {
+                /* Only supported for non pv domains */
+                vmport_rpc(&current->domain->arch.hvm_domain, regs);
+            }
+            break;
+        case BDOOR_CMD_GETGUIOPTIONS:
+            regs->rax = VMWARE_GUI_AUTO_GRAB | VMWARE_GUI_AUTO_UNGRAB |
+                VMWARE_GUI_AUTO_RAISE_DISABLED | VMWARE_GUI_SYNC_TIME |
+                VMWARE_DISABLE_CURSOR_OPTIONS;
+            break;
+        case BDOOR_CMD_SETGUIOPTIONS:
+            regs->rax = 0x0;
+            break;
+        default:
+            VMPORT_DBG_LOG(VMPORT_LOG_ERROR,
+                           "VMware bytes=%d dir=%d cmd=%d",
+                           bytes, dir, cmd);
+            break;
+        }
+#ifndef NDEBUG
+        if ( opt_vmport_debug & VMPORT_LOG_VMWARE_AFTER &&
+             (!(opt_vmport_debug & VMPORT_LOG_SKIP_MESSAGE) ||
+              (opt_vmport_debug & VMPORT_LOG_SKIP_MESSAGE &&
+               cmd != BDOOR_CMD_MESSAGE)) )
+            gdprintk(XENLOG_DEBUG,
+                     "VMware after ip=%"PRIx64" cmd=%d ax=%"PRIx64" bx=%"PRIx64
+                     " cx=%"PRIx64" dx=%"PRIx64" si=%"PRIx64" di=%"PRIx64"\n",
+                     regs->rip, cmd, regs->rax, regs->rbx, regs->rcx, regs->rdx,
+                     regs->rsi, regs->rdi);
+#endif
+        if (dir == IOREQ_READ)
+            *val = regs->rax;
+    }
+    else
+    {
+        rc = X86EMUL_UNHANDLEABLE;
+        VMPORT_DBG_LOG(VMPORT_LOG_ERROR,
+                       "Not VMware %x vs %x vs %x; ip=%"PRIx64" ax=%"PRIx64
+                       " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64" si=%"PRIx64
+                       " di=%"PRIx64"",
+                       magic, BDOOR_MAGIC, VMPORT_MAGIC,
+                       regs->rip, regs->rax, regs->rbx, regs->rcx, regs->rdx,
+                       regs->rsi, regs->rdi);
+    }
+
+    if (dir == IOREQ_READ)
+        HVMTRACE_ND(VMPORT_READ2, 0, 1/*cycles*/, 6,
+                    regs->rax, regs->rbx, regs->rcx,
+                    regs->rdx, regs->rsi, regs->rdi);
+    else
+        HVMTRACE_ND(VMPORT_WRITE2, 0, 1/*cycles*/, 6,
+                    regs->rax, regs->rbx, regs->rcx,
+                    regs->rdx, regs->rsi, regs->rdi);
+
+    return rc;
+}
+
+/* save and restore functions */
+
+static int vmport_save_domain_ctxt(struct domain *d, hvm_domain_context_t *h)
+{
+    struct hvm_vmport_context *ctxt;
+    struct hvm_save_descriptor *desc;
+    struct hvm_domain *hd = &d->arch.hvm_domain;
+    struct vmport_state *vs = hd->vmport_data;
+    char *p;
+    unsigned int guestinfo_size = 0;
+    unsigned int used_guestinfo = 0;
+    unsigned int used_guestinfo_jumbo = 0;
+    unsigned int chans_size;
+    unsigned int i;
+
+    /* Customized handling for entry since our entry is of variable length */
+    desc = (struct hvm_save_descriptor *)&h->data[h->cur];
+    if ( _hvm_init_entry(h, HVM_SAVE_CODE(VMPORT), 0,
+                         HVM_SAVE_LENGTH(VMPORT)) )
+        return 1;
+    ctxt = (struct hvm_vmport_context *)&h->data[h->cur];
+
+    spin_lock(&hd->vmport_lock);
+
+    ctxt->version = VMPORT_SAVE_VERSION;
+    ctxt->ping_time = vs->ping_time;
+    ctxt->open_cookie = vs->open_cookie;
+    ctxt->used_guestinfo = vs->used_guestinfo;
+    ctxt->used_guestinfo_jumbo = vs->used_guestinfo_jumbo;
+
+    p = ctxt->u.packed.packed_data;
+
+    for (i = 0; i < VMPORT_MAX_CHANS; i++)
+    {
+        unsigned int j;
+        unsigned int buf_max;
+
+        ctxt->u.packed.chan_ctl[i].chan = vs->chans[i].ctl;
+        buf_max = vs->chans[i].ctl.send_len;
+        if ( buf_max > VMPORT_MAX_SEND_BUF * 4 )
+            buf_max = VMPORT_MAX_SEND_BUF * 4;
+        memcpy(p, vs->chans[i].send_buf, buf_max);
+        p += buf_max;
+        for (j = 0; j < VMPORT_MAX_BKTS; j++)
+        {
+            ctxt->u.packed.chan_ctl[i].recv[j] = vs->chans[i].recv_bkt[j].ctl;
+            buf_max = vs->chans[i].recv_bkt[j].ctl.recv_len;
+            if ( buf_max > VMPORT_MAX_RECV_BUF * 4 )
+                buf_max = VMPORT_MAX_RECV_BUF * 4;
+            memcpy(p, vs->chans[i].recv_bkt[j].recv_buf, buf_max);
+            p += buf_max;
+        }
+        ctxt->u.packed.chan_ctl[i].jumbo = vs->chans[i].jumbo_recv_bkt.ctl;
+        buf_max = vs->chans[i].jumbo_recv_bkt.ctl.recv_len;
+        if ( buf_max > VMPORT_MAX_RECV_JUMBO_BUF * 4 )
+            buf_max = VMPORT_MAX_RECV_JUMBO_BUF * 4;
+        memcpy(p, vs->chans[i].jumbo_recv_bkt.recv_buf, buf_max);
+        p += buf_max;
+    }
+
+    chans_size = p - ctxt->u.packed.packed_data;
+
+    for (i = 0; i < ctxt->used_guestinfo; i++)
+    {
+        vmport_guestinfo_t *vg = vs->guestinfo[i];
+
+        if ( vg && vg->key_len )
+        {
+            guestinfo_size += sizeof(vg->key_len) + sizeof(vg->val_len) +
+                vg->key_len + vg->val_len;
+            used_guestinfo++;
+            ASSERT(sizeof(vg->key_len) == 1);
+            *p++ = (char) vg->key_len;
+            ASSERT(sizeof(vg->val_len) == 1);
+            *p++ = (char) vg->val_len;
+            if ( vg->key_len )
+            {
+                memcpy(p, vg->key_data, vg->key_len);
+                p += vg->key_len;
+                if ( vg->val_len )
+                {
+                    memcpy(p, vg->val_data, vg->val_len);
+                    p += vg->val_len;
+                }
+            }
+        }
+    }
+    ctxt->used_guestinfo = used_guestinfo;
+
+    for (i = 0; i < ctxt->used_guestinfo_jumbo; i++)
+    {
+        vmport_guestinfo_jumbo_t *vgj =
+            vs->guestinfo_jumbo[i];
+        if ( vgj && vgj->key_len )
+        {
+            guestinfo_size += sizeof(vgj->key_len) + sizeof(vgj->val_len) +
+                vgj->key_len + vgj->val_len;
+            used_guestinfo_jumbo++;
+            ASSERT(sizeof(vgj->key_len) == 1);
+            *p++ = (char) vgj->key_len;
+            memcpy(p, &vgj->val_len, sizeof(vgj->val_len));
+            p += sizeof(vgj->val_len);
+            if ( vgj->key_len )
+            {
+                memcpy(p, vgj->key_data, vgj->key_len);
+                p += vgj->key_len;
+                if ( vgj->val_len )
+                {
+                    memcpy(p, vgj->val_data, vgj->val_len);
+                    p += vgj->val_len;
+                }
+            }
+        }
+    }
+    ctxt->used_guestinfo_jumbo = used_guestinfo_jumbo;
+
+    ctxt->used_guestsize = guestinfo_size;
+
+    spin_unlock(&hd->vmport_lock);
+
+#ifndef NDEBUG
+    gdprintk(XENLOG_WARNING, "chans_size=%d guestinfo_size=%d, used=%ld\n",
+             chans_size, guestinfo_size,
+             p - ctxt->u.packed.packed_data);
+#endif
+    ASSERT(p - ctxt->u.packed.packed_data == chans_size + guestinfo_size);
+    ASSERT(desc->length >= p - (char *)ctxt);
+    desc->length = p - (char *)ctxt; /* Fixup length to be right */
+    h->cur += desc->length; /* Do _hvm_write_entry */
+    ASSERT(guestinfo_size < desc->length);
+
+    return 0;
+}
+
+static int vmport_load_domain_ctxt(struct domain *d, hvm_domain_context_t *h)
+{
+    struct hvm_vmport_context *ctxt;
+    struct hvm_save_descriptor *desc;
+    struct hvm_domain *hd = &d->arch.hvm_domain;
+    struct vmport_state *vs = hd->vmport_data;
+    unsigned int i;
+    uint8_t key_len;
+    uint16_t val_len;
+    char *p;
+    vmport_guestinfo_t *vg;
+    vmport_guestinfo_jumbo_t *vgj;
+    unsigned int loop_cnt;
+    unsigned int guestinfo_size;
+    unsigned int used_guestinfo;
+    unsigned int used_guestinfo_jumbo;
+
+    if ( !vs )
+        return -ENOMEM;
+
+    /* Customized checking for entry since our entry is of variable length */
+    desc = (struct hvm_save_descriptor *)&h->data[h->cur];
+    if ( sizeof(*desc) > h->size - h->cur )
+    {
+        printk(XENLOG_G_WARNING
+               "HVM%d restore: not enough data left to read descriptor"
+               "for type %lu\n", d->domain_id,
+               HVM_SAVE_CODE(VMPORT));
+        return -1;
+    }
+    if ( desc->length + sizeof(*desc) > h->size - h->cur )
+    {
+        printk(XENLOG_G_WARNING
+               "HVM%d restore: not enough data left to read %u bytes "
+               "for type %lu\n", d->domain_id, desc->length,
+               HVM_SAVE_CODE(VMPORT));
+        return -1;
+    }
+    if ( HVM_SAVE_CODE(VMPORT) != desc->typecode ||
+         (desc->length > HVM_SAVE_LENGTH(VMPORT)) )
+    {
+        printk(XENLOG_G_WARNING
+               "HVM%d restore mismatch: expected type %lu with max length %lu, "
+               "saw type %u length %u\n", d->domain_id, HVM_SAVE_CODE(VMPORT),
+               HVM_SAVE_LENGTH(VMPORT),
+               desc->typecode, desc->length);
+        return -1;
+    }
+    h->cur += sizeof(*desc);
+    /* Checking finished */
+
+    ctxt = (struct hvm_vmport_context *)&h->data[h->cur];
+    h->cur += desc->length;
+
+    if ( ctxt->version != VMPORT_SAVE_VERSION )
+        return -EINVAL;
+
+    spin_lock(&hd->vmport_lock);
+
+    vs->ping_time = ctxt->ping_time;
+    vs->open_cookie = ctxt->open_cookie;
+    vs->used_guestinfo = ctxt->used_guestinfo;
+    vs->used_guestinfo_jumbo = ctxt->used_guestinfo_jumbo;
+    guestinfo_size = ctxt->used_guestsize;
+    used_guestinfo = ctxt->used_guestinfo;
+    used_guestinfo_jumbo = ctxt->used_guestinfo_jumbo;
+
+    p = ctxt->u.packed.packed_data;
+
+    for (i = 0; i < VMPORT_MAX_CHANS; i++)
+    {
+        unsigned int j;
+
+        vs->chans[i].ctl = ctxt->u.packed.chan_ctl[i].chan;
+        memcpy(vs->chans[i].send_buf, p, vs->chans[i].ctl.send_len);
+        p += vs->chans[i].ctl.send_len;
+        for (j = 0; j < VMPORT_MAX_BKTS; j++)
+        {
+            vs->chans[i].recv_bkt[j].ctl = ctxt->u.packed.chan_ctl[i].recv[j];
+            memcpy(vs->chans[i].recv_bkt[j].recv_buf, p,
+                   vs->chans[i].recv_bkt[j].ctl.recv_len);
+            p += vs->chans[i].recv_bkt[j].ctl.recv_len;
+        }
+        vs->chans[i].jumbo_recv_bkt.ctl = ctxt->u.packed.chan_ctl[i].jumbo;
+        memcpy(vs->chans[i].jumbo_recv_bkt.recv_buf, p,
+               vs->chans[i].jumbo_recv_bkt.ctl.recv_len);
+        p += vs->chans[i].jumbo_recv_bkt.ctl.recv_len;
+    }
+
+
+    /* keep at least 10 total and 5 empty entries */
+    loop_cnt = (vs->used_guestinfo + 5) > 10 ?
+        (vs->used_guestinfo + 5) : 10;
+    for (i = 0; i < loop_cnt; i++)
+    {
+        if ( !vs->guestinfo[i] )
+        {
+            vs->guestinfo[i] = xzalloc(vmport_guestinfo_t);
+        }
+        if ( i < vs->used_guestinfo
+             && guestinfo_size > 0 )
+        {
+            key_len = (uint8_t)*p++;
+            val_len = (uint8_t)*p++;
+            guestinfo_size -= 2;
+            if ( guestinfo_size >= key_len + val_len )
+            {
+                vg = vs->guestinfo[i];
+                if ( key_len )
+                {
+                    vg->key_len = key_len;
+                    vg->val_len = val_len;
+                    memcpy(vg->key_data, p, key_len);
+                    p += key_len;
+                    memcpy(vg->val_data, p, val_len);
+                    p += val_len;
+                    guestinfo_size -= key_len + val_len;
+                }
+            }
+        }
+    }
+    vs->used_guestinfo = loop_cnt;
+
+    /* keep at least 2 total and 1 empty entries */
+    loop_cnt = (vs->used_guestinfo_jumbo + 1) > 2 ?
+        (vs->used_guestinfo_jumbo + 1) : 2;
+    for (i = 0; i < loop_cnt; i++)
+    {
+        if ( !vs->guestinfo_jumbo[i] )
+        {
+            vs->guestinfo_jumbo[i] = xzalloc(vmport_guestinfo_jumbo_t);
+        }
+        if ( i < vs->used_guestinfo_jumbo
+             && guestinfo_size > 0 )
+        {
+            key_len = (uint8_t)*p++;
+            memcpy(&val_len, p, 2);
+            p += 2;
+            guestinfo_size -= 3;
+            if ( guestinfo_size >= key_len + val_len )
+            {
+                vgj = vs->guestinfo_jumbo[i];
+                if ( key_len )
+                {
+                    vgj->key_len = key_len;
+                    vgj->val_len = val_len;
+                    memcpy(vgj->key_data, p, key_len);
+                    p += key_len;
+                    memcpy(vgj->val_data, p, val_len);
+                    p += val_len;
+                    guestinfo_size -= key_len + val_len;
+                }
+            }
+        }
+    }
+    vs->used_guestinfo_jumbo = loop_cnt;
+
+    spin_unlock(&hd->vmport_lock);
+
+    return 0;
+}
+
+HVM_REGISTER_SAVE_RESTORE(VMPORT, vmport_save_domain_ctxt,
+                          vmport_load_domain_ctxt, 1, HVMSR_PER_DOM);
+
+/*
+ * Local variables:
+ * mode: C
+ * c-set-style: "BSD"
+ * c-basic-offset: 4
+ * tab-width: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c
index 4a4f4e1..fa1f69a 100644
--- a/xen/arch/x86/hvm/vmx/vmcs.c
+++ b/xen/arch/x86/hvm/vmx/vmcs.c
@@ -1077,6 +1077,7 @@ static int construct_vmcs(struct vcpu *v)
 
     v->arch.hvm_vmx.exception_bitmap = HVM_TRAP_MASK
               | (paging_mode_hap(d) ? 0 : (1U << TRAP_page_fault))
+              | (1U << TRAP_gp_fault)
               | (1U << TRAP_no_device);
     vmx_update_exception_bitmap(v);
 
diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c
index 26b1ad5..f807d36 100644
--- a/xen/arch/x86/hvm/vmx/vmx.c
+++ b/xen/arch/x86/hvm/vmx/vmx.c
@@ -44,6 +44,7 @@
 #include <asm/hvm/support.h>
 #include <asm/hvm/vmx/vmx.h>
 #include <asm/hvm/vmx/vmcs.h>
+#include <asm/hvm/vmport.h>
 #include <public/sched.h>
 #include <public/hvm/ioreq.h>
 #include <asm/hvm/vpic.h>
@@ -1279,6 +1280,7 @@ static void vmx_update_guest_cr(struct vcpu *v, unsigned int cr)
                 v->arch.hvm_vmx.exception_bitmap = HVM_TRAP_MASK
                           | (paging_mode_hap(v->domain) ?
                              0 : (1U << TRAP_page_fault))
+                          | (1U << TRAP_gp_fault)
                           | (1U << TRAP_no_device);
                 vmx_update_exception_bitmap(v);
                 vmx_update_debug_state(v);
@@ -2565,6 +2567,82 @@ static void vmx_idtv_reinject(unsigned long idtv_info)
     }
 }
 
+void do_gp_fault(struct cpu_user_regs *regs, struct vcpu *v)
+{
+    unsigned long exit_qualification;
+    unsigned long inst_len;
+    unsigned long ecode;
+
+    __vmread(EXIT_QUALIFICATION, &exit_qualification);
+    __vmread(VM_EXIT_INSTRUCTION_LEN, &inst_len);
+    __vmread(VM_EXIT_INTR_ERROR_CODE, &ecode);
+    regs->error_code = ecode;
+    if ( hvm_long_mode_enabled(v) )
+        HVMTRACE_LONG_C4D(TRAP_GP, TRAP_gp_fault, inst_len, ecode,
+                          TRC_PAR_LONG(exit_qualification));
+    else
+        HVMTRACE_C4D(TRAP_GP, TRAP_gp_fault, inst_len, ecode,
+                     exit_qualification);
+
+    if ( inst_len == 1 && (regs->rdx & 0xffff) == VMPORT_PORT &&
+         exit_qualification == 0 && ecode == 0 )
+    {
+        uint32_t magic = regs->rax;
+
+        if ( magic == VMPORT_MAGIC )
+        {
+            unsigned char bytes[1] = { 0 };
+            int frc = hvm_fetch_from_guest_virt_nofault(bytes, regs->rip,
+                                                        1, PFEC_page_present);
+
+            if ( !frc && bytes[0] == 0xed ) /* in (%dx),%eax */
+            {
+                uint32_t val;
+
+                update_guest_eip();
+                vmport_ioport(IOREQ_READ, VMPORT_PORT, 4, &val);
+                VMPORT_DBG_LOG(VMPORT_LOG_VMWARE_AFTER,
+                               "gp: VMware ip=%"PRIx64" ax=%"PRIx64
+                               " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                               " si=%"PRIx64" di=%"PRIx64,
+                               regs->rip, regs->rax, regs->rbx, regs->rcx,
+                               regs->rdx, regs->rsi, regs->rdi);
+                return;
+            }
+            else
+            {
+                VMPORT_DBG_LOG(VMPORT_LOG_GP_FAIL_RD_INST,
+                               "gp: VMware? ip=%"PRIx64"=>0x%x(%d) ax=%"PRIx64
+                               " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                               " si=%"PRIx64" di=%"PRIx64,
+                               regs->rip, bytes[0], frc, regs->rax, regs->rbx,
+                               regs->rcx, regs->rdx, regs->rsi, regs->rdi);
+            }
+        }
+        else
+        {
+            VMPORT_DBG_LOG(VMPORT_LOG_GP_NOT_VMWARE,
+                           "gp: ip=%"PRIx64" ax=%"PRIx64" bx=%"PRIx64
+                           " cx=%"PRIx64" dx=%"PRIx64" si=%"PRIx64
+                           " di=%"PRIx64,
+                           regs->rip, regs->rax, regs->rbx, regs->rcx,
+                           regs->rdx, regs->rsi, regs->rdi);
+            hvm_inject_hw_exception(TRAP_gp_fault, ecode);
+        }
+    }
+    else
+    {
+        VMPORT_DBG_LOG(VMPORT_LOG_GP_UNKNOWN,
+                       "gp: eq=%lx ec=%lx ip=%"PRIx64" (%ld) ax=%"PRIx64
+                       " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                       " si=%"PRIx64" di=%"PRIx64,
+                       exit_qualification, ecode,
+                       regs->rip, inst_len, regs->rax, regs->rbx, regs->rcx,
+                       regs->rdx, regs->rsi, regs->rdi);
+        hvm_inject_hw_exception(TRAP_gp_fault, ecode);
+    }
+}
+
 static int vmx_handle_apic_write(void)
 {
     unsigned long exit_qualification;
@@ -2675,6 +2753,15 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
                  && vector != TRAP_nmi 
                  && vector != TRAP_machine_check ) 
             {
+                if ( vector == TRAP_gp_fault )
+                {
+                    VMPORT_DBG_LOG(VMPORT_LOG_REALMODE_GP,
+                                   "realmode gp: ip=%"PRIx64" ax=%"PRIx64
+                                   " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                                   " si=%"PRIx64" di=%"PRIx64,
+                                   regs->rip, regs->rax, regs->rbx, regs->rcx,
+                                   regs->rdx, regs->rsi, regs->rdi);
+                }
                 perfc_incr(realmode_exits);
                 v->arch.hvm_vmx.vmx_emulate = 1;
                 HVMTRACE_0D(REALMODE_EMULATE);
@@ -2790,6 +2877,9 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
             HVMTRACE_1D(TRAP, vector);
             vmx_fpu_dirty_intercept();
             break;
+        case TRAP_gp_fault:
+            do_gp_fault(regs, v);
+            break;
         case TRAP_page_fault:
             __vmread(EXIT_QUALIFICATION, &exit_qualification);
             __vmread(VM_EXIT_INTR_ERROR_CODE, &ecode);
diff --git a/xen/arch/x86/hvm/vmx/vvmx.c b/xen/arch/x86/hvm/vmx/vvmx.c
index 9ccc03f..734d88e 100644
--- a/xen/arch/x86/hvm/vmx/vvmx.c
+++ b/xen/arch/x86/hvm/vmx/vvmx.c
@@ -24,6 +24,7 @@
 #include <asm/types.h>
 #include <asm/mtrr.h>
 #include <asm/p2m.h>
+#include <asm/hvm/vmport.h>
 #include <asm/hvm/vmx/vmx.h>
 #include <asm/hvm/vmx/vvmx.h>
 #include <asm/hvm/nestedhvm.h>
@@ -2182,6 +2183,19 @@ int nvmx_n2_vmexit_handler(struct cpu_user_regs *regs,
             if ( v->fpu_dirtied )
                 nvcpu->nv_vmexit_pending = 1;
         }
+        else if ( vector == TRAP_gp_fault )
+        {
+#ifndef NDEBUG
+            struct cpu_user_regs *ur = guest_cpu_user_regs();
+#endif
+            VMPORT_DBG_LOG(VMPORT_LOG_VGP_UNKNOWN,
+                           "Unexpected gp: ip=%"PRIx64" ax=%"PRIx64
+                           " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
+                           " si=%"PRIx64" di=%"PRIx64,
+                           ur->rip, ur->rax, ur->rbx, ur->rcx, ur->rdx,
+                           ur->rsi, ur->rdi);
+                nvcpu->nv_vmexit_pending = 1;
+        }
         else if ( (intr_info & valid_mask) == valid_mask )
         {
             exec_bitmap =__get_vvmcs(nvcpu->nv_vvmcx, EXCEPTION_BITMAP);
diff --git a/xen/include/asm-x86/hvm/domain.h b/xen/include/asm-x86/hvm/domain.h
index 526b5da..7bbf1de 100644
--- a/xen/include/asm-x86/hvm/domain.h
+++ b/xen/include/asm-x86/hvm/domain.h
@@ -108,6 +108,10 @@ struct hvm_domain {
     /* emulated irq to pirq */
     struct radix_tree_root emuirq_pirq;
 
+    /* VMware special port */
+    spinlock_t             vmport_lock;
+    struct vmport_state   *vmport_data;
+
     uint64_t              *params;
 
     /* Memory ranges with pinned cache attributes. */
diff --git a/xen/include/asm-x86/hvm/svm/emulate.h b/xen/include/asm-x86/hvm/svm/emulate.h
index ccc2d3c..4ff1aea 100644
--- a/xen/include/asm-x86/hvm/svm/emulate.h
+++ b/xen/include/asm-x86/hvm/svm/emulate.h
@@ -39,6 +39,7 @@ enum instruction_index {
     INSTR_STGI,
     INSTR_CLGI,
     INSTR_INVLPGA,
+    INSTR_IN,
     INSTR_MAX_COUNT /* Must be last - Number of instructions supported */
 };
 
diff --git a/xen/include/asm-x86/hvm/trace.h b/xen/include/asm-x86/hvm/trace.h
index de802a6..2cd60c2 100644
--- a/xen/include/asm-x86/hvm/trace.h
+++ b/xen/include/asm-x86/hvm/trace.h
@@ -52,8 +52,15 @@
 #define DO_TRC_HVM_LMSW64      DEFAULT_HVM_MISC
 #define DO_TRC_HVM_REALMODE_EMULATE DEFAULT_HVM_MISC 
 #define DO_TRC_HVM_TRAP             DEFAULT_HVM_MISC
+#define DO_TRC_HVM_TRAP64           DEFAULT_HVM_MISC
 #define DO_TRC_HVM_TRAP_DEBUG       DEFAULT_HVM_MISC
 #define DO_TRC_HVM_VLAPIC           DEFAULT_HVM_MISC
+#define DO_TRC_HVM_TRAP_GP          DEFAULT_HVM_MISC
+#define DO_TRC_HVM_TRAP_GP64        DEFAULT_HVM_MISC
+#define DO_TRC_HVM_VMPORT_READ1     DEFAULT_HVM_IO
+#define DO_TRC_HVM_VMPORT_WRITE1    DEFAULT_HVM_IO
+#define DO_TRC_HVM_VMPORT_READ2     DEFAULT_HVM_IO
+#define DO_TRC_HVM_VMPORT_WRITE2    DEFAULT_HVM_IO
 
 
 #define TRC_PAR_LONG(par) ((par)&0xFFFFFFFF),((par)>>32)
@@ -98,6 +105,21 @@
 #define HVMTRACE_0D(evt)                            \
     HVMTRACE_ND(evt, 0, 0, 0,  0,  0,  0,  0,  0,  0)
 
+#define HVMTRACE_C6D(evt, d1, d2, d3, d4, d5, d6)    \
+    HVMTRACE_ND(evt, 0, 1, 6, d1, d2, d3, d4, d5, d6)
+#define HVMTRACE_C5D(evt, d1, d2, d3, d4, d5)        \
+    HVMTRACE_ND(evt, 0, 1, 5, d1, d2, d3, d4, d5,  0)
+#define HVMTRACE_C4D(evt, d1, d2, d3, d4)            \
+    HVMTRACE_ND(evt, 0, 1, 4, d1, d2, d3, d4,  0,  0)
+#define HVMTRACE_C3D(evt, d1, d2, d3)                \
+    HVMTRACE_ND(evt, 0, 1, 3, d1, d2, d3,  0,  0,  0)
+#define HVMTRACE_C2D(evt, d1, d2)                    \
+    HVMTRACE_ND(evt, 0, 1, 2, d1, d2,  0,  0,  0,  0)
+#define HVMTRACE_C1D(evt, d1)                        \
+    HVMTRACE_ND(evt, 0, 1, 1, d1,  0,  0,  0,  0,  0)
+#define HVMTRACE_C0D(evt)                            \
+    HVMTRACE_ND(evt, 0, 1, 0,  0,  0,  0,  0,  0,  0)
+
 #define HVMTRACE_LONG_1D(evt, d1)                  \
                    HVMTRACE_2D(evt ## 64, (d1) & 0xFFFFFFFF, (d1) >> 32)
 #define HVMTRACE_LONG_2D(evt, d1, d2, ...)              \
@@ -107,6 +129,15 @@
 #define HVMTRACE_LONG_4D(evt, d1, d2, d3, d4, ...)  \
                    HVMTRACE_5D(evt ## 64, d1, d2, d3, d4)
 
+#define HVMTRACE_LONG_C1D(evt, d1)                  \
+                   HVMTRACE_C2D(evt ## 64, (d1) & 0xFFFFFFFF, (d1) >> 32)
+#define HVMTRACE_LONG_C2D(evt, d1, d2, ...)              \
+                   HVMTRACE_C3D(evt ## 64, d1, d2)
+#define HVMTRACE_LONG_C3D(evt, d1, d2, d3, ...)      \
+                   HVMTRACE_C4D(evt ## 64, d1, d2, d3)
+#define HVMTRACE_LONG_C4D(evt, d1, d2, d3, d4, ...)  \
+                   HVMTRACE_C5D(evt ## 64, d1, d2, d3, d4)
+
 #endif /* __ASM_X86_HVM_TRACE_H__ */
 
 /*
diff --git a/xen/include/asm-x86/hvm/vmport.h b/xen/include/asm-x86/hvm/vmport.h
new file mode 100644
index 0000000..e72e422
--- /dev/null
+++ b/xen/include/asm-x86/hvm/vmport.h
@@ -0,0 +1,90 @@
+/*
+ * vmport.h: HVM VMPORT emulation
+ *
+ *
+ * Copyright (C) 2012 Verizon Corporation
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License Version 2 (GPLv2)
+ * as published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details. <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_X86_HVM_VMPORT_H__
+#define __ASM_X86_HVM_VMPORT_H__
+
+#ifndef NDEBUG
+#define VMPORT_LOG_RPC             (1 << 0)
+#define VMPORT_LOG_RECV_STATUS     (1 << 1)
+#define VMPORT_LOG_SKIP_SEND       (1 << 2)
+#define VMPORT_LOG_SEND            (1 << 3)
+#define VMPORT_LOG_SEND_SIZE_ALL   (1 << 4)
+#define VMPORT_LOG_SEND_SIZE       (1 << 5)
+#define VMPORT_LOG_RECV_SIZE_ALL   (1 << 6)
+#define VMPORT_LOG_RECV_SIZE       (1 << 7)
+#define VMPORT_LOG_CLOSE           (1 << 8)
+#define VMPORT_LOG_OPEN            (1 << 9)
+#define VMPORT_LOG_FLUSH           (1 << 10)
+#define VMPORT_LOG_TRACE           (1 << 11)
+#define VMPORT_LOG_PING            (1 << 12)
+#define VMPORT_LOG_SWEEP           (1 << 13)
+#define VMPORT_LOG_BUILD           (1 << 14)
+
+#define VMPORT_LOG_ERROR           (1 << 16)
+
+#define VMPORT_LOG_INFO_GET        (1 << 17)
+#define VMPORT_LOG_INFO_SET        (1 << 18)
+
+#define VMPORT_LOG_SKIP_MESSAGE    (1 << 19)
+
+#define VMPORT_LOG_GP_UNKNOWN      (1 << 20)
+#define VMPORT_LOG_GP_NOT_VMWARE   (1 << 21)
+#define VMPORT_LOG_GP_FAIL_RD_INST (1 << 22)
+
+#define VMPORT_LOG_GP_VMWARE_AFTER (1 << 23)
+
+#define VMPORT_LOG_VGP_UNKNOWN     (1 << 24)
+#define VMPORT_LOG_REALMODE_GP     (1 << 27)
+
+#define VMPORT_LOG_VMWARE_AFTER    (1 << 28)
+
+extern unsigned int opt_vmport_debug;
+#define VMPORT_DBG_LOG(level, _f, _a...)                                \
+    do {                                                                \
+        if ( unlikely((level) & opt_vmport_debug) )                     \
+            printk("[HVM:%d.%d] <%s> " _f "\n",                         \
+                   current->domain->domain_id, current->vcpu_id, __func__, \
+                   ## _a);                                              \
+    } while (0)
+#else
+#define VMPORT_DBG_LOG(level, _f, _a...) do {} while (0)
+#endif
+
+/* Note: VMPORT_PORT and VMPORT_MAGIC is also defined as BDOOR_PORT
+ * and BDOOR_MAGIC in backdoor_def.h Defined here so that other
+ * parts of XEN can use it.
+ */
+
+#define VMPORT_PORT 0x5658
+#define VMPORT_MAGIC 0x564D5868
+
+#include "public/arch-x86/hvm/vmporttype.h"
+
+int vmport_ioport(int dir, uint32_t port, uint32_t bytes, uint32_t *val);
+void vmport_ctrl_send(struct hvm_domain *hd, char *msg);
+void vmport_flush(struct hvm_domain *hd);
+
+#endif /* __ASM_X86_HVM_VMPORT_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/arch-x86/hvm/save.h b/xen/include/public/arch-x86/hvm/save.h
index 16d85a3..7cdcb8f 100644
--- a/xen/include/public/arch-x86/hvm/save.h
+++ b/xen/include/public/arch-x86/hvm/save.h
@@ -26,6 +26,8 @@
 #ifndef __XEN_PUBLIC_HVM_SAVE_X86_H__
 #define __XEN_PUBLIC_HVM_SAVE_X86_H__
 
+#include "vmporttype.h"
+
 /* 
  * Save/restore header: general info about the save file. 
  */
@@ -610,9 +612,44 @@ struct hvm_msr {
 
 #define CPU_MSR_CODE  20
 
+/*
+ * VMware context.
+ */
+struct hvm_vmport_context {
+    uint32_t version;
+    uint32_t used_guestsize;
+    uint64_t ping_time;
+    uint32_t open_cookie;
+    uint32_t used_guestinfo;
+    uint32_t used_guestinfo_jumbo;
+    union {
+        struct {
+            vmport_channel_t chans[VMPORT_MAX_CHANS];
+            vmport_guestinfo_t guestinfo[VMPORT_MAX_NUM_KEY];
+            vmport_guestinfo_jumbo_t guestinfo_jumbo[VMPORT_MAX_NUM_JUMBO_KEY];
+        } full;
+        struct {
+            struct {
+                vmport_channel_control_t chan;
+                vmport_bucket_control_t recv[VMPORT_MAX_BKTS];
+                vmport_bucket_control_t jumbo;
+            } chan_ctl[VMPORT_MAX_CHANS];
+#if defined(__STDC_VERSION__) && __STDC_VERSION__ >= 199901L
+            char packed_data[];
+#elif defined(__GNUC__)
+            char packed_data[0];
+#else
+            char packed_data[1 /* Variable length */];
+#endif
+        } packed;
+    } u;
+};
+
+DECLARE_HVM_SAVE_TYPE(VMPORT, 21, struct hvm_vmport_context);
+
 /* 
  * Largest type-code in use
  */
-#define HVM_SAVE_CODE_MAX 20
+#define HVM_SAVE_CODE_MAX 21
 
 #endif /* __XEN_PUBLIC_HVM_SAVE_X86_H__ */
diff --git a/xen/include/public/arch-x86/hvm/vmporttype.h b/xen/include/public/arch-x86/hvm/vmporttype.h
new file mode 100644
index 0000000..3ecbe25
--- /dev/null
+++ b/xen/include/public/arch-x86/hvm/vmporttype.h
@@ -0,0 +1,105 @@
+/*
+ * vmporttype.h: HVM VMPORT structure definitions
+ *
+ *
+ * Copyright (C) 2012 Verizon Corporation
+ *
+ * This file is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License Version 2 (GPLv2)
+ * as published by the Free Software Foundation.
+ *
+ * This file is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details. <http://www.gnu.org/licenses/>.
+ */
+
+#ifndef __ASM_X86_HVM_VMPORTTYPE_H__
+#define __ASM_X86_HVM_VMPORTTYPE_H__
+
+#define VMPORT_MAX_KEY_LEN 30
+#define VMPORT_MAX_VAL_LEN 128
+#define VMPORT_MAX_NUM_KEY  64
+#define VMPORT_MAX_NUM_JUMBO_KEY 4
+#define VMPORT_MAX_VAL_JUMBO_LEN 4096
+
+#define VMPORT_MAX_SEND_BUF ((22 + VMPORT_MAX_KEY_LEN + \
+                              VMPORT_MAX_VAL_JUMBO_LEN + 3)/4)
+#define VMPORT_MAX_RECV_BUF ((2 + VMPORT_MAX_VAL_LEN + 3)/4)
+#define VMPORT_MAX_RECV_JUMBO_BUF ((2 + VMPORT_MAX_VAL_JUMBO_LEN + 3)/4)
+#define VMPORT_MAX_CHANS    6
+#define VMPORT_MAX_BKTS     8
+
+#define VMPORT_SAVE_VERSION 0xabcd0001
+
+typedef struct {
+    uint8_t key_len;
+    uint8_t val_len;
+    char key_data[VMPORT_MAX_KEY_LEN];
+    char val_data[VMPORT_MAX_VAL_LEN];
+} vmport_guestinfo_t;
+
+typedef struct {
+    uint16_t val_len;
+    uint8_t  key_len;
+    char     key_data[VMPORT_MAX_KEY_LEN];
+    char     val_data[VMPORT_MAX_VAL_JUMBO_LEN];
+} vmport_guestinfo_jumbo_t;
+
+typedef struct __attribute__ ((packed)) {
+    uint16_t recv_len;
+    uint16_t recv_idx;
+} vmport_bucket_control_t;
+
+typedef struct __attribute__ ((packed)) {
+    vmport_bucket_control_t ctl;
+    uint32_t recv_buf[VMPORT_MAX_RECV_BUF + 1];
+} vmport_bucket_t;
+
+typedef struct __attribute__ ((packed)) {
+    vmport_bucket_control_t ctl;
+    uint32_t recv_buf[VMPORT_MAX_RECV_JUMBO_BUF + 1];
+} vmport_jumbo_bucket_t;
+
+typedef struct __attribute__ ((packed)) {
+    uint64_t active_time;
+    uint32_t chan_id;
+    uint32_t cookie;
+    uint32_t proto_num;
+    uint16_t send_len;
+    uint16_t send_idx;
+    uint8_t jumbo;
+    uint8_t recv_read;
+    uint8_t recv_write;
+    uint8_t recv_chan_pad[1];
+} vmport_channel_control_t;
+
+typedef struct __attribute__ ((packed)) {
+    vmport_channel_control_t ctl;
+    vmport_bucket_t recv_bkt[VMPORT_MAX_BKTS];
+    vmport_jumbo_bucket_t jumbo_recv_bkt;
+    uint32_t send_buf[VMPORT_MAX_SEND_BUF + 1];
+} vmport_channel_t;
+
+struct vmport_state {
+    uint64_t ping_time;
+    uint32_t open_cookie;
+    uint32_t used_guestinfo;
+    uint32_t used_guestinfo_jumbo;
+    uint8_t  max_chans;
+    uint8_t  state_pad[3];
+    vmport_channel_t chans[VMPORT_MAX_CHANS];
+    vmport_guestinfo_t *guestinfo[VMPORT_MAX_NUM_KEY];
+    vmport_guestinfo_jumbo_t *guestinfo_jumbo[VMPORT_MAX_NUM_JUMBO_KEY];
+};
+
+#endif /* __ASM_X86_HVM_VMPORTTYPE_H__ */
+
+/*
+ * Local variables:
+ * mode: C
+ * c-file-style: "BSD"
+ * c-basic-offset: 4
+ * indent-tabs-mode: nil
+ * End:
+ */
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 5b11bbf..fd4cc8a 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -458,6 +458,9 @@ DEFINE_XEN_GUEST_HANDLE(xen_domctl_address_size_t);
 #define XEN_DOMCTL_SENDTRIGGER_INIT   2
 #define XEN_DOMCTL_SENDTRIGGER_POWER  3
 #define XEN_DOMCTL_SENDTRIGGER_SLEEP  4
+#define XEN_DOMCTL_SENDTRIGGER_VTPOWER  5
+#define XEN_DOMCTL_SENDTRIGGER_VTREBOOT 6
+#define XEN_DOMCTL_SENDTRIGGER_VTPING   7
 struct xen_domctl_sendtrigger {
     uint32_t  trigger;  /* IN */
     uint32_t  vcpu;     /* IN */
diff --git a/xen/include/public/hvm/hvm_op.h b/xen/include/public/hvm/hvm_op.h
index eeb0a60..185e33d 100644
--- a/xen/include/public/hvm/hvm_op.h
+++ b/xen/include/public/hvm/hvm_op.h
@@ -369,6 +369,24 @@ DEFINE_XEN_GUEST_HANDLE(xen_hvm_set_ioreq_server_state_t);
 
 #endif /* defined(__XEN__) || defined(__XEN_TOOLS__) */
 
+/* Get/set vmport subcommands */
+#define HVMOP_get_vmport_guest_info 23
+#define HVMOP_set_vmport_guest_info 24
+#define VMPORT_GUEST_INFO_KEY_MAX 40
+#define VMPORT_GUEST_INFO_VAL_MAX 4096
+struct xen_hvm_vmport_guest_info {
+    /* Domain to be accessed */
+    domid_t   domid;
+    /* key length */
+    uint16_t   key_length;
+    /* value length */
+    uint16_t   value_length;
+    /* key and value data */
+    char      data[VMPORT_GUEST_INFO_KEY_MAX + VMPORT_GUEST_INFO_VAL_MAX];
+};
+typedef struct xen_hvm_vmport_guest_info xen_hvm_vmport_guest_info_t;
+DEFINE_XEN_GUEST_HANDLE(xen_hvm_vmport_guest_info_t);
+
 #endif /* __XEN_PUBLIC_HVM_HVM_OP_H__ */
 
 /*
diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h
index dee6d68..722e30b 100644
--- a/xen/include/public/hvm/params.h
+++ b/xen/include/public/hvm/params.h
@@ -153,7 +153,10 @@
 
 /* Params for VMware */
 #define HVM_PARAM_VMWARE_HW                 35
+#define HVM_PARAM_VMPORT_BUILD_NUMBER_TIME  36
+#define HVM_PARAM_VMPORT_BUILD_NUMBER_VALUE 37
+#define HVM_PARAM_VMPORT_RESET_TIME         38
 
-#define HVM_NR_PARAMS          36
+#define HVM_NR_PARAMS          39
 
 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */
diff --git a/xen/include/public/trace.h b/xen/include/public/trace.h
index cfcf4aa..b81fc66 100644
--- a/xen/include/public/trace.h
+++ b/xen/include/public/trace.h
@@ -224,11 +224,18 @@
 #define TRC_HVM_NPF             (TRC_HVM_HANDLER + 0x21)
 #define TRC_HVM_REALMODE_EMULATE (TRC_HVM_HANDLER + 0x22)
 #define TRC_HVM_TRAP             (TRC_HVM_HANDLER + 0x23)
+#define TRC_HVM_TRAP64           (TRC_HVM_HANDLER + TRC_64_FLAG + 0x23)
 #define TRC_HVM_TRAP_DEBUG       (TRC_HVM_HANDLER + 0x24)
 #define TRC_HVM_VLAPIC           (TRC_HVM_HANDLER + 0x25)
+#define TRC_HVM_TRAP_GP          (TRC_HVM_HANDLER + 0x26)
+#define TRC_HVM_TRAP_GP64        (TRC_HVM_HANDLER + TRC_64_FLAG + 0x26)
+#define TRC_HVM_VMPORT_READ1     (TRC_HVM_HANDLER + 0x27)
+#define TRC_HVM_VMPORT_READ2     (TRC_HVM_HANDLER + 0x28)
 
 #define TRC_HVM_IOPORT_WRITE    (TRC_HVM_HANDLER + 0x216)
 #define TRC_HVM_IOMEM_WRITE     (TRC_HVM_HANDLER + 0x217)
+#define TRC_HVM_VMPORT_WRITE1   (TRC_HVM_HANDLER + 0x227)
+#define TRC_HVM_VMPORT_WRITE2   (TRC_HVM_HANDLER + 0x228)
 
 /* Trace events for emulated devices */
 #define TRC_HVM_EMUL_HPET_START_TIMER  (TRC_HVM_EMUL + 0x1)
-- 
1.8.4

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Jan Beulich]

>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> This is both a more complete support then in currently provided by
> QEMU and/or KVM and less.  The missing part requires QEMU changes
> and has been left out until the QEMU patches are accepted upstream.
> 
> VMware's hyper-call is also known as VMware Backdoor I/O Port.
> 
> Note: this support does not depend on vmware_hw being non-zero.
> 
> Summary is that VMware treats "IN EAX, DX" (or "OUT DX, EAX"; or
> "inl %dx, %eax" in AT&T syntax ) to port 0x5658 specially.  Note:
> since many operations return data in EAX, "OUT DX, EAX" does not
> work for them on VMware, I did not support the "OUT DX, EAX", but it
> would not be hard to add.
> 
> Also this instruction is allowed to be used from ring 3.  To
> support this the vmexit for GP needs to be enabled.  I have not
> fully tested that nested HVM is doing the right thing for this.
> 
> An open source example of using this is:
> 
> http://open-vm-tools.sourceforge.net/
> 
> Which only uses "IN EAX, DX".  Also
> 
> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=disp
> layKC&externalId=1009458
> 
> Lists just "inl (%%dx)" (I assume this is AT&T syntax and is the
> same as "inl %dx, %eax").
> 
> The support included is enough to allow VMware tools to install in a
> HVM domU and provide guestinfo support.  guestinfo support is
> provide by what is known as VMware RPC support.  This guestinfo
> support is provided via libxc.  libxl support has not be written.
> 
> Note: VMware RPC support is only available on HVM domU.
> 
> This interface is an extension of __HYPERVISOR_HVM_op.  It was
> picked because xc_get_hvm_param() also uses it and VMware guest
> info is a lot like a hvm param.
> 
> The HVMOP_get_vmport_guest_info is used by two libxc functions,
> xc_get_vmport_guest_info and xc_fetch_vmport_guest_info.
> xc_fetch_vmport_guest_info is designed to be used to fetch all
> currently set guestinfo values.
> 
> To save on hypervisor heap memory, the guestinfo support in done in
> two sizes, normal and jumbo.  Normal is used to handle up to 128
> byte values and jumbo is used to handle up to 4096 byte values.
> 
> Since all this is work is done when the guest is doing a single
> instruction; it was designed to not use the hypervisor heap to
> allocate the memory at this time.  Instead a few are allocated at
> the create domain time and during the xen's hyper-call to get or set
> them.  This was picked in that if a tool stack is using the VMware
> guest info support, it should be using either of both of the get and
> set.  And so in this case the guest should only see an out of memory
> error when the compile max amount of hypervisor heap memory is in
> use.
> 
> Doing it this way does lead to a lot of pointer use and many
> sub structures.
> 
> If the domU is running VMware tools, then the "build version" of
> the tools is also available via xc_get_HVM_param().  This also
> enables the use of new triggers that will use the VMware hyper-call
> to do some limited control of the domU.  The most useful are
> poweroff and reboot.  Since a guest process needs to be running
> for these to work, a tool stack should check that the build version
> is non zero before assuming these will work.
> 
> The 2 hvm param's HVM_PARAM_VMPORT_BUILD_NUMBER_TIME and
> HVM_PARAM_VMPORT_BUILD_NUMBER_VALUE are how "build version" is
> accessed.  These 2 params are only allowed to be set to zero.  The
> HVM_PARAM_VMPORT_BUILD_NUMBER_TIME can be used to track the last
> time the VMware tools in the guest responded.  One such use would
> be the health of the tools in the guest.  The hvm param
> HVM_PARAM_VMPORT_RESET_TIME controls how often to request them in
> seconds minus 1.  The minus 1 is to handle to 0 case.  I.E. the
> fastest that can be selected is every second.  The default is 4
> times a minute.
> 
> The VMware RPC support includes the notion of channels that are
> opened, active and closed.  All RPC messages sent via a channel
> starts with normal ASCII text.  The message some times does include
> binary data.
> 
> Currently there are 2 protocols defined for VMware RPC.  They
> determine the direction for data flow, domU to tool stack or
> tool stack to domU.
> 
> There is no provided interrupt for VMware RPC.
> 
> The VMware's hyper-call state is included in live migration and
> save/restore.  Because the max size of the VMware guestinfo is
> large, then data is compressed and expanded in the
> vmport_save_domain_ctxt and vmport_load_domain_ctxt.
> 
> For a debug=y build there is a new command line option
> vmport_debug=.  It enabled output to the console of various
> stages of handling the "IN EAX, DX" instruction.  Most uses
> are the summary ones that show complete RPC actions.
> 
> Signed-off-by: Don Slutz <dslutz@verizon.com>
> ---
>  tools/libxc/xc_domain.c                      |  115 +++
>  tools/libxc/xenctrl.h                        |   24 +
>  tools/misc/xen-hvmctx.c                      |  229 ++++
>  tools/xentrace/formats                       |    8 +
>  xen/arch/x86/domctl.c                        |   34 +
>  xen/arch/x86/hvm/Makefile                    |    1 +
>  xen/arch/x86/hvm/hvm.c                       |  480 ++++++++-
>  xen/arch/x86/hvm/io.c                        |    1 +
>  xen/arch/x86/hvm/svm/emulate.c               |    3 +
>  xen/arch/x86/hvm/svm/svm.c                   |   79 ++
>  xen/arch/x86/hvm/svm/vmcb.c                  |    1 +
>  xen/arch/x86/hvm/vmport/Makefile             |    1 +
>  xen/arch/x86/hvm/vmport/includeCheck.h       |   17 +
>  xen/arch/x86/hvm/vmport/vmport.c             | 1436 ++++++++++++++++++++++++++
>  xen/arch/x86/hvm/vmx/vmcs.c                  |    1 +
>  xen/arch/x86/hvm/vmx/vmx.c                   |   90 ++
>  xen/arch/x86/hvm/vmx/vvmx.c                  |   14 +
>  xen/include/asm-x86/hvm/domain.h             |    4 +
>  xen/include/asm-x86/hvm/svm/emulate.h        |    1 +
>  xen/include/asm-x86/hvm/trace.h              |   31 +
>  xen/include/asm-x86/hvm/vmport.h             |   90 ++
>  xen/include/public/arch-x86/hvm/save.h       |   39 +-
>  xen/include/public/arch-x86/hvm/vmporttype.h |  105 ++
>  xen/include/public/domctl.h                  |    3 +
>  xen/include/public/hvm/hvm_op.h              |   18 +
>  xen/include/public/hvm/params.h              |    5 +-
>  xen/include/public/trace.h                   |    7 +
>  27 files changed, 2834 insertions(+), 3 deletions(-)
>  create mode 100644 xen/arch/x86/hvm/vmport/Makefile
>  create mode 100644 xen/arch/x86/hvm/vmport/includeCheck.h
>  create mode 100644 xen/arch/x86/hvm/vmport/vmport.c
>  create mode 100644 xen/include/asm-x86/hvm/vmport.h
>  create mode 100644 xen/include/public/arch-x86/hvm/vmporttype.h

As I think was said on v1 already - this should be split into smaller
pieces if at all possible. I'm therefore not going to do a full review
at this time, just give a couple of remarks.

> @@ -579,6 +580,39 @@ long arch_do_domctl(
>          }
>          break;
>  
> +        case XEN_DOMCTL_SENDTRIGGER_VTPOWER:
> +        {
> +            ret = -EINVAL;
> +            if ( is_hvm_domain(d) )
> +            {
> +                ret = 0;
> +                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Halt");
> +            }
> +        }
> +        break;
> +
> +        case XEN_DOMCTL_SENDTRIGGER_VTREBOOT:
> +        {
> +            ret = -EINVAL;
> +            if ( is_hvm_domain(d) )
> +            {
> +                ret = 0;
> +                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Reboot");
> +            }
> +        }
> +        break;
> +
> +        case XEN_DOMCTL_SENDTRIGGER_VTPING:
> +        {
> +            ret = -EINVAL;
> +            if ( is_hvm_domain(d) )
> +            {
> +                ret = 0;
> +                vmport_ctrl_send(&d->arch.hvm_domain, "ping");
> +            }
> +        }
> +        break;

Rather than adding three new domctls, wouldn't a single VMware
one with suitable sub-operations do?

> @@ -1480,6 +1481,7 @@ int hvm_domain_initialise(struct domain *d)
>      INIT_LIST_HEAD(&d->arch.hvm_domain.ioreq_server.list);
>      spin_lock_init(&d->arch.hvm_domain.irq_lock);
>      spin_lock_init(&d->arch.hvm_domain.uc_lock);
> +    spin_lock_init(&d->arch.hvm_domain.vmport_lock);
>  
>      INIT_LIST_HEAD(&d->arch.hvm_domain.msixtbl_list);
>      spin_lock_init(&d->arch.hvm_domain.msixtbl_list_lock);
> @@ -1492,11 +1494,36 @@ int hvm_domain_initialise(struct domain *d)
>  
>      d->arch.hvm_domain.params = xzalloc_array(uint64_t, HVM_NR_PARAMS);
>      d->arch.hvm_domain.io_handler = xmalloc(struct hvm_io_handler);
> +    d->arch.hvm_domain.vmport_data = xzalloc(struct vmport_state);
>      rc = -ENOMEM;
> -    if ( !d->arch.hvm_domain.params || !d->arch.hvm_domain.io_handler )
> +    if ( !d->arch.hvm_domain.params || !d->arch.hvm_domain.io_handler ||
> +         !d->arch.hvm_domain.vmport_data )
>          goto fail1;
>      d->arch.hvm_domain.io_handler->num_slot = 0;
>  
> +    /*
> +     * Any value is fine here. In fact a random number may better.
> +     * It is used to help validate that a both sides are talking
> +     * about the same channel.
> +     */
> +    d->arch.hvm_domain.vmport_data->open_cookie = 435;
> +
> +    d->arch.hvm_domain.vmport_data->used_guestinfo = 10;
> +    for (rc = 0;
> +         rc < d->arch.hvm_domain.vmport_data->used_guestinfo;
> +         rc++)
> +        d->arch.hvm_domain.vmport_data->guestinfo[rc] =
> +            xzalloc(vmport_guestinfo_t);
> +
> +    d->arch.hvm_domain.vmport_data->used_guestinfo_jumbo = 2;
> +    for (rc = 0;
> +         rc < d->arch.hvm_domain.vmport_data->used_guestinfo_jumbo;
> +         rc++)
> +        d->arch.hvm_domain.vmport_data->guestinfo_jumbo[rc] =
> +            xzalloc(vmport_guestinfo_jumbo_t);
> +
> +    vmport_flush(&d->arch.hvm_domain);
> +

All this would very likely better go into a separate function placed in
vmport.c. Same for many of the helper functions further down.
Please realize that this file is already huge, so any new addition
having a reasonable other home would be nice to be kept out of
this file.

> @@ -1520,6 +1548,7 @@ int hvm_domain_initialise(struct domain *d)
>  
>      register_portio_handler(d, 0xe9, 1, hvm_print_line);
>      register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
> +    register_portio_handler(d, VMPORT_PORT, 4, vmport_ioport);

So from a cursory look I wasn't able to spot any code making sure
that port doesn't get used for e.g. a passed through device.

And in any event I'm rather uncomfortable about this getting
enabled unconditionally, also due to (but not limited to this) the
up front allocation of various memory blocks that may end up
never being needed. The main issue I see with this approach is
that guests could end up being misguided into thinking they're
running on VMware when in fact they have code in place to
optimize when running on Xen. We had such detection ordering
issues with Linux checking for both Hyper-V and Xen.

> @@ -1532,6 +1561,17 @@ int hvm_domain_initialise(struct domain *d)
>      stdvga_deinit(d);
>      vioapic_deinit(d);
>   fail1:
> +    if ( d->arch.hvm_domain.vmport_data )
> +    {
> +        struct vmport_state *vs = d->arch.hvm_domain.vmport_data;
> +        int idx;
> +
> +        for (idx = 0; idx < vs->used_guestinfo; idx++)

You'll have to go through and fix coding style issues.

> --- a/xen/arch/x86/hvm/io.c
> +++ b/xen/arch/x86/hvm/io.c
> @@ -42,6 +42,7 @@
>  #include <asm/hvm/vlapic.h>
>  #include <asm/hvm/trace.h>
>  #include <asm/hvm/emulate.h>
> +#include <asm/hvm/vmport.h>

This again being the only change to this file - why?

> @@ -106,6 +107,7 @@ MAKE_INSTR(VMSAVE, 3, 0x0f, 0x01, 0xdb);
>  MAKE_INSTR(STGI,   3, 0x0f, 0x01, 0xdc);
>  MAKE_INSTR(CLGI,   3, 0x0f, 0x01, 0xdd);
>  MAKE_INSTR(INVLPGA,3, 0x0f, 0x01, 0xdf);
> +MAKE_INSTR(IN,     1, 0xed);

This name is ambiguous.

> +static void svm_vmexit_gp_intercept(struct cpu_user_regs *regs, struct vcpu *v)
> +{
> +    struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
> +    unsigned long inst_len = __get_instruction_length(v, INSTR_IN);
> +
> +    regs->error_code = vmcb->exitinfo1;
> +    if ( hvm_long_mode_enabled(v) )
> +        HVMTRACE_LONG_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
> +                          regs->error_code,
> +                          TRC_PAR_LONG(vmcb->exitinfo2));
> +    else
> +        HVMTRACE_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
> +                     regs->error_code, vmcb->exitinfo2);
> +
> +    if ( inst_len <= 1 && (regs->rdx & 0xffff) == VMPORT_PORT &&

What would inst_len being zero mean here?

> +         vmcb->exitinfo2 == 0 && regs->error_code == 0 )
> +    {
> +        uint32_t magic = regs->rax;
> +
> +        if ( magic == VMPORT_MAGIC )

This ought to be folded with the previous if(), at once reducing
code redundancy further down in the function.

> +        {
> +            unsigned char bytes[1] = { 0 };

Pointless initializer (and it being an array looks suspicious too).

> @@ -2565,6 +2567,82 @@ static void vmx_idtv_reinject(unsigned long idtv_info)
>      }
>  }
>  
> +void do_gp_fault(struct cpu_user_regs *regs, struct vcpu *v)

A VMX-specific function shouldn't be named this way. Plus I can't see
why it's not static (in which case the name may actually be okay). And
then a lot of the code further down looks rather similar to SVM's -
please avoid such duplication.

> @@ -2675,6 +2753,15 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
>                   && vector != TRAP_nmi 
>                   && vector != TRAP_machine_check ) 
>              {
> +                if ( vector == TRAP_gp_fault )
> +                {
> +                    VMPORT_DBG_LOG(VMPORT_LOG_REALMODE_GP,
> +                                   "realmode gp: ip=%"PRIx64" ax=%"PRIx64
> +                                   " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
> +                                   " si=%"PRIx64" di=%"PRIx64,
> +                                   regs->rip, regs->rax, regs->rbx, regs->rcx,
> +                                   regs->rdx, regs->rsi, regs->rdi);
> +                }

Is this really useful (no matter that it'd guarded by a !NDEBUG-only
command line option? Speaking of which - why can't you just define
a new DBG_LEVEL_VMPORT and use the existing HVM_DBG_LOG()?

> @@ -2182,6 +2183,19 @@ int nvmx_n2_vmexit_handler(struct cpu_user_regs *regs,
>              if ( v->fpu_dirtied )
>                  nvcpu->nv_vmexit_pending = 1;
>          }
> +        else if ( vector == TRAP_gp_fault )
> +        {
> +#ifndef NDEBUG
> +            struct cpu_user_regs *ur = guest_cpu_user_regs();
> +#endif
> +            VMPORT_DBG_LOG(VMPORT_LOG_VGP_UNKNOWN,
> +                           "Unexpected gp: ip=%"PRIx64" ax=%"PRIx64
> +                           " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
> +                           " si=%"PRIx64" di=%"PRIx64,
> +                           ur->rip, ur->rax, ur->rbx, ur->rcx, ur->rdx,
> +                           ur->rsi, ur->rdi);

The #endif above ought to go here for the code to look consistent.

> +#ifndef NDEBUG
> +#define VMPORT_LOG_RPC             (1 << 0)
> +#define VMPORT_LOG_RECV_STATUS     (1 << 1)
> +#define VMPORT_LOG_SKIP_SEND       (1 << 2)
> +#define VMPORT_LOG_SEND            (1 << 3)
> +#define VMPORT_LOG_SEND_SIZE_ALL   (1 << 4)
> +#define VMPORT_LOG_SEND_SIZE       (1 << 5)
> +#define VMPORT_LOG_RECV_SIZE_ALL   (1 << 6)
> +#define VMPORT_LOG_RECV_SIZE       (1 << 7)
> +#define VMPORT_LOG_CLOSE           (1 << 8)
> +#define VMPORT_LOG_OPEN            (1 << 9)
> +#define VMPORT_LOG_FLUSH           (1 << 10)
> +#define VMPORT_LOG_TRACE           (1 << 11)
> +#define VMPORT_LOG_PING            (1 << 12)
> +#define VMPORT_LOG_SWEEP           (1 << 13)
> +#define VMPORT_LOG_BUILD           (1 << 14)
> +
> +#define VMPORT_LOG_ERROR           (1 << 16)
> +
> +#define VMPORT_LOG_INFO_GET        (1 << 17)
> +#define VMPORT_LOG_INFO_SET        (1 << 18)
> +
> +#define VMPORT_LOG_SKIP_MESSAGE    (1 << 19)
> +
> +#define VMPORT_LOG_GP_UNKNOWN      (1 << 20)
> +#define VMPORT_LOG_GP_NOT_VMWARE   (1 << 21)
> +#define VMPORT_LOG_GP_FAIL_RD_INST (1 << 22)
> +
> +#define VMPORT_LOG_GP_VMWARE_AFTER (1 << 23)
> +
> +#define VMPORT_LOG_VGP_UNKNOWN     (1 << 24)
> +#define VMPORT_LOG_REALMODE_GP     (1 << 27)
> +
> +#define VMPORT_LOG_VMWARE_AFTER    (1 << 28)

Ah, here we go (as to using HVM_DBG_LOG()): Isn't this _way_ too
fine grained?

Jan

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Don Slutz]

On 09/02/14 04:16, Jan Beulich wrote:
>>>> On 01.09.14 at 17:33,<dslutz@verizon.com>  wrote:
>> This is both a more complete support then in currently provided by
>> QEMU and/or KVM and less.  The missing part requires QEMU changes
>> and has been left out until the QEMU patches are accepted upstream.
>>
>> VMware's hyper-call is also known as VMware Backdoor I/O Port.
>>
>> Note: this support does not depend on vmware_hw being non-zero.
>>
>> Summary is that VMware treats "IN EAX, DX" (or "OUT DX, EAX"; or
>> "inl %dx, %eax" in AT&T syntax ) to port 0x5658 specially.  Note:
>> since many operations return data in EAX, "OUT DX, EAX" does not
>> work for them on VMware, I did not support the "OUT DX, EAX", but it
>> would not be hard to add.
>>
>> Also this instruction is allowed to be used from ring 3.  To
>> support this the vmexit for GP needs to be enabled.  I have not
>> fully tested that nested HVM is doing the right thing for this.
>>
>> An open source example of using this is:
>>
>> http://open-vm-tools.sourceforge.net/
>>
>> Which only uses "IN EAX, DX".  Also
>>
>> http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=disp
>> layKC&externalId=1009458
>>
>> Lists just "inl (%%dx)" (I assume this is AT&T syntax and is the
>> same as "inl %dx, %eax").
>>
>> The support included is enough to allow VMware tools to install in a
>> HVM domU and provide guestinfo support.  guestinfo support is
>> provide by what is known as VMware RPC support.  This guestinfo
>> support is provided via libxc.  libxl support has not be written.
>>
>> Note: VMware RPC support is only available on HVM domU.
>>
>> This interface is an extension of __HYPERVISOR_HVM_op.  It was
>> picked because xc_get_hvm_param() also uses it and VMware guest
>> info is a lot like a hvm param.
>>
>> The HVMOP_get_vmport_guest_info is used by two libxc functions,
>> xc_get_vmport_guest_info and xc_fetch_vmport_guest_info.
>> xc_fetch_vmport_guest_info is designed to be used to fetch all
>> currently set guestinfo values.
>>
>> To save on hypervisor heap memory, the guestinfo support in done in
>> two sizes, normal and jumbo.  Normal is used to handle up to 128
>> byte values and jumbo is used to handle up to 4096 byte values.
>>
>> Since all this is work is done when the guest is doing a single
>> instruction; it was designed to not use the hypervisor heap to
>> allocate the memory at this time.  Instead a few are allocated at
>> the create domain time and during the xen's hyper-call to get or set
>> them.  This was picked in that if a tool stack is using the VMware
>> guest info support, it should be using either of both of the get and
>> set.  And so in this case the guest should only see an out of memory
>> error when the compile max amount of hypervisor heap memory is in
>> use.
>>
>> Doing it this way does lead to a lot of pointer use and many
>> sub structures.
>>
>> If the domU is running VMware tools, then the "build version" of
>> the tools is also available via xc_get_HVM_param().  This also
>> enables the use of new triggers that will use the VMware hyper-call
>> to do some limited control of the domU.  The most useful are
>> poweroff and reboot.  Since a guest process needs to be running
>> for these to work, a tool stack should check that the build version
>> is non zero before assuming these will work.
>>
>> The 2 hvm param's HVM_PARAM_VMPORT_BUILD_NUMBER_TIME and
>> HVM_PARAM_VMPORT_BUILD_NUMBER_VALUE are how "build version" is
>> accessed.  These 2 params are only allowed to be set to zero.  The
>> HVM_PARAM_VMPORT_BUILD_NUMBER_TIME can be used to track the last
>> time the VMware tools in the guest responded.  One such use would
>> be the health of the tools in the guest.  The hvm param
>> HVM_PARAM_VMPORT_RESET_TIME controls how often to request them in
>> seconds minus 1.  The minus 1 is to handle to 0 case.  I.E. the
>> fastest that can be selected is every second.  The default is 4
>> times a minute.
>>
>> The VMware RPC support includes the notion of channels that are
>> opened, active and closed.  All RPC messages sent via a channel
>> starts with normal ASCII text.  The message some times does include
>> binary data.
>>
>> Currently there are 2 protocols defined for VMware RPC.  They
>> determine the direction for data flow, domU to tool stack or
>> tool stack to domU.
>>
>> There is no provided interrupt for VMware RPC.
>>
>> The VMware's hyper-call state is included in live migration and
>> save/restore.  Because the max size of the VMware guestinfo is
>> large, then data is compressed and expanded in the
>> vmport_save_domain_ctxt and vmport_load_domain_ctxt.
>>
>> For a debug=y build there is a new command line option
>> vmport_debug=.  It enabled output to the console of various
>> stages of handling the "IN EAX, DX" instruction.  Most uses
>> are the summary ones that show complete RPC actions.
>>
>> Signed-off-by: Don Slutz<dslutz@verizon.com>
>> ---
>>   tools/libxc/xc_domain.c                      |  115 +++
>>   tools/libxc/xenctrl.h                        |   24 +
>>   tools/misc/xen-hvmctx.c                      |  229 ++++
>>   tools/xentrace/formats                       |    8 +
>>   xen/arch/x86/domctl.c                        |   34 +
>>   xen/arch/x86/hvm/Makefile                    |    1 +
>>   xen/arch/x86/hvm/hvm.c                       |  480 ++++++++-
>>   xen/arch/x86/hvm/io.c                        |    1 +
>>   xen/arch/x86/hvm/svm/emulate.c               |    3 +
>>   xen/arch/x86/hvm/svm/svm.c                   |   79 ++
>>   xen/arch/x86/hvm/svm/vmcb.c                  |    1 +
>>   xen/arch/x86/hvm/vmport/Makefile             |    1 +
>>   xen/arch/x86/hvm/vmport/includeCheck.h       |   17 +
>>   xen/arch/x86/hvm/vmport/vmport.c             | 1436 ++++++++++++++++++++++++++
>>   xen/arch/x86/hvm/vmx/vmcs.c                  |    1 +
>>   xen/arch/x86/hvm/vmx/vmx.c                   |   90 ++
>>   xen/arch/x86/hvm/vmx/vvmx.c                  |   14 +
>>   xen/include/asm-x86/hvm/domain.h             |    4 +
>>   xen/include/asm-x86/hvm/svm/emulate.h        |    1 +
>>   xen/include/asm-x86/hvm/trace.h              |   31 +
>>   xen/include/asm-x86/hvm/vmport.h             |   90 ++
>>   xen/include/public/arch-x86/hvm/save.h       |   39 +-
>>   xen/include/public/arch-x86/hvm/vmporttype.h |  105 ++
>>   xen/include/public/domctl.h                  |    3 +
>>   xen/include/public/hvm/hvm_op.h              |   18 +
>>   xen/include/public/hvm/params.h              |    5 +-
>>   xen/include/public/trace.h                   |    7 +
>>   27 files changed, 2834 insertions(+), 3 deletions(-)
>>   create mode 100644 xen/arch/x86/hvm/vmport/Makefile
>>   create mode 100644 xen/arch/x86/hvm/vmport/includeCheck.h
>>   create mode 100644 xen/arch/x86/hvm/vmport/vmport.c
>>   create mode 100644 xen/include/asm-x86/hvm/vmport.h
>>   create mode 100644 xen/include/public/arch-x86/hvm/vmporttype.h
> As I think was said on v1 already - this should be split into smaller
> pieces if at all possible. I'm therefore not going to do a full review
> at this time, just give a couple of remarks.

The last time (v1's) split was much worse (and so I think that "split 
into smaller"
was not said). I could check be most were "combine this patch with that 
patch";
and more comment message.

Having been over this code many times in the last month, a possible
quick split (which would not delay v3 too long) would be:

1) Add simple vmport commands like BDOOR_CMD_GETVERSION
2) Add the hypervisor side of VMware guest info.
3) Add the hvm params that come from VMware guest info.
4) Add the hyper calls for libxc to handle VMware guest info.

I will attempt to split the commit message to follow this also. I expect 
that
there will be duplication of text in the commit messages.


And should I add the unit tests also (as optional)?


Anything you can provide helps.

>> @@ -579,6 +580,39 @@ long arch_do_domctl(
>>           }
>>           break;
>>   
>> +        case XEN_DOMCTL_SENDTRIGGER_VTPOWER:
>> +        {
>> +            ret = -EINVAL;
>> +            if ( is_hvm_domain(d) )
>> +            {
>> +                ret = 0;
>> +                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Halt");
>> +            }
>> +        }
>> +        break;
>> +
>> +        case XEN_DOMCTL_SENDTRIGGER_VTREBOOT:
>> +        {
>> +            ret = -EINVAL;
>> +            if ( is_hvm_domain(d) )
>> +            {
>> +                ret = 0;
>> +                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Reboot");
>> +            }
>> +        }
>> +        break;
>> +
>> +        case XEN_DOMCTL_SENDTRIGGER_VTPING:
>> +        {
>> +            ret = -EINVAL;
>> +            if ( is_hvm_domain(d) )
>> +            {
>> +                ret = 0;
>> +                vmport_ctrl_send(&d->arch.hvm_domain, "ping");
>> +            }
>> +        }
>> +        break;
> Rather than adding three new domctls, wouldn't a single VMware
> one with suitable sub-operations do?

As far as I can tell these are already a "sub-operation":


case XEN_DOMCTL_sendtrigger:
{
struct vcpu *v;

ret = -EINVAL;
if ( domctl->u.sendtrigger.vcpu >= MAX_VIRT_CPUS )
break;

ret = -ESRCH;
if ( domctl->u.sendtrigger.vcpu >= d->max_vcpus ||
(v = d->vcpu[domctl->u.sendtrigger.vcpu]) == NULL )
break;

switch ( domctl->u.sendtrigger.trigger )
{
case XEN_DOMCTL_SENDTRIGGER_NMI:
...


So not sure I can do any thing here.



>> @@ -1480,6 +1481,7 @@ int hvm_domain_initialise(struct domain *d)
>>       INIT_LIST_HEAD(&d->arch.hvm_domain.ioreq_server.list);
>>       spin_lock_init(&d->arch.hvm_domain.irq_lock);
>>       spin_lock_init(&d->arch.hvm_domain.uc_lock);
>> +    spin_lock_init(&d->arch.hvm_domain.vmport_lock);
>>   
>>       INIT_LIST_HEAD(&d->arch.hvm_domain.msixtbl_list);
>>       spin_lock_init(&d->arch.hvm_domain.msixtbl_list_lock);
>> @@ -1492,11 +1494,36 @@ int hvm_domain_initialise(struct domain *d)
>>   
>>       d->arch.hvm_domain.params = xzalloc_array(uint64_t, HVM_NR_PARAMS);
>>       d->arch.hvm_domain.io_handler = xmalloc(struct hvm_io_handler);
>> +    d->arch.hvm_domain.vmport_data = xzalloc(struct vmport_state);
>>       rc = -ENOMEM;
>> -    if ( !d->arch.hvm_domain.params || !d->arch.hvm_domain.io_handler )
>> +    if ( !d->arch.hvm_domain.params || !d->arch.hvm_domain.io_handler ||
>> +         !d->arch.hvm_domain.vmport_data )
>>           goto fail1;
>>       d->arch.hvm_domain.io_handler->num_slot = 0;
>>   
>> +    /*
>> +     * Any value is fine here. In fact a random number may better.
>> +     * It is used to help validate that a both sides are talking
>> +     * about the same channel.
>> +     */
>> +    d->arch.hvm_domain.vmport_data->open_cookie = 435;
>> +
>> +    d->arch.hvm_domain.vmport_data->used_guestinfo = 10;
>> +    for (rc = 0;
>> +         rc < d->arch.hvm_domain.vmport_data->used_guestinfo;
>> +         rc++)
>> +        d->arch.hvm_domain.vmport_data->guestinfo[rc] =
>> +            xzalloc(vmport_guestinfo_t);
>> +
>> +    d->arch.hvm_domain.vmport_data->used_guestinfo_jumbo = 2;
>> +    for (rc = 0;
>> +         rc < d->arch.hvm_domain.vmport_data->used_guestinfo_jumbo;
>> +         rc++)
>> +        d->arch.hvm_domain.vmport_data->guestinfo_jumbo[rc] =
>> +            xzalloc(vmport_guestinfo_jumbo_t);
>> +
>> +    vmport_flush(&d->arch.hvm_domain);
>> +
> All this would very likely better go into a separate function placed in
> vmport.c. Same for many of the helper functions further down.
> Please realize that this file is already huge, so any new addition
> having a reasonable other home would be nice to be kept out of
> this file.

ok, Will move a all of this into vmport.c and add more functions to
handle the code I added here.


>> @@ -1520,6 +1548,7 @@ int hvm_domain_initialise(struct domain *d)
>>   
>>       register_portio_handler(d, 0xe9, 1, hvm_print_line);
>>       register_portio_handler(d, 0xcf8, 4, hvm_access_cf8);
>> +    register_portio_handler(d, VMPORT_PORT, 4, vmport_ioport);
> So from a cursory look I wasn't able to spot any code making sure
> that port doesn't get used for e.g. a passed through device.

There is no such code.

> And in any event I'm rather uncomfortable about this getting
> enabled unconditionally, also due to (but not limited to this) the
> up front allocation of various memory blocks that may end up
> never being needed. The main issue I see with this approach is
> that guests could end up being misguided into thinking they're
> running on VMware when in fact they have code in place to
> optimize when running on Xen. We had such detection ordering
> issues with Linux checking for both Hyper-V and Xen.


Ok, I do feel strongly that this would need to be independent on
vmware_hw "version". And so I will add a new xl.cfg item to control this.

QEMU on 5/19/2014 added a way to turn it's version off:

+@item vmport=on|off
+Enables emulation of VMWare IO port, for vmmouse etc. (enabled by default)

So should I name it vmware_vmport or just vmport ( I lean to just vmport )?
and unlike QEMU I am fine with the default of off.

>> @@ -1532,6 +1561,17 @@ int hvm_domain_initialise(struct domain *d)
>>       stdvga_deinit(d);
>>       vioapic_deinit(d);
>>    fail1:
>> +    if ( d->arch.hvm_domain.vmport_data )
>> +    {
>> +        struct vmport_state *vs = d->arch.hvm_domain.vmport_data;
>> +        int idx;
>> +
>> +        for (idx = 0; idx < vs->used_guestinfo; idx++)
> You'll have to go through and fix coding style issues.

Do to the many coding styles I have used in the last 20 years, I can have
style "blindness". I do not find a style issue here based on CODING_STYLE.
All hints are welcome.

If I look in tools/libxl/CODING_STYLE, my best guess is s/idx/i/.


>> --- a/xen/arch/x86/hvm/io.c
>> +++ b/xen/arch/x86/hvm/io.c
>> @@ -42,6 +42,7 @@
>>   #include <asm/hvm/vlapic.h>
>>   #include <asm/hvm/trace.h>
>>   #include <asm/hvm/emulate.h>
>> +#include <asm/hvm/vmport.h>
> This again being the only change to this file - why?
>

Looks like a left over of when I was not using register_portio_handler()

Will fix.


>> @@ -106,6 +107,7 @@ MAKE_INSTR(VMSAVE, 3, 0x0f, 0x01, 0xdb);
>>   MAKE_INSTR(STGI,   3, 0x0f, 0x01, 0xdc);
>>   MAKE_INSTR(CLGI,   3, 0x0f, 0x01, 0xdd);
>>   MAKE_INSTR(INVLPGA,3, 0x0f, 0x01, 0xdf);
>> +MAKE_INSTR(IN,     1, 0xed);
> This name is ambiguous.

There are 4 opcodes and 6 instructions. Not at all sure how to name
the one I need. Here is the info about the IN instruction.


IN—Input from Port
Opcode Instruction Op/ 64-Bit Compat/ Description
En Mode Leg Mode
E4 ib IN AL, imm8 A Valid Valid Input byte from imm8 I/O
port address into AL.
E5 ib IN AX, imm8 A Valid Valid Input word from imm8 I/O
port address into AX.
E5 ib IN EAX, imm8 A Valid Valid Input dword from imm8 I/O
port address into EAX.
EC IN AL,DX B Valid Valid Input byte from I/O port in
DX into AL.
ED IN AX,DX B Valid Valid Input word from I/O port in
DX into AX.
ED IN EAX,DX B Valid Valid Input doubleword from I/O
port in DX into EAX.


So would "inl" be any better? or is inw_dx or inl_dx better?


>> +static void svm_vmexit_gp_intercept(struct cpu_user_regs *regs, struct vcpu *v)
>> +{
>> +    struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
>> +    unsigned long inst_len = __get_instruction_length(v, INSTR_IN);
>> +
>> +    regs->error_code = vmcb->exitinfo1;
>> +    if ( hvm_long_mode_enabled(v) )
>> +        HVMTRACE_LONG_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
>> +                          regs->error_code,
>> +                          TRC_PAR_LONG(vmcb->exitinfo2));
>> +    else
>> +        HVMTRACE_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
>> +                     regs->error_code, vmcb->exitinfo2);
>> +
>> +    if ( inst_len <= 1 && (regs->rdx & 0xffff) == VMPORT_PORT &&
> What would inst_len being zero mean here?

It use to mean that AMD was missing the feature:


(XEN) [2014-08-14 20:17:28] - Next-RIP Saved on #VMEXIT

and so I needed to look at the opcode.

Now that I am using __get_instruction_length() I think it could be changed
to a 1. However I can only test on an AMD server that does have this
feature. Will switch to == 1 and hope for the best.

As far as I can tell __get_instruction_length() does not insure that the 
opcode
list passed is found, just that it might be.


>> +         vmcb->exitinfo2 == 0 && regs->error_code == 0 )
>> +    {
>> +        uint32_t magic = regs->rax;
>> +
>> +        if ( magic == VMPORT_MAGIC )
> This ought to be folded with the previous if(), at once reducing
> code redundancy further down in the function.

OK. But that does make the debug log message more complex to code.

>> +        {
>> +            unsigned char bytes[1] = { 0 };
> Pointless initializer (and it being an array looks suspicious too).

Will drop the initializer. hvm_fetch_from_guest_virt_nofault() does take 
an array.
So it is better to say "&tmp, regs->rip, 1" then "bytes, regs->rip, 1" ?

Also by dropping the initializer, you are stating that
hvm_fetch_from_guest_virt_nofault() does initialize it's output on error (
because the debug log output does include bytes[0]).


>> @@ -2565,6 +2567,82 @@ static void vmx_idtv_reinject(unsigned long idtv_info)
>>       }
>>   }
>>   
>> +void do_gp_fault(struct cpu_user_regs *regs, struct vcpu *v)
> A VMX-specific function shouldn't be named this way. Plus I can't see
> why it's not static (in which case the name may actually be okay). And
> then a lot of the code further down looks rather similar to SVM's -
> please avoid such duplication.

I will look into having a common routine.


>> @@ -2675,6 +2753,15 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
>>                    && vector != TRAP_nmi
>>                    && vector != TRAP_machine_check )
>>               {
>> +                if ( vector == TRAP_gp_fault )
>> +                {
>> +                    VMPORT_DBG_LOG(VMPORT_LOG_REALMODE_GP,
>> +                                   "realmode gp: ip=%"PRIx64" ax=%"PRIx64
>> +                                   " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
>> +                                   " si=%"PRIx64" di=%"PRIx64,
>> +                                   regs->rip, regs->rax, regs->rbx, regs->rcx,
>> +                                   regs->rdx, regs->rsi, regs->rdi);
>> +                }
> Is this really useful (no matter that it'd guarded by a !NDEBUG-only
> command line option?

I had found it so at the start. Not so sure it is now.

>   Speaking of which - why can't you just define
> a new DBG_LEVEL_VMPORT and use the existing HVM_DBG_LOG()?

I did define 23 bits to control various part of the debug. A lot of this
is because there are many times you go through the code. Basicly
a string is 1st passed a length, and the 4 bytes at a time until the
length is handled, followed by state changes...



>> @@ -2182,6 +2183,19 @@ int nvmx_n2_vmexit_handler(struct cpu_user_regs *regs,
>>               if ( v->fpu_dirtied )
>>                   nvcpu->nv_vmexit_pending = 1;
>>           }
>> +        else if ( vector == TRAP_gp_fault )
>> +        {
>> +#ifndef NDEBUG
>> +            struct cpu_user_regs *ur = guest_cpu_user_regs();
>> +#endif
>> +            VMPORT_DBG_LOG(VMPORT_LOG_VGP_UNKNOWN,
>> +                           "Unexpected gp: ip=%"PRIx64" ax=%"PRIx64
>> +                           " bx=%"PRIx64" cx=%"PRIx64" dx=%"PRIx64
>> +                           " si=%"PRIx64" di=%"PRIx64,
>> +                           ur->rip, ur->rax, ur->rbx, ur->rcx, ur->rdx,
>> +                           ur->rsi, ur->rdi);
> The #endif above ought to go here for the code to look consistent.

Ok.

>> +#ifndef NDEBUG
>> +#define VMPORT_LOG_RPC             (1 << 0)
>> +#define VMPORT_LOG_RECV_STATUS     (1 << 1)
>> +#define VMPORT_LOG_SKIP_SEND       (1 << 2)
>> +#define VMPORT_LOG_SEND            (1 << 3)
>> +#define VMPORT_LOG_SEND_SIZE_ALL   (1 << 4)
>> +#define VMPORT_LOG_SEND_SIZE       (1 << 5)
>> +#define VMPORT_LOG_RECV_SIZE_ALL   (1 << 6)
>> +#define VMPORT_LOG_RECV_SIZE       (1 << 7)
>> +#define VMPORT_LOG_CLOSE           (1 << 8)
>> +#define VMPORT_LOG_OPEN            (1 << 9)
>> +#define VMPORT_LOG_FLUSH           (1 << 10)
>> +#define VMPORT_LOG_TRACE           (1 << 11)
>> +#define VMPORT_LOG_PING            (1 << 12)
>> +#define VMPORT_LOG_SWEEP           (1 << 13)
>> +#define VMPORT_LOG_BUILD           (1 << 14)
>> +
>> +#define VMPORT_LOG_ERROR           (1 << 16)
>> +
>> +#define VMPORT_LOG_INFO_GET        (1 << 17)
>> +#define VMPORT_LOG_INFO_SET        (1 << 18)
>> +
>> +#define VMPORT_LOG_SKIP_MESSAGE    (1 << 19)
>> +
>> +#define VMPORT_LOG_GP_UNKNOWN      (1 << 20)
>> +#define VMPORT_LOG_GP_NOT_VMWARE   (1 << 21)
>> +#define VMPORT_LOG_GP_FAIL_RD_INST (1 << 22)
>> +
>> +#define VMPORT_LOG_GP_VMWARE_AFTER (1 << 23)
>> +
>> +#define VMPORT_LOG_VGP_UNKNOWN     (1 << 24)
>> +#define VMPORT_LOG_REALMODE_GP     (1 << 27)
>> +
>> +#define VMPORT_LOG_VMWARE_AFTER    (1 << 28)
> Ah, here we go (as to using HVM_DBG_LOG()): Isn't this _way_ too
> fine grained?

It was not when I was checking out all the code. And I do leave a lot
of the debug code in. I can drop some of them, but I find it hard
to guess at a good subset.

I have found that VMPORT_LOG_RPC, VMPORT_LOG_SEND, VMPORT_LOG_ERROR,
and VMPORT_LOG_SKIP_SEND are the 4 I most commonly turn on now. That
amount of logging seems fine on a 9600 baud serial console.


VMPORT_LOG_SKIP_MESSAGE is useful to see all the not BDOOR_CMD_MESSAGE (the guest info
stuff mostly, or RPC as it is also called).


And all the ones with VMPORT_LOG_GP prefix are all about the #gp handling.


Since it now includes xentrace stuff, I could drop some of the ones like
VMPORT_LOG_VMWARE_AFTER


     -Don Slutz

> Jan

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Jan Beulich]

>>> On 03.09.14 at 02:55, <dslutz@verizon.com> wrote:
> On 09/02/14 04:16, Jan Beulich wrote:
>> As I think was said on v1 already - this should be split into smaller
>> pieces if at all possible. I'm therefore not going to do a full review
>> at this time, just give a couple of remarks.
> 
> The last time (v1's) split was much worse (and so I think that "split 
> into smaller"
> was not said). I could check be most were "combine this patch with that 
> patch";
> and more comment message.
> 
> Having been over this code many times in the last month, a possible
> quick split (which would not delay v3 too long) would be:
> 
> 1) Add simple vmport commands like BDOOR_CMD_GETVERSION
> 2) Add the hypervisor side of VMware guest info.
> 3) Add the hvm params that come from VMware guest info.
> 4) Add the hyper calls for libxc to handle VMware guest info.

One thing I'd certainly like to see split out is the save/restore
code. And the second - general - consideration would be to
split hypervisor from tools side changes as much as possible.

> And should I add the unit tests also (as optional)?

No idea - that's a tools side thing iirc, and hence a question to the
tools maintainers.

>>> @@ -579,6 +580,39 @@ long arch_do_domctl(
>>>           }
>>>           break;
>>>   
>>> +        case XEN_DOMCTL_SENDTRIGGER_VTPOWER:
>>> +        {
>>> +            ret = -EINVAL;
>>> +            if ( is_hvm_domain(d) )
>>> +            {
>>> +                ret = 0;
>>> +                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Halt");
>>> +            }
>>> +        }
>>> +        break;
>>> +
>>> +        case XEN_DOMCTL_SENDTRIGGER_VTREBOOT:
>>> +        {
>>> +            ret = -EINVAL;
>>> +            if ( is_hvm_domain(d) )
>>> +            {
>>> +                ret = 0;
>>> +                vmport_ctrl_send(&d->arch.hvm_domain, "OS_Reboot");
>>> +            }
>>> +        }
>>> +        break;
>>> +
>>> +        case XEN_DOMCTL_SENDTRIGGER_VTPING:
>>> +        {
>>> +            ret = -EINVAL;
>>> +            if ( is_hvm_domain(d) )
>>> +            {
>>> +                ret = 0;
>>> +                vmport_ctrl_send(&d->arch.hvm_domain, "ping");
>>> +            }
>>> +        }
>>> +        break;
>> Rather than adding three new domctls, wouldn't a single VMware
>> one with suitable sub-operations do?
> 
> As far as I can tell these are already a "sub-operation":

And right you are. Never mind the comment then.

>> And in any event I'm rather uncomfortable about this getting
>> enabled unconditionally, also due to (but not limited to this) the
>> up front allocation of various memory blocks that may end up
>> never being needed. The main issue I see with this approach is
>> that guests could end up being misguided into thinking they're
>> running on VMware when in fact they have code in place to
>> optimize when running on Xen. We had such detection ordering
>> issues with Linux checking for both Hyper-V and Xen.
> 
> 
> Ok, I do feel strongly that this would need to be independent on
> vmware_hw "version". And so I will add a new xl.cfg item to control this.
> 
> QEMU on 5/19/2014 added a way to turn it's version off:
> 
> +@item vmport=on|off
> +Enables emulation of VMWare IO port, for vmmouse etc. (enabled by default)
> 
> So should I name it vmware_vmport or just vmport ( I lean to just vmport )?
> and unlike QEMU I am fine with the default of off.

I'd prefer the former as being more explicit (vmport alone doesn't
really provide a connection to VMware), but this again is something
to discuss with the tools maintainers.

>>> @@ -1532,6 +1561,17 @@ int hvm_domain_initialise(struct domain *d)
>>>       stdvga_deinit(d);
>>>       vioapic_deinit(d);
>>>    fail1:
>>> +    if ( d->arch.hvm_domain.vmport_data )
>>> +    {
>>> +        struct vmport_state *vs = d->arch.hvm_domain.vmport_data;
>>> +        int idx;
>>> +
>>> +        for (idx = 0; idx < vs->used_guestinfo; idx++)
>> You'll have to go through and fix coding style issues.
> 
> Do to the many coding styles I have used in the last 20 years, I can have
> style "blindness". I do not find a style issue here based on CODING_STYLE.
> All hints are welcome.

Just look at the difference between you if() and your for() - the
latter is lacking blanks inside the parentheses. And that difference
appeared to be a pretty consistent thing throughout the code I
looked a little more closely at.

>>> @@ -106,6 +107,7 @@ MAKE_INSTR(VMSAVE, 3, 0x0f, 0x01, 0xdb);
>>>   MAKE_INSTR(STGI,   3, 0x0f, 0x01, 0xdc);
>>>   MAKE_INSTR(CLGI,   3, 0x0f, 0x01, 0xdd);
>>>   MAKE_INSTR(INVLPGA,3, 0x0f, 0x01, 0xdf);
>>> +MAKE_INSTR(IN,     1, 0xed);
>> This name is ambiguous.
> 
> There are 4 opcodes and 6 instructions. Not at all sure how to name
> the one I need. Here is the info about the IN instruction.

I'd name it INL_DX, but one of the secondary problems you
may need to solve is that unlike for other opcodes you do not
want the operand size override ignored here.

>>> +static void svm_vmexit_gp_intercept(struct cpu_user_regs *regs, struct vcpu 
> *v)
>>> +{
>>> +    struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
>>> +    unsigned long inst_len = __get_instruction_length(v, INSTR_IN);
>>> +
>>> +    regs->error_code = vmcb->exitinfo1;
>>> +    if ( hvm_long_mode_enabled(v) )
>>> +        HVMTRACE_LONG_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
>>> +                          regs->error_code,
>>> +                          TRC_PAR_LONG(vmcb->exitinfo2));
>>> +    else
>>> +        HVMTRACE_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
>>> +                     regs->error_code, vmcb->exitinfo2);
>>> +
>>> +    if ( inst_len <= 1 && (regs->rdx & 0xffff) == VMPORT_PORT &&
>> What would inst_len being zero mean here?
> 
> It use to mean that AMD was missing the feature:
> 
> 
> (XEN) [2014-08-14 20:17:28] - Next-RIP Saved on #VMEXIT
> 
> and so I needed to look at the opcode.

__get_instruction_length_from_list() returns using the value resulting
from that feature only when the length is non-zero.

> Now that I am using __get_instruction_length() I think it could be changed
> to a 1. However I can only test on an AMD server that does have this
> feature. Will switch to == 1 and hope for the best.
> 
> As far as I can tell __get_instruction_length() does not insure that the 
> opcode
> list passed is found, just that it might be.

Right, and zero is an indication that it wasn't found. Also I just
noticed there's a gdprintk() in that event, which for all other
cases the function gets used for is quite fine. #GP, however,
can result from various instructions, and hence you don't want
a message printed each time it wasn't due to "inl %dx, %eax".

>>> +         vmcb->exitinfo2 == 0 && regs->error_code == 0 )
>>> +    {
>>> +        uint32_t magic = regs->rax;
>>> +
>>> +        if ( magic == VMPORT_MAGIC )
>> This ought to be folded with the previous if(), at once reducing
>> code redundancy further down in the function.
> 
> OK. But that does make the debug log message more complex to code.

Maybe, albeit (as hinted at before) I question the use of these
anyway.

>>> +        {
>>> +            unsigned char bytes[1] = { 0 };
>> Pointless initializer (and it being an array looks suspicious too).
> 
> Will drop the initializer. hvm_fetch_from_guest_virt_nofault() does take 
> an array.
> So it is better to say "&tmp, regs->rip, 1" then "bytes, regs->rip, 1" ?

Yes, I think so.

> Also by dropping the initializer, you are stating that
> hvm_fetch_from_guest_virt_nofault() does initialize it's output on error (
> because the debug log output does include bytes[0]).

Existing debug log output, or one you add? It's a mistake in any case,
I'm just trying to understand whether a fix might be one order
independent of your patch.

>>   Speaking of which - why can't you just define
>> a new DBG_LEVEL_VMPORT and use the existing HVM_DBG_LOG()?
> 
> I did define 23 bits to control various part of the debug. A lot of this
> is because there are many times you go through the code. Basicly
> a string is 1st passed a length, and the 4 bytes at a time until the
> length is handled, followed by state changes...

Which again raises the question of the value of all this debugging
output. Whether or not a niche feature, I don't think it should be
significantly more verbose than the base code we have.

Jan

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Don Slutz]

On 09/03/14 04:25, Jan Beulich wrote:
>>>> On 03.09.14 at 02:55, <dslutz@verizon.com> wrote:
>> On 09/02/14 04:16, Jan Beulich wrote:
>>> As I think was said on v1 already - this should be split into smaller
>>> pieces if at all possible. I'm therefore not going to do a full review
>>> at this time, just give a couple of remarks.
>> The last time (v1's) split was much worse (and so I think that "split
>> into smaller"
>> was not said). I could check be most were "combine this patch with that
>> patch";
>> and more comment message.
>>
>> Having been over this code many times in the last month, a possible
>> quick split (which would not delay v3 too long) would be:
>>
>> 1) Add simple vmport commands like BDOOR_CMD_GETVERSION
>> 2) Add the hypervisor side of VMware guest info.
>> 3) Add the hvm params that come from VMware guest info.
>> 4) Add the hyper calls for libxc to handle VMware guest info.
> One thing I'd certainly like to see split out is the save/restore
> code. And the second - general - consideration would be to
> split hypervisor from tools side changes as much as possible.

I will take that into account.

>> And should I add the unit tests also (as optional)?
> No idea - that's a tools side thing iirc, and hence a question to the
> tools maintainers.

Will see if I get a response before v3 code changes have been tested
and are ready to post.

>

>>> And in any event I'm rather uncomfortable about this getting
>>> enabled unconditionally, also due to (but not limited to this) the
>>> up front allocation of various memory blocks that may end up
>>> never being needed. The main issue I see with this approach is
>>> that guests could end up being misguided into thinking they're
>>> running on VMware when in fact they have code in place to
>>> optimize when running on Xen. We had such detection ordering
>>> issues with Linux checking for both Hyper-V and Xen.
>>
>> Ok, I do feel strongly that this would need to be independent on
>> vmware_hw "version". And so I will add a new xl.cfg item to control this.
>>
>> QEMU on 5/19/2014 added a way to turn it's version off:
>>
>> +@item vmport=on|off
>> +Enables emulation of VMWare IO port, for vmmouse etc. (enabled by default)
>>
>> So should I name it vmware_vmport or just vmport ( I lean to just vmport )?
>> and unlike QEMU I am fine with the default of off.
> I'd prefer the former as being more explicit (vmport alone doesn't
> really provide a connection to VMware), but this again is something
> to discuss with the tools maintainers.
>

Ok.

>>>> @@ -1532,6 +1561,17 @@ int hvm_domain_initialise(struct domain *d)
>>>>        stdvga_deinit(d);
>>>>        vioapic_deinit(d);
>>>>     fail1:
>>>> +    if ( d->arch.hvm_domain.vmport_data )
>>>> +    {
>>>> +        struct vmport_state *vs = d->arch.hvm_domain.vmport_data;
>>>> +        int idx;
>>>> +
>>>> +        for (idx = 0; idx < vs->used_guestinfo; idx++)
>>> You'll have to go through and fix coding style issues.
>> Do to the many coding styles I have used in the last 20 years, I can have
>> style "blindness". I do not find a style issue here based on CODING_STYLE.
>> All hints are welcome.
> Just look at the difference between you if() and your for() - the
> latter is lacking blanks inside the parentheses. And that difference
> appeared to be a pretty consistent thing throughout the code I
> looked a little more closely at.

like I said I do not see the difference until pointed out.  Thank you
for this.  Will fix all my "for"(s) to this style.


>>>> @@ -106,6 +107,7 @@ MAKE_INSTR(VMSAVE, 3, 0x0f, 0x01, 0xdb);
>>>>    MAKE_INSTR(STGI,   3, 0x0f, 0x01, 0xdc);
>>>>    MAKE_INSTR(CLGI,   3, 0x0f, 0x01, 0xdd);
>>>>    MAKE_INSTR(INVLPGA,3, 0x0f, 0x01, 0xdf);
>>>> +MAKE_INSTR(IN,     1, 0xed);
>>> This name is ambiguous.
>> There are 4 opcodes and 6 instructions. Not at all sure how to name
>> the one I need. Here is the info about the IN instruction.
> I'd name it INL_DX, but one of the secondary problems you
> may need to solve is that unlike for other opcodes you do not
> want the operand size override ignored here.

Ok, I will go with INL_DX.  So far I have not found any code that has the
operand size prefix.  I expect that since all code is in 32 or 64 bit
segments and it is expected that 32 bits are used, so the form "IN AX,DX"
(which would have the operand size prefix) is not used.

In any case, I will do some checking on what I get for this case.  I will
agree that handling of the operand size prefix is not clear.  And may
not be needed to be solved at this time.


>>>> +static void svm_vmexit_gp_intercept(struct cpu_user_regs *regs, struct vcpu
>> *v)
>>>> +{
>>>> +    struct vmcb_struct *vmcb = v->arch.hvm_svm.vmcb;
>>>> +    unsigned long inst_len = __get_instruction_length(v, INSTR_IN);
>>>> +
>>>> +    regs->error_code = vmcb->exitinfo1;
>>>> +    if ( hvm_long_mode_enabled(v) )
>>>> +        HVMTRACE_LONG_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
>>>> +                          regs->error_code,
>>>> +                          TRC_PAR_LONG(vmcb->exitinfo2));
>>>> +    else
>>>> +        HVMTRACE_C4D(TRAP_GP, TRAP_gp_fault, inst_len,
>>>> +                     regs->error_code, vmcb->exitinfo2);
>>>> +
>>>> +    if ( inst_len <= 1 && (regs->rdx & 0xffff) == VMPORT_PORT &&
>>> What would inst_len being zero mean here?
>> It use to mean that AMD was missing the feature:
>>
>>
>> (XEN) [2014-08-14 20:17:28] - Next-RIP Saved on #VMEXIT
>>
>> and so I needed to look at the opcode.
> __get_instruction_length_from_list() returns using the value resulting
> from that feature only when the length is non-zero.
>
>> Now that I am using __get_instruction_length() I think it could be changed
>> to a 1. However I can only test on an AMD server that does have this
>> feature. Will switch to == 1 and hope for the best.
>>
>> As far as I can tell __get_instruction_length() does not insure that the
>> opcode
>> list passed is found, just that it might be.
> Right, and zero is an indication that it wasn't found. Also I just
> noticed there's a gdprintk() in that event, which for all other
> cases the function gets used for is quite fine. #GP, however,
> can result from various instructions, and hence you don't want
> a message printed each time it wasn't due to "inl %dx, %eax".
>

Will look into how to fix this.


>>>> +         vmcb->exitinfo2 == 0 && regs->error_code == 0 )
>>>> +    {
>>>> +        uint32_t magic = regs->rax;
>>>> +
>>>> +        if ( magic == VMPORT_MAGIC )
>>> This ought to be folded with the previous if(), at once reducing
>>> code redundancy further down in the function.
>> OK. But that does make the debug log message more complex to code.
> Maybe, albeit (as hinted at before) I question the use of these
> anyway.
>

Ok.

>>>> +        {
>>>> +            unsigned char bytes[1] = { 0 };
>>> Pointless initializer (and it being an array looks suspicious too).
>> Will drop the initializer. hvm_fetch_from_guest_virt_nofault() does take
>> an array.
>> So it is better to say "&tmp, regs->rip, 1" then "bytes, regs->rip, 1" ?
> Yes, I think so.
Fine, will change to that.

>> Also by dropping the initializer, you are stating that
>> hvm_fetch_from_guest_virt_nofault() does initialize it's output on error (
>> because the debug log output does include bytes[0]).
> Existing debug log output, or one you add? It's a mistake in any case,
> I'm just trying to understand whether a fix might be one order
> independent of your patch.

It was in code I added.

>>>    Speaking of which - why can't you just define
>>> a new DBG_LEVEL_VMPORT and use the existing HVM_DBG_LOG()?
>> I did define 23 bits to control various part of the debug. A lot of this
>> is because there are many times you go through the code. Basicly
>> a string is 1st passed a length, and the 4 bytes at a time until the
>> length is handled, followed by state changes...
> Which again raises the question of the value of all this debugging
> output. Whether or not a niche feature, I don't think it should be
> significantly more verbose than the base code we have.

I will spend some time to make this have less options.

     -Don Slutz

> Jan

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Ian Campbell]

On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
> This is both a more complete support then in currently provided by
> QEMU and/or KVM and less.  The missing part requires QEMU changes
> and has been left out until the QEMU patches are accepted upstream.
> 
> VMware's hyper-call is also known as VMware Backdoor I/O Port.
> 
> Note: this support does not depend on vmware_hw being non-zero.
> 
> Summary is that VMware treats "IN EAX, DX" (or "OUT DX, EAX"; or
> "inl %dx, %eax" in AT&T syntax ) to port 0x5658 specially.  Note:
> since many operations return data in EAX, "OUT DX, EAX" does not
> work for them on VMware, I did not support the "OUT DX, EAX", but it
> would not be hard to add.

Quite a bit of this changelog seems to concern itself with the
differences between Intel and AT&T as syntax. Please can you just use
the same AT&T syntax as are used in Xen's own .S files throughout (and
without the need to comment on there being other syntaxes).

> Also this instruction is allowed to be used from ring 3.  To
> support this the vmexit for GP needs to be enabled.

Isn't that quite costly?

> The support included is enough to allow VMware tools to install in a
> HVM domU and provide guestinfo support.  guestinfo support is
> provide by what is known as VMware RPC support.  This guestinfo
> support is provided via libxc.  libxl support has not be written.

I suppose this isn't a true RPC, since there isn't any actual running
code on the remote side? (alternatively if you have added some sort of
daemon backend to libxc then we need to talk ;-))

> If the domU is running VMware tools, then the "build version" of
> the tools is also available via xc_get_HVM_param().  This also

It's just xc_get_hvm_param(), isn't it?

> +int xc_set_vmport_guest_info(xc_interface *handle,
> +                             domid_t dom,
> +                             unsigned int key_len,
> +                             char *key,
> +                             unsigned int val_len,
> +                             char *val)

Can key and val have embedded NULs?

(another way of asking if we can treat one or both as a null terminated
string)

> +int xc_get_vmport_guest_info(xc_interface *handle,
> +                             domid_t dom,
> +                             unsigned int key_len,
> +                             char *key,
> +                             unsigned int val_max,
> +                             unsigned int *val_len,
> +                             char *val)

> +int xc_fetch_vmport_guest_info(xc_interface *handle,
> +                               domid_t dom,
> +                               unsigned int idx,
> +                               unsigned int key_max,
> +                               unsigned int *key_len,
> +                               char *key,
> +                               unsigned int val_max,
> +                               unsigned int *val_len,
> +                               char *val)

How do get and fetch differ? I can see how they differ in code terms,
but I'm not sure why and what the differences imply.

Ian.

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Don Slutz]

On 09/08/14 09:35, Ian Campbell wrote:
> On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
>> This is both a more complete support then in currently provided by
>> QEMU and/or KVM and less.  The missing part requires QEMU changes
>> and has been left out until the QEMU patches are accepted upstream.
>>
>> VMware's hyper-call is also known as VMware Backdoor I/O Port.
>>
>> Note: this support does not depend on vmware_hw being non-zero.
>>
>> Summary is that VMware treats "IN EAX, DX" (or "OUT DX, EAX"; or
>> "inl %dx, %eax" in AT&T syntax ) to port 0x5658 specially.  Note:
>> since many operations return data in EAX, "OUT DX, EAX" does not
>> work for them on VMware, I did not support the "OUT DX, EAX", but it
>> would not be hard to add.
> Quite a bit of this changelog seems to concern itself with the
> differences between Intel and AT&T as syntax. Please can you just use
> the same AT&T syntax as are used in Xen's own .S files throughout (and
> without the need to comment on there being other syntaxes).
>

Ok, Will adjust to have only AT&T syntax.

>> Also this instruction is allowed to be used from ring 3.  To
>> support this the vmexit for GP needs to be enabled.
> Isn't that quite costly?

Yes.  But since that is how VMware does it, I need to do the same slow
thing.

>
>> The support included is enough to allow VMware tools to install in a
>> HVM domU and provide guestinfo support.  guestinfo support is
>> provide by what is known as VMware RPC support.  This guestinfo
>> support is provided via libxc.  libxl support has not be written.
> I suppose this isn't a true RPC, since there isn't any actual running
> code on the remote side? (alternatively if you have added some sort of
> daemon backend to libxc then we need to talk ;-))

Nope, it is not a true RPC.  However that is the way VMware's
documentation talks about it.  However it is a very slow speed
way of passing data into or out of a domU.  At some point it
does make sense to consider how libxl might change to take
advantage of this, but I am sure that this is not happening for 4.5.

This was why I provided the optional unit test code as an example
of the use of the libxc changes.

>> If the domU is running VMware tools, then the "build version" of
>> the tools is also available via xc_get_HVM_param().  This also
> It's just xc_get_hvm_param(), isn't it?
>

Yes. looks like a typo.


>> +int xc_set_vmport_guest_info(xc_interface *handle,
>> +                             domid_t dom,
>> +                             unsigned int key_len,
>> +                             char *key,
>> +                             unsigned int val_len,
>> +                             char *val)
> Can key and val have embedded NULs?
>
> (another way of asking if we can treat one or both as a null terminated
> string)

I have not seen any embedded NULs for guest info, but the way they
are tansfered do include embedded NULs.  And so it is all coded to
handle strings with embedded NULs.

>> +int xc_get_vmport_guest_info(xc_interface *handle,
>> +                             domid_t dom,
>> +                             unsigned int key_len,
>> +                             char *key,
>> +                             unsigned int val_max,
>> +                             unsigned int *val_len,
>> +                             char *val)
>> +int xc_fetch_vmport_guest_info(xc_interface *handle,
>> +                               domid_t dom,
>> +                               unsigned int idx,
>> +                               unsigned int key_max,
>> +                               unsigned int *key_len,
>> +                               char *key,
>> +                               unsigned int val_max,
>> +                               unsigned int *val_len,
>> +                               char *val)
> How do get and fetch differ? I can see how they differ in code terms,
> but I'm not sure why and what the differences imply.

get does just the specified key.  fetch is used to get all key value pairs.
It might helpp to look at v3 #15 which uses fetch vs v3 #14 which uses
get in the unit test code.

Hope this helps.
     -Don Slutz


> Ian.
>

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Ian Campbell]

On Mon, 2014-09-08 at 12:57 -0400, Don Slutz wrote:
> >> Also this instruction is allowed to be used from ring 3.  To
> >> support this the vmexit for GP needs to be enabled.
> > Isn't that quite costly?
> 
> Yes.  But since that is how VMware does it, I need to do the same slow
> thing.

Sounds from other subthreads like there might be other better ways? It's
hard to believe that vmware is really trapping every #GP.

> >> The support included is enough to allow VMware tools to install in a
> >> HVM domU and provide guestinfo support.  guestinfo support is
> >> provide by what is known as VMware RPC support.  This guestinfo
> >> support is provided via libxc.  libxl support has not be written.
> > I suppose this isn't a true RPC, since there isn't any actual running
> > code on the remote side? (alternatively if you have added some sort of
> > daemon backend to libxc then we need to talk ;-))
> 
> Nope, it is not a true RPC.  However that is the way VMware's
> documentation talks about it.  However it is a very slow speed
> way of passing data into or out of a domU.  At some point it
> does make sense to consider how libxl might change to take
> advantage of this, but I am sure that this is not happening for 4.5.
> 
> This was why I provided the optional unit test code as an example
> of the use of the libxc changes.

So is the libxc code as proposed today actually used for anything?

> >> +int xc_set_vmport_guest_info(xc_interface *handle,
> >> +                             domid_t dom,
> >> +                             unsigned int key_len,
> >> +                             char *key,
> >> +                             unsigned int val_len,
> >> +                             char *val)
> > Can key and val have embedded NULs?
> >
> > (another way of asking if we can treat one or both as a null terminated
> > string)
> 
> I have not seen any embedded NULs for guest info, but the way they
> are tansfered do include embedded NULs.  And so it is all coded to
> handle strings with embedded NULs.

OK.

> 
> >> +int xc_get_vmport_guest_info(xc_interface *handle,
> >> +                             domid_t dom,
> >> +                             unsigned int key_len,
> >> +                             char *key,
> >> +                             unsigned int val_max,
> >> +                             unsigned int *val_len,
> >> +                             char *val)
> >> +int xc_fetch_vmport_guest_info(xc_interface *handle,
> >> +                               domid_t dom,
> >> +                               unsigned int idx,
> >> +                               unsigned int key_max,
> >> +                               unsigned int *key_len,
> >> +                               char *key,
> >> +                               unsigned int val_max,
> >> +                               unsigned int *val_len,
> >> +                               char *val)
> > How do get and fetch differ? I can see how they differ in code terms,
> > but I'm not sure why and what the differences imply.
> 
> get does just the specified key.  fetch is used to get all key value pairs.
> It might helpp to look at v3 #15 which uses fetch vs v3 #14 which uses
> get in the unit test code.

Perhaps get and either get_all or fetch_all would be clearer names?

Ian.

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Don Slutz]

On 09/09/14 05:36, Ian Campbell wrote:
> On Mon, 2014-09-08 at 12:57 -0400, Don Slutz wrote:
>>>> Also this instruction is allowed to be used from ring 3.  To
>>>> support this the vmexit for GP needs to be enabled.
>>> Isn't that quite costly?
>> Yes.  But since that is how VMware does it, I need to do the same slow
>> thing.
> Sounds from other subthreads like there might be other better ways? It's
> hard to believe that vmware is really trapping every #GP.

I have not found a better way.  The simplest statement I have come
up with is that this is not a pass thru of the VMware device.  Or the
statement (in AMD land): Generate an IOIO #VMEXIT not a GP
#VMWEXIT for ioport <x> (or all ports).

And yes this sounds bad, until you think about how many GP #VMEXIT
are done.  For both Linux and Windows this is a small number (< 10).

Any others would be application ones.

I am working on making the GP #VMEXIT optional.


>>>> The support included is enough to allow VMware tools to install in a
>>>> HVM domU and provide guestinfo support.  guestinfo support is
>>>> provide by what is known as VMware RPC support.  This guestinfo
>>>> support is provided via libxc.  libxl support has not be written.
>>> I suppose this isn't a true RPC, since there isn't any actual running
>>> code on the remote side? (alternatively if you have added some sort of
>>> daemon backend to libxc then we need to talk ;-))
>> Nope, it is not a true RPC.  However that is the way VMware's
>> documentation talks about it.  However it is a very slow speed
>> way of passing data into or out of a domU.  At some point it
>> does make sense to consider how libxl might change to take
>> advantage of this, but I am sure that this is not happening for 4.5.
>>
>> This was why I provided the optional unit test code as an example
>> of the use of the libxc changes.
> So is the libxc code as proposed today actually used for anything?

Yes.  2 main areas:

1) Clean shutdown of windows guests with VMware tools installed.
      (acpi poweroff does not work if logged off).
2) set root's password and hostname at 1st boot of a template
     (done by VMware guestinfo).  Note: this could have been done with
     xenstore (XenBus?) but was not since we also use the VMware
     mouse support (not for 4.5, planned for 4.6 needs QEMU support).


>>>> +int xc_set_vmport_guest_info(xc_interface *handle,
>>>> +                             domid_t dom,
>>>> +                             unsigned int key_len,
>>>> +                             char *key,
>>>> +                             unsigned int val_len,
>>>> +                             char *val)
>>> Can key and val have embedded NULs?
>>>
>>> (another way of asking if we can treat one or both as a null terminated
>>> string)
>> I have not seen any embedded NULs for guest info, but the way they
>> are tansfered do include embedded NULs.  And so it is all coded to
>> handle strings with embedded NULs.
> OK.
>
>>>> +int xc_get_vmport_guest_info(xc_interface *handle,
>>>> +                             domid_t dom,
>>>> +                             unsigned int key_len,
>>>> +                             char *key,
>>>> +                             unsigned int val_max,
>>>> +                             unsigned int *val_len,
>>>> +                             char *val)
>>>> +int xc_fetch_vmport_guest_info(xc_interface *handle,
>>>> +                               domid_t dom,
>>>> +                               unsigned int idx,
>>>> +                               unsigned int key_max,
>>>> +                               unsigned int *key_len,
>>>> +                               char *key,
>>>> +                               unsigned int val_max,
>>>> +                               unsigned int *val_len,
>>>> +                               char *val)
>>> How do get and fetch differ? I can see how they differ in code terms,
>>> but I'm not sure why and what the differences imply.
>> get does just the specified key.  fetch is used to get all key value pairs.
>> It might helpp to look at v3 #15 which uses fetch vs v3 #14 which uses
>> get in the unit test code.
> Perhaps get and either get_all or fetch_all would be clearer names?

I will go with fetch_all.

     -Don Slutz

> Ian.
>
>

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Boris Ostrovsky]

On 09/09/2014 01:31 PM, Don Slutz wrote:
> On 09/09/14 05:36, Ian Campbell wrote:
>> On Mon, 2014-09-08 at 12:57 -0400, Don Slutz wrote:
>>>>> Also this instruction is allowed to be used from ring 3.  To
>>>>> support this the vmexit for GP needs to be enabled.
>>>> Isn't that quite costly?
>>> Yes.  But since that is how VMware does it, I need to do the same slow
>>> thing.
>> Sounds from other subthreads like there might be other better ways? It's
>> hard to believe that vmware is really trapping every #GP.
>
> I have not found a better way.  The simplest statement I have come
> up with is that this is not a pass thru of the VMware device.  Or the
> statement (in AMD land): Generate an IOIO #VMEXIT not a GP
> #VMWEXIT for ioport <x> (or all ports).



When I asked about enabling #GP intercepts only when we know that the 
guest is VMware-aware I meant that you'd do it as soon as you detect 
that you are having such a guest (e.g. when you set 
HVM_PARAM_VMWARE_PORT from libxl).  You set is_vmware=1, for example. 
And then you only add TRAP_gp_fault intercept in construct_vmcb() when 
is_vmware is true (and probably something similar for VMX). Or something 
along these lines.

I suspect you are trying to figure out how to decide this during guest 
execution, but that's not what I was referring to.


-boris

>
> And yes this sounds bad, until you think about how many GP #VMEXIT
> are done.  For both Linux and Windows this is a small number (< 10).
>
> Any others would be application ones.
>
> I am working on making the GP #VMEXIT optional.
>
>
>>>>> The support included is enough to allow VMware tools to install in a
>>>>> HVM domU and provide guestinfo support.  guestinfo support is
>>>>> provide by what is known as VMware RPC support.  This guestinfo
>>>>> support is provided via libxc.  libxl support has not be written.
>>>> I suppose this isn't a true RPC, since there isn't any actual running
>>>> code on the remote side? (alternatively if you have added some sort of
>>>> daemon backend to libxc then we need to talk ;-))
>>> Nope, it is not a true RPC.  However that is the way VMware's
>>> documentation talks about it.  However it is a very slow speed
>>> way of passing data into or out of a domU.  At some point it
>>> does make sense to consider how libxl might change to take
>>> advantage of this, but I am sure that this is not happening for 4.5.
>>>
>>> This was why I provided the optional unit test code as an example
>>> of the use of the libxc changes.
>> So is the libxc code as proposed today actually used for anything?
>
> Yes.  2 main areas:
>
> 1) Clean shutdown of windows guests with VMware tools installed.
>      (acpi poweroff does not work if logged off).
> 2) set root's password and hostname at 1st boot of a template
>     (done by VMware guestinfo).  Note: this could have been done with
>     xenstore (XenBus?) but was not since we also use the VMware
>     mouse support (not for 4.5, planned for 4.6 needs QEMU support).
>
>
>>>>> +int xc_set_vmport_guest_info(xc_interface *handle,
>>>>> +                             domid_t dom,
>>>>> +                             unsigned int key_len,
>>>>> +                             char *key,
>>>>> +                             unsigned int val_len,
>>>>> +                             char *val)
>>>> Can key and val have embedded NULs?
>>>>
>>>> (another way of asking if we can treat one or both as a null 
>>>> terminated
>>>> string)
>>> I have not seen any embedded NULs for guest info, but the way they
>>> are tansfered do include embedded NULs.  And so it is all coded to
>>> handle strings with embedded NULs.
>> OK.
>>
>>>>> +int xc_get_vmport_guest_info(xc_interface *handle,
>>>>> +                             domid_t dom,
>>>>> +                             unsigned int key_len,
>>>>> +                             char *key,
>>>>> +                             unsigned int val_max,
>>>>> +                             unsigned int *val_len,
>>>>> +                             char *val)
>>>>> +int xc_fetch_vmport_guest_info(xc_interface *handle,
>>>>> +                               domid_t dom,
>>>>> +                               unsigned int idx,
>>>>> +                               unsigned int key_max,
>>>>> +                               unsigned int *key_len,
>>>>> +                               char *key,
>>>>> +                               unsigned int val_max,
>>>>> +                               unsigned int *val_len,
>>>>> +                               char *val)
>>>> How do get and fetch differ? I can see how they differ in code terms,
>>>> but I'm not sure why and what the differences imply.
>>> get does just the specified key.  fetch is used to get all key value 
>>> pairs.
>>> It might helpp to look at v3 #15 which uses fetch vs v3 #14 which uses
>>> get in the unit test code.
>> Perhaps get and either get_all or fetch_all would be clearer names?
>
> I will go with fetch_all.
>
>     -Don Slutz
>
>> Ian.
>>
>>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Ian Campbell]

On Tue, 2014-09-09 at 13:31 -0400, Don Slutz wrote:
> On 09/09/14 05:36, Ian Campbell wrote:
> > On Mon, 2014-09-08 at 12:57 -0400, Don Slutz wrote:
> >>>> Also this instruction is allowed to be used from ring 3.  To
> >>>> support this the vmexit for GP needs to be enabled.
> >>> Isn't that quite costly?
> >> Yes.  But since that is how VMware does it, I need to do the same slow
> >> thing.
> > Sounds from other subthreads like there might be other better ways? It's
> > hard to believe that vmware is really trapping every #GP.
> 
> I have not found a better way.  The simplest statement I have come
> up with is that this is not a pass thru of the VMware device.  Or the
> statement (in AMD land): Generate an IOIO #VMEXIT not a GP
> #VMWEXIT for ioport <x> (or all ports).
> 
> And yes this sounds bad, until you think about how many GP #VMEXIT
> are done.  For both Linux and Windows this is a small number (< 10).

Hrm, well this is mainly one for the hypervisor guys and I believe there
is already a subthread exploring option here so I'll leave it at that.

> >>>> The support included is enough to allow VMware tools to install in a
> >>>> HVM domU and provide guestinfo support.  guestinfo support is
> >>>> provide by what is known as VMware RPC support.  This guestinfo
> >>>> support is provided via libxc.  libxl support has not be written.
> >>> I suppose this isn't a true RPC, since there isn't any actual running
> >>> code on the remote side? (alternatively if you have added some sort of
> >>> daemon backend to libxc then we need to talk ;-))
> >> Nope, it is not a true RPC.  However that is the way VMware's
> >> documentation talks about it.  However it is a very slow speed
> >> way of passing data into or out of a domU.  At some point it
> >> does make sense to consider how libxl might change to take
> >> advantage of this, but I am sure that this is not happening for 4.5.
> >>
> >> This was why I provided the optional unit test code as an example
> >> of the use of the libxc changes.
> > So is the libxc code as proposed today actually used for anything?
> 
> Yes.  2 main areas:
> 
> 1) Clean shutdown of windows guests with VMware tools installed.
>       (acpi poweroff does not work if logged off).
> 2) set root's password and hostname at 1st boot of a template
>      (done by VMware guestinfo).  Note: this could have been done with
>      xenstore (XenBus?) but was not since we also use the VMware
>      mouse support (not for 4.5, planned for 4.6 needs QEMU support).

Is that functionality in this series and I've just missed it?

Ian

Re: [PATCH v2 3/3] Add limited support of VMware's hyper-call

[Don Slutz]

On 09/10/14 05:32, Ian Campbell wrote:
> On Tue, 2014-09-09 at 13:31 -0400, Don Slutz wrote:
>> On 09/09/14 05:36, Ian Campbell wrote:
>>> On Mon, 2014-09-08 at 12:57 -0400, Don Slutz wrote:
>>>>>> Also this instruction is allowed to be used from ring 3.  To
>>>>>> support this the vmexit for GP needs to be enabled.
>>>>> Isn't that quite costly?
>>>> Yes.  But since that is how VMware does it, I need to do the same slow
>>>> thing.
>>> Sounds from other subthreads like there might be other better ways? It's
>>> hard to believe that vmware is really trapping every #GP.
>> I have not found a better way.  The simplest statement I have come
>> up with is that this is not a pass thru of the VMware device.  Or the
>> statement (in AMD land): Generate an IOIO #VMEXIT not a GP
>> #VMWEXIT for ioport <x> (or all ports).
>>
>> And yes this sounds bad, until you think about how many GP #VMEXIT
>> are done.  For both Linux and Windows this is a small number (< 10).
> Hrm, well this is mainly one for the hypervisor guys and I believe there
> is already a subthread exploring option here so I'll leave it at that.
>
>>>>>> The support included is enough to allow VMware tools to install in a
>>>>>> HVM domU and provide guestinfo support.  guestinfo support is
>>>>>> provide by what is known as VMware RPC support.  This guestinfo
>>>>>> support is provided via libxc.  libxl support has not be written.
>>>>> I suppose this isn't a true RPC, since there isn't any actual running
>>>>> code on the remote side? (alternatively if you have added some sort of
>>>>> daemon backend to libxc then we need to talk ;-))
>>>> Nope, it is not a true RPC.  However that is the way VMware's
>>>> documentation talks about it.  However it is a very slow speed
>>>> way of passing data into or out of a domU.  At some point it
>>>> does make sense to consider how libxl might change to take
>>>> advantage of this, but I am sure that this is not happening for 4.5.
>>>>
>>>> This was why I provided the optional unit test code as an example
>>>> of the use of the libxc changes.
>>> So is the libxc code as proposed today actually used for anything?
>> Yes.  2 main areas:
>>
>> 1) Clean shutdown of windows guests with VMware tools installed.
>>        (acpi poweroff does not work if logged off).
>> 2) set root's password and hostname at 1st boot of a template
>>       (done by VMware guestinfo).  Note: this could have been done with
>>       xenstore (XenBus?) but was not since we also use the VMware
>>       mouse support (not for 4.5, planned for 4.6 needs QEMU support).
> Is that functionality in this series and I've just missed it?

The clean shutdown of windows can be done with the optional
code in v3 (not this patch set):

A tool to send a trigger to a domU via xc_domain_send_trigger

I know this is not the best and having something in shutdown_domain (or
libxl_domain_shutdown) and reboot_domain (or libxl_domain_reboot) whould
be much better.  I think this is too late in 4.5 since the code has not
been written yet (the tool stack I use all the time does have it).

And for #2, you also need a script in the guest, VMware tools installed in
the guest and use the optional tool in v3:

A tool to get and set VMware guestinfo

which I do have.  I might have mis-understood your question.  Hope this
helps.

     -Don Slutz



> Ian
>

Re: [PATCH v2 0/3] Xen VMware tools support

[Jan Beulich]

>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
> changes RFC to v2:
> 
> Jan Beulich:
>   Add xen/arch/x86/hvm/vmware.c for cpuid_vmware_leaves
>   Fewer patches
> [...]
>  xen/arch/x86/hvm/vmport/Makefile             |    1 +
>  xen/arch/x86/hvm/vmport/backdoor_def.h       |  167 +++
>  xen/arch/x86/hvm/vmport/guest_msg_def.h      |   87 ++
>  xen/arch/x86/hvm/vmport/includeCheck.h       |   17 +
>  xen/arch/x86/hvm/vmport/vmport.c             | 1436 ++++++++++++++++++++++++++
>  xen/arch/x86/hvm/vmware.c                    |   85 ++

I don't think you should be adding a new fine in hvm/ _and_ a
new subdirectory. If VMware support requires more than one
file, everything should probably go into hvm/vmware/.

Jan

Re: [PATCH v2 0/3] Xen VMware tools support

[Don Slutz]

On 09/01/14 12:10, Jan Beulich wrote:
>>>> On 01.09.14 at 17:33, <dslutz@verizon.com> wrote:
>> changes RFC to v2:
>>
>> Jan Beulich:
>>    Add xen/arch/x86/hvm/vmware.c for cpuid_vmware_leaves
>>    Fewer patches
>> [...]
>>   xen/arch/x86/hvm/vmport/Makefile             |    1 +
>>   xen/arch/x86/hvm/vmport/backdoor_def.h       |  167 +++
>>   xen/arch/x86/hvm/vmport/guest_msg_def.h      |   87 ++
>>   xen/arch/x86/hvm/vmport/includeCheck.h       |   17 +
>>   xen/arch/x86/hvm/vmport/vmport.c             | 1436 ++++++++++++++++++++++++++
>>   xen/arch/x86/hvm/vmware.c                    |   85 ++
> I don't think you should be adding a new fine in hvm/ _and_ a
> new subdirectory. If VMware support requires more than one
> file, everything should probably go into hvm/vmware/.

Sure.
    -Don Slutz


> Jan
>

Re: [PATCH v2 0/3] Xen VMware tools support

[Ian Campbell]

On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
[...]
> I picked this subset to start with because it only has changes in
> Xen.
[...]
>  docs/man/xl.cfg.pod.5                        |    6 +
>  tools/firmware/hvmloader/pci.c               |   10 +-
>  tools/libxc/xc_domain.c                      |  115 +++
>  tools/libxc/xc_domain_restore.c              |   14 +
>  tools/libxc/xc_domain_save.c                 |   11 +
>  tools/libxc/xenctrl.h                        |   24 +
>  tools/libxc/xg_save_restore.h                |    2 +
>  tools/libxl/libxl_create.c                   |    4 +-
>  tools/libxl/libxl_dm.c                       |   33 +-
>  tools/libxl/libxl_dom.c                      |    2 +
>  tools/libxl/libxl_types.idl                  |    1 +
>  tools/libxl/xl_cmdimpl.c                     |    2 +
>  tools/misc/xen-hvmctx.c                      |  229 ++++
>  tools/xentrace/formats                       |    8 +

Bah, I thought my review-load had just decreased! ;-)

Ian.

Re: [PATCH v2 0/3] Xen VMware tools support

[Don Slutz]

On 09/08/14 09:03, Ian Campbell wrote:
> On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
> [...]
>> I picked this subset to start with because it only has changes in
>> Xen.
> [...]
>>   docs/man/xl.cfg.pod.5                        |    6 +
>>   tools/firmware/hvmloader/pci.c               |   10 +-
>>   tools/libxc/xc_domain.c                      |  115 +++
>>   tools/libxc/xc_domain_restore.c              |   14 +
>>   tools/libxc/xc_domain_save.c                 |   11 +
>>   tools/libxc/xenctrl.h                        |   24 +
>>   tools/libxc/xg_save_restore.h                |    2 +
>>   tools/libxl/libxl_create.c                   |    4 +-
>>   tools/libxl/libxl_dm.c                       |   33 +-
>>   tools/libxl/libxl_dom.c                      |    2 +
>>   tools/libxl/libxl_types.idl                  |    1 +
>>   tools/libxl/xl_cmdimpl.c                     |    2 +
>>   tools/misc/xen-hvmctx.c                      |  229 ++++
>>   tools/xentrace/formats                       |    8 +
> Bah, I thought my review-load had just decreased! ;-)
>
> Ian.
>

No need to look at v2, v3 has been sent.
      -Don Slutz

Re: [PATCH v2 0/3] Xen VMware tools support

[Ian Campbell]

On Mon, 2014-09-08 at 09:18 -0400, Don Slutz wrote:
> On 09/08/14 09:03, Ian Campbell wrote:
> > On Mon, 2014-09-01 at 11:33 -0400, Don Slutz wrote:
> > [...]
> >> I picked this subset to start with because it only has changes in
> >> Xen.
> > [...]
> >>   docs/man/xl.cfg.pod.5                        |    6 +
> >>   tools/firmware/hvmloader/pci.c               |   10 +-
> >>   tools/libxc/xc_domain.c                      |  115 +++
> >>   tools/libxc/xc_domain_restore.c              |   14 +
> >>   tools/libxc/xc_domain_save.c                 |   11 +
> >>   tools/libxc/xenctrl.h                        |   24 +
> >>   tools/libxc/xg_save_restore.h                |    2 +
> >>   tools/libxl/libxl_create.c                   |    4 +-
> >>   tools/libxl/libxl_dm.c                       |   33 +-
> >>   tools/libxl/libxl_dom.c                      |    2 +
> >>   tools/libxl/libxl_types.idl                  |    1 +
> >>   tools/libxl/xl_cmdimpl.c                     |    2 +
> >>   tools/misc/xen-hvmctx.c                      |  229 ++++
> >>   tools/xentrace/formats                       |    8 +
> > Bah, I thought my review-load had just decreased! ;-)
> >
> > Ian.
> >
> 
> No need to look at v2, v3 has been sent.

Too late ;-)