[DOCS/PATCH v2] http://wiki.xen.org/wiki/X86_Paravirtualised_Memory_Management changes

[DOCS/PATCH v2] http://wiki.xen.org/wiki/X86_Paravirtualised_Memory_Management changes

[Konrad Rzeszutek Wilk]

This v2 of the patchset that I posted a month ago. The Wiki changes
are mentioned in the first (or zeroth?) patch. If folks are OK I put
that in the Wiki.

[PATCH to X86 PV MMU Wiki v2] Expand the bootup section to include ELF and memory layout

[Konrad Rzeszutek Wilk]

v2: Integrate Ian's comments.

Signed-of-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
diff --git a/intro.txt b/intro.txt
index 8a16f92..fc80ef6 100644
--- a/intro.txt
+++ b/intro.txt
@@ -339,6 +339,173 @@ into the page table base register but will not be explicitly pinned.
 The initial virtual and pseudo-physical layout of a new guest is
 described [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday here].
 
+
+When guest has started, the kernel image is read and the ELF (PT_NOTE) program is
+parsed.The hypervisor looks in the .note sections
+for the 'Xen' NULL terminated notes. The description fields contain the
+required information to find out: where the kernel expects its virtual base address,
+what type of hypervisor it can work with, certain features the kernel image
+can support, and the location of the hypercall page. There are two variants of this:
+
+; a). A “.note.Xen” section in ELF header conforming to the ELF PT_NOTE format.
+The PT_NOTE header is described in [http://www.netbsd.org/docs/kernel/elf-notes.html]
+and in [http://www.sco.com/developers/gabi/latest/contents.html]
+
+The type field (Name, Desc, Type) are of the ELF specification. The specific
+type values and the description is defined by Xen.
+
+This structure is a 4-byte aligned structure. First section is an numerical
+key (aligned to 4 bytes); followed by either a string or a numerical value
+(again, aligned to 4 bytes). The values can up to any length, if the key is
+assumed to a string. If it is a numerical value, it is a long
+(64-bit value - which means 8 bytes).
+
+For example this XEN_ELFNOTE_XEN_VERSION (5) with the value of "xen-3.0":
+
+  04000000 08000000 05000000 58656e00 78656e2d 332e3000 ........Xen.xen-3.0
+
+Using read-elf would print out as a eight-byte length value with type 5:
+
+  Xen                  0x00000008	Unknown note type: (0x00000005)
+
+; b). The legacy ASCIIZ string with all of the keys concatenated. Each key
+being a string and the equal sign with the value also being an string
+(numeric values are typed as hexadecimal strings). The delimiter is a comma.
+The key can be up to 32 characters and the value up to 128 characters.
+For example:
+
+	GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic
+
+
+The legacy should not be used as it has limited values that can
+be used and the specification is frozen.
+
+The parameters and its purpose are explained in
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,elfnote.h.html#incontents_elfnote here].
+
+And the XEN_ELF_FEATURES values are explained in
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,features.h.html#incontents_elfnote_features here].
+
+For example, if the ELF values were as so:
+
+{| border="1" cellpadding="2" cellspacing="0"
+ | Name of Xen ELF entry
+ | Contents
+ |-
+ | XEN_ELFNOTE_GUEST_OS (6)
+ | linux
+ |-
+ | XEN_ELFNOTE_GUEST_VERSION (7)
+ | 2.6
+ |-
+ | XEN_ELFNOTE_XEN_VERSION (5)
+ | xen-3.0
+ |-
+ | XEN_ELFNOTE_VIRT_BASE (3)
+ | 0xffffffff80000000
+ |-
+ | XEN_ELFNOTE_ENTRY (1)
+ | 0xffffffff81899200
+ |-
+ | XEN_ELFNOTE_HYPERCALL_PAGE (2)
+ | 0xffffffff81001000
+ |-
+ | XEN_ELFNOTE_FEATURES (10)
+ | !writable_page_tables|pae_pgdir_above_4gb
+ |-
+ | XEN_ELFNOTE_PAE_MODE (9)
+ | yes
+ |-
+ | XEN_ELFNOTE_LOADER (8)
+ | generic
+ |-
+ | XEN_ELFNOTE_SUSPEND_CANCEL (14)
+ | 1
+ |-
+ | XEN_ELFNOTE_HV_START_LOW
+ | 0xffff800000000000
+ |-
+ | XEN_ELFNOTE_PADDR_OFFSET
+ | 0
+ |}
+
+With that setup, the hypervisor constructs an initial page table that spans the
+region from virtual start address up (0xffffffff80000000) to the end of the
+p2m map.
+
+Using that ELF program header information, the hypervisor (or toolstack)
+constructs the domain with the appropiately located data. This ELF data
+is used to construct a guest which is laid out as enumerated in this
+header:
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday]
+
+NOTE: This is an example of a 64-bit guest and not part of the ABI.
+
+{| border="1" cellpadding="2" cellspacing="0"
+ | Page Frame (PFN)
+ | Virtual Address
+ | contents
+ |-
+ | 0x0
+ | 0xffffffff80000000
+ | location of [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_shared struct shared_info]. The 3.d entry (start_info_t) contains the machine address of this structure.
+ |-
+ | 0x1000
+ | 0xffffffff81000000
+ | location of the kernel
+ |-
+ | 0x1001
+ | 0xffffffff81001000
+ | location of the hypercall within the kernel
+ |-
+ | 0x1E3E
+ | 0xffffffff81e3e000
+ | ramdisk (NOTE: This is an example and the kernel size or ramdisk will differ)
+ |-
+ | 0xFC69
+ | 0xffffffff8fc69000
+ | (NOTE): This is an example and based on the size of the kernel and ramdisk and will differ). phys2mach (P2M) - an array of machine frame numbers. The total size of this array is dependent on the nr_pages in
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday struct start_info]
+and the architecture of the guest (each entry is four bytes
+under 32-bit kernels and eight bytes under 64-bit kernels).
+ |-
+ | 0xFCE9
+ | 0xffffffff8fce9000
+ | location of
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday start_info structure]
+ |-
+ | 0xFCEA
+ | 0xffffffff8fcea000
+ | location of
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,xs_wire.h.html#incontents_xenstore_struct XenStore structure].
+Also refer to http://xenbits.xen.org/docs/unstable/misc/xenstore.txt
+ |-
+ | 0xFCEB
+ | 0xffffffff8fceb000
+ | Depending on the type of the guest (initial domain or subsequent domains), it can either be the
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_dom0_console dom0_vga_console_info structure]
+or 
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,console.h.html XenConsole structure].
+The parameters defining this location are in the
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday start_info structure]
+ |-
+ | 0xFCEC
+ | 0xffffffff8fcfc000
+ | bootstrap page tables. These pagetables are loaded in the guest at startup
+and cover from 0x0 up to 0xFD6f (the bootstack).
+ |-
+ | 0xFD6F
+ | 0xffffffff8fd6f000
+ | bootstrap stack.
+ |}
+
+When the guest is launched, per
+[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday]
+explanation, the register %esi contains the virtual address to the
+start_info_t (0xffffffff8fce9000), the %cr3 points to the beginning of the
+bootstrap page-tables (0xffffffff8fcfc000), and the %esp points to the
+bootstrap stack (0xffffffff8fd6f000).
+
 = Virtual Address Space =
 
 Xen enforces certain restrictions on the virtual addresses which are

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH 1/7] docs: Document the ELF notes

[Konrad Rzeszutek Wilk]

Mark-up for inclusion of generated docs.

Acked-by: Ian Campbell <ian.campbel@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/elfnote.h |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/xen/include/public/elfnote.h b/xen/include/public/elfnote.h
index 42d76da..3c57a06 100644
--- a/xen/include/public/elfnote.h
+++ b/xen/include/public/elfnote.h
@@ -28,6 +28,8 @@
 #define __XEN_PUBLIC_ELFNOTE_H__
 
 /*
+ * `incontents 200 elfnotes ELF notes
+ *
  * The notes should live in a PT_NOTE segment and have "Xen" in the
  * name field.
  *
-- 
1.7.7.6

[PATCH 2/7] docs: Document the ELF_FEATURES entry

[Konrad Rzeszutek Wilk]

Mark-up for inclusion of generated docs.

Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/features.h |   14 ++++++++++++++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/xen/include/public/features.h b/xen/include/public/features.h
index b4533cc..bfaf2db 100644
--- a/xen/include/public/features.h
+++ b/xen/include/public/features.h
@@ -28,6 +28,20 @@
 #define __XEN_PUBLIC_FEATURES_H__
 
 /*
+ * `incontents 200 elfnotes_features XEN_ELFNOTE_FEATURES
+ *
+ * The list of all the features the guest supports. They are set by
+ * parsing the XEN_ELFNOTE_FEATURES and XEN_ELFNOTE_SUPPORTED_FEATURES
+ * string. The format is the  feature names (as given here without the
+ * "XENFEAT_" prefix) separated by '|' characters.
+ * If a feature is required for the kernel to function then the feature name
+ * must be preceded by a '!' character.
+ *
+ * Note that if XEN_ELFNOTE_SUPPORTED_FEATURES is used, then in the
+ * XENFEAT_dom0 MUST be set if the guest is to be booted as dom0,
+ */
+
+/*
  * If set, the guest does not need to write-protect its pagetables, and can
  * update them via direct writes.
  */
-- 
1.7.7.6

[PATCH 3/7] docs: Add some extra details to the ELF note.

[Konrad Rzeszutek Wilk]

Such as how the string values MUST be NULL terminated ASCII.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/elfnote.h |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/xen/include/public/elfnote.h b/xen/include/public/elfnote.h
index 3c57a06..6cb2ded 100644
--- a/xen/include/public/elfnote.h
+++ b/xen/include/public/elfnote.h
@@ -38,6 +38,9 @@
  *
  * LEGACY indicated the fields in the legacy __xen_guest string which
  * this a note type replaces.
+ *
+ * String values (for non-legacy) are NULL terminated ASCII, also known
+ * as ASCIZ type.
  */
 
 /*
@@ -160,6 +163,9 @@
 
 /*
  * Whether or not the guest supports cooperative suspend cancellation.
+ * This is a numeric value.
+ *
+ * Default is 0
  */
 #define XEN_ELFNOTE_SUSPEND_CANCEL 14
 
-- 
1.7.7.6

[PATCH 4/7] docs: Document the shared structure.

[Konrad Rzeszutek Wilk]

Mark-up for inclusion of generated docs.

Acked-by: Ian Campbell <ian.campbell@citrix.com>
Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/xen.h |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index 5593066..e1748d1 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -622,6 +622,7 @@ typedef struct vcpu_info vcpu_info_t;
 #endif
 
 /*
+ * `incontents 200 startofday_shared Start-of-day shared data structure
  * Xen/kernel shared data -- pointer provided in start_info.
  *
  * This structure is defined to be both smaller than a page, and the
-- 
1.7.7.6

[PATCH 5/7] docs: Document the dom0_vga_console_info structure.

[Konrad Rzeszutek Wilk]

Mark-up for inclusion of generated docs.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/xen.h |    9 ++++++++-
 1 files changed, 8 insertions(+), 1 deletions(-)

diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index e1748d1..5e9bbcb 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -779,7 +779,14 @@ struct xen_multiboot_mod_list
     /* Unused, must be zero */
     uint32_t pad;
 };
-
+/*
+ * `incontents 200 startofday_dom0_console Dom9_console
+ *
+ * The console structure in start_info.console.dom0
+ *
+ * This structure includes a variety of information required to
+ * have a working VGA/VESA console.
+ */
 typedef struct dom0_vga_console_info {
     uint8_t video_type; /* DOM0_VGA_CONSOLE_??? */
 #define XEN_VGATYPE_TEXT_MODE_3 0x03
-- 
1.7.7.6

[PATCH 6/7] docs: Document start_info changes in Xen 4.2.

[Konrad Rzeszutek Wilk]

The  25833:bb85bbccb1c9. "x86/32-on-64: adjust Dom0 initial page table layout"
fixes a bug in the reported value of pt_base versus where the page tables
actually start. This documents this in the start of the world header note.

This clarifies the implied understanding that the page table space is
pointed by pt_base. As in it is ".. implied that the range of page-tables
is the range [pt_base, pt_base + nr_pt_frames), whereas that that range
here indeed is [pt_base - 2, pt_base -2 + nr_pt_frames)" (Jan Beulich).

Also make it crystal clear that pt_base == %cr3.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/xen.h |   12 +++++++++++-
 1 files changed, 11 insertions(+), 1 deletions(-)

diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
index 5e9bbcb..099ad24 100644
--- a/xen/include/public/xen.h
+++ b/xen/include/public/xen.h
@@ -693,7 +693,7 @@ typedef struct shared_info shared_info_t;
  *      c. list of allocated page frames [mfn_list, nr_pages]
  *         (unless relocated due to XEN_ELFNOTE_INIT_P2M)
  *      d. start_info_t structure        [register ESI (x86)]
- *      e. bootstrap page tables         [pt_base, CR3 (x86)]
+ *      e. bootstrap page tables         [pt_base and CR3 (x86)]
  *      f. bootstrap stack               [register ESP (x86)]
  *  4. Bootstrap elements are packed together, but each is 4kB-aligned.
  *  5. The initial ram disk may be omitted.
@@ -705,6 +705,16 @@ typedef struct shared_info shared_info_t;
  *  8. There is guaranteed to be at least 512kB padding after the final
  *     bootstrap element. If necessary, the bootstrap virtual region is
  *     extended by an extra 4MB to ensure this.
+ *
+ * Note: Prior to 25833:bb85bbccb1c9. ("x86/32-on-64 adjust Dom0 initial page
+ * table layout") a bug caused the pt_base (3.e above) and cr3 to not point
+ * to the start of the guest page tables (it was offset by two pages).
+ * This only manifested itself on 32-on-64 dom0 kernels and not 32-on-64 domU
+ * or 64-bit kernels of any colour. The page tables for a 32-on-64 dom0 got
+ * allocated in the order: 'first L1','first L2', 'first L3', so the offset
+ * to the page table base is by two pages back. The initial domain if it is
+ * 32-bit and runs under a 64-bit hypervisor should _NOT_ use two of the
+ * pages preceding pt_base and mark them as reserved/unused.
  */
 
 #define MAX_GUEST_CMDLINE 1024
-- 
1.7.7.6

[PATCH 7/7] docs: Document the XenBus structure.

[Konrad Rzeszutek Wilk]

Mark-up for inclusion of generated docs.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 xen/include/public/io/xs_wire.h |    5 ++++-
 1 files changed, 4 insertions(+), 1 deletions(-)

diff --git a/xen/include/public/io/xs_wire.h b/xen/include/public/io/xs_wire.h
index 7e454c4..761864e 100644
--- a/xen/include/public/io/xs_wire.h
+++ b/xen/include/public/io/xs_wire.h
@@ -103,7 +103,10 @@ enum xs_watch_type
     XS_WATCH_TOKEN
 };
 
-/* Inter-domain shared memory communications. */
+/*
+ * `incontents 150 xenstore_struct XenStore wire protocol.
+ *
+ * Inter-domain shared memory communications. */
 #define XENSTORE_RING_SIZE 1024
 typedef uint32_t XENSTORE_RING_IDX;
 #define MASK_XENSTORE_IDX(idx) ((idx) & (XENSTORE_RING_SIZE-1))
-- 
1.7.7.6

Re: [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout

[Ian Jackson]

Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout"):
> v2: Integrate Ian's comments.

There are some formatting oddities in this:

> +
> +When guest has started, the kernel image is read and the ELF (PT_NOTE) program is

This line is too long for a .txt file; it shoudl be 70 or maybe 75.
This is a problem throughout the file.

> +parsed.The hypervisor looks in the .note sections

And here is a missing pspace.

> +can support, and the location of the hypercall page. There are two variants of this:
> +
> +; a). A “.note.Xen” section in ELF header conforming to the ELF PT_NOTE format.

What's the semicolon for ?

Your file has some kind of high-bit-set characters in it which I don't
think are desirable here ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout

[Konrad Rzeszutek Wilk]

On Tue, Feb 26, 2013 at 04:14:25PM +0000, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout"):
> > v2: Integrate Ian's comments.
> 
> There are some formatting oddities in this:
> 
> > +
> > +When guest has started, the kernel image is read and the ELF (PT_NOTE) program is
> 
> This line is too long for a .txt file; it shoudl be 70 or maybe 75.
> This is a problem throughout the file.

Huh? This goes on the Wiki so I am not sure if we need to worry about that.

> 
> > +parsed.The hypervisor looks in the .note sections
> 
> And here is a missing pspace.
> 
> > +can support, and the location of the hypercall page. There are two variants of this:
> > +
> > +; a). A “.note.Xen” section in ELF header conforming to the ELF PT_NOTE format.
> 
> What's the semicolon for ?

For Wiki.
> 
> Your file has some kind of high-bit-set characters in it which I don't
> think are desirable here ?

Not sure why they show up, but they are OK in the Wiki.
> 
> Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout

[Ian Jackson]

Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout"):
> On Tue, Feb 26, 2013 at 04:14:25PM +0000, Ian Jackson wrote:
> > This line is too long for a .txt file; it shoudl be 70 or maybe 75.
> > This is a problem throughout the file.
> 
> Huh? This goes on the Wiki so I am not sure if we need to worry about that.

Oh sorry, I'm confused.  I saw ".txt" in your patch and thought this
was a patch to a file in the Xen tree's docs.

Ian.

Re: [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout

[Konrad Rzeszutek Wilk]

On Tue, Feb 26, 2013 at 05:09:51PM +0000, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("Re: [Xen-devel] [PATCH to X86 PV MMU Wiki v2] Expand th e bootup section to include ELF and memory layout"):
> > On Tue, Feb 26, 2013 at 04:14:25PM +0000, Ian Jackson wrote:
> > > This line is too long for a .txt file; it shoudl be 70 or maybe 75.
> > > This is a problem throughout the file.
> > 
> > Huh? This goes on the Wiki so I am not sure if we need to worry about that.
> 
> Oh sorry, I'm confused.  I saw ".txt" in your patch and thought this
> was a patch to a file in the Xen tree's docs.

Sorry - I didn't know how else to do this. This "patch" to the Wiki
content was easier to make then putting in the changes in the Wiki and
then asking folks to look there.

> 
> Ian.

Re: [PATCH 5/7] docs: Document the dom0_vga_console_info structure.

[Ian Campbell]

On Mon, 2013-02-25 at 16:27 +0000, Konrad Rzeszutek Wilk wrote:
> Mark-up for inclusion of generated docs.
> 
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  xen/include/public/xen.h |    9 ++++++++-
>  1 files changed, 8 insertions(+), 1 deletions(-)
> 
> diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
> index e1748d1..5e9bbcb 100644
> --- a/xen/include/public/xen.h
> +++ b/xen/include/public/xen.h
> @@ -779,7 +779,14 @@ struct xen_multiboot_mod_list
>      /* Unused, must be zero */
>      uint32_t pad;
>  };
> -
> +/*
> + * `incontents 200 startofday_dom0_console Dom9_console

Dom9?

> + *
> + * The console structure in start_info.console.dom0
> + *
> + * This structure includes a variety of information required to
> + * have a working VGA/VESA console.
> + */
>  typedef struct dom0_vga_console_info {
>      uint8_t video_type; /* DOM0_VGA_CONSOLE_??? */
>  #define XEN_VGATYPE_TEXT_MODE_3 0x03

Re: [PATCH 6/7] docs: Document start_info changes in Xen 4.2.

[Ian Campbell]

On Mon, 2013-02-25 at 16:27 +0000, Konrad Rzeszutek Wilk wrote:
> The  25833:bb85bbccb1c9. "x86/32-on-64: adjust Dom0 initial page table layout"
> fixes a bug in the reported value of pt_base versus where the page tables
> actually start. This documents this in the start of the world header note.
> 
> This clarifies the implied understanding that the page table space is
> pointed by pt_base. As in it is ".. implied that the range of page-tables
> is the range [pt_base, pt_base + nr_pt_frames), whereas that that range
> here indeed is [pt_base - 2, pt_base -2 + nr_pt_frames)" (Jan Beulich).

Jan -- does the updated comment below correctly reflect this bug?

> 
> Also make it crystal clear that pt_base == %cr3.
> 
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  xen/include/public/xen.h |   12 +++++++++++-
>  1 files changed, 11 insertions(+), 1 deletions(-)
> 
> diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
> index 5e9bbcb..099ad24 100644
> --- a/xen/include/public/xen.h
> +++ b/xen/include/public/xen.h
> @@ -693,7 +693,7 @@ typedef struct shared_info shared_info_t;
>   *      c. list of allocated page frames [mfn_list, nr_pages]
>   *         (unless relocated due to XEN_ELFNOTE_INIT_P2M)
>   *      d. start_info_t structure        [register ESI (x86)]
> - *      e. bootstrap page tables         [pt_base, CR3 (x86)]
> + *      e. bootstrap page tables         [pt_base and CR3 (x86)]
>   *      f. bootstrap stack               [register ESP (x86)]
>   *  4. Bootstrap elements are packed together, but each is 4kB-aligned.
>   *  5. The initial ram disk may be omitted.
> @@ -705,6 +705,16 @@ typedef struct shared_info shared_info_t;
>   *  8. There is guaranteed to be at least 512kB padding after the final
>   *     bootstrap element. If necessary, the bootstrap virtual region is
>   *     extended by an extra 4MB to ensure this.
> + *
> + * Note: Prior to 25833:bb85bbccb1c9. ("x86/32-on-64 adjust Dom0 initial page
> + * table layout") a bug caused the pt_base (3.e above) and cr3 to not point
> + * to the start of the guest page tables (it was offset by two pages).
> + * This only manifested itself on 32-on-64 dom0 kernels and not 32-on-64 domU
> + * or 64-bit kernels of any colour. The page tables for a 32-on-64 dom0 got
> + * allocated in the order: 'first L1','first L2', 'first L3', so the offset
> + * to the page table base is by two pages back. The initial domain if it is
> + * 32-bit and runs under a 64-bit hypervisor should _NOT_ use two of the
> + * pages preceding pt_base and mark them as reserved/unused.
>   */
>  
>  #define MAX_GUEST_CMDLINE 1024

Re: [PATCH to X86 PV MMU Wiki v2] Expand the bootup section to include ELF and memory layout

[Ian Campbell]

On Mon, 2013-02-25 at 16:27 +0000, Konrad Rzeszutek Wilk wrote:
> v2: Integrate Ian's comments.
> 
> Signed-of-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> diff --git a/intro.txt b/intro.txt
> index 8a16f92..fc80ef6 100644
> --- a/intro.txt
> +++ b/intro.txt
> @@ -339,6 +339,173 @@ into the page table base register but will not be explicitly pinned.
>  The initial virtual and pseudo-physical layout of a new guest is
>  described [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday here].
>  
> +
> +When guest has started, the kernel image is read and the ELF (PT_NOTE) program is

             ^ ITYM "is"? Since this happens before the guest "has"
started.

> +parsed.The hypervisor looks in the .note sections
         ^ space

> +for the 'Xen' NULL terminated notes.

NULL terminated in what sense? I don't think ELF notes are inherently
NULL terminated, there are various size fields IIRC for e.g. the dsc and
name.

>  The description fields contain the
> +required information to find out: where the kernel expects its virtual base address,
> +what type of hypervisor it can work with,

What do you mean by this? It's all Xen specific I think?

>  certain features the kernel image
> +can support, and the location of the hypercall page. There are two variants of this:
> +
> +; a). A “.note.Xen” section in ELF header conforming to the ELF PT_NOTE format.
> +The PT_NOTE header is described in [http://www.netbsd.org/docs/kernel/elf-notes.html]
> +and in [http://www.sco.com/developers/gabi/latest/contents.html]
> +
> +The type field (Name, Desc, Type) are of the ELF specification. The specific
> +type values and the description is defined by Xen.
> +
> +This structure is a 4-byte aligned structure. First section is an numerical
> +key (aligned to 4 bytes); followed by either a string or a numerical value
> +(again, aligned to 4 bytes). The values can up to any length, if the key is
                                              ^ be

> +assumed to a string. If it is a numerical value, it is a long
> +(64-bit value - which means 8 bytes).

long != 64-bits (not necessarily at least). I'd just say numerical == 64
bit. I think the Endianess is part of the ELF class, so no need to
mention it? (which you haven't, just wondering aloud)

> +
> +For example this XEN_ELFNOTE_XEN_VERSION (5) with the value of "xen-3.0":
> +
> +  04000000 08000000 05000000 58656e00 78656e2d 332e3000 ........Xen.xen-3.0
> +
> +Using read-elf would print out as a eight-byte length value with type 5:
> +
> +  Xen                  0x00000008	Unknown note type: (0x00000005)

TBH I'm not sure how valuable the above description of what an ELF note
looks like and the examples etc is. Anyone who is implementing anything
should refer to the ELF spec and not rely on anything written here as
being gospel.

> +
> +; b). The legacy ASCIIZ string with all of the keys concatenated. Each key
> +being a string and the equal sign with the value also being an string
> +(numeric values are typed as hexadecimal strings). The delimiter is a comma.
> +The key can be up to 32 characters and the value up to 128 characters.
> +For example:
> +
> +	GUEST_OS=Mini-OS,XEN_VER=xen-3.0,VIRT_BASE=0x0,ELF_PADDR_OFFSET=0x0,HYPERCALL_PAGE=0x2,LOADER=generic
> +
> +
> +The legacy should not be used as it has limited values that can
             ^ needs a word at this point, string or format ?

> +be used and the specification is frozen.
> +
> +The parameters and its purpose are explained in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,elfnote.h.html#incontents_elfnote here].
> +
> +And the XEN_ELF_FEATURES values are explained in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,features.h.html#incontents_elfnote_features here].
> +
> +For example, if the ELF values were as so:
> +
> +{| border="1" cellpadding="2" cellspacing="0"
> + | Name of Xen ELF entry
> + | Contents
> + |-
> + | XEN_ELFNOTE_GUEST_OS (6)
> + | linux
> + |-
> + | XEN_ELFNOTE_GUEST_VERSION (7)
> + | 2.6
> + |-
> + | XEN_ELFNOTE_XEN_VERSION (5)
> + | xen-3.0
> + |-
> + | XEN_ELFNOTE_VIRT_BASE (3)
> + | 0xffffffff80000000
> + |-
> + | XEN_ELFNOTE_ENTRY (1)
> + | 0xffffffff81899200
> + |-
> + | XEN_ELFNOTE_HYPERCALL_PAGE (2)
> + | 0xffffffff81001000
> + |-
> + | XEN_ELFNOTE_FEATURES (10)
> + | !writable_page_tables|pae_pgdir_above_4gb
> + |-
> + | XEN_ELFNOTE_PAE_MODE (9)
> + | yes
> + |-
> + | XEN_ELFNOTE_LOADER (8)
> + | generic
> + |-
> + | XEN_ELFNOTE_SUSPEND_CANCEL (14)
> + | 1
> + |-
> + | XEN_ELFNOTE_HV_START_LOW
> + | 0xffff800000000000
> + |-
> + | XEN_ELFNOTE_PADDR_OFFSET
> + | 0
> + |}
> +
> +With that setup, the hypervisor constructs an initial page table that spans the
> +region from virtual start address up (0xffffffff80000000) to the end of the

(up after brackets for flow)         `---------------------^ 


> +p2m map.
> +
> +Using that ELF program header information, the hypervisor (or toolstack)
> +constructs the domain with the appropiately located data. This ELF data

                                  appropriately

> +is used to construct a guest which is laid out as enumerated in this
> +header:
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday]
> +
> +NOTE: This is an example of a 64-bit guest and not part of the ABI.
> +
> +{| border="1" cellpadding="2" cellspacing="0"
> + | Page Frame (PFN)
> + | Virtual Address
> + | contents
> + |-
> + | 0x0
> + | 0xffffffff80000000
> + | location of [http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_shared struct shared_info]. The 3.d entry (start_info_t) contains the machine address of this structure.
> + |-
> + | 0x1000
> + | 0xffffffff81000000
> + | location of the kernel
> + |-
> + | 0x1001
> + | 0xffffffff81001000
> + | location of the hypercall within the kernel
> + |-
> + | 0x1E3E
> + | 0xffffffff81e3e000
> + | ramdisk (NOTE: This is an example and the kernel size or ramdisk will differ)
> + |-
> + | 0xFC69
> + | 0xffffffff8fc69000
> + | (NOTE): This is an example and based on the size of the kernel and ramdisk and will differ). phys2mach (P2M) - an array of machine frame numbers. The total size of this array is dependent on the nr_pages in
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday struct start_info]
> +and the architecture of the guest (each entry is four bytes
> +under 32-bit kernels and eight bytes under 64-bit kernels).
> + |-
> + | 0xFCE9
> + | 0xffffffff8fce9000
> + | location of
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday start_info structure]
> + |-
> + | 0xFCEA
> + | 0xffffffff8fcea000
> + | location of
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,xs_wire.h.html#incontents_xenstore_struct XenStore structure].
> +Also refer to http://xenbits.xen.org/docs/unstable/misc/xenstore.txt
> + |-
> + | 0xFCEB
> + | 0xffffffff8fceb000
> + | Depending on the type of the guest (initial domain or subsequent domains), it can either be the
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday_dom0_console dom0_vga_console_info structure]
> +or 
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,io,console.h.html XenConsole structure].
> +The parameters defining this location are in the
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday start_info structure]
> + |-
> + | 0xFCEC
> + | 0xffffffff8fcfc000
> + | bootstrap page tables. These pagetables are loaded in the guest at startup
> +and cover from 0x0 up to 0xFD6f (the bootstack).
> + |-
> + | 0xFD6F
> + | 0xffffffff8fd6f000
> + | bootstrap stack.
> + |}
> +
> +When the guest is launched, per
> +[http://xenbits.xen.org/docs/unstable/hypercall/include,public,xen.h.html#incontents_startofday]
> +explanation, the register %esi contains the virtual address to the
> +start_info_t (0xffffffff8fce9000), the %cr3 points to the beginning of the
> +bootstrap page-tables (0xffffffff8fcfc000), and the %esp points to the
> +bootstrap stack (0xffffffff8fd6f000).
> +
>  = Virtual Address Space =
>  
>  Xen enforces certain restrictions on the virtual addresses which are
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [DOCS/PATCH v2] http://wiki.xen.org/wiki/X86_Paravirtualised_Memory_Management changes

[Ian Campbell]

On Mon, 2013-02-25 at 16:27 +0000, Konrad Rzeszutek Wilk wrote:
> This v2 of the patchset that I posted a month ago. The Wiki changes
> are mentioned in the first (or zeroth?) patch. If folks are OK I put
> that in the Wiki.

I had a few very minor comments on the patches to Xen. Docs patches to
xen/include/public tend to fall betweens peoples remits. I think that
with the nits addressed I'd just go ahead an shovel them in, unless
someone objects. (NB I'm AFK from 19 March until 10 April, so "shovel"
is within those constraints)

I had a few more substantial comments to the wiki thing, but nothing
huge.

Ian.

Re: [PATCH 5/7] docs: Document the dom0_vga_console_info structure.

[Konrad Rzeszutek Wilk]

On Tue, Mar 12, 2013 at 04:06:07PM +0000, Ian Campbell wrote:
> On Mon, 2013-02-25 at 16:27 +0000, Konrad Rzeszutek Wilk wrote:
> > Mark-up for inclusion of generated docs.
> > 
> > Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > ---
> >  xen/include/public/xen.h |    9 ++++++++-
> >  1 files changed, 8 insertions(+), 1 deletions(-)
> > 
> > diff --git a/xen/include/public/xen.h b/xen/include/public/xen.h
> > index e1748d1..5e9bbcb 100644
> > --- a/xen/include/public/xen.h
> > +++ b/xen/include/public/xen.h
> > @@ -779,7 +779,14 @@ struct xen_multiboot_mod_list
> >      /* Unused, must be zero */
> >      uint32_t pad;
> >  };
> > -
> > +/*
> > + * `incontents 200 startofday_dom0_console Dom9_console
> 
> Dom9?

Well spotted. Fixed.
> 
> > + *
> > + * The console structure in start_info.console.dom0
> > + *
> > + * This structure includes a variety of information required to
> > + * have a working VGA/VESA console.
> > + */
> >  typedef struct dom0_vga_console_info {
> >      uint8_t video_type; /* DOM0_VGA_CONSOLE_??? */
> >  #define XEN_VGATYPE_TEXT_MODE_3 0x03
> 
>