[RFC for-4.6 0/2] In-tree feature documentation

[RFC for-4.6 0/2] In-tree feature documentation

[Andrew Cooper]

An issue which Xen has is an uncertain support statement for features.
Given the success seen with docs/misc/xen-command-line.markdown, and in
particular keeping it up to date, introduce a similar system for
features.

Patch 1 introduces a proposed template (and a makefile tweak to include
the new docs/features subdirectory), while patch 2 is a feature document
covering the topic of migration.

This is tagged RFC as I expect people to have different views as to what
is useful to include.  I would particilarly appreciate feedback on the
template before it starts getting used widely.

Lars: Does this look like a reasonable counterpart to your formal
support statement document?

Jim: Per your request at the summit for new information, is patch 2
suitable?

Andrew Cooper (2):
  docs: Template for feature documents
  docs: Migration feature document

 docs/Makefile                  |    2 +-
 docs/features/migration.pandoc |   90 ++++++++++++++++++++++++++++++++++++++++
 docs/features/template.pandoc  |   55 ++++++++++++++++++++++++
 3 files changed, 146 insertions(+), 1 deletion(-)
 create mode 100644 docs/features/migration.pandoc
 create mode 100644 docs/features/template.pandoc

-- 
1.7.10.4

[PATCH for-4.6 1/2] docs: Template for feature documents

[Andrew Cooper]

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 docs/Makefile                 |    2 +-
 docs/features/template.pandoc |   55 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 56 insertions(+), 1 deletion(-)
 create mode 100644 docs/features/template.pandoc

diff --git a/docs/Makefile b/docs/Makefile
index 272292c..5d620e5 100644
--- a/docs/Makefile
+++ b/docs/Makefile
@@ -16,7 +16,7 @@ MARKDOWNSRC-y := $(sort $(shell find misc -name '*.markdown' -print))
 
 TXTSRC-y := $(sort $(shell find misc -name '*.txt' -print))
 
-PANDOCSRC-y := $(sort $(shell find specs -name '*.pandoc' -print))
+PANDOCSRC-y := $(sort $(shell find features/ misc/ specs/ -name '*.pandoc' -print))
 
 # Documentation targets
 DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(MAN1SRC-y))
diff --git a/docs/features/template.pandoc b/docs/features/template.pandoc
new file mode 100644
index 0000000..d883b82
--- /dev/null
+++ b/docs/features/template.pandoc
@@ -0,0 +1,55 @@
+% Template for feature documents
+
+\clearpage
+
+This is a suggested template for formatting of a Xen feature document in tree.
+
+The purpose of this document is to provide a concrete support statement for the
+feature (indicating its security status), as well as brief user and technical
+documentation.
+
+# Basics
+
+A table with an overview of the support status and applicability.
+
+---------------- ----------------------------------------------------
+         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
+
+Architecture(s): e.g. x86, arm
+
+   Component(s): e.g. Hypervisor, toolstack, guest
+
+       Hardware: _where applicable_
+---------------- ----------------------------------------------------
+
+# Overview
+
+A short description the feature, similar to an abstract for a
+paper/presentation.
+
+# User information
+
+Information for a user attempting to use the feature.  Should include how to
+enable the feature (is it enabled by default? If not, how to turn it on?), and
+how to interact with the feature (typically via `xl`).
+
+# Limitations
+
+Information concerning incompatibilities with other features or hardware
+combinations.
+
+# Technical information
+
+Information for a developer or power user.  Should include where to look
+in-tree for detailed documents and code.
+
+# Areas for improvement
+
+List of enhancements which could be undertaken, e.g. to improve the feature
+itself, or improve interaction with other features.
+
+# Known issues
+
+List of known issues or bugs.  For tech preview or experimental features, this
+section must contain the list of items needing fixing for its status to be
+upgraded.
-- 
1.7.10.4

[PATCH for-4.6 2/2] docs: Migration feature document

[Andrew Cooper]

Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
 docs/features/migration.pandoc |   90 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 90 insertions(+)
 create mode 100644 docs/features/migration.pandoc

diff --git a/docs/features/migration.pandoc b/docs/features/migration.pandoc
new file mode 100644
index 0000000..e0422f9
--- /dev/null
+++ b/docs/features/migration.pandoc
@@ -0,0 +1,90 @@
+% Migration
+
+\clearpage
+
+# Basics
+--------------- -------------
+        Status: **Supported**
+
+  Architecture: x86
+
+     Component: Toolstack
+--------------- -------------
+
+# Overview
+
+Migration is a mechanism to move a virtual machine while the VM is running.
+Live migration moves a running virtual machine between two physical servers,
+but the same mechanism can be used for non-live migrate (pause and copy) and
+suspend/resume from disk.
+
+# User details
+
+No hardware requirements, although hypervisor logdirty support is required for
+live migration.
+
+From the command line, `xl migrate/save/restore` are the top level
+interactions.  e.g.
+
+    xl create my-vm.cfg
+    xl migrate my-vm localhost
+
+or
+
+    xl create my-vm.cfg
+    xl save my-vm /path/to/save/file
+    xl restore /path/to/save/file
+
+Xen 4.6 sees the instruction of Migration v2.  There is no change for people
+using `xl`, although the `libxl` API has had an extension.
+
+# Technical details
+
+Migration is of formed of several layers.  `libxc` is responsible for the
+contents of the VM (ram, vcpus, etc) and the live migration loop, while
+`libxl` is responsible for items such as emulator state.
+
+The format of the migration v2 stream is specified in two documents, and is
+architecture neutral.  Compatibility with legacy streams is maintained via the
+`convert-legacy-stream` script which transforms a legacy stream into a
+migration v2 stream.
+
+* Documents
+    * `docs/specs/libxc-migration-stream.pandoc`
+    * `docs/specs/libxl-migration-stream.pandoc`
+* `libxc`
+    * `tools/libxc/xc_sr_*.[hc]`
+* `libxl`
+    * `tools/libxl/libxl_stream_{read,write}.c`
+* Scripts
+    * `tools/python/xen/migration/*.py`
+    * `tools/python/scripts/convert-legacy-stream`
+    * `tools/python/scripts/verify-stream-v2`
+
+Users of the `libxl` API have a new parameter `stream_version` in
+`domain_restore_params` which is used to distinguish between legacy and v2
+migration streams, and hence whether legacy conversion is required.
+
+# Limitations
+
+Hypervisor logdirty support is incompatible with hardware passthrough, as
+IOMMU faults cannot be used to track writes.
+
+While not a bug in migration specifically, VMs are very sensitive to changes
+in cpuid information, and cpuid levelling support currently has its issues.
+Extreme care should be taken when migrating VMs between non-identical CPUs
+until the cpuid levelling improvements are complete.
+
+# Areas for improvement
+
+* Arm support
+* Linear P2M support for x86 PV
+* Live looping parameters
+
+# Known issues
+
+* x86 HVM guest physmap operations (not reflected in logdirty bitmap)
+* x86 HVM with PoD pages (attempts to map cause PoD allocations)
+* x86 HVM with nested-virt (no relevant information included in the stream)
+* x86 PV ballooning (P2M marked dirty, target frame not marked)
+* x86 PV P2M structure changes (not noticed, stale mappings used)
-- 
1.7.10.4

Re: [PATCH for-4.6 1/2] docs: Template for feature documents

[Juergen Gross]

On 08/24/2015 07:37 PM, Andrew Cooper wrote:
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>   docs/Makefile                 |    2 +-
>   docs/features/template.pandoc |   55 +++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 56 insertions(+), 1 deletion(-)
>   create mode 100644 docs/features/template.pandoc
>
> diff --git a/docs/Makefile b/docs/Makefile
> index 272292c..5d620e5 100644
> --- a/docs/Makefile
> +++ b/docs/Makefile
> @@ -16,7 +16,7 @@ MARKDOWNSRC-y := $(sort $(shell find misc -name '*.markdown' -print))
>
>   TXTSRC-y := $(sort $(shell find misc -name '*.txt' -print))
>
> -PANDOCSRC-y := $(sort $(shell find specs -name '*.pandoc' -print))
> +PANDOCSRC-y := $(sort $(shell find features/ misc/ specs/ -name '*.pandoc' -print))
>
>   # Documentation targets
>   DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(MAN1SRC-y))
> diff --git a/docs/features/template.pandoc b/docs/features/template.pandoc
> new file mode 100644
> index 0000000..d883b82
> --- /dev/null
> +++ b/docs/features/template.pandoc
> @@ -0,0 +1,55 @@
> +% Template for feature documents
> +
> +\clearpage
> +
> +This is a suggested template for formatting of a Xen feature document in tree.
> +
> +The purpose of this document is to provide a concrete support statement for the
> +feature (indicating its security status), as well as brief user and technical
> +documentation.
> +
> +# Basics
> +
> +A table with an overview of the support status and applicability.
> +
> +---------------- ----------------------------------------------------
> +         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
> +
> +Architecture(s): e.g. x86, arm
> +
> +   Component(s): e.g. Hypervisor, toolstack, guest
> +
> +       Hardware: _where applicable_
> +---------------- ----------------------------------------------------

What about adding some information when the feature was introduced or
some other historical stuff? Something like:

Experimental in Xen 4.1
Supported in Xen 4.3
xl syntax changed in Xen 4.4


Juergen

> +
> +# Overview
> +
> +A short description the feature, similar to an abstract for a
> +paper/presentation.
> +
> +# User information
> +
> +Information for a user attempting to use the feature.  Should include how to
> +enable the feature (is it enabled by default? If not, how to turn it on?), and
> +how to interact with the feature (typically via `xl`).
> +
> +# Limitations
> +
> +Information concerning incompatibilities with other features or hardware
> +combinations.
> +
> +# Technical information
> +
> +Information for a developer or power user.  Should include where to look
> +in-tree for detailed documents and code.
> +
> +# Areas for improvement
> +
> +List of enhancements which could be undertaken, e.g. to improve the feature
> +itself, or improve interaction with other features.
> +
> +# Known issues
> +
> +List of known issues or bugs.  For tech preview or experimental features, this
> +section must contain the list of items needing fixing for its status to be
> +upgraded.
>

Re: [PATCH for-4.6 1/2] docs: Template for feature documents

[Andrew Cooper]

On 24/08/2015 20:27, Juergen Gross wrote:
> On 08/24/2015 07:37 PM, Andrew Cooper wrote:
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>>   docs/Makefile                 |    2 +-
>>   docs/features/template.pandoc |   55
>> +++++++++++++++++++++++++++++++++++++++++
>>   2 files changed, 56 insertions(+), 1 deletion(-)
>>   create mode 100644 docs/features/template.pandoc
>>
>> diff --git a/docs/Makefile b/docs/Makefile
>> index 272292c..5d620e5 100644
>> --- a/docs/Makefile
>> +++ b/docs/Makefile
>> @@ -16,7 +16,7 @@ MARKDOWNSRC-y := $(sort $(shell find misc -name
>> '*.markdown' -print))
>>
>>   TXTSRC-y := $(sort $(shell find misc -name '*.txt' -print))
>>
>> -PANDOCSRC-y := $(sort $(shell find specs -name '*.pandoc' -print))
>> +PANDOCSRC-y := $(sort $(shell find features/ misc/ specs/ -name
>> '*.pandoc' -print))
>>
>>   # Documentation targets
>>   DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(MAN1SRC-y))
>> diff --git a/docs/features/template.pandoc
>> b/docs/features/template.pandoc
>> new file mode 100644
>> index 0000000..d883b82
>> --- /dev/null
>> +++ b/docs/features/template.pandoc
>> @@ -0,0 +1,55 @@
>> +% Template for feature documents
>> +
>> +\clearpage
>> +
>> +This is a suggested template for formatting of a Xen feature
>> document in tree.
>> +
>> +The purpose of this document is to provide a concrete support
>> statement for the
>> +feature (indicating its security status), as well as brief user and
>> technical
>> +documentation.
>> +
>> +# Basics
>> +
>> +A table with an overview of the support status and applicability.
>> +
>> +---------------- ----------------------------------------------------
>> +         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
>> +
>> +Architecture(s): e.g. x86, arm
>> +
>> +   Component(s): e.g. Hypervisor, toolstack, guest
>> +
>> +       Hardware: _where applicable_
>> +---------------- ----------------------------------------------------
>
> What about adding some information when the feature was introduced or
> some other historical stuff? Something like:
>
> Experimental in Xen 4.1
> Supported in Xen 4.3
> xl syntax changed in Xen 4.4
>

In the longterm, I would expect that information to be visible via `git
log`.

Having said that, it probably is useful to have a summary of history
available in the written document.

How about a #History section at the bottom?  That can at least include
"document written" as a starting point and subsequent major changes in
short form.

~Andrew

Re: [PATCH for-4.6 1/2] docs: Template for feature documents

[Juergen Gross]

On 08/25/2015 12:52 AM, Andrew Cooper wrote:
> On 24/08/2015 20:27, Juergen Gross wrote:
>> On 08/24/2015 07:37 PM, Andrew Cooper wrote:
>>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>>> ---
>>>    docs/Makefile                 |    2 +-
>>>    docs/features/template.pandoc |   55
>>> +++++++++++++++++++++++++++++++++++++++++
>>>    2 files changed, 56 insertions(+), 1 deletion(-)
>>>    create mode 100644 docs/features/template.pandoc
>>>
>>> diff --git a/docs/Makefile b/docs/Makefile
>>> index 272292c..5d620e5 100644
>>> --- a/docs/Makefile
>>> +++ b/docs/Makefile
>>> @@ -16,7 +16,7 @@ MARKDOWNSRC-y := $(sort $(shell find misc -name
>>> '*.markdown' -print))
>>>
>>>    TXTSRC-y := $(sort $(shell find misc -name '*.txt' -print))
>>>
>>> -PANDOCSRC-y := $(sort $(shell find specs -name '*.pandoc' -print))
>>> +PANDOCSRC-y := $(sort $(shell find features/ misc/ specs/ -name
>>> '*.pandoc' -print))
>>>
>>>    # Documentation targets
>>>    DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(MAN1SRC-y))
>>> diff --git a/docs/features/template.pandoc
>>> b/docs/features/template.pandoc
>>> new file mode 100644
>>> index 0000000..d883b82
>>> --- /dev/null
>>> +++ b/docs/features/template.pandoc
>>> @@ -0,0 +1,55 @@
>>> +% Template for feature documents
>>> +
>>> +\clearpage
>>> +
>>> +This is a suggested template for formatting of a Xen feature
>>> document in tree.
>>> +
>>> +The purpose of this document is to provide a concrete support
>>> statement for the
>>> +feature (indicating its security status), as well as brief user and
>>> technical
>>> +documentation.
>>> +
>>> +# Basics
>>> +
>>> +A table with an overview of the support status and applicability.
>>> +
>>> +---------------- ----------------------------------------------------
>>> +         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
>>> +
>>> +Architecture(s): e.g. x86, arm
>>> +
>>> +   Component(s): e.g. Hypervisor, toolstack, guest
>>> +
>>> +       Hardware: _where applicable_
>>> +---------------- ----------------------------------------------------
>>
>> What about adding some information when the feature was introduced or
>> some other historical stuff? Something like:
>>
>> Experimental in Xen 4.1
>> Supported in Xen 4.3
>> xl syntax changed in Xen 4.4
>>
>
> In the longterm, I would expect that information to be visible via `git
> log`.
>
> Having said that, it probably is useful to have a summary of history
> available in the written document.
>
> How about a #History section at the bottom?  That can at least include
> "document written" as a starting point and subsequent major changes in
> short form.

Sure, I'm fine with this.

Juergen

Re: [PATCH for-4.6 1/2] docs: Template for feature documents

[Wei Liu]

On Mon, Aug 24, 2015 at 11:52:34PM +0100, Andrew Cooper wrote:
> On 24/08/2015 20:27, Juergen Gross wrote:
> > On 08/24/2015 07:37 PM, Andrew Cooper wrote:
> >> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> >> ---
> >>   docs/Makefile                 |    2 +-
> >>   docs/features/template.pandoc |   55
> >> +++++++++++++++++++++++++++++++++++++++++
> >>   2 files changed, 56 insertions(+), 1 deletion(-)
> >>   create mode 100644 docs/features/template.pandoc
> >>
> >> diff --git a/docs/Makefile b/docs/Makefile
> >> index 272292c..5d620e5 100644
> >> --- a/docs/Makefile
> >> +++ b/docs/Makefile
> >> @@ -16,7 +16,7 @@ MARKDOWNSRC-y := $(sort $(shell find misc -name
> >> '*.markdown' -print))
> >>
> >>   TXTSRC-y := $(sort $(shell find misc -name '*.txt' -print))
> >>
> >> -PANDOCSRC-y := $(sort $(shell find specs -name '*.pandoc' -print))
> >> +PANDOCSRC-y := $(sort $(shell find features/ misc/ specs/ -name
> >> '*.pandoc' -print))
> >>
> >>   # Documentation targets
> >>   DOC_MAN1 := $(patsubst man/%.pod.1,man1/%.1,$(MAN1SRC-y))
> >> diff --git a/docs/features/template.pandoc
> >> b/docs/features/template.pandoc
> >> new file mode 100644
> >> index 0000000..d883b82
> >> --- /dev/null
> >> +++ b/docs/features/template.pandoc
> >> @@ -0,0 +1,55 @@
> >> +% Template for feature documents

% Revision X

> >> +
> >> +\clearpage
> >> +
> >> +This is a suggested template for formatting of a Xen feature
> >> document in tree.
> >> +
> >> +The purpose of this document is to provide a concrete support
> >> statement for the
> >> +feature (indicating its security status), as well as brief user and
> >> technical
> >> +documentation.
> >> +
> >> +# Basics
> >> +
> >> +A table with an overview of the support status and applicability.
> >> +
> >> +---------------- ----------------------------------------------------
> >> +         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
> >> +
> >> +Architecture(s): e.g. x86, arm
> >> +
> >> +   Component(s): e.g. Hypervisor, toolstack, guest
> >> +
> >> +       Hardware: _where applicable_
> >> +---------------- ----------------------------------------------------
> >
> > What about adding some information when the feature was introduced or
> > some other historical stuff? Something like:
> >
> > Experimental in Xen 4.1
> > Supported in Xen 4.3
> > xl syntax changed in Xen 4.4
> >
> 
> In the longterm, I would expect that information to be visible via `git
> log`.
> 
> Having said that, it probably is useful to have a summary of history
> available in the written document.
> 
> How about a #History section at the bottom?  That can at least include
> "document written" as a starting point and subsequent major changes in
> short form.

Or use a table for revision history at the beginning? I personally think
using a table is clearer than several paragraphs.

Don't want to bikeshed how it should look like. We can always change the
template if necessary.

Wei.

> 
> ~Andrew
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH for-4.6 2/2] docs: Migration feature document

[Wei Liu]

On Mon, Aug 24, 2015 at 06:37:02PM +0100, Andrew Cooper wrote:
> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
> ---
>  docs/features/migration.pandoc |   90 ++++++++++++++++++++++++++++++++++++++++
>  1 file changed, 90 insertions(+)
>  create mode 100644 docs/features/migration.pandoc
> 
> diff --git a/docs/features/migration.pandoc b/docs/features/migration.pandoc
> new file mode 100644
> index 0000000..e0422f9
> --- /dev/null
> +++ b/docs/features/migration.pandoc
> @@ -0,0 +1,90 @@
> +% Migration

% Revision 1

> +
> +\clearpage
> +
> +# Basics
> +--------------- -------------
> +        Status: **Supported**
> +
> +  Architecture: x86
> +
> +     Component: Toolstack
> +--------------- -------------
> +
> +# Overview
> +
> +Migration is a mechanism to move a virtual machine while the VM is running.
> +Live migration moves a running virtual machine between two physical servers,
> +but the same mechanism can be used for non-live migrate (pause and copy) and
> +suspend/resume from disk.
> +
> +# User details
> +
> +No hardware requirements, although hypervisor logdirty support is required for
> +live migration.
> +
> +From the command line, `xl migrate/save/restore` are the top level
> +interactions.  e.g.
> +
> +    xl create my-vm.cfg
> +    xl migrate my-vm localhost
> +
> +or
> +
> +    xl create my-vm.cfg
> +    xl save my-vm /path/to/save/file
> +    xl restore /path/to/save/file
> +
> +Xen 4.6 sees the instruction of Migration v2.  There is no change for people
> +using `xl`, although the `libxl` API has had an extension.
> +
> +# Technical details
> +
> +Migration is of formed of several layers.  `libxc` is responsible for the

Extraneous "of".

> +contents of the VM (ram, vcpus, etc) and the live migration loop, while
> +`libxl` is responsible for items such as emulator state.
> +
> +The format of the migration v2 stream is specified in two documents, and is
> +architecture neutral.  Compatibility with legacy streams is maintained via the
> +`convert-legacy-stream` script which transforms a legacy stream into a
> +migration v2 stream.
> +
> +* Documents
> +    * `docs/specs/libxc-migration-stream.pandoc`
> +    * `docs/specs/libxl-migration-stream.pandoc`
> +* `libxc`
> +    * `tools/libxc/xc_sr_*.[hc]`
> +* `libxl`
> +    * `tools/libxl/libxl_stream_{read,write}.c`
> +* Scripts
> +    * `tools/python/xen/migration/*.py`
> +    * `tools/python/scripts/convert-legacy-stream`
> +    * `tools/python/scripts/verify-stream-v2`
> +
> +Users of the `libxl` API have a new parameter `stream_version` in
> +`domain_restore_params` which is used to distinguish between legacy and v2
> +migration streams, and hence whether legacy conversion is required.
> +
> +# Limitations
> +
> +Hypervisor logdirty support is incompatible with hardware passthrough, as
> +IOMMU faults cannot be used to track writes.
> +
> +While not a bug in migration specifically, VMs are very sensitive to changes
> +in cpuid information, and cpuid levelling support currently has its issues.
> +Extreme care should be taken when migrating VMs between non-identical CPUs
> +until the cpuid levelling improvements are complete.
> +
> +# Areas for improvement
> +
> +* Arm support
> +* Linear P2M support for x86 PV
> +* Live looping parameters
> +
> +# Known issues
> +
> +* x86 HVM guest physmap operations (not reflected in logdirty bitmap)
> +* x86 HVM with PoD pages (attempts to map cause PoD allocations)
> +* x86 HVM with nested-virt (no relevant information included in the stream)
> +* x86 PV ballooning (P2M marked dirty, target frame not marked)
> +* x86 PV P2M structure changes (not noticed, stale mappings used)

TBH I think "Areas for improvement" and "Known issues" sections are very
cryptic to normal users, but I don't have any suggestion to make them
better.

In any case, any document is better than no documents. I'm all for this
doc going in as soon as possible.

Wei.

> -- 
> 1.7.10.4
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: [PATCH for-4.6 2/2] docs: Migration feature document

[Andrew Cooper]

On 25/08/15 09:49, Wei Liu wrote:
> On Mon, Aug 24, 2015 at 06:37:02PM +0100, Andrew Cooper wrote:
>> Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com>
>> ---
>>  docs/features/migration.pandoc |   90 ++++++++++++++++++++++++++++++++++++++++
>>  1 file changed, 90 insertions(+)
>>  create mode 100644 docs/features/migration.pandoc
>>
>> diff --git a/docs/features/migration.pandoc b/docs/features/migration.pandoc
>> new file mode 100644
>> index 0000000..e0422f9
>> --- /dev/null
>> +++ b/docs/features/migration.pandoc
>> @@ -0,0 +1,90 @@
>> +% Migration
> % Revision 1
>
>> +
>> +\clearpage
>> +
>> +# Basics
>> +--------------- -------------
>> +        Status: **Supported**
>> +
>> +  Architecture: x86
>> +
>> +     Component: Toolstack
>> +--------------- -------------
>> +
>> +# Overview
>> +
>> +Migration is a mechanism to move a virtual machine while the VM is running.
>> +Live migration moves a running virtual machine between two physical servers,
>> +but the same mechanism can be used for non-live migrate (pause and copy) and
>> +suspend/resume from disk.
>> +
>> +# User details
>> +
>> +No hardware requirements, although hypervisor logdirty support is required for
>> +live migration.
>> +
>> +From the command line, `xl migrate/save/restore` are the top level
>> +interactions.  e.g.
>> +
>> +    xl create my-vm.cfg
>> +    xl migrate my-vm localhost
>> +
>> +or
>> +
>> +    xl create my-vm.cfg
>> +    xl save my-vm /path/to/save/file
>> +    xl restore /path/to/save/file
>> +
>> +Xen 4.6 sees the instruction of Migration v2.  There is no change for people
>> +using `xl`, although the `libxl` API has had an extension.
>> +
>> +# Technical details
>> +
>> +Migration is of formed of several layers.  `libxc` is responsible for the
> Extraneous "of".
>
>> +contents of the VM (ram, vcpus, etc) and the live migration loop, while
>> +`libxl` is responsible for items such as emulator state.
>> +
>> +The format of the migration v2 stream is specified in two documents, and is
>> +architecture neutral.  Compatibility with legacy streams is maintained via the
>> +`convert-legacy-stream` script which transforms a legacy stream into a
>> +migration v2 stream.
>> +
>> +* Documents
>> +    * `docs/specs/libxc-migration-stream.pandoc`
>> +    * `docs/specs/libxl-migration-stream.pandoc`
>> +* `libxc`
>> +    * `tools/libxc/xc_sr_*.[hc]`
>> +* `libxl`
>> +    * `tools/libxl/libxl_stream_{read,write}.c`
>> +* Scripts
>> +    * `tools/python/xen/migration/*.py`
>> +    * `tools/python/scripts/convert-legacy-stream`
>> +    * `tools/python/scripts/verify-stream-v2`
>> +
>> +Users of the `libxl` API have a new parameter `stream_version` in
>> +`domain_restore_params` which is used to distinguish between legacy and v2
>> +migration streams, and hence whether legacy conversion is required.
>> +
>> +# Limitations
>> +
>> +Hypervisor logdirty support is incompatible with hardware passthrough, as
>> +IOMMU faults cannot be used to track writes.
>> +
>> +While not a bug in migration specifically, VMs are very sensitive to changes
>> +in cpuid information, and cpuid levelling support currently has its issues.
>> +Extreme care should be taken when migrating VMs between non-identical CPUs
>> +until the cpuid levelling improvements are complete.
>> +
>> +# Areas for improvement
>> +
>> +* Arm support
>> +* Linear P2M support for x86 PV
>> +* Live looping parameters
>> +
>> +# Known issues
>> +
>> +* x86 HVM guest physmap operations (not reflected in logdirty bitmap)
>> +* x86 HVM with PoD pages (attempts to map cause PoD allocations)
>> +* x86 HVM with nested-virt (no relevant information included in the stream)
>> +* x86 PV ballooning (P2M marked dirty, target frame not marked)
>> +* x86 PV P2M structure changes (not noticed, stale mappings used)
> TBH I think "Areas for improvement" and "Known issues" sections are very
> cryptic to normal users, but I don't have any suggestion to make them
> better.
>
> In any case, any document is better than no documents. I'm all for this
> doc going in as soon as possible.

This document is deliberately designed to contain both user and
technical information, and be a full overview of the feature, so it is
important for the information to be present.  I am open to alternative
suggestions for how to represent it.

They are basically "proposed new features" and "known bugs".  "Areas for
improvement" could easily be small projects for newcomers/gsoc/opw,
while "Known Issues" probably needs a deeper technical knowledge of the
area.

~Andrew

Re: [PATCH for-4.6 1/2] docs: Template for feature documents

[Andrew Cooper]

On 25/08/15 09:41, Wei Liu wrote:
>
>>>> +
>>>> +\clearpage
>>>> +
>>>> +This is a suggested template for formatting of a Xen feature
>>>> document in tree.
>>>> +
>>>> +The purpose of this document is to provide a concrete support
>>>> statement for the
>>>> +feature (indicating its security status), as well as brief user and
>>>> technical
>>>> +documentation.
>>>> +
>>>> +# Basics
>>>> +
>>>> +A table with an overview of the support status and applicability.
>>>> +
>>>> +---------------- ----------------------------------------------------
>>>> +         Status: e.g. **Supported**/**Tech Preview**/**Experimental**
>>>> +
>>>> +Architecture(s): e.g. x86, arm
>>>> +
>>>> +   Component(s): e.g. Hypervisor, toolstack, guest
>>>> +
>>>> +       Hardware: _where applicable_
>>>> +---------------- ----------------------------------------------------
>>> What about adding some information when the feature was introduced or
>>> some other historical stuff? Something like:
>>>
>>> Experimental in Xen 4.1
>>> Supported in Xen 4.3
>>> xl syntax changed in Xen 4.4
>>>
>> In the longterm, I would expect that information to be visible via `git
>> log`.
>>
>> Having said that, it probably is useful to have a summary of history
>> available in the written document.
>>
>> How about a #History section at the bottom?  That can at least include
>> "document written" as a starting point and subsequent major changes in
>> short form.
> Or use a table for revision history at the beginning? I personally think
> using a table is clearer than several paragraphs.
>
> Don't want to bikeshed how it should look like. We can always change the
> template if necessary.

I have introduced a history table at the end of the template.  I want to
avoid it getting in the way of the table of basic information and overview.

~Andrew