(no subject)

(no subject)

[Konrad Rzeszutek Wilk]

I am proposing to put this in the 'staging-4.5' tree but would
appreciate an English speaker to take a look and make sure it
sounds good.

Thank you.

[PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Konrad Rzeszutek Wilk]

Expand the README file to give a brief view of what went in
Xen 4.5.0. Also change the Makefile to not use the '-rc'
postfix.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 README       | 40 +++++++++++++++++++++++++++-------------
 xen/Makefile |  2 +-
 2 files changed, 28 insertions(+), 14 deletions(-)

diff --git a/README b/README
index 412607a..26432b4 100644
--- a/README
+++ b/README
@@ -1,9 +1,9 @@
 #################################
-__  __            _  _    ____                        _        _     _      
-\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
- \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
- /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
-/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
+
+\ \/ /___ _ __   | || |  | ___| / _ \ 
+ \  // _ \ '_ \  | || |_ |___ \| | | |
+ /  \  __/ | | | |__   _| ___) | |_| |
+/_/\_\___|_| |_|    |_|(_)____(_)___/ 
 
 #################################
 
@@ -19,14 +19,28 @@ is freely-distributable Open Source software, released under the GNU
 GPL. Since its initial public release, Xen has grown a large
 development community, spearheaded by xen.org (http://www.xen.org).
 
-The 4.3 release offers a number of improvements, including NUMA
-scheduling affinity, openvswitch integration, and defaulting to
-qemu-xen rather than qemu-traditional for non-stubdom guests.
-(qemu-xen is kept very close to the upstream project.)  We also have a
-number of updates to vTPM, and improvements to XSM and Flask to allow
-greater disaggregation.  Additionally, 4.3 contains a basic version of
-Xen for the new ARM server architecture, both 32- and 64-bit.  And as
-always, there are a number of performance, stability, and security
+The 4.5 release offers a number of improvements: including shedding
+of the Python toolstack (xend), improvements for large scale machines
+during bootup and for PCI passthrough, PVH initial domain support for Intel,
+multiple IO-REQ servers (many QEMUs for a guest), soft affinity for vCPUs
+(aka NUMA affinity), API expansion for guest introspection; Real Time
+Scheduler. We also have number of updates for CPU specific changes, such
+as Broadwell Supervisor Mode Access Prevention, Haswell Bridge Cache QoS
+Monitoring aka Intel Resource Director Technology, and further extensions
+to vAPIC (SandyBridge feature), fixes in AMD microcode loading, and
+Data Breaking Extensions and masking MSR support on AMD.
+
+Additionally, 4.5 has a huge update to the ARM code allowing us to support
+more than 1TB guests, support for more than 8 CPUs, Power State Coordination
+Interface (0.2), UEFI booting, IOMMU support (SMMUv1), Super Page (2MB) support,
+passthrough of MMIO regions to guests, and lower interrupt latency.
+
+The toolstack has expanded to support VM Generation ID (an Windows 2012 Server
+requirement), Remus initial support (for high availability), libxenlight
+JSON support and persistent configuration support, systemd support, and
+lastly support in QEMU for expanding the PCI hole.
+
+And as always, there are a number of performance, stability, and security
 improvements under-the hood.
 
 This file contains some quick-start instructions to install Xen on
diff --git a/xen/Makefile b/xen/Makefile
index 72c1313..5d70741 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -2,7 +2,7 @@
 # All other places this is stored (eg. compile.h) should be autogenerated.
 export XEN_VERSION       = 4
 export XEN_SUBVERSION    = 5
-export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
+export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
 export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
 -include xen-version
 
-- 
2.1.0

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Julien Grall]

Hi Konrad,

On 06/01/15 17:17, Konrad Rzeszutek Wilk wrote:
> +Additionally, 4.5 has a huge update to the ARM code allowing us to support
> +more than 1TB guests, support for more than 8 CPUs, Power State Coordination

We only support up to 1TB of memory and up to 8 CPUs per guest.

Regards,

-- 
Julien Grall

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Konrad Rzeszutek Wilk]

On Tue, Jan 06, 2015 at 05:40:20PM +0000, Julien Grall wrote:
> Hi Konrad,
> 
> On 06/01/15 17:17, Konrad Rzeszutek Wilk wrote:
> > +Additionally, 4.5 has a huge update to the ARM code allowing us to support
> > +more than 1TB guests, support for more than 8 CPUs, Power State Coordination
> 
> We only support up to 1TB of memory and up to 8 CPUs per guest.

Thank you. Updated.
> 
> Regards,
> 
> -- 
> Julien Grall

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Sander Eikelenboom]

Tuesday, January 6, 2015, 6:17:12 PM, you wrote:

> Expand the README file to give a brief view of what went in
> Xen 4.5.0. Also change the Makefile to not use the '-rc'
> postfix.

> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  README       | 40 +++++++++++++++++++++++++++-------------
>  xen/Makefile |  2 +-
>  2 files changed, 28 insertions(+), 14 deletions(-)

> diff --git a/README b/README
> index 412607a..26432b4 100644
> --- a/README
> +++ b/README
> @@ -1,9 +1,9 @@
>  #################################
> -__  __            _  _    ____                        _        _     _      
> -\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
> - \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> - /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
> -/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> +
> +\ \/ /___ _ __   | || |  | ___| / _ \ 
> + \  // _ \ '_ \  | || |_ |___ \| | | |
> + /  \  __/ | | | |__   _| ___) | |_| |
> +/_/\_\___|_| |_|    |_|(_)____(_)___/ 
>  
>  #################################

Hi Konrad,

Doesn't the above hunk chop of the top row of the figlet ?

--
Sander


 
> @@ -19,14 +19,28 @@ is freely-distributable Open Source software, released under the GNU
>  GPL. Since its initial public release, Xen has grown a large
>  development community, spearheaded by xen.org (http://www.xen.org).
>  
> -The 4.3 release offers a number of improvements, including NUMA
> -scheduling affinity, openvswitch integration, and defaulting to
> -qemu-xen rather than qemu-traditional for non-stubdom guests.
> -(qemu-xen is kept very close to the upstream project.)  We also have a
> -number of updates to vTPM, and improvements to XSM and Flask to allow
> -greater disaggregation.  Additionally, 4.3 contains a basic version of
> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> -always, there are a number of performance, stability, and security
> +The 4.5 release offers a number of improvements: including shedding
> +of the Python toolstack (xend), improvements for large scale machines
> +during bootup and for PCI passthrough, PVH initial domain support for Intel,
> +multiple IO-REQ servers (many QEMUs for a guest), soft affinity for vCPUs
> +(aka NUMA affinity), API expansion for guest introspection; Real Time
> +Scheduler. We also have number of updates for CPU specific changes, such
> +as Broadwell Supervisor Mode Access Prevention, Haswell Bridge Cache QoS
> +Monitoring aka Intel Resource Director Technology, and further extensions
> +to vAPIC (SandyBridge feature), fixes in AMD microcode loading, and
> +Data Breaking Extensions and masking MSR support on AMD.
> +
> +Additionally, 4.5 has a huge update to the ARM code allowing us to support
> +more than 1TB guests, support for more than 8 CPUs, Power State Coordination
> +Interface (0.2), UEFI booting, IOMMU support (SMMUv1), Super Page (2MB) support,
> +passthrough of MMIO regions to guests, and lower interrupt latency.
> +
> +The toolstack has expanded to support VM Generation ID (an Windows 2012 Server
> +requirement), Remus initial support (for high availability), libxenlight
> +JSON support and persistent configuration support, systemd support, and
> +lastly support in QEMU for expanding the PCI hole.
> +
> +And as always, there are a number of performance, stability, and security
>  improvements under-the hood.
>  
>  This file contains some quick-start instructions to install Xen on
> diff --git a/xen/Makefile b/xen/Makefile
> index 72c1313..5d70741 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -2,7 +2,7 @@
>  # All other places this is stored (eg. compile.h) should be autogenerated.
>  export XEN_VERSION       = 4
>  export XEN_SUBVERSION    = 5
> -export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
> +export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
>  export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
>  -include xen-version
>  

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Konrad Rzeszutek Wilk]

On Tue, Jan 06, 2015 at 06:54:40PM +0100, Sander Eikelenboom wrote:
> 
> Tuesday, January 6, 2015, 6:17:12 PM, you wrote:
> 
> > Expand the README file to give a brief view of what went in
> > Xen 4.5.0. Also change the Makefile to not use the '-rc'
> > postfix.
> 
> > Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > ---
> >  README       | 40 +++++++++++++++++++++++++++-------------
> >  xen/Makefile |  2 +-
> >  2 files changed, 28 insertions(+), 14 deletions(-)
> 
> > diff --git a/README b/README
> > index 412607a..26432b4 100644
> > --- a/README
> > +++ b/README
> > @@ -1,9 +1,9 @@
> >  #################################
> > -__  __            _  _    ____                        _        _     _      
> > -\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
> > - \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> > - /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
> > -/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> > +
> > +\ \/ /___ _ __   | || |  | ___| / _ \ 
> > + \  // _ \ '_ \  | || |_ |___ \| | | |
> > + /  \  __/ | | | |__   _| ___) | |_| |
> > +/_/\_\___|_| |_|    |_|(_)____(_)___/ 
> >  
> >  #################################
> 
> Hi Konrad,
> 
> Doesn't the above hunk chop of the top row of the figlet ?

Yes it does. Thank you for noticing (updated)!

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Andrew Cooper]

On 06/01/15 17:17, Konrad Rzeszutek Wilk wrote:
> Expand the README file to give a brief view of what went in
> Xen 4.5.0. Also change the Makefile to not use the '-rc'
> postfix.
>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  README       | 40 +++++++++++++++++++++++++++-------------
>  xen/Makefile |  2 +-
>  2 files changed, 28 insertions(+), 14 deletions(-)
>
> diff --git a/README b/README
> index 412607a..26432b4 100644
> --- a/README
> +++ b/README
> @@ -1,9 +1,9 @@
>  #################################
> -__  __            _  _    ____                        _        _     _      
> -\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
> - \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> - /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
> -/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> +
> +\ \/ /___ _ __   | || |  | ___| / _ \ 
> + \  // _ \ '_ \  | || |_ |___ \| | | |
> + /  \  __/ | | | |__   _| ___) | |_| |
> +/_/\_\___|_| |_|    |_|(_)____(_)___/ 

Have you missed the top line from the new figlet banner?

>  
>  #################################
>  
> @@ -19,14 +19,28 @@ is freely-distributable Open Source software, released under the GNU
>  GPL. Since its initial public release, Xen has grown a large
>  development community, spearheaded by xen.org (http://www.xen.org).
>  
> -The 4.3 release offers a number of improvements, including NUMA
> -scheduling affinity, openvswitch integration, and defaulting to
> -qemu-xen rather than qemu-traditional for non-stubdom guests.
> -(qemu-xen is kept very close to the upstream project.)  We also have a
> -number of updates to vTPM, and improvements to XSM and Flask to allow
> -greater disaggregation.  Additionally, 4.3 contains a basic version of
> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> -always, there are a number of performance, stability, and security
> +The 4.5 release offers a number of improvements: including shedding
> +of the Python toolstack (xend), improvements for large scale machines
> +during bootup and for PCI passthrough, PVH initial domain support for Intel,
> +multiple IO-REQ servers (many QEMUs for a guest), soft affinity for vCPUs
> +(aka NUMA affinity), API expansion for guest introspection; Real Time

s/;/and a new experimental/ as this is getting towards the end of the
sentence?

> +Scheduler. We also have number of updates for CPU specific changes, such
> +as Broadwell Supervisor Mode Access Prevention, Haswell Bridge Cache QoS

Haswell Bridge? (I have never heard it called that) s/bridge/server/
perhaps?

> +Monitoring aka Intel Resource Director Technology, and further extensions
> +to vAPIC (SandyBridge feature), fixes in AMD microcode loading, and
> +Data Breaking Extensions and masking MSR support on AMD.
> +
> +Additionally, 4.5 has a huge update to the ARM code allowing us to support
> +more than 1TB guests, support for more than 8 CPUs, Power State Coordination
> +Interface (0.2), UEFI booting, IOMMU support (SMMUv1), Super Page (2MB) support,
> +passthrough of MMIO regions to guests, and lower interrupt latency.
> +
> +The toolstack has expanded to support VM Generation ID (an Windows 2012 Server

"a Windows ...." rather than an

> +requirement), Remus initial support (for high availability), libxenlight

This is specifically remus support in xl, given the removal of xend.

~Andrew

> +JSON support and persistent configuration support, systemd support, and
> +lastly support in QEMU for expanding the PCI hole.
> +
> +And as always, there are a number of performance, stability, and security
>  improvements under-the hood.
>  
>  This file contains some quick-start instructions to install Xen on
> diff --git a/xen/Makefile b/xen/Makefile
> index 72c1313..5d70741 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -2,7 +2,7 @@
>  # All other places this is stored (eg. compile.h) should be autogenerated.
>  export XEN_VERSION       = 4
>  export XEN_SUBVERSION    = 5
> -export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
> +export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
>  export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
>  -include xen-version
>  

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Ian Campbell]

On Tue, 2015-01-06 at 12:17 -0500, Konrad Rzeszutek Wilk wrote:
> +The 4.5 release offers a number of improvements: including shedding
> +of the Python toolstack (xend),

I don't think this should be the headline "improvement", if it needs
mentioning at all it should be further down, perhaps as a separate para
near the end detailing things which have gone away rather than presented
as a feature.

>  improvements for large scale machines
> +during bootup and for PCI passthrough, PVH initial domain support for Intel,

PVH is a preview isn't it? Shouldn't that be made clear to avoid
disappointing? (likewise for anything else like that in this list), I'd
suggest separate paragraphs for new features, new preview and new
experimental stuff.

> +multiple IO-REQ servers (many QEMUs for a guest), soft affinity for vCPUs
> +(aka NUMA affinity), API expansion for guest introspection; Real Time
> +Scheduler. We also have number of updates for CPU specific changes, such
> +as Broadwell Supervisor Mode Access Prevention, Haswell Bridge Cache QoS
> +Monitoring aka Intel Resource Director Technology, and further extensions
> +to vAPIC (SandyBridge feature), fixes in AMD microcode loading, and
> +Data Breaking Extensions and masking MSR support on AMD.
> +
> +Additionally, 4.5 has a huge update to the ARM code allowing us to support
> +more than 1TB guests, support for more than 8 CPUs, Power State Coordination
> +Interface (0.2), UEFI booting, IOMMU support (SMMUv1), Super Page (2MB) support,
> +passthrough of MMIO regions to guests, and lower interrupt latency.
> +
> +The toolstack has expanded to support VM Generation ID (an Windows 2012 Server
> +requirement), Remus initial support (for high availability), libxenlight
> +JSON support and persistent configuration support, systemd support, and
> +lastly support in QEMU for expanding the PCI hole.
> +
> +And as always, there are a number of performance, stability, and security
>  improvements under-the hood.
>  
>  This file contains some quick-start instructions to install Xen on
> diff --git a/xen/Makefile b/xen/Makefile
> index 72c1313..5d70741 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -2,7 +2,7 @@
>  # All other places this is stored (eg. compile.h) should be autogenerated.
>  export XEN_VERSION       = 4
>  export XEN_SUBVERSION    = 5
> -export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
> +export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
>  export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
>  -include xen-version
>  

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Ian Jackson]

Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
> -The 4.3 release offers a number of improvements, including NUMA
> -scheduling affinity, openvswitch integration, and defaulting to
> -qemu-xen rather than qemu-traditional for non-stubdom guests.
> -(qemu-xen is kept very close to the upstream project.)  We also have a
> -number of updates to vTPM, and improvements to XSM and Flask to allow
> -greater disaggregation.  Additionally, 4.3 contains a basic version of
> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> -always, there are a number of performance, stability, and security
> +The 4.5 release offers a number of improvements: including shedding

Should read
  +The 4.5 release offers a number of improvements, including: shedding
(note two punctuation changes) and the list items should all be
separated with semicolons IMO.

> +(aka NUMA affinity), API expansion for guest introspection; Real Time
> +Scheduler. We also have number of updates for CPU specific changes, such

Needs "and," or "and" before "Real Time Scheduler".  And you should
say what was done to the RT scheduler, since it's not just a new
feature.

> +as Broadwell Supervisor Mode Access Prevention, Haswell Bridge Cache QoS

I would put a : after "such as" and use semicolons in this list too.

> +Monitoring aka Intel Resource Director Technology, and further extensions
> +to vAPIC (SandyBridge feature), fixes in AMD microcode loading, and
> +Data Breaking Extensions and masking MSR support on AMD.

"and" should appear only in the final list item.

> +Additionally, 4.5 has a huge update to the ARM code allowing us to support
> +more than 1TB guests, support for more than 8 CPUs, Power State Coordination
> +Interface (0.2), UEFI booting, IOMMU support (SMMUv1), Super Page (2MB) support,
> +passthrough of MMIO regions to guests, and lower interrupt latency.

All of the entries in the list should be in the same part of speech.
I would make them all nouns.  So:

  +Additionally, 4.5 has a huge update to the ARM code, including support
  +for: >=1TB guests; more than 8 CPUs; Power State Coordination
> +Interface (0.2); UEFI booting; IOMMU (SMMUv1); Super Page (2MB);
> +passthrough of MMIO regions to guests; and lower interrupt latency.

Similarly for the final paragraph.

Thanks,
Ian.

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Konrad Rzeszutek Wilk]

On Tue, Jan 06, 2015 at 06:06:23PM +0000, Ian Jackson wrote:
> Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
> > -The 4.3 release offers a number of improvements, including NUMA
> > -scheduling affinity, openvswitch integration, and defaulting to
> > -qemu-xen rather than qemu-traditional for non-stubdom guests.
> > -(qemu-xen is kept very close to the upstream project.)  We also have a
> > -number of updates to vTPM, and improvements to XSM and Flask to allow
> > -greater disaggregation.  Additionally, 4.3 contains a basic version of
> > -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> > -always, there are a number of performance, stability, and security
> > +The 4.5 release offers a number of improvements: including shedding
> 
> Should read
>   +The 4.5 release offers a number of improvements, including: shedding
> (note two punctuation changes) and the list items should all be
> separated with semicolons IMO.


Thank you for your update.

I've incorporated feedback from all folks I hope. Would this be satisfactory?

>From 89041667dfdc298dbe0f17a1f66a748a1a13089f Mon Sep 17 00:00:00 2001
From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Date: Tue, 6 Jan 2015 12:03:10 -0500
Subject: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

Expand the README file to give a brief view of what went in
Xen 4.5.0. Also change the Makefile to not use the '-rc'
postfix.

Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
---
 README       | 45 ++++++++++++++++++++++++++++++++-------------
 xen/Makefile |  2 +-
 2 files changed, 33 insertions(+), 14 deletions(-)

diff --git a/README b/README
index 412607a..641bb23 100644
--- a/README
+++ b/README
@@ -1,9 +1,9 @@
 #################################
-__  __            _  _    ____                        _        _     _      
-\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
- \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
- /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
-/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
+__  __            _  _    ____   ___
+\ \/ /___ _ __   | || |  | ___| / _ \
+ \  // _ \ '_ \  | || |_ |___ \| | | |
+ /  \  __/ | | | |__   _| ___) | |_| |
+/_/\_\___|_| |_|    |_|(_)____(_)___/
 
 #################################
 
@@ -19,14 +19,33 @@ is freely-distributable Open Source software, released under the GNU
 GPL. Since its initial public release, Xen has grown a large
 development community, spearheaded by xen.org (http://www.xen.org).
 
-The 4.3 release offers a number of improvements, including NUMA
-scheduling affinity, openvswitch integration, and defaulting to
-qemu-xen rather than qemu-traditional for non-stubdom guests.
-(qemu-xen is kept very close to the upstream project.)  We also have a
-number of updates to vTPM, and improvements to XSM and Flask to allow
-greater disaggregation.  Additionally, 4.3 contains a basic version of
-Xen for the new ARM server architecture, both 32- and 64-bit.  And as
-always, there are a number of performance, stability, and security
+The 4.5 release offers a number of improvements, including: improvements for
+large scale machines during bootup and for PCI passthrough; multiple IO-REQ
+servers (many QEMUs for a guest); soft affinity for vCPUs (aka NUMA affinity);
+and API expansion for guest introspection. We also have number of updates for
+CPU specific changes, such as: Broadwell Supervisor Mode Access Prevention;
+Haswell Server Cache QoS Monitoring aka Intel Resource Director Technology;
+further extensions to vAPIC (SandyBridge feature); fixes in AMD microcode
+loading; Data Breaking Extensions and masking MSR support on AMD.
+
+On the experimental side we have added a new Real-Time Deferrable Server Based
+CPU Scheduler (rtds), and PVH initial domain (dom0) support for Intel CPUs.
+
+Additionally, 4.5 has a huge update to the ARM code, including support for:
+up to 1TB in guests; up to 8 CPUs; Power State Coordination Interface (0.2)
+to power up and down CPUs; UEFI booting; IOMMU support (SMMUv1); Super Page
+(2MB) support; passthrough of MMIO regions to guests; and lower interrupt
+latency.
+
+The toolstack has expanded to include support for: VM Generation ID (a Windows
+2012 Server requirement); Remus initial support (for high availability)
+in libxl (since xend has been removed); libxenlight JSON support and persistent 
+configuration support, systemd support; and support in QEMU for expanding the
+PCI hole.
+
+Lastly, we have removed the Python toolstack (xend).
+
+And as always, there are a number of performance, stability, and security
 improvements under-the hood.
 
 This file contains some quick-start instructions to install Xen on
diff --git a/xen/Makefile b/xen/Makefile
index 72c1313..5d70741 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -2,7 +2,7 @@
 # All other places this is stored (eg. compile.h) should be autogenerated.
 export XEN_VERSION       = 4
 export XEN_SUBVERSION    = 5
-export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
+export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
 export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
 -include xen-version
 
-- 
2.1.0

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Andrew Cooper]

On 06/01/15 18:21, Konrad Rzeszutek Wilk wrote:
> On Tue, Jan 06, 2015 at 06:06:23PM +0000, Ian Jackson wrote:
>> Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
>>> -The 4.3 release offers a number of improvements, including NUMA
>>> -scheduling affinity, openvswitch integration, and defaulting to
>>> -qemu-xen rather than qemu-traditional for non-stubdom guests.
>>> -(qemu-xen is kept very close to the upstream project.)  We also have a
>>> -number of updates to vTPM, and improvements to XSM and Flask to allow
>>> -greater disaggregation.  Additionally, 4.3 contains a basic version of
>>> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
>>> -always, there are a number of performance, stability, and security
>>> +The 4.5 release offers a number of improvements: including shedding
>> Should read
>>   +The 4.5 release offers a number of improvements, including: shedding
>> (note two punctuation changes) and the list items should all be
>> separated with semicolons IMO.
>
> Thank you for your update.
>
> I've incorporated feedback from all folks I hope. Would this be satisfactory?
>
> From 89041667dfdc298dbe0f17a1f66a748a1a13089f Mon Sep 17 00:00:00 2001
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Date: Tue, 6 Jan 2015 12:03:10 -0500
> Subject: [PATCH] README, xen/Makefile: Update to Xen 4.5.0
>
> Expand the README file to give a brief view of what went in
> Xen 4.5.0. Also change the Makefile to not use the '-rc'
> postfix.
>
> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  README       | 45 ++++++++++++++++++++++++++++++++-------------
>  xen/Makefile |  2 +-
>  2 files changed, 33 insertions(+), 14 deletions(-)
>
> diff --git a/README b/README
> index 412607a..641bb23 100644
> --- a/README
> +++ b/README
> @@ -1,9 +1,9 @@
>  #################################
> -__  __            _  _    ____                        _        _     _      
> -\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
> - \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> - /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
> -/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> +__  __            _  _    ____   ___
> +\ \/ /___ _ __   | || |  | ___| / _ \
> + \  // _ \ '_ \  | || |_ |___ \| | | |
> + /  \  __/ | | | |__   _| ___) | |_| |
> +/_/\_\___|_| |_|    |_|(_)____(_)___/
>  
>  #################################
>  
> @@ -19,14 +19,33 @@ is freely-distributable Open Source software, released under the GNU
>  GPL. Since its initial public release, Xen has grown a large
>  development community, spearheaded by xen.org (http://www.xen.org).
>  
> -The 4.3 release offers a number of improvements, including NUMA
> -scheduling affinity, openvswitch integration, and defaulting to
> -qemu-xen rather than qemu-traditional for non-stubdom guests.
> -(qemu-xen is kept very close to the upstream project.)  We also have a
> -number of updates to vTPM, and improvements to XSM and Flask to allow
> -greater disaggregation.  Additionally, 4.3 contains a basic version of
> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> -always, there are a number of performance, stability, and security
> +The 4.5 release offers a number of improvements, including: improvements for
> +large scale machines during bootup and for PCI passthrough; multiple IO-REQ
> +servers (many QEMUs for a guest); soft affinity for vCPUs (aka NUMA affinity);
> +and API expansion for guest introspection. We also have number of updates for
> +CPU specific changes, such as: Broadwell Supervisor Mode Access Prevention;
> +Haswell Server Cache QoS Monitoring aka Intel Resource Director Technology;
> +further extensions to vAPIC (SandyBridge feature); fixes in AMD microcode
> +loading; Data Breaking Extensions and masking MSR support on AMD.

"further MSR masking support".  There was already some masking
available, but Jan found some more masks to use.

Otherwise, Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

> +
> +On the experimental side we have added a new Real-Time Deferrable Server Based
> +CPU Scheduler (rtds), and PVH initial domain (dom0) support for Intel CPUs.
> +
> +Additionally, 4.5 has a huge update to the ARM code, including support for:
> +up to 1TB in guests; up to 8 CPUs; Power State Coordination Interface (0.2)
> +to power up and down CPUs; UEFI booting; IOMMU support (SMMUv1); Super Page
> +(2MB) support; passthrough of MMIO regions to guests; and lower interrupt
> +latency.
> +
> +The toolstack has expanded to include support for: VM Generation ID (a Windows
> +2012 Server requirement); Remus initial support (for high availability)
> +in libxl (since xend has been removed); libxenlight JSON support and persistent 
> +configuration support, systemd support; and support in QEMU for expanding the
> +PCI hole.
> +
> +Lastly, we have removed the Python toolstack (xend).
> +
> +And as always, there are a number of performance, stability, and security
>  improvements under-the hood.
>  
>  This file contains some quick-start instructions to install Xen on
> diff --git a/xen/Makefile b/xen/Makefile
> index 72c1313..5d70741 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -2,7 +2,7 @@
>  # All other places this is stored (eg. compile.h) should be autogenerated.
>  export XEN_VERSION       = 4
>  export XEN_SUBVERSION    = 5
> -export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
> +export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
>  export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
>  -include xen-version
>  

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Sander Eikelenboom]

Tuesday, January 6, 2015, 7:21:58 PM, you wrote:

> On Tue, Jan 06, 2015 at 06:06:23PM +0000, Ian Jackson wrote:
>> Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
>> > -The 4.3 release offers a number of improvements, including NUMA
>> > -scheduling affinity, openvswitch integration, and defaulting to
>> > -qemu-xen rather than qemu-traditional for non-stubdom guests.
>> > -(qemu-xen is kept very close to the upstream project.)  We also have a
>> > -number of updates to vTPM, and improvements to XSM and Flask to allow
>> > -greater disaggregation.  Additionally, 4.3 contains a basic version of
>> > -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
>> > -always, there are a number of performance, stability, and security
>> > +The 4.5 release offers a number of improvements: including shedding
>> 
>> Should read
>>   +The 4.5 release offers a number of improvements, including: shedding
>> (note two punctuation changes) and the list items should all be
>> separated with semicolons IMO.


> Thank you for your update.

> I've incorporated feedback from all folks I hope. Would this be satisfactory?

>>From 89041667dfdc298dbe0f17a1f66a748a1a13089f Mon Sep 17 00:00:00 2001
> From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Date: Tue, 6 Jan 2015 12:03:10 -0500
> Subject: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

> Expand the README file to give a brief view of what went in
> Xen 4.5.0. Also change the Makefile to not use the '-rc'
> postfix.

> Signed-off-by: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> ---
>  README       | 45 ++++++++++++++++++++++++++++++++-------------
>  xen/Makefile |  2 +-
>  2 files changed, 33 insertions(+), 14 deletions(-)

> diff --git a/README b/README
> index 412607a..641bb23 100644
> --- a/README
> +++ b/README
> @@ -1,9 +1,9 @@
>  #################################
> -__  __            _  _    ____                        _        _     _      
> -\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
> - \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> - /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
> -/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> +__  __            _  _    ____   ___
> +\ \/ /___ _ __   | || |  | ___| / _ \
> + \  // _ \ '_ \  | || |_ |___ \| | | |
> + /  \  __/ | | | |__   _| ___) | |_| |
> +/_/\_\___|_| |_|    |_|(_)____(_)___/
>  
>  #################################
>  
> @@ -19,14 +19,33 @@ is freely-distributable Open Source software, released under the GNU
>  GPL. Since its initial public release, Xen has grown a large
>  development community, spearheaded by xen.org (http://www.xen.org).

Shouldn't this be "Xen-project" and "http://www.xenproject.org", since the transition 
to the Linux foundation ? 
(there are a lot more in-tree places where this could/should be changed for 4.6)

--
Sander
  
> -The 4.3 release offers a number of improvements, including NUMA
> -scheduling affinity, openvswitch integration, and defaulting to
> -qemu-xen rather than qemu-traditional for non-stubdom guests.
> -(qemu-xen is kept very close to the upstream project.)  We also have a
> -number of updates to vTPM, and improvements to XSM and Flask to allow
> -greater disaggregation.  Additionally, 4.3 contains a basic version of
> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> -always, there are a number of performance, stability, and security
> +The 4.5 release offers a number of improvements, including: improvements for
> +large scale machines during bootup and for PCI passthrough; multiple IO-REQ
> +servers (many QEMUs for a guest); soft affinity for vCPUs (aka NUMA affinity);
> +and API expansion for guest introspection. We also have number of updates for
> +CPU specific changes, such as: Broadwell Supervisor Mode Access Prevention;
> +Haswell Server Cache QoS Monitoring aka Intel Resource Director Technology;
> +further extensions to vAPIC (SandyBridge feature); fixes in AMD microcode
> +loading; Data Breaking Extensions and masking MSR support on AMD.
> +
> +On the experimental side we have added a new Real-Time Deferrable Server Based
> +CPU Scheduler (rtds), and PVH initial domain (dom0) support for Intel CPUs.
> +
> +Additionally, 4.5 has a huge update to the ARM code, including support for:
> +up to 1TB in guests; up to 8 CPUs; Power State Coordination Interface (0.2)
> +to power up and down CPUs; UEFI booting; IOMMU support (SMMUv1); Super Page
> +(2MB) support; passthrough of MMIO regions to guests; and lower interrupt
> +latency.
> +
> +The toolstack has expanded to include support for: VM Generation ID (a Windows
> +2012 Server requirement); Remus initial support (for high availability)
> +in libxl (since xend has been removed); libxenlight JSON support and persistent 
> +configuration support, systemd support; and support in QEMU for expanding the
> +PCI hole.
> +
> +Lastly, we have removed the Python toolstack (xend).
> +
> +And as always, there are a number of performance, stability, and security
>  improvements under-the hood.
>  
>  This file contains some quick-start instructions to install Xen on
> diff --git a/xen/Makefile b/xen/Makefile
> index 72c1313..5d70741 100644
> --- a/xen/Makefile
> +++ b/xen/Makefile
> @@ -2,7 +2,7 @@
>  # All other places this is stored (eg. compile.h) should be autogenerated.
>  export XEN_VERSION       = 4
>  export XEN_SUBVERSION    = 5
> -export XEN_EXTRAVERSION ?= .0-rc$(XEN_VENDORVERSION)
> +export XEN_EXTRAVERSION ?= .0$(XEN_VENDORVERSION)
>  export XEN_FULLVERSION   = $(XEN_VERSION).$(XEN_SUBVERSION)$(XEN_EXTRAVERSION)
>  -include xen-version
>  

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Ian Campbell]

On Tue, 2015-01-06 at 19:46 +0100, Sander Eikelenboom wrote:
> > diff --git a/README b/README
> > index 412607a..641bb23 100644
> > --- a/README
> > +++ b/README
> > @@ -1,9 +1,9 @@
> >  #################################
> > -__  __            _  _    ____                        _        _     _      
> > -\ \/ /___ _ __   | || |  | ___|       _   _ _ __  ___| |_ __ _| |__ | | ___ 
> > - \  // _ \ '_ \  | || |_ |___ \ _____| | | | '_ \/ __| __/ _` | '_ \| |/ _ \
> > - /  \  __/ | | | |__   _| ___) |_____| |_| | | | \__ \ || (_| | |_) | |  __/
> > -/_/\_\___|_| |_|    |_|(_)____/       \__,_|_| |_|___/\__\__,_|_.__/|_|\___|
> > +__  __            _  _    ____   ___
> > +\ \/ /___ _ __   | || |  | ___| / _ \
> > + \  // _ \ '_ \  | || |_ |___ \| | | |
> > + /  \  __/ | | | |__   _| ___) | |_| |
> > +/_/\_\___|_| |_|    |_|(_)____(_)___/
> >  
> >  #################################
> >  
> > @@ -19,14 +19,33 @@ is freely-distributable Open Source software, released under the GNU
> >  GPL. Since its initial public release, Xen has grown a large
> >  development community, spearheaded by xen.org (http://www.xen.org).
> 
> Shouldn't this be "Xen-project" and "http://www.xenproject.org", since the transition 
> to the Linux foundation ? 

For the URLs, I guess, but it's not urgent for 4.5 IMHO, the old URLs
aren't going anywhere.

For the prose, NAK, the hypervisor is called "Xen". Lets not make the
technical documentation use the long-winded and stilted "Xen Project
Hypervisor" or whatever when a simple "Xen" works fine.

Ian.

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Pasi Kärkkäinen]

On Tue, Jan 06, 2015 at 01:21:58PM -0500, Konrad Rzeszutek Wilk wrote:
> On Tue, Jan 06, 2015 at 06:06:23PM +0000, Ian Jackson wrote:
> > Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
> > > -The 4.3 release offers a number of improvements, including NUMA
> > > -scheduling affinity, openvswitch integration, and defaulting to
> > > -qemu-xen rather than qemu-traditional for non-stubdom guests.
> > > -(qemu-xen is kept very close to the upstream project.)  We also have a
> > > -number of updates to vTPM, and improvements to XSM and Flask to allow
> > > -greater disaggregation.  Additionally, 4.3 contains a basic version of
> > > -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> > > -always, there are a number of performance, stability, and security
> > > +The 4.5 release offers a number of improvements: including shedding
> > 
> > Should read
> >   +The 4.5 release offers a number of improvements, including: shedding
> > (note two punctuation changes) and the list items should all be
> > separated with semicolons IMO.
> 
> 
> Thank you for your update.
> 
> I've incorporated feedback from all folks I hope. Would this be satisfactory?
>

Do we want to mention things like HVM guest direct kernel boot,
or HVM guests MMIO hole resize support? 


And I think there was optimizations to oxenstored to support up to 1000 VMs per host.. 


-- Pasi

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Don Slutz]

On 01/06/15 14:14, Pasi Kärkkäinen wrote:
> On Tue, Jan 06, 2015 at 01:21:58PM -0500, Konrad Rzeszutek Wilk wrote:
>> On Tue, Jan 06, 2015 at 06:06:23PM +0000, Ian Jackson wrote:
>>> Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
>>>> -The 4.3 release offers a number of improvements, including NUMA
>>>> -scheduling affinity, openvswitch integration, and defaulting to
>>>> -qemu-xen rather than qemu-traditional for non-stubdom guests.
>>>> -(qemu-xen is kept very close to the upstream project.)  We also have a
>>>> -number of updates to vTPM, and improvements to XSM and Flask to allow
>>>> -greater disaggregation.  Additionally, 4.3 contains a basic version of
>>>> -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
>>>> -always, there are a number of performance, stability, and security
>>>> +The 4.5 release offers a number of improvements: including shedding
>>>
>>> Should read
>>>   +The 4.5 release offers a number of improvements, including: shedding
>>> (note two punctuation changes) and the list items should all be
>>> separated with semicolons IMO.
>>
>>
>> Thank you for your update.
>>
>> I've incorporated feedback from all folks I hope. Would this be satisfactory?
>>
> 
> Do we want to mention things like HVM guest direct kernel boot,
> or HVM guests MMIO hole resize support? 
> 

It is not "resize" support, it is configure size support.  And
yes it might be good to list it.

   -Don Slutz

> 
> And I think there was optimizations to oxenstored to support up to 1000 VMs per host.. 
> 
> 
> -- Pasi
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
> 

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Konrad Rzeszutek Wilk]

On Tue, Jan 06, 2015 at 09:14:22PM +0200, Pasi Kärkkäinen wrote:
> On Tue, Jan 06, 2015 at 01:21:58PM -0500, Konrad Rzeszutek Wilk wrote:
> > On Tue, Jan 06, 2015 at 06:06:23PM +0000, Ian Jackson wrote:
> > > Konrad Rzeszutek Wilk writes ("[Xen-devel] [PATCH] README, xen/Makefile: Update to Xen 4.5.0"):
> > > > -The 4.3 release offers a number of improvements, including NUMA
> > > > -scheduling affinity, openvswitch integration, and defaulting to
> > > > -qemu-xen rather than qemu-traditional for non-stubdom guests.
> > > > -(qemu-xen is kept very close to the upstream project.)  We also have a
> > > > -number of updates to vTPM, and improvements to XSM and Flask to allow
> > > > -greater disaggregation.  Additionally, 4.3 contains a basic version of
> > > > -Xen for the new ARM server architecture, both 32- and 64-bit.  And as
> > > > -always, there are a number of performance, stability, and security
> > > > +The 4.5 release offers a number of improvements: including shedding
> > > 
> > > Should read
> > >   +The 4.5 release offers a number of improvements, including: shedding
> > > (note two punctuation changes) and the list items should all be
> > > separated with semicolons IMO.
> > 
> > 
> > Thank you for your update.
> > 
> > I've incorporated feedback from all folks I hope. Would this be satisfactory?
> >
> 
> Do we want to mention things like HVM guest direct kernel boot,
> or HVM guests MMIO hole resize support? 

"and support in QEMU for expanding the PCI hole." which is in the README

HVM guest direct kernel boot - I forgot about that!

> 
> 
> And I think there was optimizations to oxenstored to support up to 1000 VMs per host.. 

.. and that.

diff --git a/README b/README
index 4a9cac1..e2c9e83 100644
--- a/README
+++ b/README
@@ -43,8 +43,9 @@ guests; and lower interrupt latency.
 The toolstack has expanded to include support for: VM Generation ID (a
 Windows 2012 Server requirement); Remus initial support (for high
 availability) in libxl (since xend has been removed); libxenlight JSON
-support and persistent configuration support, and systemd support; and
-support in QEMU for expanding the PCI hole.
+support, HVM guest direct kernel boot, and persistent configuration
+support; systemd support; optimizations in oxenstored to support more
+than 1000+ VMs; and support in QEMU for expanding the PCI hole.
 
 Lastly, we have removed the Python toolstack (xend).
 

Re: [PATCH] README, xen/Makefile: Update to Xen 4.5.0

[Pasi Kärkkäinen]

On Wed, Jan 07, 2015 at 10:27:37AM -0500, Konrad Rzeszutek Wilk wrote:
> > 
> > And I think there was optimizations to oxenstored to support up to 1000 VMs per host.. 
> 
> .. and that.
>

Actually I'm not sure anymore about "up to 1000 VMs per host".. I was certain I was reading about such number
in the discussion related to some patches about performance improvements related to VDI bootstorms on xen-devel, 
but I can't find a reference now..

So maybe we should drop that number.. and only state performance/scalability improvements for oxenstored.

Also the oxenstored fd limit was increased from 1024 to 4096,
and oxenstored switched from select() to poll() for increased scalability/performance.

-- Pasi

 
> diff --git a/README b/README
> index 4a9cac1..e2c9e83 100644
> --- a/README
> +++ b/README
> @@ -43,8 +43,9 @@ guests; and lower interrupt latency.
>  The toolstack has expanded to include support for: VM Generation ID (a
>  Windows 2012 Server requirement); Remus initial support (for high
>  availability) in libxl (since xend has been removed); libxenlight JSON
> -support and persistent configuration support, and systemd support; and
> -support in QEMU for expanding the PCI hole.
> +support, HVM guest direct kernel boot, and persistent configuration
> +support; systemd support; optimizations in oxenstored to support more
> +than 1000+ VMs; and support in QEMU for expanding the PCI hole.
>  
>  Lastly, we have removed the Python toolstack (xend).
>