Xen 4.5 development update (July update)

Xen 4.5 development update (July update)

[konrad.wilk]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 12417 bytes --]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is less than one month away. This being the summer some folks are talking
vacation, so that month might be a lot less.

I have moved some of the items for which I hadn't seen any traffic for
quite some time - or hadn't received any email response, and as such I think
it won't make it by the feature freeze date.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
A bunch of items had moved to the completed phase which is fantastic.

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs

For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

In terms of libvirt it has monthly releases. As such not going to track
every release - but closer to when RCs are out.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Coding time: <=== NOW, one (less than!) month left!

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM - Device assigment on ARM (good)
   Linux parts at risk.
   v2 for hypervisor out
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
   v10 posted; resent & rebased
  -  Arianna Avanzini

*  ARM Xen UEFI booting on ARM (ok)
   v2
  -  Roy Franz

*  ARM PSCI v0.2 (good)
   v11 posted
  -  Parth Dixit

*  ARM GICv3 support (ok)
   v6a patchset (refactor in, also known as v9a); v7 posted
   v9a posted which does GIC and VGIC code refactoring
   v8 posted
  -  Vijay Kilari

*  ARM remote processor iommu module (GPUs + IPUs) (fair)
   v2 posted
  -  Andrii Tseglytskyi

*  ARM - MiniOS (ok)
   v7 posted
  -  Thomas Leonard

*  ARM - VGIC emulation (ok)
   Reposted as gic and vgic fixes and improvements
   v12
  -  Stefano Stabellini

*  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (ok)
   Provide kernels an grant->MFN lookup
   v4
  -  Stefano Stabellini

*  ARM - Add Odroid-XU (Exynos5410) support (ok)
   v3
  -  Suriyan Ramasami

== x86 ==

*  New Migration (v2). (good)
   v6 posted, going on the 'good' side!
  -  Andrew Cooper & David Vrabel

*  libxl migrationv2 patches. (none)
  -  Andrew Cooper & David Vrabel

*  tmem migrationv2 patches. (none)
  -  Bob Liu & Andrew Cooper & David Vrabel

*  Remus using migration-v2 (fair)
   RFC posted - depends on v6 of 'New Migration'
  -  Yang Hongyang

*  PVH - AMD hardware support. (fair)
   Issues with TSC on modern AMD machines
  -  Mukesh Rathor

*  Xen multiboot2-EFI support (fair)
   Needed for SecureBoot
   Depends on Xen Boot info (rework multiboot and other structs)
   See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html
   RFC posted
  -  Daniel Kiper

*  HT enabled, virtualization overhead is high (Xen 4.4) (none)
   kernbench demonstrated it
   Looking and tracing
   http://www.gossamer-threads.com/lists/xen/devel/339409
  -  Dario Faggioli

*  dirty vram / IOMMU bug (fair)
   http://bugs.xenproject.org/xen/bug/38
  -  Zhang, Yang Z

*  VMware backdoor (hypercall) (ok)
   Needs to be split up and redone
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
   v9 posted
  -  Boris Ostrovsky

*  Cache QoS Monitoring - hypercalls (fair)
   Just hypercalls - no toolstack changes.
   v13.
  -  Dongxiao Xu and Shantong Kang

*  1TB slow destruction (ok)
  -  Bob Liu

*  extending mem_access support to PV domain (fair)
   RFC v2
  -  Aravindh Puthiyaparambil (aravindp)

*  Support controlling the max C-state sub-state (ok)
   v3 posted
  -  Ross Lagerwall

*  Repurpose SEDF Scheduler for Real-time (fair)
   RFC patch posted (v2)
  -  Joshua Whitehead, Robert VanVossen

*  XenRT (Preemptive Global Earliest Deadline First) (fair)
   RFC patch posted (v2)
  -  Meng Xu

*  Introspection of HVM guests (fair)
   RFC v7
  -  Razvan Cojocaru

*  vNUMA in Xen (ok)
   v6 posted
   git://gitorious.org/vnuma/xen_vnuma.git
  -  Elena Ufimtseva

== QEMU ==

*  Using qemu-upstream in a stubdomain (fair)
   Will use rump kernels.
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   v5 posted
  -  Chen, Tiejun

*  Xen PV block driver in OVMF (UEFI in guest) (fair)
   RFC posted.
  -  Anthony PERARD

== lib{xc,xl} and toolstack ==

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  Remus in Xen (libxl) (ok)
   v18
   url: https://github.com/laijs/xen remus-v18
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  COarse-grain LOck-stepping Virtual Machines in Xen (fair)
   RFC patch posted
  -  Wen Congyang
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  snapshot API extension (checkpointing disk) (ok)
   v5
  -  Bamvor Jian Zhang

*  extend the xenstore ring with a 'closing' signal (fair)
   RFC patch posted
  -  David Scott

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
   v6 posted
   git://gitorious.org/vnuma/linux_vnuma.git
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  vAPIC in PVHVM guests (Linux side) (none)
  -  Boris Ostrovsky

*  pvSCSI (ok)
   Initial patch posted.
  -  Juergen Gross

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

*  mini-os: xenbus changes for rump kernels (ok)
   git://xenbits.xen.org/people/iwj/rumpuser-xen.git
   branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
   v2 posted
  -  Ian Jackson

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to Xen 4.6 ==

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  Xen HPET interrupt fixes (fair)
   behind migration v2
  -  Andrew Cooper

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  ARM VM save/restore/live migration (none)
   Need to rebased against migrationv2 - no code posted.
  -  Junghyun Yoo

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  cpuid leveling (none)
   http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
  -  Ian Jackson

*  IO-NUMA - hwloc and xl (none)
   Andrew Cooper had an RFC patch for hwloc
  -  Boris Ostrovsky

*  HVM guest NUMA (none)
  -  Matt Wilson

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  ARM GICv2m support (none)
  -  Linaro (unknown)

== Up for grabs ==

*  OSSTest - also test Linux PVH guests

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of vHPET (done)
   aka hvm/hpet: Detect comparator values in the past
  -  Don Slutz

*  vAPIC in PVHVM guests (Xen side) (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)

*  multiple AMD container files appended together in initrd (early initramfs)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk

*  ARM  - IOMMU support (done)
  -  Julien Grall

*  ioreq-server, aka secondary emulators (done)
  -  Paul Durrant

*  Netback multiqueue (good)
  -  Wei Liu

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  Bigger PCI hole in QEMU (done)
   Needs to be rebased
  -  Don Slutz

*  ARM DRA7 support (done)
   v3 posted
   v3 with comments applied
  -  Andrii Tseglytskyi

*  rework VM Generation ID (done)
   v7 posted
  -  David Vrabel

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
   v11 posted
  -  Dario Faggioli

*  Linux pvops of Xen EFI hypercall support (done)
  -  Daniel Kiper

*  ARM: Use super pages in p2m (done)
   v5 posted
  -  Ian Campbell

*  QEMU 2.0 branch for qemu-upstream (done)
   It is v2.0 with 2.1 Xen backports.
  -  Stefano Stabellini

*  systemd support (done)
   v11
  -  Luis R. Rodriguez

*  Soft affinity for vcpus libxl/xl changes (done)
   v13 posted
  -  Dario Faggioli



[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Don Slutz]

On 08/15/14 13:23, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame.  The tentative feature freeze is scheduled for September 10th,
> which is less than one month away. This being the summer some folks are talking
> vacation, so that month might be a lot less.
>

...

> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>

...

>
> *  VMware backdoor (hypercall) (ok)
>     Needs to be split up and redone
>    -  Don Slutz

Still making review changes.  And then the re-spliting into a good patch set
still needs to be done.  So I think this is still in the ok state.
     -Don Slutz

Re: Xen 4.5 development update (July update)

[Jan Beulich]

>>> On 15.08.14 at 19:23, <konrad.wilk@oracle.com> wrote:
> *  Xen multiboot2-EFI support (fair)
>    Needed for SecureBoot
>    Depends on Xen Boot info (rework multiboot and other structs)
>    See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html 
>    RFC posted
>   -  Daniel Kiper

I think the qualification above should be changed from "Needed for
SecureBoot" to "Needed to work with GrUB2" - after all, secure
booting works quite fine without that code in place.

> *  1TB slow destruction (ok)
>   -  Bob Liu

I haven't seen any progress here for quite a while, and based on the
last iteration I don't think this will get ready in time.

> *  Support controlling the max C-state sub-state (ok)
>    v3 posted
>   -  Ross Lagerwall

This is stalled on a conceptional issue (it being unclear how to
express this via a meaningful command line option value).

Jan

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Fri, Aug 15, 2014 at 08:40:35PM +0100, Jan Beulich wrote:
> >>> On 15.08.14 at 19:23, <konrad.wilk@oracle.com> wrote:
> > *  Xen multiboot2-EFI support (fair)
> >    Needed for SecureBoot
> >    Depends on Xen Boot info (rework multiboot and other structs)
> >    See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html 
> >    RFC posted
> >   -  Daniel Kiper
> 
> I think the qualification above should be changed from "Needed for
> SecureBoot" to "Needed to work with GrUB2" - after all, secure
> booting works quite fine without that code in place.

<nods> Is there an document describing how to do that? (I presume
you need to build xen.efi with a signature?)

> 
> > *  1TB slow destruction (ok)
> >   -  Bob Liu
> 
> I haven't seen any progress here for quite a while, and based on the
> last iteration I don't think this will get ready in time.

Good point. Moving it to defered.
> 
> > *  Support controlling the max C-state sub-state (ok)
> >    v3 posted
> >   -  Ross Lagerwall
> 
> This is stalled on a conceptional issue (it being unclear how to
> express this via a meaningful command line option value).

OK, moving to defered.
> 
> Jan
> 

Re: Xen 4.5 development update (July update)

[Jan Beulich]

>>> On 15.08.14 at 21:46, <konrad.wilk@oracle.com> wrote:
> On Fri, Aug 15, 2014 at 08:40:35PM +0100, Jan Beulich wrote:
>> >>> On 15.08.14 at 19:23, <konrad.wilk@oracle.com> wrote:
>> > *  Xen multiboot2-EFI support (fair)
>> >    Needed for SecureBoot
>> >    Depends on Xen Boot info (rework multiboot and other structs)
>> >    See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html 
>> >    RFC posted
>> >   -  Daniel Kiper
>> 
>> I think the qualification above should be changed from "Needed for
>> SecureBoot" to "Needed to work with GrUB2" - after all, secure
>> booting works quite fine without that code in place.
> 
> <nods> Is there an document describing how to do that?

I don't know of one, but that doesn't mean much. Iirc this
involves a small custom patch to the boot loader unless you'd
set things up so the shim would load xen.efi directly (not
sure whether that actually works).

> (I presume you need to build xen.efi with a signature?)

Yes.

Jan

Re: Xen 4.5 development update (July update)

[Meng Xu]

[-- Attachment #1.1: Type: text/plain, Size: 1102 bytes --]

>
> The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
> A bunch of items had moved to the completed phase which is fantastic.
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>
>
*  XenRT (Preemptive Global Earliest Deadline First) (fair)
>    RFC patch posted (v2)
>   -  Meng Xu
>

​The RFC patch v3 will be released next week and the RFC patch v4 (last
one) will be released ​before the Xen 4.5 feature freeze. both of the
following two patches are focusing on improving the performance of the
scheduler.

Although it's a RFC patch, the XenRT can actually works well on the
machines we have. It currently has the preemptive global EDF scheduler. The
functionalities I'm currently working on is improving its performance based
on the review of the first version of patches.

Thanks,

Meng

-- 


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

[-- Attachment #1.2: Type: text/html, Size: 1953 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 2164 bytes --]

On ven, 2014-08-15 at 20:31 -0400, Meng Xu wrote:
> 
> 
>         
>         The "prognosis" is now the likelihood of completion in the 4.5
>         timeframe.
>         A bunch of items had moved to the completed phase which is
>         fantastic.
>         
>         none - nothing yet
>         fair - still working on it, patches are prototypes or RFC
>         ok   - patches posted, acting on review
>         good - some last minute pieces
>         done - all done, might have bugs
>         
> 
> 
>         *  XenRT (Preemptive Global Earliest Deadline First) (fair)
>            RFC patch posted (v2)
>           -  Meng Xu
> 

> Although it's a RFC patch, the XenRT can actually works well on the
> machines we have. It currently has the preemptive global EDF
> scheduler. The functionalities I'm currently working on is improving
> its performance based on the review of the first version of patches. 
> 
If this is the case, I think you should drop the RFC tag/status.

The idea is this: if you are asking the Xen dev community to provide
some feedback on an idea/prototype implementation, then keep the RFC
tag. If you are proposing something for inclusion in upstream Xen and,
at least according to your own opinion, the code you are submitting is
good enough to be included, then you need to drop the RFC tag.

Then, of course, there will still be reviews, changes will be requested,
new versions, etc., but it's important to mark the difference between
these two phases, as said, by either retaining or dripping the tag.

Basing on what you're saying (i.e., that you're working on performances
optimizations), on how the last patch series looked and on the comments
you got, I think it would be fine for you to send the next round of
patches as proper v1, rather than "RFC v3", but that's, of course, up to
you. :-)

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Meng Xu]

[-- Attachment #1.1: Type: text/plain, Size: 2128 bytes --]

Hi Dario,


> >
> >         The "prognosis" is now the likelihood of completion in the 4.5
> >         timeframe.
> >         A bunch of items had moved to the completed phase which is
> >         fantastic.
> >
> >         none - nothing yet
> >         fair - still working on it, patches are prototypes or RFC
> >         ok   - patches posted, acting on review
> >         good - some last minute pieces
> >         done - all done, might have bugs
> >
> >
> >
> >         *  XenRT (Preemptive Global Earliest Deadline First) (fair)
> >            RFC patch posted (v2)
> >           -  Meng Xu
> >
>
> > Although it's a RFC patch, the XenRT can actually works well on the
> > machines we have. It currently has the preemptive global EDF
> > scheduler. The functionalities I'm currently working on is improving
> > its performance based on the review of the first version of patches.
> >
> If this is the case, I think you should drop the RFC tag/status.
>
> The idea is this: if you are asking the Xen dev community to provide
> some feedback on an idea/prototype implementation, then keep the RFC
> tag. If you are proposing something for inclusion in upstream Xen and,
> at least according to your own opinion, the code you are submitting is
> good enough to be included, then you need to drop the RFC tag.
>
> Then, of course, there will still be reviews, changes will be requested,
> new versions, etc., but it's important to mark the difference between
> these two phases, as said, by either retaining or dripping the tag.
>
> Basing on what you're saying (i.e., that you're working on performances
> optimizations), on how the last patch series looked and on the comments
> you got, I think it would be fine for you to send the next round of
> patches as proper v1, rather than "RFC v3", but that's, of course, up to
> you. :-)
>

​Thank you very much for your advice! I will drop the RFC then in the next
round of patches (this week).

Best,

Meng ​

-- 


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

[-- Attachment #1.2: Type: text/html, Size: 3041 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1534 bytes --]

On ven, 2014-08-15 at 13:23 -0400, konrad.wilk@oracle.com wrote:

> = Open =

> == x86 == 

> *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
>    kernbench demonstrated it
>    Looking and tracing
>    http://www.gossamer-threads.com/lists/xen/devel/339409
>   -  Dario Faggioli
> 
Mmm... I realize only now that I must have missed your last email about
this... So, sorry.

In any case, didn't we settle on closing it? The bullet is here because
very high perf degradation between baremetal and HVM DomU was seen when
running `kernbench -j' (i.e., kernbench with unlimited number of build
jobs) with HT enabled.

Both me and other people tried to reproduce the issue, and here's the
number:

HT Enabled
==========
                 Host               Guest
kernbench -j4    31.598 (0.113666)  34.078 (0.168582)
kernbench -j8    26.676 (0.16772)   26.672 (0.0432435)
kernbench -j     27.674 (0.933584)  27.49 (0.364897)

To me, this means we don't have an issue of high virtualization overhead
when HT is enabled (look at the last row!).

All the other things that came up in previous threads are not to be
ignored or forgot, of course, and I won't, but they don't have anything
to do with this.

Let me know,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Tue, Aug 19, 2014 at 06:46:51PM +0200, Dario Faggioli wrote:
> On ven, 2014-08-15 at 13:23 -0400, konrad.wilk@oracle.com wrote:
> 
> > = Open =
> 
> > == x86 == 
> 
> > *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
> >    kernbench demonstrated it
> >    Looking and tracing
> >    http://www.gossamer-threads.com/lists/xen/devel/339409
> >   -  Dario Faggioli
> > 
> Mmm... I realize only now that I must have missed your last email about
> this... So, sorry.
> 
> In any case, didn't we settle on closing it? The bullet is here because
> very high perf degradation between baremetal and HVM DomU was seen when
> running `kernbench -j' (i.e., kernbench with unlimited number of build
> jobs) with HT enabled.
> 
> Both me and other people tried to reproduce the issue, and here's the
> number:
> 
> HT Enabled
> ==========
>                  Host               Guest
> kernbench -j4    31.598 (0.113666)  34.078 (0.168582)
> kernbench -j8    26.676 (0.16772)   26.672 (0.0432435)
> kernbench -j     27.674 (0.933584)  27.49 (0.364897)
> 
> To me, this means we don't have an issue of high virtualization overhead
> when HT is enabled (look at the last row!).

Ok, so false alarm. Thank you!
> 
> All the other things that came up in previous threads are not to be
> ignored or forgot, of course, and I won't, but they don't have anything
> to do with this.
> 
> Let me know,
> Dario
> 
> -- 
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
> 

Xen 4.5 development update (July update)

[konrad.wilk]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 13448 bytes --]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is less a week away!

Most of the Xen patches that impacted the hypervisor and had'fair'
(except the HVM introspection one) I moved to the Xen 4.6 list.

They might get Acked by maintainers in the next couple of days which would
be fantastic - and if so I will update the list. But perhaps not. Also
some are in 'good' or in 'ok' condition - but that does not mean they
will automatically go in Xen 4.5.

In terms of QEMU  - I only had three items and since the version of QEMU
from upstream we are using is already established (and stable) I don't see us
backporting any more patches from upstream. But perhaps Stefano has some
other plans...

In terms of the toolstack - I kept the ones that are in the 'fair' category
as I think they are easier to test/rebase/retest than the hypervisor ones.

In terms of Linux I am keeping the 'fair' ones as by the Xen 4.6 release
it could be v3.19, which means there is still an upcoming merge window
for those.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
A bunch of items had moved to the completed phase which is fantastic.

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs

For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

In terms of libvirt it has monthly releases. As such not going to track
every release - but closer to when RCs are out.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Coding time: <=== NOW, one week left!

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

If you think your patchset MUST go in Xen 4.5 I will post the procedure
for requesting an exception to get them in past the feature freeze next
week.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM - Device assigment on ARM (good)
   Linux parts at risk.
   v2 for hypervisor out
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
   v12 posted.
  -  Arianna Avanzini

*  ARM Xen UEFI booting on ARM (ok)
   v2
  -  Roy Franz

*  ARM PSCI v0.2 (good)
   v11 posted
  -  Parth Dixit

*  ARM GICv3 support (ok)
   v6a patchset (refactor in, also known as v9a); v7 posted
   v9a posted which does GIC and VGIC code refactoring
   v8 posted
  -  Vijay Kilari

*  ARM - MiniOS (ok)
   v7 posted
  -  Thomas Leonard

*  ARM - VGIC emulation (ok)
   Reposted as gic and vgic fixes and improvements
   v12
  -  Stefano Stabellini

*  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (ok)
   Provide kernels an grant->MFN lookup
   v4
  -  Stefano Stabellini

*  ARM - Add Odroid-XU (Exynos5410) support (ok)
   v4
  -  Suriyan Ramasami

*  ARM implement mem_access (ok)
   v3
   https://github.com/tklengyel/xen/tree/arm_memaccess3
  -  Tamas K Lengyel

== x86 ==

*  New Migration (v2). (good)
   v6 posted, going on the 'good' side!
  -  Andrew Cooper & David Vrabel

*  PVH - AMD hardware support. (ok)
   Posted.
  -  Mukesh Rathor

*  VMware backdoor (hypercall) (ok)
   v2 posted.
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
   v9 posted
  -  Boris Ostrovsky

*  Cache QoS Monitoring - hypercalls (ok)
   Just hypercalls - no toolstack changes.
   v14
  -  Chao Peng, Dongxiao Xu, and Shantong Kang

*  XenRT (Preemptive Global Earliest Deadline First) (ok)
   v3
  -  Meng Xu

*  Introspection of HVM guests (ok)
   RFC v11 (dropped the RFC in the list)
  -  Razvan Cojocaru

*  vNUMA in Xen (ok)
   v9 posted
   git://gitorious.org/vnuma/xen_vnuma.git:v9
  -  Elena Ufimtseva

== lib{xc,xl} and toolstack ==

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  Remus in Xen (libxl) (ok)
   v19
   url: https://github.com/laijs/xen remus-v19
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
   v6 posted
   git://gitorious.org/vnuma/linux_vnuma.git
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  vAPIC in PVHVM guests (Linux side) (none)
  -  Boris Ostrovsky

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

*  mini-os: xenbus changes for rump kernels (ok)
   git://xenbits.xen.org/people/iwj/rumpuser-xen.git
   branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
   v2 posted
  -  Ian Jackson

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to QEMU v2.next ==

*  Using qemu-upstream in a stubdomain (fair)
   Will use rump kernels.
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   v5 posted
  -  Chen, Tiejun

*  Xen PV block driver in OVMF (UEFI in guest) (fair)
   RFC posted.
  -  Anthony PERARD

== Deferred to Xen hypervisor 4.6 ==

*  extending mem_access support to PV domain (fair)
   RFC v2
  -  Aravindh Puthiyaparambil (aravindp)

*  Repurpose SEDF Scheduler for Real-time (fair)
   RFC patch posted (v2)
  -  Joshua Whitehead, Robert VanVossen

*  ARM remote processor iommu module (GPUs + IPUs) (fair)
   v3 posted
  -  Andrii Tseglytskyi

*  COarse-grain LOck-stepping Virtual Machines in Xen (fair)
   RFC patch posted
  -  Wen Congyang
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  extend the xenstore ring with a 'closing' signal (fair)
   RFC patch posted
  -  David Scott

*  libxl migrationv2 patches. (none)
  -  Andrew Cooper & David Vrabel

*  tmem migrationv2 patches. (none)
  -  Bob Liu & Andrew Cooper & David Vrabel

*  Remus using migration-v2 (fair)
   RFC posted - depends on v6 of 'New Migration'
  -  Yang Hongyang

*  Xen multiboot2-EFI support (fair)
   Needed for GrUB2
   Depends on Xen Boot info (rework multiboot and other structs)
   See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html
   RFC posted
  -  Daniel Kiper

*  dirty vram / IOMMU bug (fair)
   http://bugs.xenproject.org/xen/bug/38
  -  Zhang, Yang Z

*  snapshot API extension (checkpointing disk) (ok)
   v5
   His email bounces.
  -  Bamvor Jian Zhang

*  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Support controlling the max C-state sub-state (ok)
   v3 posted
  -  Ross Lagerwall

*  1TB slow destruction (ok)
  -  Bob Liu

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  Xen HPET interrupt fixes (fair)
   behind migration v2
  -  Andrew Cooper

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  ARM VM save/restore/live migration (none)
   Need to rebased against migrationv2 - no code posted.
  -  Junghyun Yoo

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  cpuid leveling (none)
   http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
  -  Ian Jackson

*  IO-NUMA - hwloc and xl (none)
   Andrew Cooper had an RFC patch for hwloc
   add restrictions  as to which devices cannot safely/functionally be split apart.
  -  Boris Ostrovsky

*  HVM guest NUMA (none)
  -  Matt Wilson

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  ARM GICv2m support (none)
  -  Linaro (unknown)

== Up for grabs ==

*  OSSTest - also test Linux PVH guests

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  pvSCSI in Linux (fronted and backend) (done)
   v6
  -  Juergen Gross

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of vHPET (done)
   aka hvm/hpet: Detect comparator values in the past
  -  Don Slutz

*  vAPIC in PVHVM guests (Xen side) (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)

*  multiple AMD container files appended together in initrd (early initramfs)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk

*  ARM  - IOMMU support (done)
  -  Julien Grall

*  ioreq-server, aka secondary emulators (done)
  -  Paul Durrant

*  Netback multiqueue (good)
  -  Wei Liu

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  Bigger PCI hole in QEMU (done)
   Needs to be rebased
  -  Don Slutz

*  ARM DRA7 support (done)
   v3 posted
   v3 with comments applied
  -  Andrii Tseglytskyi

*  rework VM Generation ID (done)
   v7 posted
  -  David Vrabel

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
   v11 posted
  -  Dario Faggioli

*  Linux pvops of Xen EFI hypercall support (done)
  -  Daniel Kiper

*  ARM: Use super pages in p2m (done)
   v5 posted
  -  Ian Campbell

*  QEMU 2.0 branch for qemu-upstream (done)
   It is v2.0 with 2.1 Xen backports.
  -  Stefano Stabellini

*  systemd support (done)
   v11
  -  Luis R. Rodriguez

*  Soft affinity for vcpus libxl/xl changes (done)
   v13 posted
  -  Dario Faggioli

*  HT enabled, virtualization overhead is high (Xen 4.4) (done)
   kernbench demonstrated it
   Looking and tracing
   http://www.gossamer-threads.com/lists/xen/devel/339409
   False alarm.
  -  Dario Faggioli



[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Andrew Cooper]

On 02/09/2014 21:45, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame.  The tentative feature freeze is scheduled for September 10th,
> which is less a week away!
>
> Most of the Xen patches that impacted the hypervisor and had'fair'
> (except the HVM introspection one) I moved to the Xen 4.6 list.
>
> They might get Acked by maintainers in the next couple of days which would
> be fantastic - and if so I will update the list. But perhaps not. Also
> some are in 'good' or in 'ok' condition - but that does not mean they
> will automatically go in Xen 4.5.
>
> In terms of QEMU  - I only had three items and since the version of QEMU
> from upstream we are using is already established (and stable) I don't see us
> backporting any more patches from upstream. But perhaps Stefano has some
> other plans...
>
> In terms of the toolstack - I kept the ones that are in the 'fair' category
> as I think they are easier to test/rebase/retest than the hypervisor ones.
>
> In terms of Linux I am keeping the 'fair' ones as by the Xen 4.6 release
> it could be v3.19, which means there is still an upcoming merge window
> for those.
>
> The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
> A bunch of items had moved to the completed phase which is fantastic.
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
>
> In terms of libvirt it has monthly releases. As such not going to track
> every release - but closer to when RCs are out.
>
> = Timeline =
>
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
>
> * Coding time: <=== NOW, one week left!
>
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
>
> If you think your patchset MUST go in Xen 4.5 I will post the procedure
> for requesting an exception to get them in past the feature freeze next
> week.
>
> = Prognosis =
>
> The states are: none -> fair -> ok -> good -> done
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>
>
> = Open =
>
> == ARM == 
>
> *  ARM - Device assigment on ARM (good)
>    Linux parts at risk.
>    v2 for hypervisor out
>   -  Julien Grall
>
> *  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
>    v12 posted.
>   -  Arianna Avanzini
>
> *  ARM Xen UEFI booting on ARM (ok)
>    v2
>   -  Roy Franz
>
> *  ARM PSCI v0.2 (good)
>    v11 posted
>   -  Parth Dixit
>
> *  ARM GICv3 support (ok)
>    v6a patchset (refactor in, also known as v9a); v7 posted
>    v9a posted which does GIC and VGIC code refactoring
>    v8 posted
>   -  Vijay Kilari
>
> *  ARM - MiniOS (ok)
>    v7 posted
>   -  Thomas Leonard
>
> *  ARM - VGIC emulation (ok)
>    Reposted as gic and vgic fixes and improvements
>    v12
>   -  Stefano Stabellini
>
> *  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (ok)
>    Provide kernels an grant->MFN lookup
>    v4
>   -  Stefano Stabellini
>
> *  ARM - Add Odroid-XU (Exynos5410) support (ok)
>    v4
>   -  Suriyan Ramasami
>
> *  ARM implement mem_access (ok)
>    v3
>    https://github.com/tklengyel/xen/tree/arm_memaccess3
>   -  Tamas K Lengyel
>
> == x86 == 
>
> *  New Migration (v2). (good)
>    v6 posted, going on the 'good' side!
>   -  Andrew Cooper & David Vrabel

Work on the {lib,}xl side of things are making progress.

Given the lack of stream versioning, *even* at the libxl level, both
sets of changes need to go in together, so there is no chance of
deferring the libxl work.

PV is working and HVM is getting there.  Currently have a lot of
compatibility hacks until Wei's domain json series gets accepted (or
cherrypicked into my queue).  Also, making the restore side asynchronous
is proving very difficult.

Will also require some careful integration with remus.  I believe my
libxc series now contains all the remus-specific bugfixes discovered,
but I have yet to have feedback.

~Andrew

Re: Xen 4.5 development update (July update)

[Ian Campbell]

On Tue, 2014-09-02 at 22:25 +0100, Andrew Cooper wrote:
> > *  New Migration (v2). (good)
> >    v6 posted, going on the 'good' side!
> >   -  Andrew Cooper & David Vrabel
> 
> Work on the {lib,}xl side of things are making progress.
> 
> Given the lack of stream versioning, *even* at the libxl level, both
> sets of changes need to go in together, so there is no chance of
> deferring the libxl work.

There is a protocol at the xl level which includes a (string)
indentifier which could be used if needed. See savefileheader_magic[] in
xl_cmdimpl.c, e.g. Wei modifies this as part of his changes to stream
the guest CFG as json instead of as an xl.cfg.

Ian.

Re: Xen 4.5 development update (July update)

[Wei Liu]

On Wed, Sep 03, 2014 at 11:09:54AM +0100, Ian Campbell wrote:
> On Tue, 2014-09-02 at 22:25 +0100, Andrew Cooper wrote:
> > > *  New Migration (v2). (good)
> > >    v6 posted, going on the 'good' side!
> > >   -  Andrew Cooper & David Vrabel
> > 
> > Work on the {lib,}xl side of things are making progress.
> > 
> > Given the lack of stream versioning, *even* at the libxl level, both
> > sets of changes need to go in together, so there is no chance of
> > deferring the libxl work.
> 
> There is a protocol at the xl level which includes a (string)
> indentifier which could be used if needed. See savefileheader_magic[] in
> xl_cmdimpl.c, e.g. Wei modifies this as part of his changes to stream
> the guest CFG as json instead of as an xl.cfg.

FWIW I don't modify the magic string but a feature bit of the stream
header. :-)

Wei.

> 
> Ian.

Re: Xen 4.5 development update (July update)

[Andrew Cooper]

On 03/09/14 11:09, Ian Campbell wrote:
> On Tue, 2014-09-02 at 22:25 +0100, Andrew Cooper wrote:
>>> *  New Migration (v2). (good)
>>>    v6 posted, going on the 'good' side!
>>>   -  Andrew Cooper & David Vrabel
>> Work on the {lib,}xl side of things are making progress.
>>
>> Given the lack of stream versioning, *even* at the libxl level, both
>> sets of changes need to go in together, so there is no chance of
>> deferring the libxl work.
> There is a protocol at the xl level which includes a (string)
> indentifier which could be used if needed. See savefileheader_magic[] in
> xl_cmdimpl.c, e.g. Wei modifies this as part of his changes to stream
> the guest CFG as json instead of as an xl.cfg.
>
> Ian.
>

That is fine for xl, but not libxl which lacks any protocol version. 
Consumers of the libxl api other than xl currently have an opaque handle
to raw libxc stream and no metadata.

I have just published draft A of the libxl stream specification, which
should hopefully clear up what is trying to be done along these lines.

~Andrew

Re: Xen 4.5 development update (July update)

[Shriram Rajagopalan]

[-- Attachment #1.1: Type: text/plain, Size: 5263 bytes --]

Andrew,

On Sep 2, 2014 5:29 PM, "Andrew Cooper" <andrew.cooper3@citrix.com> wrote:
>
> On 02/09/2014 21:45, konrad.wilk@oracle.com wrote:
> > Below is a summary of the projects / features being worked on for the
4.5
> > time frame.  The tentative feature freeze is scheduled for September
10th,
> > which is less a week away!
> >
> > Most of the Xen patches that impacted the hypervisor and had'fair'
> > (except the HVM introspection one) I moved to the Xen 4.6 list.
> >
> > They might get Acked by maintainers in the next couple of days which
would
> > be fantastic - and if so I will update the list. But perhaps not. Also
> > some are in 'good' or in 'ok' condition - but that does not mean they
> > will automatically go in Xen 4.5.
> >
> > In terms of QEMU  - I only had three items and since the version of QEMU
> > from upstream we are using is already established (and stable) I don't
see us
> > backporting any more patches from upstream. But perhaps Stefano has some
> > other plans...
> >
> > In terms of the toolstack - I kept the ones that are in the 'fair'
category
> > as I think they are easier to test/rebase/retest than the hypervisor
ones.
> >
> > In terms of Linux I am keeping the 'fair' ones as by the Xen 4.6 release
> > it could be v3.19, which means there is still an upcoming merge window
> > for those.
> >
> > The "prognosis" is now the likelihood of completion in the 4.5
timeframe.
> > A bunch of items had moved to the completed phase which is fantastic.
> >
> > none - nothing yet
> > fair - still working on it, patches are prototypes or RFC
> > ok   - patches posted, acting on review
> > good - some last minute pieces
> > done - all done, might have bugs
> >
> > For items involving code hosted on the Xen.org site (including
qemu-xen),
> > that means a likelihood of having the feature code-complete and mostly
> > working by the feature freeze.  (It's OK if there are still bugs to be
> > worked out.)  For items in Linux, it would mean having items on track
> > to make it into the kernel released just after the scheduled 4.5 time
frame.
> >
> > In terms of libvirt it has monthly releases. As such not going to track
> > every release - but closer to when RCs are out.
> >
> > = Timeline =
> >
> > We are planning on a 9-month release cycle.  Based on that, below are
> > our estimated dates:
> >
> > * Coding time: <=== NOW, one week left!
> >
> > * Feature Freeze: 10th September 2014
> > * First RC: 10th October
> > * Release: 10th December 2014
> >
> > The RCs and release will of course depend on stability and bugs, and
> > will therefore be fairly unpredictable.  The feature freeze may be
> > slipped for especially important features which are near completion.
> >
> > If you think your patchset MUST go in Xen 4.5 I will post the procedure
> > for requesting an exception to get them in past the feature freeze next
> > week.
> >
> > = Prognosis =
> >
> > The states are: none -> fair -> ok -> good -> done
> >
> > none - nothing yet
> > fair - still working on it, patches are prototypes or RFC
> > ok   - patches posted, acting on review
> > good - some last minute pieces
> > done - all done, might have bugs
> >
> >
> > = Open =
> >
> > == ARM ==
> >
> > *  ARM - Device assigment on ARM (good)
> >    Linux parts at risk.
> >    v2 for hypervisor out
> >   -  Julien Grall
> >
> > *  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
> >    v12 posted.
> >   -  Arianna Avanzini
> >
> > *  ARM Xen UEFI booting on ARM (ok)
> >    v2
> >   -  Roy Franz
> >
> > *  ARM PSCI v0.2 (good)
> >    v11 posted
> >   -  Parth Dixit
> >
> > *  ARM GICv3 support (ok)
> >    v6a patchset (refactor in, also known as v9a); v7 posted
> >    v9a posted which does GIC and VGIC code refactoring
> >    v8 posted
> >   -  Vijay Kilari
> >
> > *  ARM - MiniOS (ok)
> >    v7 posted
> >   -  Thomas Leonard
> >
> > *  ARM - VGIC emulation (ok)
> >    Reposted as gic and vgic fixes and improvements
> >    v12
> >   -  Stefano Stabellini
> >
> > *  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (ok)
> >    Provide kernels an grant->MFN lookup
> >    v4
> >   -  Stefano Stabellini
> >
> > *  ARM - Add Odroid-XU (Exynos5410) support (ok)
> >    v4
> >   -  Suriyan Ramasami
> >
> > *  ARM implement mem_access (ok)
> >    v3
> >    https://github.com/tklengyel/xen/tree/arm_memaccess3
> >   -  Tamas K Lengyel
> >
> > == x86 ==
> >
> > *  New Migration (v2). (good)
> >    v6 posted, going on the 'good' side!
> >   -  Andrew Cooper & David Vrabel
>
> Work on the {lib,}xl side of things are making progress.
>
> Given the lack of stream versioning, *even* at the libxl level, both
> sets of changes need to go in together, so there is no chance of
> deferring the libxl work.
>
> PV is working and HVM is getting there.  Currently have a lot of
> compatibility hacks until Wei's domain json series gets accepted (or
> cherrypicked into my queue).  Also, making the restore side asynchronous
> is proving very difficult.
>
> Will also require some careful integration with remus.  I believe my
> libxc series now contains all the remus-specific bugfixes discovered,
> but I have yet to have feedback.
>

Sorry about this. Can you point me to the specific patches that you need
feedback on?

Shriram

[-- Attachment #1.2: Type: text/html, Size: 7218 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Andrew Cooper]

[-- Attachment #1.1: Type: text/plain, Size: 6006 bytes --]

On 03/09/14 12:53, Shriram Rajagopalan wrote:
>
> Andrew,
>
> On Sep 2, 2014 5:29 PM, "Andrew Cooper" <andrew.cooper3@citrix.com
> <mailto:andrew.cooper3@citrix.com>> wrote:
> >
> > On 02/09/2014 21:45, konrad.wilk@oracle.com
> <mailto:konrad.wilk@oracle.com> wrote:
> > > Below is a summary of the projects / features being worked on for
> the 4.5
> > > time frame.  The tentative feature freeze is scheduled for
> September 10th,
> > > which is less a week away!
> > >
> > > Most of the Xen patches that impacted the hypervisor and had'fair'
> > > (except the HVM introspection one) I moved to the Xen 4.6 list.
> > >
> > > They might get Acked by maintainers in the next couple of days
> which would
> > > be fantastic - and if so I will update the list. But perhaps not. Also
> > > some are in 'good' or in 'ok' condition - but that does not mean they
> > > will automatically go in Xen 4.5.
> > >
> > > In terms of QEMU  - I only had three items and since the version
> of QEMU
> > > from upstream we are using is already established (and stable) I
> don't see us
> > > backporting any more patches from upstream. But perhaps Stefano
> has some
> > > other plans...
> > >
> > > In terms of the toolstack - I kept the ones that are in the 'fair'
> category
> > > as I think they are easier to test/rebase/retest than the
> hypervisor ones.
> > >
> > > In terms of Linux I am keeping the 'fair' ones as by the Xen 4.6
> release
> > > it could be v3.19, which means there is still an upcoming merge window
> > > for those.
> > >
> > > The "prognosis" is now the likelihood of completion in the 4.5
> timeframe.
> > > A bunch of items had moved to the completed phase which is fantastic.
> > >
> > > none - nothing yet
> > > fair - still working on it, patches are prototypes or RFC
> > > ok   - patches posted, acting on review
> > > good - some last minute pieces
> > > done - all done, might have bugs
> > >
> > > For items involving code hosted on the Xen.org site (including
> qemu-xen),
> > > that means a likelihood of having the feature code-complete and mostly
> > > working by the feature freeze.  (It's OK if there are still bugs to be
> > > worked out.)  For items in Linux, it would mean having items on track
> > > to make it into the kernel released just after the scheduled 4.5
> time frame.
> > >
> > > In terms of libvirt it has monthly releases. As such not going to
> track
> > > every release - but closer to when RCs are out.
> > >
> > > = Timeline =
> > >
> > > We are planning on a 9-month release cycle.  Based on that, below are
> > > our estimated dates:
> > >
> > > * Coding time: <=== NOW, one week left!
> > >
> > > * Feature Freeze: 10th September 2014
> > > * First RC: 10th October
> > > * Release: 10th December 2014
> > >
> > > The RCs and release will of course depend on stability and bugs, and
> > > will therefore be fairly unpredictable.  The feature freeze may be
> > > slipped for especially important features which are near completion.
> > >
> > > If you think your patchset MUST go in Xen 4.5 I will post the
> procedure
> > > for requesting an exception to get them in past the feature freeze
> next
> > > week.
> > >
> > > = Prognosis =
> > >
> > > The states are: none -> fair -> ok -> good -> done
> > >
> > > none - nothing yet
> > > fair - still working on it, patches are prototypes or RFC
> > > ok   - patches posted, acting on review
> > > good - some last minute pieces
> > > done - all done, might have bugs
> > >
> > >
> > > = Open =
> > >
> > > == ARM ==
> > >
> > > *  ARM - Device assigment on ARM (good)
> > >    Linux parts at risk.
> > >    v2 for hypervisor out
> > >   -  Julien Grall
> > >
> > > *  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
> > >    v12 posted.
> > >   -  Arianna Avanzini
> > >
> > > *  ARM Xen UEFI booting on ARM (ok)
> > >    v2
> > >   -  Roy Franz
> > >
> > > *  ARM PSCI v0.2 (good)
> > >    v11 posted
> > >   -  Parth Dixit
> > >
> > > *  ARM GICv3 support (ok)
> > >    v6a patchset (refactor in, also known as v9a); v7 posted
> > >    v9a posted which does GIC and VGIC code refactoring
> > >    v8 posted
> > >   -  Vijay Kilari
> > >
> > > *  ARM - MiniOS (ok)
> > >    v7 posted
> > >   -  Thomas Leonard
> > >
> > > *  ARM - VGIC emulation (ok)
> > >    Reposted as gic and vgic fixes and improvements
> > >    v12
> > >   -  Stefano Stabellini
> > >
> > > *  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (ok)
> > >    Provide kernels an grant->MFN lookup
> > >    v4
> > >   -  Stefano Stabellini
> > >
> > > *  ARM - Add Odroid-XU (Exynos5410) support (ok)
> > >    v4
> > >   -  Suriyan Ramasami
> > >
> > > *  ARM implement mem_access (ok)
> > >    v3
> > >    https://github.com/tklengyel/xen/tree/arm_memaccess3
> > >   -  Tamas K Lengyel
> > >
> > > == x86 ==
> > >
> > > *  New Migration (v2). (good)
> > >    v6 posted, going on the 'good' side!
> > >   -  Andrew Cooper & David Vrabel
> >
> > Work on the {lib,}xl side of things are making progress.
> >
> > Given the lack of stream versioning, *even* at the libxl level, both
> > sets of changes need to go in together, so there is no chance of
> > deferring the libxl work.
> >
> > PV is working and HVM is getting there.  Currently have a lot of
> > compatibility hacks until Wei's domain json series gets accepted (or
> > cherrypicked into my queue).  Also, making the restore side asynchronous
> > is proving very difficult.
> >
> > Will also require some careful integration with remus.  I believe my
> > libxc series now contains all the remus-specific bugfixes discovered,
> > but I have yet to have feedback.
> >
>
> Sorry about this. Can you point me to the specific patches that you
> need feedback on?
>
> Shriram
>

http://xenbits.xen.org/gitweb/?p=people/andrewcoop/xen.git;a=shortlog;h=refs/heads/saverestore2-v6.2

should contain all the underlying bugfixes identified from the remus
libxc patch series.  It also contains a few new functions which should
make the remus specific bits easier.

~Andrew

[-- Attachment #1.2: Type: text/html, Size: 9811 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Julien Grall]

Hi Konrad,

On 02/09/14 14:25, Andrew Cooper wrote:
>> == ARM ==
>>
>> *  ARM - Device assigment on ARM (good)
>>     Linux parts at risk.

I think you can move the Linux part to Xen 4.6. I plan to add support 
for a basic passthrough in Xen 4.5. It won't require modification in Linux.

>>     v2 for hypervisor out

I will try to send a new version when I will be back to the office.

As it will be after the feature freeze, I will send a separate email to 
request an exception for this feature.

>>    -  Julien Grall

>> *  ARM PSCI v0.2 (good)
>>     v11 posted
>>    -  Parth Dixit

The patch has been pushed last week on Xen unstable by Ian. I believe 
this can be moved to "done".

Regards,

-- 
Julien Grall

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Wed, Sep 03, 2014 at 12:54:01PM -0700, Julien Grall wrote:
> Hi Konrad,
> 
> On 02/09/14 14:25, Andrew Cooper wrote:
> >>== ARM ==
> >>
> >>*  ARM - Device assigment on ARM (good)
> >>    Linux parts at risk.
> 
> I think you can move the Linux part to Xen 4.6. I plan to add support for a
> basic passthrough in Xen 4.5. It won't require modification in Linux.

OK, for the Linux parts that do require modification - do they require
changes in the Xen hypervisor as well? Or is the basic passthrough support
that was added in Xen 4.5 well enough for it?
> 
> >>    v2 for hypervisor out
> 
> I will try to send a new version when I will be back to the office.

OK.
> 
> As it will be after the feature freeze, I will send a separate email to
> request an exception for this feature.
> 
> >>   -  Julien Grall
> 
> >>*  ARM PSCI v0.2 (good)
> >>    v11 posted
> >>   -  Parth Dixit
> 
> The patch has been pushed last week on Xen unstable by Ian. I believe this
> can be moved to "done".
> 
> Regards,
> 
> -- 
> Julien Grall

Re: Xen 4.5 development update (July update)

[Hongyang Yang]

Hello konrad,

在 09/03/2014 04:45 AM, konrad.wilk@oracle.com 写道:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame.  The tentative feature freeze is scheduled for September 10th,
> which is less a week away!
>
> Most of the Xen patches that impacted the hypervisor and had'fair'
> (except the HVM introspection one) I moved to the Xen 4.6 list.
>
> They might get Acked by maintainers in the next couple of days which would
> be fantastic - and if so I will update the list. But perhaps not. Also
> some are in 'good' or in 'ok' condition - but that does not mean they
> will automatically go in Xen 4.5.
>
> In terms of QEMU  - I only had three items and since the version of QEMU
> from upstream we are using is already established (and stable) I don't see us
> backporting any more patches from upstream. But perhaps Stefano has some
> other plans...
>
...snip...
>
> == lib{xc,xl} and toolstack ==
>
> *  libxl work - JSON to keep track of guest configs (ok)
>    -  Wei Liu
>
> *  pvscsi should be targeted for 4.5, a prototype exists (fair)
>    -  Olaf Hering
>
> *  Remus in Xen (libxl) (ok)
>     v19
>     url: https://github.com/laijs/xen remus-v19

Url is here now:
  https://github.com/macrosheep/xen/tree/remus-v19


>    -  Gui Jianfeng
>    -  Yang Hongyang
>    -  Dong, Eddie
>
> == Linux ==
>
> *  Linux block multiqueue (fair)
>    -  Arianna Avanzini
>
> *  Netback grant table manipulations (ok)
>    -  Zoltan Kiss
>
> *  VPMU - 'perf' support in Linux (ok)
>     Depends on Xen patches
>    -  Boris Ostrovsky
>
> *  vNUMA in Linux (ok)
>     v6 posted
>     git://gitorious.org/vnuma/linux_vnuma.git
>    -  Elena Ufimtseva
>
> *  vsyscall in Linux (fair)
>    -  Konrad Rzeszutek Wilk
>
> *  COLO Agent in Linux (fair)
>    -  Gui Jianfeng
>    -  Yang Hongyang
>    -  Dong, Eddie
>
> *  vAPIC in PVHVM guests (Linux side) (none)
>    -  Boris Ostrovsky
>
> == FreeBSD ==
>
> *  PVH FreeBSD dom0 (ok)
>     FreeBSD 11 goal. Toolstack side done in Xen 4.5
>    -  Roger Pau Monné
>
> == Other OSes (MiniOS, QNX) ==
>
> *  PV drivers for automotive kernels (fair)
>    -  Artem Mygaiev
>
> *  mini-os: xenbus changes for rump kernels (ok)
>     git://xenbits.xen.org/people/iwj/rumpuser-xen.git
>     branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
>     v2 posted
>    -  Ian Jackson
>
> == GRUB2 ==
>
> *  GRUB2 multiboot2 (fair)
>    -  Daniel Kiper
>
> == OSSTEST ==
>
> *  OSSTest: libvirt (good)
>    -  Ian Campbell
>
> == Deferred to QEMU v2.next ==
>
> *  Using qemu-upstream in a stubdomain (fair)
>     Will use rump kernels.
>    -  Ian Jackson
>
> *  Intel IGD PCI GPU passthrough (ok)
>     v5 posted
>    -  Chen, Tiejun
>
> *  Xen PV block driver in OVMF (UEFI in guest) (fair)
>     RFC posted.
>    -  Anthony PERARD
>
> == Deferred to Xen hypervisor 4.6 ==
>
> *  extending mem_access support to PV domain (fair)
>     RFC v2
>    -  Aravindh Puthiyaparambil (aravindp)
>
> *  Repurpose SEDF Scheduler for Real-time (fair)
>     RFC patch posted (v2)
>    -  Joshua Whitehead, Robert VanVossen
>
> *  ARM remote processor iommu module (GPUs + IPUs) (fair)
>     v3 posted
>    -  Andrii Tseglytskyi
>
> *  COarse-grain LOck-stepping Virtual Machines in Xen (fair)
>     RFC patch posted
>    -  Wen Congyang
>    -  Gui Jianfeng
>    -  Yang Hongyang
>    -  Dong, Eddie
>
> *  extend the xenstore ring with a 'closing' signal (fair)
>     RFC patch posted
>    -  David Scott
>
> *  libxl migrationv2 patches. (none)
>    -  Andrew Cooper & David Vrabel
>
> *  tmem migrationv2 patches. (none)
>    -  Bob Liu & Andrew Cooper & David Vrabel
>
> *  Remus using migration-v2 (fair)
>     RFC posted - depends on v6 of 'New Migration'
>    -  Yang Hongyang
>
> *  Xen multiboot2-EFI support (fair)
>     Needed for GrUB2
>     Depends on Xen Boot info (rework multiboot and other structs)
>     See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html
>     RFC posted
>    -  Daniel Kiper
>
> *  dirty vram / IOMMU bug (fair)
>     http://bugs.xenproject.org/xen/bug/38
>    -  Zhang, Yang Z
>
> *  snapshot API extension (checkpointing disk) (ok)
>     v5
>     His email bounces.
>    -  Bamvor Jian Zhang
>
> *  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
>    -  Daniel Kiper
>
> *  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
>    -  Daniel Kiper
>
> *  Support controlling the max C-state sub-state (ok)
>     v3 posted
>    -  Ross Lagerwall
>
> *  1TB slow destruction (ok)
>    -  Bob Liu
>
> *  IOMMU ABI for guests to map their DMA regions (fair)
>    -  Malcolm Crossley
>
> *  xl list --long (and some related xl commands) have some bugs (none)
>    -  Zhigang Wang
>
> *  Xen HPET interrupt fixes (fair)
>     behind migration v2
>    -  Andrew Cooper
>
> *  Convert tasklet to per-cpu tasklets (fair)
>    -  Konrad Rzeszutek Wilk
>
> *  ARM VM save/restore/live migration (none)
>     Need to rebased against migrationv2 - no code posted.
>    -  Junghyun Yoo
>
> *  Default to credit2 (none)
>     cpu pinning, numa affinity and cpu reservation
>    -  George Dunlap
>
> *  "Short" grant copy (just header) of packets. (none)
>    -  Zoltan Kiss
>
> *  cpuid leveling (none)
>     http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
>    -  Andrew Cooper
>
> *  live migration knobs, there is no suitable code yet, just ideas (none)
>      http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
>    -  Olaf Hering
>
> *  Further tmem cleanups/fixes (16TB etc) (fair)
>    -  Bob Liu
>
> *  AMD Radeon PCI GPU passthrough (none)
>     Focusing on Xen 4.2 and qemu-traditional
>    -  Kelly Zytaruk
>
> *  xl does not handle migrate interruption gracefully (none)
>     If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
>    -  Ian Jackson
>
> *  IO-NUMA - hwloc and xl (none)
>     Andrew Cooper had an RFC patch for hwloc
>     add restrictions  as to which devices cannot safely/functionally be split apart.
>    -  Boris Ostrovsky
>
> *  HVM guest NUMA (none)
>    -  Matt Wilson
>
> *  PVH - Migration of PVH DomUs. (none)
>     Depends on migration2 code
>    -  Roger Pau Monné
>
> *  PVH - Migration of guests from a PVH dom0  (none)
>     Depends on migration2 code
>    -  Roger Pau Monné
>
> *  ARM GICv2m support (none)
>    -  Linaro (unknown)
>
> == Up for grabs ==
>
> *  OSSTest - also test Linux PVH guests
>
> *  PoD fixes
>     if you boot with memory <= maxmem we have a size estimation bug
>
> *  TLB flushing without locks in Xen
>
> *  xl does not support specifying virtual function for passthrough device
>     http://bugs.xenproject.org/xen/bug/22
>
> *  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>     http://bugs.xenproject.org/xen/bug/28
>
> *  libx{c,l} error handling cleanup
>
> *  Adding missing 'xend' features in libxl
>
> *  xl list -l on a dom0-only system
>
> *  xl list -l doesn't contain tty console port
>
> *  xl: passing more defaults in configuration in xl.conf
>     There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.
>
> *  PVH - PVH working with shadow.
>     Based on Tim's work
>
> *  PVH - PCI passthrough for DomU.
>
> *  AMD performance regressions
>
> *  Performance due to hypercall preemption. More preemptions - slower. (none)
>
> == Completed ==
>
> *  pvSCSI in Linux (fronted and backend) (done)
>     v6
>    -  Juergen Gross
>
> *  alternative_asm in Xen (done)
>    -  Feng Wu
>
> *  SMAP (done)
>    -  Feng Wu
>
> *  Re-write of vHPET (done)
>     aka hvm/hpet: Detect comparator values in the past
>    -  Don Slutz
>
> *  vAPIC in PVHVM guests (Xen side) (done)
>    -  Boris Ostrovsky
>
> *  libvirt and xl discard support, so that libvirt can start using it (done)
>    -  Olaf Hering
>
> *  Xen PVH dom0 (done)
>    -  Mukesh Rathor
>
> *  Linux PVH dom0 (done)
>    -  Mukesh Rathor
>
> *  OSSTest: upstream QEMU (done)
>    -  Ian Campbell
>
> *  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)
>
> *  Data breakpoint Extension support (new-feat) (in master)
>
> *  Feature masking MSR support (enhancement) (in master)
>
> *  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)
>
> *  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)
>
> *  multiple AMD container files appended together in initrd (early initramfs)
>    -  Aravind and Suravee
>
> *  NUMA memory scrubbing (done)
>    -  Konrad Rzeszutek Wilk
>
> *  ARM  - IOMMU support (done)
>    -  Julien Grall
>
> *  ioreq-server, aka secondary emulators (done)
>    -  Paul Durrant
>
> *  Netback multiqueue (good)
>    -  Wei Liu
>
> *  ARM Interrupt latency reduction (no maintenance interrupts) (good)
>    -  Stefano Stabellini
>
> *  Bigger PCI hole in QEMU (done)
>     Needs to be rebased
>    -  Don Slutz
>
> *  ARM DRA7 support (done)
>     v3 posted
>     v3 with comments applied
>    -  Andrii Tseglytskyi
>
> *  rework VM Generation ID (done)
>     v7 posted
>    -  David Vrabel
>
> *  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
>     v11 posted
>    -  Dario Faggioli
>
> *  Linux pvops of Xen EFI hypercall support (done)
>    -  Daniel Kiper
>
> *  ARM: Use super pages in p2m (done)
>     v5 posted
>    -  Ian Campbell
>
> *  QEMU 2.0 branch for qemu-upstream (done)
>     It is v2.0 with 2.1 Xen backports.
>    -  Stefano Stabellini
>
> *  systemd support (done)
>     v11
>    -  Luis R. Rodriguez
>
> *  Soft affinity for vcpus libxl/xl changes (done)
>     v13 posted
>    -  Dario Faggioli
>
> *  HT enabled, virtualization overhead is high (Xen 4.4) (done)
>     kernbench demonstrated it
>     Looking and tracing
>     http://www.gossamer-threads.com/lists/xen/devel/339409
>     False alarm.
>    -  Dario Faggioli
>
> .
>

-- 
Thanks,
Yang.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Shriram Rajagopalan]

[-- Attachment #1.1: Type: text/plain, Size: 1668 bytes --]

Hi Konrad,
On Sep 2, 2014 9:20 PM, "Hongyang Yang" <yanghy@cn.fujitsu.com> wrote:
>
> Hello konrad,
>
> 在 09/03/2014 04:45 AM, konrad.wilk@oracle.com 写道:
>
>> Below is a summary of the projects / features being worked on for the 4.5
>> time frame.  The tentative feature freeze is scheduled for September
10th,
>> which is less a week away!
>>
>> Most of the Xen patches that impacted the hypervisor and had'fair'
>> (except the HVM introspection one) I moved to the Xen 4.6 list.
>>
>> They might get Acked by maintainers in the next couple of days which
would
>> be fantastic - and if so I will update the list. But perhaps not. Also
>> some are in 'good' or in 'ok' condition - but that does not mean they
>> will automatically go in Xen 4.5.
>>
>> In terms of QEMU  - I only had three items and since the version of QEMU
>> from upstream we are using is already established (and stable) I don't
see us
>> backporting any more patches from upstream. But perhaps Stefano has some
>> other plans...
>>
> ...snip...
>
>>
>> == lib{xc,xl} and toolstack ==
>>
>> *  libxl work - JSON to keep track of guest configs (ok)
>>    -  Wei Liu
>>
>> *  pvscsi should be targeted for 4.5, a prototype exists (fair)
>>    -  Olaf Hering
>>
>> *  Remus in Xen (libxl) (ok)
>>     v19
>>     url: https://github.com/laijs/xen remus-v19
>
>
> Url is here now:
>  https://github.com/macrosheep/xen/tree/remus-v19
>

I would say the status is 'good'. As of v18, Ian J seemed to think that the
series was converging; there were mostly nits and couple of comments, that
have been addressed in the latest series posted.

Shriram

[-- Attachment #1.2: Type: text/html, Size: 2351 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Mukesh Rathor]

On Tue,  2 Sep 2014 16:45:00 -0400 (EDT)
konrad.wilk@oracle.com wrote:

..........

> *  PVH - AMD hardware support. (ok)
>    Posted.
>   -  Mukesh Rathor

Hey Konrad,

I'm afraid this is not looking too good now. I was supposed to get
an AMD box for dom0 that still has not arrived, and handle_mmio changes
may take longer for domu work. 

I'll talk to Boris when he's caught up, and let me know if you want to
talk further too.

thanks a lot,
Mukesh

Re: Xen 4.5 development update (July update)

[Boris Ostrovsky]

On 09/02/2014 04:45 PM, konrad.wilk@oracle.com wrote:
>
> *  VPMU - 'perf' support in Xen (good)
>     v9 posted
>    -  Boris Ostrovsky

I expect to post v10 today or tomorrow, I think status is good.

> == Linux ==
...
> *  VPMU - 'perf' support in Linux (ok)
>     Depends on Xen patches
>    -  Boris Ostrovsky

v4 was posted, didn't see any comments (most patches were already 
Reviewed-by DavidV). Will post another version with small interface 
change requested by Jan.

> *  vAPIC in PVHVM guests (Linux side) (none)
>    -  Boris Ostrovsky

No change ;-(


-boris

Re: Xen 4.5 development update (July update)

[Tamas K Lengyel]

[-- Attachment #1.1: Type: text/plain, Size: 163 bytes --]

> *  ARM implement mem_access (ok)
>    v3
>    https://github.com/tklengyel/xen/tree/arm_memaccess3
>   -  Tamas K Lengyel
>

v4 soon to be posted, status is ok.

[-- Attachment #1.2: Type: text/html, Size: 488 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Roy Franz]

On Tue, Sep 2, 2014 at 1:45 PM,  <konrad.wilk@oracle.com> wrote:
> *  ARM Xen UEFI booting on ARM (ok)
>    v2
>   -  Roy Franz

fair/ok  I am almost done with a significant refactoring based on
feedback.  I should have a new patchset out in
a day or two.

Roy

Re: Xen 4.5 development update (July update)

[Ian Campbell]

I've trimmed the CC list to just xen-devel and committers/maintainers
(mostly from the existing cc list, so not exhaustive).

On Tue, 2014-09-02 at 16:45 -0400, konrad.wilk@oracle.com wrote:

> * Coding time: <=== NOW, one week left!
> 
> * Feature Freeze: 10th September 2014

I'm wondering if, with various maintainers and committers having been
away at various points over the summer (August in particular), we should
perhaps consider slipping the schedule a little bit.

I was away a fair bit myself and I still have a fairly large pile of
patches to wade through, many of which I consider important for 4.5[0].
I'm not sure how other maintainers feel about their respective areas.

One problem with a slip is that I'm also away (conference) next week, so
slipping by a week doesn't really help me all that much and slipping by
more may be unpalatable to some.

> * First RC: 10th October
> * Release: 10th December 2014
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> If you think your patchset MUST go in Xen 4.5 I will post the procedure
> for requesting an exception to get them in past the feature freeze next
> week.

Ian.

[0] toolstack side: Wei's state preservation stuff, er, I'm sure there's
more...
ARM side: GICv3 support, Stefano's 1:1, Julien's passthrough stuff, the
second half of Ariana's iomem stuff, the EFI stub stuff, perhaps the
xenaccess stuff. And that's ignoring my own outstanding serieses

Re: Xen 4.5 development update (July update)

[Jan Beulich]

>>> On 09.09.14 at 11:53, <Ian.Campbell@citrix.com> wrote:
> I've trimmed the CC list to just xen-devel and committers/maintainers
> (mostly from the existing cc list, so not exhaustive).
> 
> On Tue, 2014-09-02 at 16:45 -0400, konrad.wilk@oracle.com wrote:
> 
>> * Coding time: <=== NOW, one week left!
>> 
>> * Feature Freeze: 10th September 2014
> 
> I'm wondering if, with various maintainers and committers having been
> away at various points over the summer (August in particular), we should
> perhaps consider slipping the schedule a little bit.
> 
> I was away a fair bit myself and I still have a fairly large pile of
> patches to wade through, many of which I consider important for 4.5[0].
> I'm not sure how other maintainers feel about their respective areas.

A certain amount of slip would likely help me too, as would if the flood
of patches (or revisions thereof) would start to decrease. I think we
need to consider taking some measures to prevent everyone wanting
to rush in their changes during the last couple of weeks before the
freeze (irrespective of how long ago a first iteration may have been
posted). I can only say that with the current situation (which I don't
think was ever that bad), and with the need for me to do things other
than patch reviewing, I'm not going to feel bad at all if some or all of
those series miss 4.5 even if that's only because I didn't get to look at
the n-th iteration thereof.

Jan

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Tue, Sep 09, 2014 at 11:30:48AM +0100, Jan Beulich wrote:
> >>> On 09.09.14 at 11:53, <Ian.Campbell@citrix.com> wrote:
> > I've trimmed the CC list to just xen-devel and committers/maintainers
> > (mostly from the existing cc list, so not exhaustive).
> > 
> > On Tue, 2014-09-02 at 16:45 -0400, konrad.wilk@oracle.com wrote:
> > 
> >> * Coding time: <=== NOW, one week left!
> >> 
> >> * Feature Freeze: 10th September 2014
> > 
> > I'm wondering if, with various maintainers and committers having been
> > away at various points over the summer (August in particular), we should
> > perhaps consider slipping the schedule a little bit.
> > 
> > I was away a fair bit myself and I still have a fairly large pile of
> > patches to wade through, many of which I consider important for 4.5[0].
> > I'm not sure how other maintainers feel about their respective areas.
> 
> A certain amount of slip would likely help me too, as would if the flood
> of patches (or revisions thereof) would start to decrease. I think we
> need to consider taking some measures to prevent everyone wanting
> to rush in their changes during the last couple of weeks before the
> freeze (irrespective of how long ago a first iteration may have been
> posted). I can only say that with the current situation (which I don't
> think was ever that bad), and with the need for me to do things other
> than patch reviewing, I'm not going to feel bad at all if some or all of
> those series miss 4.5 even if that's only because I didn't get to look at
> the n-th iteration thereof.

Hehe.

What is a comfortable amount of time for folks to feel that they can get
some code in?

Would a two week slippage be too little? Three weeks?
One month slippage scares me.

> 
> Jan
> 

Re: Xen 4.5 development update (July update)

[Jan Beulich]

>>> On 09.09.14 at 16:09, <konrad.wilk@oracle.com> wrote:
> On Tue, Sep 09, 2014 at 11:30:48AM +0100, Jan Beulich wrote:
>> >>> On 09.09.14 at 11:53, <Ian.Campbell@citrix.com> wrote:
>> > I've trimmed the CC list to just xen-devel and committers/maintainers
>> > (mostly from the existing cc list, so not exhaustive).
>> > 
>> > On Tue, 2014-09-02 at 16:45 -0400, konrad.wilk@oracle.com wrote:
>> > 
>> >> * Coding time: <=== NOW, one week left!
>> >> 
>> >> * Feature Freeze: 10th September 2014
>> > 
>> > I'm wondering if, with various maintainers and committers having been
>> > away at various points over the summer (August in particular), we should
>> > perhaps consider slipping the schedule a little bit.
>> > 
>> > I was away a fair bit myself and I still have a fairly large pile of
>> > patches to wade through, many of which I consider important for 4.5[0].
>> > I'm not sure how other maintainers feel about their respective areas.
>> 
>> A certain amount of slip would likely help me too, as would if the flood
>> of patches (or revisions thereof) would start to decrease. I think we
>> need to consider taking some measures to prevent everyone wanting
>> to rush in their changes during the last couple of weeks before the
>> freeze (irrespective of how long ago a first iteration may have been
>> posted). I can only say that with the current situation (which I don't
>> think was ever that bad), and with the need for me to do things other
>> than patch reviewing, I'm not going to feel bad at all if some or all of
>> those series miss 4.5 even if that's only because I didn't get to look at
>> the n-th iteration thereof.
> 
> Hehe.
> 
> What is a comfortable amount of time for folks to feel that they can get
> some code in?
> 
> Would a two week slippage be too little? Three weeks?
> One month slippage scares me.

Me too; just any shift would help me.

Jan

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Tue, Sep 09, 2014 at 03:30:41PM +0100, Jan Beulich wrote:
> >>> On 09.09.14 at 16:09, <konrad.wilk@oracle.com> wrote:
> > On Tue, Sep 09, 2014 at 11:30:48AM +0100, Jan Beulich wrote:
> >> >>> On 09.09.14 at 11:53, <Ian.Campbell@citrix.com> wrote:
> >> > I've trimmed the CC list to just xen-devel and committers/maintainers
> >> > (mostly from the existing cc list, so not exhaustive).
> >> > 
> >> > On Tue, 2014-09-02 at 16:45 -0400, konrad.wilk@oracle.com wrote:
> >> > 
> >> >> * Coding time: <=== NOW, one week left!
> >> >> 
> >> >> * Feature Freeze: 10th September 2014
> >> > 
> >> > I'm wondering if, with various maintainers and committers having been
> >> > away at various points over the summer (August in particular), we should
> >> > perhaps consider slipping the schedule a little bit.
> >> > 
> >> > I was away a fair bit myself and I still have a fairly large pile of
> >> > patches to wade through, many of which I consider important for 4.5[0].
> >> > I'm not sure how other maintainers feel about their respective areas.
> >> 
> >> A certain amount of slip would likely help me too, as would if the flood
> >> of patches (or revisions thereof) would start to decrease. I think we
> >> need to consider taking some measures to prevent everyone wanting
> >> to rush in their changes during the last couple of weeks before the
> >> freeze (irrespective of how long ago a first iteration may have been
> >> posted). I can only say that with the current situation (which I don't
> >> think was ever that bad), and with the need for me to do things other
> >> than patch reviewing, I'm not going to feel bad at all if some or all of
> >> those series miss 4.5 even if that's only because I didn't get to look at
> >> the n-th iteration thereof.
> > 
> > Hehe.
> > 
> > What is a comfortable amount of time for folks to feel that they can get
> > some code in?
> > 
> > Would a two week slippage be too little? Three weeks?
> > One month slippage scares me.
> 
> Me too; just any shift would help me.

Two week then. I will post on xen-devel the email tomorrow if nobody
screams here 'MORE! or 'LESS!'.


> 
> Jan
> 

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Tue, Sep 09, 2014 at 10:53:44AM +0100, Ian Campbell wrote:
> I've trimmed the CC list to just xen-devel and committers/maintainers
> (mostly from the existing cc list, so not exhaustive).
> 
> On Tue, 2014-09-02 at 16:45 -0400, konrad.wilk@oracle.com wrote:
> 
> > * Coding time: <=== NOW, one week left!
> > 
> > * Feature Freeze: 10th September 2014
> 
> I'm wondering if, with various maintainers and committers having been
> away at various points over the summer (August in particular), we should
> perhaps consider slipping the schedule a little bit.
> 
> I was away a fair bit myself and I still have a fairly large pile of
> patches to wade through, many of which I consider important for 4.5[0].
> I'm not sure how other maintainers feel about their respective areas.
> 
> One problem with a slip is that I'm also away (conference) next week, so
> slipping by a week doesn't really help me all that much and slipping by
> more may be unpalatable to some.
> 
> > * First RC: 10th October
> > * Release: 10th December 2014
> > 
> > The RCs and release will of course depend on stability and bugs, and
> > will therefore be fairly unpredictable.  The feature freeze may be
> > slipped for especially important features which are near completion.
> > 
> > If you think your patchset MUST go in Xen 4.5 I will post the procedure
> > for requesting an exception to get them in past the feature freeze next
> > week.
> 
> Ian.
> 
> [0] toolstack side: Wei's state preservation stuff, er, I'm sure there's
> more...
> ARM side: GICv3 support, Stefano's 1:1, Julien's passthrough stuff, the
> second half of Ariana's iomem stuff, the EFI stub stuff, perhaps the
> xenaccess stuff. And that's ignoring my own outstanding serieses

Right. But we already have a ton of stuff in.

> 
> 

Re: Xen 4.5 development update (July update)

[Meng Xu]

Hi Konrad,

>
> The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
> A bunch of items had moved to the completed phase which is fantastic.
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>

>
> *  XenRT (Preemptive Global Earliest Deadline First) (ok)
>    v3
>   -  Meng Xu
>

PATCH v2 was posted last weekend (Sept. 7th). It is moving from ok to
good. (My plan is releasing the v3 this week to tackle George and
Dario's comments on the hypervisor part.)

The current patch status is ok. I plan to make it into good status
this week or next week.

Thanks,

Meng


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

Re: Xen 4.5 development update (July update)

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1503 bytes --]

On Tue, 2014-09-09 at 09:51 -0400, Meng Xu wrote:
> >
> > The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
> > A bunch of items had moved to the completed phase which is fantastic.
> >
> > none - nothing yet
> > fair - still working on it, patches are prototypes or RFC
> > ok   - patches posted, acting on review
> > good - some last minute pieces
> > done - all done, might have bugs
> >
> 
> >
> > *  XenRT (Preemptive Global Earliest Deadline First) (ok)
> >    v3
> >   -  Meng Xu
> >
> 
> PATCH v2 was posted last weekend (Sept. 7th). It is moving from ok to
> good. (My plan is releasing the v3 this week to tackle George and
> Dario's comments on the hypervisor part.)
> 
> The current patch status is ok. I plan to make it into good status
> this week or next week.
> 
Yes, I confirm that the status of this series is pretty good, given the
goal we have for this scheduler.

In fact, both me, George and Meng agreed on considering this an
experimental feature, similarly to what we did for, ARM or PVH. With
that in mind, although there are still some things Meng needs to change,
the code already looks quite good.

If George wants to add anything...

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Xen 4.5 development update (July update)

[konrad.wilk]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 11782 bytes --]

Below is a summary of the projects / features being worked on for the 4.5
time frame.  The tentative feature freeze is scheduled for September 10th,
which is two months away. This being the summer some folks are talking
vacation, so those two months might be a lot less.

The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
A bunch of items had moved to the completed phase which is fantastic.

However we still have some that are important but the code hadn't yet
been posted which is worrisome. I've also moved some to Xen 4.6 after
talking with folks privately.

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs

For items involving code hosted on the Xen.org site (including qemu-xen),
that means a likelihood of having the feature code-complete and mostly
working by the feature freeze.  (It's OK if there are still bugs to be
worked out.)  For items in Linux, it would mean having items on track
to make it into the kernel released just after the scheduled 4.5 time frame.

In terms of libvirt it has monthly releases. As such not going to track
every release - but closer to when RCs are out.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Coding time: <=== NOW, two months left!

* Feature Freeze: 10th September 2014
* First RC: 10th October
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs


= Open =

== ARM ==

*  ARM - Device assigment on ARM (good)
   RFC for the hypervisor side sent.
   Linux parts at risk.
  -  Julien Grall

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (good)
   v9 posted; resent & rebased
  -  Arianna Avanzini

*  ARM Xen UEFI booting on ARM (ok)
   v2
  -  Roy Franz

*  ARM PSCI v0.2 (good)
   v3 posted
  -  Parth Dixit

*  ARM GICv3 support (ok)
   v6a patchset (refactor in, also known as v9a); v7 posted
   v9a posted which does GIC and VGIC code refactoring
  -  Vijay Kilari

*  ARM VM save/restore/live migration (ok)
  -  Junghyun Yoo

*  ARM remote processor iommu module (GPUs + IPUs) (unknown)
  -  Andrii Tseglytskyi

*  ARM: Use super pages in p2m (ok)
   v5 posted
  -  Ian Campbell

*  ARM - MiniOS (ok)
   v6 posted
  -  Thomas Leonard

*  ARM - VGIC emulation (ok)
   v8
  -  Stefano Stabellini

*  ARM - XENFEAT_grant_map_11 (fair)
   Provide kernels an grant->MFN lookup
   RFC
  -  Stefano Stabellini

== x86 ==

*  New Migration (v2). (ok)
   v6 posted
   libxl is next
   tmem & remus need work
  -  Andrew Cooper & David Vrabel

*  Remus using migration-v2 (fair)
   RFC posted - depends on v6 of 'New Migration'
  -  Yang Hongyang

*  PVH - AMD hardware support. (fair)
   Issues with FSBASE MSRs
  -  Mukesh Rathor

*  Xen multiboot2-EFI support (fair)
   Needed for SecureBoot
   Depends on Xen Boot info (rework multiboot and other structs)
   See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html
  -  Daniel Kiper

*  Xen HPET interrupt fixes (fair)
   behind migration v2
  -  Andrew Cooper

*  HT enabled, virtualization overhead is high (Xen 4.4) (none)
   kernbench demonstrated it
   looking and tracing it
  -  Dario Faggioli

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  dirty vram / IOMMU bug (fair)
  -  Zhang, Yang Z

*  Convert tasklet to per-cpu tasklets (fair)
  -  Konrad Rzeszutek Wilk

*  VMware backdoor (hypercall) (ok)
   Needs to be split up and redone
  -  Don Slutz

*  VPMU - 'perf' support in Xen (good)
   v8 posted
  -  Boris Ostrovsky

*  Cache QoS Monitoring - hypercalls (fair)
   Just hypercalls - no toolstack changes.
   v12 posted
  -  Dongxiao Xu and Shantong Kang

*  1TB slow destruction (ok)
  -  Bob Liu

*  extending mem_access support to PV domain (fair)
   RFC v2
  -  Aravindh Puthiyaparambil (aravindp)

*  Stability fix (good)
  -  Jan Beulich

*  Support controlling the max C-state sub-state (ok)
   v3 posted
  -  Ross Lagerwall

*  Repurpose SEDF Scheduler for Real-time (fair)
   RFC patch posted (v2)
  -  Joshua Whitehead, Robert VanVossen

*  XenRT (Preemptive Global Earliest Deadline First) (fair)
   RFC patch posted (v1)
  -  Meng Xu

*  Introspection of HVM guests (fair)
   RFC v2
  -  Razvan Cojocaru

*  vNUMA in Xen (ok)
   v6 posted
   git://gitorious.org/vnuma/xen_vnuma.git
  -  Elena Ufimtseva

== QEMU ==

*  Rebase of QEMU 2.0 and SeaBIOS (fair)
  -  Ian Jackson

*  Using qemu-upstream in a stubdomain (fair)
   Will use rump kernels.
  -  Ian Jackson

*  Intel IGD PCI GPU passthrough (ok)
   v5 posted
  -  Chen, Tiejun

*  Xen PV block driver in OVMF (UEFI in guest) (fair)
   RFC posted.
  -  Anthony PERARD

== lib{xc,xl} and toolstack ==

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl work - JSON to keep track of guest configs (ok)
  -  Wei Liu

*  pvscsi should be targeted for 4.5, a prototype exists (fair)
  -  Olaf Hering

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  Remus in Xen (libxl) (ok)
   v17
   url: https://github.com/laijs/xen remus-v17
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  COarse-grain LOck-stepping Virtual Machines in Xen (fair)
   RFC patch posted
  -  Wen Congyang
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  systemd support (ok)
   v7
  -  Luis R. Rodriguez

*  snapshot API extension (checkpointing disk) (ok)
   v5
  -  Bamvor Jian Zhang

*  extend the xenstore ring with a 'closing' signal (fair)
   RFC patch posted
  -  David Scott

*  Soft affinity for vcpus libxl/xl changes (good)
   v12 posted
  -  Dario Faggioli

== Linux ==

*  Linux block multiqueue (fair)
  -  Arianna Avanzini

*  Netback grant table manipulations (ok)
  -  Zoltan Kiss

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
   v6 posted
   git://gitorious.org/vnuma/linux_vnuma.git
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  vAPIC in PVHVM guests (Linux side) (none)
  -  Boris Ostrovsky

*  pvSCSI (ok)
   Initial patch posted.
  -  Juergen Gross

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

*  mini-os: xenbus changes for rump kernels (ok)
   git://xenbits.xen.org/people/iwj/rumpuser-xen.git
   branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
   v2 posted
  -  Ian Jackson

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to Xen 4.6 ==

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

*  cpuid leveling (none)
   http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

*  IO-NUMA - hwloc and xl (fair)
   Andrew Cooper had an RFC patch for hwloc
  -  Boris Ostrovsky

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
  -  Ian Jackson

*  HVM guest NUMA (none)
  -  Matt Wilson

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  ARM GICv2m support (none)
  -  Linaro (unknown)

== Up for grabs ==

*  OSSTest - also test Linux PVH guests

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of vHPET (done)
   aka hvm/hpet: Detect comparator values in the past
  -  Don Slutz

*  vAPIC in PVHVM guests (Xen side) (done)
  -  Boris Ostrovsky

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk

*  ARM  - IOMMU support (done)
  -  Julien Grall

*  ioreq-server, aka secondary emulators (done)
  -  Paul Durrant

*  Netback multiqueue (good)
  -  Wei Liu

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  Bigger PCI hole in QEMU (done)
   Needs to be rebased
  -  Don Slutz

*  ARM DRA7 support (done)
   v3 posted
   v3 with comments applied
  -  Andrii Tseglytskyi

*  rework VM Generation ID (done)
   v7 posted
  -  David Vrabel

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
   v11 posted
  -  Dario Faggioli

*  Linux pvops of Xen EFI hypercall support (done)
  -  Daniel Kiper



[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Boris Ostrovsky]

On 07/22/2014 08:28 PM, konrad.wilk@oracle.com wrote:
>
> *  IO-NUMA - hwloc and xl (fair)
>     Andrew Cooper had an RFC patch for hwloc
>    -  Boris Ostrovsky


I don't think we ever planned this for 4.5. This is a "none".

-boris

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Tue, Jul 22, 2014 at 08:47:13PM -0400, Boris Ostrovsky wrote:
> On 07/22/2014 08:28 PM, konrad.wilk@oracle.com wrote:
> >
> >*  IO-NUMA - hwloc and xl (fair)
> >    Andrew Cooper had an RFC patch for hwloc
> >   -  Boris Ostrovsky
> 
> 
> I don't think we ever planned this for 4.5. This is a "none".

Right. I had it in Xen 4.6 deferred line. Will fix it to 'none'
> 
> -boris
> 
> 

Re: Xen 4.5 development update (July update)

[Andrew Cooper]

On 23/07/2014 01:28, konrad.wilk@oracle.com wrote:
> Below is a summary of the projects / features being worked on for the 4.5
> time frame.  The tentative feature freeze is scheduled for September 10th,
> which is two months away. This being the summer some folks are talking
> vacation, so those two months might be a lot less.
>
> The "prognosis" is now the likelihood of completion in the 4.5 timeframe.
> A bunch of items had moved to the completed phase which is fantastic.
>
> However we still have some that are important but the code hadn't yet
> been posted which is worrisome. I've also moved some to Xen 4.6 after
> talking with folks privately.
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>
> For items involving code hosted on the Xen.org site (including qemu-xen),
> that means a likelihood of having the feature code-complete and mostly
> working by the feature freeze.  (It's OK if there are still bugs to be
> worked out.)  For items in Linux, it would mean having items on track
> to make it into the kernel released just after the scheduled 4.5 time frame.
>
> In terms of libvirt it has monthly releases. As such not going to track
> every release - but closer to when RCs are out.
>
> = Timeline =
>
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
>
> * Coding time: <=== NOW, two months left!
>
> * Feature Freeze: 10th September 2014
> * First RC: 10th October
> * Release: 10th December 2014
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
>
> = Prognosis =
>
> The states are: none -> fair -> ok -> good -> done
>
> none - nothing yet
> fair - still working on it, patches are prototypes or RFC
> ok   - patches posted, acting on review
> good - some last minute pieces
> done - all done, might have bugs
>
>
> = Open =
>
> <snip>
>
> == x86 == 
>
> *  New Migration (v2). (ok)
>    v6 posted
>    libxl is next
>    tmem & remus need work
>   -  Andrew Cooper & David Vrabel

remus should be removed from this section as it now has its own section
below.

I think the libxl work should also be split out, which is currently
'none'.  The libxc work is 'ok' going on 'good' (but still probably on
the ok side).

~Andrew

Re: Xen 4.5 development update (July update)

[Ian Campbell]

(trimming CC to just those related to things I've commented on)

On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:

> fair - still working on it, patches are prototypes or RFC

You've used this (correctly per your definition here) on various things
which it seems to me are very unlikely to make it for 4.5, i.e. things
which are really early prototypes etc, which made me pause every time I
ready it and have to think back to the definition.

I guess what I'm saying is that "fair" is not a good label for this
state, it implies that while the thing is a WIP it is still likely to
make it for 4.5.

I think you need a lower ranked WIP label for things which are in that
state of prototype/RFC but not likely to make 4.5. "wip" or "inprogress"
perhaps.

> = Open =
> 
> == ARM == 
[...]

> *  ARM VM save/restore/live migration (ok)
>   -  Junghyun Yoo

I've not heard from Junghyun since the initial reposting, not sure what
the prognosis is but given lack of support for SMP and some of the other
issues I'm more incline to suggest fair rather than OK. Jinghyun, do you
have any thoughts?

It's possible that the "save/restore/dead-migration" bit is "ok", it's
the live migration bits where the issues mostly remain iffy.

Also this is dependent on the migration v2 by Andrew et al.

> *  ARM: Use super pages in p2m (ok)
>    v5 posted
>   -  Ian Campbell

Committed.

> == QEMU == 
> 
> *  Rebase of QEMU 2.0 and SeaBIOS (fair)
>   -  Ian Jackson

What is this? Neither of them sound like Ian J things. Stefano maintains
Qemu and I take care of SeaBIOS.

>From the SeaBIOS side we are currently using the latest upstream.

IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
with, but he should confirm...

Ian.

Re: Xen 4.5 development update (July update)

[Stefano Stabellini]

On Wed, 23 Jul 2014, Ian Campbell wrote:
> (trimming CC to just those related to things I've commented on)
> 
> On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
> 
> > fair - still working on it, patches are prototypes or RFC
> 
> You've used this (correctly per your definition here) on various things
> which it seems to me are very unlikely to make it for 4.5, i.e. things
> which are really early prototypes etc, which made me pause every time I
> ready it and have to think back to the definition.
> 
> I guess what I'm saying is that "fair" is not a good label for this
> state, it implies that while the thing is a WIP it is still likely to
> make it for 4.5.
> 
> I think you need a lower ranked WIP label for things which are in that
> state of prototype/RFC but not likely to make 4.5. "wip" or "inprogress"
> perhaps.
> 
> > = Open =
> > 
> > == ARM == 
> [...]
> 
> > *  ARM VM save/restore/live migration (ok)
> >   -  Junghyun Yoo
> 
> I've not heard from Junghyun since the initial reposting, not sure what
> the prognosis is but given lack of support for SMP and some of the other
> issues I'm more incline to suggest fair rather than OK. Jinghyun, do you
> have any thoughts?
> 
> It's possible that the "save/restore/dead-migration" bit is "ok", it's
> the live migration bits where the issues mostly remain iffy.
> 
> Also this is dependent on the migration v2 by Andrew et al.
> 
> > *  ARM: Use super pages in p2m (ok)
> >    v5 posted
> >   -  Ian Campbell
> 
> Committed.
> 
> > == QEMU == 
> > 
> > *  Rebase of QEMU 2.0 and SeaBIOS (fair)
> >   -  Ian Jackson
> 
> What is this? Neither of them sound like Ian J things. Stefano maintains
> Qemu and I take care of SeaBIOS.

Yeah, it should be me for QEMU and it is done.


> >From the SeaBIOS side we are currently using the latest upstream.
> 
> IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> with, but he should confirm...

Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
again a new QEMU upstream release before 4.5.

Re: Xen 4.5 development update (July update)

[Don Slutz]

On 07/23/14 06:20, Stefano Stabellini wrote:
> On Wed, 23 Jul 2014, Ian Campbell wrote:
>> (trimming CC to just those related to things I've commented on)
>>
>> On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
>>

>>> == QEMU ==
>>>
>>> *  Rebase of QEMU 2.0 and SeaBIOS (fair)
>>>    -  Ian Jackson
>> What is this? Neither of them sound like Ian J things. Stefano maintains
>> Qemu and I take care of SeaBIOS.
> Yeah, it should be me for QEMU and it is done.
>
>
>> >From the SeaBIOS side we are currently using the latest upstream.
>>
>> IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
>> with, but he should confirm...
> Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> again a new QEMU upstream release before 4.5.
>

Based on this,

Message-ID: <1405424067-23136-1-git-send-email-dslutz@verizon.com>
http://lists.xenproject.org/archives/html/xen-devel/2014-07/msg01923.html

Needs to wait for 4.6
     -Don Slutz




> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Stefano Stabellini]

On Wed, 23 Jul 2014, Don Slutz wrote:
> On 07/23/14 06:20, Stefano Stabellini wrote:
> > On Wed, 23 Jul 2014, Ian Campbell wrote:
> > > (trimming CC to just those related to things I've commented on)
> > > 
> > > On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
> > > 
> 
> > > > == QEMU ==
> > > > 
> > > > *  Rebase of QEMU 2.0 and SeaBIOS (fair)
> > > >    -  Ian Jackson
> > > What is this? Neither of them sound like Ian J things. Stefano maintains
> > > Qemu and I take care of SeaBIOS.
> > Yeah, it should be me for QEMU and it is done.
> > 
> > 
> > > >From the SeaBIOS side we are currently using the latest upstream.
> > > 
> > > IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> > > with, but he should confirm...
> > Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> > again a new QEMU upstream release before 4.5.
> > 
> 
> Based on this,
> 
> Message-ID: <1405424067-23136-1-git-send-email-dslutz@verizon.com>
> http://lists.xenproject.org/archives/html/xen-devel/2014-07/msg01923.html
> 
> Needs to wait for 4.6
>     -Don Slutz

I would be happy to backport that patch once it is upstream.

Re: Xen 4.5 development update (July update)

[Don Slutz]

On 07/23/14 09:03, Stefano Stabellini wrote:
> On Wed, 23 Jul 2014, Don Slutz wrote:
>> On 07/23/14 06:20, Stefano Stabellini wrote:
>>> On Wed, 23 Jul 2014, Ian Campbell wrote:
>>>> (trimming CC to just those related to things I've commented on)
>>>>
>>>> On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
>>>>
>>>>> == QEMU ==
>>>>>
>>>>> *  Rebase of QEMU 2.0 and SeaBIOS (fair)
>>>>>     -  Ian Jackson
>>>> What is this? Neither of them sound like Ian J things. Stefano maintains
>>>> Qemu and I take care of SeaBIOS.
>>> Yeah, it should be me for QEMU and it is done.
>>>
>>>
>>>> >From the SeaBIOS side we are currently using the latest upstream.
>>>>
>>>> IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
>>>> with, but he should confirm...
>>> Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
>>> again a new QEMU upstream release before 4.5.
>>>
>> Based on this,
>>
>> Message-ID: <1405424067-23136-1-git-send-email-dslutz@verizon.com>
>> http://lists.xenproject.org/archives/html/xen-devel/2014-07/msg01923.html
>>
>> Needs to wait for 4.6
>>      -Don Slutz
> I would be happy to backport that patch once it is upstream.

The change that was accepted by QEMU upstream depends on a big batch
of other changes (commit c4f5cdc53f181f6fe84a0f1bf99914598934a8a6)
So the backport to QEMU 2.0.x is not simple.

I can provide a QEMU 2.0.x patch that is based on the accepted name and
way it is used, but uses a global variable instead of a property on the
machine object.  I.E. a simple patch.

Let me know if you want this.
     -Don Slutz

Re: Xen 4.5 development update (July update)

[Stefano Stabellini]

On Wed, 23 Jul 2014, Don Slutz wrote:
> On 07/23/14 09:03, Stefano Stabellini wrote:
> > On Wed, 23 Jul 2014, Don Slutz wrote:
> > > On 07/23/14 06:20, Stefano Stabellini wrote:
> > > > On Wed, 23 Jul 2014, Ian Campbell wrote:
> > > > > (trimming CC to just those related to things I've commented on)
> > > > > 
> > > > > On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
> > > > > 
> > > > > > == QEMU ==
> > > > > > 
> > > > > > *  Rebase of QEMU 2.0 and SeaBIOS (fair)
> > > > > >     -  Ian Jackson
> > > > > What is this? Neither of them sound like Ian J things. Stefano
> > > > > maintains
> > > > > Qemu and I take care of SeaBIOS.
> > > > Yeah, it should be me for QEMU and it is done.
> > > > 
> > > > 
> > > > > >From the SeaBIOS side we are currently using the latest upstream.
> > > > > 
> > > > > IIRC Stefano has merged the version of qemu which he intends 4.5 to
> > > > > ship
> > > > > with, but he should confirm...
> > > > Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> > > > again a new QEMU upstream release before 4.5.
> > > > 
> > > Based on this,
> > > 
> > > Message-ID: <1405424067-23136-1-git-send-email-dslutz@verizon.com>
> > > http://lists.xenproject.org/archives/html/xen-devel/2014-07/msg01923.html
> > > 
> > > Needs to wait for 4.6
> > >      -Don Slutz
> > I would be happy to backport that patch once it is upstream.
> 
> The change that was accepted by QEMU upstream depends on a big batch
> of other changes (commit c4f5cdc53f181f6fe84a0f1bf99914598934a8a6)
> So the backport to QEMU 2.0.x is not simple.
> 
> I can provide a QEMU 2.0.x patch that is based on the accepted name and
> way it is used, but uses a global variable instead of a property on the
> machine object.  I.E. a simple patch.

Yes, please. Thank you.

- Stefano

Re: Xen 4.5 development update (July update)

[Fabio Fantoni]

Il 23/07/2014 12:20, Stefano Stabellini ha scritto:
> On Wed, 23 Jul 2014, Ian Campbell wrote:
>> (trimming CC to just those related to things I've commented on)
>>
>> On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
>>
>>> fair - still working on it, patches are prototypes or RFC
>> You've used this (correctly per your definition here) on various things
>> which it seems to me are very unlikely to make it for 4.5, i.e. things
>> which are really early prototypes etc, which made me pause every time I
>> ready it and have to think back to the definition.
>>
>> I guess what I'm saying is that "fair" is not a good label for this
>> state, it implies that while the thing is a WIP it is still likely to
>> make it for 4.5.
>>
>> I think you need a lower ranked WIP label for things which are in that
>> state of prototype/RFC but not likely to make 4.5. "wip" or "inprogress"
>> perhaps.
>>
>>> = Open =
>>>
>>> == ARM ==
>> [...]
>>
>>> *  ARM VM save/restore/live migration (ok)
>>>    -  Junghyun Yoo
>> I've not heard from Junghyun since the initial reposting, not sure what
>> the prognosis is but given lack of support for SMP and some of the other
>> issues I'm more incline to suggest fair rather than OK. Jinghyun, do you
>> have any thoughts?
>>
>> It's possible that the "save/restore/dead-migration" bit is "ok", it's
>> the live migration bits where the issues mostly remain iffy.
>>
>> Also this is dependent on the migration v2 by Andrew et al.
>>
>>> *  ARM: Use super pages in p2m (ok)
>>>     v5 posted
>>>    -  Ian Campbell
>> Committed.
>>
>>> == QEMU ==
>>>
>>> *  Rebase of QEMU 2.0 and SeaBIOS (fair)
>>>    -  Ian Jackson
>> What is this? Neither of them sound like Ian J things. Stefano maintains
>> Qemu and I take care of SeaBIOS.
> Yeah, it should be me for QEMU and it is done.
>
>
>> >From the SeaBIOS side we are currently using the latest upstream.
>>
>> IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
>> with, but he should confirm...
> Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> again a new QEMU upstream release before 4.5.

qemu 2.1 is near to be released, now is rc3, why not update to 2.1?

thanks for any reply

Re: Xen 4.5 development update (July update)

[Stefano Stabellini]

On Wed, 23 Jul 2014, Fabio Fantoni wrote:
> Il 23/07/2014 12:20, Stefano Stabellini ha scritto:
> > On Wed, 23 Jul 2014, Ian Campbell wrote:
> > > (trimming CC to just those related to things I've commented on)
> > > 
> > > On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
> > > 
> > > > fair - still working on it, patches are prototypes or RFC
> > > You've used this (correctly per your definition here) on various things
> > > which it seems to me are very unlikely to make it for 4.5, i.e. things
> > > which are really early prototypes etc, which made me pause every time I
> > > ready it and have to think back to the definition.
> > > 
> > > I guess what I'm saying is that "fair" is not a good label for this
> > > state, it implies that while the thing is a WIP it is still likely to
> > > make it for 4.5.
> > > 
> > > I think you need a lower ranked WIP label for things which are in that
> > > state of prototype/RFC but not likely to make 4.5. "wip" or "inprogress"
> > > perhaps.
> > > 
> > > > = Open =
> > > > 
> > > > == ARM ==
> > > [...]
> > > 
> > > > *  ARM VM save/restore/live migration (ok)
> > > >    -  Junghyun Yoo
> > > I've not heard from Junghyun since the initial reposting, not sure what
> > > the prognosis is but given lack of support for SMP and some of the other
> > > issues I'm more incline to suggest fair rather than OK. Jinghyun, do you
> > > have any thoughts?
> > > 
> > > It's possible that the "save/restore/dead-migration" bit is "ok", it's
> > > the live migration bits where the issues mostly remain iffy.
> > > 
> > > Also this is dependent on the migration v2 by Andrew et al.
> > > 
> > > > *  ARM: Use super pages in p2m (ok)
> > > >     v5 posted
> > > >    -  Ian Campbell
> > > Committed.
> > > 
> > > > == QEMU ==
> > > > 
> > > > *  Rebase of QEMU 2.0 and SeaBIOS (fair)
> > > >    -  Ian Jackson
> > > What is this? Neither of them sound like Ian J things. Stefano maintains
> > > Qemu and I take care of SeaBIOS.
> > Yeah, it should be me for QEMU and it is done.
> > 
> > 
> > > >From the SeaBIOS side we are currently using the latest upstream.
> > > 
> > > IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> > > with, but he should confirm...
> > Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> > again a new QEMU upstream release before 4.5.
> 
> qemu 2.1 is near to be released, now is rc3, why not update to 2.1?

I don't think we would have enough time to test it throughly.

Re: Xen 4.5 development update (July update)

[Fabio Fantoni]

Il 23/07/2014 15:45, Stefano Stabellini ha scritto:
> On Wed, 23 Jul 2014, Fabio Fantoni wrote:
>> Il 23/07/2014 12:20, Stefano Stabellini ha scritto:
>>> On Wed, 23 Jul 2014, Ian Campbell wrote:
>>>> (trimming CC to just those related to things I've commented on)
>>>>
>>>> On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
>>>>
>>>>> fair - still working on it, patches are prototypes or RFC
>>>> You've used this (correctly per your definition here) on various things
>>>> which it seems to me are very unlikely to make it for 4.5, i.e. things
>>>> which are really early prototypes etc, which made me pause every time I
>>>> ready it and have to think back to the definition.
>>>>
>>>> I guess what I'm saying is that "fair" is not a good label for this
>>>> state, it implies that while the thing is a WIP it is still likely to
>>>> make it for 4.5.
>>>>
>>>> I think you need a lower ranked WIP label for things which are in that
>>>> state of prototype/RFC but not likely to make 4.5. "wip" or "inprogress"
>>>> perhaps.
>>>>
>>>>> = Open =
>>>>>
>>>>> == ARM ==
>>>> [...]
>>>>
>>>>> *  ARM VM save/restore/live migration (ok)
>>>>>     -  Junghyun Yoo
>>>> I've not heard from Junghyun since the initial reposting, not sure what
>>>> the prognosis is but given lack of support for SMP and some of the other
>>>> issues I'm more incline to suggest fair rather than OK. Jinghyun, do you
>>>> have any thoughts?
>>>>
>>>> It's possible that the "save/restore/dead-migration" bit is "ok", it's
>>>> the live migration bits where the issues mostly remain iffy.
>>>>
>>>> Also this is dependent on the migration v2 by Andrew et al.
>>>>
>>>>> *  ARM: Use super pages in p2m (ok)
>>>>>      v5 posted
>>>>>     -  Ian Campbell
>>>> Committed.
>>>>
>>>>> == QEMU ==
>>>>>
>>>>> *  Rebase of QEMU 2.0 and SeaBIOS (fair)
>>>>>     -  Ian Jackson
>>>> What is this? Neither of them sound like Ian J things. Stefano maintains
>>>> Qemu and I take care of SeaBIOS.
>>> Yeah, it should be me for QEMU and it is done.
>>>
>>>
>>>> >From the SeaBIOS side we are currently using the latest upstream.
>>>>
>>>> IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
>>>> with, but he should confirm...
>>> Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
>>> again a new QEMU upstream release before 4.5.
>> qemu 2.1 is near to be released, now is rc3, why not update to 2.1?
> I don't think we would have enough time to test it throughly.
However, the major distributions will update to latest qemu tested or 
notin Xen.
I think it's always best to use the latest stable qemu in xen-unstable, 
at least until will xen reach the freeze.

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

> > >From the SeaBIOS side we are currently using the latest upstream.
> > 
> > IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> > with, but he should confirm...
> 
> Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> again a new QEMU upstream release before 4.5.

Is there an ETA when you want to push it out? So far I see that
 qemu-upstream-unstable master has not been updated.

Re: Xen 4.5 development update (July update)

[Stefano Stabellini]

On Wed, 23 Jul 2014, Konrad Rzeszutek Wilk wrote:
> > > >From the SeaBIOS side we are currently using the latest upstream.
> > > 
> > > IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> > > with, but he should confirm...
> > 
> > Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> > again a new QEMU upstream release before 4.5.
> 
> Is there an ETA when you want to push it out? So far I see that
>  qemu-upstream-unstable master has not been updated.

It has. The tip is:

commit 779640a3f4f58633e8575a534838ec79b4982107
Author: Luiz Capitulino <lcapitulino@redhat.com>
Date:   Thu Jun 19 10:13:43 2014 -0400

    fpu: softfloat: drop INLINE macro

and it is based on v2.0.0.

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Thu, Jul 24, 2014 at 12:23:17PM +0100, Stefano Stabellini wrote:
> On Wed, 23 Jul 2014, Konrad Rzeszutek Wilk wrote:
> > > > >From the SeaBIOS side we are currently using the latest upstream.
> > > > 
> > > > IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> > > > with, but he should confirm...
> > > 
> > > Yes, I have already merged QEMU 2.0.0 and I am not planning to merge
> > > again a new QEMU upstream release before 4.5.
> > 
> > Is there an ETA when you want to push it out? So far I see that
> >  qemu-upstream-unstable master has not been updated.
> 
> It has. The tip is:
> 
> commit 779640a3f4f58633e8575a534838ec79b4982107
> Author: Luiz Capitulino <lcapitulino@redhat.com>
> Date:   Thu Jun 19 10:13:43 2014 -0400
> 
>     fpu: softfloat: drop INLINE macro
> 
> and it is based on v2.0.0.

<scratches his head>

Looking at git://git.qemu.org/qemu.git I see:
a9e8aeb Update version for v2.0.0 release

oh, which I see now in the master, way way back in the log. And then
you backported a bunch of stuff from v2.1.0 in the tree.

Right, that makes more sense now.

Thank you!

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Wed, Jul 23, 2014 at 10:51:46AM +0100, Ian Campbell wrote:
> (trimming CC to just those related to things I've commented on)
> 
> On Tue, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
> 
> > fair - still working on it, patches are prototypes or RFC
> 
> You've used this (correctly per your definition here) on various things
> which it seems to me are very unlikely to make it for 4.5, i.e. things
> which are really early prototypes etc, which made me pause every time I
> ready it and have to think back to the definition.
> 
> I guess what I'm saying is that "fair" is not a good label for this
> state, it implies that while the thing is a WIP it is still likely to
> make it for 4.5.
> 
> I think you need a lower ranked WIP label for things which are in that
> state of prototype/RFC but not likely to make 4.5. "wip" or "inprogress"
> perhaps.

Or I can start weeding them out and push them in the Xen 4.6 territory.
I was thinking that in August I would do that.
> 
> > = Open =
> > 
> > == ARM == 
> [...]
> 
> > *  ARM VM save/restore/live migration (ok)
> >   -  Junghyun Yoo
> 
> I've not heard from Junghyun since the initial reposting, not sure what
> the prognosis is but given lack of support for SMP and some of the other
> issues I'm more incline to suggest fair rather than OK. Jinghyun, do you
> have any thoughts?
> 
> It's possible that the "save/restore/dead-migration" bit is "ok", it's
> the live migration bits where the issues mostly remain iffy.

OK, downgraded it.
> 
> Also this is dependent on the migration v2 by Andrew et al.

That would certainly make it 'fair' or 'none' as said code dependent
on Andrew, et al. code has not been posted.
> 
> > *  ARM: Use super pages in p2m (ok)
> >    v5 posted
> >   -  Ian Campbell
> 
> Committed.

Excellent!
> 
> > == QEMU == 
> > 
> > *  Rebase of QEMU 2.0 and SeaBIOS (fair)
> >   -  Ian Jackson
> 
> What is this? Neither of them sound like Ian J things. Stefano maintains
> Qemu and I take care of SeaBIOS.

Right.
> 
> >From the SeaBIOS side we are currently using the latest upstream.

Is there a specific tag we want to use? 
> 
> IIRC Stefano has merged the version of qemu which he intends 4.5 to ship
> with, but he should confirm...
> 
> Ian.
> 

Re: Xen 4.5 development update (July update)

[Ian Campbell]

On Wed, 2014-07-23 at 12:18 -0400, Konrad Rzeszutek Wilk wrote:

> > >From the SeaBIOS side we are currently using the latest upstream.
> 
> Is there a specific tag we want to use? 

We are using rel-1.7.5 which is the current latest.

If when 1.7.6 or 1.7.5.1 occurs I'll evaluate that based on where we are
in the release process.

Ian.

Re: Xen 4.5 development update (July update)

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 813 bytes --]

On mar, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:

> == x86 == 

> *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
>    kernbench demonstrated it
>    looking and tracing it
>   -  Dario Faggioli
> 
There is a (sub-)thread sill ongoing here:
http://www.gossamer-threads.com/lists/xen/devel/339409

However, there have been found counterexamples, i.e., numbers showing
that virt overhead is similar with and without HT, so I think this
bullet can go away (not sure with what tracking status...)

Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.5 development update (July update)

[Konrad Rzeszutek Wilk]

On Mon, Jul 28, 2014 at 07:35:22PM +0200, Dario Faggioli wrote:
> On mar, 2014-07-22 at 20:28 -0400, konrad.wilk@oracle.com wrote:
> 
> > == x86 == 
> 
> > *  HT enabled, virtualization overhead is high (Xen 4.4) (none)
> >    kernbench demonstrated it
> >    looking and tracing it
> >   -  Dario Faggioli
> > 
> There is a (sub-)thread sill ongoing here:
> http://www.gossamer-threads.com/lists/xen/devel/339409
> 
> However, there have been found counterexamples, i.e., numbers showing
> that virt overhead is similar with and without HT, so I think this
> bullet can go away (not sure with what tracking status...)

The 'This confirms, for me, that it's an SMT balancing issue that we're seen. '
is not an issue anymore? As in we can get bad and good results depending
on the workload?

Now here is another question - did you run the guests with 'xen_nopvspin'
to see if this was an PV ticketlock (or the lack of it) issue?

Also what did the " I'll try more runs, e.g. with number of VCPUs equal
less than nr_corse/2 and see what happens." come out to be?

Thank you!
> 
> Dario
> 
> -- 
> <<This happens because I choose it to happen!>> (Raistlin Majere)
> -----------------------------------------------------------------
> Dario Faggioli, Ph.D, http://about.me/dario.faggioli
> Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)
> 

Xen 4.5 development update (July update)

[Meng Xu]

[-- Attachment #1.1: Type: text/plain, Size: 842 bytes --]

 Hi Konrad,


> *  XenRT (Preemptive Global Earliest Deadline First) (fair)
>    RFC patch posted (v1)
>   -  Meng Xu
>
>
I'm going to release the version 2 of this XenRT patches soon. I hope to
settle down the xen tool for the XenRT patch in the version 2.

The current status is: The scheduler can work well and we had small tests
on this scheduler.  (Any suggestion on automatic testing the scheduler for
production?) I will include the progress details in the email of the
version 2.

The main functionality of this scheduler is almost done. (Well, we only
need to add the TRACE in the next version to finish all of the
functionality.) The current work focuses on improving the scheduler's
performance and scalability.

Thanks,

Meng



-- 


-----------
Meng Xu
PhD Student in Computer and Information Science
University of Pennsylvania

[-- Attachment #1.2: Type: text/html, Size: 1205 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel