Xen 4.5-rc1 update (RC1 is out 2014-Oct-24th)

* Xen 4.5-rc1 update (RC1 is out 2014-Oct-24th)
@ 2014-10-24 18:08 konrad.wilk
  2014-10-24 19:16 ` Andrew Cooper
                   ` (4 more replies)
  0 siblings, 5 replies; 27+ messages in thread
From: konrad.wilk @ 2014-10-24 18:08 UTC (permalink / raw)
  To: Wei.Liu2, m.a.young, tiejun.chen, avanzini.arianna,
	boris.ostrovsky, ufimtseva, guijianfeng, eddie.dong, jgross,
	roger.pau, artem.mygaiev, ian.jackson, daniel.kiper,
	ian.campbell, Ian.Jackson, Kelly.Zytaruk, anthony.perard,
	mukesh.rathor, dslutz, aravindp, josh.whitehead,
	robert.vanvossen, Paul.Skentzos, Steve.VanderLeest,
	andrii.tseglytskyi, yang.z.zhang, ross.lagerwall,
	malcolm.crossley, george.dunlap, bob.liu, yjhyun.yoo,
	serge.brosla

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain, Size: 17588 bytes --]

Feature patchsets that did not make it in by today have been put
on the deferred list. If you think your feature should make it in Xen 4.5-rc2
please make your case.

Xen 4.5-rc1 is out today. There are some issues (see 'Known Issues' below)
which are to be fixed in RC2. The official test-day is on Wednesday (Oct 29th)
but if you want to start testing it today - please do!

Details for the test-day are at

http://wiki.xen.org/wiki/Xen_4.5_RC1_test_instructions

In terms of bugs, we have:

#6 linux: pv drivers miss shutdown command if issued too early
#8 "linux, xenbus mutex hangs when rebooting dom0 and guests hung."
#11 qxl hypervisor support
#13 Re: [Xen-devel] man page example: xm block-attach
#18 xl improve support for migration over non-sshlike tunnels
#19 xl migrate transport improvements
#22 xl does not support specifying virtual function for passthrough device
#23 Remove arbitrary LIBXL_MAXMEM_CONSTANT from libxl, see what breaks
#24 xl missing support for encrypted VNC
#27 Re: [Xen-devel] xend vs xl with pci=['<bdf'] wherein the '<bdf>' are not owned by pciback or pcistub will still launch.
#28 support PCI hole resize in qemu-xen
#30 libxl should implement non-suspend-cancel based resume path
#36 credit2 only uses one runqueue instead of one runq per socket
#38 Implement VT-d large pages so we can avoid sharing between EPT
#40 linux pvops: fpu corruption due to incorrect assumptions
#42 "linux, S3 resume of PVHVM fails - missing call to xen_arch_post_suspend?"
#43 "30s delay loading xenfb driver on some systems"
#44 Security policy ambiguities - XSA-108 process post-mortem

And we should start closing them if they are fixed.

= Timeline =

We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:

* Feature Freeze: 24th September 2014
* First RC: 24th October [Friday!] <==== <WE ARE HERE>
* RC2: Unknown. Need to talk to maintainers on scheduling.
* RC2: Unknown. Need to talk to maintainers on scheduling.
* Release: 10th December 2014

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

Bug-fixes, if Acked-by by maintainer, can go anytime before the First
RC. Later on we will need to figure out the risk of regression/reward
to eliminate the possiblity of a bug introducing another bug.

= Prognosis =

The states are: none -> fair -> ok -> good -> done

none - nothing yet
fair - still working on it, patches are prototypes or RFC
ok   - patches posted, acting on review
good - some last minute pieces
done - all done, might have bugs

= Feature freeze exception =

Remember our goal for the release:
  1. A bug-free release
  2. An awesome release
  3. An on-time release

Accepting a new feature may make Xen more awesome; but it also
introduces a risk that it will introduce more bugs.  That bug may be
found before the release (threatening #3), or it may not be found
until after the release (threatening #1).  Each freeze exception
request will attempt to balance the benefits (how awesome the
exception is) vs the risks (will it cause the release to slip, or
worse, cause a bug which goes un-noticed into the final release).

The idea is that today we will be pretty permissive, but that we will
become progressively more conservative until the first RC, which is
scheduled for 3 weeks' time (October 25).  After that, we will only
accept bug fixes.

Bug fixes can be checked in without a freeze exception throughout the
code freeze, unless the maintainer thinks they are particularly high
risk.  In later RC's, we may even begin rejecting bug fixes if the
broken functionality is small and the risk to other functionality is
high.

Features which are currently marked "experimental" or do not at the
moment work at all cannot be broken really; so changes to code only
used by those features should be able to get a freeze exception
easily.

Features which change or add new interfaces which will need to be
supported in a backwards-compatible way (for instance, vNUMA) will
need freeze exceptions to make sure that the interface itself has
enough time to be considered stable.

These are guidelines and principles to give you an idea where we're
coming from; if you think there's a good reason why making an
exception for you will help us achieve goals 1-3 above better than not
doing so, feel free to make your case.

= Open =

== Known issues ==

*  Systemd integration
   Affects CentOS7, SLES12, Fedora Core 21 and Debian Jessie. Xen source contains systemd files that can be used to configure the various run-time services. In the past the distributions would carry their own version of it - but now we host them. This is not yet complete - [[http://lists.xenproject.org/archives/html/xen-devel/2014-10/msg03064.html patches]] for this are being worked on for RC2.
  -  Wei and Olaf

*  Stubdomains build issues
   stubdomains will not build. Fix is in staging (and will make RC2) or [[http://lists.xen.org/archives/html/xen-devel/2014-10/msg02925.html stubdom/Makefile should use QEMU_TRADITIONAL_LOC]]
  -  Michael Young

*  Building against libxl (outside code)
   If you are building against libxl for any APIs before Xen 4.5 you will encounter building errors.
  -  Andrew Cooper

*  Migrating large Windows guests can cause WMI service to hang
   Patch is in 'staging' and will be in RC2.

*  pygrub parsing grub1 problems ==
   pygrub (Python code) has problems parsing grub.cfg file and hence cannot boot PV guests.

*  xc_reserved_device_memory_map in hvmloader to avoid conflicting MMIO/RAM (good)
   v6
   Treating pieces as bug-fixes only.
  -  Tiejun Chen

== Linux ==

*  Linux block multiqueue (ok)
   v3 posted.
  -  Arianna Avanzini

*  VPMU - 'perf' support in Linux (ok)
   Depends on Xen patches
   Acked by David Vrabel
  -  Boris Ostrovsky

*  vNUMA in Linux (ok)
   v6 posted
   git://gitorious.org/vnuma/linux_vnuma.git
  -  Elena Ufimtseva

*  vsyscall in Linux (fair)
  -  Konrad Rzeszutek Wilk

*  COLO Agent in Linux (fair)
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  vAPIC in PVHVM guests (Linux side) (none)
  -  Boris Ostrovsky

*  Fix PAT in Linux kernel (aka Full support for PAT) (good)
   Acked and reposted for v3.18. Waiting for x86 maintainers.
  -  Juergen Gross

== FreeBSD ==

*  PVH FreeBSD dom0 (ok)
   FreeBSD 11 goal. Toolstack side done in Xen 4.5
  -  Roger Pau Monné

== Other OSes (MiniOS, QNX) ==

*  PV drivers for automotive kernels (fair)
  -  Artem Mygaiev

*  mini-os: xenbus changes for rump kernels (ok)
   git://xenbits.xen.org/people/iwj/rumpuser-xen.git
   branch: base.dev-xen-xenbus.v1..dev-xen-xenbus.v1
   v2 posted
  -  Ian Jackson

== GRUB2 ==

*  GRUB2 multiboot2 (fair)
  -  Daniel Kiper

== OSSTEST ==

*  OSSTest: libvirt (good)
  -  Ian Campbell

== Deferred to QEMU v2.next ==

*  Using qemu-upstream in a stubdomain (fair)
   Will use rump kernels.
  -  Ian Jackson

*  AMD Radeon PCI GPU passthrough (none)
   Focusing on Xen 4.2 and qemu-traditional
  -  Kelly Zytaruk

*  Intel IGD PCI GPU passthrough (ok)
   v5 posted
  -  Chen, Tiejun

*  Xen PV block driver in OVMF (UEFI in guest) (ok)
   v1
  -  Anthony PERARD

== Deferred to Xen hypervisor 4.6 ==

*  Xen Boot Information (xbi) (ok)
   Dependency for GRUB2 + EFI work
   http://lists.xen.org/archives/html/xen-devel/2014-10/msg02068.html
   v4, No go for full patchset. Only some of the patches.
   No ARM EFI hardware (yet) available to test them.
  -  Daniel Kiper

*  PVH - AMD hardware support. (fair)
   Posted.
  -  Mukesh Rathor

*  VMware backdoor (hypercall) (ok)
   v5 posted.
  -  Don Slutz

*  extending mem_access support to PV domain (fair)
   RFC v2
  -  Aravindh Puthiyaparambil (aravindp)

*  Repurpose SEDF Scheduler for Real-time (fair)
   RFC patch posted (v2)
  -  Joshua Whitehead, Robert VanVossen

*  ARM remote processor iommu module (GPUs + IPUs) (fair)
   v3 posted
  -  Andrii Tseglytskyi

*  dirty vram / IOMMU bug (fair)
   http://bugs.xenproject.org/xen/bug/38
  -  Zhang, Yang Z

*  Xen multiboot2-EFI support (fair)
   Needed for GrUB2
   Depends on Xen Boot info (rework multiboot and other structs)
   See http://lists.xen.org/archives/html/xen-devel/2013-05/msg02281.html
   RFC posted
  -  Daniel Kiper

*  Support controlling the max C-state sub-state (ok)
   v3 posted
  -  Ross Lagerwall

*  IOMMU ABI for guests to map their DMA regions (fair)
  -  Malcolm Crossley

*  Default to credit2 (none)
   cpu pinning, numa affinity and cpu reservation
  -  George Dunlap

*  Convert tasklet to per-cpu tasklets (fair)
   RFC posted
  -  Konrad Rzeszutek Wilk

*  Further tmem cleanups/fixes (16TB etc) (fair)
  -  Bob Liu

*  1TB slow destruction (ok)
  -  Bob Liu

*  ARM VM save/restore/live migration (none)
   Need to rebased against migrationv2 - no code posted.
  -  Junghyun Yoo

*  ARM GICv2m support (none)
  -  Linaro (unknown)

== Deferred to Xen toolstack 4.6 ==

*  vNUMA in Xen toolstack (ok)
   v11 posted
   Hypervisor part in
   git://gitorious.org/vnuma/xen_vnuma.git:v11
  -  Elena Ufimtseva

*  pvscsi in libxl (fair)
  -  Juergen Gross and Olaf

*  COarse-grain LOck-stepping Virtual Machines in Xen (fair)
   RFC v3 posted, based on remus-v19
  -  Wen Congyang
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  extend the xenstore ring with a 'closing' signal (fair)
   RFC patch posted
  -  David Scott

*  New Migration (v2). (good)
   v7 (libxc and libxl)
   git://xenbits.xen.org/people/andrewcoop/xen.git
   Seems that it might need to slip or we run v1 alongside v2.
  -  Andrew Cooper & David Vrabel

*  libxl migrationv2 patches. (none)
  -  Andrew Cooper & David Vrabel

*  tmem migrationv2 patches. (none)
  -  Bob Liu & Andrew Cooper & David Vrabel

*  Remus using migration-v2 (fair)
   RFC posted - depends on v6 of 'New Migration'
  -  Yang Hongyang

*  snapshot API extension (checkpointing disk) (ok)
   v5
   His email bounces.
  -  Bamvor Jian Zhang

*  Rearrange and cleanup installation destination directories (/var -> var/lib/xen) (fair)
  -  Daniel Kiper

*  libxl/xl - xm compatibility mode for mem-max and mem-set; (ok)
  -  Daniel Kiper

*  xl list --long (and some related xl commands) have some bugs (none)
  -  Zhigang Wang

*  Xen HPET interrupt fixes (fair)
   behind migration v2
  -  Andrew Cooper

*  cpuid leveling (none)
   http://xenbits.xen.org/people/andrewcoop/feature-levelling/feature-levelling-D.pdf
  -  Andrew Cooper

*  live migration knobs, there is no suitable code yet, just ideas (none)
    http://lists.xenproject.org/archives/html/xen-devel/2014-03/msg00785.html
  -  Olaf Hering

*  xl does not handle migrate interruption gracefully (none)
   If you start a localhost migrate, and press "Ctrl-C" in the middle, you get two hung domains
  -  Ian Jackson

*  IO-NUMA - hwloc and xl (none)
   Andrew Cooper had an RFC patch for hwloc
   add restrictions  as to which devices cannot safely/functionally be split apart.
  -  Boris Ostrovsky

*  HVM guest NUMA (none)
  -  Matt Wilson

*  PVH - Migration of PVH DomUs. (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  PVH - Migration of guests from a PVH dom0  (none)
   Depends on migration2 code
  -  Roger Pau Monné

*  "Short" grant copy (just header) of packets. (none)
  -  Zoltan Kiss

== Deferred to Linux's after Xen 4.6 ==

*  ARM - Device assigment usage in Linux code (arch/arm) (none)
   Depends on Xen pieces which are on the Xen 4.6 list.
  -  Julien Grall

*  Linux ARM - Device assigment (fair)
  -  Julien Grall

== Up for grabs ==

*  OSSTest - also test Linux PVH guests

*  PoD fixes
   if you boot with memory <= maxmem we have a size estimation bug

*  TLB flushing without locks in Xen

*  xl does not support specifying virtual function for passthrough device
   http://bugs.xenproject.org/xen/bug/22

*  PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
   http://bugs.xenproject.org/xen/bug/28

*  libx{c,l} error handling cleanup

*  Adding missing 'xend' features in libxl

*  xl list -l on a dom0-only system

*  xl list -l doesn't contain tty console port

*  xl: passing more defaults in configuration in xl.conf
   There are a number of options for which it might be useful to pass a default in xl.conf.  For example, if we could have a default "backend" parameter for vifs, then it would be easy to switch back and forth between a backend in a driver domain and a backend in dom0.

*  PVH - PVH working with shadow.
   Based on Tim's work

*  PVH - PCI passthrough for DomU.

*  AMD performance regressions

*  Performance due to hypercall preemption. More preemptions - slower. (none)

== Completed ==

== Hypervisor ==

*  ARM - introduce GNTTABOP_cache_flush (ok)
   v11
  -  Stefano Stabellini

*  ARM - VGIC emulation (done)
   Reposted as gic and vgic fixes and improvements
   v12
  -  Stefano Stabellini

*  ARM implement mem_access (done)
   v12, two patches for Xen 4.6
   https://github.com/tklengyel/xen/tree arm_memaccess_12-for-4.5
  -  Tamas K Lengyel

*  ARM - Add Odroid-XU (Exynos5410) support (done)
   v6
  -  Suriyan Ramasami

*  ARM GICv3 support (done)
   v11 posted
  -  Vijay Kilari

*  ARM implement mem_access (done)
   v12, two patches for Xen 4.6
   https://github.com/tklengyel/xen/tree arm_memaccess_12-for-4.5
  -  Tamas K Lengyel

*  ARM - MiniOS (done)
   v7 posted
  -  Thomas Leonard

*  ARM XEN_DOMCTL_memory_mapping hypercall for ARM (done)
   v12 posted.
  -  Arianna Avanzini

*  ARM - XENFEAT_grant_map_11 (aka map grants refs at pfn = mfn) (done)
   Provide kernels an grant->MFN lookup
   v4
  -  Stefano Stabellini

*  ARM PSCI v0.2 (done)
   v11 posted
  -  Parth Dixit

*  ARM  - IOMMU support (done)
  -  Julien Grall

*  ARM Interrupt latency reduction (no maintenance interrupts) (good)
  -  Stefano Stabellini

*  ARM DRA7 support (done)
   v3 posted
   v3 with comments applied
  -  Andrii Tseglytskyi

*  ARM: Use super pages in p2m (done)
   v5 posted
  -  Ian Campbell

*  ARM Xen UEFI booting on ARM (done)
   v5
  -  Roy Franz

*  VPMU - 'perf' support in Xen (good)
   v14 posted
   Need reviews/final ack.
  -  Boris Ostrovsky

*  Cache QoS Monitoring - hypercalls (done)
   Just hypercalls - no toolstack changes.
   v15
   Hit a snag with rdmsr/IPI/wrmsr/IPI, possible redesign
  -  Chao Peng, Dongxiao Xu, and Shantong Kang

*  XenRT (Preemptive Global Earliest Deadline First) (done)
   v3
  -  Meng Xu

*  Introspection of HVM guests (done)
   v10, split out in for 4.5 (smaller subset)
  -  Razvan Cojocaru

*  alternative_asm in Xen (done)
  -  Feng Wu

*  SMAP (done)
  -  Feng Wu

*  Re-write of vHPET (done)
   aka hvm/hpet: Detect comparator values in the past
  -  Don Slutz

*  vAPIC in PVHVM guests (Xen side) (done)
  -  Boris Ostrovsky

*  Xen PVH dom0 (done)
  -  Mukesh Rathor

*  amd_ucode cleanups, verify patch size(enhancement) (mostly in master except one patch)

*  Data breakpoint Extension support (new-feat) (in master)

*  Feature masking MSR support (enhancement) (in master)

*  Support BRCM TruManage chip (Serial over LAN support) (new-feat) (in master)

*  fix vmce_amd* functions, unify mce_amd mcheck initialization (fixes/cleanups)

*  multiple AMD container files appended together in initrd (early initramfs)
  -  Aravind and Suravee

*  NUMA memory scrubbing (done)
  -  Konrad Rzeszutek Wilk

*  ioreq-server, aka secondary emulators (done)
  -  Paul Durrant

*  Soft affinity for vcpus (was NUMA affinity for vcpus) (good)
   v11 posted
  -  Dario Faggioli

*  HT enabled, virtualization overhead is high (Xen 4.4) (done)
   kernbench demonstrated it
   Looking and tracing
   http://www.gossamer-threads.com/lists/xen/devel/339409
   False alarm.
  -  Dario Faggioli

== lib{xc,xl} and toolstack ==

*  Build systems fixes/improvements (ok)
  -  Andrew Cooper

*  libxl work - JSON to keep track of guest configs (done)
   Some patches merged, need to post more.
  -  Wei Liu

*  Remus in Xen (libxl) (done)
   v19
   url:  https://github.com/macrosheep/xen/tree/remus-v19
  -  Gui Jianfeng
  -  Yang Hongyang
  -  Dong, Eddie

*  libvirt and xl discard support, so that libvirt can start using it (done)
  -  Olaf Hering

*  OSSTest: upstream QEMU (done)
  -  Ian Campbell

*  rework VM Generation ID (done)
   v7 posted
  -  David Vrabel

*  systemd support (done)
   v11
  -  Luis R. Rodriguez

*  Soft affinity for vcpus libxl/xl changes (done)
   v13 posted
  -  Dario Faggioli

== QEMU ==

*  Bigger PCI hole in QEMU (done)
   Needs to be rebased
  -  Don Slutz

*  QEMU 2.0 branch for qemu-upstream (done)
   It is v2.0 with 2.1 Xen backports.
  -  Stefano Stabellini

== Linux 3.18 and earlier ==

*  pvSCSI in Linux (fronted and backend) (done)
   v6
  -  Juergen Gross

*  Linux PVH dom0 (done)
  -  Mukesh Rathor

*  Netback multiqueue (good)
  -  Wei Liu

*  Linux pvops of Xen EFI hypercall support (done)
  -  Daniel Kiper

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 27+ messages in thread