Xen 4.4 development update: Is PVH a blocker?

Xen 4.4 development update: Is PVH a blocker?

[George Dunlap]

This information will be mirrored on the Xen 4.4 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.4

Our timeline had us start the code freeze last Friday.  However, we
have not released an RC0 because we have been waiting for PVH dom0
support.  Adding bug fixes during RCs makes sense, but RC0 should
contain all of the functionality we expect to be in the final release.

PVH dom0 support is making progress, however it's not that clear when
it will actually be ready to be checked in.  The current discussion is
about refcounting the new p2m type, which is a tricky and delicate
operation (though luckily one which should be limited to domains which
use that type -- at the moment, exclusively PVH dom0s).

If we continue to wait, it is likely that the release will slip.  The question
then is, how long should we continue to wait before we cut our losses and
say we'll wait for PVH dom0 until 4.5?

It would be very nice to have an RC out before Christmas, so
enthusiasts can play around with it over the holidays.  One option
would be to set a date to cut the first RC -- say, next Friday 20
December -- and just use the most recent changeset to pass the
push-gate, whether it has dom0 PVH or not.

Or we can just wait, and take stock of the situation again in January.

Thoughts?

In other news, I will be on holiday and / or travelling for
conferences until 20 January; during that time, Ian Campbell has
agreed to take over my role as Release Coordinator.

= Timeline =

Here is our current timeline based on a 6-month release:

* Feature freeze: 18 October 2013
* Code freezing point: 18 November 2013
* First RCs: 6 December 2013  <== WE ARE HERE
* Release: 21 January 2014

Last updated: 13 December 2013

== Completed ==

* Event channel scalability (FIFO event channels)

* Non-udev scripts for driver domains (non-Linux driver domains)

* Multi-vector PCI MSI (Hypervisor side)

* Improved Spice support on libxl
 - Added Spice vdagent support
 - Added Spice clipboard sharing support
 - Spice usbredirection support for upstream qemu

* PHV domU (experimental only)

* pvgrub2 checked into grub upstream

* ARM64 guest

* Guest EFI booting (tianocore)

* kexec

* Testing: Xen on ARM

* Update to SeaBIOS 1.7.3.1

* Update to qemu 1.6

* SWIOTLB (in Linux 3.13)

* Disk: indirect descriptors (in 3.11)

* Reworked ocaml bindings

== Resolved since last update ==

* Supposed regression from a3513737 ("x86: allow guest to set/clear

* qemu-traditional mis-parses host bus 8 as 0
  - Invalid: parses "08" as octal, as qemu-xen does.
  - Just needs documenting (see below)

== Open ==

* xl support for vnc and vnclisten options with PV guests
 > http://bugs.xenproject.org/xen/bug/25
 Blocker?

* libxl / xl does not handle failure of remote qemu gracefully
  > Easiest way to reproduce:
  >  - set "vncunused=0" and do a local migrate
  >  - The "remote" qemu will fail because the vnc port is in use
  > The failure isn't the problem, but everything being stuck afterwards is

* xl needs to disallow PoD with PCI passthrough
  >see http://xen.1045712.n5.nabble.com/PATCH-VT-d-Dis-allow-PCI-device-assignment-if-PoD-is-enabled-td2547788.html

* qemu-upstream not freeing pirq
 > http://www.gossamer-threads.com/lists/xen/devel/281498
 > http://marc.info/?l=xen-devel&m=137265766424502
 status: patches posted; latest patches need testing
 Not a blocker.

* Race in PV shutdown between tool detection and shutdown watch
 > http://www.gossamer-threads.com/lists/xen/devel/282467
 > Nothing to do with ACPI
 status: Patches posted
 Not a blocker.

* xl does not support specifying virtual function for passthrough device
 > http://bugs.xenproject.org/xen/bug/22
 Too much work to be a blocker.

* xl does not handle migrate interruption gracefully
  > If you start a localhost migrate, and press "Ctrl-C" in the middle,
  > you get two hung domains

* Win2k3 SP2 RTC infinite loops
   > Regression introduced late in Xen-4.3 development
   owner: andrew.cooper@citrix
   status: patches posted, undergoing review. ( v2 ID
1386241748-9617-1-git-send-email-andrew.cooper3@citrix.com )

* HPET interrupt stack overflow (when using hpet_broadcast mode and MSI
capable HPETs)
  owner: andyh@citrix
  status: patches posted, undergoing review iteration.

* PCI hole resize support hvmloader/qemu-traditional/qemu-upstream
with PCI/GPU passthrough
  > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
  > Where Stefano writes:
  > 2) for Xen 4.4 rework the two patches above and improve
  > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
  > enough, it also needs to be able to resize the system memory region
  > (xen.ram) to make room for the bigger pci_hole

* qemu memory leak?
  > http://lists.xen.org/archives/html/xen-users/2013-03/msg00276.html

* qemu-* parses "008" as octal in USB bus.addr format
  > http://bugs.xenproject.org/xen/bug/15
  > just needs documenting

=== Big ticket items ===

* PVH dom0 (w/ Linux)
  blocker
  owner: mukesh@oracle, george@citrix
  status (Linux): Acked, waiting for ABI to be nailed down
  status (Xen): v6 posted; considered a blocker

* libvirt/libxl integration (external)
 - owner: jfehlig@suse, dario@citrix
 - patches posted (should be released before 4.4)
  - migration
  - PCI pass-through
 - In progress
  - integration w/ libvirt's lock manager
  - improved concurrency

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Fri, Dec 13, 2013 at 07:21:07PM +0000, George Dunlap wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> 
> Our timeline had us start the code freeze last Friday.  However, we
> have not released an RC0 because we have been waiting for PVH dom0
> support.  Adding bug fixes during RCs makes sense, but RC0 should
> contain all of the functionality we expect to be in the final release.
> 
> PVH dom0 support is making progress, however it's not that clear when
> it will actually be ready to be checked in.  The current discussion is
> about refcounting the new p2m type, which is a tricky and delicate
> operation (though luckily one which should be limited to domains which
> use that type -- at the moment, exclusively PVH dom0s).
> 
> If we continue to wait, it is likely that the release will slip.  The question
> then is, how long should we continue to wait before we cut our losses and
> say we'll wait for PVH dom0 until 4.5?
> 
> It would be very nice to have an RC out before Christmas, so
> enthusiasts can play around with it over the holidays.  One option
> would be to set a date to cut the first RC -- say, next Friday 20
> December -- and just use the most recent changeset to pass the
> push-gate, whether it has dom0 PVH or not.
> 
> Or we can just wait, and take stock of the situation again in January.

I would of course want to see PVH dom0 in it. I am right now re-working
Mukesh's patches to be applicable for Linux and I hope to get all of this
done by next week.

Which would be really neat to have it all working when v3.14 merge
windows open and Xen 4.4 RC0 is out.

In regards to having RC before X-Mas. Meeh.


> 
> Thoughts?
> 
> In other news, I will be on holiday and / or travelling for
> conferences until 20 January; during that time, Ian Campbell has
> agreed to take over my role as Release Coordinator.

Woot! Enjoy the travels.
 
> 
> = Timeline =
> == Open ==

> * qemu-upstream not freeing pirq
>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>  > http://marc.info/?l=xen-devel&m=137265766424502
>  status: patches posted; latest patches need testing
>  Not a blocker.

I can't test it, b/c qemu-upstream does not work with PCI passthrough
anymore.

> 
> * Race in PV shutdown between tool detection and shutdown watch
>  > http://www.gossamer-threads.com/lists/xen/devel/282467
>  > Nothing to do with ACPI
>  status: Patches posted
>  Not a blocker.

Need to rework them.
> === Big ticket items ===
> 
> * PVH dom0 (w/ Linux)
>   blocker
>   owner: mukesh@oracle, george@citrix
>   status (Linux): Acked, waiting for ABI to be nailed down

That is kind of down. I had reposted the Linux one and I am
reworking them to be easier to grok.

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

> == Open ==
>
Also, xl as opposed to xend, allows me to share a disk without
any fanfare.

Meaning I can do this:

xl block-attach phy:/dev/sda latest1 xvda w
xl block-attach phy:/dev/sda latest2 xvda w

while if I had used 'xend' I had to also append the '!' parameter
to denote it as 'shared'.

Re: Xen 4.4 development update: Is PVH a blocker?

[Sander Eikelenboom]

Friday, December 13, 2013, 8:39:26 PM, you wrote:

> On Fri, Dec 13, 2013 at 07:21:07PM +0000, George Dunlap wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>> 
>> Our timeline had us start the code freeze last Friday.  However, we
>> have not released an RC0 because we have been waiting for PVH dom0
>> support.  Adding bug fixes during RCs makes sense, but RC0 should
>> contain all of the functionality we expect to be in the final release.
>> 
>> PVH dom0 support is making progress, however it's not that clear when
>> it will actually be ready to be checked in.  The current discussion is
>> about refcounting the new p2m type, which is a tricky and delicate
>> operation (though luckily one which should be limited to domains which
>> use that type -- at the moment, exclusively PVH dom0s).
>> 
>> If we continue to wait, it is likely that the release will slip.  The question
>> then is, how long should we continue to wait before we cut our losses and
>> say we'll wait for PVH dom0 until 4.5?
>> 
>> It would be very nice to have an RC out before Christmas, so
>> enthusiasts can play around with it over the holidays.  One option
>> would be to set a date to cut the first RC -- say, next Friday 20
>> December -- and just use the most recent changeset to pass the
>> push-gate, whether it has dom0 PVH or not.
>> 
>> Or we can just wait, and take stock of the situation again in January.

> I would of course want to see PVH dom0 in it. I am right now re-working
> Mukesh's patches to be applicable for Linux and I hope to get all of this
> done by next week.

> Which would be really neat to have it all working when v3.14 merge
> windows open and Xen 4.4 RC0 is out.

> In regards to having RC before X-Mas. Meeh.


>> 
>> Thoughts?
>> 
>> In other news, I will be on holiday and / or travelling for
>> conferences until 20 January; during that time, Ian Campbell has
>> agreed to take over my role as Release Coordinator.

> Woot! Enjoy the travels.
>  
>> 
>> = Timeline =
>> == Open ==

>> * qemu-upstream not freeing pirq
>>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>>  > http://marc.info/?l=xen-devel&m=137265766424502
>>  status: patches posted; latest patches need testing
>>  Not a blocker.

> I can't test it, b/c qemu-upstream does not work with PCI passthrough
> anymore.

Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
(if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)

What are you trying to passthrough ?
What problems do you have / errors do you see ?


>> 
>> * Race in PV shutdown between tool detection and shutdown watch
>>  > http://www.gossamer-threads.com/lists/xen/devel/282467
>>  > Nothing to do with ACPI
>>  status: Patches posted
>>  Not a blocker.

> Need to rework them.
>> === Big ticket items ===
>> 
>> * PVH dom0 (w/ Linux)
>>   blocker
>>   owner: mukesh@oracle, george@citrix
>>   status (Linux): Acked, waiting for ABI to be nailed down

> That is kind of down. I had reposted the Linux one and I am
> reworking them to be easier to grok.

Re: Xen 4.4 development update: Is PVH a blocker?

[Jan Beulich]

>>> On 13.12.13 at 20:21, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 
> 
> Our timeline had us start the code freeze last Friday.  However, we
> have not released an RC0 because we have been waiting for PVH dom0
> support.  Adding bug fixes during RCs makes sense, but RC0 should
> contain all of the functionality we expect to be in the final release.
> 
> PVH dom0 support is making progress, however it's not that clear when
> it will actually be ready to be checked in.  The current discussion is
> about refcounting the new p2m type, which is a tricky and delicate
> operation (though luckily one which should be limited to domains which
> use that type -- at the moment, exclusively PVH dom0s).
> 
> If we continue to wait, it is likely that the release will slip.  The 
> question
> then is, how long should we continue to wait before we cut our losses and
> say we'll wait for PVH dom0 until 4.5?

Even if likely unpopular, given the condition the one critical patch
is in I'd favor not waiting any longer at all, deferring the feature
to 4.5 and cutting RC1 e.g. based on what got pushed over the
weekend.

Jan

Re: Xen 4.4 development update: Is PVH a blocker?

[George Dunlap]

On 12/13/2013 07:43 PM, Konrad Rzeszutek Wilk wrote:
>> == Open ==
>>
> Also, xl as opposed to xend, allows me to share a disk without
> any fanfare.
>
> Meaning I can do this:
>
> xl block-attach phy:/dev/sda latest1 xvda w
> xl block-attach phy:/dev/sda latest2 xvda w
>
> while if I had used 'xend' I had to also append the '!' parameter
> to denote it as 'shared'.

Is xl intended to have that level of information?  The 'l' originally 
meant "light".

  -George

Re: Xen 4.4 development update: Is PVH a blocker?

[Ian Campbell]

On Fri, 2013-12-13 at 14:39 -0500, Konrad Rzeszutek Wilk wrote:
> > * qemu-upstream not freeing pirq
> >  > http://www.gossamer-threads.com/lists/xen/devel/281498
> >  > http://marc.info/?l=xen-devel&m=137265766424502
> >  status: patches posted; latest patches need testing
> >  Not a blocker.
> 
> I can't test it, b/c qemu-upstream does not work with PCI passthrough
> anymore.

Please can we get details of this -- obviously it needs investigating
before the release.

Ian.

Re: Xen 4.4 development update: Is PVH a blocker?

[Ian Campbell]

On Fri, 2013-12-13 at 14:43 -0500, Konrad Rzeszutek Wilk wrote:
> > == Open ==
> >
> Also, xl as opposed to xend, allows me to share a disk without
> any fanfare.
> 
> Meaning I can do this:
> 
> xl block-attach phy:/dev/sda latest1 xvda w
> xl block-attach phy:/dev/sda latest2 xvda w
> 
> while if I had used 'xend' I had to also append the '!' parameter
> to denote it as 'shared'.

Do you find that restriction to be valuable in practice?

Was it ever reliable? How did it cope with /dev/mapper/FOO-BAR
vs /dev/FOO/BAR and other similar aliases (/dev/cdrom etc)?

We could certainly cause xl to swallow the '!' for compatibility but is
the feature itself necessary?

I have a feeling this is mostly implemented by checks in the block
scripts rather than the toolstack itself, perhaps libxl drives them a
bit differently.

Ian.

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

> >> * qemu-upstream not freeing pirq
> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
> >>  > http://marc.info/?l=xen-devel&m=137265766424502
> >>  status: patches posted; latest patches need testing
> >>  Not a blocker.
> 
> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
> > anymore.
> 
> Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
> (if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)
> 
> What are you trying to passthrough ?

-bash-4.1# lspci -s 01:00.0 -v 
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Flags: fast devsel, IRQ 16
        Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
        Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
        I/O ports at e020 [disabled] [size=32]
        Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
        Expansion ROM at fb400000 [disabled] [size=4M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable- Count=10 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: pciback
        Kernel modules: igb


> What problems do you have / errors do you see ?
> 

http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Mon, Dec 16, 2013 at 10:54:17AM +0000, Ian Campbell wrote:
> On Fri, 2013-12-13 at 14:43 -0500, Konrad Rzeszutek Wilk wrote:
> > > == Open ==
> > >
> > Also, xl as opposed to xend, allows me to share a disk without
> > any fanfare.
> > 
> > Meaning I can do this:
> > 
> > xl block-attach phy:/dev/sda latest1 xvda w
> > xl block-attach phy:/dev/sda latest2 xvda w
> > 
> > while if I had used 'xend' I had to also append the '!' parameter
> > to denote it as 'shared'.
> 
> Do you find that restriction to be valuable in practice?

It protects me from doing silly mistakes.
> 
> Was it ever reliable? How did it cope with /dev/mapper/FOO-BAR
> vs /dev/FOO/BAR and other similar aliases (/dev/cdrom etc)?

I am not sure - but it did work across device mapper.
> 
> We could certainly cause xl to swallow the '!' for compatibility but is
> the feature itself necessary?

Not for Xen 4.4.
> 
> I have a feeling this is mostly implemented by checks in the block
> scripts rather than the toolstack itself, perhaps libxl drives them a
> bit differently.

I can do some investigation for this. After New Year though.

> 
> Ian.
> 

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Mon, Dec 16, 2013 at 10:51:40AM +0000, Ian Campbell wrote:
> On Fri, 2013-12-13 at 14:39 -0500, Konrad Rzeszutek Wilk wrote:
> > > * qemu-upstream not freeing pirq
> > >  > http://www.gossamer-threads.com/lists/xen/devel/281498
> > >  > http://marc.info/?l=xen-devel&m=137265766424502
> > >  status: patches posted; latest patches need testing
> > >  Not a blocker.
> > 
> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
> > anymore.
> 
> Please can we get details of this -- obviously it needs investigating
> before the release.

I somehow forgot to CC you on it. Here is the link:

http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded


> 
> Ian.
> 
> 

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Mon, Dec 16, 2013 at 10:12:20AM -0500, Konrad Rzeszutek Wilk wrote:
> On Mon, Dec 16, 2013 at 10:51:40AM +0000, Ian Campbell wrote:
> > On Fri, 2013-12-13 at 14:39 -0500, Konrad Rzeszutek Wilk wrote:
> > > > * qemu-upstream not freeing pirq
> > > >  > http://www.gossamer-threads.com/lists/xen/devel/281498
> > > >  > http://marc.info/?l=xen-devel&m=137265766424502
> > > >  status: patches posted; latest patches need testing
> > > >  Not a blocker.
> > > 
> > > I can't test it, b/c qemu-upstream does not work with PCI passthrough
> > > anymore.
> > 
> > Please can we get details of this -- obviously it needs investigating
> > before the release.
> 
> I somehow forgot to CC you on it. Here is the link:
> 
> http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded

And just to make sure a bug-fix didn't crep in, I removed the qemu-xen-dir
so that it would fetch a new one and I see the problem. The top git commit
is commit b97307ecaad98360f41ea36cd9674ef810c4f8cf
Author: Matthew Daley <mattjd@gmail.com>
Date:   Thu Oct 10 14:10:48 2013 +0000

    xen_disk: mark ioreq as mapped before unmapping in error case

> 
> 
> > 
> > Ian.
> > 
> > 

Re: Xen 4.4 development update: Is PVH a blocker?

[Ian Campbell]

On Mon, 2013-12-16 at 10:12 -0500, Konrad Rzeszutek Wilk wrote:
> On Mon, Dec 16, 2013 at 10:51:40AM +0000, Ian Campbell wrote:
> > On Fri, 2013-12-13 at 14:39 -0500, Konrad Rzeszutek Wilk wrote:
> > > > * qemu-upstream not freeing pirq
> > > >  > http://www.gossamer-threads.com/lists/xen/devel/281498
> > > >  > http://marc.info/?l=xen-devel&m=137265766424502
> > > >  status: patches posted; latest patches need testing
> > > >  Not a blocker.
> > > 
> > > I can't test it, b/c qemu-upstream does not work with PCI passthrough
> > > anymore.
> > 
> > Please can we get details of this -- obviously it needs investigating
> > before the release.
> 
> I somehow forgot to CC you on it. Here is the link:
> 
> http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded

Thanks, I vaguely recall seeing that go past.

Looks like Anthony and Wei are on it.

Ian.

Re: Xen 4.4 development update: Is PVH a blocker?

[Sander Eikelenboom]

Monday, December 16, 2013, 4:09:06 PM, you wrote:

>> >> * qemu-upstream not freeing pirq
>> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>> >>  > http://marc.info/?l=xen-devel&m=137265766424502
>> >>  status: patches posted; latest patches need testing
>> >>  Not a blocker.
>> 
>> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
>> > anymore.
>> 
>> Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
>> (if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)
>> 
>> What are you trying to passthrough ?

> -bash-4.1# lspci -s 01:00.0 -v 
> 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>         Flags: fast devsel, IRQ 16
>         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>         Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>         I/O ports at e020 [disabled] [size=32]
>         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>         Expansion ROM at fb400000 [disabled] [size=4M]
>         Capabilities: [40] Power Management version 3
>         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
>         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
>         Capabilities: [a0] Express Endpoint, MSI 00
>         Capabilities: [100] Advanced Error Reporting
>         Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
>         Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
>         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
>         Kernel driver in use: pciback
>         Kernel modules: igb


>> What problems do you have / errors do you see ?
>> 

> http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded


Could it be related to the outstanding issue (since 4.3) of:

* PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
  > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
  > Where Stefano writes:
  > 2) for Xen 4.4 rework the two patches above and improve
  > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
  > enough, it also needs to be able to resize the system memory region
  > (xen.ram) to make room for the bigger pci_hole

  Your card seems to need quite some resources, and restricted to 32bit, even other SR-IOV cards seems to to with less and or support 32 bit.

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Mon, Dec 16, 2013 at 06:24:20PM +0100, Sander Eikelenboom wrote:
> 
> Monday, December 16, 2013, 4:09:06 PM, you wrote:
> 
> >> >> * qemu-upstream not freeing pirq
> >> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
> >> >>  > http://marc.info/?l=xen-devel&m=137265766424502
> >> >>  status: patches posted; latest patches need testing
> >> >>  Not a blocker.
> >> 
> >> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
> >> > anymore.
> >> 
> >> Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
> >> (if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)
> >> 
> >> What are you trying to passthrough ?
> 
> > -bash-4.1# lspci -s 01:00.0 -v 
> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> >         Flags: fast devsel, IRQ 16
> >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
> >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> >         I/O ports at e020 [disabled] [size=32]
> >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
> >         Expansion ROM at fb400000 [disabled] [size=4M]
> >         Capabilities: [40] Power Management version 3
> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
> >         Capabilities: [a0] Express Endpoint, MSI 00
> >         Capabilities: [100] Advanced Error Reporting
> >         Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
> >         Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
> >         Kernel driver in use: pciback
> >         Kernel modules: igb
> 
> 
> >> What problems do you have / errors do you see ?
> >> 
> 
> > http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded
> 
> 
> Could it be related to the outstanding issue (since 4.3) of:
> 
> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>   > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
>   > Where Stefano writes:
>   > 2) for Xen 4.4 rework the two patches above and improve
>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
>   > enough, it also needs to be able to resize the system memory region
>   > (xen.ram) to make room for the bigger pci_hole
> 
>   Your card seems to need quite some resources, and restricted to 32bit, even other SR-IOV cards seems to to with less and or support 32 bit.
> 

I don't think that is the problem. The QEMU-xen did work at some point - until
I blew away 'qemu-xen-dir' to get the newest and best. So I think the problem
I am seeing is with QEMU 1.6 regression.

Re: Xen 4.4 development update: Is PVH a blocker?

[Sander Eikelenboom]

Monday, December 16, 2013, 6:46:35 PM, you wrote:

> On Mon, Dec 16, 2013 at 06:24:20PM +0100, Sander Eikelenboom wrote:
>> 
>> Monday, December 16, 2013, 4:09:06 PM, you wrote:
>> 
>> >> >> * qemu-upstream not freeing pirq
>> >> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>> >> >>  > http://marc.info/?l=xen-devel&m=137265766424502
>> >> >>  status: patches posted; latest patches need testing
>> >> >>  Not a blocker.
>> >> 
>> >> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
>> >> > anymore.
>> >> 
>> >> Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
>> >> (if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)
>> >> 
>> >> What are you trying to passthrough ?
>> 
>> > -bash-4.1# lspci -s 01:00.0 -v 
>> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>> >         Flags: fast devsel, IRQ 16
>> >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>> >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>> >         I/O ports at e020 [disabled] [size=32]
>> >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>> >         Expansion ROM at fb400000 [disabled] [size=4M]
>> >         Capabilities: [40] Power Management version 3
>> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
>> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
>> >         Capabilities: [a0] Express Endpoint, MSI 00
>> >         Capabilities: [100] Advanced Error Reporting
>> >         Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
>> >         Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
>> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
>> >         Kernel driver in use: pciback
>> >         Kernel modules: igb
>> 
>> 
>> >> What problems do you have / errors do you see ?
>> >> 
>> 
>> > http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded
>> 
>> 
>> Could it be related to the outstanding issue (since 4.3) of:
>> 
>> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>>   > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
>>   > Where Stefano writes:
>>   > 2) for Xen 4.4 rework the two patches above and improve
>>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
>>   > enough, it also needs to be able to resize the system memory region
>>   > (xen.ram) to make room for the bigger pci_hole
>> 
>>   Your card seems to need quite some resources, and restricted to 32bit, even other SR-IOV cards seems to to with less and or support 32 bit.
>> 

> I don't think that is the problem. The QEMU-xen did work at some point - until
> I blew away 'qemu-xen-dir' to get the newest and best. So I think the problem
> I am seeing is with QEMU 1.6 regression.


Could try to get the latest upstream qemu and seabios patches then by changing your trees in Config.mk to the upstream qemu tree:

git://git.qemu.org/qemu.git

and

git://git.qemu.org/seabios.git
(at the extra config option questions, just except the defaults).

It's what i run at the moment, could be there is a off-chance there is a patch upstream that fixes your problem that's not in the xen qemu upstream tree yet.

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Mon, Dec 16, 2013 at 06:59:16PM +0100, Sander Eikelenboom wrote:
> 
> Monday, December 16, 2013, 6:46:35 PM, you wrote:
> 
> > On Mon, Dec 16, 2013 at 06:24:20PM +0100, Sander Eikelenboom wrote:
> >> 
> >> Monday, December 16, 2013, 4:09:06 PM, you wrote:
> >> 
> >> >> >> * qemu-upstream not freeing pirq
> >> >> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
> >> >> >>  > http://marc.info/?l=xen-devel&m=137265766424502
> >> >> >>  status: patches posted; latest patches need testing
> >> >> >>  Not a blocker.
> >> >> 
> >> >> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
> >> >> > anymore.
> >> >> 
> >> >> Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
> >> >> (if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)
> >> >> 
> >> >> What are you trying to passthrough ?
> >> 
> >> > -bash-4.1# lspci -s 01:00.0 -v 
> >> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
> >> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> >> >         Flags: fast devsel, IRQ 16
> >> >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
> >> >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> >> >         I/O ports at e020 [disabled] [size=32]
> >> >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
> >> >         Expansion ROM at fb400000 [disabled] [size=4M]
> >> >         Capabilities: [40] Power Management version 3
> >> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> >> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
> >> >         Capabilities: [a0] Express Endpoint, MSI 00
> >> >         Capabilities: [100] Advanced Error Reporting
> >> >         Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
> >> >         Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
> >> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
> >> >         Kernel driver in use: pciback
> >> >         Kernel modules: igb
> >> 
> >> 
> >> >> What problems do you have / errors do you see ?
> >> >> 
> >> 
> >> > http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded
> >> 
> >> 
> >> Could it be related to the outstanding issue (since 4.3) of:
> >> 
> >> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
> >>   > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
> >>   > Where Stefano writes:
> >>   > 2) for Xen 4.4 rework the two patches above and improve
> >>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
> >>   > enough, it also needs to be able to resize the system memory region
> >>   > (xen.ram) to make room for the bigger pci_hole
> >> 
> >>   Your card seems to need quite some resources, and restricted to 32bit, even other SR-IOV cards seems to to with less and or support 32 bit.
> >> 
> 
> > I don't think that is the problem. The QEMU-xen did work at some point - until
> > I blew away 'qemu-xen-dir' to get the newest and best. So I think the problem
> > I am seeing is with QEMU 1.6 regression.
> 
> 
> Could try to get the latest upstream qemu and seabios patches then by changing your trees in Config.mk to the upstream qemu tree:
> 
> git://git.qemu.org/qemu.git
> 
> and
> 
> git://git.qemu.org/seabios.git
> (at the extra config option questions, just except the defaults).
> 
> It's what i run at the moment, could be there is a off-chance there is a patch upstream that fixes your problem that's not in the xen qemu upstream tree yet.

I did this:
diff --git a/Config.mk b/Config.mk
index 2007b22..7702fff 100644
--- a/Config.mk
+++ b/Config.mk
@@ -235,7 +235,13 @@ SEABIOS_UPSTREAM_URL ?= git://xenbits.xen.org/seabios.git
 endif
 OVMF_UPSTREAM_REVISION ?= 447d264115c476142f884af0be287622cd244423
 QEMU_UPSTREAM_REVISION ?= master
-SEABIOS_UPSTREAM_TAG ?= rel-1.7.3.1
+
+SEABIOS_UPSTREAM_URL = git://git.qemu.org/seabios.git
+QEMU_UPSTREAM_URL = git://git.qemu.org/qemu.git
+
+SEABIOS_UPSTREAM_TAG = origin/master
+
+#SEABIOS_UPSTREAM_TAG ?= rel-1.7.3.1
 # Fri Aug 2 14:12:09 2013 -0400
 # Fix bug in CBFS file walking with compressed files.
 

rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
rebuilt and had the same issue :-(

> 

Re: Xen 4.4 development update: Is PVH a blocker?

[Sander Eikelenboom]

Monday, December 16, 2013, 8:34:03 PM, you wrote:

> On Mon, Dec 16, 2013 at 06:59:16PM +0100, Sander Eikelenboom wrote:
>> 
>> Monday, December 16, 2013, 6:46:35 PM, you wrote:
>> 
>> > On Mon, Dec 16, 2013 at 06:24:20PM +0100, Sander Eikelenboom wrote:
>> >> 
>> >> Monday, December 16, 2013, 4:09:06 PM, you wrote:
>> >> 
>> >> >> >> * qemu-upstream not freeing pirq
>> >> >> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>> >> >> >>  > http://marc.info/?l=xen-devel&m=137265766424502
>> >> >> >>  status: patches posted; latest patches need testing
>> >> >> >>  Not a blocker.
>> >> >> 
>> >> >> > I can't test it, b/c qemu-upstream does not work with PCI passthrough
>> >> >> > anymore.
>> >> >> 
>> >> >> Strange, pci passthrough in general works for me on "simple" pci devices (usb cards, videograbber) with latest qemu-upstream and seabios upstream.
>> >> >> (if you have no need for pci rombars, no vga passthrough, no need for specifying funky things about which virtual functions to passthrough)
>> >> >> 
>> >> >> What are you trying to passthrough ?
>> >> 
>> >> > -bash-4.1# lspci -s 01:00.0 -v 
>> >> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
>> >> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
>> >> >         Flags: fast devsel, IRQ 16
>> >> >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled] [size=128K]
>> >> >         Memory at fb800000 (32-bit, non-prefetchable) [disabled] [size=4M]
>> >> >         I/O ports at e020 [disabled] [size=32]
>> >> >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled] [size=16K]
>> >> >         Expansion ROM at fb400000 [disabled] [size=4M]
>> >> >         Capabilities: [40] Power Management version 3
>> >> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
>> >> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
>> >> >         Capabilities: [a0] Express Endpoint, MSI 00
>> >> >         Capabilities: [100] Advanced Error Reporting
>> >> >         Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
>> >> >         Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
>> >> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
>> >> >         Kernel driver in use: pciback
>> >> >         Kernel modules: igb
>> >> 
>> >> 
>> >> >> What problems do you have / errors do you see ?
>> >> >> 
>> >> 
>> >> > http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded
>> >> 
>> >> 
>> >> Could it be related to the outstanding issue (since 4.3) of:
>> >> 
>> >> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU passthrough
>> >>   > http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
>> >>   > Where Stefano writes:
>> >>   > 2) for Xen 4.4 rework the two patches above and improve
>> >>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
>> >>   > enough, it also needs to be able to resize the system memory region
>> >>   > (xen.ram) to make room for the bigger pci_hole
>> >> 
>> >>   Your card seems to need quite some resources, and restricted to 32bit, even other SR-IOV cards seems to to with less and or support 32 bit.
>> >> 
>> 
>> > I don't think that is the problem. The QEMU-xen did work at some point - until
>> > I blew away 'qemu-xen-dir' to get the newest and best. So I think the problem
>> > I am seeing is with QEMU 1.6 regression.
>> 
>> 
>> Could try to get the latest upstream qemu and seabios patches then by changing your trees in Config.mk to the upstream qemu tree:
>> 
>> git://git.qemu.org/qemu.git
>> 
>> and
>> 
>> git://git.qemu.org/seabios.git
>> (at the extra config option questions, just except the defaults).
>> 
>> It's what i run at the moment, could be there is a off-chance there is a patch upstream that fixes your problem that's not in the xen qemu upstream tree yet.

> I did this:
> diff --git a/Config.mk b/Config.mk
> index 2007b22..7702fff 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -235,7 +235,13 @@ SEABIOS_UPSTREAM_URL ?= git://xenbits.xen.org/seabios.git
>  endif
>  OVMF_UPSTREAM_REVISION ?= 447d264115c476142f884af0be287622cd244423
>  QEMU_UPSTREAM_REVISION ?= master
> -SEABIOS_UPSTREAM_TAG ?= rel-1.7.3.1
> +
> +SEABIOS_UPSTREAM_URL = git://git.qemu.org/seabios.git
> +QEMU_UPSTREAM_URL = git://git.qemu.org/qemu.git
> +
> +SEABIOS_UPSTREAM_TAG = origin/master
> +
> +#SEABIOS_UPSTREAM_TAG ?= rel-1.7.3.1
>  # Fri Aug 2 14:12:09 2013 -0400
>  # Fix bug in CBFS file walking with compressed files.
>  

> rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
> rebuilt and had the same issue :-(

Hmm so taking it forward doesn't cut it .. going back is probably going to be searching for a needle in a haystack.
Finding many incomptabilities. I tried it with vga passthrough to find back a working config i knew i once had .. but to no avail :-(

It's at least a good testdevice :-) seems to stretch the limits ...

>> 

Re: Xen 4.4 development update: Is PVH a blocker?

[Mukesh Rathor]

On Mon, 16 Dec 2013 08:47:07 +0000
"Jan Beulich" <JBeulich@suse.com> wrote:

> >>> On 13.12.13 at 20:21, George Dunlap <George.Dunlap@eu.citrix.com>
> >>> wrote:
> > This information will be mirrored on the Xen 4.4 Roadmap wiki page:
> >  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 
> > 
> > Our timeline had us start the code freeze last Friday.  However, we
> > have not released an RC0 because we have been waiting for PVH dom0
> > support.  Adding bug fixes during RCs makes sense, but RC0 should
> > contain all of the functionality we expect to be in the final
> > release.
> > 
> > PVH dom0 support is making progress, however it's not that clear
> > when it will actually be ready to be checked in.  The current
> > discussion is about refcounting the new p2m type, which is a tricky
> > and delicate operation (though luckily one which should be limited
> > to domains which use that type -- at the moment, exclusively PVH
> > dom0s).
> > 
> > If we continue to wait, it is likely that the release will slip.
> > The question
> > then is, how long should we continue to wait before we cut our
> > losses and say we'll wait for PVH dom0 until 4.5?
> 
> Even if likely unpopular, given the condition the one critical patch
> is in I'd favor not waiting any longer at all, deferring the feature
> to 4.5 and cutting RC1 e.g. based on what got pushed over the
> weekend.

Aaah, we are so close, it looks like other than patch 6, all others
are ok. The patch #6 had 3 comments from Tim:

   - switch order in write_ept_entry() : Done.
   - Add check in Non-ept code: done.
   - Release of refcnt in case of p2m teardown: Since this only applies
     to control domain being destroyed, I don't think it's that critical
     for 4.4, so submitting a separate patch for it. That should make the 
     series eligible for 4.4 assuming Tim OK's the above two changes.
     Tim kindly take a look.

Having it in 4.4 would be nice as it would all the community users and
inhouse QA to test it all once, domU and dom0. It would also allow
my work and others to then focus on all the follow on PVH work like:
32bit, performance, amd port, hotplugs, etc... 

So, I hope you'll reconsider.

thanks
mukesh

Re: Xen 4.4 development update: Is PVH a blocker?

[Jan Beulich]

>>> On 17.12.13 at 00:45, Mukesh Rathor <mukesh.rathor@oracle.com> wrote:
> Aaah, we are so close, it looks like other than patch 6, all others
> are ok. The patch #6 had 3 comments from Tim:
> 
>    - switch order in write_ept_entry() : Done.
>    - Add check in Non-ept code: done.
>    - Release of refcnt in case of p2m teardown: Since this only applies
>      to control domain being destroyed, I don't think it's that critical
>      for 4.4, so submitting a separate patch for it. That should make the 
>      series eligible for 4.4 assuming Tim OK's the above two changes.

The question really is how sure you and Tim are that the improper
refcounting can really only affect PVH control domain teardown.
If any other path can result in leaks/mistakes here, this would be
unacceptable. And considering how tricky some of the recounting
has become over time, I'm personally not convinced yet that you're
not introducing issues onto unrelated code paths.

Jan

Re: Xen 4.4 development update: Is PVH a blocker?

[Tim Deegan]

At 08:47 +0000 on 16 Dec (1387180027), Jan Beulich wrote:
> >>> On 13.12.13 at 20:21, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> > This information will be mirrored on the Xen 4.4 Roadmap wiki page:
> >  http://wiki.xen.org/wiki/Xen_Roadmap/4.4 
> > 
> > Our timeline had us start the code freeze last Friday.  However, we
> > have not released an RC0 because we have been waiting for PVH dom0
> > support.  Adding bug fixes during RCs makes sense, but RC0 should
> > contain all of the functionality we expect to be in the final release.
> > 
> > PVH dom0 support is making progress, however it's not that clear when
> > it will actually be ready to be checked in.  The current discussion is
> > about refcounting the new p2m type, which is a tricky and delicate
> > operation (though luckily one which should be limited to domains which
> > use that type -- at the moment, exclusively PVH dom0s).
> > 
> > If we continue to wait, it is likely that the release will slip.  The 
> > question
> > then is, how long should we continue to wait before we cut our losses and
> > say we'll wait for PVH dom0 until 4.5?
> 
> Even if likely unpopular, given the condition the one critical patch
> is in I'd favor not waiting any longer at all, deferring the feature
> to 4.5 and cutting RC1 e.g. based on what got pushed over the
> weekend.

+1.  The remeining changes that I'm aware of touch non-pvh code and
refcounting code, neither of which seems like a good idea at this
point, even if they were complete.

Tim.

Re: Xen 4.4 development update: Is PVH a blocker?

[George Dunlap]

On 12/17/2013 11:58 AM, Tim Deegan wrote:
> At 08:47 +0000 on 16 Dec (1387180027), Jan Beulich wrote:
>>>>> On 13.12.13 at 20:21, George Dunlap <George.Dunlap@eu.citrix.com> wrote:
>>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>>
>>> Our timeline had us start the code freeze last Friday.  However, we
>>> have not released an RC0 because we have been waiting for PVH dom0
>>> support.  Adding bug fixes during RCs makes sense, but RC0 should
>>> contain all of the functionality we expect to be in the final release.
>>>
>>> PVH dom0 support is making progress, however it's not that clear when
>>> it will actually be ready to be checked in.  The current discussion is
>>> about refcounting the new p2m type, which is a tricky and delicate
>>> operation (though luckily one which should be limited to domains which
>>> use that type -- at the moment, exclusively PVH dom0s).
>>>
>>> If we continue to wait, it is likely that the release will slip.  The
>>> question
>>> then is, how long should we continue to wait before we cut our losses and
>>> say we'll wait for PVH dom0 until 4.5?
>>
>> Even if likely unpopular, given the condition the one critical patch
>> is in I'd favor not waiting any longer at all, deferring the feature
>> to 4.5 and cutting RC1 e.g. based on what got pushed over the
>> weekend.
>
> +1.  The remeining changes that I'm aware of touch non-pvh code and
> refcounting code, neither of which seems like a good idea at this
> point, even if they were complete.

Right -- I think we're going to have to go ahead without it then.

FWIW I was always expecting dom0 PVH not to make it; it was Jan who 
first suggested making it a blocker.  But I think that was before we 
realized how tricky the p2m stuff was going to be.

I haven't taken a close look at the patches, but browsing the 
conversation, it seems like waiting is the best option.  For refcounting 
in particular, we don't want to rush things.  Bugs are subtle and may 
not manifest for quite a while.  It would be better to check this in at 
the beginning of a release cycle and we had the full time to shake 
things out.

  -George

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

> > rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
> > rebuilt and had the same issue :-(
> 
> Hmm so taking it forward doesn't cut it .. going back is probably going to be searching for a needle in a haystack.
> Finding many incomptabilities. I tried it with vga passthrough to find back a working config i knew i once had .. but to no avail :-(

Yikes. Do you get a similar error message in QEMU?
> 
> It's at least a good testdevice :-) seems to stretch the limits ...

I also tried it with my GPU:
05:00.0 VGA compatible controller: NVIDIA Corporation GF104GLM [Quadro 4000M] (rev a1) (prog-if 00 [VGA controller])
        Subsystem: Gigabyte Technology Co., Ltd Device 34fc
        Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
        Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
        Interrupt: pin A routed to IRQ 20
        Region 0: Memory at fc000000 (32-bit, non-prefetchable) [disabled] [size=32M]
        Region 1: Memory at d0000000 (64-bit, prefetchable) [disabled] [size=128M]
        Region 3: Memory at d8000000 (64-bit, prefetchable) [disabled] [size=64M]
        Region 5: I/O ports at c000 [disabled] [size=128]
        Expansion ROM at fe000000 [disabled] [size=512K]
        Capabilities: <access denied>
        Kernel driver in use: pciback


But my GPU's BAR (fc000000) is right smack in where the HVMloader stashes the ACPI tables -
so I figured I would try with something less complex, like a NIC.
  
> 
> >> 
> 
> 

Re: Xen 4.4 development update: Is PVH a blocker?

[Sander Eikelenboom]

Tuesday, December 17, 2013, 3:35:45 PM, you wrote:

>> > rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
>> > rebuilt and had the same issue :-(
>> 
>> Hmm so taking it forward doesn't cut it .. going back is probably going to be searching for a needle in a haystack.
>> Finding many incomptabilities. I tried it with vga passthrough to find back a working config i knew i once had .. but to no avail :-(

> Yikes. Do you get a similar error message in QEMU?

Nope my problem (at present) is with the rom bar, that is special cased around everywhere (hvmloader, seabios, qemu, guest kernel).
And it appears all fine, but in fact in the guest it isn't backed by the actual rom data (neither by a copy, neither by passing it through).

Already tried by letting hvmloader load it as a optionrom, but that didn't cut it.

When dumping the rom of the devices passed through in the guest,
i was seeing al kinds of romdata from the emulated devices at the address given to the passed through devive,
but not the romdata from the passedthrough device.

And that was when i stumbled upon the whole xen_platform_pci problem,
as i was trying to switch of as many emulated devices as possible (in the hope it was some interference between those.

After that tried KVM .. to see if it is in any way feasible and possible with my system (iommu + gfx card),
that resulted in almost instant succes .. so yes it should be possible ...

>> 
>> It's at least a good testdevice :-) seems to stretch the limits ...

> I also tried it with my GPU:
> 05:00.0 VGA compatible controller: NVIDIA Corporation GF104GLM [Quadro 4000M] (rev a1) (prog-if 00 [VGA controller])
>         Subsystem: Gigabyte Technology Co., Ltd Device 34fc
>         Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
>         Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>         Interrupt: pin A routed to IRQ 20
>         Region 0: Memory at fc000000 (32-bit, non-prefetchable) [disabled] [size=32M]
>         Region 1: Memory at d0000000 (64-bit, prefetchable) [disabled] [size=128M]
>         Region 3: Memory at d8000000 (64-bit, prefetchable) [disabled] [size=64M]
>         Region 5: I/O ports at c000 [disabled] [size=128]
>         Expansion ROM at fe000000 [disabled] [size=512K]
>         Capabilities: <access denied>
>         Kernel driver in use: pciback


> But my GPU's BAR (fc000000) is right smack in where the HVMloader stashes the ACPI tables -
> so I figured I would try with something less complex, like a NIC.
>   
>> 
>> >> 
>> 
>> 

Re: Xen 4.4 development update: Is PVH a blocker?

[Anthony Liguori]

[-- Attachment #1.1: Type: text/plain, Size: 5046 bytes --]

On Dec 16, 2013 11:37 AM, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
wrote:
>
> On Mon, Dec 16, 2013 at 06:59:16PM +0100, Sander Eikelenboom wrote:
> >
> > Monday, December 16, 2013, 6:46:35 PM, you wrote:
> >
> > > On Mon, Dec 16, 2013 at 06:24:20PM +0100, Sander Eikelenboom wrote:
> > >>
> > >> Monday, December 16, 2013, 4:09:06 PM, you wrote:
> > >>
> > >> >> >> * qemu-upstream not freeing pirq
> > >> >> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
> > >> >> >>  > http://marc.info/?l=xen-devel&m=137265766424502
> > >> >> >>  status: patches posted; latest patches need testing
> > >> >> >>  Not a blocker.
> > >> >>
> > >> >> > I can't test it, b/c qemu-upstream does not work with PCI
passthrough
> > >> >> > anymore.
> > >> >>
> > >> >> Strange, pci passthrough in general works for me on "simple" pci
devices (usb cards, videograbber) with latest qemu-upstream and seabios
upstream.
> > >> >> (if you have no need for pci rombars, no vga passthrough, no need
for specifying funky things about which virtual functions to passthrough)
> > >> >>
> > >> >> What are you trying to passthrough ?
> > >>
> > >> > -bash-4.1# lspci -s 01:00.0 -v
> > >> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
Network Connection (rev 01)
> > >> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server
Adapter
> > >> >         Flags: fast devsel, IRQ 16
> > >> >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled]
[size=128K]
> > >> >         Memory at fb800000 (32-bit, non-prefetchable) [disabled]
[size=4M]
> > >> >         I/O ports at e020 [disabled] [size=32]
> > >> >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled]
[size=16K]
> > >> >         Expansion ROM at fb400000 [disabled] [size=4M]
> > >> >         Capabilities: [40] Power Management version 3
> > >> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> > >> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
> > >> >         Capabilities: [a0] Express Endpoint, MSI 00
> > >> >         Capabilities: [100] Advanced Error Reporting
> > >> >         Capabilities: [140] Device Serial Number
00-1b-21-ff-ff-45-d9-ac
> > >> >         Capabilities: [150] Alternative Routing-ID Interpretation
(ARI)
> > >> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
> > >> >         Kernel driver in use: pciback
> > >> >         Kernel modules: igb
> > >>
> > >>
> > >> >> What problems do you have / errors do you see ?
> > >> >>
> > >>
> > >> >
http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded
> > >>
> > >>
> > >> Could it be related to the outstanding issue (since 4.3) of:
> > >>
> > >> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream
with PCI/GPU passthrough
> > >>   >
http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
> > >>   > Where Stefano writes:
> > >>   > 2) for Xen 4.4 rework the two patches above and improve
> > >>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is
not
> > >>   > enough, it also needs to be able to resize the system memory
region
> > >>   > (xen.ram) to make room for the bigger pci_hole
> > >>
> > >>   Your card seems to need quite some resources, and restricted to
32bit, even other SR-IOV cards seems to to with less and or support 32 bit.
> > >>
> >
> > > I don't think that is the problem. The QEMU-xen did work at some
point - until
> > > I blew away 'qemu-xen-dir' to get the newest and best. So I think the
problem
> > > I am seeing is with QEMU 1.6 regression.
> >
> >
> > Could try to get the latest upstream qemu and seabios patches then by
changing your trees in Config.mk to the upstream qemu tree:
> >
> > git://git.qemu.org/qemu.git
> >
> > and
> >
> > git://git.qemu.org/seabios.git
> > (at the extra config option questions, just except the defaults).
> >
> > It's what i run at the moment, could be there is a off-chance there is
a patch upstream that fixes your problem that's not in the xen qemu
upstream tree yet.
>
> I did this:
> diff --git a/Config.mk b/Config.mk
> index 2007b22..7702fff 100644
> --- a/Config.mk
> +++ b/Config.mk
> @@ -235,7 +235,13 @@ SEABIOS_UPSTREAM_URL ?= git://
xenbits.xen.org/seabios.git
>  endif
>  OVMF_UPSTREAM_REVISION ?= 447d264115c476142f884af0be287622cd244423
>  QEMU_UPSTREAM_REVISION ?= master

QEMU is bisectable so if the old rev worked you can find the commit that
stopped working for youe card.

Regards,

Anthony Liguori

> -SEABIOS_UPSTREAM_TAG ?= rel-1.7.3.1
> +
> +SEABIOS_UPSTREAM_URL = git://git.qemu.org/seabios.git
> +QEMU_UPSTREAM_URL = git://git.qemu.org/qemu.git
> +
> +SEABIOS_UPSTREAM_TAG = origin/master
> +
> +#SEABIOS_UPSTREAM_TAG ?= rel-1.7.3.1
>  # Fri Aug 2 14:12:09 2013 -0400
>  # Fix bug in CBFS file walking with compressed files.
>
>
> rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
> rebuilt and had the same issue :-(
>
> >
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

[-- Attachment #1.2: Type: text/html, Size: 7521 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Tue, Dec 17, 2013 at 07:04:23AM -0800, Anthony Liguori wrote:
> On Dec 16, 2013 11:37 AM, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
> wrote:
> >
> > On Mon, Dec 16, 2013 at 06:59:16PM +0100, Sander Eikelenboom wrote:
> > >
> > > Monday, December 16, 2013, 6:46:35 PM, you wrote:
> > >
> > > > On Mon, Dec 16, 2013 at 06:24:20PM +0100, Sander Eikelenboom wrote:
> > > >>
> > > >> Monday, December 16, 2013, 4:09:06 PM, you wrote:
> > > >>
> > > >> >> >> * qemu-upstream not freeing pirq
> > > >> >> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
> > > >> >> >>  > http://marc.info/?l=xen-devel&m=137265766424502
> > > >> >> >>  status: patches posted; latest patches need testing
> > > >> >> >>  Not a blocker.
> > > >> >>
> > > >> >> > I can't test it, b/c qemu-upstream does not work with PCI
> passthrough
> > > >> >> > anymore.
> > > >> >>
> > > >> >> Strange, pci passthrough in general works for me on "simple" pci
> devices (usb cards, videograbber) with latest qemu-upstream and seabios
> upstream.
> > > >> >> (if you have no need for pci rombars, no vga passthrough, no need
> for specifying funky things about which virtual functions to passthrough)
> > > >> >>
> > > >> >> What are you trying to passthrough ?
> > > >>
> > > >> > -bash-4.1# lspci -s 01:00.0 -v
> > > >> > 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit
> Network Connection (rev 01)
> > > >> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server
> Adapter
> > > >> >         Flags: fast devsel, IRQ 16
> > > >> >         Memory at fbc20000 (32-bit, non-prefetchable) [disabled]
> [size=128K]
> > > >> >         Memory at fb800000 (32-bit, non-prefetchable) [disabled]
> [size=4M]
> > > >> >         I/O ports at e020 [disabled] [size=32]
> > > >> >         Memory at fbc44000 (32-bit, non-prefetchable) [disabled]
> [size=16K]
> > > >> >         Expansion ROM at fb400000 [disabled] [size=4M]
> > > >> >         Capabilities: [40] Power Management version 3
> > > >> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> > > >> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
> > > >> >         Capabilities: [a0] Express Endpoint, MSI 00
> > > >> >         Capabilities: [100] Advanced Error Reporting
> > > >> >         Capabilities: [140] Device Serial Number
> 00-1b-21-ff-ff-45-d9-ac
> > > >> >         Capabilities: [150] Alternative Routing-ID Interpretation
> (ARI)
> > > >> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
> > > >> >         Kernel driver in use: pciback
> > > >> >         Kernel modules: igb
> > > >>
> > > >>
> > > >> >> What problems do you have / errors do you see ?
> > > >> >>
> > > >>
> > > >> >
> http://www.gossamer-threads.com/lists/xen/devel/309476?do=post_view_threaded
> > > >>
> > > >>
> > > >> Could it be related to the outstanding issue (since 4.3) of:
> > > >>
> > > >> * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream
> with PCI/GPU passthrough
> > > >>   >
> http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
> > > >>   > Where Stefano writes:
> > > >>   > 2) for Xen 4.4 rework the two patches above and improve
> > > >>   > i440fx_update_pci_mem_hole: resizing the pci_hole subregion is
> not
> > > >>   > enough, it also needs to be able to resize the system memory
> region
> > > >>   > (xen.ram) to make room for the bigger pci_hole
> > > >>
> > > >>   Your card seems to need quite some resources, and restricted to
> 32bit, even other SR-IOV cards seems to to with less and or support 32 bit.
> > > >>
> > >
> > > > I don't think that is the problem. The QEMU-xen did work at some
> point - until
> > > > I blew away 'qemu-xen-dir' to get the newest and best. So I think the
> problem
> > > > I am seeing is with QEMU 1.6 regression.
> > >
> > >
> > > Could try to get the latest upstream qemu and seabios patches then by
> changing your trees in Config.mk to the upstream qemu tree:
> > >
> > > git://git.qemu.org/qemu.git
> > >
> > > and
> > >
> > > git://git.qemu.org/seabios.git
> > > (at the extra config option questions, just except the defaults).
> > >
> > > It's what i run at the moment, could be there is a off-chance there is
> a patch upstream that fixes your problem that's not in the xen qemu
> upstream tree yet.
> >
> > I did this:
> > diff --git a/Config.mk b/Config.mk
> > index 2007b22..7702fff 100644
> > --- a/Config.mk
> > +++ b/Config.mk
> > @@ -235,7 +235,13 @@ SEABIOS_UPSTREAM_URL ?= git://
> xenbits.xen.org/seabios.git
> >  endif
> >  OVMF_UPSTREAM_REVISION ?= 447d264115c476142f884af0be287622cd244423
> >  QEMU_UPSTREAM_REVISION ?= master
> 
> QEMU is bisectable so if the old rev worked you can find the commit that
> stopped working for youe card.

That will have to wait (the bisection) after New Year. I got a whole
bunch of other things I need to get done.

I am hoping that Anthony (the other one) can reproduce this when he gets
hardware capable of doing PCI passthrough and find the bug immediately.

Re: Xen 4.4 development update: Is PVH a blocker?

[Konrad Rzeszutek Wilk]

On Tue, Dec 17, 2013 at 04:03:14PM +0100, Sander Eikelenboom wrote:
> 
> Tuesday, December 17, 2013, 3:35:45 PM, you wrote:
> 
> >> > rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
> >> > rebuilt and had the same issue :-(
> >> 
> >> Hmm so taking it forward doesn't cut it .. going back is probably going to be searching for a needle in a haystack.
> >> Finding many incomptabilities. I tried it with vga passthrough to find back a working config i knew i once had .. but to no avail :-(
> 
> > Yikes. Do you get a similar error message in QEMU?
> 
> Nope my problem (at present) is with the rom bar, that is special cased around everywhere (hvmloader, seabios, qemu, guest kernel).
> And it appears all fine, but in fact in the guest it isn't backed by the actual rom data (neither by a copy, neither by passing it through).
> 
> Already tried by letting hvmloader load it as a optionrom, but that didn't cut it.
> 
> When dumping the rom of the devices passed through in the guest,
> i was seeing al kinds of romdata from the emulated devices at the address given to the passed through devive,
> but not the romdata from the passedthrough device.
> 
> And that was when i stumbled upon the whole xen_platform_pci problem,
> as i was trying to switch of as many emulated devices as possible (in the hope it was some interference between those.
> 
> After that tried KVM .. to see if it is in any way feasible and possible with my system (iommu + gfx card),
> that resulted in almost instant succes .. so yes it should be possible ...

Yes. I would like it to work too. That is what I am trying to get working. And
neatly enough I can reproduce the problem you are seeing on my system.

I have some ideas of why it is happening but no concrete patches. Will
CC when I am ready.

Re: Xen 4.4 development update: Is PVH a blocker?

[Sander Eikelenboom]

Tuesday, December 17, 2013, 10:35:38 PM, you wrote:

> On Tue, Dec 17, 2013 at 04:03:14PM +0100, Sander Eikelenboom wrote:
>> 
>> Tuesday, December 17, 2013, 3:35:45 PM, you wrote:
>> 
>> >> > rm -Rf tools/firmware/seabios-dir* tools/qemu-xen*
>> >> > rebuilt and had the same issue :-(
>> >> 
>> >> Hmm so taking it forward doesn't cut it .. going back is probably going to be searching for a needle in a haystack.
>> >> Finding many incomptabilities. I tried it with vga passthrough to find back a working config i knew i once had .. but to no avail :-(
>> 
>> > Yikes. Do you get a similar error message in QEMU?
>> 
>> Nope my problem (at present) is with the rom bar, that is special cased around everywhere (hvmloader, seabios, qemu, guest kernel).
>> And it appears all fine, but in fact in the guest it isn't backed by the actual rom data (neither by a copy, neither by passing it through).
>> 
>> Already tried by letting hvmloader load it as a optionrom, but that didn't cut it.
>> 
>> When dumping the rom of the devices passed through in the guest,
>> i was seeing al kinds of romdata from the emulated devices at the address given to the passed through devive,
>> but not the romdata from the passedthrough device.
>> 
>> And that was when i stumbled upon the whole xen_platform_pci problem,
>> as i was trying to switch of as many emulated devices as possible (in the hope it was some interference between those.
>> 
>> After that tried KVM .. to see if it is in any way feasible and possible with my system (iommu + gfx card),
>> that resulted in almost instant succes .. so yes it should be possible ...

> Yes. I would like it to work too. That is what I am trying to get working. And
> neatly enough I can reproduce the problem you are seeing on my system.

> I have some ideas of why it is happening but no concrete patches. Will
> CC when I am ready.

Ok thx :-)

Re: Xen 4.4 development update: Is PVH a blocker?

[Ian Campbell]

create ^
title it xl: require explicit action to share a disk between vms
severity it wishlist
thanks

On Fri, 2013-12-13 at 14:43 -0500, Konrad Rzeszutek Wilk wrote:
> > == Open ==
> >
> Also, xl as opposed to xend, allows me to share a disk without
> any fanfare.
> 
> Meaning I can do this:
> 
> xl block-attach phy:/dev/sda latest1 xvda w
> xl block-attach phy:/dev/sda latest2 xvda w
> 
> while if I had used 'xend' I had to also append the '!' parameter
> to denote it as 'shared'.
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Processed: Re: Xen 4.4 development update: Is PVH a blocker?

[xen]

Processing commands for xen@bugs.xenproject.org:

> create ^
Created new bug #26 rooted at `<20131213194326.GA28712@phenom.dumpdata.com>'
Title: `Re: [Xen-devel] Xen 4.4 development update: Is PVH a blocker?'
> title it xl: require explicit action to share a disk between vms
Set title for #26 to `xl: require explicit action to share a disk between vms'
> severity it wishlist
Change severity for #26 to `wishlist'
> thanks
Finished processing.

Modified/created Bugs:
 - 26: http://bugs.xenproject.org/xen/bug/26 (new)

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues