Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[xen]

Processing commands for xen@bugs.xenproject.org:

> create ^
Created new bug #20 rooted at `<1571692646.20131009000945@eikelenboom.it>'
Title: `Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.'
> title it xen_platform_pci=0 doesn't work with qemu-xen
Set title for #20 to `xen_platform_pci=0 doesn't work with qemu-xen'
> owner it Anthony Perard <anthony.perard@citrix.com>
Change owner for #20 to `Anthony Perard <anthony.perard@citrix.com>'
> thanks
Finished processing.

Modified/created Bugs:
 - 20: http://bugs.xenproject.org/xen/bug/20 (new)

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues

Re: Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Ian Campbell]

graft 20 !
prune 20 <1043931136.20131009144924@eikelenboom.it>
thanks

On Wed, 2013-10-09 at 13:15 +0100, xen@bugs.xenproject.org wrote:
> Processing commands for xen@bugs.xenproject.org:
[...]
> Modified/created Bugs:
>  - 20: http://bugs.xenproject.org/xen/bug/20 (new)

Hrm, this bug is missing the useful content because Sander mailed
Stefano and I privately and I didn't notice.

The original message was simply:
> While trying to get to the bottom of my passthrough problem with the rom bar,
> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
> 
> Running latest xen-unstable, together with upstream qemu and upstream seabios.

Re: Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Sander Eikelenboom]

Friday, October 11, 2013, 5:04:04 PM, you wrote:

> graft 20 !
> prune 20 <1043931136.20131009144924@eikelenboom.it>
> thanks

> On Wed, 2013-10-09 at 13:15 +0100, xen@bugs.xenproject.org wrote:
>> Processing commands for xen@bugs.xenproject.org:
> [...]
>> Modified/created Bugs:
>>  - 20: http://bugs.xenproject.org/xen/bug/20 (new)

> Hrm, this bug is missing the useful content because Sander mailed
> Stefano and I privately and I didn't notice.

Sorry for that, it seemed xen-devel got dropped somewhere along that thread and i didn't notice.

Perhaps relevant if any one is going to take a look at this (since it will crash on boot for a newer kernel anyhow):

> Tried with xen_platform_pci=0 with qemu-xen-traditional and that makes a linux 3.12-rc3 kernel crash on boot
> unfortunatly the strack trace runs from the small vnc screen here :S
> It also crashes on kernel 3.9.2 and 3.9.0-rc3, and 3.8.13
> but not on the debian supplied 3.2.0 and 3.8.0-rc2.
> So the cause seems to be introduced or backported to the 3.8 series somewhere.



> The original message was simply:
>> While trying to get to the bottom of my passthrough problem with the rom bar,
>> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
>> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
>> 
>> Running latest xen-unstable, together with upstream qemu and upstream seabios.

Re: Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Anthony PERARD]

On Fri, Oct 11, 2013 at 05:22:47PM +0200, Sander Eikelenboom wrote:
> > The original message was simply:
> >> While trying to get to the bottom of my passthrough problem with the rom bar,
> >> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
> >> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
> >> 
> >> Running latest xen-unstable, together with upstream qemu and upstream seabios.

I've start looking at this, and the fix would be easy: removing the
creation of the xen-platform from QEMU, and let xl ask QEMU to create
it.

But if one uses an older version of one project (qemu/xl) and a newer
version of the other, then we can end-up with no xen-platform or two
xen-platform ...

So should we try to be clever about this, or just backport patchs for
both qemu and xl?

"Clever" option: There is a recent patch for xl that make use of
the -nodefault QEMU's command line option. So if it's present, we could
ask QEMU to not add xen-platform, and xl can decide.

Thought?

Regards,

-- 
Anthony PERARD

Re: Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[George Dunlap]

On Fri, Oct 11, 2013 at 5:53 PM, Anthony PERARD
<anthony.perard@citrix.com> wrote:
> On Fri, Oct 11, 2013 at 05:22:47PM +0200, Sander Eikelenboom wrote:
>> > The original message was simply:
>> >> While trying to get to the bottom of my passthrough problem with the rom bar,
>> >> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
>> >> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
>> >>
>> >> Running latest xen-unstable, together with upstream qemu and upstream seabios.
>
> I've start looking at this, and the fix would be easy: removing the
> creation of the xen-platform from QEMU, and let xl ask QEMU to create
> it.
>
> But if one uses an older version of one project (qemu/xl) and a newer
> version of the other, then we can end-up with no xen-platform or two
> xen-platform ...

So what's the plan for compatibility here -- full forward and backward
compatibily on both sides?  (i.e., newer qemu works with older Xen,
newer Xen works with older qemu)?

> So should we try to be clever about this, or just backport patchs for
> both qemu and xl?
>
> "Clever" option: There is a recent patch for xl that make use of
> the -nodefault QEMU's command line option. So if it's present, we could
> ask QEMU to not add xen-platform, and xl can decide.

I wouldn't mind doing something like this; but would it make more
sense to have some kind of a "capability" or "version" interface, like
we have between Xen and Linux, rather than relying on other quirks
like the presence of "-nodefault"?

 -George

Re: Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Anthony PERARD]

On Fri, Oct 11, 2013 at 06:10:32PM +0100, George Dunlap wrote:
> On Fri, Oct 11, 2013 at 5:53 PM, Anthony PERARD
> <anthony.perard@citrix.com> wrote:
> > On Fri, Oct 11, 2013 at 05:22:47PM +0200, Sander Eikelenboom wrote:
> >> > The original message was simply:
> >> >> While trying to get to the bottom of my passthrough problem with the rom bar,
> >> >> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
> >> >> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
> >> >>
> >> >> Running latest xen-unstable, together with upstream qemu and upstream seabios.
> >
> > I've start looking at this, and the fix would be easy: removing the
> > creation of the xen-platform from QEMU, and let xl ask QEMU to create
> > it.
> >
> > But if one uses an older version of one project (qemu/xl) and a newer
> > version of the other, then we can end-up with no xen-platform or two
> > xen-platform ...
> 
> So what's the plan for compatibility here -- full forward and backward
> compatibily on both sides?  (i.e., newer qemu works with older Xen,
> newer Xen works with older qemu)?
> 
> > So should we try to be clever about this, or just backport patchs for
> > both qemu and xl?
> >
> > "Clever" option: There is a recent patch for xl that make use of
> > the -nodefault QEMU's command line option. So if it's present, we could
> > ask QEMU to not add xen-platform, and xl can decide.
> 
> I wouldn't mind doing something like this; but would it make more
> sense to have some kind of a "capability" or "version" interface, like
> we have between Xen and Linux, rather than relying on other quirks
> like the presence of "-nodefault"?

Well, we can always check if the xen-platform-pci is already there after
having started QEMU (through QMP) and react by adding it, display a
warning or do nothing more, depending the result and the config of the
vm.

-- 
Anthony PERARD

Processed: Re: Processed: Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[xen]

Processing commands for xen@bugs.xenproject.org:

> graft 20 <1381503844.24708.46.camel@kazak.uk.xensource.com>
Graft `<1381503844.24708.46.camel@kazak.uk.xensource.com>' onto #20
> prune 20 <1043931136.20131009144924@eikelenboom.it>
Prune `<1043931136.20131009144924@eikelenboom.it>' from #20
> thanks
Finished processing.

Modified/created Bugs:
 - 20: http://bugs.xenproject.org/xen/bug/20

---
Xen Hypervisor Bug Tracker
See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs
Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Sander Eikelenboom]

Wednesday, October 9, 2013, 2:00:02 PM, you wrote:

> create ^
> title it xen_platform_pci=0 doesn't work with qemu-xen
> owner it Anthony Perard <anthony.perard@citrix.com>
> thanks

Perhaps in general looking at the libxl and xend .. and .. qemu-xen and qemu-xen-traditonal compatibility shoud be added too.

Perhaps i'm a bit blunt ..
but for users it's quite a mess and PITA now for quite some time, the transition of both is now smeared out over quite some major and minor releases.

With some features not working since .. some features are working again but have a gap of not working,
since old features becoming usable again are (understandably) not backported.

This seems to be at a level that it is even undocumentable ... you would need a spreadsheet
with every minor version to try to document that ... it seems to be leading to a lot of user questions as well.

I'm not blaming any one personally or anything like that .. just expressing my worries and the threat i see for Xen as a project in general.
Namely users sop

I understand that new features like the arm port are interesting .. but i think that has the cost of both
IanC and Stefano time to work on libxl and qemu-upstream seems to be quite limited, and perhaps developer time in general is.

But getting these major transitions smeared out over several major and minor releases doesn't seem to be very desirable.
This will be enlarged since distro's like debian are going to stop packaging the xen-forks. But since they will be using a "stable" release for these
projects like qemu, i will take some time for xen patches to for instance qemu to trickle down into the qemu-upstream stable.

(BTW debian unstable already seems to be not packaging qemu-traditional anymore but rely on the upstream qemu package entirely,
leading to questions on IRC already )

KVM seems to have it's act quite together with their virtio drivers at the moment and i must say i'm getting
attracted to try to switch (although i generally like Xen so i'm still in denial :-) ).

Just my few cents and a call out as a user ..
(and perhaps a topic for the developer conference to address .. also in the light of the short discussion about a long term stable release
which in my opinion can't be done before both are resolved)

Don't know how Pasi thinks about this and if he has any comments, since he seems to be quite involved in (helping) the user community ..

--
Sander



> On Wed, 2013-10-09 at 00:09 +0200, Sander Eikelenboom wrote:
>> Hi,
>> 
>> While trying to get to the bottom of my passthrough problem with the rom bar,
>> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
>> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
>> 
>> Running latest xen-unstable, together with upstream qemu and upstream seabios.
>> 
>> --
>> 
>> Sander
>> 

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Stefano Stabellini]

On Wed, 9 Oct 2013, Sander Eikelenboom wrote:
> Wednesday, October 9, 2013, 2:00:02 PM, you wrote:
> 
> > create ^
> > title it xen_platform_pci=0 doesn't work with qemu-xen
> > owner it Anthony Perard <anthony.perard@citrix.com>
> > thanks
> 
> Perhaps in general looking at the libxl and xend .. and .. qemu-xen and qemu-xen-traditonal compatibility shoud be added too.
> 
> Perhaps i'm a bit blunt ..
> but for users it's quite a mess and PITA now for quite some time, the transition of both is now smeared out over quite some major and minor releases.
> 
> With some features not working since .. some features are working again but have a gap of not working,
> since old features becoming usable again are (understandably) not backported.
> 
> This seems to be at a level that it is even undocumentable ... you would need a spreadsheet
> with every minor version to try to document that ... it seems to be leading to a lot of user questions as well.
> 
> I'm not blaming any one personally or anything like that .. just expressing my worries and the threat i see for Xen as a project in general.
> Namely users sop
> 
> I understand that new features like the arm port are interesting .. but i think that has the cost of both
> IanC and Stefano time to work on libxl and qemu-upstream seems to be quite limited, and perhaps developer time in general is.
> 
> But getting these major transitions smeared out over several major and minor releases doesn't seem to be very desirable.
> This will be enlarged since distro's like debian are going to stop packaging the xen-forks. But since they will be using a "stable" release for these
> projects like qemu, i will take some time for xen patches to for instance qemu to trickle down into the qemu-upstream stable.
> 
> (BTW debian unstable already seems to be not packaging qemu-traditional anymore but rely on the upstream qemu package entirely,
> leading to questions on IRC already )
> 
> KVM seems to have it's act quite together with their virtio drivers at the moment and i must say i'm getting
> attracted to try to switch (although i generally like Xen so i'm still in denial :-) ).
> 
> Just my few cents and a call out as a user ..
> (and perhaps a topic for the developer conference to address .. also in the light of the short discussion about a long term stable release
> which in my opinion can't be done before both are resolved)
> 
> Don't know how Pasi thinks about this and if he has any comments, since he seems to be quite involved in (helping) the user community ..

Thank you for the feedback. We need constructive criticism like this to
improve the project and the way we handle things. Let me tell you where
I stand.

In the case of xl, we really thought that we filled all the gaps. Just
when we were about to remove xend for good, people stepped up telling us
to wait because actually we missed something.
I don't think we could have done anything better, given the
circumstances. If we don't know that we have a bug or a missing feature,
we can't do anything about it.

In the case of QEMU, we could probably have done a better job at
tracking missing features. However in general once we know that we have
an issue, we usually resolve it in a timely manner.

Anthony wrote a blog post
(http://blog.xen.org/index.php/2013/09/20/qemu-vs-qemu-traditional-update/)
on this, but having a detailed well formatted wiki page listing
everything we know is missing in upstream QEMU would go a long way
toward avoiding situations like this in the future.

The other thing that could help is better testing. OSSTests doesn't
cover the xen_platform_pci option, otherwise we would have found and
fixed the issues months ago.

Anybody can write a new OSSTest, not just developers. Give a look at
Wei's recent blog post:

http://blog.xen.org/index.php/2013/09/30/osstest-standalone-mode-step-by-step/

Do you care about a particular feature? Do you want to make sure it
doesn't break in xen-unstable? Write an OSSTest for it!

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Sander Eikelenboom]

Wednesday, October 9, 2013, 5:03:56 PM, you wrote:

> On Wed, 9 Oct 2013, Sander Eikelenboom wrote:
>> Wednesday, October 9, 2013, 2:00:02 PM, you wrote:
>> 
>> > create ^
>> > title it xen_platform_pci=0 doesn't work with qemu-xen
>> > owner it Anthony Perard <anthony.perard@citrix.com>
>> > thanks
>> 
>> Perhaps in general looking at the libxl and xend .. and .. qemu-xen and qemu-xen-traditonal compatibility shoud be added too.
>> 
>> Perhaps i'm a bit blunt ..
>> but for users it's quite a mess and PITA now for quite some time, the transition of both is now smeared out over quite some major and minor releases.
>> 
>> With some features not working since .. some features are working again but have a gap of not working,
>> since old features becoming usable again are (understandably) not backported.
>> 
>> This seems to be at a level that it is even undocumentable ... you would need a spreadsheet
>> with every minor version to try to document that ... it seems to be leading to a lot of user questions as well.
>> 
>> I'm not blaming any one personally or anything like that .. just expressing my worries and the threat i see for Xen as a project in general.
>> Namely users sop
>> 
>> I understand that new features like the arm port are interesting .. but i think that has the cost of both
>> IanC and Stefano time to work on libxl and qemu-upstream seems to be quite limited, and perhaps developer time in general is.
>> 
>> But getting these major transitions smeared out over several major and minor releases doesn't seem to be very desirable.
>> This will be enlarged since distro's like debian are going to stop packaging the xen-forks. But since they will be using a "stable" release for these
>> projects like qemu, i will take some time for xen patches to for instance qemu to trickle down into the qemu-upstream stable.
>> 
>> (BTW debian unstable already seems to be not packaging qemu-traditional anymore but rely on the upstream qemu package entirely,
>> leading to questions on IRC already )
>> 
>> KVM seems to have it's act quite together with their virtio drivers at the moment and i must say i'm getting
>> attracted to try to switch (although i generally like Xen so i'm still in denial :-) ).
>> 
>> Just my few cents and a call out as a user ..
>> (and perhaps a topic for the developer conference to address .. also in the light of the short discussion about a long term stable release
>> which in my opinion can't be done before both are resolved)
>> 
>> Don't know how Pasi thinks about this and if he has any comments, since he seems to be quite involved in (helping) the user community ..

> Thank you for the feedback. We need constructive criticism like this to
> improve the project and the way we handle things. Let me tell you where
> I stand.

> In the case of xl, we really thought that we filled all the gaps. Just
> when we were about to remove xend for good, people stepped up telling us
> to wait because actually we missed something.
> I don't think we could have done anything better, given the
> circumstances. If we don't know that we have a bug or a missing feature,
> we can't do anything about it.

I know it's hard to test all the (possible combination) of options.
However a systematic screening of the documented options allowed in config files would have revealed
more before missing features in libxl even before user testing.

But i agree it has some chicken and egg problem to it, since early adopter user testing usually comes after the .0 release
and the rest after the .1 or .2.

However (with hindsight) when refactoring such major components it's perhaps better to adjust the release schedule and
focus developer attention to that refactoring for that release and make the refactoring job set the pace and be the one and only true blocker
and for the .1 at least be liberal with backporting missing features as well (to prevent a feature gap as much as possible).

That would (hopefully) prevent such a refactoring from dragging on, which is also a developer burden.
The earlier it is done and over with, the more time to work on the exciting new features and less nagging users :-p

I know that for the external projects you are not in control for both the patch acceptance and release pace,
so f.e. for the linux kernel there wasn't a way to prevent it being smeared over multiple releases.

However i think qemu is perhaps different in the sense that on distro's users will be more inclined to compile a newer kernel
than to compile a new qemu package from upstream sources. Also (PV)HVM seems to only gain importance ...

So i think it is even more important to keep xen in qemu-upstream in good shape
and to direct some test effort into it, especially prior to an upcoming qemu release (rc's).

> In the case of QEMU, we could probably have done a better job at
> tracking missing features. However in general once we know that we have
> an issue, we usually resolve it in a timely manner.

I must say in general the responsiveness is quite good, although as i mentioned earlier,
i think it is somewhat reduced for the qemu and libxl parts since the arm port is on it's way.

For instance there wasn't much response to the thread about lacking pci multifunction passthrough
options. (Neither to my personal problem about the rom bar of passthroughed pci devices.)

I don't say they have to be solved right away, the more since they are in the interaction between 2 or more components
(libxl, qemu, hvmloader, seabios), but at least *some* response on how to proceed would be nice.

Another thing i stumbled across in the qemu source in include/exec/memory.h:

From:   Avi Kivity
Subject:        [Qemu-devel] [PATCH 16/23] memory: temporarily add memory_region_get_ram_addr()
Date:   Mon, 19 Dec 2011 16:13:37 +0200
/**
 * memory_region_get_ram_addr: Get the ram address associated with a memory
 *                             region
 *
 * DO NOT USE THIS FUNCTION.  This is a temporary workaround while the Xen
 * code is being reworked.
 */
ram_addr_t memory_region_get_ram_addr(MemoryRegion *mr);


It seems it is temporary for quite some time ;-)



> Anthony wrote a blog post
> (http://blog.xen.org/index.php/2013/09/20/qemu-vs-qemu-traditional-update/)
> on this, but having a detailed well formatted wiki page listing
> everything we know is missing in upstream QEMU would go a long way
> toward avoiding situations like this in the future.

> The other thing that could help is better testing. OSSTests doesn't
> cover the xen_platform_pci option, otherwise we would have found and
> fixed the issues months ago.

Yes .. although it seemed to be in the bugzilla for quite some time ..
http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=1839

I know there was a discussion some time ago about bugzilla's not being used by developers,
but if that is the case it better can be closed with a message to just send it to the mailing
list instead. Now a user has the expectation he has reported it and the chance he won't post it again to
xen-devel and thus the project is losing a perfectly sound bug report
(and perhaps a user since it's not addressed nor replied to).

I know that most bugzilla's are actually distro bugzilla's, and are out of reach, however this one
is on xensource. I don't know how much contact there is with the various distro package maintainers to forward
the bugzilla reports that should be fixed upstream ?


> Anybody can write a new OSSTest, not just developers. Give a look at
> Wei's recent blog post:

> http://blog.xen.org/index.php/2013/09/30/osstest-standalone-mode-step-by-step/

> Do you care about a particular feature? Do you want to make sure it
> doesn't break in xen-unstable? Write an OSSTest for it!

Well one of the features happens to be the pci passthrough.

Since i use it, i can say it works in general for single devices and when the device is not requiring it's expansion rom to function.
But that last issue i just stumbled upon.

But for such an OSSTEST the test machine should have certain hardware to passthrough,
or is there a possibility to create a fake/dummy test pci device on the host and pass
that through and test its behavior (if all mem ranges, ioports and rom are read/writable as expected)
(did a fast google but didn't found something immediately useful.


--
Sander

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[George Dunlap]

On 09/10/13 13:49, Sander Eikelenboom wrote:
> Wednesday, October 9, 2013, 2:00:02 PM, you wrote:
>
>> create ^
>> title it xen_platform_pci=0 doesn't work with qemu-xen
>> owner it Anthony Perard <anthony.perard@citrix.com>
>> thanks
> Perhaps in general looking at the libxl and xend .. and .. qemu-xen and qemu-xen-traditonal compatibility shoud be added too.
>
> Perhaps i'm a bit blunt ..
> but for users it's quite a mess and PITA now for quite some time, the transition of both is now smeared out over quite some major and minor releases.
>
> With some features not working since .. some features are working again but have a gap of not working,
> since old features becoming usable again are (understandably) not backported.

Well thank you for being blunt -- we'd much rather you complain than 
just silently disappear. :-)  I can certainly see where you're coming 
from.  But on the flip side -- how are developers supposed to know what 
is broken if nobody tests it, or reports it?  We had several test days 
before the 4.3 release; if anyone had reported this feature broken, it 
would have been fixed immediately.

Something similar with xend -- many of the Xen developers were 
considering removing xend for 4.4, because xl seemed to be mature enough 
for everyone.  When we discussed it on the list, immediately a number of 
people stood up and listed features they wanted that were missing from 
xl, or ways in which xl didn't meet their needs.

We're not mind-readers, and our software doesn't have a call-home 
feature to let us know who is using what feature.  The core developers 
have already been looking at libxl, xend, and qemu-xen for some time; we 
made the switch because it looked like we had feature parity for the 
important features.  Us taking a longer look at this point isn't going 
to help things: we've seen all we're going to see.

If you have features that are missing from xl/qemu-xen that are not on 
my 4.4 development list, and not in xenbugs, the best thing you can do 
is report them, so we can put them on our list.  Better yet, as Stefano 
said, write a test case for them, to make sure that they are never 
broken in any release ever again. :-)

As for the future -- it is unfortunate that with a major transition, 
like from xend->libxl and qemu-traditional->qemu-xen, there is going to 
be some "catch-up" as bugs are ironed out and important functionality 
"faulted in".  This same thing happens in other open source projects; 
KDE, Gnome, Ubuntu's move to Unity, all come to mind as things that have 
had this happen. Even Linux: the Linux driver for the SD card reader 
broke on my 8-year-old laptop 4 years ago, and still hasn't been fixed.  
KVM isn't an exception: they just haven't had to do any major 
refactorings yet (as far as I know). Maybe that's because they have 
superior process, developers, or design; or maybe it's just because it's 
a younger project by about 4 years, and there's a major refactoring 
coming up right around the corner. :-)

I don't forsee any more big transitions like that for Xen any time on 
the horizon: once libxl and qemu-xen settle down, things should be 
stable for quite a while.  We're putting a lot more effort into writing 
test cases, and making it easy for others to do so; as our test suite 
grows and becomes more comprehensive, the number of features that can 
get broken without us noticing will get smaller and smaller.

  -George

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Sander Eikelenboom]

Wednesday, October 9, 2013, 5:32:04 PM, you wrote:

> On 09/10/13 13:49, Sander Eikelenboom wrote:
>> Wednesday, October 9, 2013, 2:00:02 PM, you wrote:
>>
>>> create ^
>>> title it xen_platform_pci=0 doesn't work with qemu-xen
>>> owner it Anthony Perard <anthony.perard@citrix.com>
>>> thanks
>> Perhaps in general looking at the libxl and xend .. and .. qemu-xen and qemu-xen-traditonal compatibility shoud be added too.
>>
>> Perhaps i'm a bit blunt ..
>> but for users it's quite a mess and PITA now for quite some time, the transition of both is now smeared out over quite some major and minor releases.
>>
>> With some features not working since .. some features are working again but have a gap of not working,
>> since old features becoming usable again are (understandably) not backported.

> Well thank you for being blunt -- we'd much rather you complain than 
> just silently disappear. :-)  I can certainly see where you're coming 
> from.  But on the flip side -- how are developers supposed to know what 
> is broken if nobody tests it, or reports it?  We had several test days 
> before the 4.3 release; if anyone had reported this feature broken, it 
> would have been fixed immediately.

Well if you do care, you can't silently disappear :-)

I know there is a almost unlimited combination of config options, so not everything can be tested in every combination (not by hand, and probably not by machine either).
However during such a refactoring one would expect that at least every option that can be disabled and enabled would be tested on it's own.
This one clearly wasn't nor do the docs say anything about it missing ...

I have no idea how much test resources there are available to the project, since there are already quite a lot of
dependencies to test.

However this makes me wonder, does the current test system have something like the randconfig on linux ?
Or is it always running tests on the same config ?

Hmm have to dive into those docs about OSStest then ...

> Something similar with xend -- many of the Xen developers were 
> considering removing xend for 4.4, because xl seemed to be mature enough 
> for everyone.  When we discussed it on the list, immediately a number of 
> people stood up and listed features they wanted that were missing from 
> xl, or ways in which xl didn't meet their needs.

> We're not mind-readers, and our software doesn't have a call-home 
> feature to let us know who is using what feature.  The core developers 
> have already been looking at libxl, xend, and qemu-xen for some time; we 
> made the switch because it looked like we had feature parity for the 
> important features.  Us taking a longer look at this point isn't going 
> to help things: we've seen all we're going to see.

The other part of it is that it's not documented that something is missing either.
So (most) users aren't mind-readers as well as code readers ;-)

The docs for example don't say all the nifty multifunction and vslot functionality of
pci passthrough isn't working.

> If you have features that are missing from xl/qemu-xen that are not on 
> my 4.4 development list, and not in xenbugs, the best thing you can do 
> is report them, so we can put them on our list.  Better yet, as Stefano 
> said, write a test case for them, to make sure that they are never 
> broken in any release ever again. :-)

> As for the future -- it is unfortunate that with a major transition, 
like from xend->>libxl and qemu-traditional->qemu-xen, there is going to 
> be some "catch-up" as bugs are ironed out and important functionality 
> "faulted in".  This same thing happens in other open source projects; 
> KDE, Gnome, Ubuntu's move to Unity, all come to mind as things that have 
> had this happen. Even Linux: the Linux driver for the SD card reader 
> broke on my 8-year-old laptop 4 years ago, and still hasn't been fixed.  
> KVM isn't an exception: they just haven't had to do any major 
> refactorings yet (as far as I know). Maybe that's because they have 
> superior process, developers, or design; or maybe it's just because it's 
> a younger project by about 4 years, and there's a major refactoring 
> coming up right around the corner. :-)

Yes i know Xen is running into the "Law of the handicap of a head start" now,
and KVM has had the opportunity to learn from that and eventually they will run
into that as well. How ever i think there could perhaps be some lessons in here
regarding to the release process when doing major code refactoring projects.

> I don't forsee any more big transitions like that for Xen any time on 
> the horizon: once libxl and qemu-xen settle down, things should be 
> stable for quite a while.  We're putting a lot more effort into writing 
> test cases, and making it easy for others to do so; as our test suite 
> grows and becomes more comprehensive, the number of features that can 
> get broken without us noticing will get smaller and smaller.

I could see at least one such transition lurking on the qemu-side of things.
The machine model transition from I440FX/PIIX4 to Q35 (or making the xen code agnostic
as seems to be done with KVM)

And the dicussion on how to proceed with PVH could lead to another one ...

>   -George
--
Sander

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[George Dunlap]

On Wed, Oct 9, 2013 at 1:49 PM, Sander Eikelenboom <linux@eikelenboom.it> wrote:
>
> Wednesday, October 9, 2013, 2:00:02 PM, you wrote:
>
>> create ^
>> title it xen_platform_pci=0 doesn't work with qemu-xen
>> owner it Anthony Perard <anthony.perard@citrix.com>
>> thanks
>
> Perhaps in general looking at the libxl and xend .. and .. qemu-xen and qemu-xen-traditonal compatibility shoud be added too.
>
> Perhaps i'm a bit blunt ..
> but for users it's quite a mess and PITA now for quite some time, the transition of both is now smeared out over quite some major and minor releases.
>
> With some features not working since .. some features are working again but have a gap of not working,
> since old features becoming usable again are (understandably) not backported.
>
> This seems to be at a level that it is even undocumentable ... you would need a spreadsheet
> with every minor version to try to document that ... it seems to be leading to a lot of user questions as well.
>
> I'm not blaming any one personally or anything like that .. just expressing my worries and the threat i see for Xen as a project in general.
> Namely users sop
>
> I understand that new features like the arm port are interesting .. but i think that has the cost of both
> IanC and Stefano time to work on libxl and qemu-upstream seems to be quite limited, and perhaps developer time in general is.

FWIW, the ARM stuff isn't just "interesting" -- it was a critical
strategic opportunity for the future of the Xen project.  The work
that was done, particularly the political stuff done by Stefano to get
Xen into Linaro, was a major achievement at just the right time.  A
delay of even a few months may have meant that Xen had basically no
presence in ARM server space, which would in turn have had an impact
on the commercial viability of the Xen project as a whole.  The vast
majority of the money used to support Xen comes from the server
market.  Growing this market means more commercial interest in Xen,
which means more developers; missing this market could have knock-on
effects in the x86 space, which would have had a much bigger impact on
PCI pass-through in the long term.

 -George

Re: [HVM} xen_platform_pci=0 doesn't prevent platform device creation and disk and nic take over by PV drivers.

[Sander Eikelenboom]

Hello Sander,

Wednesday, October 9, 2013, 1:54:52 PM, you wrote:


> Wednesday, October 9, 2013, 1:40:00 PM, you wrote:

>> On Wed, 9 Oct 2013, Sander Eikelenboom wrote:
>>> Hi,
>>> 
>>> While trying to get to the bottom of my passthrough problem with the rom bar,
>>> i noticed that specifiying "xen_platform_pci=0" in the config file does not prevent
>>> the platform device to appear in my HVM guest and consequently the disk and nic are taken over.
>>> 
>>> Running latest xen-unstable, together with upstream qemu and upstream seabios.

>> I think that xen_platform_pci has never been implemented with upstream
>> QEMU.
>> However you can pass xen_emul_unplug=never to Linux to avoid
>> initializing netfront and blkfront.


> Tried with xen_platform_pci=0 with qemu-xen-traditional and that makes a linux 3.12-rc3 kernel crash on boot
> unfortunatly the strack trace runs from the small vnc screen here :S

> screenshot of what is left attached

It also crashes on kernel 3.9.2 and 3.9.0-rc3, and 3.8.13
but not on the debian supplied 3.2.0 and 3.8.0-rc2.

So the cause seems to be introduced or backported to the 3.8 series somewhere.


> Pfrrt wat a mess


-- 
Best regards,
 Sander                            mailto:linux@eikelenboom.it