Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Jim Fehlig]

Wei Liu wrote:
> Hi libvirt maintainers,

Sorry for the delay. Slowly catching up on mail after vacation...

> 
> Xen's control library libxenlight (libxl) requires application
> (libvirt in this case) to explictily define LIBXL_API_VERSION.

Where is this requirement written? :-)

> This is
> lacking at the moment so libvirt's libxl driver always gets the latest
> APIs.

IMO, that is what we want for upstream libvirt. Downstreams can choose a
specific version if they want.

> We changed one of the APIs in libxl so libvirt's libxl driver
> can't build at the moment.

A quick glance ahead in my libvirt mail tells me you fixed this with the
preferred LIBXL_HAVE_* pattern.

Regards,
Jim

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Wei Liu]

On Tue, Apr 12, 2016 at 03:31:46PM -0600, Jim Fehlig wrote:
> Wei Liu wrote:
> > Hi libvirt maintainers,
> 
> Sorry for the delay. Slowly catching up on mail after vacation...
> 
> > 
> > Xen's control library libxenlight (libxl) requires application
> > (libvirt in this case) to explictily define LIBXL_API_VERSION.
> 
> Where is this requirement written? :-)
> 
> > This is
> > lacking at the moment so libvirt's libxl driver always gets the latest
> > APIs.
> 
> IMO, that is what we want for upstream libvirt. Downstreams can choose a
> specific version if they want.
> 
> > We changed one of the APIs in libxl so libvirt's libxl driver
> > can't build at the moment.
> 
> A quick glance ahead in my libvirt mail tells me you fixed this with the
> preferred LIBXL_HAVE_* pattern.
> 

It's already done that way. :-)

Wei.

> Regards,
> Jim

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[George Dunlap]

On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com> wrote:
> Wei Liu wrote:
>> Hi libvirt maintainers,
>
> Sorry for the delay. Slowly catching up on mail after vacation...
>
>>
>> Xen's control library libxenlight (libxl) requires application
>> (libvirt in this case) to explictily define LIBXL_API_VERSION.
>
> Where is this requirement written? :-)
>
>> This is
>> lacking at the moment so libvirt's libxl driver always gets the latest
>> APIs.
>
> IMO, that is what we want for upstream libvirt. Downstreams can choose a
> specific version if they want.

I think one of us isn't understanding the situation properly. Is it
not the case that currently, all releases of libvirt *will not
compile* against Xen 4.7 once it's released?  So people downloading
and building libvirt will have to either 1) root around and try to
figure out what version of Xen it will build against, 2) manually add
in a #define *with the correct API version* to a header somewhere to
make it build properly, or 3) update to a version of libvirt that
supports the new api (which at the moment hasn't even been released
yet)?

All of those options are completely unacceptable.  Older versions of
libvirt should Just Work when compiled against newer versions of Xen.

I think it does make sense to have the libvirt development branch not
specify an API version; but when it branches for release, it should
set LIBXL_API_VERSION to whatever the current version is at the time
of the branch.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Daniel P. Berrange]

On Wed, Apr 13, 2016 at 10:09:16AM +0100, George Dunlap wrote:
> On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com> wrote:
> > Wei Liu wrote:
> >> Hi libvirt maintainers,
> >
> > Sorry for the delay. Slowly catching up on mail after vacation...
> >
> >>
> >> Xen's control library libxenlight (libxl) requires application
> >> (libvirt in this case) to explictily define LIBXL_API_VERSION.
> >
> > Where is this requirement written? :-)
> >
> >> This is
> >> lacking at the moment so libvirt's libxl driver always gets the latest
> >> APIs.
> >
> > IMO, that is what we want for upstream libvirt. Downstreams can choose a
> > specific version if they want.
> 
> I think one of us isn't understanding the situation properly. Is it
> not the case that currently, all releases of libvirt *will not
> compile* against Xen 4.7 once it's released?  So people downloading
> and building libvirt will have to either 1) root around and try to
> figure out what version of Xen it will build against, 2) manually add
> in a #define *with the correct API version* to a header somewhere to
> make it build properly, or 3) update to a version of libvirt that
> supports the new api (which at the moment hasn't even been released
> yet)?
> 
> All of those options are completely unacceptable.  Older versions of
> libvirt should Just Work when compiled against newer versions of Xen.
> 
> I think it does make sense to have the libvirt development branch not
> specify an API version; but when it branches for release, it should
> set LIBXL_API_VERSION to whatever the current version is at the time
> of the branch.

FYI, libvirt doesn't do branching for releases - we always just cut the
release straight from the master branch. We only actually create branches
on demand, when we find we want to backport fixes to a previous release.

Does libvirt master really need to always use the latest API version ?

It feels like libvirt could just set LIBXL_API_VERSION to the lowest
version it requires in order to get the functionality we know we are
able to currently build against. IOW, we'd only need to update the
define for LIBXL_API_VERSION when we merge patches that actually need
the newer functionality.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[George Dunlap]

On 13/04/16 10:26, Daniel P. Berrange wrote:
> On Wed, Apr 13, 2016 at 10:09:16AM +0100, George Dunlap wrote:
>> On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com> wrote:
>>> Wei Liu wrote:
>>>> Hi libvirt maintainers,
>>>
>>> Sorry for the delay. Slowly catching up on mail after vacation...
>>>
>>>>
>>>> Xen's control library libxenlight (libxl) requires application
>>>> (libvirt in this case) to explictily define LIBXL_API_VERSION.
>>>
>>> Where is this requirement written? :-)
>>>
>>>> This is
>>>> lacking at the moment so libvirt's libxl driver always gets the latest
>>>> APIs.
>>>
>>> IMO, that is what we want for upstream libvirt. Downstreams can choose a
>>> specific version if they want.
>>
>> I think one of us isn't understanding the situation properly. Is it
>> not the case that currently, all releases of libvirt *will not
>> compile* against Xen 4.7 once it's released?  So people downloading
>> and building libvirt will have to either 1) root around and try to
>> figure out what version of Xen it will build against, 2) manually add
>> in a #define *with the correct API version* to a header somewhere to
>> make it build properly, or 3) update to a version of libvirt that
>> supports the new api (which at the moment hasn't even been released
>> yet)?
>>
>> All of those options are completely unacceptable.  Older versions of
>> libvirt should Just Work when compiled against newer versions of Xen.
>>
>> I think it does make sense to have the libvirt development branch not
>> specify an API version; but when it branches for release, it should
>> set LIBXL_API_VERSION to whatever the current version is at the time
>> of the branch.
> 
> FYI, libvirt doesn't do branching for releases - we always just cut the
> release straight from the master branch. We only actually create branches
> on demand, when we find we want to backport fixes to a previous release.
> 
> Does libvirt master really need to always use the latest API version ?
> 
> It feels like libvirt could just set LIBXL_API_VERSION to the lowest
> version it requires in order to get the functionality we know we are
> able to currently build against. IOW, we'd only need to update the
> define for LIBXL_API_VERSION when we merge patches that actually need
> the newer functionality.

Oh, right -- yes, if that's the libvirt development model then it makes
more sense to do what works best with that model to make sure each
release has an appropriate LIBXL_API_VERSION.

On reflection, it's probably a better idea even from a Xen development
perspective.  I was originally thinking that it would be nice to have
the testing automatically flag up an update in libxl that could use a
corresponding update in libvirt.  But in practice, since we use these
tests as a push-gate, having changesets in the xen development branch
which break against libvirt master but require changes in libvirt master
to fix is actually causes a fair amount of hassle.

It might be useful for the XenProject to have a non-pushgate test which
tests libvirt without a LIBXL_API_VERSION, just to flag things up, but
that's something we can sort out on our side with a sed script.

Thanks,
 -George




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Wei Liu]

On Wed, Apr 13, 2016 at 10:26:54AM +0100, Daniel P. Berrange wrote:
> On Wed, Apr 13, 2016 at 10:09:16AM +0100, George Dunlap wrote:
> > On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com> wrote:
> > > Wei Liu wrote:
> > >> Hi libvirt maintainers,
> > >
> > > Sorry for the delay. Slowly catching up on mail after vacation...
> > >
> > >>
> > >> Xen's control library libxenlight (libxl) requires application
> > >> (libvirt in this case) to explictily define LIBXL_API_VERSION.
> > >
> > > Where is this requirement written? :-)
> > >
> > >> This is
> > >> lacking at the moment so libvirt's libxl driver always gets the latest
> > >> APIs.
> > >
> > > IMO, that is what we want for upstream libvirt. Downstreams can choose a
> > > specific version if they want.
> > 
> > I think one of us isn't understanding the situation properly. Is it
> > not the case that currently, all releases of libvirt *will not
> > compile* against Xen 4.7 once it's released?  So people downloading
> > and building libvirt will have to either 1) root around and try to
> > figure out what version of Xen it will build against, 2) manually add
> > in a #define *with the correct API version* to a header somewhere to
> > make it build properly, or 3) update to a version of libvirt that
> > supports the new api (which at the moment hasn't even been released
> > yet)?
> > 
> > All of those options are completely unacceptable.  Older versions of
> > libvirt should Just Work when compiled against newer versions of Xen.
> > 
> > I think it does make sense to have the libvirt development branch not
> > specify an API version; but when it branches for release, it should
> > set LIBXL_API_VERSION to whatever the current version is at the time
> > of the branch.
> 
> FYI, libvirt doesn't do branching for releases - we always just cut the
> release straight from the master branch. We only actually create branches
> on demand, when we find we want to backport fixes to a previous release.
> 
> Does libvirt master really need to always use the latest API version ?
> 

No, it doesn't.

> It feels like libvirt could just set LIBXL_API_VERSION to the lowest
> version it requires in order to get the functionality we know we are
> able to currently build against. IOW, we'd only need to update the
> define for LIBXL_API_VERSION when we merge patches that actually need
> the newer functionality.
> 

That's sensible.

Wei.

> Regards,
> Daniel
> -- 
> |: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
> |: http://libvirt.org              -o-             http://virt-manager.org :|
> |: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
> |: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Jim Fehlig]

On 04/13/2016 03:09 AM, George Dunlap wrote:
> On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com> wrote:
>> Wei Liu wrote:
>>> Hi libvirt maintainers,
>> Sorry for the delay. Slowly catching up on mail after vacation...
>>
>>> Xen's control library libxenlight (libxl) requires application
>>> (libvirt in this case) to explictily define LIBXL_API_VERSION.
>> Where is this requirement written? :-)
>>
>>> This is
>>> lacking at the moment so libvirt's libxl driver always gets the latest
>>> APIs.
>> IMO, that is what we want for upstream libvirt. Downstreams can choose a
>> specific version if they want.
> I think one of us isn't understanding the situation properly. Is it
> not the case that currently, all releases of libvirt *will not
> compile* against Xen 4.7 once it's released?  So people downloading
> and building libvirt will have to either 1) root around and try to
> figure out what version of Xen it will build against, 2) manually add
> in a #define *with the correct API version* to a header somewhere to
> make it build properly, or 3) update to a version of libvirt that
> supports the new api (which at the moment hasn't even been released
> yet)?
>
> All of those options are completely unacceptable.  Older versions of
> libvirt should Just Work when compiled against newer versions of Xen.

Yes, agreed. In practice though folks want a new libvirt with the new Xen, which
is probably why no one has complained thus far.

I'll knock up a patch to set the LIBXL_API_VERSION to 0x040200. The only APIs
that have changed since 0x040200 are libxl_set_vcpuaffinity and
libxl_domain_create_restore, but libvirt does not use the changes (added
params). libvirt does use new libxl APIs added since Xen 4.2, but those aren't
tied to a specific LIBXL_API_VERSION.

Regards,
Jim


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Jim Fehlig]

On 04/13/2016 03:26 AM, Daniel P. Berrange wrote:
> On Wed, Apr 13, 2016 at 10:09:16AM +0100, George Dunlap wrote:
>> On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com> wrote:
>>> Wei Liu wrote:
>>>> Hi libvirt maintainers,
>>> Sorry for the delay. Slowly catching up on mail after vacation...
>>>
>>>> Xen's control library libxenlight (libxl) requires application
>>>> (libvirt in this case) to explictily define LIBXL_API_VERSION.
>>> Where is this requirement written? :-)
>>>
>>>> This is
>>>> lacking at the moment so libvirt's libxl driver always gets the latest
>>>> APIs.
>>> IMO, that is what we want for upstream libvirt. Downstreams can choose a
>>> specific version if they want.
>> I think one of us isn't understanding the situation properly. Is it
>> not the case that currently, all releases of libvirt *will not
>> compile* against Xen 4.7 once it's released?  So people downloading
>> and building libvirt will have to either 1) root around and try to
>> figure out what version of Xen it will build against, 2) manually add
>> in a #define *with the correct API version* to a header somewhere to
>> make it build properly, or 3) update to a version of libvirt that
>> supports the new api (which at the moment hasn't even been released
>> yet)?
>>
>> All of those options are completely unacceptable.  Older versions of
>> libvirt should Just Work when compiled against newer versions of Xen.
>>
>> I think it does make sense to have the libvirt development branch not
>> specify an API version; but when it branches for release, it should
>> set LIBXL_API_VERSION to whatever the current version is at the time
>> of the branch.
> FYI, libvirt doesn't do branching for releases - we always just cut the
> release straight from the master branch. We only actually create branches
> on demand, when we find we want to backport fixes to a previous release.
>
> Does libvirt master really need to always use the latest API version ?
>
> It feels like libvirt could just set LIBXL_API_VERSION to the lowest
> version it requires in order to get the functionality we know we are
> able to currently build against.

I think that is a good idea too. I've sent a patch setting LIBXL_API_VERSION to
0x040200

https://www.redhat.com/archives/libvir-list/2016-April/msg00771.html

Changes made to the APIs since 0x040200 (params added to
libxl_domain_create_restore and libxl_set_vcpuaffinity) are currently not used
by libvirt.

Regards,
Jim


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1054 bytes --]

On Wed, 2016-04-13 at 17:31 -0600, Jim Fehlig wrote:
> > On Tue, Apr 12, 2016 at 10:31 PM, Jim Fehlig <jfehlig@suse.com>
> > wrote:
> > > 
> I'll knock up a patch to set the LIBXL_API_VERSION to 0x040200. The
> only APIs
> that have changed since 0x040200 are libxl_set_vcpuaffinity and
> libxl_domain_create_restore, but libvirt does not use the changes
> (added
> params). 
>
I think this is a good choice.

> libvirt does use new libxl APIs added since Xen 4.2, but those aren't
> tied to a specific LIBXL_API_VERSION.
> 
And, in those cases, usage should be gated by the appropriate
LIBXL_HAVE_FOOBAR symbol, which I see in the sources (e.g.,
for LIBXL_HAVE_NO_SUSPEND_RESUME or LIBXL_HAVE_DOMAIN_NODEAFFINITY) to
be the case, so good again. :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Ian Jackson]

Dario Faggioli writes ("Re: [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> And, in those cases, usage should be gated by the appropriate
> LIBXL_HAVE_FOOBAR symbol, which I see in the sources (e.g.,
> for LIBXL_HAVE_NO_SUSPEND_RESUME or LIBXL_HAVE_DOMAIN_NODEAFFINITY) to
> be the case, so good again. :-)

If libvirt specifies LIBXL_API_VERSION then it does not need to check
any LIBXL_HAVE_*, since it will actually get an API corresponding to
the specified API_VERSION.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1041 bytes --]

On Thu, 2016-04-14 at 18:59 +0100, Ian Jackson wrote:
> Dario Faggioli writes ("Re: [Xen-devel] Fixing libvirt's libxl driver
> breakage -- where to define LIBXL_API_VERSION?"):
> > 
> > And, in those cases, usage should be gated by the appropriate
> > LIBXL_HAVE_FOOBAR symbol, which I see in the sources (e.g.,
> > for LIBXL_HAVE_NO_SUSPEND_RESUME or LIBXL_HAVE_DOMAIN_NODEAFFINITY)
> > to
> > be the case, so good again. :-)
> If libvirt specifies LIBXL_API_VERSION then it does not need to check
> any LIBXL_HAVE_*, since it will actually get an API corresponding to
> the specified API_VERSION.
> 
I concur. And in fact, this patch introduces an LIBXL_API_VERSION and
drops the use of LIBXL_HAVE_*.

And I think the patch is right. :-)

Regards,
Dario
-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

[-- Attachment #2: Type: text/plain, Size: 126 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Olaf Hering]

On Thu, Apr 14, Ian Jackson wrote:

> Dario Faggioli writes ("Re: [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> > And, in those cases, usage should be gated by the appropriate
> > LIBXL_HAVE_FOOBAR symbol, which I see in the sources (e.g.,
> > for LIBXL_HAVE_NO_SUSPEND_RESUME or LIBXL_HAVE_DOMAIN_NODEAFFINITY) to
> > be the case, so good again. :-)
> 
> If libvirt specifies LIBXL_API_VERSION then it does not need to check
> any LIBXL_HAVE_*, since it will actually get an API corresponding to
> the specified API_VERSION.

This cant be true, because otherwise it will be unable to use vscsi and
perhaps usb (and whatever else showed up after 4.2).

Does that also mean that for example things related to LIBXL_HAVE_VSCSI
need to be wrapped in "#if LIBXL_API_VERSION > $whatever"? Maybe I'm
just misunderstand what you are trying to say.

Olaf

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Ian Jackson]

Jim Fehlig writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> I think that is a good idea too. I've sent a patch setting
> LIBXL_API_VERSION to 0x040200
> 
> https://www.redhat.com/archives/libvir-list/2016-April/msg00771.html

It seems that the libvirt LIBXL_API_VERSION is now rather higher, at
0x040400, since libvirt#fccf27253ced
  libxl: switch to using libxl_domain_create_restore from v4.4 API

One unfortunate effect of this is to break the osstest tests of the
Xen 4.3 stable branch, which for the moment is still allegedly in
security support.

I can't really see a way that this kind of problem could be avoided
in principle, without
  - providing a more sophisticated way for libxl callers to set the
    requested version
  - providing more compatibility code in libvirt, too, and retaining
    it for some time

I think instead that it would probably be better for osstest to
"freeze" the version of libvirt that it is using, every time we branch
Xen.

So Xen 4.4 would be tested with whatever libvirt we were using when
the stable branch for Xen 4.4 was made, and so on.

Does that sound sensible ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Ian Jackson]

(Adding Jan Beulich)

Ian Jackson writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> It seems that the libvirt LIBXL_API_VERSION is now rather higher, at
> 0x040400, since libvirt#fccf27253ced
>   libxl: switch to using libxl_domain_create_restore from v4.4 API
> 
> One unfortunate effect of this is to break the osstest tests of the
> Xen 4.3 stable branch, which for the moment is still allegedly in
> security support.
> 
> I can't really see a way that this kind of problem could be avoided
> in principle, without
>   - providing a more sophisticated way for libxl callers to set the
>     requested version
>   - providing more compatibility code in libvirt, too, and retaining
>     it for some time
> 
> I think instead that it would probably be better for osstest to
> "freeze" the version of libvirt that it is using, every time we branch
> Xen.
> 
> So Xen 4.4 would be tested with whatever libvirt we were using when
> the stable branch for Xen 4.4 was made, and so on.
> 
> Does that sound sensible ?

In the assumption that it is, I have:

Created the following branch refs on xenbits in the toplevel
libvirt.git:

 osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
 osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
 osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
 osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
 osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc

These were those tested by the following `tolerable' osstest push gate
flights for the corresponding Xen tree:

 xen-4.3-testing 9a0c7f5f8341 86673
 xen-4.4-testing 33fb8ff18584 85031
 xen-4.5-testing cda1cc170f07 83135
 xen-4.6-testing eac167e2610d 96031
 xen-4.7-testing 1a41ed5af5e1 95728

And I have prepared the patch below, which (together with a
prerequisite, in my tree) will implement this in osstest.

Ian.

From 5d1c91d3c53b580305e96d62f8ca84f85f8d3011 Mon Sep 17 00:00:00 2001
From: Ian Jackson <ian.jackson@eu.citrix.com>
Date: Mon, 27 Jun 2016 16:49:52 +0100
Subject: [OSSTEST PATCH] cr-daily-branch: libvirt: use frozen version on
 stable branches

libvirt master might increase its LIBXL_API_VERSION.  When this feeds
through osstest it can cause the push gates of Xen stable branches to
break.

So for stable Xen branches do not track libvirt upstream.  Instead,
use a frozen revision.  (Only for main push gate tests of stable Xen
branches.)

The frozen branch is never going to be updated so it is not suitable
for other kinds of uses.  In particular it won't get security fixes.
So we call the refs   osstest/frozen/xen-K.L-testing  to discourage
users from using them.

Deployment note: The Xen release checklist needs a new item "add this
frozen libvirt branch".

Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
---
 cr-daily-branch | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/cr-daily-branch b/cr-daily-branch
index 8b7c789..21780b8 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -186,6 +186,12 @@ if [ "x$REVISION_OVMF" = x ]; then
     fi
 fi
 if [ "x$REVISION_LIBVIRT" = x ]; then
+	case "$xenbranch" in
+	xen-[0-9]*-testing)
+		BASE_TAG_LIBVIRT=osstest/frozen/$xenbranch
+		export BASE_TAG_LIBVIRT
+		;;
+	esac
 	determine_version REVISION_LIBVIRT libvirt LIBVIRT
 	export REVISION_LIBVIRT
 fi
-- 
2.1.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Daniel P. Berrange]

On Mon, Jun 27, 2016 at 04:54:35PM +0100, Ian Jackson wrote:
> (Adding Jan Beulich)
> 
> Ian Jackson writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> > It seems that the libvirt LIBXL_API_VERSION is now rather higher, at
> > 0x040400, since libvirt#fccf27253ced
> >   libxl: switch to using libxl_domain_create_restore from v4.4 API
> > 
> > One unfortunate effect of this is to break the osstest tests of the
> > Xen 4.3 stable branch, which for the moment is still allegedly in
> > security support.
> > 
> > I can't really see a way that this kind of problem could be avoided
> > in principle, without
> >   - providing a more sophisticated way for libxl callers to set the
> >     requested version
> >   - providing more compatibility code in libvirt, too, and retaining
> >     it for some time
> > 
> > I think instead that it would probably be better for osstest to
> > "freeze" the version of libvirt that it is using, every time we branch
> > Xen.
> > 
> > So Xen 4.4 would be tested with whatever libvirt we were using when
> > the stable branch for Xen 4.4 was made, and so on.
> > 
> > Does that sound sensible ?
> 
> In the assumption that it is, I have:
> 
> Created the following branch refs on xenbits in the toplevel
> libvirt.git:
> 
>  osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
>  osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
>  osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
>  osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
>  osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc

How did you pick those hashes ? Would it make more sense to pick the
nearest libvirt release tag ? eg v1.3.2 instead of 33fb8ff18584 ?

> 
> These were those tested by the following `tolerable' osstest push gate
> flights for the corresponding Xen tree:
> 
>  xen-4.3-testing 9a0c7f5f8341 86673
>  xen-4.4-testing 33fb8ff18584 85031
>  xen-4.5-testing cda1cc170f07 83135
>  xen-4.6-testing eac167e2610d 96031
>  xen-4.7-testing 1a41ed5af5e1 95728
> 
> And I have prepared the patch below, which (together with a
> prerequisite, in my tree) will implement this in osstest.
> 
> Ian.
> 
> From 5d1c91d3c53b580305e96d62f8ca84f85f8d3011 Mon Sep 17 00:00:00 2001
> From: Ian Jackson <ian.jackson@eu.citrix.com>
> Date: Mon, 27 Jun 2016 16:49:52 +0100
> Subject: [OSSTEST PATCH] cr-daily-branch: libvirt: use frozen version on
>  stable branches
> 
> libvirt master might increase its LIBXL_API_VERSION.  When this feeds
> through osstest it can cause the push gates of Xen stable branches to
> break.
> 
> So for stable Xen branches do not track libvirt upstream.  Instead,
> use a frozen revision.  (Only for main push gate tests of stable Xen
> branches.)
> 
> The frozen branch is never going to be updated so it is not suitable
> for other kinds of uses.  In particular it won't get security fixes.
> So we call the refs   osstest/frozen/xen-K.L-testing  to discourage
> users from using them.
> 
> Deployment note: The Xen release checklist needs a new item "add this
> frozen libvirt branch".
> 
> Signed-off-by: Ian Jackson <Ian.Jackson@eu.citrix.com>
> ---
>  cr-daily-branch | 6 ++++++
>  1 file changed, 6 insertions(+)
> 
> diff --git a/cr-daily-branch b/cr-daily-branch
> index 8b7c789..21780b8 100755
> --- a/cr-daily-branch
> +++ b/cr-daily-branch
> @@ -186,6 +186,12 @@ if [ "x$REVISION_OVMF" = x ]; then
>      fi
>  fi
>  if [ "x$REVISION_LIBVIRT" = x ]; then
> +	case "$xenbranch" in
> +	xen-[0-9]*-testing)
> +		BASE_TAG_LIBVIRT=osstest/frozen/$xenbranch
> +		export BASE_TAG_LIBVIRT
> +		;;
> +	esac
>  	determine_version REVISION_LIBVIRT libvirt LIBVIRT
>  	export REVISION_LIBVIRT
>  fi

Overall I think your approach makes sense.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Ian Jackson]

Daniel P. Berrange writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> On Mon, Jun 27, 2016 at 04:54:35PM +0100, Ian Jackson wrote:
> > Created the following branch refs on xenbits in the toplevel
> > libvirt.git:
> > 
> >  osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
> >  osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
> >  osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
> >  osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
> >  osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
> 
> How did you pick those hashes ? Would it make more sense to pick the
> nearest libvirt release tag ? eg v1.3.2 instead of 33fb8ff18584 ?
> 
> > These were those tested by the following `tolerable' osstest push gate
> > flights for the corresponding Xen tree:
> > 
> >  xen-4.3-testing 9a0c7f5f8341 86673
> >  xen-4.4-testing 33fb8ff18584 85031
> >  xen-4.5-testing cda1cc170f07 83135
> >  xen-4.6-testing eac167e2610d 96031
> >  xen-4.7-testing 1a41ed5af5e1 95728

I picked them by searching my mail archives for osstest `tolerable'
push gate flights - ie, passes in our CI system.

That minimises the risk that the selected versions are themselves
troublesome for some reason, needing another round of adjustment.

It might indeed be better to convert them to nearby release tags.
However:

mariner:libvirt> git-describe 9a0c7f5f834185db9017c34aabc03ad99cf37bed
v1.3.2-202-g9a0c7f5
mariner:libvirt> git-describe 33fb8ff185846a7b4974105d2c9400690a6f95cf
v1.3.2-rc2-1-g33fb8ff
mariner:libvirt> git-describe cda1cc170f07b45911b3dad03e42c8ebfc210fa1
v1.3.1-262-gcda1cc1
mariner:libvirt> git-describe eac167e2610d3e59b32f7ec7ba78cbc8c420a425
v1.3.5-318-geac167e
mariner:libvirt> git-describe 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
v1.3.5-129-g1a41ed5
mariner:libvirt> 

So in most cases these hashes are well away from a release tag.

Does libvirt have stable release branches ?  One approach would be to
have osstest track a suitable libvirt stable release branche for each
Xen stable release branch.

That would involve setting up a push gate for each of the chosen
libvirt stable branches.  That would be worthwhile if we expect those
stable branches to acquire commits which break Xen, and which we could
like to be told about.  But I'm not sure that's the case.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Daniel P. Berrange]

On Mon, Jun 27, 2016 at 05:12:52PM +0100, Ian Jackson wrote:
> Daniel P. Berrange writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> > On Mon, Jun 27, 2016 at 04:54:35PM +0100, Ian Jackson wrote:
> > > Created the following branch refs on xenbits in the toplevel
> > > libvirt.git:
> > > 
> > >  osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
> > >  osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
> > >  osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
> > >  osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
> > >  osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
> > 
> > How did you pick those hashes ? Would it make more sense to pick the
> > nearest libvirt release tag ? eg v1.3.2 instead of 33fb8ff18584 ?
> > 
> > > These were those tested by the following `tolerable' osstest push gate
> > > flights for the corresponding Xen tree:
> > > 
> > >  xen-4.3-testing 9a0c7f5f8341 86673
> > >  xen-4.4-testing 33fb8ff18584 85031
> > >  xen-4.5-testing cda1cc170f07 83135
> > >  xen-4.6-testing eac167e2610d 96031
> > >  xen-4.7-testing 1a41ed5af5e1 95728
> 
> I picked them by searching my mail archives for osstest `tolerable'
> push gate flights - ie, passes in our CI system.
> 
> That minimises the risk that the selected versions are themselves
> troublesome for some reason, needing another round of adjustment.
> 
> It might indeed be better to convert them to nearby release tags.
> However:
> 
> mariner:libvirt> git-describe 9a0c7f5f834185db9017c34aabc03ad99cf37bed
> v1.3.2-202-g9a0c7f5
> mariner:libvirt> git-describe 33fb8ff185846a7b4974105d2c9400690a6f95cf
> v1.3.2-rc2-1-g33fb8ff
> mariner:libvirt> git-describe cda1cc170f07b45911b3dad03e42c8ebfc210fa1
> v1.3.1-262-gcda1cc1
> mariner:libvirt> git-describe eac167e2610d3e59b32f7ec7ba78cbc8c420a425
> v1.3.5-318-geac167e
> mariner:libvirt> git-describe 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
> v1.3.5-129-g1a41ed5
> mariner:libvirt> 
> 
> So in most cases these hashes are well away from a release tag.
> 
> Does libvirt have stable release branches ?  One approach would be to
> have osstest track a suitable libvirt stable release branche for each
> Xen stable release branch.

Yep, there is a vN.N.N-maint branch for every release

NB, with our new numbering that'll be changing nto vN.N-maint instead.

> That would involve setting up a push gate for each of the chosen
> libvirt stable branches.  That would be worthwhile if we expect those
> stable branches to acquire commits which break Xen, and which we could
> like to be told about.  But I'm not sure that's the case.

Stuff goes onto the stable branches on an as-needed basis - mostly coming
from the distro maintainers response to bug fixes from their users. There's
probably not a whole lot that's touching xen on a regular basis and we're
quite strict in what we accept for stable.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Jim Fehlig]

On 06/27/2016 10:12 AM, Ian Jackson wrote:
> Daniel P. Berrange writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
>> On Mon, Jun 27, 2016 at 04:54:35PM +0100, Ian Jackson wrote:
>>> Created the following branch refs on xenbits in the toplevel
>>> libvirt.git:
>>>
>>>   osstest/frozen/xen-4.3-testing 9a0c7f5f834185db9017c34aabc03ad99cf37bed
>>>   osstest/frozen/xen-4.4-testing 33fb8ff185846a7b4974105d2c9400690a6f95cf
>>>   osstest/frozen/xen-4.5-testing cda1cc170f07b45911b3dad03e42c8ebfc210fa1
>>>   osstest/frozen/xen-4.6-testing eac167e2610d3e59b32f7ec7ba78cbc8c420a425
>>>   osstest/frozen/xen-4.7-testing 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
>> How did you pick those hashes ? Would it make more sense to pick the
>> nearest libvirt release tag ? eg v1.3.2 instead of 33fb8ff18584 ?
>>
>>> These were those tested by the following `tolerable' osstest push gate
>>> flights for the corresponding Xen tree:
>>>
>>>   xen-4.3-testing 9a0c7f5f8341 86673
>>>   xen-4.4-testing 33fb8ff18584 85031
>>>   xen-4.5-testing cda1cc170f07 83135
>>>   xen-4.6-testing eac167e2610d 96031
>>>   xen-4.7-testing 1a41ed5af5e1 95728
> I picked them by searching my mail archives for osstest `tolerable'
> push gate flights - ie, passes in our CI system.
>
> That minimises the risk that the selected versions are themselves
> troublesome for some reason, needing another round of adjustment.
>
> It might indeed be better to convert them to nearby release tags.
> However:
>
> mariner:libvirt> git-describe 9a0c7f5f834185db9017c34aabc03ad99cf37bed
> v1.3.2-202-g9a0c7f5
> mariner:libvirt> git-describe 33fb8ff185846a7b4974105d2c9400690a6f95cf
> v1.3.2-rc2-1-g33fb8ff
> mariner:libvirt> git-describe cda1cc170f07b45911b3dad03e42c8ebfc210fa1
> v1.3.1-262-gcda1cc1

It seems odd that Xen 4.5 would use an older libvirt release than Xen 4.3.

> mariner:libvirt> git-describe eac167e2610d3e59b32f7ec7ba78cbc8c420a425
> v1.3.5-318-geac167e
> mariner:libvirt> git-describe 1a41ed5af5e1704dd9b0bdca40df5c9cacbdabfc
> v1.3.5-129-g1a41ed5
> mariner:libvirt>
>
> So in most cases these hashes are well away from a release tag.
>
> Does libvirt have stable release branches ?  One approach would be to
> have osstest track a suitable libvirt stable release branche for each
> Xen stable release branch.

I see Daniel already answered this question.

>
> That would involve setting up a push gate for each of the chosen
> libvirt stable branches.  That would be worthwhile if we expect those
> stable branches to acquire commits which break Xen, and which we could
> like to be told about.  But I'm not sure that's the case.

I occasionally backport Xen bug fixes to -maint branches. Cole has also grabbed 
some Xen bug fixes when making a stable release of a -maint branch. But such 
backports should be trivial and obvious bug fixes that shouldn't cause build or 
runtime breakage with Xen.

Regards,
Jim


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Ian Jackson]

Jim Fehlig writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> On 06/27/2016 10:12 AM, Ian Jackson wrote:
> > Does libvirt have stable release branches ?  One approach would be to
> > have osstest track a suitable libvirt stable release branche for each
> > Xen stable release branch.
> 
> I see Daniel already answered this question.
> 
> >
> > That would involve setting up a push gate for each of the chosen
> > libvirt stable branches.  That would be worthwhile if we expect those
> > stable branches to acquire commits which break Xen, and which we could
> > like to be told about.  But I'm not sure that's the case.
> 
> I occasionally backport Xen bug fixes to -maint branches. Cole has
> also grabbed some Xen bug fixes when making a stable release of a
> -maint branch. But such backports should be trivial and obvious bug
> fixes that shouldn't cause build or runtime breakage with Xen.

OK.  Thanks for the feedback.  I'll go ahead with my plan with the
git commit ids named in my earlier email.

Thanks,
Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Jan Beulich]

>>> On 27.06.16 at 18:54, <ian.jackson@eu.citrix.com> wrote:
> Jim Fehlig writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver 
> breakage -- where to define LIBXL_API_VERSION?"):
>> On 06/27/2016 10:12 AM, Ian Jackson wrote:
>> > Does libvirt have stable release branches ?  One approach would be to
>> > have osstest track a suitable libvirt stable release branche for each
>> > Xen stable release branch.
>> 
>> I see Daniel already answered this question.
>> 
>> >
>> > That would involve setting up a push gate for each of the chosen
>> > libvirt stable branches.  That would be worthwhile if we expect those
>> > stable branches to acquire commits which break Xen, and which we could
>> > like to be told about.  But I'm not sure that's the case.
>> 
>> I occasionally backport Xen bug fixes to -maint branches. Cole has
>> also grabbed some Xen bug fixes when making a stable release of a
>> -maint branch. But such backports should be trivial and obvious bug
>> fixes that shouldn't cause build or runtime breakage with Xen.
> 
> OK.  Thanks for the feedback.  I'll go ahead with my plan with the
> git commit ids named in my earlier email.

The only (hopefully highly theoretical) problem I see with this is that
we may end up picking a libvirt commit which subsequently (e.g. via
a libxl backport) turns out to have an issue. Such a problem could be
dealt with in the suggested the stable branch tracking model (or any
other model not dealing with something completely frozen).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [libvirt] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Ian Jackson]

Jan Beulich writes ("Re: [libvirt] [Xen-devel] Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?"):
> On 27.06.16 at 18:54, <ian.jackson@eu.citrix.com> wrote:
> > OK.  Thanks for the feedback.  I'll go ahead with my plan with the
> > git commit ids named in my earlier email.
> 
> The only (hopefully highly theoretical) problem I see with this is that
> we may end up picking a libvirt commit which subsequently (e.g. via
> a libxl backport) turns out to have an issue. Such a problem could be
> dealt with in the suggested the stable branch tracking model (or any
> other model not dealing with something completely frozen).

I don't think there is anything stopping us manually updating one of
these frozen "branches", should such a situation occur.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Fixing libvirt's libxl driver breakage -- where to define LIBXL_API_VERSION?

[Wei Liu]

Hi libvirt maintainers,

Xen's control library libxenlight (libxl) requires application
(libvirt in this case) to explictily define LIBXL_API_VERSION. This is
lacking at the moment so libvirt's libxl driver always gets the latest
APIs. We changed one of the APIs in libxl so libvirt's libxl driver
can't build at the moment.

I want to submit a patch for libvirt to define LIBXL_API_VERSION, but
I'm not sure where I should add that to. Can you give me some
pointers? Or even better -- a specific file that I should edit?
src/Makefile.am and src/Makfile.in are good candicate to me as far as
I can tell.

Thanks
Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel