HVM support to be removed from Debian Squeeze: call for volunteers

HVM support to be removed from Debian Squeeze: call for volunteers

[Thomas Goirand]

[message cross posted to the pkg-xen and xen-devel list]

Dear everyone,

Bastian Blank - which is the person (among others, but mainly him) that
is packaging Xen in Debian -, has decided last summer that he doesn't
want to deal with the qemu-dm of Xen, thus removing Xen Qemu and support
for HVM in Debian. Here is what he wrote:

http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2009-July/002366.html
http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2009-July/002382.html

Many people have expressed their concerns about this regression.

Before anyone flames Bastian Blank, I'd like to highlight the fact that
maintaining a PV dom0 in a distribution needs quite some work, he does
it for free, and that he should be thanks for his work within Debian,
rather than being blame for his maintainers choice. If he is not
interested or doesn't have time to maintain the Qemu version of Xen, he
has right to do that, and in fact I think it's not a bad idea to have a
binary package separated from the hypervisor itself.

Now, if the qemu-dm binary have been removed from the hypervisor
packages, nothing prevents anyone to make a new Debian package that
would have only the HVM bits of Xen, depending on the hypervisor
package, and then having it uploaded to SID (so that it would go in
testing later). Later, we could ask the maintainers of the
xen-linux-system package to add yet another dependency to the new
"xen-hvm-support" package that we would create (are you ok with this new
package name?).

The issue here is that Squeeze (the current Debian testing) is said to
be going to be functionality frozen next march. That means that if we
want to have a xen-hvm-support package in the next Debian, and recover
from Bastian's choice, then we need to act FAST, otherwise after the
deadline, there's a big chance that the release managers will not accept
the package into testing.

I am quite comfortable with Debian packaging myself, so I believe I can
write Debian packages with a quality good enough to enter the
distribution without too much hassle. I manage already a dozen of
packages in Debian, and I intend to add even more. But I do not feel
comfortable enough with the Xen sources to do it alone. So I'm calling
here for volunteers to work together with me on this package. I also
don't want to be the only person responsible for the package, as I
wouldn't be able to address any security bug fixing issue (as I don't
know the internals of the Xen and Qemu code, and don't have the time to
dive into it). Someone that is willing to do this job of security
maintenance HAS to raise hand here, otherwise I will NOT shoot myself in
the foot by asking for an upload of a package that would have a bad
security maintenance team (this is the kind of behavior that people hate
in Debian, which I think is right: no package with poor maintenance
should enter the distribution).

Mainly, the work would be to have a script (or some Makefile entries
with DESTDIR= for example) that would install all needed files in
debian/xen-hvm-support (if we call that package xen-hvm-support). Also,
the source package should contain ONLY what is strictly concerning
Qemu-dm (not a big issue, but it would be cleaner this way and make the
source package smaller). As of now, I don't know who is maintaining Qemu
in Xen at all. Any volunteer here that would work with me does NOT need
to know anything about Debian or Debian packaging, but only about Qemu
and Xen itself.

Also, I'm not a registered Debian developer (yet), so I can't upload
directly in the distribution without going by a sponsor for my package,
which can seriously slow down everything. If a registered DD could raise
hand, that would help as well.

I think that the best way here, is to build a (small, 3 or 4 persons)
team to work on this issue so it can be addressed fast. I'd like to
highlight also, that my company (GPLHost) doesn't really need the HVM
part of Xen (we sell PV only VMs), so I'd be doing it only for the
community, and not for myself. So please volunteer, as this is the only
way it can work. Best could be to create a project in Alioth (which I
never did, maybe someone else who had this experience should do it for
the team), or anywhere else. GPLHost can host the git repository and/or
web space for that.

If you are interested in helping, please reply to this message.

Thomas

P.S. to Bastian: Are you willing to help the team with advices, even if
you don't take any responsibility in the package?

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Ian Jackson]

Thomas Goirand writes ("[Xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers"):
>  Someone that is willing to do this job of security
> maintenance HAS to raise hand here,

Hello.  Yes, I volunteer (with my day job hat on).  I've done security
patching before in other contexts.

> Mainly, the work would be to have a script (or some Makefile entries
> with DESTDIR= for example) that would install all needed files in

You mean in the upstream package ?  This isn't quite trivial but it
should be doable.  There is a slight difficulty in that the filesystem
layout of the Debian Xen packages is quite different to upstream's.

But partitioning the output of `make dist-tools' etc. to provide
something that can be used for qemu-dm build should be easy enough.

> debian/xen-hvm-support (if we call that package xen-hvm-support). Also,
> the source package should contain ONLY what is strictly concerning
> Qemu-dm (not a big issue, but it would be cleaner this way and make the
> source package smaller). As of now, I don't know who is maintaining Qemu
> in Xen at all. Any volunteer here that would work with me does NOT need
> to know anything about Debian or Debian packaging, but only about Qemu
> and Xen itself.

I'm very happy to help out with this

> Also, I'm not a registered Debian developer (yet), so I can't upload
> directly in the distribution without going by a sponsor for my package,
> which can seriously slow down everything. If a registered DD could raise
> hand, that would help as well.

[swaps hats]

Also, I am a Debian Developer although I haven't been very active in
Debian recently.  I would be happy to sponsor uploads, provided they
seemed sane to me.

> I think that the best way here, is to build a (small, 3 or 4 persons)
> team to work on this issue so it can be addressed fast.

Yes.

Regards,
Ian.

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Stefano Stabellini]

On Tue, 15 Dec 2009, Ian Jackson wrote:
> Thomas Goirand writes ("[Xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers"):
> >  Someone that is willing to do this job of security
> > maintenance HAS to raise hand here,
> 
> Hello.  Yes, I volunteer (with my day job hat on).  I've done security
> patching before in other contexts.
> 

I can provide help as well (I maintain the XenServer qemu-dm tree that
is publicly available as part of the XCP project).

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Thomas Goirand]

Ian Jackson wrote:
> Thomas Goirand writes ("[Xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers"):
>>  Someone that is willing to do this job of security
>> maintenance HAS to raise hand here,
> 
> Hello.  Yes, I volunteer (with my day job hat on).  I've done security
> patching before in other contexts.

Great.

>> Mainly, the work would be to have a script (or some Makefile entries
>> with DESTDIR= for example) that would install all needed files in
> 
> You mean in the upstream package ?  This isn't quite trivial but it
> should be doable.

To me, that's the most easy way, yes, because this avoids the
double-work of listing files to install (once in the "make install" and
once in the debian/install).

> There is a slight difficulty in that the filesystem layout of the
> Debian Xen packages is quite different to upstream's.

Yes, because Debian uses versionned folders for /usr/lib/xen. Maybe we
could overwrite this in the Makefile with a variable. That's what I did
for numerous projects so that "make install" could be used by a spec
file using "make install MANUAL_DIR=%{_mandir}" for example. What do you
think? Could this apply to this project?

> But partitioning the output of `make dist-tools' etc. to provide
> something that can be used for qemu-dm build should be easy enough.

What do you mean here here? What's the use of "make dist-tools"?

One of my issue is to create the .orig.tar.gz with only the relevant
code, I'm not sure where to start. Could you do that? Maybe add a target
to the "upstream" Makefile? How should we do it?

>> debian/xen-hvm-support (if we call that package xen-hvm-support). Also,
>> the source package should contain ONLY what is strictly concerning
>> Qemu-dm (not a big issue, but it would be cleaner this way and make the
>> source package smaller). As of now, I don't know who is maintaining Qemu
>> in Xen at all. Any volunteer here that would work with me does NOT need
>> to know anything about Debian or Debian packaging, but only about Qemu
>> and Xen itself.
> 
> I'm very happy to help out with this

Cool.

> [swaps hats]
> 
> Also, I am a Debian Developer although I haven't been very active in
> Debian recently.  I would be happy to sponsor uploads, provided they
> seemed sane to me.

Great.

>> I think that the best way here, is to build a (small, 3 or 4 persons)
>> team to work on this issue so it can be addressed fast.
> 
> Yes.
> 
> Regards,
> Ian.

Can you start a project on Alioth, with a git repo? Would it be useful
(I never used alioth or gforge yet)?

Thomas

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Thomas Goirand]

Stefano Stabellini wrote:
> On Tue, 15 Dec 2009, Ian Jackson wrote:
>> Thomas Goirand writes ("[Xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers"):
>>>  Someone that is willing to do this job of security
>>> maintenance HAS to raise hand here,
>> Hello.  Yes, I volunteer (with my day job hat on).  I've done security
>> patching before in other contexts.
>>
> 
> I can provide help as well (I maintain the XenServer qemu-dm tree that
> is publicly available as part of the XCP project).

Is there a way to have the qemu-dm source tarball packaged separated
from anything else, so we can have something to work with? What source
should I fetch?

Thomas

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Stefano Stabellini]

On Tue, 15 Dec 2009, Thomas Goirand wrote:
> Stefano Stabellini wrote:
> > On Tue, 15 Dec 2009, Ian Jackson wrote:
> >> Thomas Goirand writes ("[Xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers"):
> >>>  Someone that is willing to do this job of security
> >>> maintenance HAS to raise hand here,
> >> Hello.  Yes, I volunteer (with my day job hat on).  I've done security
> >> patching before in other contexts.
> >>
> > 
> > I can provide help as well (I maintain the XenServer qemu-dm tree that
> > is publicly available as part of the XCP project).
> 
> Is there a way to have the qemu-dm source tarball packaged separated
> from anything else, so we can have something to work with? What source
> should I fetch?
> 

Yes, there is.
As a matter of fact we package qemu-xen in its own rpm on XCP, called
xen-device-model.
The qemu-xen XCP tree is available as a patchqueue against CS
284d056851f7 of qemu-xen-unstable.hg here:

http://xenbits.xen.org/xapi/qemu-xen-3.4.pq.hg

you might be interested in the spec file: mk/xen-device-model.spec.in.

However I don't suggest using the qemu-xen XCP tree because the last few
patches in the patchqueue require changes in the toolstack or other
patches in the xen tree available on XCP but not available on plain
unpatched systems.

Re: [Pkg-xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers

[Bastian Blank]

On Tue, Dec 15, 2009 at 12:59:13PM +0800, Thomas Goirand wrote:
> Now, if the qemu-dm binary have been removed from the hypervisor
> packages, nothing prevents anyone to make a new Debian package that
> would have only the HVM bits of Xen, depending on the hypervisor
> package, and then having it uploaded to SID (so that it would go in
> testing later).

I fact I already worked on such a package. The qemu-fork just need some
small patches to link against the provided libraries. However it is
again a fork and the patches are not clean enough to merge that into
qemu upstream yet and build it from their.

>                 Later, we could ask the maintainers of the
> xen-linux-system package to add yet another dependency to the new
> "xen-hvm-support" package that we would create (are you ok with this new
> package name?).

No, this is xen-qemu-dm. HVM works properly without, using a stubdomain.

>                Someone that is willing to do this job of security
> maintenance HAS to raise hand here, otherwise I will NOT shoot myself in
> the foot by asking for an upload of a package that would have a bad
> security maintenance team (this is the kind of behavior that people hate
> in Debian, which I think is right: no package with poor maintenance
> should enter the distribution).

The Xen in stable and oldstable is currently in this state. I fixed
several bugs in the core hypervisor but ignored the qemu part.

>                  Best could be to create a project in Alioth (which I
> never did, maybe someone else who had this experience should do it for
> the team), or anywhere else. GPLHost can host the git repository and/or
> web space for that.

This is best maintained as part of the existing Xen team, which needs
some cleanups.

> P.S. to Bastian: Are you willing to help the team with advices, even if
> you don't take any responsibility in the package?

Sure. I'm always available for advices.

I would like to have some further packages anyway: pv-grub,
qemu-stubdom. But the build environment for that needs a complete
rewrite to be able to produce acceptable packages. It needs to reuse
existing packages.

Bastian

-- 
A Vulcan can no sooner be disloyal than he can exist without breathing.
		-- Kirk, "The Menagerie", stardate 3012.4

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Thomas Goirand]

Stefano Stabellini wrote:
> Yes, there is.
> As a matter of fact we package qemu-xen in its own rpm on XCP, called
> xen-device-model.
> The qemu-xen XCP tree is available as a patchqueue against CS
> 284d056851f7 of qemu-xen-unstable.hg here:
> 
> http://xenbits.xen.org/xapi/qemu-xen-3.4.pq.hg
> 
> you might be interested in the spec file: mk/xen-device-model.spec.in.
> 
> However I don't suggest using the qemu-xen XCP tree because the last few
> patches in the patchqueue require changes in the toolstack or other
> patches in the xen tree available on XCP but not available on plain
> unpatched systems.

Our target should be Xen 3.4 in Debian, ATM. If you don't suggest to use
the qemu from XCP, which one should I use?

Do you think we should call the Debian package xen-device-model as well
(and not xen-hvm-support as I suggested...)?

Thomas

P.S: Form now on, I'll be writing only to the xen-devel@l.x.com list for
this thread, please do the same for future messages.

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Stefano Stabellini]

On Tue, 15 Dec 2009, Thomas Goirand wrote:
> Stefano Stabellini wrote:
> > Yes, there is.
> > As a matter of fact we package qemu-xen in its own rpm on XCP, called
> > xen-device-model.
> > The qemu-xen XCP tree is available as a patchqueue against CS
> > 284d056851f7 of qemu-xen-unstable.hg here:
> > 
> > http://xenbits.xen.org/xapi/qemu-xen-3.4.pq.hg
> > 
> > you might be interested in the spec file: mk/xen-device-model.spec.in.
> > 
> > However I don't suggest using the qemu-xen XCP tree because the last few
> > patches in the patchqueue require changes in the toolstack or other
> > patches in the xen tree available on XCP but not available on plain
> > unpatched systems.
> 
> Our target should be Xen 3.4 in Debian, ATM. If you don't suggest to use
> the qemu from XCP, which one should I use?

you should be using this one:

http://xenbits.xensource.com/git-http/qemu-xen-3.4-testing.git


> Do you think we should call the Debian package xen-device-model as well
> (and not xen-hvm-support as I suggested...)?
> 

After reading Bastian's email, I think xen-qemu-dm is probably the right
name for it.

BTW what package is going to provide ioemu-stubdom.gz?
Is it going to be provided by the main xen package or is it going to be
provided by a different one (xen-stubdom-dm maybe)?

Re: [Xen-devel] Re: [Pkg-xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers

[Samuel Thibault]

(Bcc-ing debian-hurd for the pv-grub part)

Bastian Blank, le Tue 15 Dec 2009 19:02:28 +0100, a écrit :
> I would like to have some further packages anyway: pv-grub,

I cannot but agree with this one, since we'll need it to nicely boot
GNU/Mach-based Xen domUs :)

> qemu-stubdom.

This one will probably be a bit difficult: it's both using the dm fork
of qemu, and the cross-compilation stubdom environment.

> But the build environment for that needs a complete rewrite to be able
> to produce acceptable packages. It needs to reuse existing packages.

Yes, each time I've thought about it I got a headache :)

Samuel


-- 
To UNSUBSCRIBE, email to debian-hurd-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Re: [Pkg-xen-devel] Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Bastian Blank]

On Tue, Dec 15, 2009 at 07:32:58PM +0100, Samuel Thibault wrote:
> Bastian Blank, le Tue 15 Dec 2009 19:02:28 +0100, a écrit :
> > I would like to have some further packages anyway: pv-grub,
> I cannot but agree with this one, since we'll need it to nicely boot
> GNU/Mach-based Xen domUs :)

Why?

> > qemu-stubdom.
> This one will probably be a bit difficult: it's both using the dm fork
> of qemu, and the cross-compilation stubdom environment.

No, there is no real cross-compilation involved.

> > But the build environment for that needs a complete rewrite to be able
> > to produce acceptable packages. It needs to reuse existing packages.
> Yes, each time I've thought about it I got a headache :)

The problem is that that the three parts (qemu/grub, newlib/libpci/libz
and mini-os) are not independant from each other. IMHO in a perfect
world all this variants could use one mini-os and combining them only
takes a linker call with a special linker script.

Also qemu-stubdom needs this fs-backend, which runs as root on the dom0,
just to read the keymaps (they are combined less than 1MiB, so building
them into the binary should be no problem).

Bastian

-- 
You can't evaluate a man by logic alone.
		-- McCoy, "I, Mudd", stardate 4513.3

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Pasi Kärkkäinen]

On Tue, Dec 15, 2009 at 06:27:10PM +0000, Stefano Stabellini wrote:
> On Tue, 15 Dec 2009, Thomas Goirand wrote:
> > Stefano Stabellini wrote:
> > > Yes, there is.
> > > As a matter of fact we package qemu-xen in its own rpm on XCP, called
> > > xen-device-model.
> > > The qemu-xen XCP tree is available as a patchqueue against CS
> > > 284d056851f7 of qemu-xen-unstable.hg here:
> > > 
> > > http://xenbits.xen.org/xapi/qemu-xen-3.4.pq.hg
> > > 
> > > you might be interested in the spec file: mk/xen-device-model.spec.in.
> > > 
> > > However I don't suggest using the qemu-xen XCP tree because the last few
> > > patches in the patchqueue require changes in the toolstack or other
> > > patches in the xen tree available on XCP but not available on plain
> > > unpatched systems.
> > 
> > Our target should be Xen 3.4 in Debian, ATM. If you don't suggest to use
> > the qemu from XCP, which one should I use?
> 
> you should be using this one:
> 
> http://xenbits.xensource.com/git-http/qemu-xen-3.4-testing.git
>

And here are the gitweb links for checking changelogs online:
http://xenbits.xen.org/gitweb?p=qemu-xen-3.4-testing.git;a=summary
http://xenbits.xen.org/gitweb?p=qemu-xen-unstable.git;a=summary

-- Pasi

Re: [Pkg-xen-devel] Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Samuel Thibault]

Bastian Blank, le Tue 15 Dec 2009 20:04:44 +0100, a écrit :
> On Tue, Dec 15, 2009 at 07:32:58PM +0100, Samuel Thibault wrote:
> > Bastian Blank, le Tue 15 Dec 2009 19:02:28 +0100, a écrit :
> > > I would like to have some further packages anyway: pv-grub,
> > I cannot but agree with this one, since we'll need it to nicely boot
> > GNU/Mach-based Xen domUs :)
> 
> Why?

Because it permits to load /hurd/ext2fs and /lib/ld.so directly from the
guest itself without having to patch pygrub yet more.

> > > qemu-stubdom.
> > This one will probably be a bit difficult: it's both using the dm fork
> > of qemu, and the cross-compilation stubdom environment.
> 
> No, there is no real cross-compilation involved.

Then I don't know what you call building with -nostdinc -isystem etc.

> > > But the build environment for that needs a complete rewrite to be able
> > > to produce acceptable packages. It needs to reuse existing packages.
> > Yes, each time I've thought about it I got a headache :)
> 
> The problem is that that the three parts (qemu/grub, newlib/libpci/libz
> and mini-os) are not independant from each other. IMHO in a perfect
> world all this variants could use one mini-os and combining them only
> takes a linker call with a special linker script.

So it would be an extended multilib compilation: x86, x86_64,
minios-x86, minios-x86_64?  Mini-os could indeed provide a library to be
linked with and you just need to provide main().  The eventual linkers
of all this would be qemu and grub.  There is still the problem of
having to rebuild whenever a library changes since there's no dynamic
sharing, but yes.

> Also qemu-stubdom needs this fs-backend, which runs as root on the dom0,
> just to read the keymaps (they are combined less than 1MiB, so building
> them into the binary should be no problem).

It also needs it for files non backed by a blktap, e.g. cdroms iirc.

Samuel

Re: [Pkg-xen-devel] Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Bastian Blank]

On Tue, Dec 15, 2009 at 08:33:12PM +0100, Samuel Thibault wrote:
> Bastian Blank, le Tue 15 Dec 2009 20:04:44 +0100, a écrit :
> > On Tue, Dec 15, 2009 at 07:32:58PM +0100, Samuel Thibault wrote:
> > > Bastian Blank, le Tue 15 Dec 2009 19:02:28 +0100, a écrit :
> > > > I would like to have some further packages anyway: pv-grub,
> > > I cannot but agree with this one, since we'll need it to nicely boot
> > > GNU/Mach-based Xen domUs :)
> > Why?
> Because it permits to load /hurd/ext2fs and /lib/ld.so directly from the
> guest itself without having to patch pygrub yet more.

I'm not sure if pv-grub support multiboot at all. Okay, I've never tried
it but relied on the information from other people.

> > > > qemu-stubdom.
> > No, there is no real cross-compilation involved.
> Then I don't know what you call building with -nostdinc -isystem etc.

The Linux kernel uses the same technique.

>                                      There is still the problem of
> having to rebuild whenever a library changes since there's no dynamic
> sharing, but yes.

Thats the normal static-linking problem.

> > Also qemu-stubdom needs this fs-backend, which runs as root on the dom0,
> > just to read the keymaps (they are combined less than 1MiB, so building
> > them into the binary should be no problem).
> It also needs it for files non backed by a blktap, e.g. cdroms iirc.

No, this is done with the normal block backend.

Bastian

-- 
Most legends have their basis in facts.
		-- Kirk, "And The Children Shall Lead", stardate 5029.5

Re: [Pkg-xen-devel] Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Samuel Thibault]

Bastian Blank, le Tue 15 Dec 2009 21:27:26 +0100, a écrit :
> On Tue, Dec 15, 2009 at 08:33:12PM +0100, Samuel Thibault wrote:
> > Bastian Blank, le Tue 15 Dec 2009 20:04:44 +0100, a écrit :
> > > On Tue, Dec 15, 2009 at 07:32:58PM +0100, Samuel Thibault wrote:
> > > > Bastian Blank, le Tue 15 Dec 2009 19:02:28 +0100, a écrit :
> > > > > I would like to have some further packages anyway: pv-grub,
> > > > I cannot but agree with this one, since we'll need it to nicely boot
> > > > GNU/Mach-based Xen domUs :)
> > > Why?
> > Because it permits to load /hurd/ext2fs and /lib/ld.so directly from the
> > guest itself without having to patch pygrub yet more.
> 
> I'm not sure if pv-grub support multiboot at all.

It doesn't yet, that's precisely the content of a patch being discussed
ATM on xen-devel :)

> > > > > qemu-stubdom.
> > > No, there is no real cross-compilation involved.
> > Then I don't know what you call building with -nostdinc -isystem etc.
> 
> The Linux kernel uses the same technique.

Except Linux is all self-contained and doesn't need to link against a
series of libraries.

> > > Also qemu-stubdom needs this fs-backend, which runs as root on the dom0,
> > > just to read the keymaps (they are combined less than 1MiB, so building
> > > them into the binary should be no problem).
> > It also needs it for files non backed by a blktap, e.g. cdroms iirc.
> 
> No, this is done with the normal block backend.

Even after an eject/insert cycle?  I do remember issues with blktap
restart in that case.

Samuel

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Ian Jackson]

Thomas Goirand writes ("Re: [Xen-devel] HVM support to be removed from Debian Squeeze: call for	volunteers"):
> To me, that's the most easy way, yes, because this avoids the
> double-work of listing files to install (once in the "make install" and
> once in the debian/install).

Quite so.

> Yes, because Debian uses versionned folders for /usr/lib/xen. Maybe we
> could overwrite this in the Makefile with a variable. That's what I did
> for numerous projects so that "make install" could be used by a spec
> file using "make install MANUAL_DIR=%{_mandir}" for example. What do you
> think? Could this apply to this project?

I've been thinking about this and I'm not sure the versioned folders
still make sense.  But if you want to send sensible patches to the Xen
build system to allow the interposing of a version number, I guess we
would accept them (after review).

> > But partitioning the output of `make dist-tools' etc. to provide
> > something that can be used for qemu-dm build should be easy enough.
> 
> What do you mean here here? What's the use of "make dist-tools"?

It's one of the targets in the upstream top-level Makefile.  The Xen
top-level Makefile doesn't use the conventional target names.
"dist-foo" means install "foo" in dist/install/.

> One of my issue is to create the .orig.tar.gz with only the relevant
> code, I'm not sure where to start. Could you do that? Maybe add a target
> to the "upstream" Makefile? How should we do it?

To build qemu-dm you need two things: the actual source code to
qemu-dm which is in a separate repository.  When we make the official
Xen tarballs we don't provide a separate tarball; we include it in the
same tree.  But the xen-unstable tree downloads the qemu-dm source
from the git repository.

So I would suggest you invent an orig.tar.gz by using git-archive
after fetching a suitably tagged qemu tree from
 git://xenbits.xensource.com/qemu-xen-VERSION.git


The other thing you need is the development headers and libraries
whose source code is in the Xen hg tree.  The qemu-dm source code
isn't set up for building from an "installed" copy of those libraries.
For example it includes, at build-time, fragments of makefiles from
the Xen build tree.

It would probably be possible to make a build work given a version of
these libraries and headers.  I can help with this but first we need a
libxen-dev package that contains libxc, libxs, libxenguest, etc.

> Can you start a project on Alioth, with a git repo? Would it be useful
> (I never used alioth or gforge yet)?

I'm happy to do that.  I'll try to get that set up.  I find Alioth's
git support confusing at times so it may not happen right away.

Ian.

Re: [Pkg-xen-devel] Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 1504 bytes --]

On Tue, 15 Dec 2009, Bastian Blank wrote:
> On Tue, Dec 15, 2009 at 08:33:12PM +0100, Samuel Thibault wrote:
> > Bastian Blank, le Tue 15 Dec 2009 20:04:44 +0100, a écrit :
> > > On Tue, Dec 15, 2009 at 07:32:58PM +0100, Samuel Thibault wrote:
> > > > Bastian Blank, le Tue 15 Dec 2009 19:02:28 +0100, a écrit :
> > > > > I would like to have some further packages anyway: pv-grub,
> > > > I cannot but agree with this one, since we'll need it to nicely boot
> > > > GNU/Mach-based Xen domUs :)
> > > Why?
> > Because it permits to load /hurd/ext2fs and /lib/ld.so directly from the
> > guest itself without having to patch pygrub yet more.
> 
> I'm not sure if pv-grub support multiboot at all. Okay, I've never tried
> it but relied on the information from other people.

pv-grub has multiboot-like module support on unstable now, thanks to
Samuel's work.

> > > Also qemu-stubdom needs this fs-backend, which runs as root on the dom0,
> > > just to read the keymaps (they are combined less than 1MiB, so building
> > > them into the binary should be no problem).
> > It also needs it for files non backed by a blktap, e.g. cdroms iirc.
> 
> No, this is done with the normal block backend.

Just to clarify: qemu stubdoms don't need fs-backend to boot anymore,
and certainly not for the keymaps.
On xen-unstable (and if my memory serves me right on 3.4 too) fs-backend
is only needed for suspend\resume and migration to write the qemu state
file in the dom0 filesystem.

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Thomas Goirand]

Ian Jackson wrote:
>> Yes, because Debian uses versionned folders for /usr/lib/xen. Maybe we
>> could overwrite this in the Makefile with a variable. That's what I did
>> for numerous projects so that "make install" could be used by a spec
>> file using "make install MANUAL_DIR=%{_mandir}" for example. What do you
>> think? Could this apply to this project?
> 
> I've been thinking about this and I'm not sure the versioned folders
> still make sense.  But if you want to send sensible patches to the Xen
> build system to allow the interposing of a version number, I guess we
> would accept them (after review).

I don't think it does either, but since Bastian and others want it, I
don't see why we shouldn't support it. I'll try to make a patch, yes.

>>> But partitioning the output of `make dist-tools' etc. to provide
>>> something that can be used for qemu-dm build should be easy enough.
>> What do you mean here here? What's the use of "make dist-tools"?
> 
> It's one of the targets in the upstream top-level Makefile.  The Xen
> top-level Makefile doesn't use the conventional target names.
> "dist-foo" means install "foo" in dist/install/.

Strange! :)

> To build qemu-dm you need two things: the actual source code to
> qemu-dm which is in a separate repository.  When we make the official
> Xen tarballs we don't provide a separate tarball; we include it in the
> same tree.  But the xen-unstable tree downloads the qemu-dm source
> from the git repository.
> 
> So I would suggest you invent an orig.tar.gz by using git-archive
> after fetching a suitably tagged qemu tree from
>  git://xenbits.xensource.com/qemu-xen-VERSION.git

Ok, I'll start from that. In fact, I'm quite happy xen-qemu is on Git
and not hg, as it's been years that I am used to git. But however...

In Lenny:

zigo@GPLHost:buzzig>_ /usr/src/xen$ git clone
http://xenbits.xensource.com/qemu-xen-3.4-testing.git
Initialized empty Git repository in /usr/src/xen/qemu-xen-3.4-testing/.git/
warning: remote HEAD refers to nonexistent ref, unable to checkout.

In SID:

root@GPLHost:xen018407>_ ~/sources/gplhost# git clone
http://xenbits.xensource.com/qemu-xen-3.4-testing.git
Initialized empty Git repository in
/root/sources/gplhost/qemu-xen-3.4-testing/.git/
fatal: http://xenbits.xensource.com/qemu-xen-3.4-testing.git/info/refs
not found: did you run git update-server-info on the server?

What's wrong here?

> The other thing you need is the development headers and libraries
> whose source code is in the Xen hg tree.  The qemu-dm source code
> isn't set up for building from an "installed" copy of those libraries.
> For example it includes, at build-time, fragments of makefiles from
> the Xen build tree.
>
> It would probably be possible to make a build work given a version of
> these libraries and headers.  I can help with this but first we need a
> libxen-dev package that contains libxc, libxs, libxenguest, etc.

That sounds, indeed, the way to go.

>> Can you start a project on Alioth, with a git repo? Would it be useful
>> (I never used alioth or gforge yet)?
> 
> I'm happy to do that.  I'll try to get that set up.  I find Alioth's
> git support confusing at times so it may not happen right away.
> 
> Ian.

I really don't know gforge, so again, it might not be the way to go. I
do have a public git myself, so if you can use the one of xensource,
that could be enough. Just please, let me know why I can't clone! :)

Thomas

Re: HVM support to be removed from Debian Squeeze: call for volunteers

[Keith Coleman]

On Wed, Dec 16, 2009 at 10:24 AM, Thomas Goirand <thomas@goirand.fr> wrote:
> Ok, I'll start from that. In fact, I'm quite happy xen-qemu is on Git
> and not hg, as it's been years that I am used to git. But however...
>
> In Lenny:
>
> zigo@GPLHost:buzzig>_ /usr/src/xen$ git clone
> http://xenbits.xensource.com/qemu-xen-3.4-testing.git
> Initialized empty Git repository in /usr/src/xen/qemu-xen-3.4-testing/.git/
> warning: remote HEAD refers to nonexistent ref, unable to checkout.
>
> In SID:
>
> root@GPLHost:xen018407>_ ~/sources/gplhost# git clone
> http://xenbits.xensource.com/qemu-xen-3.4-testing.git
> Initialized empty Git repository in
> /root/sources/gplhost/qemu-xen-3.4-testing/.git/
> fatal: http://xenbits.xensource.com/qemu-xen-3.4-testing.git/info/refs
> not found: did you run git update-server-info on the server?
>
> What's wrong here?

git clone http://xenbits.xensource.com/git-http/qemu-xen-3.4-testing.git


Keith Coleman

Re: [Pkg-xen-devel] HVM support to be removed from Debian Squeeze: call for volunteers

[Bastian Blank]

[-- Attachment #1.1: Type: text/plain, Size: 754 bytes --]

On Tue, Dec 15, 2009 at 07:02:28PM +0100, Bastian Blank wrote:
> I fact I already worked on such a package. The qemu-fork just need some
> small patches to link against the provided libraries. However it is
> again a fork and the patches are not clean enough to merge that into
> qemu upstream yet and build it from their.

http://hermes.jura.uni-tuebingen.de/~blank/debian/xen-test/
d7ac980cf0ba90ed2aa8dd0122643fc591132f605f73e98bcdc984dd10e0b1df  qemu-xen_3.4.2-1.debian.tar.gz
2fc5b931ae8f706f0a9a1da9dd1c9f5158d3d50a5cfc4b214dc88c9fb122a815  qemu-xen_3.4.2.orig.tar.gz

A little bit raw, but should work.

Bastian

-- 
There is a multi-legged creature crawling on your shoulder.
		-- Spock, "A Taste of Armageddon", stardate 3193.9

[-- Attachment #1.2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Qemu-dm compiling error with SDL

[Thomas Goirand]

Doing the following:

cp -f /usr/share/misc/config.sub .
cp -f /usr/share/misc/config.guess
./configure --audio-drv-list="oss alsa sdl pa esd" \
--audio-card-list="ac97 es1370 sb16 cs4231a adlib gus" --enable-mixemu
$(MAKE) install DESTDIR=debian/$(PKG_NAME)

generates the following error:

  CC    sdl.o
sdl.c: In function ‘sdl_update_caption’:
sdl.c:499: error: ‘domain_name’ undeclared (first use in this function)
sdl.c:499: error: (Each undeclared identifier is reported only once
sdl.c:499: error: for each function it appears in.)
make[1]: *** [sdl.o] Error 1
make[1]: Leaving directory `/usr/src/xen/xen-qemu-dm-3.4'
make: *** [install] Error 2
dpkg-buildpackage: failure: fakeroot debian/rules binary gave error exit
status 2

this did NOT happen with the Debian source package from Bastian, and it
still does this when I apply the debian/patches from him So I'm
wondering what's going on here. Does anyone know? Note that this happens
in both Lenny and SID.

Thomas

Re: Qemu-dm compiling error with SDL

[Ian Jackson]

Thomas Goirand writes ("Qemu-dm compiling error with SDL"):
> Doing the following:
> 
> cp -f /usr/share/misc/config.sub .
> cp -f /usr/share/misc/config.guess
> ./configure --audio-drv-list="oss alsa sdl pa esd" \
> --audio-card-list="ac97 es1370 sb16 cs4231a adlib gus" --enable-mixemu
> $(MAKE) install DESTDIR=debian/$(PKG_NAME)

The qemu-xen-*.git tree needs to be built with the "xen-setup"
script, rather than just running ./configure.

Ian.

Re: Re: Qemu-dm compiling error with SDL

[Thomas Goirand]

Ian Jackson wrote:
> Thomas Goirand writes ("Qemu-dm compiling error with SDL"):
>> Doing the following:
>>
>> cp -f /usr/share/misc/config.sub .
>> cp -f /usr/share/misc/config.guess
>> ./configure --audio-drv-list="oss alsa sdl pa esd" \
>> --audio-card-list="ac97 es1370 sb16 cs4231a adlib gus" --enable-mixemu
>> $(MAKE) install DESTDIR=debian/$(PKG_NAME)
> 
> The qemu-xen-*.git tree needs to be built with the "xen-setup"
> script, rather than just running ./configure.
> 
> Ian.

Thanks for this, I have now a package (which is litian clean). Resulting
work is here:

http://ftparchive.gplhost.com/debian/pool/lenny/main/x/xen-qemu-dm-3.4/

I have few questions though.

1- What is qemu-nbd-xen for, and do I need to package it (together with
its manpage)? Should it go in /usr/bin?

2- Is there anything else than qemu-dm, or qemu-img-xen, that needs to
be present in this package?

3- By default, the xen-setup script does:

./configure --disable-gfx-check --disable-curses \
   --disable-slirp "$@" --prefix=/usr

but I saw in the help of ./configure that there is:
--audio-drv-list

that can have the value oss alsa sdl esd pa fmod. I really believe that
OSS support only is not a good idea, and that ALSA, Pulse Audio, SDL and
ESD support would be good. Are they activated by default?

4- Is this normal, and could it be avoided:

dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
"debian/xen-qemu-dm-3.4/usr/lib/xen-3.4/boot/qemu-dm" were not uselessly
linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libXext.so.6 could be avoided if
"debian/xen-qemu-dm-3.4/usr/lib/xen-3.4/boot/qemu-dm" were not uselessly
linked against it (they use none of its symbols).

Another topic,

The list of persons that have volunteered for working on the package are
the persons in Cc: in this messages. Now, I need to make a common email
address for it. I can make a xenqemudm@gplhost.com, redirecting to us 4,
but I'm open to any other solution. Especially considering that I will
still need sponsoring and that Ian J. would be the one uploading, maybe
an email address @debian.org, signed with Ian J.'s Debian key would make
more sense. Ian J., what do you think, and can you take care of that?

Thomas