[ANNOUNCE] Project Raisin

[ANNOUNCE] Project Raisin

[Stefano Stabellini]

Hi all,

A few weeks ago I started hacking on a new project, consisting in a set
of bash scripts to build xen-unstable and a few other useful elements.
It is still in very early stages.

The name of the new project is "Raisin", from Raise Xen, pun intended
;-)
It is a bit like DevStack[1] for Xen: its purpose is to simply and
quickly retrieve and build Xen and any other related components from
source, such as Libvirt and Grub2. Optionally it can install and
configure the system for you. In the future it could also build and
configure stub domains and driver domains. Another important goal of the
project is to effectively act as a working guideline on how to setup a
Xen system, so it is important that the code is easy to read and
understand. See the README for more information.

With the introduction of Raisin, keeping xen-unstable lightweight, clean
and well suited for distro packages is going to be easier because we
won't have to make compromises with the ease of installation from
source. Raisin is going to take care of setting up a full Xen
development environment from scratch. As an example, going forward we
can move the QEMU build from xen-unstable to Raisin, and integrate
Raisin in OSSTest.

I would like Raisin to work on as many distros as possible. At the
moment Raisin builds have been tested on Debian and Fedora.
Installations and system configurations have been tested only on Debian.

I would very much appreciate if anybody with expertise on Fedora, CentOS
and other distros could help add proper support for them.


The code is available here:

git://xenbits.xen.org/people/sstabellini/raisin.git
http://xenbits.xen.org/gitweb/?p=people/sstabellini/raisin.git


Happy Hacking!

- Stefano


[1] https://github.com/openstack-dev/devstack

Re: [ANNOUNCE] Project Raisin

[George Dunlap]

On Tue, Mar 31, 2015 at 11:16 AM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> Hi all,
>
> A few weeks ago I started hacking on a new project, consisting in a set
> of bash scripts to build xen-unstable and a few other useful elements.
> It is still in very early stages.
>
> The name of the new project is "Raisin", from Raise Xen, pun intended
> ;-)
> It is a bit like DevStack[1] for Xen: its purpose is to simply and
> quickly retrieve and build Xen and any other related components from
> source, such as Libvirt and Grub2. Optionally it can install and
> configure the system for you. In the future it could also build and
> configure stub domains and driver domains. Another important goal of the
> project is to effectively act as a working guideline on how to setup a
> Xen system, so it is important that the code is easy to read and
> understand. See the README for more information.
>
> With the introduction of Raisin, keeping xen-unstable lightweight, clean
> and well suited for distro packages is going to be easier because we
> won't have to make compromises with the ease of installation from
> source. Raisin is going to take care of setting up a full Xen
> development environment from scratch. As an example, going forward we
> can move the QEMU build from xen-unstable to Raisin, and integrate
> Raisin in OSSTest.
>
> I would like Raisin to work on as many distros as possible. At the
> moment Raisin builds have been tested on Debian and Fedora.
> Installations and system configurations have been tested only on Debian.
>
> I would very much appreciate if anybody with expertise on Fedora, CentOS
> and other distros could help add proper support for them.
>
>
> The code is available here:
>
> git://xenbits.xen.org/people/sstabellini/raisin.git
> http://xenbits.xen.org/gitweb/?p=people/sstabellini/raisin.git

Hey Stefano!  Looks like a pretty useful package overall.

Some initial feedback:

Automatically installing packages via sudo without asking scares me.

I think "raise" should take commands.  The main way package
prerequisites should be installed, I think, is for raise to give a
list of packages to the user, and the user to "sudo $PACKAGEMANAGER
install $LIST" themselves.

e.g.:
---
$ ./raise check_build_dependencies
Packages required for build:
build-essential libtool autoconf autopoint [...]

Optional packages for additional functionality:
[blah]
---

or (for example)

$ sudo yum install -y $(./raise check_build_dependencies -min -all)

Then the normal way of doing things should be:
* Edit config file
* ./raise check_build_dependencies
* [Install]
* ./raise build
* sudo ./raise install

Other functionality it might be nice to have:
* "build-srcpkg", which will make source rpms/debs
* "build-pkg", which would make installable packages; perhaps with
"install-pkg" which would automatically install them all for you
* "build [component]" -- if you only want to build a single component
(or set of them)

What do you think?

 -George

Re: [ANNOUNCE] Project Raisin

[Stefano Stabellini]

On Wed, 1 Apr 2015, George Dunlap wrote:
> On Tue, Mar 31, 2015 at 11:16 AM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > Hi all,
> >
> > A few weeks ago I started hacking on a new project, consisting in a set
> > of bash scripts to build xen-unstable and a few other useful elements.
> > It is still in very early stages.
> >
> > The name of the new project is "Raisin", from Raise Xen, pun intended
> > ;-)
> > It is a bit like DevStack[1] for Xen: its purpose is to simply and
> > quickly retrieve and build Xen and any other related components from
> > source, such as Libvirt and Grub2. Optionally it can install and
> > configure the system for you. In the future it could also build and
> > configure stub domains and driver domains. Another important goal of the
> > project is to effectively act as a working guideline on how to setup a
> > Xen system, so it is important that the code is easy to read and
> > understand. See the README for more information.
> >
> > With the introduction of Raisin, keeping xen-unstable lightweight, clean
> > and well suited for distro packages is going to be easier because we
> > won't have to make compromises with the ease of installation from
> > source. Raisin is going to take care of setting up a full Xen
> > development environment from scratch. As an example, going forward we
> > can move the QEMU build from xen-unstable to Raisin, and integrate
> > Raisin in OSSTest.
> >
> > I would like Raisin to work on as many distros as possible. At the
> > moment Raisin builds have been tested on Debian and Fedora.
> > Installations and system configurations have been tested only on Debian.
> >
> > I would very much appreciate if anybody with expertise on Fedora, CentOS
> > and other distros could help add proper support for them.
> >
> >
> > The code is available here:
> >
> > git://xenbits.xen.org/people/sstabellini/raisin.git
> > http://xenbits.xen.org/gitweb/?p=people/sstabellini/raisin.git
> 
> Hey Stefano!  Looks like a pretty useful package overall.
> 
> Some initial feedback:
> 
> Automatically installing packages via sudo without asking scares me.

I understand your concerns, however using sudo is the norm nowadays in
similar projects. DevStack, after which this project is modelled, does
that for example. Not the Docker is a great model but Docker also does
that.

You can still avoid this behaviour by passing -n to raise.sh. We should
probably clarify that you can (should?) avoid sudo entirely by passing -n.


> I think "raise" should take commands.  The main way package
> prerequisites should be installed, I think, is for raise to give a
> list of packages to the user, and the user to "sudo $PACKAGEMANAGER
> install $LIST" themselves.

Giving commands to raise could be a good UI improvement, but I disagree
on the package installation. In any case there is a way around that,
using the -n option. Maybe we could consider making that the default.


> e.g.:
> ---
> $ ./raise check_build_dependencies
> Packages required for build:
> build-essential libtool autoconf autopoint [...]
> 
> Optional packages for additional functionality:
> [blah]
> ---
> 
> or (for example)
> 
> $ sudo yum install -y $(./raise check_build_dependencies -min -all)
> 
> Then the normal way of doing things should be:
> * Edit config file
> * ./raise check_build_dependencies
> * [Install]
> * ./raise build
> * sudo ./raise install
> 
> Other functionality it might be nice to have:
> * "build-srcpkg", which will make source rpms/debs
> * "build-pkg", which would make installable packages; perhaps with
> "install-pkg" which would automatically install them all for you
> * "build [component]" -- if you only want to build a single component
> (or set of them)
> 
> What do you think?

At the moment the user can specify what to build using the config file.
I guess that being able to do it via command line too could be a nice
addition. build-srcpkg would be nice to have. I already introduced an
implicit build-pkg while install-pkg is basically raise.sh -i. So what
you are proposing is mostly a UI change, rather than a change in
functionality.

So far the idea has been that most users will just want to run raise.sh,
without any arguments or parameters, and everything will be set and
ready when the script completes.

Re: [ANNOUNCE] Project Raisin

[Ian Campbell]

On Wed, 2015-04-01 at 16:13 +0100, Stefano Stabellini wrote:
> > I think "raise" should take commands.  The main way package
> > prerequisites should be installed, I think, is for raise to give a
> > list of packages to the user, and the user to "sudo $PACKAGEMANAGER
> > install $LIST" themselves.
> 
> Giving commands to raise could be a good UI improvement, but I disagree
> on the package installation. In any case there is a way around that,
> using the -n option. Maybe we could consider making that the default.

Yeah, I think the default should be to do as George suggests and print
the prereqs as a list to be installed by the user, with an option
--auto-install-dependencies (or something shorter!)

Ian.

Re: [ANNOUNCE] Project Raisin

[George Dunlap]

On Wed, Apr 1, 2015 at 4:13 PM, Stefano Stabellini
<stefano.stabellini@eu.citrix.com> wrote:
> Giving commands to raise could be a good UI improvement, but I disagree
> on the package installation. In any case there is a way around that,
> using the -n option. Maybe we could consider making that the default.

Re commands, what do you think of how this looks?

git://xenbits.xen.org/people/gdunlap/raisin.git out/librarify/rfc-v1

I've only done a little bit, but take a look and see if you think
that's a direction you might want to go or not.  If you like it I'll
do more work (including rebasing to master).

 -George

Re: [ANNOUNCE] Project Raisin

[Ian Campbell]

On Tue, 2015-03-31 at 11:16 +0100, Stefano Stabellini wrote:
> Hi all,
> 
> A few weeks ago I started hacking on a new project, consisting in a set
> of bash scripts to build xen-unstable and a few other useful elements.
> It is still in very early stages.
> 
> The name of the new project is "Raisin", from Raise Xen, pun intended
> ;-)
> It is a bit like DevStack[1] for Xen: its purpose is to simply and
> quickly retrieve and build Xen and any other related components from
> source, such as Libvirt and Grub2.

I think this is great and I look forward to it being integrated with
osstest so we can start moving 3rd party components out of xen.git

>From a UI point of view I think it would be useful to move as much
non-user-facing stuff as possible into a subdirectory such that a user
just gets something like:
    $ ls
    README     raisin.sh*     lib      components
    COPYING    config

rather than being overwhelmed by all sorts of scripts etc.

I see you've moved the components into a subdir since the first posting.
which is good, I'd suggest to move as much of the other stuff
(common-funcitons, git-checkout, mk*) down a level too.

Ian.

Re: [ANNOUNCE] Project Raisin

[Stefano Stabellini]

On Wed, 1 Apr 2015, George Dunlap wrote:
> On Wed, Apr 1, 2015 at 4:13 PM, Stefano Stabellini
> <stefano.stabellini@eu.citrix.com> wrote:
> > Giving commands to raise could be a good UI improvement, but I disagree
> > on the package installation. In any case there is a way around that,
> > using the -n option. Maybe we could consider making that the default.
> 
> Re commands, what do you think of how this looks?
> 
> git://xenbits.xen.org/people/gdunlap/raisin.git out/librarify/rfc-v1
> 
> I've only done a little bit, but take a look and see if you think
> that's a direction you might want to go or not.  If you like it I'll
> do more work (including rebasing to master).

I think it is pretty good.

It would be nice if it followed the (unwritten) coding style, mostly
using test instead [, then and do on new lines, use the function
keyword.

core.sh is truly scary but hopefully it doesn't need to be touched.
Ever.

If we are going to have nodeps set by default, we should probably rename
it and on configuration failure print an helpful message such as "run
sudo raisin --deps if you want raisin to automatically install build
dependencies for you". It might actually be better to do that on a
separate commit from the one that introduces core.sh and build().

As we are now able to give commands to raise, I would remove the install
option and turn it into a different command.