[PATCH OSSTEST 0/5] Debian Jessie patches

[PATCH OSSTEST 0/5] Debian Jessie patches

[Wei Liu]

I worked a little bit on upgrading Osstest to Jessie a few months ago but
dropped the ball. Here are the patches I wrote.

With these patches I can run build jobs (Xen and kernel) with Jessie. Running
test jobs requires more work because now Jessie uses systemd a lot of things
have changed.

Wei.

Wei Liu (5):
  mg-debian-installer-update: use new url for armhf packages
  Debian.pm: install grub to default device
  Cope with Jessie's d-i vg name
  ts-xen-build-prep: install libnl-route-3-dev on Jessie
  ts-xen-build-prep: install libc6-dev-i386 for jessie

 Osstest/Debian.pm          |  2 ++
 mg-debian-installer-update |  9 +++++++--
 ts-host-install            |  4 +++-
 ts-xen-build-prep          | 37 +++++++++++++++++++++----------------
 4 files changed, 33 insertions(+), 19 deletions(-)

-- 
1.9.1

[PATCH OSSTEST 1/5] mg-debian-installer-update: use new url for armhf packages

[Wei Liu]

When we use Wheezy the arm kernel is only available in wheezy-backports.
In Jessie it is not in backports.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
Not sure if this patch does the right thing.
---
 mg-debian-installer-update | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 4ff346f..a6693d0 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -93,8 +93,13 @@ done
 # platforms. Construct something suitable from the latest kernel in
 # wheezy-backports.
 if [ $arch = armhf ]; then
-    bp="$sbase-backports"
-    pfile=$bp/main/binary-armhf/Packages.gz
+    if [ x$suite = xwheezy ]; then
+	bp="$sbase-backports"
+	pfile=$bp/main/binary-armhf/Packages.gz
+    else
+	# wheezy onward, starting from jessie
+        pfile=$sbase/main/binary-armhf/Packages.gz
+    fi
 
     fetch $pfile >Packages.gz
 
-- 
1.9.1

[PATCH OSSTEST 2/5] Debian.pm: install grub to default device

[Wei Liu]

Debian installer used to have the behaviour to install grub on the
default device. But it's changed in Jessie.

See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759737.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 Osstest/Debian.pm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 537ccbe..e95ca90 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -672,6 +672,8 @@ popularity-contest popularity-contest/participate boolean false
 tasksel tasksel/first multiselect standard, web-server
 
 d-i grub-installer/only_debian boolean true
+# see https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=759737
+d-i grub-installer/bootdev string default
 
 d-i finish-install/keep-consoles boolean true
 d-i finish-install/reboot_in_progress note
-- 
1.9.1

[PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name

[Wei Liu]

In ts-xen-build-prep, replace matching lenny with jessie since lenny is
not used in production anymore.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 ts-host-install   | 4 +++-
 ts-xen-build-prep | 4 ++--
 2 files changed, 5 insertions(+), 3 deletions(-)

diff --git a/ts-host-install b/ts-host-install
index b73c7ff..6e2106e 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -87,7 +87,9 @@ END
 
     await_tcp(get_timeout($ho,'reboot',$timeout{Sshd}), 14,$ho);
 
-    our $vgname= $ho->{Name};
+    our $vgname= $ho->{Suite} =~ m/jessie/
+                 ? "$ho->{Name}-vg"
+                 : $ho->{Name};
 
     if ($ho->{Suite} =~ m/lenny/) {
         $vgname .= ".$c{TestHostDomain}";
diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 4dece65..25ba3d4 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -53,8 +53,8 @@ sub overall_limit_pe ($) {
 
 sub determine_vg_lv () {
     $vg=
-        $ho->{Suite} =~ m/lenny/
-        ? "$ho->{Name}.$c{TestHostDomain}"
+        $ho->{Suite} =~ m/jessie/
+        ? "$ho->{Name}-vg"
         : $ho->{Name};
     $lv = "/dev/$vg/$lvleaf";
 }
-- 
1.9.1

[PATCH OSSTEST 4/5] ts-xen-build-prep: install libnl-route-3-dev on Jessie

[Wei Liu]

There is no package called libnl-dev on Jessie.  To build remus, use
libnl-route-3-dev instead.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 ts-xen-build-prep | 31 ++++++++++++++++++-------------
 1 file changed, 18 insertions(+), 13 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 25ba3d4..312ae78 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -198,19 +198,24 @@ END
 }
 
 sub prep () {
-    target_install_packages($ho,
-                            qw(mercurial
-                               build-essential bin86 bcc iasl bc
-                               flex bison cmake
-                               libpci-dev libncurses5-dev libssl-dev python-dev
-                               libx11-dev git-core uuid-dev gettext gawk
-                               libsdl-dev libyajl-dev libaio-dev libpixman-1-dev
-                               libglib2.0-dev liblzma-dev pkg-config
-                               autoconf automake libtool xsltproc
-                               libxml2-utils libxml2-dev libnl-dev
-                               libdevmapper-dev w3c-dtd-xhtml libxml-xpath-perl
-                               ccache nasm checkpolicy));
-
+    my @packages = qw(mercurial
+                      build-essential bin86 bcc iasl bc
+                      flex bison cmake
+                      libpci-dev libncurses5-dev libssl-dev python-dev
+                      libx11-dev git-core uuid-dev gettext gawk
+                      libsdl-dev libyajl-dev libaio-dev libpixman-1-dev
+                      libglib2.0-dev liblzma-dev pkg-config
+                      autoconf automake libtool xsltproc
+                      libxml2-utils libxml2-dev
+                      libdevmapper-dev w3c-dtd-xhtml libxml-xpath-perl
+                      ccache nasm checkpolicy);
+
+    if ($ho->{Suite} =~ m/jessie/) {
+	push(@packages, "libnl-route-3-dev");
+    } else {
+	push(@packages, "libnl-dev");
+    }
+    target_install_packages($ho, @packages);
     target_cmd_root($ho, "chmod -R a+r /usr/share/git-core/templates");
     # workaround for Debian #595728
 
-- 
1.9.1

[PATCH OSSTEST 5/5] ts-xen-build-prep: install libc6-dev-i386 for jessie

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 ts-xen-build-prep | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 312ae78..76758ac 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -229,7 +229,7 @@ END
     if ($ho->{Suite} !~ m/lenny|squeeze/) {
         target_install_packages($ho, 'libfdt-dev');
     }
-    if ($r{arch} eq 'amd64' && $ho->{Suite} =~ m/wheezy/) {
+    if ($r{arch} eq 'amd64' && $ho->{Suite} =~ m/wheezy|jessie/) {
         target_install_packages($ho, 'libc6-dev-i386');
     }
 }
-- 
1.9.1

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Wei Liu]

On Wed, May 20, 2015 at 06:56:23PM +0100, Wei Liu wrote:
> I worked a little bit on upgrading Osstest to Jessie a few months ago but
> dropped the ball. Here are the patches I wrote.
> 
> With these patches I can run build jobs (Xen and kernel) with Jessie. Running
> test jobs requires more work because now Jessie uses systemd a lot of things
> have changed.

Actually the problems I see might not be systemd related. The problem is
that Osstest cannot setup test host. Last time I didn't look into
details.

Wei.

Re: [PATCH OSSTEST 1/5] mg-debian-installer-update: use new url for armhf packages

[Ian Campbell]

On Wed, 2015-05-20 at 18:56 +0100, Wei Liu wrote:
> When we use Wheezy the arm kernel is only available in wheezy-backports.
> In Jessie it is not in backports.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
> Not sure if this patch does the right thing.

I have some patches addressing this in a more flexible way in my local
branch where I am working on arm64 support, which overlaps a bit with
"make Jessie work" since arm64 only arrived with Jessie

I'm hoping to post an RFC of that stuff shortly...

Re: [PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name

[Ian Campbell]

On Wed, 2015-05-20 at 18:56 +0100, Wei Liu wrote:
> In ts-xen-build-prep, replace matching lenny with jessie since lenny is
> not used in production anymore.

AIUI the default VG name scheme changed? Can you say how here please.

Also I think in general we should aim for the "non-exceptional" case to
be the default one, and use explicit matching for issues with historical
versions. IOW test for wheezy and earlier and override the default (i.e.
Jessie) name. The rationale is that we shouldn't need to add Stretch
here too unless it also differs from Jessie.

That principal seems to apply to a some of the following patches too.

WRT "wheezy and earlier" I've been wondering about providing a function
to transform a Debian code name to a sortable integer. e.g.

sub debver($) {
    my %vers = qw(Lenny 50
                  Squeeze 60
                  Wheezy 70
                  Jessie 80
                  Stretch 90
                  Sid 1000000);
    # Unknown is > sid...
    return $vers{ucfirst($_[0])) // 1000001;
}

so you can do things like
    if ( debver($ho->{Suite}) < debver('jessie') ) {
        workaround_a_thing();

Thoughts?
Ian.

> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>  ts-host-install   | 4 +++-
>  ts-xen-build-prep | 4 ++--
>  2 files changed, 5 insertions(+), 3 deletions(-)
> 
> diff --git a/ts-host-install b/ts-host-install
> index b73c7ff..6e2106e 100755
> --- a/ts-host-install
> +++ b/ts-host-install
> @@ -87,7 +87,9 @@ END
>  
>      await_tcp(get_timeout($ho,'reboot',$timeout{Sshd}), 14,$ho);
>  
> -    our $vgname= $ho->{Name};
> +    our $vgname= $ho->{Suite} =~ m/jessie/
> +                 ? "$ho->{Name}-vg"
> +                 : $ho->{Name};
>  
>      if ($ho->{Suite} =~ m/lenny/) {
>          $vgname .= ".$c{TestHostDomain}";
> diff --git a/ts-xen-build-prep b/ts-xen-build-prep
> index 4dece65..25ba3d4 100755
> --- a/ts-xen-build-prep
> +++ b/ts-xen-build-prep
> @@ -53,8 +53,8 @@ sub overall_limit_pe ($) {
>  
>  sub determine_vg_lv () {
>      $vg=
> -        $ho->{Suite} =~ m/lenny/
> -        ? "$ho->{Name}.$c{TestHostDomain}"
> +        $ho->{Suite} =~ m/jessie/
> +        ? "$ho->{Name}-vg"
>          : $ho->{Name};
>      $lv = "/dev/$vg/$lvleaf";
>  }

Re: [PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name

[Wei Liu]

On Thu, May 21, 2015 at 04:10:02PM +0100, Ian Campbell wrote:
> On Wed, 2015-05-20 at 18:56 +0100, Wei Liu wrote:
> > In ts-xen-build-prep, replace matching lenny with jessie since lenny is
> > not used in production anymore.
> 
> AIUI the default VG name scheme changed? Can you say how here please.
> 

Sure.

> Also I think in general we should aim for the "non-exceptional" case to
> be the default one, and use explicit matching for issues with historical
> versions. IOW test for wheezy and earlier and override the default (i.e.
> Jessie) name. The rationale is that we shouldn't need to add Stretch
> here too unless it also differs from Jessie.
> 
> That principal seems to apply to a some of the following patches too.
> 
> WRT "wheezy and earlier" I've been wondering about providing a function
> to transform a Debian code name to a sortable integer. e.g.
> 
> sub debver($) {
>     my %vers = qw(Lenny 50
>                   Squeeze 60
>                   Wheezy 70
>                   Jessie 80
>                   Stretch 90
>                   Sid 1000000);
>     # Unknown is > sid...
>     return $vers{ucfirst($_[0])) // 1000001;
> }
> 
> so you can do things like
>     if ( debver($ho->{Suite}) < debver('jessie') ) {
>         workaround_a_thing();
> 

You would need to test for "$ho->{Suite} == A" or "A < $ho->{Suite} < B".

And it's just another form of saying "if ($ho->{Suite} =~ /XXX/)" IMHO.

> Thoughts?

That would be nice but I doubt it would be of much use -- who
doesn't want the latest version in production... ;-)

Wei.

> Ian.
> 
> > 
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> >  ts-host-install   | 4 +++-
> >  ts-xen-build-prep | 4 ++--
> >  2 files changed, 5 insertions(+), 3 deletions(-)
> > 
> > diff --git a/ts-host-install b/ts-host-install
> > index b73c7ff..6e2106e 100755
> > --- a/ts-host-install
> > +++ b/ts-host-install
> > @@ -87,7 +87,9 @@ END
> >  
> >      await_tcp(get_timeout($ho,'reboot',$timeout{Sshd}), 14,$ho);
> >  
> > -    our $vgname= $ho->{Name};
> > +    our $vgname= $ho->{Suite} =~ m/jessie/
> > +                 ? "$ho->{Name}-vg"
> > +                 : $ho->{Name};
> >  
> >      if ($ho->{Suite} =~ m/lenny/) {
> >          $vgname .= ".$c{TestHostDomain}";
> > diff --git a/ts-xen-build-prep b/ts-xen-build-prep
> > index 4dece65..25ba3d4 100755
> > --- a/ts-xen-build-prep
> > +++ b/ts-xen-build-prep
> > @@ -53,8 +53,8 @@ sub overall_limit_pe ($) {
> >  
> >  sub determine_vg_lv () {
> >      $vg=
> > -        $ho->{Suite} =~ m/lenny/
> > -        ? "$ho->{Name}.$c{TestHostDomain}"
> > +        $ho->{Suite} =~ m/jessie/
> > +        ? "$ho->{Name}-vg"
> >          : $ho->{Name};
> >      $lv = "/dev/$vg/$lvleaf";
> >  }
> 

Re: [PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name

[Ian Campbell]

On Thu, 2015-05-21 at 17:04 +0100, Wei Liu wrote:
> On Thu, May 21, 2015 at 04:10:02PM +0100, Ian Campbell wrote:
> > On Wed, 2015-05-20 at 18:56 +0100, Wei Liu wrote:
> > > In ts-xen-build-prep, replace matching lenny with jessie since lenny is
> > > not used in production anymore.
> > 
> > AIUI the default VG name scheme changed? Can you say how here please.
> > 
> 
> Sure.
> 
> > Also I think in general we should aim for the "non-exceptional" case to
> > be the default one, and use explicit matching for issues with historical
> > versions. IOW test for wheezy and earlier and override the default (i.e.
> > Jessie) name. The rationale is that we shouldn't need to add Stretch
> > here too unless it also differs from Jessie.
> > 
> > That principal seems to apply to a some of the following patches too.
> > 
> > WRT "wheezy and earlier" I've been wondering about providing a function
> > to transform a Debian code name to a sortable integer. e.g.
> > 
> > sub debver($) {
> >     my %vers = qw(Lenny 50
> >                   Squeeze 60
> >                   Wheezy 70
> >                   Jessie 80
> >                   Stretch 90
> >                   Sid 1000000);
> >     # Unknown is > sid...
> >     return $vers{ucfirst($_[0])) // 1000001;
> > }
> > 
> > so you can do things like
> >     if ( debver($ho->{Suite}) < debver('jessie') ) {
> >         workaround_a_thing();
> > 
> 
> You would need to test for "$ho->{Suite} == A" or "A < $ho->{Suite} < B".

if ( debver($ho->{Suite}) <= debver('squeeze') )
    handle Squeeze or earlier
elseif ( debver($ho->{Suite}) <= debver('wheezy') )
    handle Wheezy
else
    handle Jessie onwards

I'm not sure if Perl supports
        debver('squeeze') < debver($ho->{Suite}) <= debver('wheezy')
like some languages do.

> And it's just another form of saying "if ($ho->{Suite} =~ /XXX/)" IMHO.

The difference is you don't need to enumerate all the names. e.g.
"lenny|squeeze|wheezy" becomes "< debver('wheezy')"

> > Thoughts?
> 
> That would be nice but I doubt it would be of much use -- who
> doesn't want the latest version in production... ;-)

People who think it is useful to continue to test Wheezy for some
reason? People who have other constraints preventing them from upgrading
right away? People who want to use a _newer_ version for some reason and
want to make Stretch work while not breaking Wheezy/Jessie for everyone
else.

If it were expensive to support I'd agree, but I don't think it is,
especially if you arrange to never need to touch it again...

BTW, at some point I'm hoping to extend my distros flight stuff to cover
testing the distro host packages, at which point it would naturally be
useful to cover at least current stable and current testing, if not
oldstable or sid.

Ian.

Re: [PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name

[Wei Liu]

On Thu, May 21, 2015 at 05:15:55PM +0100, Ian Campbell wrote:
> On Thu, 2015-05-21 at 17:04 +0100, Wei Liu wrote:
> > On Thu, May 21, 2015 at 04:10:02PM +0100, Ian Campbell wrote:
> > > On Wed, 2015-05-20 at 18:56 +0100, Wei Liu wrote:
> > > > In ts-xen-build-prep, replace matching lenny with jessie since lenny is
> > > > not used in production anymore.
> > > 
> > > AIUI the default VG name scheme changed? Can you say how here please.
> > > 
> > 
> > Sure.
> > 
> > > Also I think in general we should aim for the "non-exceptional" case to
> > > be the default one, and use explicit matching for issues with historical
> > > versions. IOW test for wheezy and earlier and override the default (i.e.
> > > Jessie) name. The rationale is that we shouldn't need to add Stretch
> > > here too unless it also differs from Jessie.
> > > 
> > > That principal seems to apply to a some of the following patches too.
> > > 
> > > WRT "wheezy and earlier" I've been wondering about providing a function
> > > to transform a Debian code name to a sortable integer. e.g.
> > > 
> > > sub debver($) {
> > >     my %vers = qw(Lenny 50
> > >                   Squeeze 60
> > >                   Wheezy 70
> > >                   Jessie 80
> > >                   Stretch 90
> > >                   Sid 1000000);
> > >     # Unknown is > sid...
> > >     return $vers{ucfirst($_[0])) // 1000001;
> > > }
> > > 
> > > so you can do things like
> > >     if ( debver($ho->{Suite}) < debver('jessie') ) {
> > >         workaround_a_thing();
> > > 
> > 
> > You would need to test for "$ho->{Suite} == A" or "A < $ho->{Suite} < B".
> 
> if ( debver($ho->{Suite}) <= debver('squeeze') )
>     handle Squeeze or earlier
> elseif ( debver($ho->{Suite}) <= debver('wheezy') )
>     handle Wheezy
> else
>     handle Jessie onwards
> 
> I'm not sure if Perl supports
>         debver('squeeze') < debver($ho->{Suite}) <= debver('wheezy')
> like some languages do.
> 

I was just writing pseudo code.

> > And it's just another form of saying "if ($ho->{Suite} =~ /XXX/)" IMHO.
> 
> The difference is you don't need to enumerate all the names. e.g.
> "lenny|squeeze|wheezy" becomes "< debver('wheezy')"
> 
> > > Thoughts?
> > 
> > That would be nice but I doubt it would be of much use -- who
> > doesn't want the latest version in production... ;-)
> 
> People who think it is useful to continue to test Wheezy for some
> reason? People who have other constraints preventing them from upgrading
> right away? People who want to use a _newer_ version for some reason and
> want to make Stretch work while not breaking Wheezy/Jessie for everyone
> else.
> 
> If it were expensive to support I'd agree, but I don't think it is,
> especially if you arrange to never need to touch it again...
> 
> BTW, at some point I'm hoping to extend my distros flight stuff to cover
> testing the distro host packages, at which point it would naturally be
> useful to cover at least current stable and current testing, if not
> oldstable or sid.
> 

Yeah, this is a valid use case for that.

Wei.

> Ian.

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Ian Campbell]

On Wed, 2015-05-20 at 18:56 +0100, Wei Liu wrote:
> I worked a little bit on upgrading Osstest to Jessie a few months ago but
> dropped the ball. Here are the patches I wrote.

With the recent change to support submenus I find I now need the below,
I think update-grub on Jessie must include some sort of
whitespace/reindent functionality compared with Wheezy.

----8>-----

>From 98079d6eb0a93c12e5e65c999bf87b5364b300bc Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Thu, 28 May 2015 09:24:51 +0100
Subject: [PATCH] Debian: grub2: Allow submenu and menuentry items to be
 indented.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 Osstest/Debian.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 0bf108b..16ea68b 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -444,12 +444,12 @@ sub setupboot_grub2 ($$$$) {
             if (m/^function.*\{/) {
                 $entry= { StartLine => $. };
             }
-            if (m/^menuentry\s+[\'\"](.*)[\'\"].*\{\s*$/) {
+            if (m/^\s*menuentry\s+[\'\"](.*)[\'\"].*\{\s*$/) {
                 die $entry->{StartLine} if $entry;
                 $entry= { Title => $1, StartLine => $., MenuEntryPath => join ">", @offsets };
                 $offsets[$#offsets]++;
             }
-            if (m/^submenu\s+[\'\"](.*)[\'\"].*\{\s*$/) {
+            if (m/^\s*submenu\s+[\'\"](.*)[\'\"].*\{\s*$/) {
                 $submenu={ StartLine =>$., MenuEntryPath => join ">", @offsets };
                 push @offsets,(0);
             }
-- 
2.1.4

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Ian Campbell]

On Wed, 2015-05-20 at 19:04 +0100, Wei Liu wrote:
> On Wed, May 20, 2015 at 06:56:23PM +0100, Wei Liu wrote:
> > I worked a little bit on upgrading Osstest to Jessie a few months ago but
> > dropped the ball. Here are the patches I wrote.
> > 
> > With these patches I can run build jobs (Xen and kernel) with Jessie. Running
> > test jobs requires more work because now Jessie uses systemd a lot of things
> > have changed.
> 
> Actually the problems I see might not be systemd related. The problem is
> that Osstest cannot setup test host. Last time I didn't look into
> details.

I had some issues too, to do with /boot and /boot/efi not being mounted
on boot which I couldn't figure out. It was complaining that the magic
dev...by-uuid event hadn't triggered despite the fact I could see the
devices and mount them and they were tagged systemd in the udev db.

Eventually I gave in and went with this patch, essentially deferring the
problem to Stretch...

-----



commit 1bc2eff194a4c02071b75cc671058977d23d60d8
Author: Ian Campbell <ian.campbell@citrix.com>
Date:   Tue Jun 9 11:23:25 2015 +0100

    Force sysvinit with Jessie.
    
    systemd seems to have some issues with mounting /boot and /boot/efi.
    See bug references in the comments.
    
    (I suspect there ust be more to it, but I can't figure out what...)
    
    Signed-off-by: Ian Campbell <ian.campbell@citrix.com>

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index aa3648b..8458335 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -982,6 +982,15 @@ in-target update-initramfs -u -k all
 END
     }
 
+    # With systemd it seems that /boot and/or /boot/efi occasionally
+    # do not boot, Debian bugs #718190 and #719738 seem to be relevant.
+    # For the time being force sysvinit with Jessie.
+    if ( $ho->{Suite} =~ /jessie/ ) {
+	preseed_hook_command($ho, 'late_command', $sfx, <<END)
+in-target apt-get install -y sysvinit-core
+END
+    }
+
     my @extra_packages = ();
     push(@extra_packages, "u-boot-tools") if $ho->{Flags}{'need-uboot-bootscr'};
 



> 
> Wei.

Re: [PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name

[Ian Jackson]

Ian Campbell writes ("Re: [PATCH OSSTEST 3/5] Cope with Jessie's d-i vg name"):
> The difference is you don't need to enumerate all the names. e.g.
> "lenny|squeeze|wheezy" becomes "< debver('wheezy')"

I'm not sure this is worth the candle.  Because we only use this for
backward compatibility, we only ever need to enumerate old codenames.

I agrree with the rest of your comments to Wei.

Ian.

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Wei Liu]

On Wed, Jun 10, 2015 at 10:50:33AM +0100, Ian Campbell wrote:
> On Wed, 2015-05-20 at 19:04 +0100, Wei Liu wrote:
> > On Wed, May 20, 2015 at 06:56:23PM +0100, Wei Liu wrote:
> > > I worked a little bit on upgrading Osstest to Jessie a few months ago but
> > > dropped the ball. Here are the patches I wrote.
> > > 
> > > With these patches I can run build jobs (Xen and kernel) with Jessie. Running
> > > test jobs requires more work because now Jessie uses systemd a lot of things
> > > have changed.
> > 
> > Actually the problems I see might not be systemd related. The problem is
> > that Osstest cannot setup test host. Last time I didn't look into
> > details.
> 
> I had some issues too, to do with /boot and /boot/efi not being mounted
> on boot which I couldn't figure out. It was complaining that the magic
> dev...by-uuid event hadn't triggered despite the fact I could see the
> devices and mount them and they were tagged systemd in the udev db.
> 
> Eventually I gave in and went with this patch, essentially deferring the
> problem to Stretch...
> 

The same problem also manifests when booting DomU. However booting DomU
with stock Jessie kernel works fine.

I now think this issue might be related to the 3.14 kernel we use for
both Dom0 and DomU.

Wei.

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Wei Liu]

On Fri, Jun 12, 2015 at 09:43:11PM +0100, Wei Liu wrote:
> On Wed, Jun 10, 2015 at 10:50:33AM +0100, Ian Campbell wrote:
> > On Wed, 2015-05-20 at 19:04 +0100, Wei Liu wrote:
> > > On Wed, May 20, 2015 at 06:56:23PM +0100, Wei Liu wrote:
> > > > I worked a little bit on upgrading Osstest to Jessie a few months ago but
> > > > dropped the ball. Here are the patches I wrote.
> > > > 
> > > > With these patches I can run build jobs (Xen and kernel) with Jessie. Running
> > > > test jobs requires more work because now Jessie uses systemd a lot of things
> > > > have changed.
> > > 
> > > Actually the problems I see might not be systemd related. The problem is
> > > that Osstest cannot setup test host. Last time I didn't look into
> > > details.
> > 
> > I had some issues too, to do with /boot and /boot/efi not being mounted
> > on boot which I couldn't figure out. It was complaining that the magic
> > dev...by-uuid event hadn't triggered despite the fact I could see the
> > devices and mount them and they were tagged systemd in the udev db.
> > 
> > Eventually I gave in and went with this patch, essentially deferring the
> > problem to Stretch...
> > 
> 
> The same problem also manifests when booting DomU. However booting DomU
> with stock Jessie kernel works fine.
> 
> I now think this issue might be related to the 3.14 kernel we use for
> both Dom0 and DomU.
> 

I tracked down the issue.

---8<---
>From 756cb022b4273c75878d4b446ca19b69661ebcb6 Mon Sep 17 00:00:00 2001
From: Wei Liu <wei.liu2@citrix.com>
Date: Fri, 12 Jun 2015 23:50:55 +0100
Subject: [PATCH OSSTEST] ts-kernel-build: enable CONFIG_FHANDLE
Cc: ian.campbell@citrix.com, ian.jackson@eu.citrix.com

According to Debian bug #764539, systemd needs this kernel feature to be
able to mount disks. Complete list of required kernel features can be
found in /usr/share/doc/systemd/README.gz.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 ts-kernel-build | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/ts-kernel-build b/ts-kernel-build
index e7cbdc1..438be6d 100755
--- a/ts-kernel-build
+++ b/ts-kernel-build
@@ -525,6 +525,8 @@ setopt CONFIG_FUSION_LOGGING n
 
 setopt CONFIG_BLK_CPQ_CISS_DA m
 
+setopt CONFIG_FHANDLE y
+
 case ${XEN_TARGET_ARCH} in
     x86_32) setopt CONFIG_64BIT n ;;
     x86_64)
-- 
1.9.1

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Ian Campbell]

On Mon, 2015-06-15 at 09:29 +0100, Wei Liu wrote:
> On Fri, Jun 12, 2015 at 09:43:11PM +0100, Wei Liu wrote:
> > On Wed, Jun 10, 2015 at 10:50:33AM +0100, Ian Campbell wrote:
> > > On Wed, 2015-05-20 at 19:04 +0100, Wei Liu wrote:
> > > > On Wed, May 20, 2015 at 06:56:23PM +0100, Wei Liu wrote:
> > > > > I worked a little bit on upgrading Osstest to Jessie a few months ago but
> > > > > dropped the ball. Here are the patches I wrote.
> > > > > 
> > > > > With these patches I can run build jobs (Xen and kernel) with Jessie. Running
> > > > > test jobs requires more work because now Jessie uses systemd a lot of things
> > > > > have changed.
> > > > 
> > > > Actually the problems I see might not be systemd related. The problem is
> > > > that Osstest cannot setup test host. Last time I didn't look into
> > > > details.
> > > 
> > > I had some issues too, to do with /boot and /boot/efi not being mounted
> > > on boot which I couldn't figure out. It was complaining that the magic
> > > dev...by-uuid event hadn't triggered despite the fact I could see the
> > > devices and mount them and they were tagged systemd in the udev db.
> > > 
> > > Eventually I gave in and went with this patch, essentially deferring the
> > > problem to Stretch...
> > > 
> > 
> > The same problem also manifests when booting DomU. However booting DomU
> > with stock Jessie kernel works fine.
> > 
> > I now think this issue might be related to the 3.14 kernel we use for
> > both Dom0 and DomU.

I was about to say: I've seen it with one or both of 3.18 and 4.1 too
(since I am using ARM and need newer).
> > 
> 
> I tracked down the issue.

Well spotted!

> 
> ---8<---
> From 756cb022b4273c75878d4b446ca19b69661ebcb6 Mon Sep 17 00:00:00 2001
> From: Wei Liu <wei.liu2@citrix.com>
> Date: Fri, 12 Jun 2015 23:50:55 +0100
> Subject: [PATCH OSSTEST] ts-kernel-build: enable CONFIG_FHANDLE
> Cc: ian.campbell@citrix.com, ian.jackson@eu.citrix.com
> 
> According to Debian bug #764539, systemd needs this kernel feature to be
> able to mount disks. Complete list of required kernel features can be
> found in /usr/share/doc/systemd/README.gz.

A useful document indeed, I somehow didn't manage to find any of these
reference despite my searching.

> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>  ts-kernel-build | 2 ++
>  1 file changed, 2 insertions(+)
> 
> diff --git a/ts-kernel-build b/ts-kernel-build
> index e7cbdc1..438be6d 100755
> --- a/ts-kernel-build
> +++ b/ts-kernel-build
> @@ -525,6 +525,8 @@ setopt CONFIG_FUSION_LOGGING n
>  
>  setopt CONFIG_BLK_CPQ_CISS_DA m
>  
> +setopt CONFIG_FHANDLE y
> +
>  case ${XEN_TARGET_ARCH} in
>      x86_32) setopt CONFIG_64BIT n ;;
>      x86_64)

Re: [PATCH OSSTEST 0/5] Debian Jessie patches

[Ian Campbell]

On Mon, 2015-06-15 at 09:37 +0100, Ian Campbell wrote:
>  
> > ---8<---
> > From 756cb022b4273c75878d4b446ca19b69661ebcb6 Mon Sep 17 00:00:00 2001
> > From: Wei Liu <wei.liu2@citrix.com>
> > Date: Fri, 12 Jun 2015 23:50:55 +0100
> > Subject: [PATCH OSSTEST] ts-kernel-build: enable CONFIG_FHANDLE
> > Cc: ian.campbell@citrix.com, ian.jackson@eu.citrix.com
> > 
> > According to Debian bug #764539, systemd needs this kernel feature to be
> > able to mount disks. Complete list of required kernel features can be
> > found in /usr/share/doc/systemd/README.gz.
> 
> A useful document indeed, I somehow didn't manage to find any of these
> reference despite my searching.

What I really meant was:
Acked-by: Ian Campbell <ian.campbell@citrix.com>

(Also, WTF is a _syscall_ a Kconfig option!)

> 
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> >  ts-kernel-build | 2 ++
> >  1 file changed, 2 insertions(+)
> > 
> > diff --git a/ts-kernel-build b/ts-kernel-build
> > index e7cbdc1..438be6d 100755
> > --- a/ts-kernel-build
> > +++ b/ts-kernel-build
> > @@ -525,6 +525,8 @@ setopt CONFIG_FUSION_LOGGING n
> >  
> >  setopt CONFIG_BLK_CPQ_CISS_DA m
> >  
> > +setopt CONFIG_FHANDLE y
> > +
> >  case ${XEN_TARGET_ARCH} in
> >      x86_32) setopt CONFIG_64BIT n ;;
> >      x86_64)
>