[PATCH 1/6] Revert "ts-xen-build-prep: Run the backgrounded resize2fs with ionice"

[PATCH 1/6] Revert "ts-xen-build-prep: Run the backgrounded resize2fs with ionice"

[Ian Jackson]

This did not help.  We are going to do something different.

This reverts commit ee31a32af1fc9a38ad08b4a36ee69e1c58ea0828.
---
 ts-xen-build-prep |    5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 05a7857..0e68cd3 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -159,10 +159,7 @@ sub lvextend1 ($$$) {
     if (!$background) {
 	target_cmd_root($ho, $cmd, $timeout);
     } else {
-	target_cmd_root($ho,
-			"ionice -cBest-Effort -n7 ".
-			"$cmd >>resizefs.log 2>&1 </dev/null &",
-			30);
+	target_cmd_root($ho, "$cmd >>resizefs.log 2>&1 </dev/null &", 30);
     }
 }
 
-- 
1.7.10.4

[PATCH 2/6] ts-xen-build-prep: Abolish "background" parameter to lvextend

[Ian Jackson]

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 ts-xen-build-prep |   12 ++++--------
 1 file changed, 4 insertions(+), 8 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 0e68cd3..6443351 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -65,11 +65,11 @@ sub lvextend_stage1 () {
         fi
 END
 
-    lvextend1('disk extend initial',       30, 0);
+    lvextend1('disk extend initial',       300);
 }
 
 sub lvextend_stage2 () {
-    lvextend1('disk extend background', undef, 1);
+    lvextend1('disk extend subsequent',  undef);
 }
 
 sub vginfo () {
@@ -80,7 +80,7 @@ sub vginfo () {
 }
 
 sub lvextend1 ($$$) {
-    my ($what, $max_more_gb, $background)  = @_;
+    my ($what, $max_more_gb)  = @_;
 
     my $lv= "/dev/$vg/root";
 
@@ -156,11 +156,7 @@ sub lvextend1 ($$$) {
     my $timeout = 2000 + int($pe_size * 0.000003 * $more_pe);
     logm("$what: ${pe_size}k x $more_pe (timeout=$timeout)");
     my $cmd = "resize2fs $lv";
-    if (!$background) {
-	target_cmd_root($ho, $cmd, $timeout);
-    } else {
-	target_cmd_root($ho, "$cmd >>resizefs.log 2>&1 </dev/null &", 30);
-    }
+    target_cmd_root($ho, $cmd, $timeout);
 }
 
 sub prep () {
-- 
1.7.10.4

Re: [PATCH 2/6] ts-xen-build-prep: Abolish "background" parameter to lvextend

[Ian Campbell]

On Wed, 2014-06-11 at 21:47 +0100, Ian Jackson wrote:
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  ts-xen-build-prep |   12 ++++--------
>  1 file changed, 4 insertions(+), 8 deletions(-)
> 
> diff --git a/ts-xen-build-prep b/ts-xen-build-prep
> index 0e68cd3..6443351 100755
> --- a/ts-xen-build-prep
> +++ b/ts-xen-build-prep
> @@ -65,11 +65,11 @@ sub lvextend_stage1 () {
>          fi
>  END
>  
> -    lvextend1('disk extend initial',       30, 0);
> +    lvextend1('disk extend initial',       300);

Did you also intentionally increase max_more_gb here?

>  }
>  
>  sub lvextend_stage2 () {
> -    lvextend1('disk extend background', undef, 1);
> +    lvextend1('disk extend subsequent',  undef);

Nit: Extra space

[PATCH 3/6] TestSupport: break out lv_dev_mapper

[Ian Jackson]

Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
---
 Osstest/TestSupport.pm |   17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 93c8fcf..593f06c 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -59,7 +59,8 @@ BEGIN {
                       target_run_apt
                       target_install_packages target_install_packages_norec
                       target_jobdir target_extract_jobdistpath_subdir
-                      target_extract_jobdistpath target_guest_lv_name
+                      target_extract_jobdistpath
+                      lv_dev_mapper target_guest_lv_name
 
                       poll_loop tcpconnect await_tcp
                       contents_make_cpio file_simple_write_contents
@@ -639,13 +640,17 @@ sub poll_loop ($$$&) {
     logm("$what: ok. (${waited}s)");
 }
 
-sub target_guest_lv_name($$) {
-    my ($ho, $lv) = @_;
-
-    my $vg = "$ho->{Name}";
-    # Dashes are escaped in the VG name
+sub lv_dev_mapper ($$) {
+    my ($vg,$lv) = @_;
     $vg =~ s/-/--/g;
+    $lv =~ s/-/--/g;
+    # Dashes are doubled in the VG and LV names
     return "/dev/mapper/$vg-$lv";
+}    
+
+sub target_guest_lv_name($$) {
+    my ($ho, $lv) = @_;
+    return lv_dev_mapper("$ho->{Name}", $lv);
 }
 
 #---------- dhcp watching ----------
-- 
1.7.10.4

Re: [PATCH 3/6] TestSupport: break out lv_dev_mapper

[Ian Campbell]

On Wed, 2014-06-11 at 21:47 +0100, Ian Jackson wrote:
> Signed-off-by: Ian Jackson <ijackson@chiark.greenend.org.uk>
> ---
>  Osstest/TestSupport.pm |   17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
> index 93c8fcf..593f06c 100644
> --- a/Osstest/TestSupport.pm
> +++ b/Osstest/TestSupport.pm
> @@ -59,7 +59,8 @@ BEGIN {
>                        target_run_apt
>                        target_install_packages target_install_packages_norec
>                        target_jobdir target_extract_jobdistpath_subdir
> -                      target_extract_jobdistpath target_guest_lv_name
> +                      target_extract_jobdistpath
> +                      lv_dev_mapper target_guest_lv_name
>  
>                        poll_loop tcpconnect await_tcp
>                        contents_make_cpio file_simple_write_contents
> @@ -639,13 +640,17 @@ sub poll_loop ($$$&) {
>      logm("$what: ok. (${waited}s)");
>  }
>  
> -sub target_guest_lv_name($$) {
> -    my ($ho, $lv) = @_;
> -
> -    my $vg = "$ho->{Name}";
> -    # Dashes are escaped in the VG name
> +sub lv_dev_mapper ($$) {
> +    my ($vg,$lv) = @_;
>      $vg =~ s/-/--/g;
> +    $lv =~ s/-/--/g;
> +    # Dashes are doubled in the VG and LV names

Doesn't this belong before the two substitutions?

>      return "/dev/mapper/$vg-$lv";
> +}    
> +
> +sub target_guest_lv_name($$) {
> +    my ($ho, $lv) = @_;
> +    return lv_dev_mapper("$ho->{Name}", $lv);
>  }
>  
>  #---------- dhcp watching ----------

[PATCH 4/6] ts-xen-build-prep: Break out determine_vg_lv

[Ian Jackson]

And make $lv a global.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-xen-build-prep |   11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 6443351..d600285 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -28,8 +28,9 @@ $whhost ||= 'host';
 our $ho= selecthost($whhost);
 exit 0 if $ho->{SharedReady};
 
-our ($vg);
+our ($vg,$lv);
 
+our $lvleaf = 'root';
 our $pe_size;
 our $extended_pes = 0;
 
@@ -50,12 +51,15 @@ sub overall_limit_pe ($) {
     $extended_pes += $$pevar;
 }
 
-sub lvextend_stage1 () {
+sub determine_vg_lv () {
     $vg=
         $ho->{Suite} =~ m/lenny/
         ? "$ho->{Name}.$c{TestHostDomain}"
         : $ho->{Name};
+    $lv = "/dev/$vg/$lvleaf";
+}
 
+sub lvextend_stage1 () {
     target_cmd_root($ho, <<END);
         set -ex; if ! test -f /root/swap_osstest_enabled; then
             lvcreate -L 10G -n swap_osstest_build $vg ||:
@@ -82,8 +86,6 @@ sub vginfo () {
 sub lvextend1 ($$$) {
     my ($what, $max_more_gb)  = @_;
 
-    my $lv= "/dev/$vg/root";
-
     my @vginfo = vginfo();
     $pe_size= $vginfo[12];
 
@@ -199,6 +201,7 @@ sub ccache_setup () {
 }
 
 if (!$ho->{Flags}{'no-reinstall'}) {
+    determine_vg_lv();
     lvextend_stage1();
     prep();
     ccache_setup();
-- 
1.7.10.4

[PATCH 5/6] ts-xen-build-prep: mkfs a new /home/osstest, don't resize2fs

[Ian Jackson]

Online resize is 40x slower than mkfs.  It appears that the
backgrounded resize2fs can starve build tasks of IO bandwidth.

So instead, use mkfs to make a new filesystem for /home/osstest.
We use rsync to copy in the old contents.

For convenience of (a) review (b) possible reversion, we keep (for
now) the lvextend machinery.  So we create a new 1-extent LV for the
lvextend machinery to work on.

But we don't call resize2fs when we extend it, because now it doesn't
have a fs on it yet.  We make the filesystem later.

We move the ccache_setup until after this is done because it's a bit
pointless to put things in the to-be-removed /home/osstest when they
could be put in the new one after it had been set up.

We take slight care to make the rune slightly idempotent: if it
completed successfully we detect this and do not run it again.  But if
it didn't, things may be messed up and running it again is unlikely to
help and may make things worse.

I have tested this on rice-weevil and the whole new target command
(including rsync, mkfs, mount etc.) takes 126s.

Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
---
 ts-xen-build-prep |   46 ++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 42 insertions(+), 4 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index d600285..ab1346d 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -30,7 +30,7 @@ exit 0 if $ho->{SharedReady};
 
 our ($vg,$lv);
 
-our $lvleaf = 'root';
+our $lvleaf = 'osstest_home';
 our $pe_size;
 our $extended_pes = 0;
 
@@ -83,6 +83,11 @@ sub vginfo () {
     return @vginfo;
 }
 
+sub lvcreate () {
+    target_cmd_output_root($ho,
+			   "lvdisplay $lv || lvcreate -l 1 -n $lvleaf $vg");
+}
+
 sub lvextend1 ($$$) {
     my ($what, $max_more_gb)  = @_;
 
@@ -157,8 +162,39 @@ sub lvextend1 ($$$) {
 
     my $timeout = 2000 + int($pe_size * 0.000003 * $more_pe);
     logm("$what: ${pe_size}k x $more_pe (timeout=$timeout)");
-    my $cmd = "resize2fs $lv";
-    target_cmd_root($ho, $cmd, $timeout);
+}
+
+sub replace_home () {
+    my $dir = '/home/osstest';
+    my $mapper = lv_dev_mapper($vg,$lvleaf);
+    my ($fstype,@opts) = qw(ext3 -m 0 -O sparse_super);
+    target_cmd_root($ho, <<END, 1000);
+        set -ex
+	if mount | sed -e 's/^[^ ].* on //; s/ .*//' | grep -F '$dir'; then
+	    exit 0
+	fi
+	mkfs -t $fstype @opts $lv
+        mount $lv /mnt
+	rsync -aHx --numeric-ids $dir/. /mnt/.
+	rm -rf $dir
+	mkdir -m 2700 $dir
+	echo '$mapper $dir $fstype defaults 0 0' >>/etc/fstab
+	umount /mnt
+	mount $dir
+END
+
+        # for convenience, here is a small scriptlet to undo this:
+        <<'END';
+#!/bin/sh
+set -ex
+cd /home
+rm -rf osstest.new
+rsync -aH --numeric-ids osstest/. osstest.new
+umount /home/osstest
+rmdir osstest
+mv osstest.new osstest
+lvremove -f /dev/`uname -n`/osstest_home
+END
 }
 
 sub prep () {
@@ -202,10 +238,12 @@ sub ccache_setup () {
 
 if (!$ho->{Flags}{'no-reinstall'}) {
     determine_vg_lv();
+    lvcreate();
     lvextend_stage1();
     prep();
-    ccache_setup();
     lvextend_stage2();
+    replace_home();
+    ccache_setup();
 }
 $mjobdb->jobdb_resource_shared_mark_ready
    ($ho->{Ident}, $ho->{Name}, "build-".$ho->{Suite}."-".$r{arch});
-- 
1.7.10.4

Re: [PATCH 5/6] ts-xen-build-prep: mkfs a new /home/osstest, don't resize2fs

[Ian Campbell]

On Wed, 2014-06-11 at 21:47 +0100, Ian Jackson wrote:
> Online resize is 40x slower than mkfs.  It appears that the
> backgrounded resize2fs can starve build tasks of IO bandwidth.
> 
> So instead, use mkfs to make a new filesystem for /home/osstest.
> We use rsync to copy in the old contents.
> 
> For convenience of (a) review (b) possible reversion, we keep (for
> now) the lvextend machinery.  So we create a new 1-extent LV for the
> lvextend machinery to work on.
> 
> But we don't call resize2fs when we extend it, because now it doesn't
> have a fs on it yet.  We make the filesystem later.
> 
> We move the ccache_setup until after this is done because it's a bit
> pointless to put things in the to-be-removed /home/osstest when they
> could be put in the new one after it had been set up.
> 
> We take slight care to make the rune slightly idempotent: if it
> completed successfully we detect this and do not run it again.  But if
> it didn't, things may be messed up and running it again is unlikely to
> help and may make things worse.
> 
> I have tested this on rice-weevil and the whole new target command
> (including rsync, mkfs, mount etc.) takes 126s.
> 
> Signed-off-by: Ian Jackson <ian.jackson@eu.citrix.com>
> ---
>  ts-xen-build-prep |   46 ++++++++++++++++++++++++++++++++++++++++++----
>  1 file changed, 42 insertions(+), 4 deletions(-)
> 
> diff --git a/ts-xen-build-prep b/ts-xen-build-prep
> index d600285..ab1346d 100755
> --- a/ts-xen-build-prep
> +++ b/ts-xen-build-prep
> @@ -30,7 +30,7 @@ exit 0 if $ho->{SharedReady};
>  
>  our ($vg,$lv);
>  
> -our $lvleaf = 'root';
> +our $lvleaf = 'osstest_home';
>  our $pe_size;
>  our $extended_pes = 0;
>  
> @@ -83,6 +83,11 @@ sub vginfo () {
>      return @vginfo;
>  }
>  
> +sub lvcreate () {
> +    target_cmd_output_root($ho,
> +			   "lvdisplay $lv || lvcreate -l 1 -n $lvleaf $vg");
> +}
> +
>  sub lvextend1 ($$$) {
>      my ($what, $max_more_gb)  = @_;
>  
> @@ -157,8 +162,39 @@ sub lvextend1 ($$$) {
>  
>      my $timeout = 2000 + int($pe_size * 0.000003 * $more_pe);
>      logm("$what: ${pe_size}k x $more_pe (timeout=$timeout)");
> -    my $cmd = "resize2fs $lv";
> -    target_cmd_root($ho, $cmd, $timeout);
> +}
> +
> +sub replace_home () {
> +    my $dir = '/home/osstest';
> +    my $mapper = lv_dev_mapper($vg,$lvleaf);
> +    my ($fstype,@opts) = qw(ext3 -m 0 -O sparse_super);
> +    target_cmd_root($ho, <<END, 1000);
> +        set -ex
> +	if mount | sed -e 's/^[^ ].* on //; s/ .*//' | grep -F '$dir'; then
> +	    exit 0
> +	fi
> +	mkfs -t $fstype @opts $lv
> +        mount $lv /mnt

Some mixture hard and soft tabs here.

> +	rsync -aHx --numeric-ids $dir/. /mnt/.
> +	rm -rf $dir
> +	mkdir -m 2700 $dir
> +	echo '$mapper $dir $fstype defaults 0 0' >>/etc/fstab
> +	umount /mnt
> +	mount $dir
> +END
> +
> +        # for convenience, here is a small scriptlet to undo this:
> +        <<'END';
> +#!/bin/sh
> +set -ex
> +cd /home
> +rm -rf osstest.new
> +rsync -aH --numeric-ids osstest/. osstest.new

This made me wonder why not move the original one to osstest.foo instead
of rm -rf it? Doesn't really matter of course.

Ian.

[PATCH 6/6] Revert "ts-logs-capture: collect resize2fs.log"

[Ian Jackson]

This is no longer generated.

This reverts commit 9f299b936d1812bfd958d2771676a8de3cf9c060.
---
 ts-logs-capture |    2 --
 1 file changed, 2 deletions(-)

diff --git a/ts-logs-capture b/ts-logs-capture
index 3ccfc00..5fa7b82 100755
--- a/ts-logs-capture
+++ b/ts-logs-capture
@@ -124,8 +124,6 @@ sub fetch_logs_host_guests () {
 
                   /var/log/xen-tools/*
 
-                  /root/resizefs.log
-
                   /boot/config*
 
                   /etc/xen/*
-- 
1.7.10.4

Re: [PATCH 1/6] Revert "ts-xen-build-prep: Run the backgrounded resize2fs with ionice"

[Ian Campbell]

On Wed, 2014-06-11 at 21:47 +0100, Ian Jackson wrote:

FYI this series looked mostly fine. I had a few minor quibbles on
individual patches.

> This did not help.  We are going to do something different.
> 
> This reverts commit ee31a32af1fc9a38ad08b4a36ee69e1c58ea0828.
> ---
>  ts-xen-build-prep |    5 +----
>  1 file changed, 1 insertion(+), 4 deletions(-)
> 
> diff --git a/ts-xen-build-prep b/ts-xen-build-prep
> index 05a7857..0e68cd3 100755
> --- a/ts-xen-build-prep
> +++ b/ts-xen-build-prep
> @@ -159,10 +159,7 @@ sub lvextend1 ($$$) {
>      if (!$background) {
>  	target_cmd_root($ho, $cmd, $timeout);
>      } else {
> -	target_cmd_root($ho,
> -			"ionice -cBest-Effort -n7 ".
> -			"$cmd >>resizefs.log 2>&1 </dev/null &",
> -			30);
> +	target_cmd_root($ho, "$cmd >>resizefs.log 2>&1 </dev/null &", 30);
>      }
>  }
>