[PATCH OSSTEST 0/3] fixes to erase-other-disks d-i hook

[PATCH OSSTEST 0/3] fixes to erase-other-disks d-i hook

[Ian Campbell]

This series of patches fix host installation on disks with multiple disks,
which has been failing since the update to Jessie. This has been affecting
only the *-mite and *-frog machines in the Citrix instance in Cambridge so
far, but second disks are on order for several machines in the production
colo.

#2 is the actual fix, #1 is some logging improvements and #3 is really just
belt and braces.

Since this has been blocking osstest updates in the Cambridge instance I'd
like to push this one to the head of the queue for application (to the colo
pretest).

Ian.

[PATCH OSSTEST 1/3] Debian: erase-other-disks: add a log() helper

[Ian Campbell]

Writing it out each time is too verbose.

At the same time log the set of devices present before and after each
batch of erasing, with a udev settle before the second to ensure any
changes to /dev have happened.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 Osstest/Debian.pm | 17 +++++++++++++----
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 76171c0..cf3486b 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1033,22 +1033,29 @@ if test -f \$stamp; then
     exit 0
 fi
 >\$stamp
-logger -t osstest-erase-other-disks-\$\$ "Running..."
+
+log () {
+    logger -t osstest-erase-other-disks-\$\$ "\$\@"
+}
+
+log "Running..."
+
 zero () {
     if test -b \$dev; then
-        logger -t osstest-erase-other-disks-\$\$ "Erasing \$dev"
+        log "Erasing \$dev"
         dd if=/dev/zero of=\$dev count=64 ||:
         if ! test -b \$dev; then
-            logger -t osstest-erase-other-disks-\$\$ "\$dev is no longer a block device!"
+            log "\$dev is no longer a block device!"
             exit 1
         fi
     else
-        logger -t osstest-erase-other-disks-\$\$ "\$dev does not exist or is not a block device."
+        log "\$dev does not exist or is not a block device."
     fi
 }
 
 udevadm settle
 for sd in sd hd; do
+    log "\${sd} devices present before: `echo /dev/\${sd}*`"
     for b in a b c d e f; do
         dev=/dev/\${sd}\${b}
         zero
@@ -1056,6 +1063,8 @@ for sd in sd hd; do
     for dev in /dev/\${sd}a[0-9]; do
         zero
     done
+    udevadm settle
+    log "\${sd} devices present after: `echo /dev/\${sd}*`"
 done
 for dev in ${disk}*; do
     zero
-- 
2.6.1

[PATCH OSSTEST 2/3] Debian: erase-other-disks: erase partitions first

[Ian Campbell]

It seems that when sdX is zeroed there is some chance that sdX[0-9]
will disappear before we get to them.

When partman comes along and recreates the partitions it is likely
that they will occupy the same disk space as before (since d-i's
autopartition is deterministic), meaning that LVM will find the old
PV headers again.

This is in particular problematic on multi disk systems where we end
up with an LV spanning sda5 and sdb. sdb is successfully erased here
but sda5 is not, however LVM will still find the LV with missing PV,
which is sufficient to trigger partman-lvm's checks for erasing
devices which weren't explicitly listed, resulting in:

    !! ERROR: Unable to automatically remove LVM data

    Because the volume group(s) on the selected device also consist of physical
    volumes on other devices, it is not considered safe to remove its LVM data
    automatically. If you wish to use this device for partitioning, please remove
    its LVM data first.

which cannot be preseeded around.

If the autopartitioning is not deterministic (as might be the case
when installing a different version of Debian to last time) then
going from layout A -> B -> A' risks B (by chance) not destroying the
headers created by A, meaning that A' will find them and suffer again
from the problem above. This is handled via the use of
ts-host-install-twice which will cause A' to run twice, i.e. A -> B
-> (A' -> A''). In this case A' will fail as above, but A'' will
startup seeing the partition layout put in place by A' (which matches
A) and erase those partitions, leading to success later on.

Also erase partitions for all sd/hd? not just sda+hda.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 Osstest/Debian.pm | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index cf3486b..20f3de7 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1057,12 +1057,13 @@ udevadm settle
 for sd in sd hd; do
     log "\${sd} devices present before: `echo /dev/\${sd}*`"
     for b in a b c d e f; do
+        for dev in /dev/\${sd}\${b}[0-9]; do
+            zero
+        done
+
         dev=/dev/\${sd}\${b}
         zero
     done
-    for dev in /dev/\${sd}a[0-9]; do
-        zero
-    done
     udevadm settle
     log "\${sd} devices present after: `echo /dev/\${sd}*`"
 done
-- 
2.6.1

[PATCH OSSTEST 3/3] Debian: erase-other-disks: rescan partition tables after erasing whole disk

[Ian Campbell]

This appears to happen anyway, but force it to be sure.

Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
 Osstest/Debian.pm | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 20f3de7..e7b5538 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1053,6 +1053,14 @@ zero () {
     fi
 }
 
+rescan () {
+    if ! test -b \${dev}; then
+	return
+    fi
+    log "Rescaning partition table on \${dev}"
+    echo 1 > /sys/block/\${dev#/dev/}/device/rescan
+}
+
 udevadm settle
 for sd in sd hd; do
     log "\${sd} devices present before: `echo /dev/\${sd}*`"
@@ -1063,6 +1071,7 @@ for sd in sd hd; do
 
         dev=/dev/\${sd}\${b}
         zero
+        rescan
     done
     udevadm settle
     log "\${sd} devices present after: `echo /dev/\${sd}*`"
-- 
2.6.1

Re: [PATCH OSSTEST 1/3] Debian: erase-other-disks: add a log() helper

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST 1/3] Debian: erase-other-disks: add a log() helper"):
> Writing it out each time is too verbose.
> 
> At the same time log the set of devices present before and after each
> batch of erasing, with a udev settle before the second to ensure any
> changes to /dev have happened.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

Re: [PATCH OSSTEST 2/3] Debian: erase-other-disks: erase partitions first

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST 2/3] Debian: erase-other-disks: erase partitions first"):
> It seems that when sdX is zeroed there is some chance that sdX[0-9]
> will disappear before we get to them.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>

We're really pushing the commit-message-to-code ratio here :-).

Ian.

Re: [PATCH OSSTEST 3/3] Debian: erase-other-disks: rescan partition tables after erasing whole disk

[Ian Jackson]

Ian Campbell writes ("[PATCH OSSTEST 3/3] Debian: erase-other-disks: rescan partition tables after erasing whole disk"):
> This appears to happen anyway, but force it to be sure.

Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>