[OSSTEST PATCH 00/36] Switch to Debian Bookworm

[OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Anthony PERARD]

Patch series available in this git branch:
https://xenbits.xen.org/git-http/people/aperard/osstest.git br.bookworm-v1

Hi,

I intend to push this series in two waves.

First, push up to commit "Temporally switch "qemu-mainline" branch to
Bookworm". This is to test that osstest still works fine if we need to use
"buster" for a branch. Also upstream QEMU doesn't build on buster anymore, so
I've included a commit to use bookworm for it.

Second, push the remaning two patches, at least a week later, which will switch
the default debian suite.

I did run many tests, but hopefully nothing breaks.

Cheers,

Anthony PERARD (36):
  production-config: Add bookworm debian install media filename
  ts-xen-build-prep: Only force git protocol v2 on buster
  mgi-common: Fix fetch_debian_package error message
  mg-debian-installer-update: Download non-free firmware from new repo.
  ts-host-install: fix ntp.conf path on bookworm
  ts-host-install: fix ntp server setting
  ts-host-install: Restart ntp service
  preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI
    installation
  preseed_create: osstest-erase-other-disks: workaround creating
    /dev/sdXD files
  preseed_create: Workaround fail grub-install on arndale
  ts-host-install,preseed_create: Do lvm vgextend at install time
  di_installcmdline_core: Add link_wait_timeout to install cmdline
  Disable persistent net generator on Bookworm
  preseed_base, ts-host-install: Change NIC NamePolicy to "mac"
  ts-xen-build-prep: Change package selection for Bookworm
  bl_getmenu_open: Read grub.cfg as root
  target_editfile: Use the same user to retrieve and send
  ts-xen-install: remove "libc6-xen" package installation
  overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
  overlay-bookworm: 20_linux_xen: Fix XSM entries generation
  ts-xtf-install: Install python symlink
  setupboot_grub2: Parse arm64 uefi grub verbes
  bookworm: Extend ARM clock workaround
  ts-nested-setup, setup l1 lvm volume groupe in guest
  ts-leak-check: add new name for udevd workers
  ts-debian-hvm-install: Allow udev failure in install media
  ts-debian-fixup: Fix nic names for bookworm
  ts-debian-install: keep avoiding to use pygrub
  ts-debian-hvm-install: Increase min guest ram size
  bookworm: Extend guest bootloader workaround
  ts-debian-*-install: Replace dots in hostnames by dashs
  ts-xen-install: Fix bridge setup, ask to copy MAC addr
  make-flight: Keep using buster for L2 guest in nested tests
  Temporally switch "qemu-mainline" branch to Bookworm
  Switch to Debian Bookworm as default suite
  make-hosts-flight: default to bookworm

 Osstest.pm                               |   2 +-
 Osstest/Debian.pm                        |  92 +++++-
 Osstest/TestSupport.pm                   |   4 +-
 make-flight                              |  16 +-
 make-hosts-flight                        |   2 +-
 mfi-common                               |   5 +-
 mg-debian-installer-update               |   9 +-
 mgi-common                               |   2 +-
 overlay-bookworm/etc/grub.d/20_linux_xen | 379 +++++++++++++++++++++++
 production-config                        |   2 +
 ts-debian-di-install                     |   4 +-
 ts-debian-fixup                          |  15 +
 ts-debian-hvm-install                    |  21 +-
 ts-debian-install                        |  17 +-
 ts-host-install                          |  42 ++-
 ts-leak-check                            |   1 +
 ts-nested-setup                          |  10 +-
 ts-xen-build-prep                        |  10 +-
 ts-xen-install                           |   5 +-
 ts-xtf-install                           |   6 +
 20 files changed, 591 insertions(+), 53 deletions(-)
 create mode 100755 overlay-bookworm/etc/grub.d/20_linux_xen

-- 
Anthony PERARD

[OSSTEST PATCH 01/36] production-config: Add bookworm debian install media filename

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 production-config | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/production-config b/production-config
index 2c44805c..6345c40c 100644
--- a/production-config
+++ b/production-config
@@ -101,6 +101,8 @@ DebianImageVersion_jessie 8.2.0
 DebianImageVersion_stretch 9.4.0
 DebianImageFile_buster_amd64 debian-10.12.0-amd64-xfce-CD-1.iso
 DebianImageFile_buster_i386 debian-10.12.0-i386-xfce-CD-1.iso
+DebianImageFile_bookworm_amd64 debian-12.5.0-amd64-netinst.iso
+DebianImageFile_bookworm_i386 debian-12.5.0-i386-netinst.iso
 
 
 # Update with ./mg-netgrub-loader-update
-- 
Anthony PERARD

[OSSTEST PATCH 02/36] ts-xen-build-prep: Only force git protocol v2 on buster

[Anthony PERARD]

Newer version of Debian and thus git would use this automatically, no
need to force it.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index f0e087aa..0dded9b2 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -3257,7 +3257,7 @@ END
 
     # Adding mandadory use of Git protocol v2
     # necessary on buster when building QEMU v8.1
-    $gitcfg .= <<END;
+    $gitcfg .= <<END if $ho->{Suite} =~ m/buster/;
 [protocol]
 	version = 2
 END
-- 
Anthony PERARD

Re: [OSSTEST PATCH 02/36] ts-xen-build-prep: Only force git protocol v2 on buster

[Roger Pau Monné]

On Mon, Mar 18, 2024 at 04:55:11PM +0000, Anthony PERARD wrote:
> Newer version of Debian and thus git would use this automatically, no
> need to force it.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  Osstest/TestSupport.pm | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
> index f0e087aa..0dded9b2 100644
> --- a/Osstest/TestSupport.pm
> +++ b/Osstest/TestSupport.pm
> @@ -3257,7 +3257,7 @@ END
>  
>      # Adding mandadory use of Git protocol v2
>      # necessary on buster when building QEMU v8.1
> -    $gitcfg .= <<END;
> +    $gitcfg .= <<END if $ho->{Suite} =~ m/buster/;
>  [protocol]
>  	version = 2

Do we need to limit ourselves to version 2 for the git cache stuff, or
that doesn't matter?

I'm wondering whether it case version 3 of the protocol appears we
would be in trouble by not having version = 2 in the config file.

Thanks, Roger.

Re: [OSSTEST PATCH 02/36] ts-xen-build-prep: Only force git protocol v2 on buster

[Anthony PERARD]

On Wed, Mar 20, 2024 at 04:20:00PM +0100, Roger Pau Monné wrote:
> On Mon, Mar 18, 2024 at 04:55:11PM +0000, Anthony PERARD wrote:
> > Newer version of Debian and thus git would use this automatically, no
> > need to force it.
> > 
> > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> > ---
> >  Osstest/TestSupport.pm | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
> > index f0e087aa..0dded9b2 100644
> > --- a/Osstest/TestSupport.pm
> > +++ b/Osstest/TestSupport.pm
> > @@ -3257,7 +3257,7 @@ END
> >  
> >      # Adding mandadory use of Git protocol v2
> >      # necessary on buster when building QEMU v8.1
> > -    $gitcfg .= <<END;
> > +    $gitcfg .= <<END if $ho->{Suite} =~ m/buster/;
> >  [protocol]
> >  	version = 2
> 
> Do we need to limit ourselves to version 2 for the git cache stuff, or
> that doesn't matter?

It's not a limitation, this settings forces `git` to use an experimental
feature. In buster, git still default to version 1, which was an issue
for a single git repo, one of QEMU's meson wrap dependency.

In bookworm, git default to version 2, so I don't think there's need to
deviate from default beyond buster.

Once git-cache-proxy knows how to handle this version=2, and pass it on
to `git-upload-pack`, it will know how to handle v3 and beyond.

> I'm wondering whether it case version 3 of the protocol appears we
> would be in trouble by not having version = 2 in the config file.

No, git should be able to negotiate with itself. Even today, if a git
server only have protocol v1, it will just ignore that a client ask for
v2.

Thanks,

-- 
Anthony PERARD

[OSSTEST PATCH 03/36] mgi-common: Fix fetch_debian_package error message

[Anthony PERARD]

$@ expand to two or more words, but fail() only take one argument.
So if there's more than one argument left, fail() only shows the first
one, and don't event display "not found".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 mgi-common | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mgi-common b/mgi-common
index 6ce34162..98f795b1 100644
--- a/mgi-common
+++ b/mgi-common
@@ -49,7 +49,7 @@ fetch_debian_package () {
     set -e
 
     if [ $rc -ne 0 ] || [ x$pkgfile = x ]; then
-	fail "package matching $@ not found";
+	fail "package matching $* not found";
     fi
 
     fetch "$site/$pkgfile"
-- 
Anthony PERARD

[OSSTEST PATCH 04/36] mg-debian-installer-update: Download non-free firmware from new repo.

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 mg-debian-installer-update | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/mg-debian-installer-update b/mg-debian-installer-update
index 4fb4bc21..31b8a192 100755
--- a/mg-debian-installer-update
+++ b/mg-debian-installer-update
@@ -105,7 +105,14 @@ if [ "x$dtbs" != "x" ] ; then
     gzip -9nf dtbs.tar
 fi
 
-pfile=$sbase/non-free/binary-$arch/Packages.gz
+case $suite in
+    wheezy|jessie|stretch|buster|bullseye)
+        pfile=$sbase/non-free/binary-$arch/Packages.gz
+        ;;
+    *)
+        # Starting with bookworm, firmware are in a new repo
+        pfile=$sbase/non-free-firmware/binary-$arch/Packages.gz
+esac
 
 fetch $pfile >Packages.gz
 
-- 
Anthony PERARD

[OSSTEST PATCH 05/36] ts-host-install: fix ntp.conf path on bookworm

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-host-install | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/ts-host-install b/ts-host-install
index f79a1beb..61433e64 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -151,7 +151,13 @@ END
 
     my $ntpserver = get_target_property($ho, 'NtpServer');
     if ($ntpserver) {
-	target_editfile_root($ho, '/etc/ntp.conf', sub {
+        my $ntpconf_path;
+        if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie|stretch|buster/) {
+            $ntpconf_path = '/etc/ntp.conf';
+        } else {
+            $ntpconf_path = '/etc/ntpsec/ntp.conf';
+        }
+	target_editfile_root($ho, $ntpconf_path, sub {
 	    my $done= 0;
 	    while (<EI>) {
 		if (m/^server\b|^pool\b\s/) {
-- 
Anthony PERARD

[OSSTEST PATCH 06/36] ts-host-install: fix ntp server setting

[Anthony PERARD]

The Debian #778564 bug report is still open, the ntp.conf file doesn't
contain the setting from NtpServer after installation, so we still
need to edit ntp.conf on Bookworm.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-host-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-host-install b/ts-host-install
index 61433e64..00277485 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -161,7 +161,7 @@ END
 	    my $done= 0;
 	    while (<EI>) {
 		if (m/^server\b|^pool\b\s/) {
-		    if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie|stretch|buster/) {
+		    if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie|stretch|buster|bookworm/) {
 			$_= $done ? "" : "server $ntpserver\n";
 		    } else {
 			m/^server \Q$ntpserver\E\s/ or
-- 
Anthony PERARD

[OSSTEST PATCH 07/36] ts-host-install: Restart ntp service

[Anthony PERARD]

Otherwise, the change to the config file isn't taken into account
until the next reboot.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-host-install | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/ts-host-install b/ts-host-install
index 00277485..43ed9285 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -152,10 +152,12 @@ END
     my $ntpserver = get_target_property($ho, 'NtpServer');
     if ($ntpserver) {
         my $ntpconf_path;
+        my $ntp_service;
         if ($ho->{Suite} =~ m/lenny|squeeze|wheezy|jessie|stretch|buster/) {
             $ntpconf_path = '/etc/ntp.conf';
         } else {
             $ntpconf_path = '/etc/ntpsec/ntp.conf';
+            $ntp_service = 'ntpsec';
         }
 	target_editfile_root($ho, $ntpconf_path, sub {
 	    my $done= 0;
@@ -173,6 +175,9 @@ END
 	    }
 	    $done or die;
 	});
+	if ($ntp_service) {
+	    target_cmd_root($ho, "service $ntp_service restart");
+	}
     }
 
     host_install_postboot_complete($ho);
-- 
Anthony PERARD

[OSSTEST PATCH 08/36] preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI installation

[Anthony PERARD]

Instead of "grub-installer/no-nvram" proposed in Debian bug #789798,
we have "grub-installer/update-nvram". Make use of it, and remove
workaround.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 9b30a375..7d6f9778 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1550,13 +1550,17 @@ d-i partman-auto/expert_recipe string					\\
 			lv_name{ dummy }				\\
 		.
 
+# Prevent grub-install from making a new Debian boot entry, so
+# we always reboot from the network.
+d-i grub-installer/update-nvram boolean false
 END
 
-    if (get_host_property($ho, "firmware") eq "uefi") {
-	die unless $ho->{Suite} =~ m/jessie|stretch|buster/;
+    if (get_host_property($ho, "firmware") eq "uefi" &&
+	$ho->{Suite} =~ m/jessie|stretch|buster/) {
 	# Prevent grub-install from making a new Debian boot entry, so
 	# we always reboot from the network. Debian bug #789798 proposes a
-	# properly preseedable solution to this.
+	# properly preseedable solution to this. Implemented with
+	# "grub-installer/update-nvram".
 	preseed_hook_installscript($ho, $sfx,
 		'/usr/lib/base-installer.d/',
 		'osstest-disable-grub-nvram', <<'END');
-- 
Anthony PERARD

[OSSTEST PATCH 09/36] preseed_create: osstest-erase-other-disks: workaround creating /dev/sdXD files

[Anthony PERARD]

Sometime, it can happen that the erase-other-disks script fails and
indicate that /dev/sda1 isn't a block device anymore. This happened
when /dev/sda was erased before /dev/sda1. So to avoid this, we will
zero the partitions of ${disk} first, and hopefully the error won't
happen again.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 7d6f9778..36c05551 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1305,7 +1305,7 @@ for sd in sd hd; do
     udevadm settle
     log "\${sd} devices present after: `echo /dev/\${sd}*`"
 done
-for dev in ${disk}*; do
+for dev in ${disk}?* ${disk}*; do
     zero
 done
 echo ===
-- 
Anthony PERARD

[OSSTEST PATCH 10/36] preseed_create: Workaround fail grub-install on arndale

[Anthony PERARD]

grub-installer on arndale-* machine fails with Debian Bookworm. It
tries to install "grub-pc" which doesn't exist. Skip installation.

Somehow, cubietruck-* installation works fine.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 36c05551..98b2fc86 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1581,6 +1581,14 @@ chmod +x /target/$t
 END
     }
 
+    if ( $ho->{Flags}{'need-uboot-bootscr'} &&
+        $ho->{Suite} =~ m/bookworm/) {
+
+        # grub-installer fails on the arndale machine, it's trying to install
+        # "grub-pc" which doesn't exist. Skip installation.
+        $preseed_file .= "d-i grub-installer/skip boolean true\n";
+    }
+
     $preseed_file .= preseed_hook_cmds();
 
     if ($backports_kernel || $ho->{Flags}{'no-di-kernel'}) {
-- 
Anthony PERARD

Re: [OSSTEST PATCH 10/36] preseed_create: Workaround fail grub-install on arndale

[Roger Pau Monné]

On Mon, Mar 18, 2024 at 04:55:19PM +0000, Anthony PERARD wrote:
> grub-installer on arndale-* machine fails with Debian Bookworm. It
> tries to install "grub-pc" which doesn't exist. Skip installation.
> 
> Somehow, cubietruck-* installation works fine.

I'm kind of puzzled by this, as cubietruck and arndales are both armhf
IIRC? (IOW: they should use the same repo?)

Does the install of grub-pc succeed on the cubietruck, or is skipped
for some other reason that doesn't require us to intervene?

Thanks, Roger.

Re: [OSSTEST PATCH 10/36] preseed_create: Workaround fail grub-install on arndale

[Anthony PERARD]

On Wed, Mar 20, 2024 at 05:38:17PM +0100, Roger Pau Monné wrote:
> On Mon, Mar 18, 2024 at 04:55:19PM +0000, Anthony PERARD wrote:
> > grub-installer on arndale-* machine fails with Debian Bookworm. It
> > tries to install "grub-pc" which doesn't exist. Skip installation.
> > 
> > Somehow, cubietruck-* installation works fine.
> 
> I'm kind of puzzled by this, as cubietruck and arndales are both armhf
> IIRC? (IOW: they should use the same repo?)
> 
> Does the install of grub-pc succeed on the cubietruck, or is skipped
> for some other reason that doesn't require us to intervene?

It's skipped.

The difference I can see is that, on the cubietruck, there's an extra
step. And it's due to this extra logging:
    debconf: --> METAGET debian-installer/flash-kernel-installer/title Description
    debconf: <-- 0 Make the system bootable

On the cubietruck, there would be these installation step:
    ..., Select and install software, Make the system bootable, Install the GRUB boot loader, ...
But on the arndale, it would be instead:
    ..., Select and install software, Install the GRUB boot loader, ...

Somehow, flash-kernel-installer does something different on both
machine.

Thanks,

-- 
Anthony PERARD

[OSSTEST PATCH 11/36] ts-host-install,preseed_create: Do lvm vgextend at install time

[Anthony PERARD]

When there's more than one disk, like the "pinot?" machine, the name
assigned to e.g. "sda" may change after a reboot, at least when
installing Debian Bookworm, which is using Linux 6.1.

I believe Linux probes disk controller in parallel and assign "sda"
to the first controller to respond, or something like that, so disk
aren't assigned a name in a predictable order.

So, instead of extending lvm volume group to a new disk on reboot, do
that at install time when we know that lvm is on sda.

Alternatively, we could try to set "d-i partman-auto/disk" with a list
of all disk, but that would mean knowing all the disk before starting
the machine.

This new shell script in /lib/partman/finish.d/ should work in all
debian, so replace the script in ts-host-install by this new one.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 21 +++++++++++++++++++++
 ts-host-install   |  9 ---------
 2 files changed, 21 insertions(+), 9 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 98b2fc86..14ec2293 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1312,6 +1312,27 @@ echo ===
 set +e
 ls -l /dev/sd*
 true
+END
+
+    # Extend lvm vg to other disks
+    preseed_hook_installscript($ho, $sfx,
+          '/lib/partman/finish.d', '99extend-vg', <<END, 0);
+#!/bin/sh
+set -ex
+
+log () {
+    logger -t extend-vg-\$\$ "\$\@"
+}
+
+vgname="\$(hostname)-vg"
+devices="\$(sed -nE 's/^[[:space:]]*[[:digit:]]+[[:space:]]+[[:digit:]]+[[:space:]]+[[:digit:]]+[[:space:]]+([sh]d[b-z])[[:space:]]*\$/\\1/p' /proc/partitions)"
+
+for dev in \$devices; do
+    log "Extend \$vgname to \$dev"
+    pvcreate /dev/\$dev
+    vgextend \$vgname /dev/\$dev
+done
+log done
 END
 
     my $dtbs = "$d_i/dtbs.tar.gz";
diff --git a/ts-host-install b/ts-host-install
index 43ed9285..17a1bf78 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -116,15 +116,6 @@ END
         target_cmd_root($ho, "lvremove -f $1");
     }
 
-    my $partitions= target_cmd_output_root($ho, 'cat /proc/partitions');
-    foreach my $l (split /\n/, $partitions) {
-        logm("| $l");
-        next unless $l =~ m/^\s*\d+\s+\d+\s+\d+\s+((?:sd|hd)[b-z])\s*$/;
-        my $dev= "/dev/$1";
-        target_cmd_root($ho, "pvcreate $dev");
-        target_cmd_root($ho, "vgextend $vgname $dev");
-    }
-
     my $kpath= $c{TestHostKeypairPath};
     my $kleaf= $kpath;
     $kleaf =~ s,.*/,,;
-- 
Anthony PERARD

[OSSTEST PATCH 12/36] di_installcmdline_core: Add link_wait_timeout to install cmdline

[Anthony PERARD]

The default wait time of 3 seconds isn't always enough get an IP from
the DHCP, give more time to the installer to find a NIC that works.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 14ec2293..cf5a3bd1 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -723,6 +723,7 @@ sub di_installcmdline_core ($$;@) {
                "hostname=$tho->{Name}",
                "$xopts{PreseedScheme}=$ps_url",
                "netcfg/dhcp_timeout=150",
+               "netcfg/link_wait_timeout=10",
                "netcfg/choose_interface=$netcfg_interface",
                );
 
-- 
Anthony PERARD

[OSSTEST PATCH 13/36] Disable persistent net generator on Bookworm

[Anthony PERARD]

This schema doesn't work. Even if the udev rule is there, the name of
the different NIC are different from one boot to the next. On a
machine (sabro*) with 3 different NIC, the name of each interface is
basically random and could take on of three name, "eth[0-2]".

net.ifnames=0 does still mean that we have "eth*" names, but the
generated file "75-persistent-net-generator" doesn't seems to name the
interfaces as expected.

The new predictable network interface names schema seems to work well
enough. The names are still the same after rebooting into Xen, at
least on the few machine tested so far. But, we will change to a "mac"
name policy in a follow-up commit.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 12 +++++++-----
 ts-host-install   |  2 +-
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index cf5a3bd1..102b0246 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -87,10 +87,12 @@ sub debian_boot_setup ($$$$$;$) {
         $kopt .= ' '.$targkopt;
     }
 
-    # https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
-    # In fact these are anything but predictable.  We use the scheme
-    # from Debian jessie and earlier, persistent-net-generator etc.
-    $kopt .= ' net.ifnames=0';
+    if ($ho->{Suite} =~ m/stretch|buster/) {
+        # https://www.freedesktop.org/wiki/Software/systemd/PredictableNetworkInterfaceNames/
+        # In fact these are anything but predictable.  We use the scheme
+        # from Debian jessie and earlier, persistent-net-generator etc.
+        $kopt .= ' net.ifnames=0';
+    }
 
     foreach my $hook ($hooks ? @$hooks : ()) {
         my $bo_hook= $hook->{EditBootOptions};
@@ -800,7 +802,7 @@ sub debian_overlays ($$) {
     # network device names', is the default; but it is anything but
     # predictable, so we disable it.  Instead, we restore the
     # 75-persistent-net-generator mechanism from jessie and earlier.
-    $maybe->("overlay-persistent-net") if $ho->{Suite} !~ m/wheezy|jessie/;
+    $maybe->("overlay-persistent-net") if $ho->{Suite} =~ m/stretch|buster/;
 
     $maybe->("overlay-$suite");
     $maybe->($c{OverlayLocal}, 'overlay-local.tar');
diff --git a/ts-host-install b/ts-host-install
index 17a1bf78..88481038 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -214,7 +214,7 @@ sub setup_netboot_firstboot($) {
     system qw(rm -rf --),"$initrd_overlay.d";
     mkdir "$initrd_overlay.d" or die "$initrd_overlay.d: $!";
 
-    if ($ho->{Suite} !~ m/wheezy|jessie/) {
+    if ($ho->{Suite} =~ m/stretch|buster/) {
 	my @cmd = (qw(cp -dR overlay-persistent-net/.),
 		   "$initrd_overlay.d/.");
 	logm("using persistent-net-generator: @cmd");
-- 
Anthony PERARD

[OSSTEST PATCH 14/36] preseed_base, ts-host-install: Change NIC NamePolicy to "mac"

[Anthony PERARD]

On "italia?" machine, the two network interfaces are competing to have
"eno1", base on the "onboard" naming policy. So the name of the
network interface can change between "eno1" and "eth0".

Switching to "mac" should avoid the unpredictable name based on
"onboard" or "slot" policy.

The "mac" naming policy break `vif-bridge` for the "vif*.*" network
interfaces. So we will avoid the "mac" policy if the driver is "vif".
This also have an impact on guest created with ./ts-debian-install, as
they use the initrd from dom0, so the interface in the guest will be
renamed according to default policy (which rename eth0 to enX0).

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 22 ++++++++++++++++++++++
 ts-host-install   | 16 ++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 102b0246..31d32d6f 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -962,6 +962,28 @@ cp /$RULESDIR/70-persistent-*.rules /target/$RULESDIR 2>/dev/null || true
 
 END
 
+    if ($ho->{Suite} !~ m/lenny|squeeze|wheezy|jessie|stretch|buster/) {
+        # Always use MAC address in network interface names.
+        #
+        # But keep the default policy if the driver is "vif", which match the
+        # "vif$domid.$idx" interface in dom0, and the interface in domU. This file
+        # is going to be added to dom0's initrd, which is used by some guests
+        # (created with ts-debian-install).
+        preseed_hook_installscript($ho, $sfx,
+            '/usr/lib/base-installer.d/', '05ifnamepolicy', <<'END');
+#!/bin/sh -e
+linkfile=/target/etc/systemd/network/90-eth-mac-policy.link
+mkdir -p `dirname $linkfile`
+cat > $linkfile <<EOF
+[Match]
+Type=ether
+Driver=!vif
+[Link]
+NamePolicy=mac
+EOF
+END
+    }
+
     debian_overlays($ho, sub {
 	my ($srcdir, $tfilename) = @_;
 	preseed_hook_overlay($ho, $sfx, $srcdir, $tfilename);
diff --git a/ts-host-install b/ts-host-install
index 88481038..0b6aaeea 100755
--- a/ts-host-install
+++ b/ts-host-install
@@ -248,6 +248,22 @@ END
     print CANARY "\n# - canary - came via initramfs\n" or die $!;
     close CANARY or die $!;
 
+    if ($ho->{Suite} !~ m/lenny|squeeze|wheezy|jessie|stretch|buster/) {
+        # Switch to more predictale nic name based on mac address, instead of the
+        # policy "onboard" which can try to set the same name ("eno1") to two
+        # differents nic, or "slot". New names are "enx$mac".
+        system_checked(qw(mkdir -p --), "$initrd_overlay.d/lib/systemd/network");
+        file_simple_write_contents
+            ("$initrd_overlay.d/lib/systemd/network/90-eth-mac-policy.link",
+                <<END);
+[Match]
+Type=ether
+Driver=!vif
+[Link]
+NamePolicy=mac
+END
+    }
+
     my %xopts;
 
     di_special_kernel($ho, sub {
-- 
Anthony PERARD

[OSSTEST PATCH 15/36] ts-xen-build-prep: Change package selection for Bookworm

[Anthony PERARD]

python-dev:
    It doesn't exist on bookworm, and python2 shouldn't be needed
    anymore.

libsdl-dev:
    On buster this already select "libsdl1.2-dev", but to not change
    buster installation we will only use the new package name on
    Bookworm.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-xen-build-prep | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/ts-xen-build-prep b/ts-xen-build-prep
index 547bbc16..48d3fb75 100755
--- a/ts-xen-build-prep
+++ b/ts-xen-build-prep
@@ -200,9 +200,9 @@ sub prep () {
     my @packages = qw(mercurial rsync figlet
                       build-essential bin86 bcc iasl bc
                       flex bison cmake ninja-build meson
-                      libpci-dev libncurses5-dev libssl-dev python-dev
+                      libpci-dev libncurses5-dev libssl-dev
                       libx11-dev git-core uuid-dev gettext gawk
-                      libsdl-dev libyajl-dev libaio-dev libpixman-1-dev
+                      libyajl-dev libaio-dev libpixman-1-dev
                       libglib2.0-dev liblzma-dev pkg-config
                       autoconf automake libtool xsltproc
                       libxml2-utils libxml2-dev
@@ -229,6 +229,12 @@ sub prep () {
         push(@packages, qw(texinfo autopoint libpciaccess-dev));
     }
 
+    if ($ho->{Suite} =~ m/squeeze|wheezy|jessie|stretch|buster/) {
+        push(@packages, qw(python-dev libsdl-dev))
+    } else {
+        push(@packages, qw(libsdl1.2-dev))
+    }
+
     # The in-tree ext4 support in libfsimage can't cope with 64bit ext4 on
     # 32bit build. Use the packaged library.
     if ($ho->{Suite} !~ m/squeeze|wheezy|jessie/) {
-- 
Anthony PERARD

[OSSTEST PATCH 16/36] bl_getmenu_open: Read grub.cfg as root

[Anthony PERARD]

On bookworm, "/boot/grub/grub.cfg" isn't accessible by user "osstest",
so read the file as user "root".

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 31d32d6f..57f31977 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -139,7 +139,7 @@ sub debian_boot_setup ($$$$$;$) {
 
 sub bl_getmenu_open ($$$) {
     my ($ho, $rmenu, $lmenu) = @_;
-    target_getfile($ho, 60, $rmenu, $lmenu);
+    target_getfile_root($ho, 60, $rmenu, $lmenu);
     my $f= new IO::File $lmenu, 'r' or die "$lmenu $?";
     return $f;
 }
-- 
Anthony PERARD

[OSSTEST PATCH 17/36] target_editfile: Use the same user to retrieve and send

[Anthony PERARD]

The file "/boot/grub/grub.cfg" on Debian Bookworm isn't accessible
from the "osstest" user, but target_editfile_root() tries to grab the
file as "osstest" then edit it as "root.

Change teditfileex() to use the same $user also to get the file. This
will fix ts-examine-serial-pre step.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/TestSupport.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/TestSupport.pm b/Osstest/TestSupport.pm
index 0dded9b2..b86f1d96 100644
--- a/Osstest/TestSupport.pm
+++ b/Osstest/TestSupport.pm
@@ -783,7 +783,7 @@ sub teditfileex {
         logm("editing $rfile to $rdest as $lfile".'{,.new}');
     }
 
-    target_getfile($ho, 60, $rfile, $lfile);
+    tgetfileex($user, $ho, 60, $rfile, $lfile);
     open '::EI', "$lfile" or die "$lfile: $!";
     open '::EO', "> $lfile.new" or die "$lfile.new: $!";
 
-- 
Anthony PERARD

[OSSTEST PATCH 18/36] ts-xen-install: remove "libc6-xen" package installation

[Anthony PERARD]

libc6-xen packaged have been removed from Debian Bookworm.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-xen-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-xen-install b/ts-xen-install
index bf55d4e5..3a913fce 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -64,7 +64,7 @@ sub packages () {
     if ($ho->{Suite} !~ m/lenny|squeeze/) {
         target_install_packages($ho, qw(libfdt1));
     }
-    if ($ho->{Arch} eq 'i386') {
+    if ($ho->{Arch} eq 'i386' && $ho->{Suite} =~ m/wheezy|jessie|stretch|buster/) {
 	target_install_packages($ho, qw(libc6-xen));
     }
     target_install_packages($ho, @{toolstack($ho)->{ExtraPackages}})
-- 
Anthony PERARD

[OSSTEST PATCH 19/36] overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm

[Anthony PERARD]

This is a copy of the file installed, from grub-common package.

We are going to edit it shortly.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 overlay-bookworm/etc/grub.d/20_linux_xen | 379 +++++++++++++++++++++++
 1 file changed, 379 insertions(+)
 create mode 100755 overlay-bookworm/etc/grub.d/20_linux_xen

diff --git a/overlay-bookworm/etc/grub.d/20_linux_xen b/overlay-bookworm/etc/grub.d/20_linux_xen
new file mode 100755
index 00000000..3a27fc6f
--- /dev/null
+++ b/overlay-bookworm/etc/grub.d/20_linux_xen
@@ -0,0 +1,379 @@
+#! /bin/sh
+set -e
+
+# grub-mkconfig helper script.
+# Copyright (C) 2006,2007,2008,2009,2010  Free Software Foundation, Inc.
+#
+# GRUB is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# GRUB is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GRUB.  If not, see <http://www.gnu.org/licenses/>.
+
+prefix="/usr"
+exec_prefix="/usr"
+datarootdir="/usr/share"
+
+. "$pkgdatadir/grub-mkconfig_lib"
+
+export TEXTDOMAIN=grub
+export TEXTDOMAINDIR="${datarootdir}/locale"
+
+CLASS="--class gnu-linux --class gnu --class os --class xen"
+SUPPORTED_INITS="sysvinit:/lib/sysvinit/init systemd:/lib/systemd/systemd upstart:/sbin/upstart"
+
+if [ "x${GRUB_DISTRIBUTOR}" = "x" ] ; then
+  OS=GNU/Linux
+else
+  OS="${GRUB_DISTRIBUTOR} GNU/Linux"
+  CLASS="--class $(echo ${GRUB_DISTRIBUTOR} | tr 'A-Z' 'a-z' | cut -d' ' -f1|LC_ALL=C sed 's,[^[:alnum:]_],_,g') ${CLASS}"
+fi
+
+# loop-AES arranges things so that /dev/loop/X can be our root device, but
+# the initrds that Linux uses don't like that.
+case ${GRUB_DEVICE} in
+  /dev/loop/*|/dev/loop[0-9])
+    GRUB_DEVICE=`losetup ${GRUB_DEVICE} | sed -e "s/^[^(]*(\([^)]\+\)).*/\1/"`
+    # We can't cope with devices loop-mounted from files here.
+    case ${GRUB_DEVICE} in
+      /dev/*) ;;
+      *) exit 0 ;;
+    esac
+  ;;
+esac
+
+# Default to disabling partition uuid support to maintian compatibility with
+# older kernels.
+GRUB_DISABLE_LINUX_PARTUUID=${GRUB_DISABLE_LINUX_PARTUUID-true}
+
+# btrfs may reside on multiple devices. We cannot pass them as value of root= parameter
+# and mounting btrfs requires user space scanning, so force UUID in this case.
+if ( [ "x${GRUB_DEVICE_UUID}" = "x" ] && [ "x${GRUB_DEVICE_PARTUUID}" = "x" ] ) \
+    || ( [ "x${GRUB_DISABLE_LINUX_UUID}" = "xtrue" ] \
+	&& [ "x${GRUB_DISABLE_LINUX_PARTUUID}" = "xtrue" ] ) \
+    || ( ! test -e "/dev/disk/by-uuid/${GRUB_DEVICE_UUID}" \
+	&& ! test -e "/dev/disk/by-partuuid/${GRUB_DEVICE_PARTUUID}" ) \
+    || ( test -e "${GRUB_DEVICE}" && uses_abstraction "${GRUB_DEVICE}" lvm ); then
+  LINUX_ROOT_DEVICE=${GRUB_DEVICE}
+elif [ "x${GRUB_DEVICE_UUID}" = "x" ] \
+    || [ "x${GRUB_DISABLE_LINUX_UUID}" = "xtrue" ]; then
+  LINUX_ROOT_DEVICE=PARTUUID=${GRUB_DEVICE_PARTUUID}
+else
+  LINUX_ROOT_DEVICE=UUID=${GRUB_DEVICE_UUID}
+fi
+
+# Allow overriding GRUB_CMDLINE_LINUX and GRUB_CMDLINE_LINUX_DEFAULT.
+if [ "${GRUB_CMDLINE_LINUX_XEN_REPLACE}" ]; then
+  GRUB_CMDLINE_LINUX="${GRUB_CMDLINE_LINUX_XEN_REPLACE}"
+fi
+if [ "${GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT}" ]; then
+  GRUB_CMDLINE_LINUX_DEFAULT="${GRUB_CMDLINE_LINUX_XEN_REPLACE_DEFAULT}"
+fi
+
+case x"$GRUB_FS" in
+    xbtrfs)
+	rootsubvol="`make_system_path_relative_to_its_root /`"
+	rootsubvol="${rootsubvol#/}"
+	if [ "x${rootsubvol}" != x ]; then
+	    GRUB_CMDLINE_LINUX="rootflags=subvol=${rootsubvol} ${GRUB_CMDLINE_LINUX}"
+	fi;;
+    xzfs)
+	rpool=`${grub_probe} --device ${GRUB_DEVICE} --target=fs_label 2>/dev/null || true`
+	bootfs="`make_system_path_relative_to_its_root / | sed -e "s,@$,,"`"
+	LINUX_ROOT_DEVICE="ZFS=${rpool}${bootfs%/}"
+	;;
+esac
+
+title_correction_code=
+
+linux_entry ()
+{
+  linux_entry_xsm "$@" false
+  linux_entry_xsm "$@" true
+}
+linux_entry_xsm ()
+{
+  os="$1"
+  version="$2"
+  xen_version="$3"
+  type="$4"
+  args="$5"
+  xen_args="$6"
+  xsm="$7"
+  # If user wants to enable XSM support, make sure there's
+  # corresponding policy file.
+  if ${xsm} ; then
+      xenpolicy="xenpolicy-$xen_version"
+      if test ! -e "${xen_dirname}/${xenpolicy}" ; then
+	  return
+      fi
+      xen_args="$xen_args flask=enforcing"
+      xen_version="$(gettext_printf "%s (XSM enabled)" "$xen_version")"
+      # xen_version is used for messages only; actual file is xen_basename
+  fi
+  if [ -z "$boot_device_id" ]; then
+      boot_device_id="$(grub_get_device_id "${GRUB_DEVICE}")"
+  fi
+  if [ x$type != xsimple ] ; then
+      if [ x$type = xrecovery ] ; then
+	  title="$(gettext_printf "%s, with Xen %s and Linux %s (%s)" "${os}" "${xen_version}" "${version}" "$(gettext "${GRUB_RECOVERY_TITLE}")")"
+      elif [ "${type#init-}" != "$type" ] ; then
+	  title="$(gettext_printf "%s, with Xen %s and Linux %s (%s)" "${os}" "${xen_version}" "${version}" "${type#init-}")"
+      else
+	  title="$(gettext_printf "%s, with Xen %s and Linux %s" "${os}" "${xen_version}" "${version}")"
+      fi
+      replacement_title="$(echo "Advanced options for ${OS}" | sed 's,>,>>,g')>$(echo "$title" | sed 's,>,>>,g')"
+      if [ x"Xen ${xen_version}>$title" = x"$GRUB_ACTUAL_DEFAULT" ]; then
+         quoted="$(echo "$GRUB_ACTUAL_DEFAULT" | grub_quote)"
+         title_correction_code="${title_correction_code}if [ \"x\$default\" = '$quoted' ]; then default='$(echo "$replacement_title" | grub_quote)'; fi;"
+         grub_warn "$(gettext_printf "Please don't use old title \`%s' for GRUB_DEFAULT, use \`%s' (for versions before 2.00) or \`%s' (for 2.00 or later)" "$GRUB_ACTUAL_DEFAULT" "$replacement_title" "gnulinux-advanced-$boot_device_id>gnulinux-$version-$type-$boot_device_id")"
+      fi
+      echo "menuentry '$(echo "$title" | grub_quote)' ${CLASS} \$menuentry_id_option 'xen-gnulinux-$version-$type-$boot_device_id' {" | sed "s/^/$submenu_indentation/"
+  else
+      title="$(gettext_printf "%s, with Xen hypervisor" "${os}")"
+      echo "menuentry '$(echo "$title" | grub_quote)' ${CLASS} \$menuentry_id_option 'xen-gnulinux-simple-$boot_device_id' {" | sed "s/^/$submenu_indentation/"
+  fi
+  if [ x$type != xrecovery ] ; then
+      save_default_entry | grub_add_tab | sed "s/^/$submenu_indentation/"
+  fi
+
+  if [ -z "${prepare_boot_cache}" ]; then
+    prepare_boot_cache="$(prepare_grub_to_access_device ${GRUB_DEVICE_BOOT} | grub_add_tab)"
+  fi
+  printf '%s\n' "${prepare_boot_cache}" | sed "s/^/$submenu_indentation/"
+  xmessage="$(gettext_printf "Loading Xen %s ..." ${xen_version})"
+  lmessage="$(gettext_printf "Loading Linux %s ..." ${version})"
+  sed "s/^/$submenu_indentation/" << EOF
+	echo	'$(echo "$xmessage" | grub_quote)'
+        if [ "\$grub_platform" = "pc" -o "\$grub_platform" = "" ]; then
+            xen_rm_opts=
+        else
+            xen_rm_opts="no-real-mode edd=off"
+        fi
+	${xen_loader}	${rel_xen_dirname}/${xen_basename} placeholder ${xen_args} \${xen_rm_opts}
+	echo	'$(echo "$lmessage" | grub_quote)'
+	${module_loader}	${rel_dirname}/${basename} placeholder root=${linux_root_device_thisversion} ro ${args}
+EOF
+  if test -n "${initrd}" ; then
+    # TRANSLATORS: ramdisk isn't identifier. Should be translated.
+    message="$(gettext_printf "Loading initial ramdisk ...")"
+    initrd_path=
+    for i in ${initrd}; do
+       initrd_path="${initrd_path} ${rel_dirname}/${i}"
+    done
+    sed "s/^/$submenu_indentation/" << EOF
+	echo	'$(echo "$message" | grub_quote)'
+	${module_loader}	--nounzip   $(echo $initrd_path)
+EOF
+  fi
+  if ${xsm} && test -n "${xenpolicy}" ; then
+    message="$(gettext_printf "Loading XSM policy ...")"
+    sed "s/^/$submenu_indentation/" << EOF
+	echo	'$(echo "$message" | grub_quote)'
+	${module_loader}     ${rel_dirname}/${xenpolicy}
+EOF
+  fi
+  sed "s/^/$submenu_indentation/" << EOF
+}
+EOF
+}
+
+linux_list=
+for i in /boot/vmlinu[xz]-* /vmlinu[xz]-* /boot/kernel-*; do
+    if grub_file_is_not_garbage "$i"; then
+    	basename=$(basename $i)
+	version=$(echo $basename | sed -e "s,^[^0-9]*-,,g")
+	dirname=$(dirname $i)
+	config=
+	for j in "${dirname}/config-${version}" "${dirname}/config-${alt_version}" "/etc/kernels/kernel-config-${version}" ; do
+	    if test -e "${j}" ; then
+		config="${j}"
+		break
+	    fi
+	done
+        if (grep -qx "CONFIG_XEN_DOM0=y" "${config}" 2> /dev/null || grep -qx "CONFIG_XEN_PRIVILEGED_GUEST=y" "${config}" 2> /dev/null); then linux_list="$linux_list $i" ; fi
+    fi
+done
+if [ "x${linux_list}" = "x" ] ; then
+    exit 0
+fi
+
+file_is_not_xen_garbage () {
+    case "$1" in
+	*/xen-syms-*)
+	    return 1;;
+	*/xenpolicy-*)
+	    return 1;;
+	*/*.config)
+	    return 1;;
+	*)
+	    return 0;;
+    esac
+}
+
+xen_list=
+for i in /boot/xen*; do
+    if grub_file_is_not_garbage "$i" && file_is_not_xen_garbage "$i" ; then xen_list="$xen_list $i" ; fi
+done
+prepare_boot_cache=
+boot_device_id=
+
+title_correction_code=
+
+machine=`uname -m`
+
+case "$machine" in
+    i?86) GENKERNEL_ARCH="x86" ;;
+    mips|mips64) GENKERNEL_ARCH="mips" ;;
+    mipsel|mips64el) GENKERNEL_ARCH="mipsel" ;;
+    arm*) GENKERNEL_ARCH="arm" ;;
+    *) GENKERNEL_ARCH="$machine" ;;
+esac
+
+# Extra indentation to add to menu entries in a submenu. We're not in a submenu
+# yet, so it's empty. In a submenu it will be equal to '\t' (one tab).
+submenu_indentation=""
+
+is_top_level=true
+
+while [ "x${xen_list}" != "x" ] ; do
+    list="${linux_list}"
+    current_xen=`version_find_latest $xen_list`
+    xen_basename=`basename ${current_xen}`
+    xen_dirname=`dirname ${current_xen}`
+    rel_xen_dirname=`make_system_path_relative_to_its_root $xen_dirname`
+    xen_version=`echo $xen_basename | sed -e "s,.gz$,,g;s,^xen-,,g"`
+    if [ -z "$boot_device_id" ]; then
+	boot_device_id="$(grub_get_device_id "${GRUB_DEVICE}")"
+    fi
+    if [ "x$is_top_level" != xtrue ]; then
+	echo "	submenu '$(gettext_printf "Xen hypervisor, version %s" "${xen_version}" | grub_quote)' \$menuentry_id_option 'xen-hypervisor-$xen_version-$boot_device_id' {"
+    fi
+    if ($grub_file --is-arm64-efi $current_xen); then
+	xen_loader="xen_hypervisor"
+	module_loader="xen_module"
+    else
+	if ($grub_file --is-x86-multiboot2 $current_xen); then
+	    xen_loader="multiboot2"
+	    module_loader="module2"
+	else
+	    xen_loader="multiboot"
+	    module_loader="module"
+        fi
+    fi
+
+    initrd_early=
+    for i in ${GRUB_EARLY_INITRD_LINUX_STOCK} \
+             ${GRUB_EARLY_INITRD_LINUX_CUSTOM}; do
+       if test -e "${xen_dirname}/${i}" ; then
+          initrd_early="${initrd_early} ${i}"
+       fi
+    done
+
+    while [ "x$list" != "x" ] ; do
+	linux=`version_find_latest $list`
+	gettext_printf "Found linux image: %s\n" "$linux" >&2
+	basename=`basename $linux`
+	dirname=`dirname $linux`
+	rel_dirname=`make_system_path_relative_to_its_root $dirname`
+	version=`echo $basename | sed -e "s,^[^0-9]*-,,g"`
+	alt_version=`echo $version | sed -e "s,\.old$,,g"`
+	linux_root_device_thisversion="${LINUX_ROOT_DEVICE}"
+
+	initrd_real=
+	for i in "initrd.img-${version}" "initrd-${version}.img" "initrd-${version}.gz" \
+	   "initrd-${version}" "initramfs-${version}.img" \
+	   "initrd.img-${alt_version}" "initrd-${alt_version}.img" \
+	   "initrd-${alt_version}" "initramfs-${alt_version}.img" \
+	   "initramfs-genkernel-${version}" \
+	   "initramfs-genkernel-${alt_version}" \
+	   "initramfs-genkernel-${GENKERNEL_ARCH}-${version}" \
+	   "initramfs-genkernel-${GENKERNEL_ARCH}-${alt_version}" ; do
+	    if test -e "${dirname}/${i}" ; then
+		initrd_real="$i"
+		break
+	    fi
+	done
+
+	initrd=
+	if test -n "${initrd_early}" || test -n "${initrd_real}"; then
+	    initrd="${initrd_early} ${initrd_real}"
+
+	    initrd_display=
+	    for i in ${initrd}; do
+		initrd_display="${initrd_display} ${dirname}/${i}"
+	    done
+	    gettext_printf "Found initrd image: %s\n" "$(echo $initrd_display)" >&2
+	fi
+
+	if test -z "${initrd_real}"; then
+    # "UUID=" magic is parsed by initrds.  Since there's no initrd, it can't work here.
+	    if [ "x${GRUB_DEVICE_PARTUUID}" = "x" ] \
+		|| [ "x${GRUB_DISABLE_LINUX_PARTUUID}" = "xtrue" ]; then
+
+		linux_root_device_thisversion=${GRUB_DEVICE}
+	    else
+		linux_root_device_thisversion=PARTUUID=${GRUB_DEVICE_PARTUUID}
+	    fi
+	fi
+
+	# The GRUB_DISABLE_SUBMENU option used to be different than others since it was
+	# mentioned in the documentation that has to be set to 'y' instead of 'true' to
+	# enable it. This caused a lot of confusion to users that set the option to 'y',
+	# 'yes' or 'true'. This was fixed but all of these values must be supported now.
+	if [ "x${GRUB_DISABLE_SUBMENU}" = xyes ] || [ "x${GRUB_DISABLE_SUBMENU}" = xy ]; then
+	    GRUB_DISABLE_SUBMENU="true"
+	fi
+
+	if [ "x$is_top_level" = xtrue ] && [ "x${GRUB_DISABLE_SUBMENU}" != xtrue ]; then
+	    linux_entry "${OS}" "${version}" "${xen_version}" simple \
+		"${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT}" "${GRUB_CMDLINE_XEN} ${GRUB_CMDLINE_XEN_DEFAULT}"
+
+	    submenu_indentation="$grub_tab$grub_tab"
+    
+	    if [ -z "$boot_device_id" ]; then
+		boot_device_id="$(grub_get_device_id "${GRUB_DEVICE}")"
+	    fi
+            # TRANSLATORS: %s is replaced with an OS name
+	    echo "submenu '$(gettext_printf "Advanced options for %s (with Xen hypervisor)" "${OS}" | grub_quote)' \$menuentry_id_option 'gnulinux-advanced-$boot_device_id' {"
+	echo "	submenu '$(gettext_printf "Xen hypervisor, version %s" "${xen_version}" | grub_quote)' \$menuentry_id_option 'xen-hypervisor-$xen_version-$boot_device_id' {"
+	   is_top_level=false
+	fi
+
+	linux_entry "${OS}" "${version}" "${xen_version}" advanced \
+	    "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT}" "${GRUB_CMDLINE_XEN} ${GRUB_CMDLINE_XEN_DEFAULT}"
+	for supported_init in ${SUPPORTED_INITS}; do
+	    init_path="${supported_init#*:}"
+	    if [ -x "${init_path}" ] && [ "$(readlink -f /sbin/init)" != "$(readlink -f "${init_path}")" ]; then
+		linux_entry "${OS}" "${version}" "${xen_version}" "init-${supported_init%%:*}" \
+		    "${GRUB_CMDLINE_LINUX} ${GRUB_CMDLINE_LINUX_DEFAULT} init=${init_path}" "${GRUB_CMDLINE_XEN} ${GRUB_CMDLINE_XEN_DEFAULT}"
+
+	    fi
+	done
+	if [ "x${GRUB_DISABLE_RECOVERY}" != "xtrue" ]; then
+	    linux_entry "${OS}" "${version}" "${xen_version}" recovery \
+		"single ${GRUB_CMDLINE_LINUX}" "${GRUB_CMDLINE_XEN}"
+	fi
+
+	list=`echo $list | tr ' ' '\n' | fgrep -vx "$linux" | tr '\n' ' '`
+    done
+    if [ x"$is_top_level" != xtrue ]; then
+	echo '	}'
+    fi
+    xen_list=`echo $xen_list | tr ' ' '\n' | fgrep -vx "$current_xen" | tr '\n' ' '`
+done
+
+# If at least one kernel was found, then we need to
+# add a closing '}' for the submenu command.
+if [ x"$is_top_level" != xtrue ]; then
+  echo '}'
+fi
+
+echo "$title_correction_code"
-- 
Anthony PERARD

[OSSTEST PATCH 20/36] overlay-bookworm: 20_linux_xen: Fix XSM entries generation

[Anthony PERARD]

It turns out that setting $xen_version in linux_entry_xsm() override
$xen_version in the loop over $xen_list. This means that only one
entry per Xen version is going to enable XSM, but all further entries
are going to have "(XSM enabled)" in their titles without enabling
XSM.

When a "xenpolicy-$xen_version" file was found for the current
$xen_version, it would be overwrite $xen_version to add "(XSM
enabled)" to the menu entry title. Once change, the next call to
linux_entry_xsm() would also have this modified $xen_version and would
look for the file "xenpolicy-*(XSM enabled)" and fail.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
(upstream commit: https://git.savannah.gnu.org/cgit/grub.git/commit/?id=db1faedccdce3cf83336155a95c04a8db03744c5)
---
 overlay-bookworm/etc/grub.d/20_linux_xen | 18 +++++++++---------
 1 file changed, 9 insertions(+), 9 deletions(-)

diff --git a/overlay-bookworm/etc/grub.d/20_linux_xen b/overlay-bookworm/etc/grub.d/20_linux_xen
index 3a27fc6f..85593525 100755
--- a/overlay-bookworm/etc/grub.d/20_linux_xen
+++ b/overlay-bookworm/etc/grub.d/20_linux_xen
@@ -102,7 +102,7 @@ linux_entry_xsm ()
 {
   os="$1"
   version="$2"
-  xen_version="$3"
+  entry_xen_version="$3"
   type="$4"
   args="$5"
   xen_args="$6"
@@ -110,27 +110,27 @@ linux_entry_xsm ()
   # If user wants to enable XSM support, make sure there's
   # corresponding policy file.
   if ${xsm} ; then
-      xenpolicy="xenpolicy-$xen_version"
+      xenpolicy="xenpolicy-$entry_xen_version"
       if test ! -e "${xen_dirname}/${xenpolicy}" ; then
 	  return
       fi
       xen_args="$xen_args flask=enforcing"
-      xen_version="$(gettext_printf "%s (XSM enabled)" "$xen_version")"
-      # xen_version is used for messages only; actual file is xen_basename
+      entry_xen_version="$(gettext_printf "%s (XSM enabled)" "$entry_xen_version")"
+      # entry_xen_version is used for messages only; actual file is xen_basename
   fi
   if [ -z "$boot_device_id" ]; then
       boot_device_id="$(grub_get_device_id "${GRUB_DEVICE}")"
   fi
   if [ x$type != xsimple ] ; then
       if [ x$type = xrecovery ] ; then
-	  title="$(gettext_printf "%s, with Xen %s and Linux %s (%s)" "${os}" "${xen_version}" "${version}" "$(gettext "${GRUB_RECOVERY_TITLE}")")"
+	  title="$(gettext_printf "%s, with Xen %s and Linux %s (%s)" "${os}" "${entry_xen_version}" "${version}" "$(gettext "${GRUB_RECOVERY_TITLE}")")"
       elif [ "${type#init-}" != "$type" ] ; then
-	  title="$(gettext_printf "%s, with Xen %s and Linux %s (%s)" "${os}" "${xen_version}" "${version}" "${type#init-}")"
+	  title="$(gettext_printf "%s, with Xen %s and Linux %s (%s)" "${os}" "${entry_xen_version}" "${version}" "${type#init-}")"
       else
-	  title="$(gettext_printf "%s, with Xen %s and Linux %s" "${os}" "${xen_version}" "${version}")"
+	  title="$(gettext_printf "%s, with Xen %s and Linux %s" "${os}" "${entry_xen_version}" "${version}")"
       fi
       replacement_title="$(echo "Advanced options for ${OS}" | sed 's,>,>>,g')>$(echo "$title" | sed 's,>,>>,g')"
-      if [ x"Xen ${xen_version}>$title" = x"$GRUB_ACTUAL_DEFAULT" ]; then
+      if [ x"Xen ${entry_xen_version}>$title" = x"$GRUB_ACTUAL_DEFAULT" ]; then
          quoted="$(echo "$GRUB_ACTUAL_DEFAULT" | grub_quote)"
          title_correction_code="${title_correction_code}if [ \"x\$default\" = '$quoted' ]; then default='$(echo "$replacement_title" | grub_quote)'; fi;"
          grub_warn "$(gettext_printf "Please don't use old title \`%s' for GRUB_DEFAULT, use \`%s' (for versions before 2.00) or \`%s' (for 2.00 or later)" "$GRUB_ACTUAL_DEFAULT" "$replacement_title" "gnulinux-advanced-$boot_device_id>gnulinux-$version-$type-$boot_device_id")"
@@ -148,7 +148,7 @@ linux_entry_xsm ()
     prepare_boot_cache="$(prepare_grub_to_access_device ${GRUB_DEVICE_BOOT} | grub_add_tab)"
   fi
   printf '%s\n' "${prepare_boot_cache}" | sed "s/^/$submenu_indentation/"
-  xmessage="$(gettext_printf "Loading Xen %s ..." ${xen_version})"
+  xmessage="$(gettext_printf "Loading Xen %s ..." ${entry_xen_version})"
   lmessage="$(gettext_printf "Loading Linux %s ..." ${version})"
   sed "s/^/$submenu_indentation/" << EOF
 	echo	'$(echo "$xmessage" | grub_quote)'
-- 
Anthony PERARD

[OSSTEST PATCH 21/36] ts-xtf-install: Install python symlink

[Anthony PERARD]

ts-xtf-run does run ./xtf-runner, which run `python` in its shebang.
So install a `python` symlink to `python3`.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-xtf-install | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/ts-xtf-install b/ts-xtf-install
index a64fd329..fea737ff 100755
--- a/ts-xtf-install
+++ b/ts-xtf-install
@@ -28,6 +28,11 @@ $whhost ||= 'host';
 
 our $ho= selecthost($whhost);
 
+sub packages () {
+    if ($ho->{Suite} !~ m/wheezy|jessie|stretch|buster/) {
+        target_install_packages($ho, qw(python-is-python3));
+    }
+}
 sub extract () {
     my %distpath;
 
@@ -35,4 +40,5 @@ sub extract () {
                                $r{xtfbuildjob}, \%distpath);
 }
 
+packages();
 extract();
-- 
Anthony PERARD

[OSSTEST PATCH 22/36] setupboot_grub2: Parse arm64 uefi grub verbes

[Anthony PERARD]

20_linux_xen now uses "xen_hypervisor" and "xen_module" in place of
"multiboot2?" and "module2?" when generating a config file for
arm64-uefi.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 57f31977..4f07cdef 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -541,7 +541,7 @@ sub setupboot_grub2 ($$$$) {
                 $entry->{Hv}= $1;
                 $entry->{Chainload} = 1;
             }
-            if (m/^\s*multiboot2?\s+(?:\/boot)?\/(xen\-[0-9][-+.0-9a-z]*\S+)\s+(.*)/) {
+            if (m/^\s*(?:multiboot2?|xen_hypervisor)\s+(?:\/boot)?\/(xen\-[0-9][-+.0-9a-z]*\S+)\s+(.*)/) {
                 die unless $entry;
                 $entry->{Hv}= $1;
                 $entry->{Chainload} = 0;
@@ -553,16 +553,16 @@ sub setupboot_grub2 ($$$$) {
                 $entry->{KernVer}= $2;
                 $entry->{KernOpts}= $3;
             }
-            if (m/^\s*module2?\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))\s+(.*)/) {
+            if (m/^\s*(?:module2?|xen_module)\s*(?:\/boot)?\/(vmlinu[xz]-(\S+))\s+(.*)/) {
                 die unless $entry;
                 $entry->{KernDom0}= $1;
                 $entry->{KernVer}= $2;
                 $entry->{KernOpts}= $3;
             }
-            if (m/^\s*module2?\s+(?:--nounzip\s+)*(?:\/boot)?\/(initrd\S+)/) {
+            if (m/^\s*(?:module2?|xen_module)\s+(?:--nounzip\s+)*(?:\/boot)?\/(initrd\S+)/) {
                 $entry->{Initrd}= $1;
             }
-	    if (m/^\s*module2?\s*\/(xenpolicy\S+)/) {
+	    if (m/^\s*(?:module2?|xen_module)\s*\/(xenpolicy\S+)/) {
                 $entry->{Xenpolicy}= $1;
             }
         }
-- 
Anthony PERARD

[OSSTEST PATCH 23/36] bookworm: Extend ARM clock workaround

[Anthony PERARD]

Still broken on arndale, serial stop working early, then the machine
timeout when working on creating a xen guest with xen-create-guest.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 4f07cdef..68f1be60 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -253,8 +253,10 @@ END
 	my @xenkopt = @kopt;
 	push @xenkopt, $xenkopt;
 	# https://bugs.xenproject.org/xen/bug/45
+	# #45 - arm: domain 0 disables clocks which are in fact being used
+	# https://lore.kernel.org/xen-devel/1414672390.2064.31.camel@citrix.com/
 	push @xenkopt, "clk_ignore_unused"
-	    if $ho->{Suite} =~ m/wheezy|jessie|stretch|buster/;
+	    if $ho->{Suite} =~ m/wheezy|jessie|stretch|buster|bookworm/;
 
 	$xenkopt = join ' ', @xenkopt;
 	logm("Dom0 Linux options: $xenkopt");
-- 
Anthony PERARD

[OSSTEST PATCH 24/36] ts-nested-setup, setup l1 lvm volume groupe in guest

[Anthony PERARD]

LVM in l0 doesn't let us run pvcreate on the host LV, `pvcreate
$outer_lvdev` fails with:
    Cannot use /dev/$l0-vg/l1_gueststorage_outer_lv: device is an LV

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-nested-setup | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/ts-nested-setup b/ts-nested-setup
index be3d3733..7fc8beef 100755
--- a/ts-nested-setup
+++ b/ts-nested-setup
@@ -80,14 +80,14 @@ sub guest_storage () {
     target_cmd_root($l0, "vgremove -f $inner_vg ||:");
     my $outer_lvdev = lv_create($l0, $outer_vg, $outer_lv, $size);
 
-    target_cmd_root($l0, <<END);
-        pvcreate $outer_lvdev
-	vgcreate $inner_vg $outer_lvdev
-END
-
     toolstack($l0)->block_attach($l1, "$outer_lvdev,raw,sdb,rw");
     # NB this does not update the l1 guest config so if the l1 is shut
     # down and recreated in the l0, this will vanish.
+
+    target_cmd_root($l1, <<END);
+	pvcreate /dev/xvdb
+	vgcreate $inner_vg /dev/xvdb
+END
 }
 
 packages();
-- 
Anthony PERARD

[OSSTEST PATCH 25/36] ts-leak-check: add new name for udevd workers

[Anthony PERARD]

udevd on Bookworm shows as "(udev-worker)" in the process list.
Suppress them.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-leak-check | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ts-leak-check b/ts-leak-check
index f3cca8aa..023a945f 100755
--- a/ts-leak-check
+++ b/ts-leak-check
@@ -203,6 +203,7 @@ xenstore /libxl
 
 process .* udevd
 process .* .*/systemd-udevd
+process .* \(udev-worker\)
 process .* /.+/systemd-shim
 
 file /var/run/xenstored/db
-- 
Anthony PERARD

[OSSTEST PATCH 26/36] ts-debian-hvm-install: Allow udev failure in install media

[Anthony PERARD]

Kernel in "debian-12.1.0-amd64-netinst.iso" prevent debian installer
from booting. Early on, it does `udevadm trigger --action=add`, which
fails, the same way as the following runes fails:

$ cat /sys/devices/virtual/input/input2/name
Xen Virtual Keyboard
$ echo add > /sys/devices/virtual/input/input2/uevent
[   25.884403] synth uevent: /devices/virtual/input/input2: failed to send uevent
[   25.916498] input input2: uevent: failed to send synthetic uevent: -12
sh: write error: Cannot allocate memory
$ uname -a
Linux (none) 6.1.0-10-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.38-1 (2023-07-14) x86_64 GNU/Linux

This doesn't looks like a new issue, Debian Buster ISO seems to do the
same thing (early boot command, and error in Linux logs), so it's
probable that now `udevadm trigger --action=add` return an error when
there's a failure.

Bug report in the kernel and in Debian:
    https://bugzilla.kernel.org/show_bug.cgi?id=207695
    https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983357

One way to workaround the issue is to remove the vkb device, with xl's
"vkb_device=0", but libvirt doesn't have support for this config
option.

The second option is to edit the installation media, and allow the
failure.

Once installed, the guest boot fine.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

Notes:
    There's a potential fix for the kernel:
    
    https://lore.kernel.org/xen-devel/20221209142615.33574-1-jandryuk@gmail.com/

 ts-debian-hvm-install | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 4deb443e..44eb3ab1 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -174,13 +174,14 @@ sub isolinux_cfg () {
 END
 }
 
-sub prepare_initrd ($$$) {
-    my ($initrddir,$newiso,$preseed_file_path) = @_;
+sub prepare_initrd ($$$$) {
+    my ($initrddir,$newiso,$preseed_file_path,$extra_rune) = @_;
     return <<"END";
       rm -rf $initrddir
       mkdir $initrddir
       cd $initrddir
       gzip -d < $newiso$ramdisk | cpio --extract --make-directories --no-absolute-filename
+      $extra_rune
       cp $preseed_file_path preseed.cfg
       find . | cpio -H newc --create | gzip -9 > $newiso$ramdisk
       cd -
@@ -268,7 +269,17 @@ sub prep () {
         target_putfilecontents_root_stash($ho, 10, preseed(),
                                           $preseed_file_path);
 
-        $cmds = prepare_initrd($initrddir,$newiso,$preseed_file_path);
+        my $extra_preseed_rune = '';
+        if ($gsuite =~ m/bookworm/) {
+            # Xen Virtual Keyboard initialisation fails and return ENOMEM
+            # https://bugzilla.kernel.org/show_bug.cgi?id=207695
+            # https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=983357
+            $extra_preseed_rune .= <<END;
+sed -i '/udevadm trigger --action=add/s/\$/ ||:/' lib/debian-installer/start-udev
+END
+        }
+        $cmds = prepare_initrd($initrddir,$newiso,$preseed_file_path,
+            $extra_preseed_rune);
         target_cmd_root($ho, $cmds, $isotimeout);
 
         target_putfilecontents_root_stash($ho, 10, grub_cfg(),
-- 
Anthony PERARD

[OSSTEST PATCH 27/36] ts-debian-fixup: Fix nic names for bookworm

[Anthony PERARD]

`xen-create-image` doesn't create image for bookworm with a working
network, we need to fix the interface name.

For reference, there's a bug report upstream:
    "UnPredictableNetworkInterfaceNames 'fun' with Bookworm domU: eth0 -> enX0"
    https://github.com/xen-tools/xen-tools/issues/65

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-debian-fixup | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/ts-debian-fixup b/ts-debian-fixup
index 810b3aba..4cf5f980 100755
--- a/ts-debian-fixup
+++ b/ts-debian-fixup
@@ -142,6 +142,20 @@ sub filesystems () {
 END
 }
 
+sub fix_networking () {
+    return if debian_guest_suite($gho) !~ m/bookworm/;
+
+    # `xen-create-image` doesn't setup network in a way that work with bookworm.
+    # The guest boots with "enX0" interface name, but it only try to setup "eth0".
+    # https://github.com/xen-tools/xen-tools/issues/65
+    target_editfile_root($ho, $mountpoint."/etc/network/interfaces", sub {
+        while (<::EI>) {
+            s/\beth0\b/enX0/g;
+            print ::EO or die $!;
+        }
+    });
+}
+
 sub unmount () {
     guest_umount_lv($ho, $gho);
 }
@@ -243,6 +257,7 @@ target_cmd_root($ho, debian_dhcp_rofs_fix($ho, $mountpoint));
 console();
 randomseed();
 filesystems();
+fix_networking();
 otherfixupcfg();
 writecfg();
 unmount();
-- 
Anthony PERARD

[OSSTEST PATCH 28/36] ts-debian-install: keep avoiding to use pygrub

[Anthony PERARD]

xen-tools commit 83c37b476a75 ("Start all Debian releases since
Stretch (9) with pygrub by default") started to use pygrub by default.
Revert this.

With "pygrub" setting, xen-create-guest fails on armhf, the
80-install-kernel hook fails because it doesn't know about "armhf".

https://github.com/xen-tools/xen-tools/commit/83c37b476a7534c432ecc9941817aeb989677da6

There's "--nopygrub" but that doesn't work due to several issues, so
removing "pygrub" from "distributions.conf" is the only way.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-debian-install | 15 +++++++++++++++
 1 file changed, 15 insertions(+)

diff --git a/ts-debian-install b/ts-debian-install
index c42e8a37..a737bec9 100755
--- a/ts-debian-install
+++ b/ts-debian-install
@@ -78,6 +78,21 @@ sub ginstall () {
             fi
 END
     }
+
+    if ($ho->{Suite} =~ m/bookworm/) {
+        # remove "pygrub" from /etc/xen-tools/distributions.conf
+        # The "--nopygrub" option doesn't work.
+        # https://github.com/xen-tools/xen-tools/issues/67
+        # https://github.com/xen-tools/xen-tools/issues/68
+        target_editfile_root($ho, "/etc/xen-tools/distributions.conf", sub {
+            while (<::EI>) {
+                unless (m/^#/) {
+                    s/ pygrub\b//;
+                }
+                print ::EO or die $!;
+            }
+        });
+    }
     my $cmd = '';
     my $useproxy = $c{DebianMirrorProxy} // $c{HttpProxy};
     $cmd .= <<END if defined $useproxy;
-- 
Anthony PERARD

[OSSTEST PATCH 29/36] ts-debian-hvm-install: Increase min guest ram size

[Anthony PERARD]

768 MB for the guest isn't enough anymore for Debian Bookworm, at boot
it print this message:

    Low memory
    ----------

    Entering low memory mode

    This system has relatively little free memory, so it will enter low memory
    mode. Among other things, this means that this program will proceed in English.
    You should set up swap space as soon as possible.
    [Press enter to continue]

So, automatic installation fails. An empiric test shows that the min
was 817M for one particular situation (test-amd64-amd64-xl-qemuu-ovmf-amd64)
but we probably need some leg room, so increase to 1 GB.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-debian-hvm-install | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 44eb3ab1..60c95b37 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -241,7 +241,7 @@ sub prep () {
          if ($host_freemem_mb > $ram_lots * 2 + $ram_minslop) {
               $ram_mb = $ram_lots;
          } else {
-              $ram_mb = 768;
+              $ram_mb = 1024;
          }
     }
     logm("Host has $host_freemem_mb MB free memory, setting guest memory size to $ram_mb MB");
-- 
Anthony PERARD

[OSSTEST PATCH 30/36] bookworm: Extend guest bootloader workaround

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest/Debian.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Osstest/Debian.pm b/Osstest/Debian.pm
index 68f1be60..3545f3fd 100644
--- a/Osstest/Debian.pm
+++ b/Osstest/Debian.pm
@@ -1176,7 +1176,7 @@ END
     logm("\$arch is $arch, \$suite is $suite");
     if ($xopts{PvMenuLst} &&
 	$arch =~ /^arm/ &&
-	$suite =~ /wheezy|jessie|stretch|buster|sid/ ) {
+	$suite =~ /wheezy|jessie|stretch|buster|bookworm|sid/ ) {
 
 	# Debian doesn't currently know what bootloader to install in
 	# a Xen guest on ARM. We install pv-grub-menu above which
-- 
Anthony PERARD

[OSSTEST PATCH 31/36] ts-debian-*-install: Replace dots in hostnames by dashs

[Anthony PERARD]

When running ./ts-debian-di-install, hostname on the command line is
interpreted by the debian installer. As the installer find it to be a
FQDN, it uses part of the hostname as the domain, thus overwriting the
value from the DHCP and from d-i netcfg/get_domain setting (maybe).

But the result is that /etc/resolv.conf contains "search
bookworm.guest.osstest" and can't find an IP for "cache". So the
installation fails.

Also replace ".guest.osstest" in a few other places, even if it
may not be an issue.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-debian-di-install  | 4 ++--
 ts-debian-hvm-install | 2 +-
 ts-debian-install     | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/ts-debian-di-install b/ts-debian-di-install
index d84407cf..06c7e1f4 100755
--- a/ts-debian-di-install
+++ b/ts-debian-di-install
@@ -68,8 +68,8 @@ our $ram_mb= $r{arch} =~ m/^armhf/ ? 768 : 2048;
 our $disk_mb= 10000;
 
 our $guesthost= $gn.
-    ($r{"${gn}_suite"} ? ".".$r{"${gn}_suite"} : "").
-    ".guest.osstest";
+    ($r{"${gn}_suite"} ? "-".$r{"${gn}_suite"} : "").
+    "-guest-osstest";
 our $gho;
 
 sub prep () {
diff --git a/ts-debian-hvm-install b/ts-debian-hvm-install
index 60c95b37..99e9acae 100755
--- a/ts-debian-hvm-install
+++ b/ts-debian-hvm-install
@@ -46,7 +46,7 @@ our $ho= selecthost($whhost);
 our $ram_mb;
 our $disk_mb= 10000;
 
-our $guesthost= "$gn.guest.osstest";
+our $guesthost= "$gn-guest-osstest";
 our $gho;
 
 our ($kernel, $ramdisk);
diff --git a/ts-debian-install b/ts-debian-install
index a737bec9..62db487a 100755
--- a/ts-debian-install
+++ b/ts-debian-install
@@ -34,7 +34,7 @@ our $ram_mb=    512;
 our $swap_mb=  1000;
 our $disk_mb= 10000;
 
-our $guesthost= "$gn.guest.osstest";
+our $guesthost= "$gn-guest-osstest";
 our $gho;
 
 sub prep () {
-- 
Anthony PERARD

[OSSTEST PATCH 32/36] ts-xen-install: Fix bridge setup, ask to copy MAC addr

[Anthony PERARD]

This ask to copy the MAC address from $physif on the bridge.

On Debian Bookworm, when running as a Xen guest for nested tests, the
bridge does get a random MAC address and a different IP address from
DHCP than before setting up the bridge, so the test fails.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 ts-xen-install | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-xen-install b/ts-xen-install
index 3a913fce..645d8a79 100755
--- a/ts-xen-install
+++ b/ts-xen-install
@@ -312,6 +312,9 @@ sub ensurebridge () {
     bridge_ports $physif
     bridge_fd 0
     bridge_stp off
+END
+            $bridgex .= <<END if ($ho->{Suite} !~ m/wheezy|jessie|stretch|buster/);
+    bridge_hw $physif
 END
 	} else {
 	    $iface= $physif;
-- 
Anthony PERARD

[OSSTEST PATCH 33/36] make-flight: Keep using buster for L2 guest in nested tests

[Anthony PERARD]

When starting the installation of the L2 guest, L0 kills L1. Switching
the L2 guest back to Debian Buster works fine, so do that to prevent
regression in the test.

Part of the logs from the host L0:

> domain_crash called from arch/x86/hvm/vmx/vvmx.c:2770
> Domain 3 (vcpu#0) crashed on cpu#4:
> d3v0 vmentry failure (reason 0x80000021): Invalid guest state (2)

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 make-flight | 15 ++++++++++++++-
 mfi-common  |  5 +++--
 2 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/make-flight b/make-flight
index 155a0c1f..d0c950bc 100755
--- a/make-flight
+++ b/make-flight
@@ -360,6 +360,19 @@ do_hvm_debian_nested_tests () {
     xen-4.3-testing)      return;;
   esac
 
+  local l2_runvar
+  case $guestsuite in
+      bookworm)
+          # Bookworm install image lead to a crash of l1, so keep using
+          # Buster's image.
+          l2_runvar=(l2_suite=buster)
+          l2_runvar+=(l2_image=$(usual_debianhvm_image amd64 buster))
+          ;;
+      *)
+          l2_runvar=(l2_image=$(usual_debianhvm_image amd64))
+          ;;
+  esac
+
   for cpuvendor in amd intel; do
 
     job_create_test test-$xenarch$kern-$dom0arch$qemuu_suffix-nested-$cpuvendor \
@@ -368,7 +381,7 @@ do_hvm_debian_nested_tests () {
             l1_vifmodel='e1000'                                    \
             l1_memsize='3072'                                      \
             l1_enable_nestedhvm='true'                             \
-            l2_image=$(usual_debianhvm_image amd64)                \
+            ${l2_runvar[@]}                                        \
             bios=$bios                                             \
             all_hostflags=$most_hostflags,hvm-$cpuvendor
 
diff --git a/mfi-common b/mfi-common
index 6dc39422..bbe714bf 100644
--- a/mfi-common
+++ b/mfi-common
@@ -547,18 +547,19 @@ job_create_test () {
 
 usual_debianhvm_image () {
   local arch=$1; shift
+  local suite=${1:-$guestsuite}
   if [ -n "$DEBIAN_IMAGE_FILE" ]; then
       echo $DEBIAN_IMAGE_FILE
       return
   fi
-  local file=`getconfig DebianImageFile_${guestsuite}_${arch}`
+  local file=`getconfig DebianImageFile_${suite}_${arch}`
   if [ -n "$file" ]; then
       echo $file
       return
   fi
   local ver=$DEBIAN_IMAGE_VERSION
   if [ -z "$ver" ] ; then
-      ver=`getconfig DebianImageVersion_$guestsuite`
+      ver=`getconfig DebianImageVersion_$suite`
   fi
   echo debian-$ver-$arch-CD-1.iso
 }
-- 
Anthony PERARD

Re: [OSSTEST PATCH 33/36] make-flight: Keep using buster for L2 guest in nested tests

[Roger Pau Monné]

On Mon, Mar 18, 2024 at 04:55:42PM +0000, Anthony PERARD wrote:
> When starting the installation of the L2 guest, L0 kills L1. Switching
> the L2 guest back to Debian Buster works fine, so do that to prevent
> regression in the test.
> 
> Part of the logs from the host L0:
> 
> > domain_crash called from arch/x86/hvm/vmx/vvmx.c:2770
> > Domain 3 (vcpu#0) crashed on cpu#4:
> > d3v0 vmentry failure (reason 0x80000021): Invalid guest state (2)

Hm, I guess we need this as otherwise the changes themselves won't
past the self-push gate, but this is just masking a real issue.  I
would be tempted to (after the switch to bookworm has gone in) revert
this and force-push it into osstest, so that the failure is visible.

Thanks, Roger.

Re: [OSSTEST PATCH 33/36] make-flight: Keep using buster for L2 guest in nested tests

[Anthony PERARD]

On Wed, Mar 20, 2024 at 06:24:18PM +0100, Roger Pau Monné wrote:
> On Mon, Mar 18, 2024 at 04:55:42PM +0000, Anthony PERARD wrote:
> > When starting the installation of the L2 guest, L0 kills L1. Switching
> > the L2 guest back to Debian Buster works fine, so do that to prevent
> > regression in the test.
> > 
> > Part of the logs from the host L0:
> > 
> > > domain_crash called from arch/x86/hvm/vmx/vvmx.c:2770
> > > Domain 3 (vcpu#0) crashed on cpu#4:
> > > d3v0 vmentry failure (reason 0x80000021): Invalid guest state (2)
> 
> Hm, I guess we need this as otherwise the changes themselves won't
> past the self-push gate, but this is just masking a real issue.  I
> would be tempted to (after the switch to bookworm has gone in) revert
> this and force-push it into osstest, so that the failure is visible.

That means that we might have to force-push on every branches. Or fix
Xen.

Thanks,

-- 
Anthony PERARD

[OSSTEST PATCH 34/36] Temporally switch "qemu-mainline" branch to Bookworm

[Anthony PERARD]

QEMU doesn't build on buster anymore.

This should be remove once bookworm is the default suite.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 make-flight | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/make-flight b/make-flight
index d0c950bc..6e88cb13 100755
--- a/make-flight
+++ b/make-flight
@@ -49,6 +49,11 @@ case "$xenbranch" in
     ;;
 esac
 
+# QEMU doesn't build on buster anymore, use bookworm for it.
+case "$branch" in
+    qemu-mainline) defsuite="bookworm" ;;
+esac
+
 # Pick default Debian Installer version to correspond to the chosen
 # suite.
 if [ -z "$defdi_version" ] ; then
-- 
Anthony PERARD

Re: [OSSTEST PATCH 34/36] Temporally switch "qemu-mainline" branch to Bookworm

[Roger Pau Monné]

On Mon, Mar 18, 2024 at 04:55:43PM +0000, Anthony PERARD wrote:
> QEMU doesn't build on buster anymore.
> 
> This should be remove once bookworm is the default suite.

Is this change required anymore, patch 35 makes bookworm the default,
hence it seems pointless to switch QEMU just one patch ahead.

Thanks, Roger.

Re: [OSSTEST PATCH 34/36] Temporally switch "qemu-mainline" branch to Bookworm

[Anthony PERARD]

On Wed, Mar 20, 2024 at 06:25:35PM +0100, Roger Pau Monné wrote:
> On Mon, Mar 18, 2024 at 04:55:43PM +0000, Anthony PERARD wrote:
> > QEMU doesn't build on buster anymore.
> > 
> > This should be remove once bookworm is the default suite.
> 
> Is this change required anymore, patch 35 makes bookworm the default,
> hence it seems pointless to switch QEMU just one patch ahead.

You didn't read the cover letter?

But I guess I can skip this patch, it's not that important.

Cheers,

-- 
Anthony PERARD

[OSSTEST PATCH 35/36] Switch to Debian Bookworm as default suite

[Anthony PERARD]

Xen 4.17 doesn't build with Debian Bookworm. It fails to build
IPXE/etherboot, on "amd64". So we keep using Debian Buster on Xen 4.18
and earlier branches. Xen 4.18 builds 4.17 via job "build-amd64-prev".

Xen 4.17 would needs 18a36b4a9b08 ("tools: ipxe: update for fixing
build with GCC12") which is only in Xen 4.18.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 Osstest.pm  | 2 +-
 make-flight | 6 +-----
 2 files changed, 2 insertions(+), 6 deletions(-)

diff --git a/Osstest.pm b/Osstest.pm
index a559ca4e..9ac2dc5c 100644
--- a/Osstest.pm
+++ b/Osstest.pm
@@ -87,7 +87,7 @@ our %c = qw(
 
     Images images
 
-    DebianSuite buster
+    DebianSuite bookworm
     DebianMirrorSubpath debian
 
     TestHostKeypairPath id_rsa_osstest
diff --git a/make-flight b/make-flight
index 6e88cb13..0b528629 100755
--- a/make-flight
+++ b/make-flight
@@ -43,17 +43,13 @@ case "$xenbranch" in
   xen-4.3-testing) defsuite="wheezy"; defguestsuite="wheezy";;
   xen-4.9-testing)  defsuite="stretch"; defguestsuite="stretch";;
   xen-4.10-testing) defsuite="stretch"; defguestsuite="stretch";;
+  xen-4.1[1-8]-testing) defsuite="buster"; defguestsuite="buster";;
   *)
     defsuite=`getconfig DebianSuite`
     defguestsuite=`getconfig GuestDebianSuite`
     ;;
 esac
 
-# QEMU doesn't build on buster anymore, use bookworm for it.
-case "$branch" in
-    qemu-mainline) defsuite="bookworm" ;;
-esac
-
 # Pick default Debian Installer version to correspond to the chosen
 # suite.
 if [ -z "$defdi_version" ] ; then
-- 
Anthony PERARD

Re: [OSSTEST PATCH 35/36] Switch to Debian Bookworm as default suite

[Roger Pau Monné]

On Mon, Mar 18, 2024 at 04:55:44PM +0000, Anthony PERARD wrote:
> Xen 4.17 doesn't build with Debian Bookworm. It fails to build
> IPXE/etherboot, on "amd64". So we keep using Debian Buster on Xen 4.18
> and earlier branches. Xen 4.18 builds 4.17 via job "build-amd64-prev".
> 
> Xen 4.17 would needs 18a36b4a9b08 ("tools: ipxe: update for fixing
> build with GCC12") which is only in Xen 4.18.

I would request the backport to 4.17.

Thanks, Roger.

[OSSTEST PATCH 36/36] make-hosts-flight: default to bookworm

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 make-hosts-flight | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/make-hosts-flight b/make-hosts-flight
index 63ac7b71..0177f605 100755
--- a/make-hosts-flight
+++ b/make-hosts-flight
@@ -26,7 +26,7 @@ blessing=$4
 buildflight=$5
 
 : ${ALL_ARCHES:=amd64 i386 arm64 armhf}
-: ${ALL_SUITES:=buster stretch jessie}
+: ${ALL_SUITES:=bookworm buster stretch jessie}
 # ^ most preferred suite first
 
 : ${PERHOST_MAXWAIT:=20000} # seconds
-- 
Anthony PERARD

Re: [OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Roger Pau Monné]

On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> Patch series available in this git branch:
> https://xenbits.xen.org/git-http/people/aperard/osstest.git br.bookworm-v1
> 
> Hi,
> 
> I intend to push this series in two waves.
> 
> First, push up to commit "Temporally switch "qemu-mainline" branch to
> Bookworm". This is to test that osstest still works fine if we need to use
> "buster" for a branch. Also upstream QEMU doesn't build on buster anymore, so
> I've included a commit to use bookworm for it.
> 
> Second, push the remaning two patches, at least a week later, which will switch
> the default debian suite.
> 
> I did run many tests, but hopefully nothing breaks.
> 
> Cheers,
> 
> Anthony PERARD (36):
>   production-config: Add bookworm debian install media filename
>   ts-xen-build-prep: Only force git protocol v2 on buster
>   mgi-common: Fix fetch_debian_package error message
>   mg-debian-installer-update: Download non-free firmware from new repo.
>   ts-host-install: fix ntp.conf path on bookworm
>   ts-host-install: fix ntp server setting
>   ts-host-install: Restart ntp service
>   preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI
>     installation
>   preseed_create: osstest-erase-other-disks: workaround creating
>     /dev/sdXD files
>   preseed_create: Workaround fail grub-install on arndale
>   ts-host-install,preseed_create: Do lvm vgextend at install time
>   di_installcmdline_core: Add link_wait_timeout to install cmdline
>   Disable persistent net generator on Bookworm
>   preseed_base, ts-host-install: Change NIC NamePolicy to "mac"
>   ts-xen-build-prep: Change package selection for Bookworm
>   bl_getmenu_open: Read grub.cfg as root
>   target_editfile: Use the same user to retrieve and send
>   ts-xen-install: remove "libc6-xen" package installation
>   overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
>   overlay-bookworm: 20_linux_xen: Fix XSM entries generation
>   ts-xtf-install: Install python symlink
>   setupboot_grub2: Parse arm64 uefi grub verbes
>   bookworm: Extend ARM clock workaround
>   ts-nested-setup, setup l1 lvm volume groupe in guest
>   ts-leak-check: add new name for udevd workers
>   ts-debian-hvm-install: Allow udev failure in install media
>   ts-debian-fixup: Fix nic names for bookworm
>   ts-debian-install: keep avoiding to use pygrub
>   ts-debian-hvm-install: Increase min guest ram size
>   bookworm: Extend guest bootloader workaround
>   ts-debian-*-install: Replace dots in hostnames by dashs
>   ts-xen-install: Fix bridge setup, ask to copy MAC addr
>   make-flight: Keep using buster for L2 guest in nested tests
>   Temporally switch "qemu-mainline" branch to Bookworm
>   Switch to Debian Bookworm as default suite
>   make-hosts-flight: default to bookworm

Acked-by: Roger Pau Monné <roger.pau@citrix.com>

I just had a look over the commit messages mostly, as I'm not very
much into perl as to provide more accurate reviews, sorry.  LGTM, just
some comments/questions on a few patches.

Thanks, Roger.

Re: [OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Anthony PERARD]

On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> I intend to push this series in two waves.
> 
> First, push up to commit "Temporally switch "qemu-mainline" branch to
> Bookworm". This is to test that osstest still works fine if we need to use
> "buster" for a branch. Also upstream QEMU doesn't build on buster anymore, so
> I've included a commit to use bookworm for it.
> 
> Second, push the remaning two patches, at least a week later, which will switch
> the default debian suite.
> 
> Anthony PERARD (36):
>   production-config: Add bookworm debian install media filename
>   ts-xen-build-prep: Only force git protocol v2 on buster
>   mgi-common: Fix fetch_debian_package error message
>   mg-debian-installer-update: Download non-free firmware from new repo.
>   ts-host-install: fix ntp.conf path on bookworm
>   ts-host-install: fix ntp server setting
>   ts-host-install: Restart ntp service
>   preseed_create: Use new "d-i grub-installer/update-nvram" for UEFI
>     installation
>   preseed_create: osstest-erase-other-disks: workaround creating
>     /dev/sdXD files
>   preseed_create: Workaround fail grub-install on arndale
>   ts-host-install,preseed_create: Do lvm vgextend at install time
>   di_installcmdline_core: Add link_wait_timeout to install cmdline
>   Disable persistent net generator on Bookworm
>   preseed_base, ts-host-install: Change NIC NamePolicy to "mac"
>   ts-xen-build-prep: Change package selection for Bookworm
>   bl_getmenu_open: Read grub.cfg as root
>   target_editfile: Use the same user to retrieve and send
>   ts-xen-install: remove "libc6-xen" package installation
>   overlay-bookworm: Import grub's 20_linux_xen from Debian Bookworm
>   overlay-bookworm: 20_linux_xen: Fix XSM entries generation
>   ts-xtf-install: Install python symlink
>   setupboot_grub2: Parse arm64 uefi grub verbes
>   bookworm: Extend ARM clock workaround
>   ts-nested-setup, setup l1 lvm volume groupe in guest
>   ts-leak-check: add new name for udevd workers
>   ts-debian-hvm-install: Allow udev failure in install media
>   ts-debian-fixup: Fix nic names for bookworm
>   ts-debian-install: keep avoiding to use pygrub
>   ts-debian-hvm-install: Increase min guest ram size
>   bookworm: Extend guest bootloader workaround
>   ts-debian-*-install: Replace dots in hostnames by dashs
>   ts-xen-install: Fix bridge setup, ask to copy MAC addr
>   make-flight: Keep using buster for L2 guest in nested tests

I've pushed up to here. I'll push the last two patches later.

>   Temporally switch "qemu-mainline" branch to Bookworm

And I'll drop this one.

>   Switch to Debian Bookworm as default suite
>   make-hosts-flight: default to bookworm


Cheers,

-- 
Anthony PERARD

Re: [OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Anthony PERARD]

On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:

I've now pushed
"production-config: Set Bookworm's debian-installer version"

and these two:

> >   Switch to Debian Bookworm as default suite
> >   make-hosts-flight: default to bookworm

osstest should start to use Debian Bookworm soon.

Cheers,

-- 
Anthony PERARD

Re: [OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Anthony PERARD]

On Fri, Apr 05, 2024 at 04:54:30PM +0100, Anthony PERARD wrote:
> On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> > On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> 
> I've now pushed
> "production-config: Set Bookworm's debian-installer version"
> 
> and these two:
> 
> > >   Switch to Debian Bookworm as default suite
> > >   make-hosts-flight: default to bookworm
> 
> osstest should start to use Debian Bookworm soon.

osstest failed it's own push gate, I didn't notice that the Arm* builds
was still using an old kernel (from our linux-arm-xen branch) instead of
6.1. So I've rewind this push and pushed an other fix instead:
https://lore.kernel.org/xen-devel/20240408092542.36711-1-anthony.perard@citrix.com/

Cheers,

-- 
Anthony PERARD

Re: [OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Anthony PERARD]

On Mon, Apr 08, 2024 at 10:29:08AM +0100, Anthony PERARD wrote:
> On Fri, Apr 05, 2024 at 04:54:30PM +0100, Anthony PERARD wrote:
> > On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> > > On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> > 
> > I've now pushed
> > "production-config: Set Bookworm's debian-installer version"
> > 
> > and these two:
> > 
> > > >   Switch to Debian Bookworm as default suite
> > > >   make-hosts-flight: default to bookworm
> > 
> > osstest should start to use Debian Bookworm soon.
> 
> osstest failed it's own push gate, I didn't notice that the Arm* builds
> was still using an old kernel (from our linux-arm-xen branch) instead of
> 6.1. So I've rewind this push and pushed an other fix instead:
> https://lore.kernel.org/xen-devel/20240408092542.36711-1-anthony.perard@citrix.com/

I've pushed the switch to Debian Bookworm again, and this time it when
through.

osstest have started to use Bookworm for testing.

Cheers,

-- 
Anthony PERARD

Re: [OSSTEST PATCH 00/36] Switch to Debian Bookworm

[Roger Pau Monné]

On Fri, Apr 19, 2024 at 09:34:21AM +0100, Anthony PERARD wrote:
> On Mon, Apr 08, 2024 at 10:29:08AM +0100, Anthony PERARD wrote:
> > On Fri, Apr 05, 2024 at 04:54:30PM +0100, Anthony PERARD wrote:
> > > On Thu, Mar 28, 2024 at 05:54:04PM +0000, Anthony PERARD wrote:
> > > > On Mon, Mar 18, 2024 at 04:55:09PM +0000, Anthony PERARD wrote:
> > > 
> > > I've now pushed
> > > "production-config: Set Bookworm's debian-installer version"
> > > 
> > > and these two:
> > > 
> > > > >   Switch to Debian Bookworm as default suite
> > > > >   make-hosts-flight: default to bookworm
> > > 
> > > osstest should start to use Debian Bookworm soon.
> > 
> > osstest failed it's own push gate, I didn't notice that the Arm* builds
> > was still using an old kernel (from our linux-arm-xen branch) instead of
> > 6.1. So I've rewind this push and pushed an other fix instead:
> > https://lore.kernel.org/xen-devel/20240408092542.36711-1-anthony.perard@citrix.com/
> 
> I've pushed the switch to Debian Bookworm again, and this time it when
> through.
> 
> osstest have started to use Bookworm for testing.

Thank you very much for doing this.

Roger.

[OSSTEST PATCH 37/36] production-config: Set Bookworm's debian-installer version

[Anthony PERARD]

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---

I've got this one more patch to add to the series, which I forgot to
write. I'll put it before "Switch to Debian Bookworm as default suite"
when I'll push the rest of the series.

 production-config | 1 +
 1 file changed, 1 insertion(+)

diff --git a/production-config b/production-config
index 6345c40c..0eb8f0f3 100644
--- a/production-config
+++ b/production-config
@@ -92,6 +92,7 @@ TftpDiVersion_wheezy 2016-06-08
 TftpDiVersion_jessie 2018-06-26
 TftpDiVersion_stretch 2020-09-24
 TftpDiVersion_buster 2023-06-20
+TftpDiVersion_bookworm 2024-03-26
 
 DebianSnapshotBackports_jessie http://snapshot.debian.org/archive/debian/20190206T211314Z/
 
-- 
Anthony PERARD