All of lore.kernel.org
 help / color / mirror / Atom feed
* [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix
@ 2017-03-24 18:59 Géza Gémes
  2017-03-24 18:59 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
                   ` (10 more replies)
  0 siblings, 11 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

First patch moves python2.7/site-packages/* to python2.7/dist-packages/
at debian package build, this fixes pygrub imports.

Cirros tests are also introduced by this series of patches.
Changes since the previous patchsets:
 - Downloaded cirros components are kept in a separate folder
 - Each test sets up its own images, keeping the downloaded
parts intact
 - qemu-img and pvgrub detection moved to common-functions

There is one patch which fixes lopartsetup with fdisk from newer util-linux

Géza Gémes (11):
  Fix installation of python libs
  Introduce cirros tests
  Introduce cirros-separate-kernel-pv test
  Introduce cirros-pygrub-pv test
  Introduce cirros-pvgrub2-pv test
  Introduce cirros-qemu-hvm test
  Introduce cirros-minios-stubdom-hvm test
  Introduce cirros-qemu-pvhvm test
  Introduce cirros-minios-stubdom-pvhvm test
  Fix lopartsetup for util-linux >= 2.27.1
  Enable cirros tests in the default config

 defconfig                          |   2 +-
 lib/common-functions.sh            |  23 +++++++
 lib/common-tests.sh                | 126 +++++++++++++++++++++++++++++++++++++
 scripts/lopartsetup                |  18 +++++-
 scripts/mkdeb                      |   3 +
 tests-configs/config-cirros_x86_32 |  13 ++++
 tests-configs/config-cirros_x86_64 |  13 ++++
 tests/cirros-minios-stubdom-hvm    |  26 ++++++++
 tests/cirros-minios-stubdom-pvhvm  |  27 ++++++++
 tests/cirros-pvgrub2-pv            |  24 +++++++
 tests/cirros-pygrub-pv             |  24 +++++++
 tests/cirros-qemu-hvm              |  25 ++++++++
 tests/cirros-qemu-pvhvm            |  26 ++++++++
 tests/cirros-separate-kernel-pv    |  26 ++++++++
 tests/series                       |   7 +++
 15 files changed, 381 insertions(+), 2 deletions(-)
 create mode 100644 tests-configs/config-cirros_x86_32
 create mode 100644 tests-configs/config-cirros_x86_64
 create mode 100644 tests/cirros-minios-stubdom-hvm
 create mode 100644 tests/cirros-minios-stubdom-pvhvm
 create mode 100644 tests/cirros-pvgrub2-pv
 create mode 100644 tests/cirros-pygrub-pv
 create mode 100644 tests/cirros-qemu-hvm
 create mode 100644 tests/cirros-qemu-pvhvm
 create mode 100644 tests/cirros-separate-kernel-pv

-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH 01/11] Fix installation of python libs
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-25  0:12   ` Stefano Stabellini
  2017-03-24 18:59 ` [PATCH 02/11] Introduce cirros tests Géza Gémes
                   ` (9 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

Change deb package build in order to move the site-packages
folder to dist-packages in this way having the libs inluded
in the default PYTHONPATH

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 scripts/mkdeb | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/scripts/mkdeb b/scripts/mkdeb
index 3796300..fcaf757 100755
--- a/scripts/mkdeb
+++ b/scripts/mkdeb
@@ -46,6 +46,9 @@ then
     rm -rf deb/usr/lib64
 fi
 
+# Make sure that python libs will be in the path
+mv deb/usr/lib/python2.7/site-packages deb/usr/lib/python2.7/dist-packages
+
 # Fill in the debian boilerplate
 mkdir -p deb/DEBIAN
 cat >deb/DEBIAN/control <<EOF
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 02/11] Introduce cirros tests
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
  2017-03-24 18:59 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 21:01   ` Stefano Stabellini
  2017-03-24 18:59 ` [PATCH 03/11] Introduce cirros-separate-kernel-pv test Géza Gémes
                   ` (8 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

Add support for using cirros images in raisin tests

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 lib/common-functions.sh            |  23 +++++++
 lib/common-tests.sh                | 126 +++++++++++++++++++++++++++++++++++++
 tests-configs/config-cirros_x86_32 |  13 ++++
 tests-configs/config-cirros_x86_64 |  13 ++++
 4 files changed, 175 insertions(+)
 create mode 100644 tests-configs/config-cirros_x86_32
 create mode 100644 tests-configs/config-cirros_x86_64

diff --git a/lib/common-functions.sh b/lib/common-functions.sh
index d4476f3..d82d7ab 100644
--- a/lib/common-functions.sh
+++ b/lib/common-functions.sh
@@ -439,3 +439,26 @@ function uninstall_package() {
         error_echo "Don't know how to uninstall packages on $DISTRO"
     fi
 }
+
+function get-qemu-img() {
+    set +e
+    QEMU_IMG=`which qemu-img`
+    set -e
+    if [[ -z "$QEMU_IMG" ]]
+    then
+        QEMU_IMG="/usr/lib/xen/bin/qemu-img"
+    fi
+    export QEMU_IMG
+}
+
+function get-pvgrub() {
+    ARCH=$1
+    set +e
+    PVGRUB=`which grub-${ARCH}-xen`
+    set -e
+    if [[ -z "$PVGRUB" ]]
+    then
+        PVGRUB="/usr/lib/xen/boot/grub-${ARCH}-xen"
+    fi
+    export PVGRUB
+}
diff --git a/lib/common-tests.sh b/lib/common-tests.sh
index d346af4..1deaf4f 100644
--- a/lib/common-tests.sh
+++ b/lib/common-tests.sh
@@ -178,3 +178,129 @@ function get_host_initrd() {
         exit 1
     fi
 }
+
+function cirros_network_init() {
+    rootdir=$1
+    ifile=`mktemp`
+    # Create static network config
+    cat >$ifile <<EOF
+auto lo
+iface lo inet loopback
+
+auto eth0
+iface eth0 inet static
+    address 169.254.0.2
+    network 169.254.0.0
+    broadcast 169.254.0.255
+    netmask 255.255.255.0
+EOF
+    $SUDO mv $ifile $rootdir/etc/network/interfaces
+    # Disable cloud-init
+    $SUDO rm -f ${rootdir}/etc/rc3.d/S*cirros*ds*
+    $SUDO rm -f ${rootdir}/etc/rc3.d/S*-cirros-userdata
+}
+
+function get_cirros_kernel() {
+    bootdir=$1
+    basename `find $bootdir -name vmlinuz* 2>/dev/null | head -1`
+}
+
+function get_cirros_initrd() {
+    bootdir=$1
+    basename `find $bootdir -name initrd* 2>/dev/null | head -1`
+}
+
+function cirros_grub_cfg() {
+    rootdir=$1
+    grubroot="`echo $CIRROS_GRUB_CFG | cut -d ')' -f 1`)"
+    grubcfg="`echo $CIRROS_GRUB_CFG | cut -d ')' -f 2`"
+    grubdir=`dirname $grubcfg`
+    bootdir=`dirname $grubdir`
+    tmpgrubcfg=`mktemp`
+    cat > $tmpgrubcfg <<EOF
+root="$grubroot"
+insmod xzio
+insmod gzio
+insmod btrfs
+insmod ext2
+set timeout=1
+set default=0
+menuentry Cirros {
+    linux `echo $bootdir`/`get_cirros_kernel ${rootdir}/${bootdir}` root=/dev/xvda1 ro
+    initrd `echo $bootdir`/`get_cirros_initrd ${rootdir}/${bootdir}`
+}
+EOF
+    $SUDO mv $tmpgrubcfg ${rootdir}/${grubcfg}
+}
+
+function download_cirros_components() {
+    verbose_echo "Downloading cirros components"
+    . tests-configs/config-cirros_$RAISIN_ARCH
+    mkdir -p $CIRROS_DOWNLOADS
+    wget -q -c $CIRROS_KERNEL_URL -P $CIRROS_DOWNLOADS
+    wget -q -c $CIRROS_INITRD_URL -P $CIRROS_DOWNLOADS
+    wget -q -c $CIRROS_ROOTFS_URL -P $CIRROS_DOWNLOADS
+    wget -q -c $CIRROS_DISK_URL -P $CIRROS_DOWNLOADS
+}
+
+function set_up_cirros_kernel_initrd() {
+    local testdir=$1
+    mkdir -p $testdir
+    download_cirros_components
+    verbose_echo "Setting up kernel and initrd for cirros test"
+    cp $CIRROS_DOWNLOADS/$CIRROS_KERNEL_FILE $testdir
+    cp $CIRROS_DOWNLOADS/$CIRROS_INITRD_FILE $testdir
+}
+
+function set_up_cirros_rootfs() {
+    local testdir=$1
+    mkdir -p $testdir
+    download_cirros_components
+    verbose_echo "Setting up rootfs for cirros test"
+    cp $CIRROS_DOWNLOADS/$CIRROS_ROOTFS_FILE.gz $testdir
+    gunzip $testdir/$CIRROS_ROOTFS_FILE.gz
+    local cirros_rootfs_loop=`create_loop $testdir/$CIRROS_ROOTFS_FILE`
+    local cirros_rootfs_mntpt=`mktemp -d`
+    $SUDO mount $cirros_rootfs_loop $cirros_rootfs_mntpt
+    cirros_network_init $cirros_rootfs_mntpt
+    $SUDO umount $cirros_rootfs_mntpt
+    $SUDO rmdir $cirros_rootfs_mntpt
+    $SUDO losetup -d $cirros_rootfs_loop
+}
+
+function set_up_cirros_disk() {
+    local testdir=$1
+    mkdir -p $testdir
+    local grub_install=False
+    if [[ $# -gt 1 ]]
+    then
+        grub_install=$2
+    else
+        unset grub_install
+    fi
+    download_cirros_components
+    verbose_echo "Setting up disk for cirros test"
+    $QEMU_IMG convert -f qcow2 -O raw $CIRROS_DOWNLOADS/$CIRROS_DISK_FILE $testdir/$CIRROS_DISK_FILE
+    local cirros_disk_loop=`$SUDO $BASEDIR/scripts/lopartsetup $testdir/$CIRROS_DISK_FILE | head -1 | cut -d ":" -f 1`
+    local cirros_disk_mntpt=`mktemp -d`
+    $SUDO mount $cirros_disk_loop $cirros_disk_mntpt
+    cirros_network_init $cirros_disk_mntpt
+    if [[ -n $grub_install ]]
+    then
+        cirros_grub_cfg $cirros_disk_mntpt
+    fi
+    $SUDO umount $cirros_disk_mntpt
+    $SUDO rmdir $cirros_disk_mntpt
+    $SUDO losetup -d $cirros_disk_loop
+}
+
+function tear_down_cirros_test() {
+    testdir=$1
+    if [[ `$SUDO xl vm-list | grep -e "raisin-test" | wc -l` -gt 0 ]]
+    then
+        $SUDO xl destroy "raisin-test"
+    fi
+    verbose_echo "$PREPEND deleting environment of cirros test"
+    $SUDO rm -rf $testdir
+}
+
diff --git a/tests-configs/config-cirros_x86_32 b/tests-configs/config-cirros_x86_32
new file mode 100644
index 0000000..5402415
--- /dev/null
+++ b/tests-configs/config-cirros_x86_32
@@ -0,0 +1,13 @@
+CIRROS_ARCH=i386
+CIRROS_BASE_URL="https://download.cirros-cloud.net/"
+CIRROS_VERSION="0.3.5"
+CIRROS_DOWNLOADS=$BASEDIR/downloads
+CIRROS_KERNEL_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-kernel
+CIRROS_INITRD_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-initramfs
+CIRROS_ROOTFS_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img
+CIRROS_DISK_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img
+CIRROS_KERNEL_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_KERNEL_FILE}
+CIRROS_INITRD_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_INITRD_FILE}
+CIRROS_ROOTFS_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_ROOTFS_FILE}.gz
+CIRROS_DISK_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_DISK_FILE}
+CIRROS_GRUB_CFG="(xen/xvda,msdos1)/boot/grub/grub.cfg"
diff --git a/tests-configs/config-cirros_x86_64 b/tests-configs/config-cirros_x86_64
new file mode 100644
index 0000000..72cf489
--- /dev/null
+++ b/tests-configs/config-cirros_x86_64
@@ -0,0 +1,13 @@
+CIRROS_ARCH=x86_64
+CIRROS_BASE_URL="https://download.cirros-cloud.net/"
+CIRROS_VERSION="0.3.5"
+CIRROS_DOWNLOADS=$BASEDIR/downloads
+CIRROS_KERNEL_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-kernel
+CIRROS_INITRD_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-initramfs
+CIRROS_ROOTFS_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img
+CIRROS_DISK_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img
+CIRROS_KERNEL_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_KERNEL_FILE}
+CIRROS_INITRD_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_INITRD_FILE}
+CIRROS_ROOTFS_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_ROOTFS_FILE}.gz
+CIRROS_DISK_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_DISK_FILE}
+CIRROS_GRUB_CFG="(xen/xvda,msdos1)/boot/grub/grub.cfg"
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 03/11] Introduce cirros-separate-kernel-pv test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
  2017-03-24 18:59 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
  2017-03-24 18:59 ` [PATCH 02/11] Introduce cirros tests Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-25  0:17   ` Stefano Stabellini
  2017-03-24 18:59 ` [PATCH 04/11] Introduce cirros-pygrub-pv test Géza Gémes
                   ` (7 subsequent siblings)
  10 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test is the cirros equivalent of the bussybox-pv test

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-separate-kernel-pv | 26 ++++++++++++++++++++++++++
 tests/series                    |  1 +
 2 files changed, 27 insertions(+)
 create mode 100644 tests/cirros-separate-kernel-pv

diff --git a/tests/cirros-separate-kernel-pv b/tests/cirros-separate-kernel-pv
new file mode 100644
index 0000000..9432481
--- /dev/null
+++ b/tests/cirros-separate-kernel-pv
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-separate-kernel-pv-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-separate-kernel-pv-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_kernel_initrd $testdir
+    set_up_cirros_rootfs $testdir
+    cat >$testdir/cirros-separate-kernel-pv.cfg <<EOF
+name = "raisin-test"
+memory = 128
+vcpus = 2
+kernel = "$testdir/$CIRROS_KERNEL_FILE"
+ramdisk = "$testdir/$CIRROS_INITRD_FILE"
+disk = [ '$testdir/$CIRROS_ROOTFS_FILE,raw,xvda,rw' ]
+extra = "root=/dev/xvda"
+vif = [ 'bridge=xenbr1' ]
+EOF
+
+    $SUDO xl create $testdir/cirros-separate-kernel-pv.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index a2f4b34..38ee35a 100644
--- a/tests/series
+++ b/tests/series
@@ -1,3 +1,4 @@
 busybox-pv
 busybox-hvm
 busybox-hvm-migrate
+cirros-separate-kernel-pv
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 04/11] Introduce cirros-pygrub-pv test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (2 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 03/11] Introduce cirros-separate-kernel-pv test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 18:59 ` [PATCH 05/11] Introduce cirros-pvgrub2-pv test Géza Gémes
                   ` (6 subsequent siblings)
  10 siblings, 0 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test verifies booting the cirros image using pygrub

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-pygrub-pv | 24 ++++++++++++++++++++++++
 tests/series           |  1 +
 2 files changed, 25 insertions(+)
 create mode 100644 tests/cirros-pygrub-pv

diff --git a/tests/cirros-pygrub-pv b/tests/cirros-pygrub-pv
new file mode 100644
index 0000000..149501f
--- /dev/null
+++ b/tests/cirros-pygrub-pv
@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-pygrub-pv-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-pygrub-pv-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_disk $testdir
+    cat >$testdir/cirros-pygrub-pv.cfg <<EOF
+name = "raisin-test"
+memory = 128
+vcpus = 2
+bootloader = "pygrub"
+disk = [ '$testdir/$CIRROS_DISK_FILE,raw,xvda,rw' ]
+extra = "root=/dev/xvda1"
+vif = [ 'bridge=xenbr1' ]
+EOF
+
+    $SUDO xl create $testdir/cirros-pygrub-pv.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index 38ee35a..372fafb 100644
--- a/tests/series
+++ b/tests/series
@@ -2,3 +2,4 @@ busybox-pv
 busybox-hvm
 busybox-hvm-migrate
 cirros-separate-kernel-pv
+cirros-pygrub-pv
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 05/11] Introduce cirros-pvgrub2-pv test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (3 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 04/11] Introduce cirros-pygrub-pv test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 18:59 ` [PATCH 06/11] Introduce cirros-qemu-hvm test Géza Gémes
                   ` (5 subsequent siblings)
  10 siblings, 0 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test verifies booting the cirros image using pvgrub2

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-pvgrub2-pv | 24 ++++++++++++++++++++++++
 tests/series            |  1 +
 2 files changed, 25 insertions(+)
 create mode 100644 tests/cirros-pvgrub2-pv

diff --git a/tests/cirros-pvgrub2-pv b/tests/cirros-pvgrub2-pv
new file mode 100644
index 0000000..f48617e
--- /dev/null
+++ b/tests/cirros-pvgrub2-pv
@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-pvgrub2-pv-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-pvgrub2-pv-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_disk $testdir grub_install
+    cat >$testdir/cirros-pvgrub2-pv.cfg <<EOF
+name = "raisin-test"
+memory = 128
+vcpus = 2
+kernel = "$PVGRUB"
+disk = [ '${testdir}/${CIRROS_DISK_FILE},raw,xvda,rw' ]
+extra = "$CIRROS_GRUB_CFG"
+vif = [ 'bridge=xenbr1' ]
+EOF
+
+    $SUDO xl create $testdir/cirros-pvgrub2-pv.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index 372fafb..1fec245 100644
--- a/tests/series
+++ b/tests/series
@@ -3,3 +3,4 @@ busybox-hvm
 busybox-hvm-migrate
 cirros-separate-kernel-pv
 cirros-pygrub-pv
+cirros-pvgrub2-pv
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 06/11] Introduce cirros-qemu-hvm test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (4 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 05/11] Introduce cirros-pvgrub2-pv test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 18:59 ` [PATCH 07/11] Introduce cirros-minios-stubdom-hvm test Géza Gémes
                   ` (4 subsequent siblings)
  10 siblings, 0 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test is the cirros equivalent of the bussybox-hvm test

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-qemu-hvm | 25 +++++++++++++++++++++++++
 tests/series          |  1 +
 2 files changed, 26 insertions(+)
 create mode 100644 tests/cirros-qemu-hvm

diff --git a/tests/cirros-qemu-hvm b/tests/cirros-qemu-hvm
new file mode 100644
index 0000000..7a2be1b
--- /dev/null
+++ b/tests/cirros-qemu-hvm
@@ -0,0 +1,25 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-qemu-hvm-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-qemu-hvm-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_disk $testdir
+    cat >$testdir/cirros-qemu-hvm.cfg <<EOF
+name = "raisin-test"
+builder = "hvm"
+memory = 128
+vcpus = 2
+disk = [ '$testdir/$CIRROS_DISK_FILE,raw,hda,rw' ]
+vif = [ 'bridge=xenbr1' ]
+serial="pty"
+boot="c"
+EOF
+
+    $SUDO xl create $testdir/cirros-qemu-hvm.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index 1fec245..49a9257 100644
--- a/tests/series
+++ b/tests/series
@@ -4,3 +4,4 @@ busybox-hvm-migrate
 cirros-separate-kernel-pv
 cirros-pygrub-pv
 cirros-pvgrub2-pv
+cirros-qemu-hvm
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 07/11] Introduce cirros-minios-stubdom-hvm test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (5 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 06/11] Introduce cirros-qemu-hvm test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 18:59 ` [PATCH 08/11] Introduce cirros-qemu-pvhvm test Géza Gémes
                   ` (3 subsequent siblings)
  10 siblings, 0 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test verifies a cirros based hvm domain using a minios based
stubdom as its device model

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-minios-stubdom-hvm | 26 ++++++++++++++++++++++++++
 tests/series                    |  1 +
 2 files changed, 27 insertions(+)
 create mode 100644 tests/cirros-minios-stubdom-hvm

diff --git a/tests/cirros-minios-stubdom-hvm b/tests/cirros-minios-stubdom-hvm
new file mode 100644
index 0000000..614b487
--- /dev/null
+++ b/tests/cirros-minios-stubdom-hvm
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-minios-stubdom-hvm-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-minios-stubdom-hvm-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_disk $testdir
+    cat >$testdir/cirros-minios-stubdom-hvm.cfg <<EOF
+name = "raisin-test"
+builder = "hvm"
+device_model_stubdomain_override = 1
+memory = 128
+vcpus = 2
+disk = [ '$testdir/$CIRROS_DISK_FILE,raw,hda,rw' ]
+vif = [ 'bridge=xenbr1' ]
+serial="pty"
+boot="c"
+EOF
+
+    $SUDO xl create $testdir/cirros-minios-stubdom-hvm.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index 49a9257..7b1ad8e 100644
--- a/tests/series
+++ b/tests/series
@@ -5,3 +5,4 @@ cirros-separate-kernel-pv
 cirros-pygrub-pv
 cirros-pvgrub2-pv
 cirros-qemu-hvm
+cirros-minios-stubdom-hvm
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 08/11] Introduce cirros-qemu-pvhvm test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (6 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 07/11] Introduce cirros-minios-stubdom-hvm test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 18:59 ` [PATCH 09/11] Introduce cirros-minios-stubdom-pvhvm test Géza Gémes
                   ` (2 subsequent siblings)
  10 siblings, 0 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test is similar to cirros-qemu-hvm test, differing by
explicitly exposing the xen netfront and blockfront drivers

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-qemu-pvhvm | 26 ++++++++++++++++++++++++++
 tests/series            |  1 +
 2 files changed, 27 insertions(+)
 create mode 100644 tests/cirros-qemu-pvhvm

diff --git a/tests/cirros-qemu-pvhvm b/tests/cirros-qemu-pvhvm
new file mode 100644
index 0000000..bf74529
--- /dev/null
+++ b/tests/cirros-qemu-pvhvm
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-qemu-pvhvm-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-qemu-pvhvm-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_disk $testdir
+    cat >$testdir/cirros-qemu-pvhvm.cfg <<EOF
+name = "raisin-test"
+builder = "hvm"
+xen_platform_pci = 1
+memory = 128
+vcpus = 2
+disk = [ '$testdir/$CIRROS_DISK_FILE,raw,xvda,rw' ]
+vif = [ 'bridge=xenbr1' ]
+serial="pty"
+boot="c"
+EOF
+
+    $SUDO xl create $testdir/cirros-qemu-pvhvm.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index 7b1ad8e..51dbe17 100644
--- a/tests/series
+++ b/tests/series
@@ -6,3 +6,4 @@ cirros-pygrub-pv
 cirros-pvgrub2-pv
 cirros-qemu-hvm
 cirros-minios-stubdom-hvm
+cirros-qemu-pvhvm
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 09/11] Introduce cirros-minios-stubdom-pvhvm test
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (7 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 08/11] Introduce cirros-qemu-pvhvm test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-24 18:59 ` [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1 Géza Gémes
  2017-03-24 18:59 ` [PATCH 11/11] Enable cirros tests in the default config Géza Gémes
  10 siblings, 0 replies; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

This test is similar to cirros-minios-stubdom-hvm test, differing by
explicitly exposing the xen netfront and blockfront drivers

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 tests/cirros-minios-stubdom-pvhvm | 27 +++++++++++++++++++++++++++
 tests/series                      |  1 +
 2 files changed, 28 insertions(+)
 create mode 100644 tests/cirros-minios-stubdom-pvhvm

diff --git a/tests/cirros-minios-stubdom-pvhvm b/tests/cirros-minios-stubdom-pvhvm
new file mode 100644
index 0000000..cff4863
--- /dev/null
+++ b/tests/cirros-minios-stubdom-pvhvm
@@ -0,0 +1,27 @@
+#!/usr/bin/env bash
+
+set -e
+
+function cirros-minios-stubdom-pvhvm-cleanup() {
+    tear_down_cirros_test $testdir
+}
+
+function cirros-minios-stubdom-pvhvm-test() {
+    testdir=`mktemp -d`
+    set_up_cirros_disk $testdir
+    cat >$testdir/cirros-minios-stubdom-pvhvm.cfg <<EOF
+name = "raisin-test"
+builder = "hvm"
+device_model_stubdomain_override = 1
+xen_platform_pci = 1
+memory = 128
+vcpus = 2
+disk = [ '$testdir/$CIRROS_DISK_FILE,raw,xvda,rw' ]
+vif = [ 'bridge=xenbr1' ]
+serial="pty"
+boot="c"
+EOF
+
+    $SUDO xl create $testdir/cirros-minios-stubdom-pvhvm.cfg
+    check_guest_alive
+}
diff --git a/tests/series b/tests/series
index 51dbe17..ffa8e69 100644
--- a/tests/series
+++ b/tests/series
@@ -7,3 +7,4 @@ cirros-pvgrub2-pv
 cirros-qemu-hvm
 cirros-minios-stubdom-hvm
 cirros-qemu-pvhvm
+cirros-minios-stubdom-pvhvm
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (8 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 09/11] Introduce cirros-minios-stubdom-pvhvm test Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-25  0:26   ` Stefano Stabellini
  2017-03-24 18:59 ` [PATCH 11/11] Enable cirros tests in the default config Géza Gémes
  10 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

fdisk from util-linux >= 2.27.1 returns units in a slightly different
structure than earlier versions

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 scripts/lopartsetup | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/scripts/lopartsetup b/scripts/lopartsetup
index 04ce3cc..b675e03 100755
--- a/scripts/lopartsetup
+++ b/scripts/lopartsetup
@@ -53,7 +53,23 @@ then
     exit 1
 fi
 
-unit="`fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | cut -d " " -f 9`"
+unitstring=""
+column=0
+if [ `fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | wc -l` -eq 1 ]
+then
+    unitstring="^Units = "
+    column=9
+else if [ `fdisk -lu $filename 2>/dev/null | grep -e "^Units:" | wc -l` -eq 1 ]
+then
+    unitstring="^Units:"
+    column=8
+else
+    exit 1
+fi
+fi
+
+unit="`fdisk -lu $filename 2>/dev/null | grep -e "$unitstring" | cut -d " " -f $column`"
+
 index=0
 for i in "`fdisk -lu $filename 2>/dev/null | grep -e "^$filename"`"
 do
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* [PATCH 11/11] Enable cirros tests in the default config
  2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
                   ` (9 preceding siblings ...)
  2017-03-24 18:59 ` [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1 Géza Gémes
@ 2017-03-24 18:59 ` Géza Gémes
  2017-03-25  0:08   ` Stefano Stabellini
  10 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-24 18:59 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

The existing cirros tests are enabled, with the following
exceptions:
cirros-minios-stubdom-hvm and cirros-minios-stubdom-pvhvm are
skipped as raisin does not install the stubdom

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 defconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/defconfig b/defconfig
index f8ef398..bdacc7e 100644
--- a/defconfig
+++ b/defconfig
@@ -31,4 +31,4 @@ GIT_TRANSPORT="git"
 # Tests
 ## All tests: busybox-pv busybox-hvm
 ## ENABLED_TESTS is the list of test run by raise test
-ENABLED_TESTS="busybox-pv busybox-hvm"
+ENABLED_TESTS="busybox-pv busybox-hvm cirros-separate-kernel-pv cirros-pygrub-pv cirros-pvgrub2-pv cirros-qemu-hvm cirros-qemu-pvhvm"
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

* Re: [PATCH 02/11] Introduce cirros tests
  2017-03-24 18:59 ` [PATCH 02/11] Introduce cirros tests Géza Gémes
@ 2017-03-24 21:01   ` Stefano Stabellini
  2017-03-25 19:24     ` Gémes Géza
  0 siblings, 1 reply; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-24 21:01 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, sstabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 10060 bytes --]

On Fri, 24 Mar 2017, Géza Gémes wrote:
> Add support for using cirros images in raisin tests
> 
> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>

Thank you! Much, much better, almost ready :)

I have only a couple of suggestions and questions left, see below.


>  lib/common-functions.sh            |  23 +++++++
>  lib/common-tests.sh                | 126 +++++++++++++++++++++++++++++++++++++
>  tests-configs/config-cirros_x86_32 |  13 ++++
>  tests-configs/config-cirros_x86_64 |  13 ++++
>  4 files changed, 175 insertions(+)
>  create mode 100644 tests-configs/config-cirros_x86_32
>  create mode 100644 tests-configs/config-cirros_x86_64
> 
> diff --git a/lib/common-functions.sh b/lib/common-functions.sh
> index d4476f3..d82d7ab 100644
> --- a/lib/common-functions.sh
> +++ b/lib/common-functions.sh
> @@ -439,3 +439,26 @@ function uninstall_package() {
>          error_echo "Don't know how to uninstall packages on $DISTRO"
>      fi
>  }
> +
> +function get-qemu-img() {
> +    set +e
> +    QEMU_IMG=`which qemu-img`
> +    set -e
> +    if [[ -z "$QEMU_IMG" ]]
> +    then
> +        QEMU_IMG="/usr/lib/xen/bin/qemu-img"
> +    fi
> +    export QEMU_IMG

we probably want to test that $QEMU_IMG exists (test -f), and print an
error if it doesn't


> +}
> +
> +function get-pvgrub() {
> +    ARCH=$1
> +    set +e
> +    PVGRUB=`which grub-${ARCH}-xen`
> +    set -e
> +    if [[ -z "$PVGRUB" ]]
> +    then
> +        PVGRUB="/usr/lib/xen/boot/grub-${ARCH}-xen"
> +    fi
> +    export PVGRUB

Same here.

Also, I don't see who is calling get-qemu-img and get-pvgrub in this
patch series.


> +}
> diff --git a/lib/common-tests.sh b/lib/common-tests.sh
> index d346af4..1deaf4f 100644
> --- a/lib/common-tests.sh
> +++ b/lib/common-tests.sh
> @@ -178,3 +178,129 @@ function get_host_initrd() {
>          exit 1
>      fi
>  }
> +
> +function cirros_network_init() {
> +    rootdir=$1
> +    ifile=`mktemp`
> +    # Create static network config
> +    cat >$ifile <<EOF
> +auto lo
> +iface lo inet loopback
> +
> +auto eth0
> +iface eth0 inet static
> +    address 169.254.0.2
> +    network 169.254.0.0
> +    broadcast 169.254.0.255
> +    netmask 255.255.255.0
> +EOF
> +    $SUDO mv $ifile $rootdir/etc/network/interfaces

This is OK. However, we need to to "mv -f" otherwise it might ask for
confirmation before overwriting.


> +    # Disable cloud-init
> +    $SUDO rm -f ${rootdir}/etc/rc3.d/S*cirros*ds*
> +    $SUDO rm -f ${rootdir}/etc/rc3.d/S*-cirros-userdata
> +}
> +
> +function get_cirros_kernel() {
> +    bootdir=$1
> +    basename `find $bootdir -name vmlinuz* 2>/dev/null | head -1`
> +}
> +
> +function get_cirros_initrd() {
> +    bootdir=$1
> +    basename `find $bootdir -name initrd* 2>/dev/null | head -1`
> +}
> +
> +function cirros_grub_cfg() {
> +    rootdir=$1
> +    grubroot="`echo $CIRROS_GRUB_CFG | cut -d ')' -f 1`)"
> +    grubcfg="`echo $CIRROS_GRUB_CFG | cut -d ')' -f 2`"
> +    grubdir=`dirname $grubcfg`
> +    bootdir=`dirname $grubdir`
> +    tmpgrubcfg=`mktemp`
> +    cat > $tmpgrubcfg <<EOF
> +root="$grubroot"
> +insmod xzio
> +insmod gzio
> +insmod btrfs
> +insmod ext2
> +set timeout=1
> +set default=0
> +menuentry Cirros {
> +    linux `echo $bootdir`/`get_cirros_kernel ${rootdir}/${bootdir}` root=/dev/xvda1 ro
> +    initrd `echo $bootdir`/`get_cirros_initrd ${rootdir}/${bootdir}`
> +}
> +EOF
> +    $SUDO mv $tmpgrubcfg ${rootdir}/${grubcfg}

mv -f


> +}
> +
> +function download_cirros_components() {
> +    verbose_echo "Downloading cirros components"
> +    . tests-configs/config-cirros_$RAISIN_ARCH
> +    mkdir -p $CIRROS_DOWNLOADS
> +    wget -q -c $CIRROS_KERNEL_URL -P $CIRROS_DOWNLOADS
> +    wget -q -c $CIRROS_INITRD_URL -P $CIRROS_DOWNLOADS
> +    wget -q -c $CIRROS_ROOTFS_URL -P $CIRROS_DOWNLOADS
> +    wget -q -c $CIRROS_DISK_URL -P $CIRROS_DOWNLOADS

If the downloads are already there, does wget do the right thing?  I
guess it does. Otherwise, do we need to test -f each of the files?


> +}
> +
> +function set_up_cirros_kernel_initrd() {
> +    local testdir=$1
> +    mkdir -p $testdir
> +    download_cirros_components

I would probably move the download_cirros_components call to the test
scripts, for example cirros-separate-kernel-pv-test.


> +    verbose_echo "Setting up kernel and initrd for cirros test"
> +    cp $CIRROS_DOWNLOADS/$CIRROS_KERNEL_FILE $testdir
> +    cp $CIRROS_DOWNLOADS/$CIRROS_INITRD_FILE $testdir
> +}
> +
> +function set_up_cirros_rootfs() {
> +    local testdir=$1
> +    mkdir -p $testdir
> +    download_cirros_components
> +    verbose_echo "Setting up rootfs for cirros test"
> +    cp $CIRROS_DOWNLOADS/$CIRROS_ROOTFS_FILE.gz $testdir
> +    gunzip $testdir/$CIRROS_ROOTFS_FILE.gz

I would leave the unzipped file under $CIRROS_DOWNLOADS, so that we
don't have to unzip the same rootfs file twice.

I would also test for the existance of the unzipped file here and skip
the whole setup if it is already present. So the first time, we unzip
and setup the network interfaces file; the second time we just test that
the unzipped file exists and return. Would that work?


> +    local cirros_rootfs_loop=`create_loop $testdir/$CIRROS_ROOTFS_FILE`
> +    local cirros_rootfs_mntpt=`mktemp -d`
> +    $SUDO mount $cirros_rootfs_loop $cirros_rootfs_mntpt
> +    cirros_network_init $cirros_rootfs_mntpt
> +    $SUDO umount $cirros_rootfs_mntpt
> +    $SUDO rmdir $cirros_rootfs_mntpt
> +    $SUDO losetup -d $cirros_rootfs_loop
> +}
> +
> +function set_up_cirros_disk() {
> +    local testdir=$1
> +    mkdir -p $testdir
> +    local grub_install=False
> +    if [[ $# -gt 1 ]]
> +    then
> +        grub_install=$2
> +    else
> +        unset grub_install
> +    fi

Is this test actually needed? It doesn't look like anything is calling
set_up_cirros_disk with more than one argument.


> +    download_cirros_components
> +    verbose_echo "Setting up disk for cirros test"
> +    $QEMU_IMG convert -f qcow2 -O raw $CIRROS_DOWNLOADS/$CIRROS_DISK_FILE $testdir/$CIRROS_DISK_FILE

Similarly to the rootfs, I would output the conversion to
$CIRROS_DOWNLOADS to avoid doing it twice. The second time the function
gets called, I would just test for the existance of the raw file and
return. (It is probably best to rename the raw file to something like
$CIRROS_DISK_FILE.raw). In addition, if possible, I would also do the
following setup only the first time around. Does it make sense?


> +    local cirros_disk_loop=`$SUDO $BASEDIR/scripts/lopartsetup $testdir/$CIRROS_DISK_FILE | head -1 | cut -d ":" -f 1`
> +    local cirros_disk_mntpt=`mktemp -d`
> +    $SUDO mount $cirros_disk_loop $cirros_disk_mntpt
> +    cirros_network_init $cirros_disk_mntpt
> +    if [[ -n $grub_install ]]
> +    then
> +        cirros_grub_cfg $cirros_disk_mntpt
> +    fi
> +    $SUDO umount $cirros_disk_mntpt
> +    $SUDO rmdir $cirros_disk_mntpt
> +    $SUDO losetup -d $cirros_disk_loop

This is just for you information. If you remove the loop device, it
means that you'll end up using the QEMU userspace disk backend.
Otherwise if you keep the loop device around and use it for the backend,
you end up using the in-kernel disk backend, which is typically faster.
In other words, the following:

  disk = [ '$testdir/$CIRROS_DISK_FILE,raw,hda,rw' ]

ends up using the QEMU disk backend, while the following:

  disk = [ '$cirros_disk_loop,raw,hda,rw' ]

ends up using the kernel disk backend.
Either choice is fine for the tests, I just wanted to make sure you knew :)


> +}
> +
> +function tear_down_cirros_test() {
> +    testdir=$1
> +    if [[ `$SUDO xl vm-list | grep -e "raisin-test" | wc -l` -gt 0 ]]

Why "grep -e"? I think grep is sufficient.


> +    then
> +        $SUDO xl destroy "raisin-test"
> +    fi
> +    verbose_echo "$PREPEND deleting environment of cirros test"
> +    $SUDO rm -rf $testdir
> +}
> +
> diff --git a/tests-configs/config-cirros_x86_32 b/tests-configs/config-cirros_x86_32
> new file mode 100644
> index 0000000..5402415
> --- /dev/null
> +++ b/tests-configs/config-cirros_x86_32
> @@ -0,0 +1,13 @@
> +CIRROS_ARCH=i386
> +CIRROS_BASE_URL="https://download.cirros-cloud.net/"
> +CIRROS_VERSION="0.3.5"
> +CIRROS_DOWNLOADS=$BASEDIR/downloads
> +CIRROS_KERNEL_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-kernel
> +CIRROS_INITRD_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-initramfs
> +CIRROS_ROOTFS_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img
> +CIRROS_DISK_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img
> +CIRROS_KERNEL_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_KERNEL_FILE}
> +CIRROS_INITRD_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_INITRD_FILE}
> +CIRROS_ROOTFS_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_ROOTFS_FILE}.gz
> +CIRROS_DISK_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_DISK_FILE}
> +CIRROS_GRUB_CFG="(xen/xvda,msdos1)/boot/grub/grub.cfg"

No need for {} around variables. For example, this should work:

  CIRROS_DISK_FILE=cirros-$CIRROS_VERSION-$CIRROS_ARCH-disk.img

But other than that, the two config files look fine.


> diff --git a/tests-configs/config-cirros_x86_64 b/tests-configs/config-cirros_x86_64
> new file mode 100644
> index 0000000..72cf489
> --- /dev/null
> +++ b/tests-configs/config-cirros_x86_64
> @@ -0,0 +1,13 @@
> +CIRROS_ARCH=x86_64
> +CIRROS_BASE_URL="https://download.cirros-cloud.net/"
> +CIRROS_VERSION="0.3.5"
> +CIRROS_DOWNLOADS=$BASEDIR/downloads
> +CIRROS_KERNEL_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-kernel
> +CIRROS_INITRD_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-initramfs
> +CIRROS_ROOTFS_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img
> +CIRROS_DISK_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img
> +CIRROS_KERNEL_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_KERNEL_FILE}
> +CIRROS_INITRD_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_INITRD_FILE}
> +CIRROS_ROOTFS_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_ROOTFS_FILE}.gz
> +CIRROS_DISK_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_DISK_FILE}
> +CIRROS_GRUB_CFG="(xen/xvda,msdos1)/boot/grub/grub.cfg"

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 11/11] Enable cirros tests in the default config
  2017-03-24 18:59 ` [PATCH 11/11] Enable cirros tests in the default config Géza Gémes
@ 2017-03-25  0:08   ` Stefano Stabellini
  0 siblings, 0 replies; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-25  0:08 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, sstabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1185 bytes --]

On Fri, 24 Mar 2017, Géza Gémes wrote:
> The existing cirros tests are enabled, with the following
> exceptions:
> cirros-minios-stubdom-hvm and cirros-minios-stubdom-pvhvm are
> skipped as raisin does not install the stubdom
> 
> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>

You can keep my Reviewed-by tag from before. Usually once the maintainer
replies with his Reviewed-by, the next time the submitter sends the
patch series he adds the Reviewed-by right below the Signed-off-by,
unless he makes significant changes to the patch.

Basically, all patches need a Reviewed-by from one of the maintainers.
Once all patches have it, they get committed.


>  defconfig | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/defconfig b/defconfig
> index f8ef398..bdacc7e 100644
> --- a/defconfig
> +++ b/defconfig
> @@ -31,4 +31,4 @@ GIT_TRANSPORT="git"
>  # Tests
>  ## All tests: busybox-pv busybox-hvm
>  ## ENABLED_TESTS is the list of test run by raise test
> -ENABLED_TESTS="busybox-pv busybox-hvm"
> +ENABLED_TESTS="busybox-pv busybox-hvm cirros-separate-kernel-pv cirros-pygrub-pv cirros-pvgrub2-pv cirros-qemu-hvm cirros-qemu-pvhvm"
> -- 
> 2.7.4
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 01/11] Fix installation of python libs
  2017-03-24 18:59 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
@ 2017-03-25  0:12   ` Stefano Stabellini
  0 siblings, 0 replies; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-25  0:12 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, sstabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1031 bytes --]

On Fri, 24 Mar 2017, Géza Gémes wrote:
> Change deb package build in order to move the site-packages
> folder to dist-packages in this way having the libs inluded
> in the default PYTHONPATH
> 
> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
> ---
>  scripts/mkdeb | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/scripts/mkdeb b/scripts/mkdeb
> index 3796300..fcaf757 100755
> --- a/scripts/mkdeb
> +++ b/scripts/mkdeb
> @@ -46,6 +46,9 @@ then
>      rm -rf deb/usr/lib64
>  fi
>  
> +# Make sure that python libs will be in the path
> +mv deb/usr/lib/python2.7/site-packages deb/usr/lib/python2.7/dist-packages

I think we need to do the following, just in case dist-packages is
already present (we don't want to end up with
deb/usr/lib/python2.7/dist-packages/site-packages):

mkdir -p deb/usr/lib/python2.7/dist-packages
mv deb/usr/lib/python2.7/site-packages/* deb/usr/lib/python2.7/dist-packages


>  # Fill in the debian boilerplate
>  mkdir -p deb/DEBIAN
>  cat >deb/DEBIAN/control <<EOF
> -- 
> 2.7.4
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 03/11] Introduce cirros-separate-kernel-pv test
  2017-03-24 18:59 ` [PATCH 03/11] Introduce cirros-separate-kernel-pv test Géza Gémes
@ 2017-03-25  0:17   ` Stefano Stabellini
  0 siblings, 0 replies; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-25  0:17 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, sstabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1659 bytes --]

On Fri, 24 Mar 2017, Géza Gémes wrote:
> This test is the cirros equivalent of the bussybox-pv test
                                            ^ busybox-pv


> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
> ---
>  tests/cirros-separate-kernel-pv | 26 ++++++++++++++++++++++++++
>  tests/series                    |  1 +
>  2 files changed, 27 insertions(+)
>  create mode 100644 tests/cirros-separate-kernel-pv
> 
> diff --git a/tests/cirros-separate-kernel-pv b/tests/cirros-separate-kernel-pv
> new file mode 100644
> index 0000000..9432481
> --- /dev/null
> +++ b/tests/cirros-separate-kernel-pv
> @@ -0,0 +1,26 @@
> +#!/usr/bin/env bash
> +
> +set -e
> +
> +function cirros-separate-kernel-pv-cleanup() {
> +    tear_down_cirros_test $testdir
> +}
> +
> +function cirros-separate-kernel-pv-test() {
> +    testdir=`mktemp -d`

I would move the call to download_cirros_components here. The test looks
fine though.


> +    set_up_cirros_kernel_initrd $testdir
> +    set_up_cirros_rootfs $testdir
> +    cat >$testdir/cirros-separate-kernel-pv.cfg <<EOF
> +name = "raisin-test"
> +memory = 128
> +vcpus = 2
> +kernel = "$testdir/$CIRROS_KERNEL_FILE"
> +ramdisk = "$testdir/$CIRROS_INITRD_FILE"
> +disk = [ '$testdir/$CIRROS_ROOTFS_FILE,raw,xvda,rw' ]
> +extra = "root=/dev/xvda"
> +vif = [ 'bridge=xenbr1' ]
> +EOF
> +
> +
> +    $SUDO xl create $testdir/cirros-separate-kernel-pv.cfg
> +    check_guest_alive
> +}
> diff --git a/tests/series b/tests/series
> index a2f4b34..38ee35a 100644
> --- a/tests/series
> +++ b/tests/series
> @@ -1,3 +1,4 @@
>  busybox-pv
>  busybox-hvm
>  busybox-hvm-migrate
> +cirros-separate-kernel-pv
> -- 
> 2.7.4
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1
  2017-03-24 18:59 ` [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1 Géza Gémes
@ 2017-03-25  0:26   ` Stefano Stabellini
  2017-03-25 19:21     ` Gémes Géza
  0 siblings, 1 reply; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-25  0:26 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, sstabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1211 bytes --]

On Fri, 24 Mar 2017, Géza Gémes wrote:
> fdisk from util-linux >= 2.27.1 returns units in a slightly different
> structure than earlier versions
> 
> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
> ---
>  scripts/lopartsetup | 18 +++++++++++++++++-
>  1 file changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/scripts/lopartsetup b/scripts/lopartsetup
> index 04ce3cc..b675e03 100755
> --- a/scripts/lopartsetup
> +++ b/scripts/lopartsetup
> @@ -53,7 +53,23 @@ then
>      exit 1
>  fi
>  
> -unit="`fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | cut -d " " -f 9`"
> +unitstring=""
> +column=0
> +if [ `fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | wc -l` -eq 1 ]
> +then
> +    unitstring="^Units = "
> +    column=9
> +else if [ `fdisk -lu $filename 2>/dev/null | grep -e "^Units:" | wc -l` -eq 1 ]
> +then
> +    unitstring="^Units:"
> +    column=8
> +else
> +    exit 1
> +fi
> +fi

two fi?

Aside from that, the patch looks good, so I fixed it myself and
committed it.


> +unit="`fdisk -lu $filename 2>/dev/null | grep -e "$unitstring" | cut -d " " -f $column`"
> +
>  index=0
>  for i in "`fdisk -lu $filename 2>/dev/null | grep -e "^$filename"`"
>  do
> -- 
> 2.7.4
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1
  2017-03-25  0:26   ` Stefano Stabellini
@ 2017-03-25 19:21     ` Gémes Géza
  0 siblings, 0 replies; 23+ messages in thread
From: Gémes Géza @ 2017-03-25 19:21 UTC (permalink / raw)
  To: Stefano Stabellini; +Cc: xen-devel, george.dunlap

Hi Stefano,

Unfortunately, the patch has an other typo too. Please see below:

> On Fri, 24 Mar 2017, Géza Gémes wrote:
>> fdisk from util-linux >= 2.27.1 returns units in a slightly different
>> structure than earlier versions
>>
>> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
>> ---
>>   scripts/lopartsetup | 18 +++++++++++++++++-
>>   1 file changed, 17 insertions(+), 1 deletion(-)
>>
>> diff --git a/scripts/lopartsetup b/scripts/lopartsetup
>> index 04ce3cc..b675e03 100755
>> --- a/scripts/lopartsetup
>> +++ b/scripts/lopartsetup
>> @@ -53,7 +53,23 @@ then
>>       exit 1
>>   fi
>>   
>> -unit="`fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | cut -d " " -f 9`"
>> +unitstring=""
>> +column=0
>> +if [ `fdisk -lu $filename 2>/dev/null | grep -e "^Units = " | wc -l` -eq 1 ]
>> +then
>> +    unitstring="^Units = "
>> +    column=9
>> +else if [ `fdisk -lu $filename 2>/dev/null | grep -e "^Units:" | wc -l` -eq 1 ]
this should have been elif [ `fdisk -lu $filename 2>/dev/null | grep -e 
"^Units:" | wc -l` -eq 1 ]
>> +then
>> +    unitstring="^Units:"
>> +    column=8
>> +else
>> +    exit 1
>> +fi
>> +fi
> two fi?
>
> Aside from that, the patch looks good, so I fixed it myself and
> committed it.
>
>
>> +unit="`fdisk -lu $filename 2>/dev/null | grep -e "$unitstring" | cut -d " " -f $column`"
>> +
>>   index=0
>>   for i in "`fdisk -lu $filename 2>/dev/null | grep -e "^$filename"`"
>>   do
>> -- 
>> 2.7.4

I'll send a patch fixing this in a couple of seconds.

Cheers,

Geza


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 02/11] Introduce cirros tests
  2017-03-24 21:01   ` Stefano Stabellini
@ 2017-03-25 19:24     ` Gémes Géza
  0 siblings, 0 replies; 23+ messages in thread
From: Gémes Géza @ 2017-03-25 19:24 UTC (permalink / raw)
  To: Stefano Stabellini; +Cc: xen-devel, george.dunlap


[-- Attachment #1.1: Type: text/plain, Size: 11070 bytes --]

Hi Stefano,

A few answers/comments inline:
> On Fri, 24 Mar 2017, Géza Gémes wrote:
>> Add support for using cirros images in raisin tests
>>
>> Signed-off-by: Géza Gémes<geza.gemes@gmail.com>
> Thank you! Much, much better, almost ready :)
>
> I have only a couple of suggestions and questions left, see below.
>
>
>>   lib/common-functions.sh            |  23 +++++++
>>   lib/common-tests.sh                | 126 +++++++++++++++++++++++++++++++++++++
>>   tests-configs/config-cirros_x86_32 |  13 ++++
>>   tests-configs/config-cirros_x86_64 |  13 ++++
>>   4 files changed, 175 insertions(+)
>>   create mode 100644 tests-configs/config-cirros_x86_32
>>   create mode 100644 tests-configs/config-cirros_x86_64
>>
>> diff --git a/lib/common-functions.sh b/lib/common-functions.sh
>> index d4476f3..d82d7ab 100644
>> --- a/lib/common-functions.sh
>> +++ b/lib/common-functions.sh
>> @@ -439,3 +439,26 @@ function uninstall_package() {
>>           error_echo "Don't know how to uninstall packages on $DISTRO"
>>       fi
>>   }
>> +
>> +function get-qemu-img() {
>> +    set +e
>> +    QEMU_IMG=`which qemu-img`
>> +    set -e
>> +    if [[ -z "$QEMU_IMG" ]]
>> +    then
>> +        QEMU_IMG="/usr/lib/xen/bin/qemu-img"
>> +    fi
>> +    export QEMU_IMG
> we probably want to test that $QEMU_IMG exists (test -f), and print an
> error if it doesn't
For QEMU_IMG I would rather check if it is executable, as that is needed.
>> +}
>> +
>> +function get-pvgrub() {
>> +    ARCH=$1
>> +    set +e
>> +    PVGRUB=`which grub-${ARCH}-xen`
>> +    set -e
>> +    if [[ -z "$PVGRUB" ]]
>> +    then
>> +        PVGRUB="/usr/lib/xen/boot/grub-${ARCH}-xen"
>> +    fi
>> +    export PVGRUB
> Same here.
>
> Also, I don't see who is calling get-qemu-img and get-pvgrub in this
> patch series.
>
This is strange. I've called them now explicitly.
>> +}
>> diff --git a/lib/common-tests.sh b/lib/common-tests.sh
>> index d346af4..1deaf4f 100644
>> --- a/lib/common-tests.sh
>> +++ b/lib/common-tests.sh
>> @@ -178,3 +178,129 @@ function get_host_initrd() {
>>           exit 1
>>       fi
>>   }
>> +
>> +function cirros_network_init() {
>> +    rootdir=$1
>> +    ifile=`mktemp`
>> +    # Create static network config
>> +    cat >$ifile <<EOF
>> +auto lo
>> +iface lo inet loopback
>> +
>> +auto eth0
>> +iface eth0 inet static
>> +    address 169.254.0.2
>> +    network 169.254.0.0
>> +    broadcast 169.254.0.255
>> +    netmask 255.255.255.0
>> +EOF
>> +    $SUDO mv $ifile $rootdir/etc/network/interfaces
> This is OK. However, we need to to "mv -f" otherwise it might ask for
> confirmation before overwriting.
>
Done.
>> +    # Disable cloud-init
>> +    $SUDO rm -f ${rootdir}/etc/rc3.d/S*cirros*ds*
>> +    $SUDO rm -f ${rootdir}/etc/rc3.d/S*-cirros-userdata
>> +}
>> +
>> +function get_cirros_kernel() {
>> +    bootdir=$1
>> +    basename `find $bootdir -name vmlinuz* 2>/dev/null | head -1`
>> +}
>> +
>> +function get_cirros_initrd() {
>> +    bootdir=$1
>> +    basename `find $bootdir -name initrd* 2>/dev/null | head -1`
>> +}
>> +
>> +function cirros_grub_cfg() {
>> +    rootdir=$1
>> +    grubroot="`echo $CIRROS_GRUB_CFG | cut -d ')' -f 1`)"
>> +    grubcfg="`echo $CIRROS_GRUB_CFG | cut -d ')' -f 2`"
>> +    grubdir=`dirname $grubcfg`
>> +    bootdir=`dirname $grubdir`
>> +    tmpgrubcfg=`mktemp`
>> +    cat > $tmpgrubcfg <<EOF
>> +root="$grubroot"
>> +insmod xzio
>> +insmod gzio
>> +insmod btrfs
>> +insmod ext2
>> +set timeout=1
>> +set default=0
>> +menuentry Cirros {
>> +    linux `echo $bootdir`/`get_cirros_kernel ${rootdir}/${bootdir}` root=/dev/xvda1 ro
>> +    initrd `echo $bootdir`/`get_cirros_initrd ${rootdir}/${bootdir}`
>> +}
>> +EOF
>> +    $SUDO mv $tmpgrubcfg ${rootdir}/${grubcfg}
> mv -f
Done.
>> +}
>> +
>> +function download_cirros_components() {
>> +    verbose_echo "Downloading cirros components"
>> +    . tests-configs/config-cirros_$RAISIN_ARCH
>> +    mkdir -p $CIRROS_DOWNLOADS
>> +    wget -q -c $CIRROS_KERNEL_URL -P $CIRROS_DOWNLOADS
>> +    wget -q -c $CIRROS_INITRD_URL -P $CIRROS_DOWNLOADS
>> +    wget -q -c $CIRROS_ROOTFS_URL -P $CIRROS_DOWNLOADS
>> +    wget -q -c $CIRROS_DISK_URL -P $CIRROS_DOWNLOADS
> If the downloads are already there, does wget do the right thing?  I
> guess it does. Otherwise, do we need to test -f each of the files?
wget -c tries to continue the download. If the file is fully downloaded 
it does nothing. However given your other suggestion to do unzipping 
right in this function, I'll change that into checking for the existence 
of the files.
>> +}
>> +
>> +function set_up_cirros_kernel_initrd() {
>> +    local testdir=$1
>> +    mkdir -p $testdir
>> +    download_cirros_components
> I would probably move the download_cirros_components call to the test
> scripts, for example cirros-separate-kernel-pv-test.
Done.
>> +    verbose_echo "Setting up kernel and initrd for cirros test"
>> +    cp $CIRROS_DOWNLOADS/$CIRROS_KERNEL_FILE $testdir
>> +    cp $CIRROS_DOWNLOADS/$CIRROS_INITRD_FILE $testdir
>> +}
>> +
>> +function set_up_cirros_rootfs() {
>> +    local testdir=$1
>> +    mkdir -p $testdir
>> +    download_cirros_components
>> +    verbose_echo "Setting up rootfs for cirros test"
>> +    cp $CIRROS_DOWNLOADS/$CIRROS_ROOTFS_FILE.gz $testdir
>> +    gunzip $testdir/$CIRROS_ROOTFS_FILE.gz
> I would leave the unzipped file under $CIRROS_DOWNLOADS, so that we
> don't have to unzip the same rootfs file twice.
>
> I would also test for the existance of the unzipped file here and skip
> the whole setup if it is already present. So the first time, we unzip
> and setup the network interfaces file; the second time we just test that
> the unzipped file exists and return. Would that work?
>
Done.
>> +    local cirros_rootfs_loop=`create_loop $testdir/$CIRROS_ROOTFS_FILE`
>> +    local cirros_rootfs_mntpt=`mktemp -d`
>> +    $SUDO mount $cirros_rootfs_loop $cirros_rootfs_mntpt
>> +    cirros_network_init $cirros_rootfs_mntpt
>> +    $SUDO umount $cirros_rootfs_mntpt
>> +    $SUDO rmdir $cirros_rootfs_mntpt
>> +    $SUDO losetup -d $cirros_rootfs_loop
>> +}
>> +
>> +function set_up_cirros_disk() {
>> +    local testdir=$1
>> +    mkdir -p $testdir
>> +    local grub_install=False
>> +    if [[ $# -gt 1 ]]
>> +    then
>> +        grub_install=$2
>> +    else
>> +        unset grub_install
>> +    fi
> Is this test actually needed? It doesn't look like anything is calling
> set_up_cirros_disk with more than one argument.
The pvgrub test, which needs a grub2 config file (not included by 
cirros) calls this with two arguments. However as I restructured the 
code this function is dropped.

>> +    download_cirros_components
>> +    verbose_echo "Setting up disk for cirros test"
>> +    $QEMU_IMG convert -f qcow2 -O raw $CIRROS_DOWNLOADS/$CIRROS_DISK_FILE $testdir/$CIRROS_DISK_FILE
> Similarly to the rootfs, I would output the conversion to
> $CIRROS_DOWNLOADS to avoid doing it twice. The second time the function
> gets called, I would just test for the existance of the raw file and
> return. (It is probably best to rename the raw file to something like
> $CIRROS_DISK_FILE.raw). In addition, if possible, I would also do the
> following setup only the first time around. Does it make sense?
Done
>> +    local cirros_disk_loop=`$SUDO $BASEDIR/scripts/lopartsetup $testdir/$CIRROS_DISK_FILE | head -1 | cut -d ":" -f 1`
>> +    local cirros_disk_mntpt=`mktemp -d`
>> +    $SUDO mount $cirros_disk_loop $cirros_disk_mntpt
>> +    cirros_network_init $cirros_disk_mntpt
>> +    if [[ -n $grub_install ]]
>> +    then
>> +        cirros_grub_cfg $cirros_disk_mntpt
>> +    fi
>> +    $SUDO umount $cirros_disk_mntpt
>> +    $SUDO rmdir $cirros_disk_mntpt
>> +    $SUDO losetup -d $cirros_disk_loop
> This is just for you information. If you remove the loop device, it
> means that you'll end up using the QEMU userspace disk backend.
> Otherwise if you keep the loop device around and use it for the backend,
> you end up using the in-kernel disk backend, which is typically faster.
> In other words, the following:
>
>    disk = [ '$testdir/$CIRROS_DISK_FILE,raw,hda,rw' ]
>
> ends up using the QEMU disk backend, while the following:
>
>    disk = [ '$cirros_disk_loop,raw,hda,rw' ]
>
> ends up using the kernel disk backend.
> Either choice is fine for the tests, I just wanted to make sure you knew :)
>
Thanks for the info. I didn't know. However for simplicity I'll keep the 
image files.
>> +}
>> +
>> +function tear_down_cirros_test() {
>> +    testdir=$1
>> +    if [[ `$SUDO xl vm-list | grep -e "raisin-test" | wc -l` -gt 0 ]]
> Why "grep -e"? I think grep is sufficient.
>
Done
>> +    then
>> +        $SUDO xl destroy "raisin-test"
>> +    fi
>> +    verbose_echo "$PREPEND deleting environment of cirros test"
>> +    $SUDO rm -rf $testdir
>> +}
>> +
>> diff --git a/tests-configs/config-cirros_x86_32 b/tests-configs/config-cirros_x86_32
>> new file mode 100644
>> index 0000000..5402415
>> --- /dev/null
>> +++ b/tests-configs/config-cirros_x86_32
>> @@ -0,0 +1,13 @@
>> +CIRROS_ARCH=i386
>> +CIRROS_BASE_URL="https://download.cirros-cloud.net/"
>> +CIRROS_VERSION="0.3.5"
>> +CIRROS_DOWNLOADS=$BASEDIR/downloads
>> +CIRROS_KERNEL_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-kernel
>> +CIRROS_INITRD_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-initramfs
>> +CIRROS_ROOTFS_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img
>> +CIRROS_DISK_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img
>> +CIRROS_KERNEL_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_KERNEL_FILE}
>> +CIRROS_INITRD_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_INITRD_FILE}
>> +CIRROS_ROOTFS_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_ROOTFS_FILE}.gz
>> +CIRROS_DISK_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_DISK_FILE}
>> +CIRROS_GRUB_CFG="(xen/xvda,msdos1)/boot/grub/grub.cfg"
> No need for {} around variables. For example, this should work:
>
>    CIRROS_DISK_FILE=cirros-$CIRROS_VERSION-$CIRROS_ARCH-disk.img
>
> But other than that, the two config files look fine.
Done
>
>> diff --git a/tests-configs/config-cirros_x86_64 b/tests-configs/config-cirros_x86_64
>> new file mode 100644
>> index 0000000..72cf489
>> --- /dev/null
>> +++ b/tests-configs/config-cirros_x86_64
>> @@ -0,0 +1,13 @@
>> +CIRROS_ARCH=x86_64
>> +CIRROS_BASE_URL="https://download.cirros-cloud.net/"
>> +CIRROS_VERSION="0.3.5"
>> +CIRROS_DOWNLOADS=$BASEDIR/downloads
>> +CIRROS_KERNEL_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-kernel
>> +CIRROS_INITRD_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-initramfs
>> +CIRROS_ROOTFS_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-rootfs.img
>> +CIRROS_DISK_FILE=cirros-${CIRROS_VERSION}-${CIRROS_ARCH}-disk.img
>> +CIRROS_KERNEL_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_KERNEL_FILE}
>> +CIRROS_INITRD_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_INITRD_FILE}
>> +CIRROS_ROOTFS_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_ROOTFS_FILE}.gz
>> +CIRROS_DISK_URL=${CIRROS_BASE_URL}/${CIRROS_VERSION}/${CIRROS_DISK_FILE}
>> +CIRROS_GRUB_CFG="(xen/xvda,msdos1)/boot/grub/grub.cfg"

Cheers,

Geza


[-- Attachment #1.2: Type: text/html, Size: 14285 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 01/11] Fix installation of python libs
  2017-03-21  6:27     ` Géza Gémes
@ 2017-03-21 17:04       ` Stefano Stabellini
  0 siblings, 0 replies; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-21 17:04 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, Stefano Stabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 1603 bytes --]

On Tue, 21 Mar 2017, Géza Gémes wrote:
> 2017. márc. 21. 0:54 ezt írta ("Stefano Stabellini" <sstabellini@kernel.org>):
>       On Sun, 19 Mar 2017, Géza Gémes wrote:
>       > Change deb package build in order to symlink the files installed
>       > to site-packages to dist-packages to have them inluded in the
>       > default PYTHONPATH
>       >
>       > Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
>       > ---
>       >  scripts/mkdeb | 9 +++++++++
>       >  1 file changed, 9 insertions(+)
>       >
>       > diff --git a/scripts/mkdeb b/scripts/mkdeb
>       > index 3796300..23bcc2a 100755
>       > --- a/scripts/mkdeb
>       > +++ b/scripts/mkdeb
>       > @@ -46,6 +46,15 @@ then
>       >      rm -rf deb/usr/lib64
>       >  fi
>       >
>       > +# Make sure that python libs will be in the path
>       > +mkdir -p deb/usr/lib/python2.7/dist-packages/
>       > +cd deb/usr/lib/python2.7/dist-packages/
>       > +for file in `ls ../site-packages`
>       > +do
>       > +    ln -s ../site-packages/$file .
>       > +done
>       > +cd -
> 
> Actually for Debian I think it makes sense to move (not link) the files
> to dist-packages, what do you think?
> 
> 
> As I haven't tried on anything else than ubuntu 14.04 I was afraid of breaking it elsewhere. But if you are aware that Debian and forks are using dist-packages, I will change
> the symlink to a move.

Yes, I think that's the case.




>       >  # Fill in the debian boilerplate
>       >  mkdir -p deb/DEBIAN
>       >  cat >deb/DEBIAN/control <<EOF
>       > --
>       > 2.7.4
>       >

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 01/11] Fix installation of python libs
  2017-03-20 23:54   ` Stefano Stabellini
@ 2017-03-21  6:27     ` Géza Gémes
  2017-03-21 17:04       ` Stefano Stabellini
  0 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-21  6:27 UTC (permalink / raw)
  To: Stefano Stabellini; +Cc: xen-devel, george.dunlap


[-- Attachment #1.1: Type: text/plain, Size: 1316 bytes --]

2017. márc. 21. 0:54 ezt írta ("Stefano Stabellini" <sstabellini@kernel.org
>):

On Sun, 19 Mar 2017, Géza Gémes wrote:
> Change deb package build in order to symlink the files installed
> to site-packages to dist-packages to have them inluded in the
> default PYTHONPATH
>
> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
> ---
>  scripts/mkdeb | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
> diff --git a/scripts/mkdeb b/scripts/mkdeb
> index 3796300..23bcc2a 100755
> --- a/scripts/mkdeb
> +++ b/scripts/mkdeb
> @@ -46,6 +46,15 @@ then
>      rm -rf deb/usr/lib64
>  fi
>
> +# Make sure that python libs will be in the path
> +mkdir -p deb/usr/lib/python2.7/dist-packages/
> +cd deb/usr/lib/python2.7/dist-packages/
> +for file in `ls ../site-packages`
> +do
> +    ln -s ../site-packages/$file .
> +done
> +cd -

Actually for Debian I think it makes sense to move (not link) the files
to dist-packages, what do you think?


As I haven't tried on anything else than ubuntu 14.04 I was afraid of
breaking it elsewhere. But if you are aware that Debian and forks are using
dist-packages, I will change the symlink to a move.



>  # Fill in the debian boilerplate
>  mkdir -p deb/DEBIAN
>  cat >deb/DEBIAN/control <<EOF
> --
> 2.7.4
>


Cheers

Géza

[-- Attachment #1.2: Type: text/html, Size: 2377 bytes --]

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* Re: [PATCH 01/11] Fix installation of python libs
  2017-03-19 15:23 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
@ 2017-03-20 23:54   ` Stefano Stabellini
  2017-03-21  6:27     ` Géza Gémes
  0 siblings, 1 reply; 23+ messages in thread
From: Stefano Stabellini @ 2017-03-20 23:54 UTC (permalink / raw)
  To: Géza Gémes; +Cc: xen-devel, sstabellini, george.dunlap

[-- Attachment #1: Type: TEXT/PLAIN, Size: 970 bytes --]

On Sun, 19 Mar 2017, Géza Gémes wrote:
> Change deb package build in order to symlink the files installed
> to site-packages to dist-packages to have them inluded in the
> default PYTHONPATH
> 
> Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
> ---
>  scripts/mkdeb | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/scripts/mkdeb b/scripts/mkdeb
> index 3796300..23bcc2a 100755
> --- a/scripts/mkdeb
> +++ b/scripts/mkdeb
> @@ -46,6 +46,15 @@ then
>      rm -rf deb/usr/lib64
>  fi
>  
> +# Make sure that python libs will be in the path
> +mkdir -p deb/usr/lib/python2.7/dist-packages/
> +cd deb/usr/lib/python2.7/dist-packages/
> +for file in `ls ../site-packages`
> +do
> +    ln -s ../site-packages/$file .
> +done
> +cd -

Actually for Debian I think it makes sense to move (not link) the files
to dist-packages, what do you think?


>  # Fill in the debian boilerplate
>  mkdir -p deb/DEBIAN
>  cat >deb/DEBIAN/control <<EOF
> -- 
> 2.7.4
> 

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 23+ messages in thread

* [PATCH 01/11] Fix installation of python libs
  2017-03-19 15:23 [raisin][PATCH 00/11] pygrub fix + cirros tests Géza Gémes
@ 2017-03-19 15:23 ` Géza Gémes
  2017-03-20 23:54   ` Stefano Stabellini
  0 siblings, 1 reply; 23+ messages in thread
From: Géza Gémes @ 2017-03-19 15:23 UTC (permalink / raw)
  To: xen-devel; +Cc: sstabellini, george.dunlap, Géza Gémes

Change deb package build in order to symlink the files installed
to site-packages to dist-packages to have them inluded in the
default PYTHONPATH

Signed-off-by: Géza Gémes <geza.gemes@gmail.com>
---
 scripts/mkdeb | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/scripts/mkdeb b/scripts/mkdeb
index 3796300..23bcc2a 100755
--- a/scripts/mkdeb
+++ b/scripts/mkdeb
@@ -46,6 +46,15 @@ then
     rm -rf deb/usr/lib64
 fi
 
+# Make sure that python libs will be in the path
+mkdir -p deb/usr/lib/python2.7/dist-packages/
+cd deb/usr/lib/python2.7/dist-packages/
+for file in `ls ../site-packages`
+do
+    ln -s ../site-packages/$file .
+done
+cd -
+
 # Fill in the debian boilerplate
 mkdir -p deb/DEBIAN
 cat >deb/DEBIAN/control <<EOF
-- 
2.7.4


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

^ permalink raw reply related	[flat|nested] 23+ messages in thread

end of thread, other threads:[~2017-03-25 19:24 UTC | newest]

Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-24 18:59 [raisin][PATCH 00/11] pygrub fix + cirros tests + lopartsetup fix Géza Gémes
2017-03-24 18:59 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
2017-03-25  0:12   ` Stefano Stabellini
2017-03-24 18:59 ` [PATCH 02/11] Introduce cirros tests Géza Gémes
2017-03-24 21:01   ` Stefano Stabellini
2017-03-25 19:24     ` Gémes Géza
2017-03-24 18:59 ` [PATCH 03/11] Introduce cirros-separate-kernel-pv test Géza Gémes
2017-03-25  0:17   ` Stefano Stabellini
2017-03-24 18:59 ` [PATCH 04/11] Introduce cirros-pygrub-pv test Géza Gémes
2017-03-24 18:59 ` [PATCH 05/11] Introduce cirros-pvgrub2-pv test Géza Gémes
2017-03-24 18:59 ` [PATCH 06/11] Introduce cirros-qemu-hvm test Géza Gémes
2017-03-24 18:59 ` [PATCH 07/11] Introduce cirros-minios-stubdom-hvm test Géza Gémes
2017-03-24 18:59 ` [PATCH 08/11] Introduce cirros-qemu-pvhvm test Géza Gémes
2017-03-24 18:59 ` [PATCH 09/11] Introduce cirros-minios-stubdom-pvhvm test Géza Gémes
2017-03-24 18:59 ` [PATCH 10/11] Fix lopartsetup for util-linux >= 2.27.1 Géza Gémes
2017-03-25  0:26   ` Stefano Stabellini
2017-03-25 19:21     ` Gémes Géza
2017-03-24 18:59 ` [PATCH 11/11] Enable cirros tests in the default config Géza Gémes
2017-03-25  0:08   ` Stefano Stabellini
  -- strict thread matches above, loose matches on Subject: below --
2017-03-19 15:23 [raisin][PATCH 00/11] pygrub fix + cirros tests Géza Gémes
2017-03-19 15:23 ` [PATCH 01/11] Fix installation of python libs Géza Gémes
2017-03-20 23:54   ` Stefano Stabellini
2017-03-21  6:27     ` Géza Gémes
2017-03-21 17:04       ` Stefano Stabellini

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.