[PATCH 0/4] automation: QEMU-based smoke test

[PATCH 0/4] automation: QEMU-based smoke test

[Wei Liu]

See https://gitlab.com/xen-project/people/liuw/xen/pipelines/37173009

A pipeline while I was developing this series.

Wei Liu (4):
  automation: introduce CONTAINER_NO_PULL for containerize
  automation: stash default config file for artifact extraction
  automation: also specify xen binary as artifact on x86_64
  automation: add qemu smoke test

 .gitlab-ci.yml                          | 22 +++++++++++++++++++++-
 automation/build/README.md              |  3 +++
 automation/scripts/build                |  7 +++++++
 automation/scripts/containerize         |  8 +++++---
 automation/scripts/qemu-smoke-x86-64.sh | 23 +++++++++++++++++++++++
 5 files changed, 59 insertions(+), 4 deletions(-)
 create mode 100755 automation/scripts/qemu-smoke-x86-64.sh

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 1/4] automation: introduce CONTAINER_NO_PULL for containerize

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 automation/build/README.md      | 3 +++
 automation/scripts/containerize | 8 +++++---
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/automation/build/README.md b/automation/build/README.md
index f6cfd46f1e..d8c8a18e33 100644
--- a/automation/build/README.md
+++ b/automation/build/README.md
@@ -52,6 +52,9 @@ understands.
 
 - CONTAINER_UID0: This specifies whether root is used inside the container.
 
+- CONTAINER_NO_PULL: If set to 1, the script will not pull from docker hub.
+  This is useful when testing container locally.
+
 - XEN_CONFIG_EXPERT: If this is defined in your shell it will be
   automatically passed through to the container.
 
diff --git a/automation/scripts/containerize b/automation/scripts/containerize
index aa08274eba..a3e5d79c70 100755
--- a/automation/scripts/containerize
+++ b/automation/scripts/containerize
@@ -42,9 +42,11 @@ tty -s && termint=t
 # Fetch the latest version of the container in hub.docker.com,
 # unless it's a newly created local copy.
 #
-einfo "*** Ensuring ${CONTAINER} is up to date"
-docker pull ${CONTAINER} > /dev/null ||     \
-    die "Failed to update docker container"
+if [[ "_${CONTAINER_NO_PULL}" != "_1" ]]; then
+    einfo "*** Ensuring ${CONTAINER} is up to date"
+    docker pull ${CONTAINER} > /dev/null ||     \
+        die "Failed to update docker container"
+fi
 
 if hash greadlink > /dev/null 2>&1; then
     READLINK=greadlink
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 2/4] automation: stash default config file for artifact extraction

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 .gitlab-ci.yml           | 2 +-
 automation/scripts/build | 3 +++
 2 files changed, 4 insertions(+), 1 deletion(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 5678b552c4..28152e906d 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -8,7 +8,7 @@ stages:
     - ./automation/scripts/build 2>&1 | tee build.log
   artifacts:
     paths:
-      - xen/.config
+      - xen-config
       - '*.log'
     when: always
   except:
diff --git a/automation/scripts/build b/automation/scripts/build
index 48e398ea20..a1f9a5da56 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -26,6 +26,9 @@ fi
 
 make -j$(nproc) dist
 
+# Extract artifacts to avoid getting rewritten by customised builds
+cp xen/.config xen-config
+
 # Build all the configs we care about
 case ${XEN_TARGET_ARCH} in
     x86_64) arch=x86 ;;
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 3/4] automation: also specify xen binary as artifact on x86_64

[Wei Liu]

... so that it can be passed on to test stage.

Note that xen is only extracted for x86_64 build since others may not
have that. Use a directory to account for possibly different file
names on Arm.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 .gitlab-ci.yml           | 1 +
 automation/scripts/build | 4 ++++
 2 files changed, 5 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 28152e906d..d11459b117 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -8,6 +8,7 @@ stages:
     - ./automation/scripts/build 2>&1 | tee build.log
   artifacts:
     paths:
+      - binaries/
       - xen-config
       - '*.log'
     when: always
diff --git a/automation/scripts/build b/automation/scripts/build
index a1f9a5da56..d4aceb745f 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -28,6 +28,10 @@ make -j$(nproc) dist
 
 # Extract artifacts to avoid getting rewritten by customised builds
 cp xen/.config xen-config
+mkdir binaries
+if [[ "x${XEN_TARGET_ARCH}" == "xx86_64" ]]; then
+    cp xen/xen binaries/xen
+fi
 
 # Build all the configs we care about
 case ${XEN_TARGET_ARCH} in
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH 4/4] automation: add qemu smoke test

[Wei Liu]

This patch introduces a new test stage into the pipeline and provides
a simple QEMU based smoke test.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 .gitlab-ci.yml                          | 19 +++++++++++++++++++
 automation/scripts/qemu-smoke-x86-64.sh | 23 +++++++++++++++++++++++
 2 files changed, 42 insertions(+)
 create mode 100755 automation/scripts/qemu-smoke-x86-64.sh

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index d11459b117..b9ccba6ab2 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -1,5 +1,6 @@
 stages:
   - build
+  - test
 
 .build-tmpl: &build
   stage: build
@@ -379,3 +380,21 @@ debian-unstable-gcc-debug-arm64-randconfig:
   variables:
     CONTAINER: debian:unstable-arm64v8
     RANDCONFIG: y
+
+
+# Test jobs
+qemu-smoke-x86-64:
+  stage: test
+  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
+  variables:
+    CONTAINER: debian:stretch
+  script:
+    - ./automation/scripts/qemu-smoke-x86-64.sh 2>&1 | tee qemu-smoke-x86-64.log
+  artifacts:
+    paths:
+      - smoke.serial
+    when: always
+  dependencies:
+    - debian-stretch-gcc-debug
+  tags:
+    - x86_64
diff --git a/automation/scripts/qemu-smoke-x86-64.sh b/automation/scripts/qemu-smoke-x86-64.sh
new file mode 100755
index 0000000000..7dc2c8542b
--- /dev/null
+++ b/automation/scripts/qemu-smoke-x86-64.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+
+set -ex
+
+# Install QEMU
+export DEBIAN_FRONTENT=noninteractive
+apt-get -qy update
+apt-get -qy install qemu-system-x86
+
+# Clone and build XTF
+git clone https://xenbits.xen.org/git-http/xtf.git
+cd xtf && make -j$(nproc) && cd -
+
+rm -f smoke.serial
+set +e
+timeout -k 1 10 \
+qemu-system-x86_64 -nographic -kernel binaries/xen \
+        -initrd xtf/tests/example/test-pv32pae-example \
+        -append 'loglvl=all com1=115200,,8n1 console=com1 noreboot' \
+        -m 512 -monitor none -serial file:smoke.serial
+set -e
+grep -q 'Test result: SUCCESS' smoke.serial || exit 1
+exit 0
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 0/4] automation: QEMU-based smoke test

[Wei Liu]

On Mon, Nov 19, 2018 at 09:30:58PM +0000, Wei Liu wrote:
> See https://gitlab.com/xen-project/people/liuw/xen/pipelines/37173009
> 
> A pipeline while I was developing this series.

A (still in progress as of now) full pipeline
https://gitlab.com/xen-project/people/liuw/xen/pipelines/37175006

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 1/4] automation: introduce CONTAINER_NO_PULL for containerize

[Doug Goldstein]

On Mon, Nov 19, 2018 at 09:30:59PM +0000, Wei Liu wrote:
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>  automation/build/README.md      | 3 +++
>  automation/scripts/containerize | 8 +++++---
>  2 files changed, 8 insertions(+), 3 deletions(-)
> 
> diff --git a/automation/build/README.md b/automation/build/README.md
> index f6cfd46f1e..d8c8a18e33 100644
> --- a/automation/build/README.md
> +++ b/automation/build/README.md
> @@ -52,6 +52,9 @@ understands.
>  
>  - CONTAINER_UID0: This specifies whether root is used inside the container.
>  
> +- CONTAINER_NO_PULL: If set to 1, the script will not pull from docker hub.
> +  This is useful when testing container locally.
> +

Good addition.

Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 2/4] automation: stash default config file for artifact extraction

[Doug Goldstein]

On Mon, Nov 19, 2018 at 09:31:00PM +0000, Wei Liu wrote:
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>  .gitlab-ci.yml           | 2 +-
>  automation/scripts/build | 3 +++
>  2 files changed, 4 insertions(+), 1 deletion(-)
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 5678b552c4..28152e906d 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -8,7 +8,7 @@ stages:
>      - ./automation/scripts/build 2>&1 | tee build.log
>    artifacts:
>      paths:
> -      - xen/.config
> +      - xen-config

I assume the goal here is to aid in troubleshooting when we notice a
failure which makes sense. You could toss that into the commit message
while committing it.

Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 3/4] automation: also specify xen binary as artifact on x86_64

[Doug Goldstein]

On Mon, Nov 19, 2018 at 09:31:01PM +0000, Wei Liu wrote:
> ... so that it can be passed on to test stage.
> 
> Note that xen is only extracted for x86_64 build since others may not
> have that. Use a directory to account for possibly different file
> names on Arm.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 4/4] automation: add qemu smoke test

[Doug Goldstein]

On Mon, Nov 19, 2018 at 09:31:02PM +0000, Wei Liu wrote:
> This patch introduces a new test stage into the pipeline and provides
> a simple QEMU based smoke test.
> 
> Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> ---
>  .gitlab-ci.yml                          | 19 +++++++++++++++++++
>  automation/scripts/qemu-smoke-x86-64.sh | 23 +++++++++++++++++++++++
>  2 files changed, 42 insertions(+)
>  create mode 100755 automation/scripts/qemu-smoke-x86-64.sh
> 
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index d11459b117..b9ccba6ab2 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -1,5 +1,6 @@
>  stages:
>    - build
> +  - test
>  
>  .build-tmpl: &build
>    stage: build
> @@ -379,3 +380,21 @@ debian-unstable-gcc-debug-arm64-randconfig:
>    variables:
>      CONTAINER: debian:unstable-arm64v8
>      RANDCONFIG: y
> +
> +
> +# Test jobs
> +qemu-smoke-x86-64:
> +  stage: test
> +  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> +  variables:
> +    CONTAINER: debian:stretch
> +  script:
> +    - ./automation/scripts/qemu-smoke-x86-64.sh 2>&1 | tee qemu-smoke-x86-64.log

Do you want to capture this log in the artifacts below too?

> +  artifacts:
> +    paths:
> +      - smoke.serial
> +    when: always
> +  dependencies:
> +    - debian-stretch-gcc-debug
> +  tags:
> +    - x86_64

Just one quick question. If you do that can be fixed up at commit time
as well so I have no problem saying:

Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 4/4] automation: add qemu smoke test

[Wei Liu]

On Tue, Nov 20, 2018 at 10:41:07AM -0600, Doug Goldstein wrote:
> On Mon, Nov 19, 2018 at 09:31:02PM +0000, Wei Liu wrote:
> > This patch introduces a new test stage into the pipeline and provides
> > a simple QEMU based smoke test.
> > 
> > Signed-off-by: Wei Liu <wei.liu2@citrix.com>
> > ---
> >  .gitlab-ci.yml                          | 19 +++++++++++++++++++
> >  automation/scripts/qemu-smoke-x86-64.sh | 23 +++++++++++++++++++++++
> >  2 files changed, 42 insertions(+)
> >  create mode 100755 automation/scripts/qemu-smoke-x86-64.sh
> > 
> > diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> > index d11459b117..b9ccba6ab2 100644
> > --- a/.gitlab-ci.yml
> > +++ b/.gitlab-ci.yml
> > @@ -1,5 +1,6 @@
> >  stages:
> >    - build
> > +  - test
> >  
> >  .build-tmpl: &build
> >    stage: build
> > @@ -379,3 +380,21 @@ debian-unstable-gcc-debug-arm64-randconfig:
> >    variables:
> >      CONTAINER: debian:unstable-arm64v8
> >      RANDCONFIG: y
> > +
> > +
> > +# Test jobs
> > +qemu-smoke-x86-64:
> > +  stage: test
> > +  image: registry.gitlab.com/xen-project/xen/${CONTAINER}
> > +  variables:
> > +    CONTAINER: debian:stretch
> > +  script:
> > +    - ./automation/scripts/qemu-smoke-x86-64.sh 2>&1 | tee qemu-smoke-x86-64.log
> 
> Do you want to capture this log in the artifacts below too?

Good catch. I will add it to the list.

> 
> > +  artifacts:
> > +    paths:
> > +      - smoke.serial
> > +    when: always
> > +  dependencies:
> > +    - debian-stretch-gcc-debug
> > +  tags:
> > +    - x86_64
> 
> Just one quick question. If you do that can be fixed up at commit time
> as well so I have no problem saying:
> 
> Acked-by: Doug Goldstein <cardoe@cardoe.com>

Thank you.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 3/4] automation: also specify xen binary as artifact on x86_64

[Andrew Cooper]

On 19/11/2018 21:31, Wei Liu wrote:
> diff --git a/automation/scripts/build b/automation/scripts/build
> index a1f9a5da56..d4aceb745f 100755
> --- a/automation/scripts/build
> +++ b/automation/scripts/build
> @@ -28,6 +28,10 @@ make -j$(nproc) dist
>  
>  # Extract artifacts to avoid getting rewritten by customised builds
>  cp xen/.config xen-config
> +mkdir binaries
> +if [[ "x${XEN_TARGET_ARCH}" == "xx86_64" ]]; then

Either quote the strings, or use the pre-x trick, but you don't need to
do both together.

~Andrew

> +    cp xen/xen binaries/xen
> +fi
>  
>  # Build all the configs we care about
>  case ${XEN_TARGET_ARCH} in


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH 3/4] automation: also specify xen binary as artifact on x86_64

[Wei Liu]

On Tue, Nov 20, 2018 at 05:35:54PM +0000, Andrew Cooper wrote:
> On 19/11/2018 21:31, Wei Liu wrote:
> > diff --git a/automation/scripts/build b/automation/scripts/build
> > index a1f9a5da56..d4aceb745f 100755
> > --- a/automation/scripts/build
> > +++ b/automation/scripts/build
> > @@ -28,6 +28,10 @@ make -j$(nproc) dist
> >  
> >  # Extract artifacts to avoid getting rewritten by customised builds
> >  cp xen/.config xen-config
> > +mkdir binaries
> > +if [[ "x${XEN_TARGET_ARCH}" == "xx86_64" ]]; then
> 
> Either quote the strings, or use the pre-x trick, but you don't need to
> do both together.

Fixed, thanks.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel