[Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Dario Faggioli]

The openSUSE distribution comes in two flavours: Leap, which is
non-rolling, and released annualy, and Tumbleweed, which is rolling.

In general, they are quite similar, but the versions of the software
they ship can be significantly different. As it is easy to imagine,
Tumbleweed, being rolling, has much more recent and updated packages.

Actually, Tumbleweed often ships the most updated releases of various
software and projects, among many distribution around.

For instance, right now, Tumbleweed has gcc 9.1.1. Fedora Rawhide also
seems to have 9.1.1, while Fedora 30 has 9.0.1. Debian Unstable and
Ubuntu Disco both have 8.3.0, AFAICT.

Of course, it's not at all like "the more updated the better", or
anything like that! It's just that I see some value in having, as a part
of our CI:
- more diversity, in terms of versions of the packages/software we try
  to build Xen with and against;
- a "bleeding edge" environment, in order to catch, or at least be
  aware of, build issues with latest compilers and/or when linking
  against the latest version of this and that library.

Interestingly, in the past couple of days, a few build issues of Xen,
qemu-xen and ipxe (at the commit that we were checking out) with gcc
9.1.1 where discovered and fixed. And --at least as far as the ones
I've reported/fixed-- I found about them while building Xen in
openSUSE Tumbleweed (while working on this patch series :-D ).

Happy to hear what people think about all this. :-)

Oh, BTW, apart from adding the Tumbleweed dockerfile and CI jobs, the
series also does some minor tweaking of the already present openSUSE
Leap dockerfile (in patch 1).

Thanks and Regards.
---
Dario Faggioli (3):
      automation: try to keep openSUSE Leap image a little smaller
      automation: add openSUSE Tumbleweed CI image
      automation: build in openSUSE Tumbleweed

 automation/build/suse/opensuse-leap.dockerfile     |    6 +-
 .../build/suse/opensuse-tumbleweed.dockerfile      |   68 ++++++++++++++++++++
 automation/gitlab-ci/build.yaml                    |   20 ++++++
 3 files changed, 91 insertions(+), 3 deletions(-)
 create mode 100644 automation/build/suse/opensuse-tumbleweed.dockerfile
--
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 1/3] automation: try to keep openSUSE Leap image a little smaller

[Dario Faggioli]

Using `--no-recommends` when updating or installing commands should
prevent non strictly necessary packages to be installed.

doing a `clean -a` after installing all the packages, should, in
theory, free more space (as opposed to using just `clean`).

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Wei Liu <wl@xen.org>
---
 automation/build/suse/opensuse-leap.dockerfile |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/automation/build/suse/opensuse-leap.dockerfile b/automation/build/suse/opensuse-leap.dockerfile
index 614a5c8405..eb70419002 100644
--- a/automation/build/suse/opensuse-leap.dockerfile
+++ b/automation/build/suse/opensuse-leap.dockerfile
@@ -7,8 +7,8 @@ ENV USER root
 RUN mkdir /build
 WORKDIR /build
 
-RUN zypper ref && zypper up -y
-RUN zypper install -y \
+RUN zypper ref && zypper up -y --no-recommends
+RUN zypper install -y --no-recommends \
         acpica \
         bc \
         bin86 \
@@ -64,4 +64,4 @@ RUN zypper install -y \
         xz-devel \
         zlib-devel \
         && \
-        zypper clean
+        zypper clean -a


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 2/3] automation: add openSUSE Tumbleweed CI image

[Dario Faggioli]

openSUSE comes in two flavours: Leap, which is non-rolling, and released
annualy, and Tumbleweed, which is rolling.

Reasons why it makes sense to have both (despite both being openSUSE,
package lists in dockerfiles being quite similar, etc) are:
- Leap share a lot with SUSE Linux Enterprise. So, regressions on Leap,
  not only means regressions for all openSUSE Leap users, but also helps
  prevent/catch regressions on SLE;
- Tumbleweed often has the most bleeding-edge software, so it will help
  us prevent/catch regressions with newly released versions of
  libraries, compilers, etc (e.g., at the time of writing this commit,
  some build issues, with GCC9, where discovered while trying to build
  in a Tumbleweed image).

Note that, considering the rolling nature of Tumbleweed, the container
would need to be rebuilt (e.g., periodically), even if the docker file
does not change.

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Wei Liu <wl@xen.org>
---
 .../build/suse/opensuse-tumbleweed.dockerfile      |   68 ++++++++++++++++++++
 1 file changed, 68 insertions(+)
 create mode 100644 automation/build/suse/opensuse-tumbleweed.dockerfile

diff --git a/automation/build/suse/opensuse-tumbleweed.dockerfile b/automation/build/suse/opensuse-tumbleweed.dockerfile
new file mode 100644
index 0000000000..2676a87c85
--- /dev/null
+++ b/automation/build/suse/opensuse-tumbleweed.dockerfile
@@ -0,0 +1,68 @@
+FROM opensuse/tumbleweed
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+RUN zypper ref && zypper up -y --no-recommends
+RUN zypper install -y --no-recommends \
+        acpica \
+        bc \
+        bin86 \
+        bison \
+        bzip2 \
+        checkpolicy \
+        clang \
+        cmake \
+        dev86 \
+        discount \
+        flex \
+        gcc \
+        gcc-c++ \
+        gettext-tools \
+        git \
+        glib2-devel \
+        glibc-devel \
+        glibc-devel-32bit \
+        gzip \
+        hostname \
+        libSDL2-devel \
+        libaio-devel \
+        libbz2-devel \
+        libext2fs-devel \
+        libgnutls-devel \
+        libjpeg62-devel \
+        libnl3-devel \
+        libnuma-devel \
+        libpixman-1-0-devel \
+        libpng16-devel \
+        libssh2-devel \
+        libtasn1-devel \
+        libuuid-devel \
+        libyajl-devel \
+        lzo-devel \
+        make \
+        nasm \
+        ncurses-devel \
+        ocaml \
+        ocaml-findlib-devel \
+        ocaml-ocamlbuild \
+        ocaml-ocamldoc \
+        pandoc \
+        patch \
+        pkg-config \
+        python \
+        python-devel \
+        systemd-devel \
+        tar \
+        transfig \
+        valgrind-devel \
+        wget \
+        which \
+        xz-devel \
+        zlib-devel \
+        && \
+        zypper clean -a


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [PATCH 3/3] automation: build in openSUSE Tumbleweed

[Dario Faggioli]

Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
---
Cc: Doug Goldstein <cardoe@cardoe.com>
Cc: Wei Liu <wl@xen.org>
---
 automation/gitlab-ci/build.yaml |   20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml
index 1e61d30c85..94877f9d01 100644
--- a/automation/gitlab-ci/build.yaml
+++ b/automation/gitlab-ci/build.yaml
@@ -420,6 +420,26 @@ opensuse-leap-gcc-debug:
   variables:
     CONTAINER: suse:opensuse-leap
 
+opensuse-tumbleweed-clang:
+  extends: .clang-x86-64-build
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+
+opensuse-tumbleweed-clang-debug:
+  extends: .clang-x86-64-build-debug
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+
+opensuse-tumbleweed-gcc:
+  extends: .gcc-x86-64-build
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+
+opensuse-tumbleweed-gcc-debug:
+  extends: .gcc-x86-64-build-debug
+  variables:
+    CONTAINER: suse:opensuse-tumbleweed
+
 # Arm builds
 
 debian-unstable-gcc-arm64:


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 1/3] automation: try to keep openSUSE Leap image a little smaller

[Doug Goldstein]

On 7/31/19 11:58 AM, Dario Faggioli wrote:

> Using `--no-recommends` when updating or installing commands should
> prevent non strictly necessary packages to be installed.
>
> doing a `clean -a` after installing all the packages, should, in
> theory, free more space (as opposed to using just `clean`).
>
> Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 2/3] automation: add openSUSE Tumbleweed CI image

[Doug Goldstein]

On 7/31/19 11:58 AM, Dario Faggioli wrote:

> openSUSE comes in two flavours: Leap, which is non-rolling, and released
> annualy, and Tumbleweed, which is rolling.
>
> Reasons why it makes sense to have both (despite both being openSUSE,
> package lists in dockerfiles being quite similar, etc) are:
> - Leap share a lot with SUSE Linux Enterprise. So, regressions on Leap,
>    not only means regressions for all openSUSE Leap users, but also helps
>    prevent/catch regressions on SLE;
> - Tumbleweed often has the most bleeding-edge software, so it will help
>    us prevent/catch regressions with newly released versions of
>    libraries, compilers, etc (e.g., at the time of writing this commit,
>    some build issues, with GCC9, where discovered while trying to build
>    in a Tumbleweed image).
>
> Note that, considering the rolling nature of Tumbleweed, the container
> would need to be rebuilt (e.g., periodically), even if the docker file
> does not change.
>
> Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 3/3] automation: build in openSUSE Tumbleweed

[Doug Goldstein]

On 7/31/19 11:58 AM, Dario Faggioli wrote:

> Signed-off-by: Dario Faggioli <dfaggioli@suse.com>
Acked-by: Doug Goldstein <cardoe@cardoe.com>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Andrew Cooper]

On 31/07/2019 17:58, Dario Faggioli wrote:
> The openSUSE distribution comes in two flavours: Leap, which is
> non-rolling, and released annualy, and Tumbleweed, which is rolling.
>
> In general, they are quite similar, but the versions of the software
> they ship can be significantly different. As it is easy to imagine,
> Tumbleweed, being rolling, has much more recent and updated packages.
>
> Actually, Tumbleweed often ships the most updated releases of various
> software and projects, among many distribution around.
>
> For instance, right now, Tumbleweed has gcc 9.1.1. Fedora Rawhide also
> seems to have 9.1.1, while Fedora 30 has 9.0.1. Debian Unstable and
> Ubuntu Disco both have 8.3.0, AFAICT.
>
> Of course, it's not at all like "the more updated the better", or
> anything like that! It's just that I see some value in having, as a part
> of our CI:
> - more diversity, in terms of versions of the packages/software we try
>   to build Xen with and against;
> - a "bleeding edge" environment, in order to catch, or at least be
>   aware of, build issues with latest compilers and/or when linking
>   against the latest version of this and that library.
>
> Interestingly, in the past couple of days, a few build issues of Xen,
> qemu-xen and ipxe (at the commit that we were checking out) with gcc
> 9.1.1 where discovered and fixed. And --at least as far as the ones
> I've reported/fixed-- I found about them while building Xen in
> openSUSE Tumbleweed (while working on this patch series :-D ).
>
> Happy to hear what people think about all this. :-)
>
> Oh, BTW, apart from adding the Tumbleweed dockerfile and CI jobs, the
> series also does some minor tweaking of the already present openSUSE
> Leap dockerfile (in patch 1).
>
> Thanks and Regards.
> ---
> Dario Faggioli (3):
>       automation: try to keep openSUSE Leap image a little smaller

I've pushed patch 1, and refreshed the leap container.

I have also pushed a tumbleweed container, but the build is currently
failing in Qemu.

https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/265301303

I have also pushed patch 2, because that is the dockerfile which matches
the current container.  However, I won't push patch 3 (adding it into
CI) until the build errors are resolved.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1564 bytes --]

On Sat, 2019-08-03 at 17:22 +0100, Andrew Cooper wrote:
> On 31/07/2019 17:58, Dario Faggioli wrote:
> > 
> > Interestingly, in the past couple of days, a few build issues of
> > Xen,
> > qemu-xen and ipxe (at the commit that we were checking out) with
> > gcc
> > 9.1.1 where discovered and fixed. And --at least as far as the ones
> > I've reported/fixed-- I found about them while building Xen in
> > openSUSE Tumbleweed (while working on this patch series :-D ).
> > 
> > ---
> > Dario Faggioli (3):
> >       automation: try to keep openSUSE Leap image a little smaller
> 
> I've pushed patch 1, and refreshed the leap container.
> 
> I have also pushed a tumbleweed container, but the build is currently
> failing in Qemu.
> 
> https://gitlab.com/xen-project/people/andyhhp/xen/-/jobs/265301303
> 
Ah, interesting... I tested locally and wasn't seeing this.

Anyway, it seems that, in addition to the one I mentioned to Anthony a
few days ago, we also miss this from upstream?

commit 2d2023c3b99edb33ad4bb9791f70456ea1a1c049
sockets: avoid string truncation warnings when copying UNIX path

Anthony, do you think it's fine to import that patch? Or should I try
to rebuild and see if run into the issue this time, and if that patch
fixes it?

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Anthony PERARD]

On Mon, Aug 05, 2019 at 11:44:11AM +0200, Dario Faggioli wrote:
> Anyway, it seems that, in addition to the one I mentioned to Anthony a
> few days ago, we also miss this from upstream?
> 
> commit 2d2023c3b99edb33ad4bb9791f70456ea1a1c049
> sockets: avoid string truncation warnings when copying UNIX path
> 
> Anthony, do you think it's fine to import that patch? Or should I try
> to rebuild and see if run into the issue this time, and if that patch
> fixes it?

I've pushed that patch and that other one:
  ccb799313a5926a6aa49018bbc67fe6165fad7f3
  hw/usb: avoid format truncation warning when formatting port name

with those two I could build a qemu in the tumbleweed container, but
there are maybe other commit missing.

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Dario Faggioli]

[-- Attachment #1.1: Type: text/plain, Size: 1354 bytes --]

On Tue, 2019-08-06 at 11:12 +0100, Anthony PERARD wrote:
> On Mon, Aug 05, 2019 at 11:44:11AM +0200, Dario Faggioli wrote:
> > Anyway, it seems that, in addition to the one I mentioned to
> > Anthony a
> > few days ago, we also miss this from upstream?
> > 
> > commit 2d2023c3b99edb33ad4bb9791f70456ea1a1c049
> > sockets: avoid string truncation warnings when copying UNIX path
> > 
> > Anthony, do you think it's fine to import that patch? Or should I
> > try
> > to rebuild and see if run into the issue this time, and if that
> > patch
> > fixes it?
> 
> I've pushed that patch and that other one:
>   ccb799313a5926a6aa49018bbc67fe6165fad7f3
>   hw/usb: avoid format truncation warning when formatting port name
> 
> with those two I could build a qemu in the tumbleweed container, but
> there are maybe other commit missing.
> 
You mean the build is still failing for you, but not in QEMU?

I've just tried, and the only problem I'm seeing is the
test-cpu-policy.c one that has already been reported, but is believed
to be a gcc problem.

Regards
-- 
Dario Faggioli, Ph.D
http://about.me/dario.faggioli
Virtualization Software Engineer
SUSE Labs, SUSE https://www.suse.com/
-------------------------------------------------------------------
<<This happens because _I_ choose it to happen!>> (Raistlin Majere)


[-- Attachment #1.2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [PATCH 0/3] automation: build Xen in openSUSE Tumbleweed

[Anthony PERARD]

On Tue, Aug 06, 2019 at 03:17:26PM +0200, Dario Faggioli wrote:
> On Tue, 2019-08-06 at 11:12 +0100, Anthony PERARD wrote:
> > with those two I could build a qemu in the tumbleweed container, but
> > there are maybe other commit missing.
> > 
> You mean the build is still failing for you, but not in QEMU?

I have built QEMU but without Xen. The container I used didn't have the
Xen libraries and I didn't try to build or install them.

> I've just tried, and the only problem I'm seeing is the
> test-cpu-policy.c one that has already been reported, but is believed
> to be a gcc problem.

Thanks for confirming it all works :).

-- 
Anthony PERARD

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel