[PATCH RFC 0/3] Improvements to docker-based build automation

[PATCH RFC 0/3] Improvements to docker-based build automation

[Wei Liu]

RFC because I haven't tested all dockerfiles.

Wei Liu (3):
  automation: install texinfo in debian
  automation: build stubdom and rombios, and tools on 32 bit
  automation: introduce stretch-i386.dockerfile

 automation/build/debian/jessie.dockerfile       |  1 +
 automation/build/debian/stretch-i386.dockerfile | 50 +++++++++++++++++++++++++
 automation/build/debian/stretch.dockerfile      |  1 +
 automation/scripts/build                        |  8 ----
 4 files changed, 52 insertions(+), 8 deletions(-)
 create mode 100644 automation/build/debian/stretch-i386.dockerfile

-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 1/3] automation: install texinfo in debian

[Wei Liu]

Stubdom build requires that.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 automation/build/debian/jessie.dockerfile  | 1 +
 automation/build/debian/stretch.dockerfile | 1 +
 2 files changed, 2 insertions(+)

diff --git a/automation/build/debian/jessie.dockerfile b/automation/build/debian/jessie.dockerfile
index 9bb1bdf104..3377a63334 100644
--- a/automation/build/debian/jessie.dockerfile
+++ b/automation/build/debian/jessie.dockerfile
@@ -41,6 +41,7 @@ RUN apt-get update && \
         checkpolicy \
         wget \
         git \
+        texinfo \
         && \
         apt-get autoremove -y && \
         apt-get clean && \
diff --git a/automation/build/debian/stretch.dockerfile b/automation/build/debian/stretch.dockerfile
index f068457ab6..3d3f140737 100644
--- a/automation/build/debian/stretch.dockerfile
+++ b/automation/build/debian/stretch.dockerfile
@@ -41,6 +41,7 @@ RUN apt-get update && \
         checkpolicy \
         wget \
         git \
+        texinfo \
         && \
         apt-get autoremove -y && \
         apt-get clean && \
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 2/3] automation: build stubdom and rombios, and tools on 32 bit

[Wei Liu]

Now we're using docker to build. They shouldn't be a problem anymore.

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
RFC because I haven't tested if other dockerfiles need changes.
---
 automation/scripts/build | 8 --------
 1 file changed, 8 deletions(-)

diff --git a/automation/scripts/build b/automation/scripts/build
index b90fc43d26..21fb029e8a 100755
--- a/automation/scripts/build
+++ b/automation/scripts/build
@@ -11,8 +11,6 @@ fi
 
 # build up our configure options
 cfgargs=()
-cfgargs+=("--disable-stubdom") # more work needed into building this
-cfgargs+=("--disable-rombios")
 cfgargs+=("--enable-docs")
 
 # SeaBIOS cannot be built with clang
@@ -20,12 +18,6 @@ if [[ "${CC}" == "clang" ]]; then
     cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin")
 fi
 
-if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
-    cfgargs+=("--enable-tools")
-else
-    cfgargs+=("--disable-tools") # we don't have the cross depends installed
-fi
-
 ./configure "${cfgargs[@]}"
 
 make dist
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH RFC 3/3] automation: introduce stretch-i386.dockerfile

[Wei Liu]

Signed-off-by: Wei Liu <wei.liu2@citrix.com>
---
 automation/build/debian/stretch-i386.dockerfile | 50 +++++++++++++++++++++++++
 1 file changed, 50 insertions(+)
 create mode 100644 automation/build/debian/stretch-i386.dockerfile

diff --git a/automation/build/debian/stretch-i386.dockerfile b/automation/build/debian/stretch-i386.dockerfile
new file mode 100644
index 0000000000..add2204535
--- /dev/null
+++ b/automation/build/debian/stretch-i386.dockerfile
@@ -0,0 +1,50 @@
+FROM i386/debian:stretch
+LABEL maintainer.name="The Xen Project" \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
+ENV DEBIAN_FRONTEND=noninteractive
+ENV USER root
+
+RUN mkdir /build
+WORKDIR /build
+
+ENTRYPOINT ["/usr/bin/setarch", "i686", "/bin/bash"]
+
+# build depends
+RUN apt-get update && \
+    apt-get --quiet --yes install \
+        build-essential \
+        zlib1g-dev \
+        libncurses5-dev \
+        libssl-dev \
+        python2.7-dev \
+        xorg-dev \
+        uuid-dev \
+        libyajl-dev \
+        libaio-dev \
+        libglib2.0-dev \
+        clang \
+        libpixman-1-dev \
+        pkg-config \
+        flex \
+        bison \
+        gettext \
+        acpica-tools \
+        bin86 \
+        bcc \
+        liblzma-dev \
+        libc6-dev \
+        libnl-3-dev \
+        ocaml-nox \
+        libfindlib-ocaml-dev \
+        markdown \
+        transfig \
+        pandoc \
+        checkpolicy \
+        wget \
+        git \
+        texinfo \
+        && \
+        apt-get autoremove -y && \
+        apt-get clean && \
+        rm -rf /var/lib/apt/lists* /tmp/* /var/tmp/*
-- 
2.11.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 2/3] automation: build stubdom and rombios, and tools on 32 bit

[Doug Goldstein]

On 5/23/18 5:20 AM, Wei Liu wrote:

>  
>  # SeaBIOS cannot be built with clang
> @@ -20,12 +18,6 @@ if [[ "${CC}" == "clang" ]]; then
>      cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin")
>  fi
>  
> -if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
> -    cfgargs+=("--enable-tools")
> -else
> -    cfgargs+=("--disable-tools") # we don't have the cross depends installed
> -fi
> -

This still horribly fails when targeting arm because we have to actually
execute a compiled binary on the host. The tools build system doesn't
seem to comprehend host != target.

A few people have stepped forward with ideas and said they'd take point
on the patches but I haven't seen anything land yet.

-- 
Doug Goldstein

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 1/3] automation: install texinfo in debian

[Doug Goldstein]

On 5/23/18 5:20 AM, Wei Liu wrote:
> Stubdom build requires that.
> 

CentOS & Ubuntu will need the change too. Though in a way I'd almost ask
why --enable-docs doesn't need it but stubdoms do? Seems like a weird
dep for stubdom...

-- 
Doug Goldstein

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 1/3] automation: install texinfo in debian

[Wei Liu]

On Tue, May 29, 2018 at 12:21:18AM -0500, Doug Goldstein wrote:
> On 5/23/18 5:20 AM, Wei Liu wrote:
> > Stubdom build requires that.
> > 
> 
> CentOS & Ubuntu will need the change too. Though in a way I'd almost ask
> why --enable-docs doesn't need it but stubdoms do? Seems like a weird
> dep for stubdom...
> 

It is needed by newlib IIRC.

Wei.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 2/3] automation: build stubdom and rombios, and tools on 32 bit

[Wei Liu]

On Tue, May 29, 2018 at 12:18:40AM -0500, Doug Goldstein wrote:
> On 5/23/18 5:20 AM, Wei Liu wrote:
> 
> >  
> >  # SeaBIOS cannot be built with clang
> > @@ -20,12 +18,6 @@ if [[ "${CC}" == "clang" ]]; then
> >      cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin")
> >  fi
> >  
> > -if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
> > -    cfgargs+=("--enable-tools")
> > -else
> > -    cfgargs+=("--disable-tools") # we don't have the cross depends installed
> > -fi
> > -
> 
> This still horribly fails when targeting arm because we have to actually
> execute a compiled binary on the host. The tools build system doesn't
> seem to comprehend host != target.
> 

OK. Let's only enable tools build for x86 for now.

> A few people have stepped forward with ideas and said they'd take point
> on the patches but I haven't seen anything land yet.
> 

Any reference?

Wei.

> -- 
> Doug Goldstein

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 2/3] automation: build stubdom and rombios, and tools on 32 bit

[Doug Goldstein]

On Wed, May 30, 2018 at 12:28:24AM +0100, Wei Liu wrote:
> On Tue, May 29, 2018 at 12:18:40AM -0500, Doug Goldstein wrote:
> > On 5/23/18 5:20 AM, Wei Liu wrote:
> > 
> > >  
> > >  # SeaBIOS cannot be built with clang
> > > @@ -20,12 +18,6 @@ if [[ "${CC}" == "clang" ]]; then
> > >      cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin")
> > >  fi
> > >  
> > > -if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
> > > -    cfgargs+=("--enable-tools")
> > > -else
> > > -    cfgargs+=("--disable-tools") # we don't have the cross depends installed
> > > -fi
> > > -
> > 
> > This still horribly fails when targeting arm because we have to actually
> > execute a compiled binary on the host. The tools build system doesn't
> > seem to comprehend host != target.
> > 
> 
> OK. Let's only enable tools build for x86 for now.
> 
> > A few people have stepped forward with ideas and said they'd take point
> > on the patches but I haven't seen anything land yet.
> > 
> 
> Any reference?
> 
> Wei.

Stefano originally offered I believe. He also offered to help with some
of the bits again at the Summit. Matt Spencer suggested that ARM could
help out with info on how to build the containers and potentially some
resources where they could run as well.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH RFC 2/3] automation: build stubdom and rombios, and tools on 32 bit

[Stefano Stabellini]

On Fri, 6 Jul 2018, Doug Goldstein wrote:
> On Wed, May 30, 2018 at 12:28:24AM +0100, Wei Liu wrote:
> > On Tue, May 29, 2018 at 12:18:40AM -0500, Doug Goldstein wrote:
> > > On 5/23/18 5:20 AM, Wei Liu wrote:
> > > 
> > > >  
> > > >  # SeaBIOS cannot be built with clang
> > > > @@ -20,12 +18,6 @@ if [[ "${CC}" == "clang" ]]; then
> > > >      cfgargs+=("--with-system-seabios=/usr/share/seabios/bios.bin")
> > > >  fi
> > > >  
> > > > -if [[ "${XEN_TARGET_ARCH}" == "x86_64" ]]; then
> > > > -    cfgargs+=("--enable-tools")
> > > > -else
> > > > -    cfgargs+=("--disable-tools") # we don't have the cross depends installed
> > > > -fi
> > > > -
> > > 
> > > This still horribly fails when targeting arm because we have to actually
> > > execute a compiled binary on the host. The tools build system doesn't
> > > seem to comprehend host != target.
> > > 
> > 
> > OK. Let's only enable tools build for x86 for now.
> > 
> > > A few people have stepped forward with ideas and said they'd take point
> > > on the patches but I haven't seen anything land yet.
> > > 
> > 
> > Any reference?
> > 
> > Wei.
> 
> Stefano originally offered I believe. He also offered to help with some
> of the bits again at the Summit. 

Yes, containerized builds and stubdoms support are on the ViryaOS
roadmap (as mentioned in my talk). We are not there yet, but we should
have something to share by the end of the year.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel