[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

In order to run our test.py suite on a number of platforms in CI we use
QEMU.  Configure and build a copy here so that all our jobs can rely on
it as our code is under test and not QEMU.

Signed-off-by: Tom Rini <trini@konsulko.com>
---
 Dockerfile | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/Dockerfile b/Dockerfile
index b599af8af6a0..73c1b8b1e021 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -76,6 +76,14 @@ RUN apt-get update && apt-get install -y \
 	zip \
 	&& rm -rf /var/lib/apt/lists/*
 
+RUN git clone git://git.qemu.org/qemu.git /tmp/qemu && \
+	cd /tmp/qemu && \
+	git submodule update --init dtc && \
+	git checkout 506179e42112be77bfd071f050b15762d3b2cd43 && \
+	./configure --prefix=/opt/qemu --target-list="aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,ppc-softmmu,x86_64-softmmu,xtensa-softmmu" && \
+	make -j$(nproc) all install && \
+	rm -rf /tmp/qemu
+
 # Create the buildman config file
 RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman
 RUN /bin/echo -e "kernelorg = /opt/gcc-7.3.0-nolibc/*" >> ~/.buildman
-- 
2.7.4

[U-Boot] [PATCH] gitlab-ci: Switch to a Docker image that contains a QEMU we build

[Tom Rini]

Rather than buiding QEMU for each test.py build it once in our
Dockerfile and re-use it as needed.

Signed-off-by: Tom Rini <trini@konsulko.com>
---
 .gitlab-ci.yml | 36 +++++-------------------------------
 1 file changed, 5 insertions(+), 31 deletions(-)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index e27d86f96281..e2b73c1272b5 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -2,12 +2,12 @@
 
 # Grab our configured image.  The source for this is found at:
 # https://gitlab.denx.de/u-boot/gitlab-ci-runner
-image: trini/u-boot-gitlab-ci-runner:xenial-20190222-24April2019
+image: trini/u-boot-gitlab-ci-runner:xenial-20190720-24Jul2019-b
 
 # We run some tests in different order, to catch some failures quicker.
 stages:
-  - test.py
   - testsuites
+  - test.py
   - world build
 
 .buildman_and_testpy_template: &buildman_and_testpy_dfn
@@ -28,17 +28,9 @@ stages:
     - ( cd ~/grub2-arm; wget -O - http://download.opensuse.org/ports/armv7hl/distribution/leap/42.2/repo/oss/suse/armv7hl/grub2-arm-efi-2.02~beta2-87.1.armv7hl.rpm | rpm2cpio | cpio -di )
     - mkdir ~/grub2-arm64
     - ( cd ~/grub2-arm64; wget -O - http://download.opensuse.org/ports/aarch64/distribution/leap/42.2/repo/oss/suse/aarch64/grub2-arm64-efi-2.02~beta2-87.1.aarch64.rpm | rpm2cpio | cpio -di )
-    - if [[ "${QEMU_TARGET}" != "" ]]; then
-        git clone git://git.qemu.org/qemu.git /tmp/qemu;
-        pushd /tmp/qemu;
-        git submodule update --init dtc &&
-        git checkout ${QEMU_VERSION} &&
-        ./configure --prefix=/tmp/qemu-install --target-list=${QEMU_TARGET} &&
-        make -j$(nproc) all install;
-        popd;
-      fi
+
   after_script:
-    - rm -rf ~/grub2* /tmp/uboot-test-hooks /tmp/qemu /tmp/venv
+    - rm -rf ~/grub2* /tmp/uboot-test-hooks /tmp/venv
   script:
     # From buildman, exit code 129 means warnings only.  If we've been asked to
     # use clang only do one configuration.
@@ -55,7 +47,7 @@ stages:
     # "-k something" even when $TEST_PY_TEST_SPEC doesnt need a custom
     # value.
     - export UBOOT_TRAVIS_BUILD_DIR=`cd .. && pwd`/.bm-work/${TEST_PY_BD};
-      export PATH=/tmp/qemu-install/bin:/tmp/uboot-test-hooks/bin:/usr/bin:/bin;
+      export PATH=/opt/qemu/bin:/tmp/uboot-test-hooks/bin:/usr/bin:/bin;
       export PYTHONPATH=/tmp/uboot-test-hooks/py/travis-ci;
       if [[ "${TEST_PY_BD}" != "" ]]; then
         ./test/py/test.py --bd ${TEST_PY_BD} ${TEST_PY_ID}
@@ -200,8 +192,6 @@ evb-ast2500 test.py:
   variables:
     TEST_PY_BD: "evb-ast2500"
     TEST_PY_ID: "--id qemu"
-    QEMU_TARGET: "arm-softmmu"
-    QEMU_VERSION: "506179e42112be77bfd071f050b15762d3b2cd43"
     BUILDMAN: "^evb-ast2500$"
   <<: *buildman_and_testpy_dfn
 
@@ -217,8 +207,6 @@ vexpress_ca15_tc2 test.py:
   variables:
     TEST_PY_BD: "vexpress_ca15_tc2"
     TEST_PY_ID: "--id qemu"
-    QEMU_TARGET: "arm-softmmu"
-    QEMU_VERSION: "v3.0.0"
     BUILDMAN: "^vexpress_ca15_tc2$"
   <<: *buildman_and_testpy_dfn
 
@@ -227,7 +215,6 @@ vexpress_ca9x4 test.py:
   variables:
     TEST_PY_BD: "vexpress_ca9x4"
     TEST_PY_ID: "--id qemu"
-    QEMU_TARGET: "arm-softmmu"
     BUILDMAN: "^vexpress_ca9x4$"
   <<: *buildman_and_testpy_dfn
 
@@ -237,7 +224,6 @@ integratorcp_cm926ejs test.py:
     TEST_PY_BD: "integratorcp_cm926ejs"
     TEST_PY_TEST_SPEC: "not sleep"
     TEST_PY_ID: "--id qemu"
-    QEMU_TARGET: "arm-softmmu"
     BUILDMAN: "^integratorcp_cm926ejs$"
   <<: *buildman_and_testpy_dfn
 
@@ -246,7 +232,6 @@ qemu_arm test.py:
   variables:
     TEST_PY_BD: "qemu_arm"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "arm-softmmu"
     BUILDMAN: "^qemu_arm$"
   <<: *buildman_and_testpy_dfn
 
@@ -255,7 +240,6 @@ qemu_arm64 test.py:
   variables:
     TEST_PY_BD: "qemu_arm64"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "aarch64-softmmu"
     BUILDMAN: "^qemu_arm64$"
   <<: *buildman_and_testpy_dfn
 
@@ -264,7 +248,6 @@ qemu_mips test.py:
   variables:
     TEST_PY_BD: "qemu_mips"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "mips-softmmu"
     BUILDMAN: "^qemu_mips$"
     TOOLCHAIN: "mips"
   <<: *buildman_and_testpy_dfn
@@ -274,7 +257,6 @@ qemu_mipsel test.py:
   variables:
     TEST_PY_BD: "qemu_mipsel"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "mipsel-softmmu"
     BUILDMAN: "^qemu_mipsel$"
     TOOLCHAIN: "mips"
   <<: *buildman_and_testpy_dfn
@@ -284,7 +266,6 @@ qemu_mips64 test.py:
   variables:
     TEST_PY_BD: "qemu_mips64"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "mips64-softmmu"
     BUILDMAN: "^qemu_mips64$"
     TOOLCHAIN: "mips"
   <<: *buildman_and_testpy_dfn
@@ -294,7 +275,6 @@ qemu_mips64el test.py:
   variables:
     TEST_PY_BD: "qemu_mips64el"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "mips64el-softmmu"
     BUILDMAN: "^qemu_mips64el$"
     TOOLCHAIN: "mips"
   <<: *buildman_and_testpy_dfn
@@ -304,7 +284,6 @@ qemu-ppce500 test.py:
   variables:
     TEST_PY_BD: "qemu-ppce500"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "ppc-softmmu"
     BUILDMAN: "^qemu-ppce500$"
     TOOLCHAIN: "powerpc"
   <<: *buildman_and_testpy_dfn
@@ -314,7 +293,6 @@ qemu-x86 test.py:
   variables:
     TEST_PY_BD: "qemu-x86"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "i386-softmmu"
     BUILDMAN: "^qemu-x86$"
     TOOLCHAIN: "i386"
   <<: *buildman_and_testpy_dfn
@@ -324,7 +302,6 @@ qemu-x86_64 test.py:
   variables:
     TEST_PY_BD: "qemu-x86_64"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "x86_64-softmmu"
     BUILDMAN: "^qemu-x86_64$"
     TOOLCHAIN: "i386"
   <<: *buildman_and_testpy_dfn
@@ -334,7 +311,6 @@ zynq_zc702 test.py:
   variables:
     TEST_PY_BD: "zynq_zc702"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "arm-softmmu"
     TEST_PY_ID: "--id qemu"
     BUILDMAN: "^zynq_zc702$"
   <<: *buildman_and_testpy_dfn
@@ -344,7 +320,6 @@ xilinx_versal_virt test.py:
   variables:
     TEST_PY_BD: "xilinx_versal_virt"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "aarch64-softmmu"
     TEST_PY_ID: "--id qemu"
     BUILDMAN: "^xilinx_versal_virt$"
   <<: *buildman_and_testpy_dfn
@@ -354,7 +329,6 @@ xtfpga test.py:
   variables:
     TEST_PY_BD: "xtfpga"
     TEST_PY_TEST_SPEC: "not sleep"
-    QEMU_TARGET: "xtensa-softmmu"
     TEST_PY_ID: "--id qemu"
     BUILDMAN: "^xtfpga$"
     TOOLCHAIN: "xtensa-dc233c-elf"
-- 
2.7.4

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

On Wed, Jul 24, 2019 at 03:51:14PM -0400, Tom Rini wrote:

> In order to run our test.py suite on a number of platforms in CI we use
> QEMU.  Configure and build a copy here so that all our jobs can rely on
> it as our code is under test and not QEMU.
> 
> Signed-off-by: Tom Rini <trini@konsulko.com>

I looked into using the artifact method in GitLab.  The problem there is
that artifacts are stored within the job and kept.  So rather than a
stand-alone project that we can reference and pass we would build it
once per pipeline and pass it, but then store it until we purge things
later when we notice gitlab.denx.de is out of space.  Going this route I
think makes the most sense for what we do.  It does remove the ability
to roll back a single platform (or platforms) to an older QEMU but I'm
not sure that's a big deal and we COULD re-do that as needed.  Over at
https://gitlab.denx.de/u-boot/u-boot/pipelines/324 all QEMU platforms
pass again so we can use the latest version which allows evb-ast2500 to
be used.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190724/6ba64411/attachment.sig>

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

On Wed, Jul 24, 2019 at 03:56:21PM -0400, Tom Rini wrote:
> On Wed, Jul 24, 2019 at 03:51:14PM -0400, Tom Rini wrote:
> 
> > In order to run our test.py suite on a number of platforms in CI we use
> > QEMU.  Configure and build a copy here so that all our jobs can rely on
> > it as our code is under test and not QEMU.
> > 
> > Signed-off-by: Tom Rini <trini@konsulko.com>
> 
> I looked into using the artifact method in GitLab.  The problem there is
> that artifacts are stored within the job and kept.  So rather than a
> stand-alone project that we can reference and pass we would build it
> once per pipeline and pass it, but then store it until we purge things
> later when we notice gitlab.denx.de is out of space.  Going this route I
> think makes the most sense for what we do.  It does remove the ability
> to roll back a single platform (or platforms) to an older QEMU but I'm
> not sure that's a big deal and we COULD re-do that as needed.  Over at
> https://gitlab.denx.de/u-boot/u-boot/pipelines/324 all QEMU platforms
> pass again so we can use the latest version which allows evb-ast2500 to
> be used.

Heinrich, what do you think about extending this for GRUB binaries for
test.py?

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190725/59db4113/attachment.sig>

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Heinrich Schuchardt]

On 7/24/19 9:51 PM, Tom Rini wrote:
> In order to run our test.py suite on a number of platforms in CI we use
> QEMU.  Configure and build a copy here so that all our jobs can rely on
> it as our code is under test and not QEMU.
>
> Signed-off-by: Tom Rini <trini@konsulko.com>
> ---
>   Dockerfile | 8 ++++++++
>   1 file changed, 8 insertions(+)
>
> diff --git a/Dockerfile b/Dockerfile
> index b599af8af6a0..73c1b8b1e021 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -76,6 +76,14 @@ RUN apt-get update && apt-get install -y \
>   	zip \
>   	&& rm -rf /var/lib/apt/lists/*
>
> +RUN git clone git://git.qemu.org/qemu.git /tmp/qemu && \
> +	cd /tmp/qemu && \
> +	git submodule update --init dtc && \
> +	git checkout 506179e42112be77bfd071f050b15762d3b2cd43 && \
> +	./configure --prefix=/opt/qemu --target-list="aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,ppc-softmmu,x86_64-softmmu,xtensa-softmmu" && \
> +	make -j$(nproc) all install && \
> +	rm -rf /tmp/qemu
> +
>   # Create the buildman config file
>   RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman
>   RUN /bin/echo -e "kernelorg = /opt/gcc-7.3.0-nolibc/*" >> ~/.buildman
>

The file '/Dockerfile' does not exist in the current U-Boot
origin/master. It is hard to evaluate the patch without having the
complete file.

Where would I find the file?
It might be useful to add it to tools/ in the U-Boot git.

Best regards

Heinrich

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

On Thu, Jul 25, 2019 at 09:43:10PM +0200, Heinrich Schuchardt wrote:
> On 7/24/19 9:51 PM, Tom Rini wrote:
> >In order to run our test.py suite on a number of platforms in CI we use
> >QEMU.  Configure and build a copy here so that all our jobs can rely on
> >it as our code is under test and not QEMU.
> >
> >Signed-off-by: Tom Rini <trini@konsulko.com>
> >---
> >  Dockerfile | 8 ++++++++
> >  1 file changed, 8 insertions(+)
> >
> >diff --git a/Dockerfile b/Dockerfile
> >index b599af8af6a0..73c1b8b1e021 100644
> >--- a/Dockerfile
> >+++ b/Dockerfile
> >@@ -76,6 +76,14 @@ RUN apt-get update && apt-get install -y \
> >  	zip \
> >  	&& rm -rf /var/lib/apt/lists/*
> >
> >+RUN git clone git://git.qemu.org/qemu.git /tmp/qemu && \
> >+	cd /tmp/qemu && \
> >+	git submodule update --init dtc && \
> >+	git checkout 506179e42112be77bfd071f050b15762d3b2cd43 && \
> >+	./configure --prefix=/opt/qemu --target-list="aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,ppc-softmmu,x86_64-softmmu,xtensa-softmmu" && \
> >+	make -j$(nproc) all install && \
> >+	rm -rf /tmp/qemu
> >+
> >  # Create the buildman config file
> >  RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman
> >  RUN /bin/echo -e "kernelorg = /opt/gcc-7.3.0-nolibc/*" >> ~/.buildman
> >
> 
> The file '/Dockerfile' does not exist in the current U-Boot
> origin/master. It is hard to evaluate the patch without having the
> complete file.
> 
> Where would I find the file?

Sorry, it's over at https://gitlab.denx.de/u-boot/gitlab-ci-runner

> It might be useful to add it to tools/ in the U-Boot git.

Currently I have:
$ head -n4 .gitlab-ci.yml
# SPDX-License-Identifier: GPL-2.0+

# Grab our configured image.  The source for this is found at:
# https://gitlab.denx.de/u-boot/gitlab-ci-runner

But suggestions on where to note this are welcome.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190725/4ac91f08/attachment.sig>

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Heinrich Schuchardt]

On 7/25/19 9:17 PM, Tom Rini wrote:
> On Wed, Jul 24, 2019 at 03:56:21PM -0400, Tom Rini wrote:
>> On Wed, Jul 24, 2019 at 03:51:14PM -0400, Tom Rini wrote:
>>
>>> In order to run our test.py suite on a number of platforms in CI we use
>>> QEMU.  Configure and build a copy here so that all our jobs can rely on
>>> it as our code is under test and not QEMU.
>>>
>>> Signed-off-by: Tom Rini <trini@konsulko.com>
>>
>> I looked into using the artifact method in GitLab.  The problem there is
>> that artifacts are stored within the job and kept.  So rather than a
>> stand-alone project that we can reference and pass we would build it
>> once per pipeline and pass it, but then store it until we purge things
>> later when we notice gitlab.denx.de is out of space.  Going this route I
>> think makes the most sense for what we do.  It does remove the ability
>> to roll back a single platform (or platforms) to an older QEMU but I'm
>> not sure that's a big deal and we COULD re-do that as needed.  Over at
>> https://gitlab.denx.de/u-boot/u-boot/pipelines/324 all QEMU platforms
>> pass again so we can use the latest version which allows evb-ast2500 to
>> be used.
>
> Heinrich, what do you think about extending this for GRUB binaries for
> test.py?
>
Hello Tom,

it saves a lot of CPU time if GRUB and QEMU are not built for every
single push to the U-Boot git. It should be sufficient to build them
once per release cycle or on demand.

So GRUB should be built in the Dockerfile.

I would currently use GRUB 2.04 as reference release.

Best regards

Heinrich

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Heinrich Schuchardt]

On 7/25/19 9:57 PM, Tom Rini wrote:
> On Thu, Jul 25, 2019 at 09:43:10PM +0200, Heinrich Schuchardt wrote:
>> On 7/24/19 9:51 PM, Tom Rini wrote:
>>> In order to run our test.py suite on a number of platforms in CI we use
>>> QEMU.  Configure and build a copy here so that all our jobs can rely on
>>> it as our code is under test and not QEMU.
>>>
>>> Signed-off-by: Tom Rini <trini@konsulko.com>
>>> ---
>>>   Dockerfile | 8 ++++++++
>>>   1 file changed, 8 insertions(+)
>>>
>>> diff --git a/Dockerfile b/Dockerfile
>>> index b599af8af6a0..73c1b8b1e021 100644
>>> --- a/Dockerfile
>>> +++ b/Dockerfile
>>> @@ -76,6 +76,14 @@ RUN apt-get update && apt-get install -y \
>>>   	zip \
>>>   	&& rm -rf /var/lib/apt/lists/*
>>>
>>> +RUN git clone git://git.qemu.org/qemu.git /tmp/qemu && \
>>> +	cd /tmp/qemu && \
>>> +	git submodule update --init dtc && \
>>> +	git checkout 506179e42112be77bfd071f050b15762d3b2cd43 && \
>>> +	./configure --prefix=/opt/qemu --target-list="aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,ppc-softmmu,x86_64-softmmu,xtensa-softmmu" && \
>>> +	make -j$(nproc) all install && \
>>> +	rm -rf /tmp/qemu
>>> +
>>>   # Create the buildman config file
>>>   RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman
>>>   RUN /bin/echo -e "kernelorg = /opt/gcc-7.3.0-nolibc/*" >> ~/.buildman
>>>
>>
>> The file '/Dockerfile' does not exist in the current U-Boot
>> origin/master. It is hard to evaluate the patch without having the
>> complete file.
>>
>> Where would I find the file?
>
> Sorry, it's over at https://gitlab.denx.de/u-boot/gitlab-ci-runner
>
>> It might be useful to add it to tools/ in the U-Boot git.
>
> Currently I have:
> $ head -n4 .gitlab-ci.yml
> # SPDX-License-Identifier: GPL-2.0+
>
> # Grab our configured image.  The source for this is found at:
> # https://gitlab.denx.de/u-boot/gitlab-ci-runner
>
> But suggestions on where to note this are welcome.
>

To me it would make sense to move Dockerfile to U-Boot's tools/ directory.

I am missing RISC-V in the patch. Cf.

[PATCH 1/1] travis-ci: add qemu-riscv64
https://lists.denx.de/pipermail/u-boot/2019-July/375218.html

qemu-system-* is installed in ./opt/qemu/bin/ and not in /usr/local/bin.
Don't we want those binaries in the search path for binaries?

Best regards

Heinrich

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

On Thu, Jul 25, 2019 at 10:29:31PM +0200, Heinrich Schuchardt wrote:
> On 7/25/19 9:57 PM, Tom Rini wrote:
> >On Thu, Jul 25, 2019 at 09:43:10PM +0200, Heinrich Schuchardt wrote:
> >>On 7/24/19 9:51 PM, Tom Rini wrote:
> >>>In order to run our test.py suite on a number of platforms in CI we use
> >>>QEMU.  Configure and build a copy here so that all our jobs can rely on
> >>>it as our code is under test and not QEMU.
> >>>
> >>>Signed-off-by: Tom Rini <trini@konsulko.com>
> >>>---
> >>>  Dockerfile | 8 ++++++++
> >>>  1 file changed, 8 insertions(+)
> >>>
> >>>diff --git a/Dockerfile b/Dockerfile
> >>>index b599af8af6a0..73c1b8b1e021 100644
> >>>--- a/Dockerfile
> >>>+++ b/Dockerfile
> >>>@@ -76,6 +76,14 @@ RUN apt-get update && apt-get install -y \
> >>>  	zip \
> >>>  	&& rm -rf /var/lib/apt/lists/*
> >>>
> >>>+RUN git clone git://git.qemu.org/qemu.git /tmp/qemu && \
> >>>+	cd /tmp/qemu && \
> >>>+	git submodule update --init dtc && \
> >>>+	git checkout 506179e42112be77bfd071f050b15762d3b2cd43 && \
> >>>+	./configure --prefix=/opt/qemu --target-list="aarch64-softmmu,arm-softmmu,i386-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,ppc-softmmu,x86_64-softmmu,xtensa-softmmu" && \
> >>>+	make -j$(nproc) all install && \
> >>>+	rm -rf /tmp/qemu
> >>>+
> >>>  # Create the buildman config file
> >>>  RUN /bin/echo -e "[toolchain]\nroot = /usr" > ~/.buildman
> >>>  RUN /bin/echo -e "kernelorg = /opt/gcc-7.3.0-nolibc/*" >> ~/.buildman
> >>>
> >>
> >>The file '/Dockerfile' does not exist in the current U-Boot
> >>origin/master. It is hard to evaluate the patch without having the
> >>complete file.
> >>
> >>Where would I find the file?
> >
> >Sorry, it's over at https://gitlab.denx.de/u-boot/gitlab-ci-runner
> >
> >>It might be useful to add it to tools/ in the U-Boot git.
> >
> >Currently I have:
> >$ head -n4 .gitlab-ci.yml
> ># SPDX-License-Identifier: GPL-2.0+
> >
> ># Grab our configured image.  The source for this is found at:
> ># https://gitlab.denx.de/u-boot/gitlab-ci-runner
> >
> >But suggestions on where to note this are welcome.
> 
> To me it would make sense to move Dockerfile to U-Boot's tools/ directory.

I'm not sure.  Usual best practice for Dockerfiles is they live in their
own repository, no?

> I am missing RISC-V in the patch. Cf.
> 
> [PATCH 1/1] travis-ci: add qemu-riscv64
> https://lists.denx.de/pipermail/u-boot/2019-July/375218.html

Yes, I also haven't applied that yet nor done a follow-up to add it to
the .gitlab-ci.yml file.

> qemu-system-* is installed in ./opt/qemu/bin/ and not in /usr/local/bin.
> Don't we want those binaries in the search path for binaries?

Today we look in /tmp/qemu-install/bin for QEMU so moving it to /opt
made a certain amount of sense.

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190725/1342eac3/attachment.sig>

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

On Thu, Jul 25, 2019 at 10:14:04PM +0200, Heinrich Schuchardt wrote:
> On 7/25/19 9:17 PM, Tom Rini wrote:
> >On Wed, Jul 24, 2019 at 03:56:21PM -0400, Tom Rini wrote:
> >>On Wed, Jul 24, 2019 at 03:51:14PM -0400, Tom Rini wrote:
> >>
> >>>In order to run our test.py suite on a number of platforms in CI we use
> >>>QEMU.  Configure and build a copy here so that all our jobs can rely on
> >>>it as our code is under test and not QEMU.
> >>>
> >>>Signed-off-by: Tom Rini <trini@konsulko.com>
> >>
> >>I looked into using the artifact method in GitLab.  The problem there is
> >>that artifacts are stored within the job and kept.  So rather than a
> >>stand-alone project that we can reference and pass we would build it
> >>once per pipeline and pass it, but then store it until we purge things
> >>later when we notice gitlab.denx.de is out of space.  Going this route I
> >>think makes the most sense for what we do.  It does remove the ability
> >>to roll back a single platform (or platforms) to an older QEMU but I'm
> >>not sure that's a big deal and we COULD re-do that as needed.  Over at
> >>https://gitlab.denx.de/u-boot/u-boot/pipelines/324 all QEMU platforms
> >>pass again so we can use the latest version which allows evb-ast2500 to
> >>be used.
> >
> >Heinrich, what do you think about extending this for GRUB binaries for
> >test.py?
> >
> Hello Tom,
> 
> it saves a lot of CPU time if GRUB and QEMU are not built for every
> single push to the U-Boot git. It should be sufficient to build them
> once per release cycle or on demand.
> 
> So GRUB should be built in the Dockerfile.
> 
> I would currently use GRUB 2.04 as reference release.

Can you please do a follow-up patch to do so, for all platforms we need
a copy of GRUB for to test with?  Thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190725/841582cc/attachment.sig>

[U-Boot] [PATCH] Dockerfile: Build a copy of QEMU and install it under /opt

[Tom Rini]

On Wed, Jul 24, 2019 at 03:51:14PM -0400, Tom Rini wrote:

> In order to run our test.py suite on a number of platforms in CI we use
> QEMU.  Configure and build a copy here so that all our jobs can rely on
> it as our code is under test and not QEMU.
> 
> Signed-off-by: Tom Rini <trini@konsulko.com>

Applied to u-boot-gitlab-ci-runner/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190729/8a420bd4/attachment.sig>

[U-Boot] [PATCH] gitlab-ci: Switch to a Docker image that contains a QEMU we build

[Tom Rini]

On Wed, Jul 24, 2019 at 03:53:28PM -0400, Tom Rini wrote:

> Rather than buiding QEMU for each test.py build it once in our
> Dockerfile and re-use it as needed.
> 
> Signed-off-by: Tom Rini <trini@konsulko.com>

Applied to u-boot/master, thanks!

-- 
Tom
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://lists.denx.de/pipermail/u-boot/attachments/20190729/8cab5da1/attachment.sig>