All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/2] gitlab-ci: Add riscv64 cross builds
@ 2021-09-14 18:58 Richard Henderson
  2021-09-14 18:58 ` [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image Richard Henderson
                   ` (2 more replies)
  0 siblings, 3 replies; 13+ messages in thread
From: Richard Henderson @ 2021-09-14 18:58 UTC (permalink / raw)
  To: qemu-devel; +Cc: willianr, thuth, alex.bennee, wainersm

The only tcg host that does not have build coverage is riscv64.
Filling this hole will help with tcg reorgs I have in the works.

Thanks to Alex for help debugging the docker image creation.


r~


Alex Bennée (1):
  tests/docker: promote debian-riscv64-cross to a full image

Richard Henderson (1):
  gitlab: Add cross-riscv64-system, cross-riscv64-user

 .gitlab-ci.d/container-cross.yml              |  1 -
 .gitlab-ci.d/crossbuilds.yml                  | 14 ++++++
 tests/docker/Makefile.include                 |  2 -
 .../dockerfiles/debian-riscv64-cross.docker   | 46 +++++++++++++++++--
 4 files changed, 55 insertions(+), 8 deletions(-)

-- 
2.25.1



^ permalink raw reply	[flat|nested] 13+ messages in thread

* [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image
  2021-09-14 18:58 [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Richard Henderson
@ 2021-09-14 18:58 ` Richard Henderson
  2021-09-15  7:35   ` Philippe Mathieu-Daudé
  2021-09-14 18:58 ` [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user Richard Henderson
  2021-09-15  6:15 ` [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Alex Bennée
  2 siblings, 1 reply; 13+ messages in thread
From: Richard Henderson @ 2021-09-14 18:58 UTC (permalink / raw)
  To: qemu-devel
  Cc: willianr, Daniel P. Berrangé, thuth, alex.bennee, wainersm

From: Alex Bennée <alex.bennee@linaro.org>

To be able to cross build QEMU itself we need to include a few more
libraries. These are only available in Debian's unstable ports repo
for now so we need to base the riscv64 image on sid with the the
minimal libs needed to build QEMU (glib/pixman).

The result works but is not as clean as using build-dep to bring in
more dependencies. However sid is by definition a shifting pile of
sand and by keeping the list of libs minimal we reduce the chance of
having an image we can't build. It's good enough for a basic cross
build testing of TCG.

Cc: "Daniel P. Berrangé" <berrange@redhat.com>
Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 .gitlab-ci.d/container-cross.yml              |  1 -
 tests/docker/Makefile.include                 |  2 -
 .../dockerfiles/debian-riscv64-cross.docker   | 46 +++++++++++++++++--
 3 files changed, 41 insertions(+), 8 deletions(-)

diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
index 0fcebe363a..05996200e1 100644
--- a/.gitlab-ci.d/container-cross.yml
+++ b/.gitlab-ci.d/container-cross.yml
@@ -134,7 +134,6 @@ ppc64el-debian-cross-container:
 riscv64-debian-cross-container:
   extends: .container_job_template
   stage: containers-layer2
-  needs: ['amd64-debian10-container']
   variables:
     NAME: debian-riscv64-cross
 
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index ff5d732889..3b03763186 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -141,7 +141,6 @@ docker-image-debian-mips64-cross: docker-image-debian10
 docker-image-debian-mips64el-cross: docker-image-debian10
 docker-image-debian-mipsel-cross: docker-image-debian10
 docker-image-debian-ppc64el-cross: docker-image-debian10
-docker-image-debian-riscv64-cross: docker-image-debian10
 docker-image-debian-s390x-cross: docker-image-debian10
 docker-image-debian-sh4-cross: docker-image-debian10
 docker-image-debian-sparc64-cross: docker-image-debian10
@@ -180,7 +179,6 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
 DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
 DOCKER_PARTIAL_IMAGES += debian-hppa-cross
 DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
-DOCKER_PARTIAL_IMAGES += debian-riscv64-cross
 DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
 DOCKER_PARTIAL_IMAGES += debian-tricore-cross
 DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
index 2bbff19772..594d97982c 100644
--- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
+++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
@@ -1,12 +1,48 @@
 #
-# Docker cross-compiler target
+# Docker cross-compiler target for riscv64
 #
-# This docker target builds on the debian Buster base image.
+# Currently the only distro that gets close to cross compiling riscv64
+# images is Debian Sid (with unofficial ports). As this is a moving
+# target we keep the library list minimal and are aiming to migrate
+# from this hack as soon as we are able.
 #
-FROM qemu/debian10
+FROM docker.io/library/debian:sid-slim
+
+# Add ports
+RUN apt update && \
+    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
+    DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \
+    DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy
+
+# Install common build utilities
+RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
+    bc \
+    build-essential \
+    ca-certificates \
+    debian-ports-archive-keyring \
+    dpkg-dev \
+    gettext \
+    git \
+    ninja-build \
+    pkg-config \
+    python3
+
+# Add ports and riscv64 architecture
+RUN echo "deb http://ftp.ports.debian.org/debian-ports/ sid main" >> /etc/apt/sources.list
+RUN dpkg --add-architecture riscv64
+
+# Duplicate deb line as deb-src
+RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
 
 RUN apt update && \
     DEBIAN_FRONTEND=noninteractive eatmydata \
     apt install -y --no-install-recommends \
-        gcc-riscv64-linux-gnu \
-        libc6-dev-riscv64-cross
+         gcc-riscv64-linux-gnu \
+         libc6-dev-riscv64-cross \
+         libffi-dev:riscv64 \
+         libglib2.0-dev:riscv64 \
+         libpixman-1-dev:riscv64
+
+# Specify the cross prefix for this image (see tests/docker/common.rc)
+ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu-
+ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user
  2021-09-14 18:58 [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Richard Henderson
  2021-09-14 18:58 ` [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image Richard Henderson
@ 2021-09-14 18:58 ` Richard Henderson
  2021-09-15  7:36   ` Philippe Mathieu-Daudé
                     ` (2 more replies)
  2021-09-15  6:15 ` [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Alex Bennée
  2 siblings, 3 replies; 13+ messages in thread
From: Richard Henderson @ 2021-09-14 18:58 UTC (permalink / raw)
  To: qemu-devel; +Cc: willianr, thuth, alex.bennee, wainersm

Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
 .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
index f10168db2e..0fe4a55ac5 100644
--- a/.gitlab-ci.d/crossbuilds.yml
+++ b/.gitlab-ci.d/crossbuilds.yml
@@ -124,6 +124,20 @@ cross-ppc64el-user:
   variables:
     IMAGE: debian-ppc64el-cross
 
+cross-riscv64-system:
+  extends: .cross_system_build_job
+  needs:
+    job: riscv64-debian-cross-container
+  variables:
+    IMAGE: debian-riscv64-cross
+
+cross-riscv64-user:
+  extends: .cross_user_build_job
+  needs:
+    job: riscv64-debian-cross-container
+  variables:
+    IMAGE: debian-riscv64-cross
+
 cross-s390x-system:
   extends: .cross_system_build_job
   needs:
-- 
2.25.1



^ permalink raw reply related	[flat|nested] 13+ messages in thread

* Re: [PATCH 0/2] gitlab-ci: Add riscv64 cross builds
  2021-09-14 18:58 [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Richard Henderson
  2021-09-14 18:58 ` [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image Richard Henderson
  2021-09-14 18:58 ` [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user Richard Henderson
@ 2021-09-15  6:15 ` Alex Bennée
  2 siblings, 0 replies; 13+ messages in thread
From: Alex Bennée @ 2021-09-15  6:15 UTC (permalink / raw)
  To: Richard Henderson; +Cc: willianr, thuth, qemu-devel, wainersm


Richard Henderson <richard.henderson@linaro.org> writes:

> The only tcg host that does not have build coverage is riscv64.
> Filling this hole will help with tcg reorgs I have in the works.
>
> Thanks to Alex for help debugging the docker image creation.

Queued to testing/next, thanks.

-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image
  2021-09-14 18:58 ` [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image Richard Henderson
@ 2021-09-15  7:35   ` Philippe Mathieu-Daudé
  2021-09-15  8:15     ` Alex Bennée
  0 siblings, 1 reply; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-09-15  7:35 UTC (permalink / raw)
  To: qemu-devel, alex.bennee
  Cc: willianr, Daniel P. Berrangé, thuth, Richard Henderson, wainersm

On 9/14/21 8:58 PM, Richard Henderson wrote:
> From: Alex Bennée <alex.bennee@linaro.org>
> 
> To be able to cross build QEMU itself we need to include a few more
> libraries. These are only available in Debian's unstable ports repo
> for now so we need to base the riscv64 image on sid with the the
> minimal libs needed to build QEMU (glib/pixman).
> 
> The result works but is not as clean as using build-dep to bring in
> more dependencies. However sid is by definition a shifting pile of
> sand and by keeping the list of libs minimal we reduce the chance of
> having an image we can't build. It's good enough for a basic cross
> build testing of TCG.
> 
> Cc: "Daniel P. Berrangé" <berrange@redhat.com>
> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
>  .gitlab-ci.d/container-cross.yml              |  1 -
>  tests/docker/Makefile.include                 |  2 -
>  .../dockerfiles/debian-riscv64-cross.docker   | 46 +++++++++++++++++--
>  3 files changed, 41 insertions(+), 8 deletions(-)
> 
> diff --git a/.gitlab-ci.d/container-cross.yml b/.gitlab-ci.d/container-cross.yml
> index 0fcebe363a..05996200e1 100644
> --- a/.gitlab-ci.d/container-cross.yml
> +++ b/.gitlab-ci.d/container-cross.yml
> @@ -134,7 +134,6 @@ ppc64el-debian-cross-container:
>  riscv64-debian-cross-container:
>    extends: .container_job_template
>    stage: containers-layer2
> -  needs: ['amd64-debian10-container']
>    variables:
>      NAME: debian-riscv64-cross
>  
> diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
> index ff5d732889..3b03763186 100644
> --- a/tests/docker/Makefile.include
> +++ b/tests/docker/Makefile.include
> @@ -141,7 +141,6 @@ docker-image-debian-mips64-cross: docker-image-debian10
>  docker-image-debian-mips64el-cross: docker-image-debian10
>  docker-image-debian-mipsel-cross: docker-image-debian10
>  docker-image-debian-ppc64el-cross: docker-image-debian10
> -docker-image-debian-riscv64-cross: docker-image-debian10
>  docker-image-debian-s390x-cross: docker-image-debian10
>  docker-image-debian-sh4-cross: docker-image-debian10
>  docker-image-debian-sparc64-cross: docker-image-debian10
> @@ -180,7 +179,6 @@ DOCKER_PARTIAL_IMAGES += debian-arm64-test-cross
>  DOCKER_PARTIAL_IMAGES += debian-powerpc-test-cross
>  DOCKER_PARTIAL_IMAGES += debian-hppa-cross
>  DOCKER_PARTIAL_IMAGES += debian-m68k-cross debian-mips64-cross
> -DOCKER_PARTIAL_IMAGES += debian-riscv64-cross
>  DOCKER_PARTIAL_IMAGES += debian-sh4-cross debian-sparc64-cross
>  DOCKER_PARTIAL_IMAGES += debian-tricore-cross
>  DOCKER_PARTIAL_IMAGES += debian-xtensa-cross
> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
> index 2bbff19772..594d97982c 100644
> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
> @@ -1,12 +1,48 @@
>  #
> -# Docker cross-compiler target
> +# Docker cross-compiler target for riscv64
>  #
> -# This docker target builds on the debian Buster base image.
> +# Currently the only distro that gets close to cross compiling riscv64
> +# images is Debian Sid (with unofficial ports). As this is a moving
> +# target we keep the library list minimal and are aiming to migrate
> +# from this hack as soon as we are able.
>  #
> -FROM qemu/debian10
> +FROM docker.io/library/debian:sid-slim
> +
> +# Add ports
> +RUN apt update && \
> +    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
> +    DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \
> +    DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy
> +
> +# Install common build utilities
> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
> +    bc \
> +    build-essential \
> +    ca-certificates \
> +    debian-ports-archive-keyring \
> +    dpkg-dev \
> +    gettext \
> +    git \
> +    ninja-build \
> +    pkg-config \
> +    python3

Alex, why not using a debian-sid intermediate image?

> +# Add ports and riscv64 architecture
> +RUN echo "deb http://ftp.ports.debian.org/debian-ports/ sid main" >> /etc/apt/sources.list
> +RUN dpkg --add-architecture riscv64
> +
> +# Duplicate deb line as deb-src
> +RUN cat /etc/apt/sources.list | sed "s/^deb\ /deb-src /" >> /etc/apt/sources.list
>  
>  RUN apt update && \
>      DEBIAN_FRONTEND=noninteractive eatmydata \
>      apt install -y --no-install-recommends \
> -        gcc-riscv64-linux-gnu \
> -        libc6-dev-riscv64-cross
> +         gcc-riscv64-linux-gnu \
> +         libc6-dev-riscv64-cross \
> +         libffi-dev:riscv64 \
> +         libglib2.0-dev:riscv64 \
> +         libpixman-1-dev:riscv64
> +
> +# Specify the cross prefix for this image (see tests/docker/common.rc)
> +ENV QEMU_CONFIGURE_OPTS --cross-prefix=riscv64-linux-gnu-
> +ENV DEF_TARGET_LIST riscv64-softmmu,riscv64-linux-user
> 



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user
  2021-09-14 18:58 ` [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user Richard Henderson
@ 2021-09-15  7:36   ` Philippe Mathieu-Daudé
  2021-09-15 14:31   ` Willian Rampazzo
  2021-09-15 15:03   ` Philippe Mathieu-Daudé
  2 siblings, 0 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-09-15  7:36 UTC (permalink / raw)
  To: Richard Henderson, qemu-devel; +Cc: willianr, thuth, alex.bennee, wainersm

On 9/14/21 8:58 PM, Richard Henderson wrote:
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
>  .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)

Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image
  2021-09-15  7:35   ` Philippe Mathieu-Daudé
@ 2021-09-15  8:15     ` Alex Bennée
  2021-09-15 15:01       ` Philippe Mathieu-Daudé
  0 siblings, 1 reply; 13+ messages in thread
From: Alex Bennée @ 2021-09-15  8:15 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé
  Cc: thuth, Daniel P. Berrangé,
	Richard Henderson, qemu-devel, wainersm, willianr


Philippe Mathieu-Daudé <f4bug@amsat.org> writes:

> On 9/14/21 8:58 PM, Richard Henderson wrote:
>> From: Alex Bennée <alex.bennee@linaro.org>
>> 
>> To be able to cross build QEMU itself we need to include a few more
>> libraries. These are only available in Debian's unstable ports repo
>> for now so we need to base the riscv64 image on sid with the the
>> minimal libs needed to build QEMU (glib/pixman).
>> 
>> The result works but is not as clean as using build-dep to bring in
>> more dependencies. However sid is by definition a shifting pile of
>> sand and by keeping the list of libs minimal we reduce the chance of
>> having an image we can't build. It's good enough for a basic cross
>> build testing of TCG.
>> 
<snip>
>> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
>> index 2bbff19772..594d97982c 100644
>> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
>> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
>> @@ -1,12 +1,48 @@
>>  #
>> -# Docker cross-compiler target
>> +# Docker cross-compiler target for riscv64
>>  #
>> -# This docker target builds on the debian Buster base image.
>> +# Currently the only distro that gets close to cross compiling riscv64
>> +# images is Debian Sid (with unofficial ports). As this is a moving
>> +# target we keep the library list minimal and are aiming to migrate
>> +# from this hack as soon as we are able.
>>  #
>> -FROM qemu/debian10
>> +FROM docker.io/library/debian:sid-slim
>> +
>> +# Add ports
>> +RUN apt update && \
>> +    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
>> +    DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \
>> +    DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy
>> +
>> +# Install common build utilities
>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
>> +    bc \
>> +    build-essential \
>> +    ca-certificates \
>> +    debian-ports-archive-keyring \
>> +    dpkg-dev \
>> +    gettext \
>> +    git \
>> +    ninja-build \
>> +    pkg-config \
>> +    python3
>
> Alex, why not using a debian-sid intermediate image?

Well we removed the original debian-sid image a while ago having excised
images that used it. Basing things on sid does cause problems given it's
such a moving feast. Rather than encouraging others to build sid based
images we decided it made more sense to make this a pure standalone
image which we can migrate away from at the first opportunity.

-- 
Alex Bennée


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user
  2021-09-14 18:58 ` [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user Richard Henderson
  2021-09-15  7:36   ` Philippe Mathieu-Daudé
@ 2021-09-15 14:31   ` Willian Rampazzo
  2021-09-15 15:03   ` Philippe Mathieu-Daudé
  2 siblings, 0 replies; 13+ messages in thread
From: Willian Rampazzo @ 2021-09-15 14:31 UTC (permalink / raw)
  To: Richard Henderson
  Cc: Thomas Huth, Alex Bennée, qemu-devel, Wainer Moschetta

On Tue, Sep 14, 2021 at 3:58 PM Richard Henderson
<richard.henderson@linaro.org> wrote:
>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
>  .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
>

Reviewed-by: Willian Rampazzo <willianr@redhat.com>



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image
  2021-09-15  8:15     ` Alex Bennée
@ 2021-09-15 15:01       ` Philippe Mathieu-Daudé
  2021-09-15 15:03         ` Daniel P. Berrangé
  0 siblings, 1 reply; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-09-15 15:01 UTC (permalink / raw)
  To: Alex Bennée
  Cc: thuth, Daniel P. Berrangé,
	Richard Henderson, qemu-devel, wainersm, willianr

On 9/15/21 10:15 AM, Alex Bennée wrote:
> Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
>> On 9/14/21 8:58 PM, Richard Henderson wrote:
>>> From: Alex Bennée <alex.bennee@linaro.org>
>>>
>>> To be able to cross build QEMU itself we need to include a few more
>>> libraries. These are only available in Debian's unstable ports repo
>>> for now so we need to base the riscv64 image on sid with the the
>>> minimal libs needed to build QEMU (glib/pixman).
>>>
>>> The result works but is not as clean as using build-dep to bring in
>>> more dependencies. However sid is by definition a shifting pile of
>>> sand and by keeping the list of libs minimal we reduce the chance of
>>> having an image we can't build. It's good enough for a basic cross
>>> build testing of TCG.
>>>
> <snip>
>>> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
>>> index 2bbff19772..594d97982c 100644
>>> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
>>> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
>>> @@ -1,12 +1,48 @@
>>>  #
>>> -# Docker cross-compiler target
>>> +# Docker cross-compiler target for riscv64
>>>  #
>>> -# This docker target builds on the debian Buster base image.
>>> +# Currently the only distro that gets close to cross compiling riscv64
>>> +# images is Debian Sid (with unofficial ports). As this is a moving
>>> +# target we keep the library list minimal and are aiming to migrate
>>> +# from this hack as soon as we are able.
>>>  #
>>> -FROM qemu/debian10
>>> +FROM docker.io/library/debian:sid-slim
>>> +
>>> +# Add ports
>>> +RUN apt update && \
>>> +    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
>>> +    DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \
>>> +    DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy
>>> +
>>> +# Install common build utilities
>>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
>>> +    bc \
>>> +    build-essential \
>>> +    ca-certificates \
>>> +    debian-ports-archive-keyring \
>>> +    dpkg-dev \
>>> +    gettext \
>>> +    git \
>>> +    ninja-build \
>>> +    pkg-config \
>>> +    python3
>>
>> Alex, why not using a debian-sid intermediate image?
> 
> Well we removed the original debian-sid image a while ago having excised
> images that used it. Basing things on sid does cause problems given it's
> such a moving feast. Rather than encouraging others to build sid based
> images we decided it made more sense to make this a pure standalone
> image which we can migrate away from at the first opportunity.

Yeah, which is why I'm worried we'll get the same problem...

So this image can't be used as gating IMO.


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user
  2021-09-14 18:58 ` [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user Richard Henderson
  2021-09-15  7:36   ` Philippe Mathieu-Daudé
  2021-09-15 14:31   ` Willian Rampazzo
@ 2021-09-15 15:03   ` Philippe Mathieu-Daudé
  2021-09-15 16:32     ` Richard Henderson
  2 siblings, 1 reply; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-09-15 15:03 UTC (permalink / raw)
  To: Richard Henderson, qemu-devel; +Cc: willianr, thuth, alex.bennee, wainersm

On 9/14/21 8:58 PM, Richard Henderson wrote:
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
>  .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++
>  1 file changed, 14 insertions(+)
> 
> diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
> index f10168db2e..0fe4a55ac5 100644
> --- a/.gitlab-ci.d/crossbuilds.yml
> +++ b/.gitlab-ci.d/crossbuilds.yml
> @@ -124,6 +124,20 @@ cross-ppc64el-user:
>    variables:
>      IMAGE: debian-ppc64el-cross
>  
> +cross-riscv64-system:
> +  extends: .cross_system_build_job
> +  needs:
> +    job: riscv64-debian-cross-container
> +  variables:
> +    IMAGE: debian-riscv64-cross
> +
> +cross-riscv64-user:
> +  extends: .cross_user_build_job
> +  needs:
> +    job: riscv64-debian-cross-container
> +  variables:
> +    IMAGE: debian-riscv64-cross
> +

Pending discussion on patch #1 of this series, I believe
this job is likely going to fail, so must use the
'allow_failure: true' tag or something else (manual?).
Meanwhile this is sorted out:
NAcked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image
  2021-09-15 15:01       ` Philippe Mathieu-Daudé
@ 2021-09-15 15:03         ` Daniel P. Berrangé
  0 siblings, 0 replies; 13+ messages in thread
From: Daniel P. Berrangé @ 2021-09-15 15:03 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé
  Cc: thuth, Richard Henderson, qemu-devel, wainersm, willianr,
	Alex Bennée

On Wed, Sep 15, 2021 at 05:01:23PM +0200, Philippe Mathieu-Daudé wrote:
> On 9/15/21 10:15 AM, Alex Bennée wrote:
> > Philippe Mathieu-Daudé <f4bug@amsat.org> writes:
> >> On 9/14/21 8:58 PM, Richard Henderson wrote:
> >>> From: Alex Bennée <alex.bennee@linaro.org>
> >>>
> >>> To be able to cross build QEMU itself we need to include a few more
> >>> libraries. These are only available in Debian's unstable ports repo
> >>> for now so we need to base the riscv64 image on sid with the the
> >>> minimal libs needed to build QEMU (glib/pixman).
> >>>
> >>> The result works but is not as clean as using build-dep to bring in
> >>> more dependencies. However sid is by definition a shifting pile of
> >>> sand and by keeping the list of libs minimal we reduce the chance of
> >>> having an image we can't build. It's good enough for a basic cross
> >>> build testing of TCG.
> >>>
> > <snip>
> >>> diff --git a/tests/docker/dockerfiles/debian-riscv64-cross.docker b/tests/docker/dockerfiles/debian-riscv64-cross.docker
> >>> index 2bbff19772..594d97982c 100644
> >>> --- a/tests/docker/dockerfiles/debian-riscv64-cross.docker
> >>> +++ b/tests/docker/dockerfiles/debian-riscv64-cross.docker
> >>> @@ -1,12 +1,48 @@
> >>>  #
> >>> -# Docker cross-compiler target
> >>> +# Docker cross-compiler target for riscv64
> >>>  #
> >>> -# This docker target builds on the debian Buster base image.
> >>> +# Currently the only distro that gets close to cross compiling riscv64
> >>> +# images is Debian Sid (with unofficial ports). As this is a moving
> >>> +# target we keep the library list minimal and are aiming to migrate
> >>> +# from this hack as soon as we are able.
> >>>  #
> >>> -FROM qemu/debian10
> >>> +FROM docker.io/library/debian:sid-slim
> >>> +
> >>> +# Add ports
> >>> +RUN apt update && \
> >>> +    DEBIAN_FRONTEND=noninteractive apt install -yy eatmydata && \
> >>> +    DEBIAN_FRONTEND=noninteractive eatmydata apt update -yy && \
> >>> +    DEBIAN_FRONTEND=noninteractive eatmydata apt upgrade -yy
> >>> +
> >>> +# Install common build utilities
> >>> +RUN DEBIAN_FRONTEND=noninteractive eatmydata apt install -yy \
> >>> +    bc \
> >>> +    build-essential \
> >>> +    ca-certificates \
> >>> +    debian-ports-archive-keyring \
> >>> +    dpkg-dev \
> >>> +    gettext \
> >>> +    git \
> >>> +    ninja-build \
> >>> +    pkg-config \
> >>> +    python3
> >>
> >> Alex, why not using a debian-sid intermediate image?
> > 
> > Well we removed the original debian-sid image a while ago having excised
> > images that used it. Basing things on sid does cause problems given it's
> > such a moving feast. Rather than encouraging others to build sid based
> > images we decided it made more sense to make this a pure standalone
> > image which we can migrate away from at the first opportunity.
> 
> Yeah, which is why I'm worried we'll get the same problem...
> 
> So this image can't be used as gating IMO.

Yeah, IME from babysitting libvirt CI, 'sid' containers break in dep
solving more often than is acceptable for a mandatory job, especially
for non-x86 arches.

Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|



^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user
  2021-09-15 15:03   ` Philippe Mathieu-Daudé
@ 2021-09-15 16:32     ` Richard Henderson
  2021-09-17 18:18       ` Philippe Mathieu-Daudé
  0 siblings, 1 reply; 13+ messages in thread
From: Richard Henderson @ 2021-09-15 16:32 UTC (permalink / raw)
  To: Philippe Mathieu-Daudé, qemu-devel
  Cc: willianr, thuth, alex.bennee, wainersm

On 9/15/21 8:03 AM, Philippe Mathieu-Daudé wrote:
> On 9/14/21 8:58 PM, Richard Henderson wrote:
>> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>> ---
>>   .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++
>>   1 file changed, 14 insertions(+)
>>
>> diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
>> index f10168db2e..0fe4a55ac5 100644
>> --- a/.gitlab-ci.d/crossbuilds.yml
>> +++ b/.gitlab-ci.d/crossbuilds.yml
>> @@ -124,6 +124,20 @@ cross-ppc64el-user:
>>     variables:
>>       IMAGE: debian-ppc64el-cross
>>   
>> +cross-riscv64-system:
>> +  extends: .cross_system_build_job
>> +  needs:
>> +    job: riscv64-debian-cross-container
>> +  variables:
>> +    IMAGE: debian-riscv64-cross
>> +
>> +cross-riscv64-user:
>> +  extends: .cross_user_build_job
>> +  needs:
>> +    job: riscv64-debian-cross-container
>> +  variables:
>> +    IMAGE: debian-riscv64-cross
>> +
> 
> Pending discussion on patch #1 of this series, I believe
> this job is likely going to fail, so must use the
> 'allow_failure: true' tag or something else (manual?).
> Meanwhile this is sorted out:
> NAcked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>

My only question is whether it's the image creation step that should be allowed to fail. 
If the dpkg deps weren't so broken as to not install, I *think* we should be able to rely 
on the result for the cross-build.

At least that's how I'd prefer to arrange things, if it works...

r~


^ permalink raw reply	[flat|nested] 13+ messages in thread

* Re: [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user
  2021-09-15 16:32     ` Richard Henderson
@ 2021-09-17 18:18       ` Philippe Mathieu-Daudé
  0 siblings, 0 replies; 13+ messages in thread
From: Philippe Mathieu-Daudé @ 2021-09-17 18:18 UTC (permalink / raw)
  To: Richard Henderson, qemu-devel; +Cc: willianr, thuth, alex.bennee, wainersm

On 9/15/21 6:32 PM, Richard Henderson wrote:
> On 9/15/21 8:03 AM, Philippe Mathieu-Daudé wrote:
>> On 9/14/21 8:58 PM, Richard Henderson wrote:
>>> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>>> ---
>>>   .gitlab-ci.d/crossbuilds.yml | 14 ++++++++++++++
>>>   1 file changed, 14 insertions(+)
>>>
>>> diff --git a/.gitlab-ci.d/crossbuilds.yml b/.gitlab-ci.d/crossbuilds.yml
>>> index f10168db2e..0fe4a55ac5 100644
>>> --- a/.gitlab-ci.d/crossbuilds.yml
>>> +++ b/.gitlab-ci.d/crossbuilds.yml
>>> @@ -124,6 +124,20 @@ cross-ppc64el-user:
>>>     variables:
>>>       IMAGE: debian-ppc64el-cross
>>>   +cross-riscv64-system:
>>> +  extends: .cross_system_build_job
>>> +  needs:
>>> +    job: riscv64-debian-cross-container
>>> +  variables:
>>> +    IMAGE: debian-riscv64-cross
>>> +
>>> +cross-riscv64-user:
>>> +  extends: .cross_user_build_job
>>> +  needs:
>>> +    job: riscv64-debian-cross-container
>>> +  variables:
>>> +    IMAGE: debian-riscv64-cross
>>> +
>>
>> Pending discussion on patch #1 of this series, I believe
>> this job is likely going to fail, so must use the
>> 'allow_failure: true' tag or something else (manual?).
>> Meanwhile this is sorted out:
>> NAcked-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
> 
> My only question is whether it's the image creation step that should be
> allowed to fail. If the dpkg deps weren't so broken as to not install, I
> *think* we should be able to rely on the result for the cross-build.

We have multiple possibilities:

(1) Have all jobs depending on riscv64-debian-cross-container
    use 'allow_failure: true' (see commit d3a4e41da25 "gitlab-ci:
    Fix 'when:' condition in acceptance...")

(2) Use a working snapshot date, see commit b4048a7cd10 ("docker:
    Use a stable snapshot for Debian Sid")

(3) Manually push a built image to mainstream (qemu-project namespace)
    and have forks (try to build) or pull from mainstream

Hmm I had a (4) but forgot about it... Anyhow, I personally prefer (2)
over (1) because we can use it as gating. 'allow_failure' jobs are
*not* gating and we often miss their failures.


^ permalink raw reply	[flat|nested] 13+ messages in thread

end of thread, other threads:[~2021-09-17 18:19 UTC | newest]

Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-14 18:58 [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Richard Henderson
2021-09-14 18:58 ` [PATCH 1/2] tests/docker: promote debian-riscv64-cross to a full image Richard Henderson
2021-09-15  7:35   ` Philippe Mathieu-Daudé
2021-09-15  8:15     ` Alex Bennée
2021-09-15 15:01       ` Philippe Mathieu-Daudé
2021-09-15 15:03         ` Daniel P. Berrangé
2021-09-14 18:58 ` [PATCH 2/2] gitlab: Add cross-riscv64-system, cross-riscv64-user Richard Henderson
2021-09-15  7:36   ` Philippe Mathieu-Daudé
2021-09-15 14:31   ` Willian Rampazzo
2021-09-15 15:03   ` Philippe Mathieu-Daudé
2021-09-15 16:32     ` Richard Henderson
2021-09-17 18:18       ` Philippe Mathieu-Daudé
2021-09-15  6:15 ` [PATCH 0/2] gitlab-ci: Add riscv64 cross builds Alex Bennée

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.