All of lore.kernel.org
 help / color / mirror / Atom feed
From: Bertrand Marquis <bertrand.marquis@arm.com>
To: xen-devel@lists.xenproject.org
Cc: Doug Goldstein <cardoe@cardoe.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: [PATCH v2 2/3] automation: Add yocto containers with cache
Date: Wed, 12 Oct 2022 09:02:24 +0100	[thread overview]
Message-ID: <3003e61047b648a703768d3ab1e2b251190ee591.1665561024.git.bertrand.marquis@arm.com> (raw)
In-Reply-To: <cover.1665561024.git.bertrand.marquis@arm.com>

Create yocto containers for kirkstone with a builder step to fill the
yocto cache by running the build script once.
Those containers depend on the kirkstone.dockerfile container to be
created and use it as base image.

To run a build test on one image the following command must be called
inside the container:
build-yocto.sh --xen-dir=XENSRC TARGET where
- XENSRC is the directory where xen sources are available in the
container
- TARGET is the build target (qemuarm64, qemuarm or qemux86_64)

In case of error, the script will return an error code and the logs will
be in /home/docker-build/logs.

!!!
The build of each container can take several hours depending on the
network bandwith (for downloading all sources) and the amount of
computing power on the machine where the docker images are created.
!!!

Signed-off-by: Bertrand Marquis <bertrand.marquis@arm.com>
---
Changes in v2:
- pass --dump-log option to build-yocto.sh during the image generation
so that we have more info when there is an error during the image
creation.
Changes in v1:
- fix dependency in Makefile
---
 automation/build/Makefile                     |  2 ++
 .../build/yocto/kirkstone-qemuarm.dockerfile  | 28 +++++++++++++++++++
 .../yocto/kirkstone-qemuarm64.dockerfile      | 28 +++++++++++++++++++
 .../yocto/kirkstone-qemux86-64.dockerfile     | 28 +++++++++++++++++++
 4 files changed, 86 insertions(+)
 create mode 100644 automation/build/yocto/kirkstone-qemuarm.dockerfile
 create mode 100644 automation/build/yocto/kirkstone-qemuarm64.dockerfile
 create mode 100644 automation/build/yocto/kirkstone-qemux86-64.dockerfile

diff --git a/automation/build/Makefile b/automation/build/Makefile
index a4b2b85178cf..b0dc68f00b70 100644
--- a/automation/build/Makefile
+++ b/automation/build/Makefile
@@ -16,5 +16,7 @@ help:
 		$(DOCKER_CMD) push $(REGISTRY)/$(@D):$(@F); \
 	fi
 
+$(subst .dockerfile,,$(wildcard yocto/kirkstone-*)): yocto/kirkstone
+
 .PHONY: all
 all: $(CONTAINERS)
diff --git a/automation/build/yocto/kirkstone-qemuarm.dockerfile b/automation/build/yocto/kirkstone-qemuarm.dockerfile
new file mode 100644
index 000000000000..b6bd8ceee9a7
--- /dev/null
+++ b/automation/build/yocto/kirkstone-qemuarm.dockerfile
@@ -0,0 +1,28 @@
+# Docker file to populate the initial sstate and download cache for yocto
+# for a specific target
+#
+# This requires the build-yocto.sh script to be in the same directory as this
+# dockerfile during the image creation
+#
+# Arguments that can be passed during image creation using --build-arg:
+# "from_image=img" to select the docker image and tag to start from
+# "target=qemuxxx": to select which target to build in this image
+
+ARG from_image=registry.gitlab.com/xen-project/xen/yocto:kirkstone
+
+FROM $from_image AS builder
+
+ARG target=qemuarm
+
+# This step can take one to several hours depending on your download bandwith
+# and the speed of your computer
+RUN /home/$USER_NAME/bin/build-yocto.sh --dump-log $target
+
+FROM $from_image
+
+# Only copy the cache status
+COPY --from=builder /home/$USER_NAME/yocto-cache /home/$USER_NAME/yocto-cache/.
+
+LABEL maintainer.name="The Xen Project " \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
diff --git a/automation/build/yocto/kirkstone-qemuarm64.dockerfile b/automation/build/yocto/kirkstone-qemuarm64.dockerfile
new file mode 100644
index 000000000000..f279a7af928c
--- /dev/null
+++ b/automation/build/yocto/kirkstone-qemuarm64.dockerfile
@@ -0,0 +1,28 @@
+# Docker file to populate the initial sstate and download cache for yocto
+# for a specific target
+#
+# This requires the build-yocto.sh script to be in the same directory as this
+# dockerfile during the image creation
+#
+# Arguments that can be passed during image creation using --build-arg:
+# "from_image=img" to select the docker image and tag to start from
+# "target=qemuxxx": to select which target to build in this image
+
+ARG from_image=registry.gitlab.com/xen-project/xen/yocto:kirkstone
+
+FROM $from_image AS builder
+
+ARG target=qemuarm64
+
+# This step can take one to several hours depending on your download bandwith
+# and the speed of your computer
+RUN /home/$USER_NAME/bin/build-yocto.sh --dump-log $target
+
+FROM $from_image
+
+# Only copy the cache status
+COPY --from=builder /home/$USER_NAME/yocto-cache /home/$USER_NAME/yocto-cache/.
+
+LABEL maintainer.name="The Xen Project " \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
diff --git a/automation/build/yocto/kirkstone-qemux86-64.dockerfile b/automation/build/yocto/kirkstone-qemux86-64.dockerfile
new file mode 100644
index 000000000000..595643df5905
--- /dev/null
+++ b/automation/build/yocto/kirkstone-qemux86-64.dockerfile
@@ -0,0 +1,28 @@
+# Docker file to populate the initial sstate and download cache for yocto
+# for a specific target
+#
+# This requires the build-yocto.sh script to be in the same directory as this
+# dockerfile during the image creation
+#
+# Arguments that can be passed during image creation using --build-arg:
+# "from_image=img" to select the docker image and tag to start from
+# "target=qemuxxx": to select which target to build in this image
+
+ARG from_image=registry.gitlab.com/xen-project/xen/yocto:kirkstone
+
+FROM $from_image AS builder
+
+ARG target=qemux86-64
+
+# This step can take one to several hours depending on your download bandwith
+# and the speed of your computer
+RUN /home/$USER_NAME/bin/build-yocto.sh --dump-log $target
+
+FROM $from_image
+
+# Only copy the cache status
+COPY --from=builder /home/$USER_NAME/yocto-cache /home/$USER_NAME/yocto-cache/.
+
+LABEL maintainer.name="The Xen Project " \
+      maintainer.email="xen-devel@lists.xenproject.org"
+
-- 
2.25.1



  parent reply	other threads:[~2022-10-12  8:03 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2022-10-12  8:02 [PATCH v2 0/3] Yocto Gitlab CI Bertrand Marquis
2022-10-12  8:02 ` [PATCH v2 1/3] automation: Add elements for Yocto test and run Bertrand Marquis
2022-10-12  8:02 ` Bertrand Marquis [this message]
2022-10-12  8:02 ` [PATCH v2 3/3] automation: Add a clean rule for containers Bertrand Marquis
2022-10-14 20:27 ` [PATCH v2 0/3] Yocto Gitlab CI Stefano Stabellini
2022-10-17  9:21   ` Bertrand Marquis
2022-10-18  1:27     ` Stefano Stabellini
2022-10-19  0:02       ` Stefano Stabellini
2022-10-19  8:10         ` Bertrand Marquis
2022-10-19  9:06         ` Michal Orzel
2022-10-19 10:40           ` Bertrand Marquis
2022-10-19 10:53             ` Michal Orzel
2022-10-19 22:12               ` Stefano Stabellini
2022-10-19 22:11             ` Stefano Stabellini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=3003e61047b648a703768d3ab1e2b251190ee591.1665561024.git.bertrand.marquis@arm.com \
    --to=bertrand.marquis@arm.com \
    --cc=cardoe@cardoe.com \
    --cc=sstabellini@kernel.org \
    --cc=xen-devel@lists.xenproject.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.