[xenomai-images][PATCH v2 0/8] Update to Debian 11

[xenomai-images][PATCH v2 0/8] Update to Debian 11

[Florian Bezdeka]

Hi all,

with this series the Xenomai demo image will be based on Debian 11.

All Debian 11 related topics that had to be adressed are now merged to
the Xenomai next branch. The stable branches have not yet been updated
so CI will report some failures for such branches.

Patch 4 made it to the list but has not been applied to next yet. That's
why I resend it, integrated into this series at the right place.

The Xenomai LAVA instance has been down while working on this topic. The
job submission to LAVA failed, but the test jobs where still marked as
successful. That is now fixed as well.

Best regards,
Florian

Changes since v1:
 - Renamed opt-lava-test.yml to opt-ci-board.yml
 - Moved removal of expand-on-first-boot from opt-ci.yml to opt-ci-board.yml

Florian Bezdeka (7):
  kas: Rename opt-lava-test.yml to opt-ci-board.yml
  ci: Disable partition expanding for all real test boards
  kas: Introduce kas option file for CI builds
  kas: Integrate opt-debug.yml into opt-ci.yml
  demo-image: Fix sshd-regen-keys problems on arm / qemu
  xenomai-demo: Switch to Debian bullseye
  ci: Fail test job when an error occurred during lava job submission

Jan Kiszka (1):
  linux-xenomai: arm: Enable CONFIG_ARM_MODULE_PLTS

 ci/gitlab-ci-base.yml                         | 10 ++---
 conf/distro/xenomai-demo.conf                 |  4 +-
 ...Start-key-generation-after-entropy-s.patch | 40 +++++++++++++++++++
 ...Fix-some-systemd-obsolete-warnings-a.patch | 34 ++++++++++++++++
 isar-patches/series                           |  2 +
 kas.yml                                       |  4 ++
 opt-lava-test.yml => opt-ci-board.yml         |  2 +
 opt-ci.yml                                    | 20 ++++++++++
 recipes-kernel/linux/files/armhf_defconfig    |  1 -
 scripts/run-lava-tests.sh                     |  1 +
 10 files changed, 110 insertions(+), 8 deletions(-)
 create mode 100644 isar-patches/0001-sshd-regen-keys-Start-key-generation-after-entropy-s.patch
 create mode 100644 isar-patches/0002-sshd-regen-keys-Fix-some-systemd-obsolete-warnings-a.patch
 create mode 100644 isar-patches/series
 rename opt-lava-test.yml => opt-ci-board.yml (83%)
 create mode 100644 opt-ci.yml

-- 
2.31.1

[xenomai-images][PATCH v2 1/8] kas: Rename opt-lava-test.yml to opt-ci-board.yml

[Florian Bezdeka]

This kas option file is used for real test / ci boards only. The
previous name indicated that it is used for all ci tests including
virtual devices like (qemu).

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 ci/gitlab-ci-base.yml                 | 6 +++---
 opt-lava-test.yml => opt-ci-board.yml | 0
 2 files changed, 3 insertions(+), 3 deletions(-)
 rename opt-lava-test.yml => opt-ci-board.yml (100%)

diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
index 8a1dbfb..3ea2467 100644
--- a/ci/gitlab-ci-base.yml
+++ b/ci/gitlab-ci-base.yml
@@ -97,7 +97,7 @@ default:
   extends: .build
   variables:
     TARGET: hikey
-    BUILD_OPTIONS: ":opt-lava-test.yml"
+    BUILD_OPTIONS: ":opt-ci-board.yml"
 
 .lava-test:hikey:
   extends: .test
@@ -109,7 +109,7 @@ default:
   extends: .build
   variables:
     TARGET: beagle-bone-black
-    BUILD_OPTIONS: ":opt-lava-test.yml"
+    BUILD_OPTIONS: ":opt-ci-board.yml"
 
 .lava-test:beagle-bone-black:
   extends: .test
@@ -121,7 +121,7 @@ default:
   extends: .build
   variables:
     TARGET: x86-64-efi
-    BUILD_OPTIONS: ":opt-lava-test.yml"
+    BUILD_OPTIONS: ":opt-ci-board.yml"
 
 .lava-test:x86-64-efi:
   extends: .test
diff --git a/opt-lava-test.yml b/opt-ci-board.yml
similarity index 100%
rename from opt-lava-test.yml
rename to opt-ci-board.yml
-- 
2.31.1

[xenomai-images][PATCH v2 2/8] ci: Disable partition expanding for all real test boards

[Florian Bezdeka]

Trying to expand the partition during first boot doesn't make sense
as we use NFS for deployment. Removing it helps to reduce the first boot
time a bit.

It is disabled on all qemu machines in the machine config, as we can
handle all qemu machines at once there. For real boards we want it to
stay, but not when building for CI tests.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 opt-ci-board.yml | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/opt-ci-board.yml b/opt-ci-board.yml
index 9c33da6..d894fe7 100644
--- a/opt-ci-board.yml
+++ b/opt-ci-board.yml
@@ -18,3 +18,5 @@ header:
 local_conf_header:
   image-type: |
     IMAGE_TYPE = "targz-img"
+  xenomai-ci: |
+    IMAGE_INSTALL_remove += "expand-on-first-boot"
\ No newline at end of file
-- 
2.31.1

[xenomai-images][PATCH v2 3/8] kas: Introduce kas option file for CI builds

[Florian Bezdeka]

For now the difference between the demo and the "ci image" is the
removed sshd-regen-keys. That reduces the (first) boot time especially
on low end CI hardware. Helps to stay within LAVA timeouts.

CI builds will now use that file.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 ci/gitlab-ci-base.yml |  4 ++--
 opt-ci.yml            | 18 ++++++++++++++++++
 2 files changed, 20 insertions(+), 2 deletions(-)
 create mode 100644 opt-ci.yml

diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
index 3ea2467..f7cdea5 100644
--- a/ci/gitlab-ci-base.yml
+++ b/ci/gitlab-ci-base.yml
@@ -39,8 +39,8 @@ default:
   stage: build
   script:
     - sudo sh -c "echo 'nameserver 8.8.8.8' > /etc/resolv.conf"
-    - echo "Building kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-debug.yml"
-    - kas build kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-debug.yml
+    - echo "Building kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-debug.yml:opt-ci.yml"
+    - kas build kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-debug.yml:opt-ci.yml
     - if [ -n "${USE_S3_BUCKET}" ]; then scripts/deploy_to_aws.sh ${TARGET}; fi
 
 .test:
diff --git a/opt-ci.yml b/opt-ci.yml
new file mode 100644
index 0000000..08400cc
--- /dev/null
+++ b/opt-ci.yml
@@ -0,0 +1,18 @@
+#
+# Xenomai Real-Time System
+#
+# Copyright (c) Siemens AG, 2021
+#
+# Authors:
+#  Florian Bezdeka <florian.bezdeka@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+header:
+  version: 10
+
+local_conf_header:
+  xenomai-ci: |
+    ROOTFS_POSTPROCESS_COMMAND_remove = "image_postprocess_sshd_key_regen"
+    IMAGE_INSTALL_remove += "sshd-regen-keys"
\ No newline at end of file
-- 
2.31.1

[xenomai-images][PATCH v2 4/8] kas: Integrate opt-debug.yml into opt-ci.yml

[Florian Bezdeka]

Whenever the kas option file opt-ci.yml is used we want the same build
options as the CI does, which includes setting the debug options.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 ci/gitlab-ci-base.yml | 4 ++--
 opt-ci.yml            | 2 ++
 2 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
index f7cdea5..cb2f23b 100644
--- a/ci/gitlab-ci-base.yml
+++ b/ci/gitlab-ci-base.yml
@@ -39,8 +39,8 @@ default:
   stage: build
   script:
     - sudo sh -c "echo 'nameserver 8.8.8.8' > /etc/resolv.conf"
-    - echo "Building kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-debug.yml:opt-ci.yml"
-    - kas build kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-debug.yml:opt-ci.yml
+    - echo "Building kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-ci.yml"
+    - kas build kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}:opt-ci.yml
     - if [ -n "${USE_S3_BUCKET}" ]; then scripts/deploy_to_aws.sh ${TARGET}; fi
 
 .test:
diff --git a/opt-ci.yml b/opt-ci.yml
index 08400cc..7429a3a 100644
--- a/opt-ci.yml
+++ b/opt-ci.yml
@@ -11,6 +11,8 @@
 
 header:
   version: 10
+  includes:
+    - opt-debug.yml
 
 local_conf_header:
   xenomai-ci: |
-- 
2.31.1

[xenomai-images][PATCH v2 5/8] demo-image: Fix sshd-regen-keys problems on arm / qemu

[Florian Bezdeka]

See patch descriptions for details. The main problem was that the key
generation took extremly long and several other services (like the
serial consoles) were not started because the configured timeout was
reached.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 ...Start-key-generation-after-entropy-s.patch | 40 +++++++++++++++++++
 ...Fix-some-systemd-obsolete-warnings-a.patch | 34 ++++++++++++++++
 isar-patches/series                           |  2 +
 kas.yml                                       |  4 ++
 4 files changed, 80 insertions(+)
 create mode 100644 isar-patches/0001-sshd-regen-keys-Start-key-generation-after-entropy-s.patch
 create mode 100644 isar-patches/0002-sshd-regen-keys-Fix-some-systemd-obsolete-warnings-a.patch
 create mode 100644 isar-patches/series

diff --git a/isar-patches/0001-sshd-regen-keys-Start-key-generation-after-entropy-s.patch b/isar-patches/0001-sshd-regen-keys-Start-key-generation-after-entropy-s.patch
new file mode 100644
index 0000000..56fb7d6
--- /dev/null
+++ b/isar-patches/0001-sshd-regen-keys-Start-key-generation-after-entropy-s.patch
@@ -0,0 +1,40 @@
+From 94b24ec5be3e3b0d32db4dbfe7cc9b63d8a0ce0a Mon Sep 17 00:00:00 2001
+From: Florian Bezdeka <florian.bezdeka@siemens.com>
+Date: Thu, 7 Oct 2021 17:59:44 +0200
+Subject: [PATCH 1/2] sshd-regen-keys: Start key generation after entropy seed
+
+Especially on low-end systems (including arm on qemu) it could happen
+that the key generation took extremly long. As side effect some other
+services (like the ones for serial consoles) run into the configured
+timeouts.
+
+[ TIME ] Timed out waiting for device /dev/hvc0.
+[DEPEND] Dependency failed for Serial Getty on hvc0.
+[ TIME ] Timed out waiting for device /dev/ttyAMA0.
+[DEPEND] Dependency failed for Serial Getty on ttyAMA0.
+
+Delaying the key generation to the point in time where
+systemd-random-seed finished solves this problem. My current
+understanding is that the regeneration is waiting for enough entropy
+which is being seeded in parallel.
+
+Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
+---
+ .../sshd-regen-keys/files/sshd-regen-keys.service                | 1 +
+ 1 file changed, 1 insertion(+)
+
+diff --git a/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service b/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service
+index a05e1a9..4d0a8c6 100644
+--- a/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service
++++ b/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service
+@@ -3,6 +3,7 @@ Description=Regenerate sshd host keys
+ DefaultDependencies=no
+ Conflicts=shutdown.target
+ After=systemd-remount-fs.service
++After=systemd-random-seed.service
+ Before=shutdown.target sshd.service
+ ConditionPathIsReadWrite=/etc
+ 
+-- 
+2.31.1
+
diff --git a/isar-patches/0002-sshd-regen-keys-Fix-some-systemd-obsolete-warnings-a.patch b/isar-patches/0002-sshd-regen-keys-Fix-some-systemd-obsolete-warnings-a.patch
new file mode 100644
index 0000000..1b8d2e0
--- /dev/null
+++ b/isar-patches/0002-sshd-regen-keys-Fix-some-systemd-obsolete-warnings-a.patch
@@ -0,0 +1,34 @@
+From 4d20f39987326d1a545015c9bbb385b6d9219c67 Mon Sep 17 00:00:00 2001
+From: Florian Bezdeka <florian.bezdeka@siemens.com>
+Date: Thu, 7 Oct 2021 18:05:15 +0200
+Subject: [PATCH 2/2] sshd-regen-keys: Fix some systemd obsolete warnings about
+ using syslog
+
+Fixes the following warnings:
+
+[   11.450104] systemd[1]: /lib/systemd/system/sshd-regen-keys.service:15:
+[   11.450806] systemd[1]: /lib/systemd/system/sshd-regen-keys.service:16:
+	Standard output type syslog is obsolete, automatically updating to journal.
+	Please update your unit file, and consider removing the setting altogether.
+
+Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
+---
+ .../sshd-regen-keys/files/sshd-regen-keys.service               | 2 --
+ 1 file changed, 2 deletions(-)
+
+diff --git a/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service b/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service
+index 4d0a8c6..ebcee86 100644
+--- a/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service
++++ b/meta/recipes-support/sshd-regen-keys/files/sshd-regen-keys.service
+@@ -13,8 +13,6 @@ RemainAfterExit=yes
+ Environment=DEBIAN_FRONTEND=noninteractive
+ ExecStart=/usr/sbin/sshd-regen-keys.sh
+ ExecStartPost=-/bin/systemctl disable sshd-regen-keys.service
+-StandardOutput=syslog
+-StandardError=syslog
+ 
+ [Install]
+ WantedBy=sysinit.target
+-- 
+2.31.1
+
diff --git a/isar-patches/series b/isar-patches/series
new file mode 100644
index 0000000..3e76460
--- /dev/null
+++ b/isar-patches/series
@@ -0,0 +1,2 @@
+0001-sshd-regen-keys-Start-key-generation-after-entropy-s.patch
+0002-sshd-regen-keys-Fix-some-systemd-obsolete-warnings-a.patch
diff --git a/kas.yml b/kas.yml
index c8bafcd..345fccc 100644
--- a/kas.yml
+++ b/kas.yml
@@ -23,6 +23,10 @@ repos:
   isar:
     url: https://github.com/ilbers/isar.git
     refspec: 1cfe166c5e53ae96bc07b895a92bf6cd7ace7bc9
+    patches:
+      "isar-patches":
+        repo: xenomai
+        path: isar-patches
     layers:
       meta:
 
-- 
2.31.1

[xenomai-images][PATCH v2 6/8] linux-xenomai: arm: Enable CONFIG_ARM_MODULE_PLTS

[Florian Bezdeka]

From: Jan Kiszka <jan.kiszka@siemens.com>

No reason to turn this off, in fact, we have to turn it on with
bullseye's gcc-10 and debug options enabled to avoid relocation
range problems when loading modules.

Reported-by: Florian Bezdeka <florian.bezdeka@siemens.com>
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 recipes-kernel/linux/files/armhf_defconfig | 1 -
 1 file changed, 1 deletion(-)

diff --git a/recipes-kernel/linux/files/armhf_defconfig b/recipes-kernel/linux/files/armhf_defconfig
index ef4981d..eda674c 100644
--- a/recipes-kernel/linux/files/armhf_defconfig
+++ b/recipes-kernel/linux/files/armhf_defconfig
@@ -1112,7 +1112,6 @@ CONFIG_HIGHPTE=y
 CONFIG_CPU_SW_DOMAIN_PAN=y
 CONFIG_HW_PERF_EVENTS=y
 CONFIG_ARCH_WANT_GENERAL_HUGETLB=y
-# CONFIG_ARM_MODULE_PLTS is not set
 CONFIG_FLATMEM=y
 CONFIG_FLAT_NODE_MEM_MAP=y
 CONFIG_HAVE_MEMBLOCK=y
-- 
2.31.1

[xenomai-images][PATCH v2 7/8] xenomai-demo: Switch to Debian bullseye

[Florian Bezdeka]

Switching from Debian buster to Debian bullseye as base for or xenomai-demo
distribution.

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 conf/distro/xenomai-demo.conf | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/conf/distro/xenomai-demo.conf b/conf/distro/xenomai-demo.conf
index 1c9209d..4794584 100644
--- a/conf/distro/xenomai-demo.conf
+++ b/conf/distro/xenomai-demo.conf
@@ -9,12 +9,12 @@
 # SPDX-License-Identifier: MIT
 #
 
-require conf/distro/debian-buster.conf
+require conf/distro/debian-bullseye.conf
 
 PREFERRED_VERSION_xenomai ?= "3.1%"
 include conf/distro/include/xenomai-version.inc
 
-HOST_DISTRO ?= "debian-buster"
+HOST_DISTRO ?= "debian-bullseye"
 
 KERNEL_NAME = "xenomai"
 
-- 
2.31.1

[xenomai-images][PATCH v2 8/8] ci: Fail test job when an error occurred during lava job submission

[Florian Bezdeka]

It can happen that the LAVA job submission fails. If so, the CI job
should be marked as failed. Previously we had something like the
following in the log, but the job was still marked as "OK".

Log:
connect to lava server: https://lava.xenomai.org
Deploy artifacts from 'https://xenomai-images-artifacts.s3.amazonaws.com/artifacts'
submit lava job
<ProtocolError for https://[snip]/RPC2: 500 HTTPSConnectionPool(host='lava.xenomai.org', port=443): Read timed out. (read timeout=20.0)>
<ProtocolError for https://[snip]/RPC2: 500 HTTPSConnectionPool(host='lava.xenomai.org', port=443): Read timed out. (read timeout=20.0)>
scripts/run-lava-tests.sh: 56: [: Illegal number:

Signed-off-by: Florian Bezdeka <florian.bezdeka@siemens.com>
---
 scripts/run-lava-tests.sh | 1 +
 1 file changed, 1 insertion(+)

diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index dde542c..de63b28 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -9,6 +9,7 @@
 #
 # SPDX-License-Identifier: MIT
 #
+set -e
 TARGET=$1
 if [ -z "${TARGET}" ]; then
     echo "no target was given"
-- 
2.31.1

Re: [xenomai-images][PATCH v2 3/8] kas: Introduce kas option file for CI builds

[Jan Kiszka]

On 13.10.21 14:17, Florian Bezdeka wrote:
> For now the difference between the demo and the "ci image" is the
> removed sshd-regen-keys. That reduces the (first) boot time especially
> on low end CI hardware. Helps to stay within LAVA timeouts.
> 
> CI builds will now use that file.
> 

Too many options. Can't we consolidate over a single CI/CT thing that
also includes the changes for ci-board?

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [xenomai-images][PATCH v2 3/8] kas: Introduce kas option file for CI builds

[Bezdeka, Florian]

On Wed, 2021-10-13 at 14:22 +0200, Jan Kiszka wrote:
> On 13.10.21 14:17, Florian Bezdeka wrote:
> > For now the difference between the demo and the "ci image" is the
> > removed sshd-regen-keys. That reduces the (first) boot time especially
> > on low end CI hardware. Helps to stay within LAVA timeouts.
> > 
> > CI builds will now use that file.
> > 
> 
> Too many options. Can't we consolidate over a single CI/CT thing that
> also includes the changes for ci-board?

It has been a while, but coming back to this now...

I agree, we have a lot of such option files, but after checking them
all again: They all have a unique purpose.

We don't have a single CI/CT option file because the qemu test targets
are different in comparison to the real board targets. We do not boot
up a "disk"/wic image, we pass a kernel image to qemu.

Without introducing opt-ci.yml I would have to spread the ci
optimizations into several board specific configs, which would enable
the ci optimization for "real board builds" as well. That is not what
we want.

Further ideas?

What about introducing a ci image? Does that make sense? The content
would nearly be the same as within opt-ci.yml now.

> 
> Jan
> 

Re: [xenomai-images][PATCH v2 3/8] kas: Introduce kas option file for CI builds

[Jan Kiszka]

On 10.11.21 14:23, Bezdeka, Florian (T RDA IOT SES-DE) wrote:
> On Wed, 2021-10-13 at 14:22 +0200, Jan Kiszka wrote:
>> On 13.10.21 14:17, Florian Bezdeka wrote:
>>> For now the difference between the demo and the "ci image" is the
>>> removed sshd-regen-keys. That reduces the (first) boot time especially
>>> on low end CI hardware. Helps to stay within LAVA timeouts.
>>>
>>> CI builds will now use that file.
>>>
>>
>> Too many options. Can't we consolidate over a single CI/CT thing that
>> also includes the changes for ci-board?
> 
> It has been a while, but coming back to this now...
> 
> I agree, we have a lot of such option files, but after checking them
> all again: They all have a unique purpose.

For users? I doubt that. We have

- opt-ext4-gz.yml: It enables QEMU targets, and only those, to be
  compatible with the LAVA test setup by compressing its ext4 images

- opt-lava-test.yml: It switches wic-based targets, and that are only
  the physical ones, to tarballs - again for LAVA purposes

- opt-ci.yml: Removes packages, not relevant for the above images, and
  only those

I really see no separate purposes, just separate files. That is likely
only for technical reasons.

> 
> We don't have a single CI/CT option file because the qemu test targets
> are different in comparison to the real board targets. We do not boot
> up a "disk"/wic image, we pass a kernel image to qemu.
> 
> Without introducing opt-ci.yml I would have to spread the ci
> optimizations into several board specific configs, which would enable
> the ci optimization for "real board builds" as well. That is not what
> we want.
> 
> Further ideas?
> 
> What about introducing a ci image? Does that make sense? The content
> would nearly be the same as within opt-ci.yml now.

An image alone would not consolidate those changes above into a single
one that can be controlled from a single opt-ci.yml file.

We likely need a CI_BUILD bitbake variable that controls with bitbake
logic the changes needed for CI, with bitbake conditionals.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [xenomai-images][PATCH v2 3/8] kas: Introduce kas option file for CI builds

[Bezdeka, Florian]

On Wed, 2021-11-10 at 14:42 +0100, Jan Kiszka wrote:
> On 10.11.21 14:23, Bezdeka, Florian (T RDA IOT SES-DE) wrote:
> > On Wed, 2021-10-13 at 14:22 +0200, Jan Kiszka wrote:
> > > On 13.10.21 14:17, Florian Bezdeka wrote:
> > > > For now the difference between the demo and the "ci image" is the
> > > > removed sshd-regen-keys. That reduces the (first) boot time especially
> > > > on low end CI hardware. Helps to stay within LAVA timeouts.
> > > > 
> > > > CI builds will now use that file.
> > > > 
> > > 
> > > Too many options. Can't we consolidate over a single CI/CT thing that
> > > also includes the changes for ci-board?
> > 
> > It has been a while, but coming back to this now...
> > 
> > I agree, we have a lot of such option files, but after checking them
> > all again: They all have a unique purpose.
> 
> For users? I doubt that. We have
> 
> - opt-ext4-gz.yml: It enables QEMU targets, and only those, to be
>   compatible with the LAVA test setup by compressing its ext4 images

It's the gitlab-ci config that limits it to qemu targets currently. But
that could evolve with new targets being added. 

> 
> - opt-lava-test.yml: It switches wic-based targets, and that are only
>   the physical ones, to tarballs - again for LAVA purposes
> 

This one was renamed in one of the previous patches to opt-ci-board.yml
to make sure that this one is useful for real CI targets (only). It's
up to the CI configuration to add it to the list of used options for
new build/test tasks.

> - opt-ci.yml: Removes packages, not relevant for the above images, and
>   only those

That's the new one introduced here. Collecting only CI specific things.
Nothing else.

> 
> I really see no separate purposes, just separate files. That is likely
> only for technical reasons.
> 
> > 
> > We don't have a single CI/CT option file because the qemu test targets
> > are different in comparison to the real board targets. We do not boot
> > up a "disk"/wic image, we pass a kernel image to qemu.
> > 
> > Without introducing opt-ci.yml I would have to spread the ci
> > optimizations into several board specific configs, which would enable
> > the ci optimization for "real board builds" as well. That is not what
> > we want.
> > 
> > Further ideas?
> > 
> > What about introducing a ci image? Does that make sense? The content
> > would nearly be the same as within opt-ci.yml now.
> 
> An image alone would not consolidate those changes above into a single
> one that can be controlled from a single opt-ci.yml file.
> 
> We likely need a CI_BUILD bitbake variable that controls with bitbake
> logic the changes needed for CI, with bitbake conditionals.

Currently we can define new CI tasks/jobs by just adding them to the
(gitlab) CI config, selecting the right option files for each task. As
we have at least board specific adjustments: Wouldn't that mean that
adding new targets would require recipe / bitbake adjustments?

Purpose of this series was moving up to Debian 11. Maybe we could avoid
this quite big refactoring here by just removing the CI optimizations
again, and postpone that?

> 
> Jan
>