[isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[Alice Ferrazzi]

This patch series add a new image with settings for
KernelCI.

This new image is called cip-core-image-kernelci and is
based on isar-cip-core general image with some changes
needed for make it work with KernelCI and taken from
KernelCI image base settings.
The cip-core-image-kernelci images are built by GitlabCI
and uploaded to KernelCI production fileserver.
https://storage.kernelci.org/images/rootfs/cip/

These patches are already integrated in the
isar-cip-core:alicef/kernelci_master_refactor repository branch
and are generating the images that are currently
used by KernelCI.

cip-core-image-kernelci as been tested and are
already used in KernelCI production with good results.
https://linux.kernelci.org/test/job/stable-rc/branch/queue%2F5.14/kernel/v5.14.17-9-g9f7eecaa70b3/plan/baseline-cip-nfs/

v2:
* drop trailing semicolon from gitlab-ci.yml


Alice Ferrazzi (4):
  Add cip-core-image-kernelci
  Add dmesg filter needed for lava test result
  Add script deploy-kernelci.py for upload the cip-core-image-kernelci
  enable cip-core-image-kernelci

 .gitlab-ci.yml                                | 38 +++++++++++++
 kas/opt/kernelci.yml                          | 16 ++++++
 .../images/cip-core-image-kernelci.bb         | 16 ++++++
 .../files/99-silent-printk.conf               |  1 +
 .../kernelci-customizations/files/dmesg.sh    | 23 ++++++++
 .../kernelci-customizations/files/ethernet    | 23 ++++++++
 .../kernelci-customizations/files/postinst    | 34 ++++++++++++
 .../kernelci-customizations.bb                | 35 ++++++++++++
 scripts/deploy-kernelci.py                    | 55 +++++++++++++++++++
 9 files changed, 241 insertions(+)
 create mode 100644 kas/opt/kernelci.yml
 create mode 100644 recipes-core/images/cip-core-image-kernelci.bb
 create mode 100644 recipes-core/kernelci-customizations/files/99-silent-printk.conf
 create mode 100644 recipes-core/kernelci-customizations/files/dmesg.sh
 create mode 100644 recipes-core/kernelci-customizations/files/ethernet
 create mode 100644 recipes-core/kernelci-customizations/files/postinst
 create mode 100644 recipes-core/kernelci-customizations/kernelci-customizations.bb
 create mode 100755 scripts/deploy-kernelci.py

-- 
2.33.1

[isar-cip-core][PATCH v2 1/4] Add cip-core-image-kernelci

[Alice Ferrazzi]

This image is currently used by KernelCI production for testing purpose.
The purpose of this image is that KernelCI need to be able to autologin
without password and to detect the login shell using special characters.
Currently added are the default settings used by KernelCI images.

Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
---
 kas/opt/kernelci.yml                          | 16 +++++++++
 .../images/cip-core-image-kernelci.bb         | 16 +++++++++
 .../files/99-silent-printk.conf               |  1 +
 .../kernelci-customizations/files/ethernet    | 23 +++++++++++++
 .../kernelci-customizations/files/postinst    | 34 +++++++++++++++++++
 .../kernelci-customizations.bb                | 34 +++++++++++++++++++
 6 files changed, 124 insertions(+)
 create mode 100644 kas/opt/kernelci.yml
 create mode 100644 recipes-core/images/cip-core-image-kernelci.bb
 create mode 100644 recipes-core/kernelci-customizations/files/99-silent-printk.conf
 create mode 100644 recipes-core/kernelci-customizations/files/ethernet
 create mode 100644 recipes-core/kernelci-customizations/files/postinst
 create mode 100644 recipes-core/kernelci-customizations/kernelci-customizations.bb

diff --git a/kas/opt/kernelci.yml b/kas/opt/kernelci.yml
new file mode 100644
index 0000000..9c67864
--- /dev/null
+++ b/kas/opt/kernelci.yml
@@ -0,0 +1,16 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Toshiba Corporation, 2020
+# Copyright (c) Cybertrust Japan Co., Ltd., 2021
+#
+# Authors:
+#  Venkata Pyla <venkata.pyla@toshiba-tsip.com>
+#  Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
+#
+# SPDX-License-Identifier: MIT
+#
+header:
+  version: 10
+
+target: cip-core-image-kernelci
diff --git a/recipes-core/images/cip-core-image-kernelci.bb b/recipes-core/images/cip-core-image-kernelci.bb
new file mode 100644
index 0000000..479c14c
--- /dev/null
+++ b/recipes-core/images/cip-core-image-kernelci.bb
@@ -0,0 +1,16 @@
+#
+# A reference image for KernelCI
+#
+# Copyright (c) Cybertrust Japan Co., Ltd., 2021
+#
+# Authors:
+#  Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit image
+
+DESCRIPTION = "CIP Core image for KernelCI"
+
+IMAGE_INSTALL += "kernelci-customizations"
diff --git a/recipes-core/kernelci-customizations/files/99-silent-printk.conf b/recipes-core/kernelci-customizations/files/99-silent-printk.conf
new file mode 100644
index 0000000..ad24d3a
--- /dev/null
+++ b/recipes-core/kernelci-customizations/files/99-silent-printk.conf
@@ -0,0 +1 @@
+kernel.printk = 3 4 1 3
diff --git a/recipes-core/kernelci-customizations/files/ethernet b/recipes-core/kernelci-customizations/files/ethernet
new file mode 100644
index 0000000..fa47d1a
--- /dev/null
+++ b/recipes-core/kernelci-customizations/files/ethernet
@@ -0,0 +1,23 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Siemens AG, 2019
+#
+# Authors:
+#  Jan Kiszka <jan.kiszka@siemens.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+allow-hotplug eth0
+allow-hotplug enp0s2
+allow-hotplug enp2s0
+
+# used on BBB
+iface eth0 inet dhcp
+
+# used on qemu-amd64
+iface enp0s2 inet dhcp
+
+# used on simatic-ipc227e
+iface enp2s0 inet dhcp
diff --git a/recipes-core/kernelci-customizations/files/postinst b/recipes-core/kernelci-customizations/files/postinst
new file mode 100644
index 0000000..7ae30e8
--- /dev/null
+++ b/recipes-core/kernelci-customizations/files/postinst
@@ -0,0 +1,34 @@
+#!/bin/sh
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Siemens AG, 2019
+# Copyright (c) Cybertrust Japan Co., Ltd., 2021
+#
+# Authors:
+#  Jan Kiszka <jan.kiszka@siemens.com>
+#  Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+echo "CIP Core Demo & Test Image" > /etc/issue
+
+# permit root login without password
+echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
+echo "PermitEmptyPasswords yes" >> /etc/ssh/sshd_config
+passwd root -d
+
+# serial getty service for autologin
+mkdir -p /etc/systemd/system/serial-getty@.service.d/
+echo "[Service]" > /etc/systemd/system/serial-getty@.service.d/override.conf
+echo "ExecStart=" >> /etc/systemd/system/serial-getty@.service.d/override.conf
+echo "ExecStart=-/sbin/agetty --autologin root --keep-baud 115200,38400,9600 %I $TERM" >> /etc/systemd/system/serial-getty@.service.d/override.conf
+
+# set the profile for KernelCI
+echo "PS1='\$(pwd) # '" > /root/.profile
+echo "cd /" >> /root/.profile
+
+HOSTNAME=demo
+echo "$HOSTNAME" > /etc/hostname
+echo "127.0.0.1 $HOSTNAME" >> /etc/hosts
diff --git a/recipes-core/kernelci-customizations/kernelci-customizations.bb b/recipes-core/kernelci-customizations/kernelci-customizations.bb
new file mode 100644
index 0000000..dca0891
--- /dev/null
+++ b/recipes-core/kernelci-customizations/kernelci-customizations.bb
@@ -0,0 +1,34 @@
+#
+# CIP Core, generic profile
+#
+# Copyright (c) Siemens AG, 2019
+# Copyright (c) Cybertrust Japan Co., Ltd., 2021
+#
+# Authors:
+#  Jan Kiszka <jan.kiszka@siemens.com>
+#  Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
+#
+# SPDX-License-Identifier: MIT
+#
+
+inherit dpkg-raw
+
+DESCRIPTION = "CIP Core image demo & customizations"
+
+SRC_URI = " \
+    file://postinst \
+    file://ethernet \
+    file://99-silent-printk.conf"
+
+DEPENDS += "sshd-regen-keys"
+
+DEBIAN_DEPENDS = " \
+    ifupdown, isc-dhcp-client, net-tools, iputils-ping, ssh, sshd-regen-keys"
+
+do_install() {
+  install -v -d ${D}/etc/network/interfaces.d
+  install -v -m 644 ${WORKDIR}/ethernet ${D}/etc/network/interfaces.d/
+
+  install -v -d ${D}/etc/sysctl.d
+  install -v -m 644 ${WORKDIR}/99-silent-printk.conf ${D}/etc/sysctl.d/
+}
-- 
2.33.1

[isar-cip-core][PATCH v2 2/4] Add dmesg filter needed for lava test result

[Alice Ferrazzi]

KernelCI is using a dmesg filter script for checking dmesg
logs result with lava.
Adding the script to the cip-core-image-kernelci.

Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
---
 .../kernelci-customizations/files/dmesg.sh    | 23 +++++++++++++++++++
 .../kernelci-customizations.bb                |  1 +
 2 files changed, 24 insertions(+)
 create mode 100644 recipes-core/kernelci-customizations/files/dmesg.sh

diff --git a/recipes-core/kernelci-customizations/files/dmesg.sh b/recipes-core/kernelci-customizations/files/dmesg.sh
new file mode 100644
index 0000000..3b096e1
--- /dev/null
+++ b/recipes-core/kernelci-customizations/files/dmesg.sh
@@ -0,0 +1,23 @@
+#!/bin/sh
+
+set -e
+
+if [ "$KERNELCI_LAVA" = "y" ]; then
+    alias test-result='lava-test-case'
+else
+    alias test-result='echo'
+fi
+
+for level in crit alert emerg; do
+    dmesg --level=$level --notime -x -k > dmesg.$level
+    test -s dmesg.$level && res=fail || res=pass
+    count=$(cat dmesg.$level | wc -l)
+    cat dmesg.$level
+    test-result \
+        $level \
+        --result $res \
+        --measurement $count \
+        --units lines
+done
+
+exit 0
diff --git a/recipes-core/kernelci-customizations/kernelci-customizations.bb b/recipes-core/kernelci-customizations/kernelci-customizations.bb
index dca0891..c3f677c 100644
--- a/recipes-core/kernelci-customizations/kernelci-customizations.bb
+++ b/recipes-core/kernelci-customizations/kernelci-customizations.bb
@@ -18,6 +18,7 @@ DESCRIPTION = "CIP Core image demo & customizations"
 SRC_URI = " \
     file://postinst \
     file://ethernet \
+    file://dmesg.sh \
     file://99-silent-printk.conf"
 
 DEPENDS += "sshd-regen-keys"
-- 
2.33.1

[isar-cip-core][PATCH v2 3/4] Add script deploy-kernelci.py for upload the cip-core-image-kernelci

[Alice Ferrazzi]

The cip-core-image-kernelci need to be uploaded to the KernelCI
production storage for been used by KernelCI.
This script use the KernelCI API for uploading the
cip-core-image-kernelci to the production storage.
The images are uploaded in the following link:
https://storage.kernelci.org/images/rootfs/cip/

Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
---
 scripts/deploy-kernelci.py | 55 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 55 insertions(+)
 create mode 100755 scripts/deploy-kernelci.py

diff --git a/scripts/deploy-kernelci.py b/scripts/deploy-kernelci.py
new file mode 100755
index 0000000..5a8adca
--- /dev/null
+++ b/scripts/deploy-kernelci.py
@@ -0,0 +1,55 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+
+import subprocess
+import requests
+import os
+import sys
+import time
+from urllib.parse import urljoin
+
+cdate=time.strftime("%Y%m%d")
+api="https://api.kernelci.org/upload"
+token=os.getenv("KERNELCI_TOKEN")
+
+release=sys.argv[1]
+target=sys.argv[2]
+extension=sys.argv[3]
+
+rootfs_filename="cip-core-image-kernelci-cip-core-"+release+"-"+target+".tar.gz"
+initrd_filename="cip-core-image-kernelci-cip-core-"+release+"-"+target+"-initrd.img"
+initrd_gz_filename="cip-core-image-kernelci-cip-core-"+release+"-"+target+"-initrd.img.gz"
+
+input_dir="build/tmp/deploy/images/"+target
+upload_path="/images/rootfs/cip/"+cdate+"/"+target+"/"
+upload_path_latest="/images/rootfs/cip/latest/"+target+"/"
+rootfs=input_dir+"/"+rootfs_filename
+initrd=input_dir+"/"+initrd_filename
+
+def upload_file(api, token, path, input_file, input_filename):
+    headers = {
+        'Authorization': token,
+    }
+    data = {
+        'path': path,
+    }
+    files = {
+        'file': (input_filename, open(input_file, 'rb').read()),
+    }
+    url = urljoin(api, 'upload')
+    resp = requests.post(url, headers=headers, data=data, files=files)
+    resp.raise_for_status()
+
+if os.path.exists(rootfs) and os.path.exists(initrd):
+    print("uploading rootfs to KernelCI")
+    upload_file(api, token, upload_path, rootfs, rootfs_filename)
+    print("uploading initrd to KernelCI")
+    upload_file(api, token, upload_path, initrd, initrd_gz_filename)
+    print("uploaded to: https://storage.kernelci.org"+upload_path)
+
+    # Upload latest
+    print("uploading rootfs to KernelCI CIP latest")
+    upload_file(api, token, upload_path_latest, rootfs, rootfs_filename)
+    print("uploading initrd to KernelCI CIP latest")
+    upload_file(api, token, upload_path_latest, initrd, initrd_gz_filename)
+    print("uploaded to: https://storage.kernelci.org"+upload_path_latest)
-- 
2.33.1

[isar-cip-core][PATCH v2 4/4] enable cip-core-image-kernelci

[Alice Ferrazzi]

enable CIP core image KernelCI to be built with gitlab pipeline
and uploaded to the KernelCI storage using KernelCI API.
enable cip-core-image-kernelci to be built for amd64, arm and arm64.

Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
---
 .gitlab-ci.yml | 38 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index e9cbc60..c06c783 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -9,6 +9,7 @@ variables:
   targz: disable
   dtb: none
   deploy: enable
+  deploy_kernelci: disable
 
 stages:
   - build
@@ -38,6 +39,7 @@ default:
     - echo "Building ${base_yaml}"
     - kas build ${base_yaml}
     - if [ "${deploy}" = "enable" ]; then scripts/deploy-cip-core.sh ${release} ${target} ${extension} ${dtb} ${CI_COMMIT_REF_SLUG}; fi
+    - if [ "${deploy_kernelci}" = "enable" ]; then scripts/deploy-kernelci.py ${release} ${target} ${extension} ${dtb}; fi
 
 # base image
 build:simatic-ipc227e-base:
@@ -77,6 +79,18 @@ build:qemu-amd64-base:
     wic_targz: disable
     targz: enable
 
+build:qemu-amd64-base-kernelci:
+  extends:
+    - .build_base
+  variables:
+    target: qemu-amd64
+    extension: kernelci
+    use_rt: disable
+    wic_targz: disable
+    targz: enable
+    deploy: disable
+    deploy_kernelci: enable
+
 build:qemu-arm64-base:
   extends:
     - .build_base
@@ -87,6 +101,18 @@ build:qemu-arm64-base:
     wic_targz: disable
     targz: enable
 
+build:qemu-arm64-base-kernelci:
+  extends:
+    - .build_base
+  variables:
+    target: qemu-arm64
+    extension: kernelci
+    use_rt: disable
+    wic_targz: disable
+    targz: enable
+    deploy: disable
+    deploy_kernelci: enable
+
 build:qemu-arm-base:
   extends:
     - .build_base
@@ -97,6 +123,18 @@ build:qemu-arm-base:
     wic_targz: disable
     targz: enable
 
+build:qemu-arm-base-kernelci:
+  extends:
+    - .build_base
+  variables:
+    target: qemu-arm
+    extension: kernelci
+    use_rt: disable
+    wic_targz: disable
+    targz: enable
+    deploy: disable
+    deploy_kernelci: enable
+
 # test
 build:simatic-ipc227e-test:
   extends:
-- 
2.33.1

Re: [isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[Jan Kiszka]

On 20.01.22 13:54, Alice Ferrazzi wrote:
> This patch series add a new image with settings for
> KernelCI.
> 
> This new image is called cip-core-image-kernelci and is
> based on isar-cip-core general image with some changes
> needed for make it work with KernelCI and taken from
> KernelCI image base settings.
> The cip-core-image-kernelci images are built by GitlabCI
> and uploaded to KernelCI production fileserver.
> https://storage.kernelci.org/images/rootfs/cip/
> 
> These patches are already integrated in the
> isar-cip-core:alicef/kernelci_master_refactor repository branch
> and are generating the images that are currently
> used by KernelCI.
> 
> cip-core-image-kernelci as been tested and are
> already used in KernelCI production with good results.
> https://linux.kernelci.org/test/job/stable-rc/branch/queue%2F5.14/kernel/v5.14.17-9-g9f7eecaa70b3/plan/baseline-cip-nfs/
> 
> v2:
> * drop trailing semicolon from gitlab-ci.yml
> 

Didn't you also want to de-duplicate the customization recipe, build 
kernelci on top of the common one?

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[Alice Ferrazzi]

On Wed, Jan 26, 2022 at 6:37 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>
> On 20.01.22 13:54, Alice Ferrazzi wrote:
> > This patch series add a new image with settings for
> > KernelCI.
> >
> > This new image is called cip-core-image-kernelci and is
> > based on isar-cip-core general image with some changes
> > needed for make it work with KernelCI and taken from
> > KernelCI image base settings.
> > The cip-core-image-kernelci images are built by GitlabCI
> > and uploaded to KernelCI production fileserver.
> > https://storage.kernelci.org/images/rootfs/cip/
> >
> > These patches are already integrated in the
> > isar-cip-core:alicef/kernelci_master_refactor repository branch
> > and are generating the images that are currently
> > used by KernelCI.
> >
> > cip-core-image-kernelci as been tested and are
> > already used in KernelCI production with good results.
> > https://linux.kernelci.org/test/job/stable-rc/branch/queue%2F5.14/kernel/v5.14.17-9-g9f7eecaa70b3/plan/baseline-cip-nfs/
> >
> > v2:
> > * drop trailing semicolon from gitlab-ci.yml
> >
>
> Didn't you also want to de-duplicate the customization recipe, build
> kernelci on top of the common one?
>
from my understanding we decided to don't go in that way,
because changes in the common part could break the more generics
KernelCI settings.

Thanks,
Alicef

-- 
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================

Re: [isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[Jan Kiszka]

On 26.01.22 11:27, Alice Ferrazzi wrote:
> On Wed, Jan 26, 2022 at 6:37 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>
>> On 20.01.22 13:54, Alice Ferrazzi wrote:
>>> This patch series add a new image with settings for
>>> KernelCI.
>>>
>>> This new image is called cip-core-image-kernelci and is
>>> based on isar-cip-core general image with some changes
>>> needed for make it work with KernelCI and taken from
>>> KernelCI image base settings.
>>> The cip-core-image-kernelci images are built by GitlabCI
>>> and uploaded to KernelCI production fileserver.
>>> https://storage.kernelci.org/images/rootfs/cip/
>>>
>>> These patches are already integrated in the
>>> isar-cip-core:alicef/kernelci_master_refactor repository branch
>>> and are generating the images that are currently
>>> used by KernelCI.
>>>
>>> cip-core-image-kernelci as been tested and are
>>> already used in KernelCI production with good results.
>>> https://linux.kernelci.org/test/job/stable-rc/branch/queue%2F5.14/kernel/v5.14.17-9-g9f7eecaa70b3/plan/baseline-cip-nfs/
>>>
>>> v2:
>>> * drop trailing semicolon from gitlab-ci.yml
>>>
>>
>> Didn't you also want to de-duplicate the customization recipe, build
>> kernelci on top of the common one?
>>
> from my understanding we decided to don't go in that way,
> because changes in the common part could break the more generics
> KernelCI settings.
> 

Fork where there is relevant different, use the same one where there is 
not. Duplicating files/99-silent-printk.conf and files/ethernet is 
surely pointless. The latter will actually cause unneeded maintenance 
efforts when adding new boards.

Please also avoid needless diffs between the recipes so that they can be 
re-aligned easily when boards are added.

And I still think we will eventually miss the wireless setting from the 
generic customization packages in kernelci, ie. the only real difference 
in those recipes now.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[Alice Ferrazzi]

On Wed, Jan 26, 2022 at 7:37 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>
> On 26.01.22 11:27, Alice Ferrazzi wrote:
> > On Wed, Jan 26, 2022 at 6:37 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
> >>
> >> On 20.01.22 13:54, Alice Ferrazzi wrote:
> >>> This patch series add a new image with settings for
> >>> KernelCI.
> >>>
> >>> This new image is called cip-core-image-kernelci and is
> >>> based on isar-cip-core general image with some changes
> >>> needed for make it work with KernelCI and taken from
> >>> KernelCI image base settings.
> >>> The cip-core-image-kernelci images are built by GitlabCI
> >>> and uploaded to KernelCI production fileserver.
> >>> https://storage.kernelci.org/images/rootfs/cip/
> >>>
> >>> These patches are already integrated in the
> >>> isar-cip-core:alicef/kernelci_master_refactor repository branch
> >>> and are generating the images that are currently
> >>> used by KernelCI.
> >>>
> >>> cip-core-image-kernelci as been tested and are
> >>> already used in KernelCI production with good results.
> >>> https://linux.kernelci.org/test/job/stable-rc/branch/queue%2F5.14/kernel/v5.14.17-9-g9f7eecaa70b3/plan/baseline-cip-nfs/
> >>>
> >>> v2:
> >>> * drop trailing semicolon from gitlab-ci.yml
> >>>
> >>
> >> Didn't you also want to de-duplicate the customization recipe, build
> >> kernelci on top of the common one?
> >>
> > from my understanding we decided to don't go in that way,
> > because changes in the common part could break the more generics
> > KernelCI settings.
> >
>
> Fork where there is relevant different, use the same one where there is
> not. Duplicating files/99-silent-printk.conf and files/ethernet is
> surely pointless. The latter will actually cause unneeded maintenance
> efforts when adding new boards.
>
> Please also avoid needless diffs between the recipes so that they can be
> re-aligned easily when boards are added.
>
> And I still think we will eventually miss the wireless setting from the
> generic customization packages in kernelci, ie. the only real difference
> in those recipes now.

Is not as we are adding dmesg.sh script (please check the next commit).
KernelCI have a different environment than CIP current gitlab testing.

That's why the last time we decided to merge as is.
https://lists.cip-project.org/g/cip-dev/topic/88235357#7341
https://lists.cip-project.org/g/cip-dev/message/7337
https://lists.cip-project.org/g/cip-dev/message/7340





--
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================

Re: [isar-cip-core][PATCH v2 0/4] Add cip-core-image-kernelci building and uploading system

[Jan Kiszka]

On 26.01.22 12:43, Alice Ferrazzi wrote:
> On Wed, Jan 26, 2022 at 7:37 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>
>> On 26.01.22 11:27, Alice Ferrazzi wrote:
>>> On Wed, Jan 26, 2022 at 6:37 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>>>
>>>> On 20.01.22 13:54, Alice Ferrazzi wrote:
>>>>> This patch series add a new image with settings for
>>>>> KernelCI.
>>>>>
>>>>> This new image is called cip-core-image-kernelci and is
>>>>> based on isar-cip-core general image with some changes
>>>>> needed for make it work with KernelCI and taken from
>>>>> KernelCI image base settings.
>>>>> The cip-core-image-kernelci images are built by GitlabCI
>>>>> and uploaded to KernelCI production fileserver.
>>>>> https://storage.kernelci.org/images/rootfs/cip/
>>>>>
>>>>> These patches are already integrated in the
>>>>> isar-cip-core:alicef/kernelci_master_refactor repository branch
>>>>> and are generating the images that are currently
>>>>> used by KernelCI.
>>>>>
>>>>> cip-core-image-kernelci as been tested and are
>>>>> already used in KernelCI production with good results.
>>>>> https://linux.kernelci.org/test/job/stable-rc/branch/queue%2F5.14/kernel/v5.14.17-9-g9f7eecaa70b3/plan/baseline-cip-nfs/
>>>>>
>>>>> v2:
>>>>> * drop trailing semicolon from gitlab-ci.yml
>>>>>
>>>>
>>>> Didn't you also want to de-duplicate the customization recipe, build
>>>> kernelci on top of the common one?
>>>>
>>> from my understanding we decided to don't go in that way,
>>> because changes in the common part could break the more generics
>>> KernelCI settings.
>>>
>>
>> Fork where there is relevant different, use the same one where there is
>> not. Duplicating files/99-silent-printk.conf and files/ethernet is
>> surely pointless. The latter will actually cause unneeded maintenance
>> efforts when adding new boards.
>>
>> Please also avoid needless diffs between the recipes so that they can be
>> re-aligned easily when boards are added.
>>
>> And I still think we will eventually miss the wireless setting from the
>> generic customization packages in kernelci, ie. the only real difference
>> in those recipes now.
> 
> Is not as we are adding dmesg.sh script (please check the next commit).
> KernelCI have a different environment than CIP current gitlab testing.
> 
> That's why the last time we decided to merge as is.
> https://lists.cip-project.org/g/cip-dev/topic/88235357#7341
> https://lists.cip-project.org/g/cip-dev/message/7337
> https://lists.cip-project.org/g/cip-dev/message/7340
> 

I misinterpreted this.

Then I'll merge and clean up on top. Duplications should be avoided 
where possible, and it is perfectly possible here, to a certain degree.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [isar-cip-core][PATCH v2 2/4] Add dmesg filter needed for lava test result

[Jan Kiszka]

On 20.01.22 13:54, Alice Ferrazzi wrote:
> KernelCI is using a dmesg filter script for checking dmesg
> logs result with lava.
> Adding the script to the cip-core-image-kernelci.
> 
> Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
> ---
>   .../kernelci-customizations/files/dmesg.sh    | 23 +++++++++++++++++++
>   .../kernelci-customizations.bb                |  1 +
>   2 files changed, 24 insertions(+)
>   create mode 100644 recipes-core/kernelci-customizations/files/dmesg.sh
> 
> diff --git a/recipes-core/kernelci-customizations/files/dmesg.sh b/recipes-core/kernelci-customizations/files/dmesg.sh
> new file mode 100644
> index 0000000..3b096e1
> --- /dev/null
> +++ b/recipes-core/kernelci-customizations/files/dmesg.sh
> @@ -0,0 +1,23 @@
> +#!/bin/sh
> +
> +set -e
> +
> +if [ "$KERNELCI_LAVA" = "y" ]; then
> +    alias test-result='lava-test-case'
> +else
> +    alias test-result='echo'
> +fi
> +
> +for level in crit alert emerg; do
> +    dmesg --level=$level --notime -x -k > dmesg.$level
> +    test -s dmesg.$level && res=fail || res=pass
> +    count=$(cat dmesg.$level | wc -l)
> +    cat dmesg.$level
> +    test-result \
> +        $level \
> +        --result $res \
> +        --measurement $count \
> +        --units lines
> +done
> +
> +exit 0
> diff --git a/recipes-core/kernelci-customizations/kernelci-customizations.bb b/recipes-core/kernelci-customizations/kernelci-customizations.bb
> index dca0891..c3f677c 100644
> --- a/recipes-core/kernelci-customizations/kernelci-customizations.bb
> +++ b/recipes-core/kernelci-customizations/kernelci-customizations.bb
> @@ -18,6 +18,7 @@ DESCRIPTION = "CIP Core image demo & customizations"
>   SRC_URI = " \
>       file://postinst \
>       file://ethernet \
> +    file://dmesg.sh \
>       file://99-silent-printk.conf"
>   
>   DEPENDS += "sshd-regen-keys"

This is a nop, you forgot to install it.

Where should that script actually end up?

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [isar-cip-core][PATCH v2 2/4] Add dmesg filter needed for lava test result

[Alice Ferrazzi]

On Thu, Jan 27, 2022 at 8:19 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>
> On 20.01.22 13:54, Alice Ferrazzi wrote:
> > KernelCI is using a dmesg filter script for checking dmesg
> > logs result with lava.
> > Adding the script to the cip-core-image-kernelci.
> >
> > Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
> > ---
> >   .../kernelci-customizations/files/dmesg.sh    | 23 +++++++++++++++++++
> >   .../kernelci-customizations.bb                |  1 +
> >   2 files changed, 24 insertions(+)
> >   create mode 100644 recipes-core/kernelci-customizations/files/dmesg.sh
> >
> > diff --git a/recipes-core/kernelci-customizations/files/dmesg.sh b/recipes-core/kernelci-customizations/files/dmesg.sh
> > new file mode 100644
> > index 0000000..3b096e1
> > --- /dev/null
> > +++ b/recipes-core/kernelci-customizations/files/dmesg.sh
> > @@ -0,0 +1,23 @@
> > +#!/bin/sh
> > +
> > +set -e
> > +
> > +if [ "$KERNELCI_LAVA" = "y" ]; then
> > +    alias test-result='lava-test-case'
> > +else
> > +    alias test-result='echo'
> > +fi
> > +
> > +for level in crit alert emerg; do
> > +    dmesg --level=$level --notime -x -k > dmesg.$level
> > +    test -s dmesg.$level && res=fail || res=pass
> > +    count=$(cat dmesg.$level | wc -l)
> > +    cat dmesg.$level
> > +    test-result \
> > +        $level \
> > +        --result $res \
> > +        --measurement $count \
> > +        --units lines
> > +done
> > +
> > +exit 0
> > diff --git a/recipes-core/kernelci-customizations/kernelci-customizations.bb b/recipes-core/kernelci-customizations/kernelci-customizations.bb
> > index dca0891..c3f677c 100644
> > --- a/recipes-core/kernelci-customizations/kernelci-customizations.bb
> > +++ b/recipes-core/kernelci-customizations/kernelci-customizations.bb
> > @@ -18,6 +18,7 @@ DESCRIPTION = "CIP Core image demo & customizations"
> >   SRC_URI = " \
> >       file://postinst \
> >       file://ethernet \
> > +    file://dmesg.sh \
> >       file://99-silent-printk.conf"
> >
> >   DEPENDS += "sshd-regen-keys"
>
> This is a nop, you forgot to install it.
>
> Where should that script actually end up?

you are right.
The installation script got lost in the rebase.
sending a patch v3

--
======================================
Cybertrust Japan Co.,Ltd.
Alice Ferrazzi
alice.ferrazzi@miraclelinux.com
======================================

Re: [isar-cip-core][PATCH v2 2/4] Add dmesg filter needed for lava test result

[Jan Kiszka]

On 27.01.22 14:09, Alice Ferrazzi wrote:
> On Thu, Jan 27, 2022 at 8:19 PM Jan Kiszka <jan.kiszka@siemens.com> wrote:
>>
>> On 20.01.22 13:54, Alice Ferrazzi wrote:
>>> KernelCI is using a dmesg filter script for checking dmesg
>>> logs result with lava.
>>> Adding the script to the cip-core-image-kernelci.
>>>
>>> Signed-off-by: Alice Ferrazzi <alice.ferrazzi@miraclelinux.com>
>>> ---
>>>    .../kernelci-customizations/files/dmesg.sh    | 23 +++++++++++++++++++
>>>    .../kernelci-customizations.bb                |  1 +
>>>    2 files changed, 24 insertions(+)
>>>    create mode 100644 recipes-core/kernelci-customizations/files/dmesg.sh
>>>
>>> diff --git a/recipes-core/kernelci-customizations/files/dmesg.sh b/recipes-core/kernelci-customizations/files/dmesg.sh
>>> new file mode 100644
>>> index 0000000..3b096e1
>>> --- /dev/null
>>> +++ b/recipes-core/kernelci-customizations/files/dmesg.sh
>>> @@ -0,0 +1,23 @@
>>> +#!/bin/sh
>>> +
>>> +set -e
>>> +
>>> +if [ "$KERNELCI_LAVA" = "y" ]; then
>>> +    alias test-result='lava-test-case'
>>> +else
>>> +    alias test-result='echo'
>>> +fi
>>> +
>>> +for level in crit alert emerg; do
>>> +    dmesg --level=$level --notime -x -k > dmesg.$level
>>> +    test -s dmesg.$level && res=fail || res=pass
>>> +    count=$(cat dmesg.$level | wc -l)
>>> +    cat dmesg.$level
>>> +    test-result \
>>> +        $level \
>>> +        --result $res \
>>> +        --measurement $count \
>>> +        --units lines
>>> +done
>>> +
>>> +exit 0
>>> diff --git a/recipes-core/kernelci-customizations/kernelci-customizations.bb b/recipes-core/kernelci-customizations/kernelci-customizations.bb
>>> index dca0891..c3f677c 100644
>>> --- a/recipes-core/kernelci-customizations/kernelci-customizations.bb
>>> +++ b/recipes-core/kernelci-customizations/kernelci-customizations.bb
>>> @@ -18,6 +18,7 @@ DESCRIPTION = "CIP Core image demo & customizations"
>>>    SRC_URI = " \
>>>        file://postinst \
>>>        file://ethernet \
>>> +    file://dmesg.sh \
>>>        file://99-silent-printk.conf"
>>>
>>>    DEPENDS += "sshd-regen-keys"
>>
>> This is a nop, you forgot to install it.
>>
>> Where should that script actually end up?
> 
> you are right.
> The installation script got lost in the rebase.
> sending a patch v3
> 

Found the missing bits:

   install -v -d ${D}/opt/kernelci
   install -v -m 744 ${WORKDIR}/dmesg.sh ${D}/opt/kernelci/

I can add them to your patch if you prefer.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux