* [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
@ 2020-08-20 8:24 Nobuhiro Iwamatsu
2020-08-20 8:24 ` [cip-dev] [isar-cip-core][PATCH 2/2] ci: Add support for test image Nobuhiro Iwamatsu
2020-08-21 15:04 ` [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Jan Kiszka
0 siblings, 2 replies; 9+ messages in thread
From: Nobuhiro Iwamatsu @ 2020-08-20 8:24 UTC (permalink / raw)
To: cip-dev; +Cc: jan.kiszka, Nobuhiro Iwamatsu
[-- Attachment #1: Type: text/plain, Size: 3873 bytes --]
Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
.gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
scripts/deploy-cip-core.sh | 8 +++--
2 files changed, 60 insertions(+), 18 deletions(-)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 3fe7af2..e23345f 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -2,10 +2,17 @@ image: kasproject/kas-isar:1.1
variables:
GIT_STRATEGY: clone
+ release: buster
+ extention: base
+ use_rt: enable
+ targz: enable
+ dtb: none
-all:
- stage: build
- script:
+stages:
+ - build
+
+default:
+ before_script:
- export http_proxy=$HTTP_PROXY
- export https_proxy=$HTTPS_PROXY
- export ftp_proxy=$FTP_PROXY
@@ -13,20 +20,51 @@ all:
- export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
- export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
- - kas build kas-cip.yml:kas/board/simatic-ipc227e.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster simatic-ipc227e
-
+.build_base:
+ stage: build
+ variables:
+ base_yaml: "kas-cip.yml:kas/board/${target}.yml"
+ script:
- sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/bbb.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster bbb am335x-boneblack.dtb
+ - if [ "${use_rt}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/rt.yml"; fi;
+ - if [ "${extention}" != "base" ]; then base_yaml="${base_yaml}:kas/opt/${extention}.yml"; fi;
+ - if [ "${targz}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/targz-img.yml"; fi;
+ - kas build ${base_yaml}
+ - scripts/deploy-cip-core.sh ${release} ${target} ${extention} ${dtb}
- - sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/iwg20m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster iwg20m r8a7743-iwg20d-q7-dbcm-ca.dtb
+# base image
+build:simatic-ipc227e-base:
+ extends:
+ - .build_base
+ variables:
+ target: simatic-ipc227e
- - sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/rzg2m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
- - scripts/deploy-cip-core.sh buster hihope-rzg2m renesas/r8a774a1-hihope-rzg2m-ex.dtb
+build:bbb-base:
+ extends:
+ - .build_base
+ variables:
+ target: bbb
+ dtb: am335x-boneblack.dtb
- - sudo rm -rf build/tmp
- - kas build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/security.yml
+build:iwg20m-base:
+ extends:
+ - .build_base
+ variables:
+ target: iwg20m
+ dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
+
+build:hihope-rzg2m-base:
+ extends:
+ - .build_base
+ variables:
+ target: rzg2m
+ dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
+
+build:qemu-amd64-base:
+ extends:
+ - .build_base
+ variables:
+ target: qemu-amd64
+ extention: security
+ use_rt: disable
+ targz: disable
diff --git a/scripts/deploy-cip-core.sh b/scripts/deploy-cip-core.sh
index 4c8d4c9..5b7eab9 100755
--- a/scripts/deploy-cip-core.sh
+++ b/scripts/deploy-cip-core.sh
@@ -16,9 +16,13 @@ fi
RELEASE=$1
TARGET=$2
-DTB=$3
+EXTENSION=$3
+DTB=$4
BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET
+if [ "${EXTENSION}" != "base" ] ; then
+ BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET-$EXTENSION
+fi
echo "Compressing cip-core-image-cip-core-$RELEASE-$TARGET.wic.img..."
xz -9 -k $BASE_PATH.wic.img
@@ -38,6 +42,6 @@ fi
aws s3 cp --no-progress $KERNEL_IMAGE s3://download.cip-project.org/cip-core/$TARGET/
aws s3 cp --no-progress $BASE_PATH-initrd.img s3://download.cip-project.org/cip-core/$TARGET/
-if [ -n "$DTB" ]; then
+if [ "$DTB" != "none" ]; then
aws s3 cp --no-progress build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB s3://download.cip-project.org/cip-core/$TARGET/
fi
--
2.27.0
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5158): https://lists.cip-project.org/g/cip-dev/message/5158
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [cip-dev] [isar-cip-core][PATCH 2/2] ci: Add support for test image
2020-08-20 8:24 [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Nobuhiro Iwamatsu
@ 2020-08-20 8:24 ` Nobuhiro Iwamatsu
2020-08-21 5:38 ` Jan Kiszka
2020-08-21 15:04 ` [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Jan Kiszka
1 sibling, 1 reply; 9+ messages in thread
From: Nobuhiro Iwamatsu @ 2020-08-20 8:24 UTC (permalink / raw)
To: cip-dev; +Cc: jan.kiszka, Nobuhiro Iwamatsu
[-- Attachment #1: Type: text/plain, Size: 973 bytes --]
Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
---
.gitlab-ci.yml | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index e23345f..802dbcd 100644
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -68,3 +68,35 @@ build:qemu-amd64-base:
extention: security
use_rt: disable
targz: disable
+
+# test
+build:simatic-ipc227e-test:
+ extends:
+ - .build_base
+ variables:
+ target: simatic-ipc227e
+ extention: test
+
+build:bbb-test:
+ extends:
+ - .build_base
+ variables:
+ target: bbb
+ extention: test
+ dtb: am335x-boneblack.dtb
+
+build:iwg20m-test:
+ extends:
+ - .build_base
+ variables:
+ target: iwg20m
+ extention: test
+ dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
+
+build:hihope-rzg2m-test:
+ extends:
+ - .build_base
+ variables:
+ target: rzg2m
+ extention: test
+ dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
--
2.27.0
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5159): https://lists.cip-project.org/g/cip-dev/message/5159
Mute This Topic: https://lists.cip-project.org/mt/76304033/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 2/2] ci: Add support for test image
2020-08-20 8:24 ` [cip-dev] [isar-cip-core][PATCH 2/2] ci: Add support for test image Nobuhiro Iwamatsu
@ 2020-08-21 5:38 ` Jan Kiszka
0 siblings, 0 replies; 9+ messages in thread
From: Jan Kiszka @ 2020-08-21 5:38 UTC (permalink / raw)
To: Nobuhiro Iwamatsu, cip-dev
[-- Attachment #1: Type: text/plain, Size: 1234 bytes --]
On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> ---
> .gitlab-ci.yml | 32 ++++++++++++++++++++++++++++++++
> 1 file changed, 32 insertions(+)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index e23345f..802dbcd 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -68,3 +68,35 @@ build:qemu-amd64-base:
> extention: security
> use_rt: disable
> targz: disable
> +
> +# test
> +build:simatic-ipc227e-test:
> + extends:
> + - .build_base
> + variables:
> + target: simatic-ipc227e
> + extention: test
> +
> +build:bbb-test:
> + extends:
> + - .build_base
> + variables:
> + target: bbb
> + extention: test
> + dtb: am335x-boneblack.dtb
> +
> +build:iwg20m-test:
> + extends:
> + - .build_base
> + variables:
> + target: iwg20m
> + extention: test
> + dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
> +
> +build:hihope-rzg2m-test:
> + extends:
> + - .build_base
> + variables:
> + target: rzg2m
> + extention: test
> + dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
>
Thanks, both applied to next.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5167): https://lists.cip-project.org/g/cip-dev/message/5167
Mute This Topic: https://lists.cip-project.org/mt/76304033/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
2020-08-20 8:24 [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Nobuhiro Iwamatsu
2020-08-20 8:24 ` [cip-dev] [isar-cip-core][PATCH 2/2] ci: Add support for test image Nobuhiro Iwamatsu
@ 2020-08-21 15:04 ` Jan Kiszka
2020-08-21 16:44 ` Jan Kiszka
1 sibling, 1 reply; 9+ messages in thread
From: Jan Kiszka @ 2020-08-21 15:04 UTC (permalink / raw)
To: Nobuhiro Iwamatsu, cip-dev
[-- Attachment #1: Type: text/plain, Size: 4602 bytes --]
On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> ---
> .gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
> scripts/deploy-cip-core.sh | 8 +++--
> 2 files changed, 60 insertions(+), 18 deletions(-)
>
> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
> index 3fe7af2..e23345f 100644
> --- a/.gitlab-ci.yml
> +++ b/.gitlab-ci.yml
> @@ -2,10 +2,17 @@ image: kasproject/kas-isar:1.1
>
> variables:
> GIT_STRATEGY: clone
> + release: buster
> + extention: base
> + use_rt: enable
> + targz: enable
> + dtb: none
>
> -all:
> - stage: build
> - script:
> +stages:
> + - build
> +
> +default:
> + before_script:
> - export http_proxy=$HTTP_PROXY
> - export https_proxy=$HTTPS_PROXY
> - export ftp_proxy=$FTP_PROXY
> @@ -13,20 +20,51 @@ all:
> - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
> - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
>
> - - kas build kas-cip.yml:kas/board/simatic-ipc227e.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
> - - scripts/deploy-cip-core.sh buster simatic-ipc227e
> -
> +.build_base:
> + stage: build
> + variables:
> + base_yaml: "kas-cip.yml:kas/board/${target}.yml"
> + script:
> - sudo rm -rf build/tmp
> - - kas build kas-cip.yml:kas/board/bbb.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
> - - scripts/deploy-cip-core.sh buster bbb am335x-boneblack.dtb
> + - if [ "${use_rt}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/rt.yml"; fi;
> + - if [ "${extention}" != "base" ]; then base_yaml="${base_yaml}:kas/opt/${extention}.yml"; fi;
> + - if [ "${targz}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/targz-img.yml"; fi;
> + - kas build ${base_yaml}
> + - scripts/deploy-cip-core.sh ${release} ${target} ${extention} ${dtb}
>
> - - sudo rm -rf build/tmp
> - - kas build kas-cip.yml:kas/board/iwg20m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
> - - scripts/deploy-cip-core.sh buster iwg20m r8a7743-iwg20d-q7-dbcm-ca.dtb
> +# base image
> +build:simatic-ipc227e-base:
> + extends:
> + - .build_base
> + variables:
> + target: simatic-ipc227e
>
> - - sudo rm -rf build/tmp
> - - kas build kas-cip.yml:kas/board/rzg2m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
> - - scripts/deploy-cip-core.sh buster hihope-rzg2m renesas/r8a774a1-hihope-rzg2m-ex.dtb
> +build:bbb-base:
> + extends:
> + - .build_base
> + variables:
> + target: bbb
> + dtb: am335x-boneblack.dtb
>
> - - sudo rm -rf build/tmp
> - - kas build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/security.yml
> +build:iwg20m-base:
> + extends:
> + - .build_base
> + variables:
> + target: iwg20m
> + dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
> +
> +build:hihope-rzg2m-base:
> + extends:
> + - .build_base
> + variables:
> + target: rzg2m
> + dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
> +
> +build:qemu-amd64-base:
> + extends:
> + - .build_base
> + variables:
> + target: qemu-amd64
> + extention: security
> + use_rt: disable
> + targz: disable
> diff --git a/scripts/deploy-cip-core.sh b/scripts/deploy-cip-core.sh
> index 4c8d4c9..5b7eab9 100755
> --- a/scripts/deploy-cip-core.sh
> +++ b/scripts/deploy-cip-core.sh
> @@ -16,9 +16,13 @@ fi
>
> RELEASE=$1
> TARGET=$2
> -DTB=$3
> +EXTENSION=$3
> +DTB=$4
>
> BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET
> +if [ "${EXTENSION}" != "base" ] ; then
> + BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET-$EXTENSION
> +fi
>
> echo "Compressing cip-core-image-cip-core-$RELEASE-$TARGET.wic.img..."
> xz -9 -k $BASE_PATH.wic.img
> @@ -38,6 +42,6 @@ fi
> aws s3 cp --no-progress $KERNEL_IMAGE s3://download.cip-project.org/cip-core/$TARGET/
> aws s3 cp --no-progress $BASE_PATH-initrd.img s3://download.cip-project.org/cip-core/$TARGET/
>
> -if [ -n "$DTB" ]; then
> +if [ "$DTB" != "none" ]; then
> aws s3 cp --no-progress build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB s3://download.cip-project.org/cip-core/$TARGET/
> fi
>
Unfortunately, this scale out to multiple jobs seem to cause download
issues, see e.g.
- https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
- https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649
First thought it was a sporadic hic-up, but it reoccurs, now with
master. Is kernel.org throttling us here?
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5223): https://lists.cip-project.org/g/cip-dev/message/5223
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
2020-08-21 15:04 ` [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Jan Kiszka
@ 2020-08-21 16:44 ` Jan Kiszka
2020-08-25 5:17 ` Jan Kiszka
2020-08-25 5:33 ` Nobuhiro Iwamatsu
0 siblings, 2 replies; 9+ messages in thread
From: Jan Kiszka @ 2020-08-21 16:44 UTC (permalink / raw)
To: Nobuhiro Iwamatsu, cip-dev
[-- Attachment #1: Type: text/plain, Size: 4932 bytes --]
On 21.08.20 17:04, Jan Kiszka wrote:
> On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
>> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>> ---
>> .gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
>> scripts/deploy-cip-core.sh | 8 +++--
>> 2 files changed, 60 insertions(+), 18 deletions(-)
>>
>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>> index 3fe7af2..e23345f 100644
>> --- a/.gitlab-ci.yml
>> +++ b/.gitlab-ci.yml
>> @@ -2,10 +2,17 @@ image: kasproject/kas-isar:1.1
>>
>> variables:
>> GIT_STRATEGY: clone
>> + release: buster
>> + extention: base
>> + use_rt: enable
>> + targz: enable
>> + dtb: none
>>
>> -all:
>> - stage: build
>> - script:
>> +stages:
>> + - build
>> +
>> +default:
>> + before_script:
>> - export http_proxy=$HTTP_PROXY
>> - export https_proxy=$HTTPS_PROXY
>> - export ftp_proxy=$FTP_PROXY
>> @@ -13,20 +20,51 @@ all:
>> - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
>> - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
>>
>> - - kas build kas-cip.yml:kas/board/simatic-ipc227e.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>> - - scripts/deploy-cip-core.sh buster simatic-ipc227e
>> -
>> +.build_base:
>> + stage: build
>> + variables:
>> + base_yaml: "kas-cip.yml:kas/board/${target}.yml"
>> + script:
>> - sudo rm -rf build/tmp
>> - - kas build kas-cip.yml:kas/board/bbb.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>> - - scripts/deploy-cip-core.sh buster bbb am335x-boneblack.dtb
>> + - if [ "${use_rt}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/rt.yml"; fi;
>> + - if [ "${extention}" != "base" ]; then base_yaml="${base_yaml}:kas/opt/${extention}.yml"; fi;
>> + - if [ "${targz}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/targz-img.yml"; fi;
>> + - kas build ${base_yaml}
>> + - scripts/deploy-cip-core.sh ${release} ${target} ${extention} ${dtb}
>>
>> - - sudo rm -rf build/tmp
>> - - kas build kas-cip.yml:kas/board/iwg20m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>> - - scripts/deploy-cip-core.sh buster iwg20m r8a7743-iwg20d-q7-dbcm-ca.dtb
>> +# base image
>> +build:simatic-ipc227e-base:
>> + extends:
>> + - .build_base
>> + variables:
>> + target: simatic-ipc227e
>>
>> - - sudo rm -rf build/tmp
>> - - kas build kas-cip.yml:kas/board/rzg2m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>> - - scripts/deploy-cip-core.sh buster hihope-rzg2m renesas/r8a774a1-hihope-rzg2m-ex.dtb
>> +build:bbb-base:
>> + extends:
>> + - .build_base
>> + variables:
>> + target: bbb
>> + dtb: am335x-boneblack.dtb
>>
>> - - sudo rm -rf build/tmp
>> - - kas build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/security.yml
>> +build:iwg20m-base:
>> + extends:
>> + - .build_base
>> + variables:
>> + target: iwg20m
>> + dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
>> +
>> +build:hihope-rzg2m-base:
>> + extends:
>> + - .build_base
>> + variables:
>> + target: rzg2m
>> + dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
>> +
>> +build:qemu-amd64-base:
>> + extends:
>> + - .build_base
>> + variables:
>> + target: qemu-amd64
>> + extention: security
>> + use_rt: disable
>> + targz: disable
>> diff --git a/scripts/deploy-cip-core.sh b/scripts/deploy-cip-core.sh
>> index 4c8d4c9..5b7eab9 100755
>> --- a/scripts/deploy-cip-core.sh
>> +++ b/scripts/deploy-cip-core.sh
>> @@ -16,9 +16,13 @@ fi
>>
>> RELEASE=$1
>> TARGET=$2
>> -DTB=$3
>> +EXTENSION=$3
>> +DTB=$4
>>
>> BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET
>> +if [ "${EXTENSION}" != "base" ] ; then
>> + BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET-$EXTENSION
>> +fi
>>
>> echo "Compressing cip-core-image-cip-core-$RELEASE-$TARGET.wic.img..."
>> xz -9 -k $BASE_PATH.wic.img
>> @@ -38,6 +42,6 @@ fi
>> aws s3 cp --no-progress $KERNEL_IMAGE s3://download.cip-project.org/cip-core/$TARGET/
>> aws s3 cp --no-progress $BASE_PATH-initrd.img s3://download.cip-project.org/cip-core/$TARGET/
>>
>> -if [ -n "$DTB" ]; then
>> +if [ "$DTB" != "none" ]; then
>> aws s3 cp --no-progress build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB s3://download.cip-project.org/cip-core/$TARGET/
>> fi
>>
>
> Unfortunately, this scale out to multiple jobs seem to cause download
> issues, see e.g.
>
> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649
>
> First thought it was a sporadic hic-up, but it reoccurs, now with
> master. Is kernel.org throttling us here?
>
And I'm afraid there is more broken, namely in the deployment that is
only triggered over master. Please have a look at the failing jobs.
Thanks,
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5226): https://lists.cip-project.org/g/cip-dev/message/5226
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
2020-08-21 16:44 ` Jan Kiszka
@ 2020-08-25 5:17 ` Jan Kiszka
2020-08-25 5:33 ` Nobuhiro Iwamatsu
1 sibling, 0 replies; 9+ messages in thread
From: Jan Kiszka @ 2020-08-25 5:17 UTC (permalink / raw)
To: Nobuhiro Iwamatsu, cip-dev
[-- Attachment #1: Type: text/plain, Size: 5149 bytes --]
On 21.08.20 18:44, Jan Kiszka wrote:
> On 21.08.20 17:04, Jan Kiszka wrote:
>> On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
>>> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>>> ---
>>> .gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
>>> scripts/deploy-cip-core.sh | 8 +++--
>>> 2 files changed, 60 insertions(+), 18 deletions(-)
>>>
>>> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
>>> index 3fe7af2..e23345f 100644
>>> --- a/.gitlab-ci.yml
>>> +++ b/.gitlab-ci.yml
>>> @@ -2,10 +2,17 @@ image: kasproject/kas-isar:1.1
>>>
>>> variables:
>>> GIT_STRATEGY: clone
>>> + release: buster
>>> + extention: base
>>> + use_rt: enable
>>> + targz: enable
>>> + dtb: none
>>>
>>> -all:
>>> - stage: build
>>> - script:
>>> +stages:
>>> + - build
>>> +
>>> +default:
>>> + before_script:
>>> - export http_proxy=$HTTP_PROXY
>>> - export https_proxy=$HTTPS_PROXY
>>> - export ftp_proxy=$FTP_PROXY
>>> @@ -13,20 +20,51 @@ all:
>>> - export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
>>> - export AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
>>>
>>> - - kas build kas-cip.yml:kas/board/simatic-ipc227e.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>>> - - scripts/deploy-cip-core.sh buster simatic-ipc227e
>>> -
>>> +.build_base:
>>> + stage: build
>>> + variables:
>>> + base_yaml: "kas-cip.yml:kas/board/${target}.yml"
>>> + script:
>>> - sudo rm -rf build/tmp
>>> - - kas build kas-cip.yml:kas/board/bbb.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>>> - - scripts/deploy-cip-core.sh buster bbb am335x-boneblack.dtb
>>> + - if [ "${use_rt}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/rt.yml"; fi;
>>> + - if [ "${extention}" != "base" ]; then base_yaml="${base_yaml}:kas/opt/${extention}.yml"; fi;
>>> + - if [ "${targz}" = "enable" ]; then base_yaml="${base_yaml}:kas/opt/targz-img.yml"; fi;
>>> + - kas build ${base_yaml}
>>> + - scripts/deploy-cip-core.sh ${release} ${target} ${extention} ${dtb}
>>>
>>> - - sudo rm -rf build/tmp
>>> - - kas build kas-cip.yml:kas/board/iwg20m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>>> - - scripts/deploy-cip-core.sh buster iwg20m r8a7743-iwg20d-q7-dbcm-ca.dtb
>>> +# base image
>>> +build:simatic-ipc227e-base:
>>> + extends:
>>> + - .build_base
>>> + variables:
>>> + target: simatic-ipc227e
>>>
>>> - - sudo rm -rf build/tmp
>>> - - kas build kas-cip.yml:kas/board/rzg2m.yml:kas/opt/rt.yml:kas/opt/targz-img.yml
>>> - - scripts/deploy-cip-core.sh buster hihope-rzg2m renesas/r8a774a1-hihope-rzg2m-ex.dtb
>>> +build:bbb-base:
>>> + extends:
>>> + - .build_base
>>> + variables:
>>> + target: bbb
>>> + dtb: am335x-boneblack.dtb
>>>
>>> - - sudo rm -rf build/tmp
>>> - - kas build kas-cip.yml:kas/board/qemu-amd64.yml:kas/opt/security.yml
>>> +build:iwg20m-base:
>>> + extends:
>>> + - .build_base
>>> + variables:
>>> + target: iwg20m
>>> + dtb: r8a7743-iwg20d-q7-dbcm-ca.dtb
>>> +
>>> +build:hihope-rzg2m-base:
>>> + extends:
>>> + - .build_base
>>> + variables:
>>> + target: rzg2m
>>> + dtb: renesas/r8a774a1-hihope-rzg2m-ex.dtb
>>> +
>>> +build:qemu-amd64-base:
>>> + extends:
>>> + - .build_base
>>> + variables:
>>> + target: qemu-amd64
>>> + extention: security
>>> + use_rt: disable
>>> + targz: disable
>>> diff --git a/scripts/deploy-cip-core.sh b/scripts/deploy-cip-core.sh
>>> index 4c8d4c9..5b7eab9 100755
>>> --- a/scripts/deploy-cip-core.sh
>>> +++ b/scripts/deploy-cip-core.sh
>>> @@ -16,9 +16,13 @@ fi
>>>
>>> RELEASE=$1
>>> TARGET=$2
>>> -DTB=$3
>>> +EXTENSION=$3
>>> +DTB=$4
>>>
>>> BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET
>>> +if [ "${EXTENSION}" != "base" ] ; then
>>> + BASE_PATH=build/tmp/deploy/images/$TARGET/cip-core-image-cip-core-$RELEASE-$TARGET-$EXTENSION
>>> +fi
>>>
>>> echo "Compressing cip-core-image-cip-core-$RELEASE-$TARGET.wic.img..."
>>> xz -9 -k $BASE_PATH.wic.img
>>> @@ -38,6 +42,6 @@ fi
>>> aws s3 cp --no-progress $KERNEL_IMAGE s3://download.cip-project.org/cip-core/$TARGET/
>>> aws s3 cp --no-progress $BASE_PATH-initrd.img s3://download.cip-project.org/cip-core/$TARGET/
>>>
>>> -if [ -n "$DTB" ]; then
>>> +if [ "$DTB" != "none" ]; then
>>> aws s3 cp --no-progress build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB s3://download.cip-project.org/cip-core/$TARGET/
>>> fi
>>>
>>
>> Unfortunately, this scale out to multiple jobs seem to cause download
>> issues, see e.g.
>>
>> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
>> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649
>>
>> First thought it was a sporadic hic-up, but it reoccurs, now with
>> master. Is kernel.org throttling us here?
>>
>
> And I'm afraid there is more broken, namely in the deployment that is
> only triggered over master. Please have a look at the failing jobs.
>
Ping - or should I revert those two for now?
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5237): https://lists.cip-project.org/g/cip-dev/message/5237
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
2020-08-21 16:44 ` Jan Kiszka
2020-08-25 5:17 ` Jan Kiszka
@ 2020-08-25 5:33 ` Nobuhiro Iwamatsu
2020-08-25 5:59 ` Jan Kiszka
1 sibling, 1 reply; 9+ messages in thread
From: Nobuhiro Iwamatsu @ 2020-08-25 5:33 UTC (permalink / raw)
To: jan.kiszka, cip-dev
[-- Attachment #1: Type: text/plain, Size: 1976 bytes --]
Hi,
> -----Original Message-----
> From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
> Sent: Saturday, August 22, 2020 1:45 AM
> To: iwamatsu nobuhiro(岩松 信洋 □SWC◯ACT) <nobuhiro1.iwamatsu@toshiba.co.jp>; cip-dev@lists.cip-project.org
> Subject: Re: [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
>
> On 21.08.20 17:04, Jan Kiszka wrote:
> > On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
> >> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >> ---
> >> .gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
> >> scripts/deploy-cip-core.sh | 8 +++--
> >> 2 files changed, 60 insertions(+), 18 deletions(-)
<snip>
> >> -if [ -n "$DTB" ]; then
> >> +if [ "$DTB" != "none" ]; then
> >> aws s3 cp --no-progress
> build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB
> s3://download.cip-project.org/cip-core/$TARGET/
> >> fi
> >>
> >
> > Unfortunately, this scale out to multiple jobs seem to cause download
> > issues, see e.g.
> >
> > - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
> > - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649
> >
> > First thought it was a sporadic hic-up, but it reoccurs, now with
> > master. Is kernel.org throttling us here?
> >
This may be a burden on kernel.org as snapshot reconstructs the image
on the server side. I'm not sure, when downloading images continuously,
access may be controlled by kernel.org.
>
> And I'm afraid there is more broken, namely in the deployment that is
> only triggered over master. Please have a look at the failing jobs.
I don't have no idea about this.
If we retry the test, it is successful and there may be other causes.
>
> Thanks,
> Jan
>
Best regards,
Nobuhiro
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5238): https://lists.cip-project.org/g/cip-dev/message/5238
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
2020-08-25 5:33 ` Nobuhiro Iwamatsu
@ 2020-08-25 5:59 ` Jan Kiszka
2020-08-26 1:18 ` Nobuhiro Iwamatsu
0 siblings, 1 reply; 9+ messages in thread
From: Jan Kiszka @ 2020-08-25 5:59 UTC (permalink / raw)
To: nobuhiro1.iwamatsu, cip-dev
[-- Attachment #1: Type: text/plain, Size: 2214 bytes --]
On 25.08.20 07:33, nobuhiro1.iwamatsu@toshiba.co.jp wrote:
> Hi,
>
>> -----Original Message-----
>> From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
>> Sent: Saturday, August 22, 2020 1:45 AM
>> To: iwamatsu nobuhiro(岩松 信洋 □SWC◯ACT) <nobuhiro1.iwamatsu@toshiba.co.jp>; cip-dev@lists.cip-project.org
>> Subject: Re: [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
>>
>> On 21.08.20 17:04, Jan Kiszka wrote:
>>> On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
>>>> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
>>>> ---
>>>> .gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
>>>> scripts/deploy-cip-core.sh | 8 +++--
>>>> 2 files changed, 60 insertions(+), 18 deletions(-)
>
> <snip>
>
>>>> -if [ -n "$DTB" ]; then
>>>> +if [ "$DTB" != "none" ]; then
>>>> aws s3 cp --no-progress
>> build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB
>> s3://download.cip-project.org/cip-core/$TARGET/
>>>> fi
>>>>
>>>
>>> Unfortunately, this scale out to multiple jobs seem to cause download
>>> issues, see e.g.
>>>
>>> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
>>> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649
>>>
>>> First thought it was a sporadic hic-up, but it reoccurs, now with
>>> master. Is kernel.org throttling us here?
>>>
>
> This may be a burden on kernel.org as snapshot reconstructs the image
> on the server side. I'm not sure, when downloading images continuously,
> access may be controlled by kernel.org.
>
So we need to avoid that for now, I guess. Back to single-job build, or
do we have an alternative download source?
>>
>> And I'm afraid there is more broken, namely in the deployment that is
>> only triggered over master. Please have a look at the failing jobs.
>
> I don't have no idea about this.
> If we retry the test, it is successful and there may be other causes.
>
The wic image names changed, and that broke compression and uploading.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5239): https://lists.cip-project.org/g/cip-dev/message/5239
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
2020-08-25 5:59 ` Jan Kiszka
@ 2020-08-26 1:18 ` Nobuhiro Iwamatsu
0 siblings, 0 replies; 9+ messages in thread
From: Nobuhiro Iwamatsu @ 2020-08-26 1:18 UTC (permalink / raw)
To: cip-dev
[-- Attachment #1: Type: text/plain, Size: 2773 bytes --]
Hi,
> -----Original Message-----
> From: cip-dev@lists.cip-project.org [mailto:cip-dev@lists.cip-project.org] On Behalf Of Jan Kiszka
> Sent: Tuesday, August 25, 2020 2:59 PM
> To: iwamatsu nobuhiro(岩松 信洋 □SWC◯ACT) <nobuhiro1.iwamatsu@toshiba.co.jp>; cip-dev@lists.cip-project.org
> Subject: Re: [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
>
> On 25.08.20 07:33, nobuhiro1.iwamatsu@toshiba.co.jp wrote:
> > Hi,
> >
> >> -----Original Message-----
> >> From: Jan Kiszka [mailto:jan.kiszka@siemens.com]
> >> Sent: Saturday, August 22, 2020 1:45 AM
> >> To: iwamatsu nobuhiro(岩松 信洋 □SWC◯ACT) <nobuhiro1.iwamatsu@toshiba.co.jp>;
> cip-dev@lists.cip-project.org
> >> Subject: Re: [isar-cip-core][PATCH 1/2] ci: Rewrite using extends
> >>
> >> On 21.08.20 17:04, Jan Kiszka wrote:
> >>> On 20.08.20 10:24, Nobuhiro Iwamatsu wrote:
> >>>> Signed-off-by: Nobuhiro Iwamatsu <nobuhiro1.iwamatsu@toshiba.co.jp>
> >>>> ---
> >>>> .gitlab-ci.yml | 70 +++++++++++++++++++++++++++++---------
> >>>> scripts/deploy-cip-core.sh | 8 +++--
> >>>> 2 files changed, 60 insertions(+), 18 deletions(-)
> >
> > <snip>
> >
> >>>> -if [ -n "$DTB" ]; then
> >>>> +if [ "$DTB" != "none" ]; then
> >>>> aws s3 cp --no-progress
> >> build/tmp/work/cip-core-*/linux-cip*/*/linux-cip-*/debian/linux-image-cip*/usr/lib/linux-image-*/$DTB
> >> s3://download.cip-project.org/cip-core/$TARGET/
> >>>> fi
> >>>>
> >>>
> >>> Unfortunately, this scale out to multiple jobs seem to cause download
> >>> issues, see e.g.
> >>>
> >>> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/697844276
> >>> - https://gitlab.com/cip-project/cip-core/isar-cip-core/-/jobs/696946649
> >>>
> >>> First thought it was a sporadic hic-up, but it reoccurs, now with
> >>> master. Is kernel.org throttling us here?
> >>>
> >
> > This may be a burden on kernel.org as snapshot reconstructs the image
> > on the server side. I'm not sure, when downloading images continuously,
> > access may be controlled by kernel.org.
> >
>
> So we need to avoid that for now, I guess. Back to single-job build, or
> do we have an alternative download source?
OK, because I don't know any other good download source, I will change to single job.
>
> >>
> >> And I'm afraid there is more broken, namely in the deployment that is
> >> only triggered over master. Please have a look at the failing jobs.
> >
> > I don't have no idea about this.
> > If we retry the test, it is successful and there may be other causes.
> >
>
> The wic image names changed, and that broke compression and uploading.
>
OK, I will fix this as well.
> Jan
>
Best regards,
Nobuhiro
[-- Attachment #2: Type: text/plain, Size: 419 bytes --]
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#5252): https://lists.cip-project.org/g/cip-dev/message/5252
Mute This Topic: https://lists.cip-project.org/mt/76304024/4520388
Group Owner: cip-dev+owner@lists.cip-project.org
Unsubscribe: https://lists.cip-project.org/g/cip-dev/leave/8129055/727948398/xyzzy [cip-dev@archiver.kernel.org]
-=-=-=-=-=-=-=-=-=-=-=-
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2020-08-26 1:18 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-08-20 8:24 [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Nobuhiro Iwamatsu
2020-08-20 8:24 ` [cip-dev] [isar-cip-core][PATCH 2/2] ci: Add support for test image Nobuhiro Iwamatsu
2020-08-21 5:38 ` Jan Kiszka
2020-08-21 15:04 ` [cip-dev] [isar-cip-core][PATCH 1/2] ci: Rewrite using extends Jan Kiszka
2020-08-21 16:44 ` Jan Kiszka
2020-08-25 5:17 ` Jan Kiszka
2020-08-25 5:33 ` Nobuhiro Iwamatsu
2020-08-25 5:59 ` Jan Kiszka
2020-08-26 1:18 ` Nobuhiro Iwamatsu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).