From: "Q. Gylstorff" <Quirin.Gylstorff@siemens.com>
To: xenomai@xenomai.org, jan.kiszka@siemens.com
Subject: [xenomai-images][PATCH v2 2/3] ci: Remove deploy with ssh
Date: Mon, 19 Jul 2021 18:29:48 +0200 [thread overview]
Message-ID: <20210719162949.4342-3-Quirin.Gylstorff@siemens.com> (raw)
In-Reply-To: <20210719162949.4342-1-Quirin.Gylstorff@siemens.com>
From: Quirin Gylstorff <quirin.gylstorff@siemens.com>
We will use AWS S3 deployment for LAVA Build Artifacts. Therefore
remove the Deployment via ssh as the LAVA Lab is now public
available.
Signed-off-by: Quirin Gylstorff <quirin.gylstorff@siemens.com>
---
ci/gitlab-ci-base.yml | 11 +++-----
scripts/deploy_for_testing.sh | 51 -----------------------------------
scripts/run-lava-tests.sh | 37 +++----------------------
tests/README.md | 17 +++---------
4 files changed, 10 insertions(+), 106 deletions(-)
delete mode 100755 scripts/deploy_for_testing.sh
diff --git a/ci/gitlab-ci-base.yml b/ci/gitlab-ci-base.yml
index 362a636..d80efa1 100644
--- a/ci/gitlab-ci-base.yml
+++ b/ci/gitlab-ci-base.yml
@@ -27,29 +27,24 @@ variables:
default:
image: ghcr.io/siemens/kas/kas-isar:2.4
-# add lavacli to the container and install the ssh keys
-# for the test infrastructure
-.add-lava-ssh-config:
+.add-proxy-config:
before_script:
- mkdir -p -m=700 ~/.ssh
- if [ -n "$https_proxy" ]; then
echo "ProxyCommand socat - PROXY:$(echo $https_proxy | sed 's|.*://\([^:]*\).*|\1|'):%h:%p,proxyport=$(echo $https_proxy | sed 's|.*:\([0-9]*\)$|\1|')" >> ~/.ssh/config;
chmod 600 ~/.ssh/config;
fi
- - echo "$LAVA_SSH_UPLOAD_KEY" | tr -d '\r' > ~/.ssh/lava_id_rsa && chmod 600 ~/.ssh/lava_id_rsa
- - echo "$LAVA_SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts && chmod 644 ~/.ssh/known_hosts
.build:
- extends: .add-lava-ssh-config
+ extends: .add-proxy-config
stage: build
script:
- echo "Building kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}"
- kas build kas.yml:board-${TARGET}.yml${XENOMAI_BUILD_OPTION}${LINUX_BUILD_OPTION}${BUILD_OPTIONS}
- - if [ -z "${USE_S3_BUCKET}" ]; then scripts/deploy_for_testing.sh ${TARGET}; fi
- if [ -n "${USE_S3_BUCKET}" ]; then scripts/deploy_to_aws.sh ${TARGET}; fi
.test:
- extends: .add-lava-ssh-config
+ extends: .add-proxy-config
stage: test
script:
- scripts/install-lavacli.sh
diff --git a/scripts/deploy_for_testing.sh b/scripts/deploy_for_testing.sh
deleted file mode 100755
index 3878e7d..0000000
--- a/scripts/deploy_for_testing.sh
+++ /dev/null
@@ -1,51 +0,0 @@
-#!/bin/sh
-
-#
-# Xenomai Real-Time System
-#
-# Copyright (c) Siemens AG, 2019-2020
-#
-# Authors:
-# Quirin Gylstorff <quirin.gylstorff@siemens.com>
-#
-# SPDX-License-Identifier: MIT
-#
-set -e
-target="$1"
-
-if [ -z "${target}" ]; then
- exit 1
-fi
-images_dir=build/tmp/deploy/images
-
-if [ -z "${LAVA_SSH_USER}" ] || [ -z "${LAVA_SSH_HOST}" ] || [ -z "${LAVA_SSH_PORT}" ]; then
- echo "Lava environment not available or incomplete - do not deploy"
- exit 0
-fi
-
-lava_ssh_destination="${LAVA_SSH_USER}@${LAVA_SSH_HOST}"
-
-isar_base_name="${ISAR_IMAGE}-${ISAR_DISTRIBUTION}-${target}"
-lava_deploy_dir="${LAVA_DEPLOY_DIR:-/var/lib/lava/artifacts}"
-deploy_dir="${lava_deploy_dir}/${CI_PIPELINE_ID}/${DEPLOY_DIR_EXTENSION}"
-lava_identity="-i ${LAVA_SSH_KEY_PATH:-~/.ssh/lava_id_rsa}"
-if [ -n "${CI_PIPELINE_ID}" ]; then
- ssh -p ${LAVA_SSH_PORT} ${lava_identity} ${lava_ssh_destination} 'install -d -m 755 "'${deploy_dir}'"'
-fi
-#KERNEL
-scp -P ${LAVA_SSH_PORT} ${lava_identity} ${images_dir}/${target}/${isar_base_name}-vmlinuz \
- ${lava_ssh_destination}:${deploy_dir}
-# INITRD
-scp -P ${LAVA_SSH_PORT} ${lava_identity} ${images_dir}/${target}/${isar_base_name}-initrd.img \
- ${lava_ssh_destination}:${deploy_dir}
-# ROOTFS
-if [ ! -e ${images_dir}/${target}/${isar_base_name}.*.gz ]; then
- gzip ${images_dir}/${target}/${isar_base_name}.*
-fi
-scp -P ${LAVA_SSH_PORT} ${lava_identity} ${images_dir}/${target}/${isar_base_name}.* \
- ${lava_ssh_destination}:${deploy_dir}
-# DTB
-dtb="${images_dir}/${target}/*.dtb"
-if [ -e ${dtb} ]; then
- scp -P ${LAVA_SSH_PORT} ${lava_identity} ${dtb} ${lava_ssh_destination}:${deploy_dir}
-fi
diff --git a/scripts/run-lava-tests.sh b/scripts/run-lava-tests.sh
index 13fac31..b1b51bb 100755
--- a/scripts/run-lava-tests.sh
+++ b/scripts/run-lava-tests.sh
@@ -15,36 +15,10 @@ if [ -z "${TARGET}" ]; then
exit 1
fi
-if [ -n "${LAVA_SSH_HOST}" ]; then
- echo "use SSH to connect to lava server"
- # private master with ssh forward access
- lava_master_port="${LAVA_MASTER_PORT:-28080}"
- lava_ssh_port="-p ${LAVA_SSH_PORT:-22}"
- lava_identity="-i ${LAVA_SSH_KEY_PATH:-~/.ssh/lava_id_rsa}"
- lava_ssh_destination="${LAVA_SSH_USER}@${LAVA_SSH_HOST}"
- # open connection for ssh port forwarding
- ssh -N ${lava_ssh_port} ${lava_identity} -o 'LocalForward localhost:'${lava_master_port}' localhost:80' ${lava_ssh_destination} &
- ssh_pid="$!"
- # wait for connection
- interval=1
- timeout=60
- until ss -tulw | grep -q "${lava_master_port}"
- do
- if [ ${timeout} -le 0 ]; then
- echo "could not open connection to LAVA Master"
- exit 1
- fi
- sleep ${interval}
- timeout=$(( timeout - interval ))
- done
- artifact_url="${LAVA_ARTIFACTS_URL:-http://localhost/artifacts}"
- lava_master_uri="${LAVA_MASTER_URL:-http://localhost}:${lava_master_port}"
-else
- # S3 artifacts
- artifact_url="${LAVA_ARTIFACTS_URL:-https://xenomai-images-artifacts.s3.eu-central-1.amazonaws.com/artifacts}"
- # public master
- lava_master_uri="${LAVA_MASTER_URL:-https://lava.xenomai.org}"
-fi
+# S3 artifacts
+artifact_url="${LAVA_ARTIFACTS_URL:-https://xenomai-images-artifacts.s3.eu-central-1.amazonaws.com/artifacts}"
+# public master
+lava_master_uri="${LAVA_MASTER_URL:-https://lava.xenomai.org}"
echo "connect to lava server: ${lava_master_uri}"
@@ -79,9 +53,6 @@ curl --silent "${url}" -o report.xml
# change return code to generate a error in gitlab-ci if a test is failed
number_of_fails="$(grep -o "errors=\"[0-9]*\"" report.xml | head -1 | cut -d\" -f2 )"
-if [ -n "${ssh_pid}" ];then
- kill "${ssh_pid}"
-fi
if [ "${number_of_fails}" -gt "0" ]; then
exit 1
fi
diff --git a/tests/README.md b/tests/README.md
index e0d50a9..3e17271 100644
--- a/tests/README.md
+++ b/tests/README.md
@@ -46,9 +46,9 @@ is used:
| Target 1 |
/--| beagle |
+-----------+ +---------+ +---------+ /----- | bone |
-| | ssh | LAVA | | LAVA | --- +----------+
+| | | LAVA | | LAVA | --- +----------+
| gitlab- | ------ | master |------ | Dis- | --
-| runner | tunnel | | | patcher | \ \--- +----------+
+| runner | | | | patcher | \ \--- +----------+
+-----------+ +---------+ +---------+ \- \--- | Target 2 |
\ \-| x86-64 |
\ | |
@@ -62,8 +62,7 @@ is used:
```
A test is deployed in the following steps:
1. gitlab-runner: builds the artifacts
-2. After the build is successful the build artifacts are deployed to the
- Lava master per scp.
+2. After the build is successful the build artifacts are deployed to a AWS S3 bucket.
3. The runner sends a lava job description to the LAVA master, who triggers the
job execution on LAVA Dispatcher.
@@ -103,20 +102,10 @@ The following variables are used and set by the ci system:
- `FTP_PROXY` : ftp proxy
- `NO_PROXY` : no proxy
-- SSH settings:
- - `LAVA_SSH_USER` : ssh user to connect to the LAVA master
- - `LAVA_SSH_HOST` : ssh host name to connect to the LAVA master
- - `LAVA_SSH_PORT` : ssh port used for the ssh tunnel
- - `LAVA_SSH_UPLOAD_KEY` : private ssh key to connect to the LAVA master
- - `LAVA_SSH_KNOWN_HOSTS` : Known hosts to connect via ssh to the host given by `LAVA_SSH_HOST`
-
- LAVA settings:
- `LAVA_MASTER_ACCOUNT` : lava master account name to register lavacli for test execution
- `LAVA_MASTER_TOKEN` : token to connect with the lava master
- - `LAVA_DEPLOY_DIR` : optional variable to define directory to store the build artifacts
- `LAVA_ARTIFACTS_URL` : optional variable where to get the artifacts for testing
- - `LAVA_DEPLOY_DIR`: Directory to deploy the Artifacts in a build. Default /var/lib/lava/artifacts
- - `LAVA_MASTER_PORT`: PORT to access the LAVA Master
- `LAVA_MASTER_URL`: URL to the LAVA Master
- General build settings
--
2.20.1
next prev parent reply other threads:[~2021-07-19 16:29 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-19 16:29 [xenomai-images][PATCH v2 0/3] ci: cleanup Q. Gylstorff
2021-07-19 16:29 ` [xenomai-images][PATCH v2 1/3] ci: Remove artifact deployment Q. Gylstorff
2021-07-19 16:29 ` Q. Gylstorff [this message]
2021-07-19 16:29 ` [xenomai-images][PATCH v2 3/3] ci/gitlab-ci-base: use rules Q. Gylstorff
2021-07-19 19:40 ` [xenomai-images][PATCH v2 0/3] ci: cleanup Jan Kiszka
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210719162949.4342-3-Quirin.Gylstorff@siemens.com \
--to=quirin.gylstorff@siemens.com \
--cc=jan.kiszka@siemens.com \
--cc=xenomai@xenomai.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.