* [Fuego] [PATCH v2 0/1] Use common scripts to install fuego on host or docker system
@ 2021-07-06 9:00 venkata.pyla
2021-07-06 9:00 ` [Fuego] [PATCH v2 1/1] install: Refactor fuego installation scripts to use common scripts venkata.pyla
0 siblings, 1 reply; 2+ messages in thread
From: venkata.pyla @ 2021-07-06 9:00 UTC (permalink / raw)
To: tim.bird; +Cc: fuego
From: venkata pyla <venkata.pyla@toshiba-tsip.com>
Hi Tim
I have refactored the installation scripts based on your feedback.
All the installation scripts moved under install-scripts folder and each
script is called in serial for different installation steps.
- install-debian-common.sh - all the common Debian packages
- install-buster.sh - the buster-specific Debian packages
or
- install-stretch.sh - the stretch-specific Debian packages
- install-jenkins.sh (stuff for Jenkins, called if --nojenkins was not specified)
- install-extras.sh (for ttc, serio, ebf, labcontrol, fserver, and anything else
we get directly from git)
Also added makefile targets to initiate different installations, incase
if someone needs it.
Please review the changes and provide your feedback.
you can also review the changes using my BitBucket account commit-id
https://bitbucket.org/venkata-pyla/fuego/commits/00254bbad34b626b2a78d0b8e25b065fd2220084?at=use-common-script-install-feedback
venkata pyla (1):
install: Refactor fuego installation scripts to use common scripts
Dockerfile | 306 ++---------------------
Dockerfile.nojenkins | 151 -----------
Makefile | 43 ++++
README | 14 ++
entrypoint.sh | 7 +
fuego-host-scripts/docker-build-image.sh | 10 +-
install-debian.sh | 268 --------------------
install-native.sh | 129 ++++++++++
install-scripts/install-debian-common.sh | 83 ++++++
install-scripts/install-extras.sh | 69 +++++
install-scripts/install-jenkins.sh | 157 ++++++++++++
install-scripts/install-stretch.sh | 12 +
install.sh | 14 +-
uninstall-fuego.sh | 52 ++++
14 files changed, 610 insertions(+), 705 deletions(-)
delete mode 100644 Dockerfile.nojenkins
create mode 100644 Makefile
create mode 100755 entrypoint.sh
delete mode 100755 install-debian.sh
create mode 100755 install-native.sh
create mode 100755 install-scripts/install-debian-common.sh
create mode 100755 install-scripts/install-extras.sh
create mode 100755 install-scripts/install-jenkins.sh
create mode 100755 install-scripts/install-stretch.sh
create mode 100755 uninstall-fuego.sh
--
2.20.1
The information contained in this e-mail message and in any
attachments/annexure/appendices is confidential to the
recipient and may contain privileged information.
If you are not the intended recipient, please notify the
sender and delete the message along with any
attachments/annexure/appendices. You should not disclose,
copy or otherwise use the information contained in the
message or any annexure. Any views expressed in this e-mail
are those of the individual sender except where the sender
specifically states them to be the views of
Toshiba Software India Pvt. Ltd. (TSIP),Bangalore.
Although this transmission and any attachments are believed to be
free of any virus or other defect that might affect any computer
system into which it is received and opened, it is the responsibility
of the recipient to ensure that it is virus free and no responsibility
is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or
damage arising in any way from its use.
^ permalink raw reply [flat|nested] 2+ messages in thread
* [Fuego] [PATCH v2 1/1] install: Refactor fuego installation scripts to use common scripts
2021-07-06 9:00 [Fuego] [PATCH v2 0/1] Use common scripts to install fuego on host or docker system venkata.pyla
@ 2021-07-06 9:00 ` venkata.pyla
0 siblings, 0 replies; 2+ messages in thread
From: venkata.pyla @ 2021-07-06 9:00 UTC (permalink / raw)
To: tim.bird; +Cc: fuego
From: venkata pyla <venkata.pyla@toshiba-tsip.com>
- Currently the fuego install steps are duplicated at different places of
installations like
* Dockefile
* Dockerfile.nojenkins
* install-debian.sh (for local install)
Now it is refactored to use a common installation scripts in all different
installations.
All common installation scripts are placed under folder `install-scripts`,
and in top folder it contains main install scripts for installing
fuego in docker or native system
install.sh --> Install fuego on docker system
install-native.sh --> Install fuego on native system.
- Created Makefile targets to initiate different installations
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
Dockerfile | 306 ++---------------------
Dockerfile.nojenkins | 151 -----------
Makefile | 43 ++++
README | 14 ++
entrypoint.sh | 7 +
fuego-host-scripts/docker-build-image.sh | 10 +-
install-debian.sh | 268 --------------------
install-native.sh | 129 ++++++++++
install-scripts/install-debian-common.sh | 83 ++++++
| 69 +++++
install-scripts/install-jenkins.sh | 157 ++++++++++++
install-scripts/install-stretch.sh | 12 +
install.sh | 14 +-
uninstall-fuego.sh | 52 ++++
14 files changed, 610 insertions(+), 705 deletions(-)
delete mode 100644 Dockerfile.nojenkins
create mode 100644 Makefile
create mode 100755 entrypoint.sh
delete mode 100755 install-debian.sh
create mode 100755 install-native.sh
create mode 100755 install-scripts/install-debian-common.sh
create mode 100755 install-scripts/install-extras.sh
create mode 100755 install-scripts/install-jenkins.sh
create mode 100755 install-scripts/install-stretch.sh
create mode 100755 uninstall-fuego.sh
diff --git a/Dockerfile b/Dockerfile
index 7f63798..8784427 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -2,9 +2,10 @@
# WARNING: this Dockerfile assumes that the container will be created with
# several bind mounts (see docker-create-container.sh)
# ==============================================================================
-# FIXTHIS: build this as an extension of the nonjenkins image
-FROM debian:stretch-slim
+ARG DEBIAN_VERSION=stretch
+
+FROM debian:${DEBIAN_VERSION}-slim
MAINTAINER tim.bird@sony.com
# ==============================================================================
@@ -15,300 +16,45 @@ ARG HTTP_PROXY
ENV http_proxy ${HTTP_PROXY}
ENV https_proxy ${HTTP_PROXY}
-# ==============================================================================
-# Prepare basic image
-# ==============================================================================
-
-ARG DEBIAN_FRONTEND=noninteractive
-
-WORKDIR /
-RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-
-# Fuego python dependencies
-# - python-lxml: ftc, loggen
-# - python-simplejson: ftc
-# - python-yaml: ftc
-# - python-openpyxl: ftc (also LTP)
-# - python-requests: ftc (also fuego_release_test)
-# - python-reportlab: ftc
-# - python-parsedatetime: ftc
-# - python-pexpect: ssh_exec (part of ttc)
-# - python-pip: to install filelock, flake8
-# - filelock: parser
-RUN mkdir -p /usr/share/man/man1
-RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pexpect python-pip python-setuptools python-wheel
-RUN pip install filelock
-
-# Fuego command dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion iproute2
-
-# Default SDK for testing locally or on an x86 board
-RUN apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
-
-# Default test host dependencies
-# - iperf iperf3 netperf: used as servers on the host
-# - bzip2 bc: used for local tests by Functional.bzip2/bc
-# - python-matplotlib: Benchmark.iperf3 parser
-# - python-xmltodict: AGL tests
-# - flake8: Functional.fuego_lint
-# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
-# - iputils-ping - for /bin/ping command, used by some tests
-# FIXTHIS: install dependencies dynamically on the tests that need them
-RUN apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
- netpipe-tcp iputils-ping
-RUN pip install flake8
-
-# miscelaneous packages:
-# python-serial - used by serio
-# diffstat and vim - used by Tim
-# time - useful for timing command duration
-RUN apt-get -q=2 -V --no-install-recommends install \
- python-serial \
- diffstat \
- vim \
- time
-
-# install packages used by NuttX SDK
-RUN apt-get -q=2 -V --no-install-recommends install \
- genromfs
-
-# FIXTHIS: determine if these tools are really necessary
-#RUN apt-get -q=2 -V --no-install-recommends install \
-# apt-utils python-paramiko \
-# xmlstarlet rsync \
-# inotify-tools gettext netpipe-tcp \
-# at minicom lzop bsdmainutils \
-# mc netcat openssh-server
-
-# bsdmainutils provides hexdump, which is needed for netperf test
-RUN apt-get -q=2 -V --no-install-recommends install \
- bsdmainutils
-
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
+# Copy the files to the docker that are required for installation
+RUN mkdir /fuego
+COPY . /fuego/
+WORKDIR /fuego
-# ==============================================================================
-# Install Jenkins with the same UID/GID as the host user
-# ==============================================================================
-
-ARG user=jenkins
-ARG group=jenkins
ARG uid=1000
ARG gid=${uid}
ARG JENKINS_PORT=8090
-# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
-#ARG JENKINS_VERSION=2.164.2
-#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
-#ARG JENKINS_VERSION=2.235.3
-#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
-ARG JENKINS_VERSION=2.249.3
-ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
-
-ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
-ARG JENKINS_UC=https://updates.jenkins.io
-ARG REF=/var/lib/jenkins/plugins
+ARG DEBIAN_VERSION=stretch
+ARG NO_JENKINS=
+ENV NO_JENKINS=$NO_JENKINS
ENV JENKINS_HOME=/var/lib/jenkins
ENV JENKINS_PORT=$JENKINS_PORT
-
-# Jenkins dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- default-jdk daemon psmisc adduser procps unzip
-RUN pip install python-jenkins==1.4.0
-
-RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
-RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
-RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN wget -nv ${JENKINS_URL}
-RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
-# allow Jenkins to start and install plugins, as part of dpkg installation
-RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
-RUN rm jenkins_${JENKINS_VERSION}_all.deb
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-# as of ttc version 2.0.2, this should not be needed
-#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# get ebf script and helpers
-# ==============================================================================
-RUN apt-get -q=2 -V --no-install-recommends install jq
-RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
- cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
- chmod a+x /usr/local/bin/ebf
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-
-RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
-COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
- frontend-install/0002-Output-data-from-port-during-command-execution.patch \
- /tmp/
-RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
- patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
- chown -R jenkins /usr/local/src/serio ; \
- cp /usr/local/src/serio/serio /usr/local/bin/ ; \
- ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
- ln -s /usr/local/bin/serio /usr/local/bin/sersh'
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
- ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
-
+ARG JENKINS_VERSION=2.249.3
+ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
# ==============================================================================
-# Jenkins post installation
+# Install fuego with the script
# ==============================================================================
+# Install debian common packages
+RUN ./install-scripts/install-debian-common.sh $DEBIAN_VERSION
-RUN source /etc/default/jenkins && \
- JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
- sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
-
-RUN source /etc/default/jenkins && \
- JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
- if [ -n "$HTTP_PROXY" ]; then \
- PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
- PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
- JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
- fi && \
- sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
-
-RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
-
-# set up Jenkins plugins
-COPY frontend-install/install-plugins.sh \
- frontend-install/jenkins-support \
- frontend-install/clitest \
- /usr/local/bin/
-
-# start Jenkins onces to initialize directories?
-RUN service jenkins start && \
- sleep 30 && \
- service jenkins stop
+# Install debian version specific packages or configurations
+RUN if [ -f ./install-scripts/install-${DEBIAN_VERSION}.sh ]; then ./install-scripts/install-${DEBIAN_VERSION}.sh; fi
-# install plugins - these versions are for Jenkins version 2.164.2
-# Explicitly install script-security v1.68, otherwise
-# v1.74 will automatically be installed as a dependency of
-# the junit plugin. Make sure to install before junit plugin.
-# Explicitly install junit:1.27
+# Install Jenkins
+RUN if [ -z $NO_JENKINS ]; then ./install-scripts/install-jenkins.sh ${JENKINS_PORT} ${JENKINS_VERSION} ${JENKINS_SHA} ${uid} ${gid}; fi
-# install other plugins from Jenkins update center
-# NOTE: not sure all of these are needed, but keep list
-# as compatible as possible with 1.2.1 release for now
-# Do NOT change the order of the plugins, unless you know what
-# you are doing. Otherwise, the install-plugins.sh script will
-# install the wrong versions of other plugins based on dependency
-# information in the plugin files.
+# Install extra binaries downloading from repositories
+RUN ./install-scripts/install-extras.sh $NO_JENKINS
-#RUN /usr/local/bin/install-plugins.sh \
-# script-security:1.68 \
-# structs:1.20 \
-# workflow-step-api:2.22 \
-# workflow-api:2.40 \
-# junit:1.27 \
-# scm-api:2.6.3 \
-# ant:1.9 \
-# antisamy-markup-formatter:1.5 \
-# bouncycastle-api:2.17 \
-# command-launcher:1.3 \
-# description-setter:1.10 \
-# display-url-api:2.3.1 \
-# external-monitor-job:1.7 \
-# greenballs:1.15 \
-# icon-shim:2.0.3 \
-# javadoc:1.5 \
-# jdk-tool:1.2 \
-# ldap:1.20 \
-# mailer:1.23 \
-# matrix-auth:2.3 \
-# matrix-project:1.14 \
-# pam-auth:1.5 \
-# pegdown-formatter:1.3 \
-# windows-slaves:1.4
+COPY entrypoint.sh /usr/bin/
-# install plugins for latest LTS Jenkins (currently 2.249.3)
-RUN /usr/local/bin/install-plugins.sh \
- script-security \
- structs \
- workflow-step-api \
- workflow-api \
- junit \
- scm-api \
- ant \
- antisamy-markup-formatter \
- bouncycastle-api \
- command-launcher \
- description-setter \
- display-url-api \
- external-monitor-job \
- greenballs \
- icon-shim \
- javadoc \
- jdk-tool \
- mailer \
- matrix-auth \
- matrix-project \
- pam-auth \
- pegdown-formatter \
- windows-slaves
-
-COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
-
-# jenkins should automatically unzip any plugins in the plugin dir
-# make a symlink for mod.js well after flot is installed
-RUN service jenkins start && sleep 30 && \
- rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
- ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-
-RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
-
-COPY frontend-install/config.xml $JENKINS_HOME/config.xml
-COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-
-RUN chown -R jenkins:jenkins $JENKINS_HOME/
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-
-RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
-COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-RUN echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
+# Delete the files that are copied for installation
+RUN rm -rf /fuego
+WORKDIR /
# ==============================================================================
# Setup startup command
# ==============================================================================
-
# FIXTHIS: when running multiple Fuego containers, or if the host is already
# running the netperf server, netperf will complain because the port is taken
-ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
+ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
deleted file mode 100644
index 56de98a..0000000
--- a/Dockerfile.nojenkins
+++ /dev/null
@@ -1,151 +0,0 @@
-# ==============================================================================
-# WARNING: this Dockerfile assumes that the container will be created with
-# several bind mounts (see docker-create-container.sh)
-# ==============================================================================
-# Usage:
-# host$ vi fuego-ro/conf/fuego.conf
-# jenkins_enabled=0
-# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
-# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
-# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
-# docker# ftc list-boards
-# docker, bbb, ...
-# docker# ftc list-tests
-# Functional.hello_world, ...
-# docker# ftc run-test -b docker -t Functional.hello_world
-# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
-
-FROM debian:stretch-slim
-MAINTAINER tim.bird@sony.com
-
-# ==============================================================================
-# Proxy variables
-# ==============================================================================
-
-ARG HTTP_PROXY
-ENV http_proxy ${HTTP_PROXY}
-ENV https_proxy ${HTTP_PROXY}
-
-# ==============================================================================
-# Prepare basic image
-# ==============================================================================
-
-ARG DEBIAN_FRONTEND=noninteractive
-
-WORKDIR /
-RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-
-# Fuego python dependencies
-# - python-lxml: ftc, loggen
-# - python-simplejson: ftc
-# - python-yaml: ftc
-# - python-openpyxl: ftc (also LTP)
-# - python-requests: ftc (also fuego_release_test)
-# - python-reportlab: ftc
-# - python-parsedatetime: ftc
-# - python-pexpect: ssh_exec (part of ttc)
-# - python-pip: to install filelock, flake8
-# - filelock: parser
-RUN mkdir -p /usr/share/man/man1
-RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pexpect python-pip python-setuptools python-wheel
-RUN pip install filelock
-
-# Fuego command dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion iproute2
-
-# Default SDK for testing locally or on an x86 board
-RUN apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
-
-# Default test host dependencies
-# - iperf iperf3 netperf: used as servers on the host
-# - bzip2 bc: used for local tests by Functional.bzip2/bc
-# - python-matplotlib: Benchmark.iperf3 parser
-# - python-xmltodict: AGL tests
-# - flake8: Functional.fuego_lint
-# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
-# - iputils-ping - for /bin/ping command, used by some tests
-# FIXTHIS: install dependencies dynamically on the tests that need them
-RUN apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
- netpipe-tcp iputils-ping
-RUN pip install flake8
-
-# miscelaneous packages:
-# python-serial - used by serio
-# diffstat and vim - used by Tim
-# time - useful for timing command duration
-RUN apt-get -q=2 -V --no-install-recommends install \
- python-serial \
- diffstat \
- vim \
- time
-
-# FIXTHIS: determine if these tools are really necessary
-# RUN apt-get -q=2 -V --no-install-recommends install \
-# apt-utils python-paramiko \
-# xmlstarlet rsync \
-# inotify-tools gettext netpipe-tcp \
-# at minicom lzop bsdmainutils \
-# mc netcat openssh-server
-
-# bsdmainutils provides hexdump, which is needed for netperf test
-RUN apt-get -q=2 -V --no-install-recommends install \
- bsdmainutils
-
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-# as of ttc version 2.0.2, this should not be needed
-#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-
-RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
-COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-RUN echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-
-# ==============================================================================
-# Setup startup command
-# ==============================================================================
-
-# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
-ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..d73510c
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,43 @@
+# To change docker image name or jenkins port pass below arguments
+# e.g. make install-fuego-docker NAME="fuego-buster" PORT=8091
+NAME?=fuego
+PORT?=8090
+DEBIAN_VERSION?=stretch
+
+# To Change Jenkins version
+JENKINS_VERSION?=2.249.3
+JENKINS_SHA?=534014c007edbb533a1833fe6f2dc115faf3faa2
+
+# for nocache and privilage options to docker image pass as argument
+# e.g. make install-fuego-docker EXTRA="--nocache --priv"
+EXTRA?=
+
+# Install fuego on Docker
+install-fuego-docker:
+ ./install.sh $(NAME) $(PORT) $(DEBIAN_VERSION) $(EXTRA)
+
+install-fuego-docker-nojenkins:
+ make install-fuego-docker EXTRA="--nojenkins $(EXTRA)"
+
+install-fuego-docker-buster:
+ make install-fuego-docker DEBIAN_VERSION=buster EXTRA=$(EXTRA)
+
+install-fuego-docker-buster-nojenkins:
+ make install-fuego-docker-buster EXTRA="--nojenkins $(EXTRA)"
+
+# Start fuego on docker
+run-fuego-docker:
+ ./start.sh $(NAME)-container
+
+# Install fuego on native machine
+install-fuego-native:
+ sudo ./install-native.sh -s $(DEBIAN_VERSION) -p $(PORT) $(EXTRA)
+
+install-fuego-native-nojenkins:
+ make install-fuego-native EXTRA="--nojenkins $(EXTRA)"
+
+install-fuego-native-buster:
+ make install-fuego-native DEBIAN_VERSION=buster EXTRA="$(EXTRA)"
+
+install-fuego-native-buster-nojenkins:
+ make install-fuego-native-buster EXTRA="--nojenkins $(EXTRA)"
diff --git a/README b/README
index 882d84e..30ffe34 100644
--- a/README
+++ b/README
@@ -44,6 +44,20 @@ You can change the name of the docker image and the port used by Jenkins
by passing the corresponding parameters to install.sh.
./install.sh fuego-8082 8082
+You can also install fuego on other debian distribution by passing additional
+argument to the install.sh.
+ ./install.sh fuego-buster-8083 8083 buster
+
+Installing Fuego on native system
+-------------------------------------------------------------------------------
+Run install-native.sh from the top directory. This installs the fuego on local
+system instead on docker.
+ ./install-native.sh
+
+This takes the default values for installation as stretch for debin version,
+8090 for jenkins port, you can change the value by passing them in arguements.
+ ./install-native.sh -s <debian version> -p <jenkins port>
+
Running
-------------------------------------------------------------------------------
To start your Fuego container, issue the following command top directory:
diff --git a/entrypoint.sh b/entrypoint.sh
new file mode 100755
index 0000000..6eac895
--- /dev/null
+++ b/entrypoint.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+[ "$1" != "--nojenkins" ] && service jenkins start
+
+#TODO: netperf not present in buster, remove this condition once alternative found
+[ -f /etc/init.d/netperf ] && service netperf start
+iperf3 -V -s -D -f M
diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
index 5823357..a4ee622 100755
--- a/fuego-host-scripts/docker-build-image.sh
+++ b/fuego-host-scripts/docker-build-image.sh
@@ -13,7 +13,11 @@ fi
DOCKERIMAGE=${1:-fuego}
JENKINS_PORT=${2:-8090}
-DOCKERFILE=${3:-Dockerfile}
+DEBIAN_VERSION=${3:-stretch}
+NO_JENKINS=${4:-}
+DOCKERFILE="Dockerfile"
+: "${JENKINS_VERSION:=2.249.3}"
+: "${JENKINS_SHA:=534014c007edbb533a1833fe6f2dc115faf3faa2}"
if [ "$(id -u)" == "0" ]; then
JENKINS_UID=$(id -u $SUDO_USER)
@@ -27,4 +31,8 @@ echo "Using Port $JENKINS_PORT"
sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
--build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
+ --build-arg DEBIAN_VERSION=$DEBIAN_VERSION \
+ --build-arg NO_JENKINS=$NO_JENKINS \
+ --build-arg JENKINS_VERSION=$JENKINS_VERSION \
+ --build-arg JENKINS_SHA=$JENKINS_SHA \
--build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
diff --git a/install-debian.sh b/install-debian.sh
deleted file mode 100755
index ccca032..0000000
--- a/install-debian.sh
+++ /dev/null
@@ -1,268 +0,0 @@
-#!/bin/bash
-# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
-#
-# Usage:
-# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
-#
-if [ -n "$1" ]; then
- if [ "$1" = "--help" -o "$1" = "-h" ]; then
- cat <<HERE
-Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
-
-Installs fuego on the host Debian filesystem.
-
-options:
- --help Show usage help
- --nojenkins Install Fuego without Jenkins
-HERE
- exit 0
- fi
-fi
-
-if [[ $EUID -ne 0 ]]; then
- echo "Sorry, you need root permissions"
- exit 1
-fi
-
-if [ "$1" = "--nojenkins" ]; then
- nojenkins=1
- shift
-else
- nojenkins=0
- port=${1:-8090}
-fi
-
-# ==============================================================================
-# Install Fuego dependencies
-# ==============================================================================
-
-# netperf is in non-free
-echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-
-apt-get update -q=2
-
-# Fuego python dependencies
-apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pip
-pip install filelock
-
-# Fuego command dependencies
-apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion
-
-# Default SDK for testing locally or on an x86 board
-apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
-
-# Default test host dependencies
-apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
-pip install flake8 bsdmainutils
-
-echo "dash dash/sh boolean false" | debconf-set-selections
-DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
-if [ -n "$http_proxy" ]; then
- sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
- echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
-fi
-
-# ==============================================================================
-# Clone fuego and fuego-core
-# ==============================================================================
-if [ ! -d "/fuego" ]; then
- cd /
- git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
- ln -s /fuego/fuego-ro /fuego-ro
- ln -s /fuego/fuego-rw /fuego-rw
- cd fuego
- git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
- ln -s /fuego/fuego-core /fuego-core
-fi
-
-# ==============================================================================
-# Install Jenkins
-# ==============================================================================
-
-if [ $nojenkins -eq 0 ]; then
- #JENKINS_VERSION=2.164.2
- #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
- JENKINS_VERSION=2.249.3
- JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
- JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
- JENKINS_UC=https://updates.jenkins.io
- REF=/var/lib/jenkins/plugins
- JENKINS_HOME=/var/lib/jenkins
- JENKINS_PORT=$port
-
- # Jenkins dependencies
- apt-get -q=2 -V --no-install-recommends install \
- default-jdk daemon psmisc adduser procps unzip
- pip install python-jenkins==1.4.0
-
- echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
- groupadd jenkins
- useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
- cd
- wget ${JENKINS_URL}
- echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
-
- # allow Jenkins to start and install plugins, as part of dpkg installation
- printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
- dpkg -i jenkins_${JENKINS_VERSION}_all.deb
- rm jenkins_${JENKINS_VERSION}_all.deb
-
- # update ownership
- chown -R jenkins:jenkins /fuego
- chown -R jenkins:jenkins /fuego-ro
- chown -R jenkins:jenkins /fuego-rw
- chown -R jenkins:jenkins /fuego-core
-
- source /etc/default/jenkins && \
- JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
- sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
-
- source /etc/default/jenkins && \
- JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
- if [ -n "$http_proxy" ]; then \
- PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
- PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
- JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
- fi && \
- sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
-
- sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
-
- cp /fuego/frontend-install/install-plugins.sh \
- /fuego/frontend-install/jenkins-support \
- /fuego/frontend-install/clitest \
- /usr/local/bin/
-
- cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
- ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
- mkdir $JENKINS_HOME/userContent/docs
- cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
- jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
- sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
- chown -R jenkins:jenkins $JENKINS_HOME/
-
- # start and stop jenkins to pre-populate some settings
- service jenkins start && \
- sleep 30 && \
- service jenkins stop
-
- # install other plugins from Jenkins update center
- # NOTE: not sure all of these are needed, but keep list
- # compatible with 1.2.1 release for now
- #/usr/local/bin/install-plugins.sh \
- # ant:1.9 \
- # antisamy-markup-formatter:1.5 \
- # bouncycastle-api:2.17 \
- # command-launcher:1.3 \
- # description-setter:1.10 \
- # display-url-api:2.3.1 \
- # external-monitor-job:1.7 \
- # greenballs:1.15 \
- # icon-shim:2.0.3 \
- # javadoc:1.5 \
- # jdk-tool:1.2 \
- # junit:1.27 \
- # ldap:1.20 \
- # mailer:1.23 \
- # matrix-auth:2.3 \
- # matrix-project:1.14 \
- # pam-auth:1.4 \
- # pegdown-formatter:1.3 \
- # structs:1.17 \
- # windows-slaves:1.4
-
- # install plugins for latest LTS Jenkins (currently 2.249.3)
- /usr/local/bin/install-plugins.sh \
- script-security \
- structs \
- workflow-step-api \
- workflow-api \
- junit \
- scm-api \
- ant \
- antisamy-markup-formatter \
- bouncycastle-api \
- command-launcher \
- description-setter \
- display-url-api \
- external-monitor-job \
- greenballs \
- icon-shim \
- javadoc \
- jdk-tool \
- mailer \
- matrix-auth \
- matrix-project \
- pam-auth \
- pegdown-formatter \
- windows-slaves
-
- cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
-
- # make the mod.js symlink well after flot is installed
- service jenkins start && sleep 30 && \
- rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
- ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-
- chown -R jenkins:jenkins $JENKINS_HOME/
-else
- sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
-fi
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-/usr/local/src/ttc/install.sh /usr/local/bin
-perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-if [ $nojenkins -eq 0 ]; then
- /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
- /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-else
- /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
- /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-fi
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
-ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-ln -s /fuego-core/scripts/ftc /usr/local/bin/
-cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-
-# ==============================================================================
-# Small guide
-# ==============================================================================
-echo "Run 'service netperf start' to start a netperf server"
-echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
-echo "Run 'ftc list-boards' to see the available boards"
-echo "Run 'ftc list-tests' to see the available tests"
-echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
-echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
-echo "Run 'ftc gen-report' to get results"
diff --git a/install-native.sh b/install-native.sh
new file mode 100755
index 0000000..0b196bb
--- /dev/null
+++ b/install-native.sh
@@ -0,0 +1,129 @@
+#!/bin/bash
+# 2021 (c) Toshiba corp.
+#
+# Install feugo on native system
+set -e
+
+: "${JENKINS_VERSION:=2.249.3}"
+: "${JENKINS_SHA:=534014c007edbb533a1833fe6f2dc115faf3faa2}"
+
+usage(){
+ echo ""
+ echo "Usage: sudo ./install-native.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
+ echo "Install fuego on Debian filesystem."
+ echo ""
+ echo "options:"
+ echo " --help | -h Show usage help"
+ echo " --suite | -s <suite> Specify Debian suite"
+ echo " --port | -p <port> Jenkins port"
+ echo " --nojenkins Install Fuego without Jenkins"
+ echo ""
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+NO_JENKINS=
+JENKINS_PORT=8090
+DEBIAN_VERSION=stretch
+uid=
+gid=
+
+while [ "$1" != "" ]; do
+ case $1 in
+ -s | --suite )
+ shift
+ DEBIAN_VERSION=$1
+ ;;
+ -p | --port )
+ shift
+ JENKINS_PORT=$1
+ ;;
+ -u | --uid )
+ shift
+ uid=$1
+ ;;
+ -g | --gid )
+ shift
+ gid=$1
+ ;;
+ --nojenkins )
+ NO_JENKINS="--nojenkins"
+ ;;
+ -h | --help )
+ usage
+ exit 0
+ ;;
+ * )
+ break
+ esac
+ shift
+done
+
+# get fuego-core repository, if not already present
+if [ ! -f fuego-core/scripts/ftc ] ; then
+ # set fuego-core branch to same as current fuego branch
+ # get current git branch
+ set -o noglob
+ while IFS=" " read -r part1 part2 ; do
+ if [ $part1 = "*" ] ; then
+ branch=$part2
+ fi
+ done < <(git branch)
+ set +o noglob
+ git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
+fi
+
+fuego_dir=$(pwd)
+
+# Install debian common packages
+./install-scripts/install-debian-common.sh $DEBIAN_VERSION
+
+# Install debian version specific packages or configurations
+if [ -f ./install-scripts/install-${DEBIAN_VERSION}.sh ]; then ./install-scripts/install-${DEBIAN_VERSION}.sh; fi
+
+# Install Jenkins
+if [ -z $NO_JENKINS ]; then ./install-scripts/install-jenkins.sh ${JENKINS_PORT} ${JENKINS_VERSION} ${JENKINS_SHA} ${uid} ${gid}; fi
+
+# Install extra binaries downloading from repositories
+./install-scripts/install-extras.sh $NO_JENKINS
+
+# Set Jenkins enabled status in fuego.conf
+[ "$NO_JENKINS" == "--nojenkins" ] && val=0 || val=1
+sed -i "s/\(^jenkins_enabled=\).*/\1$val/" fuego-ro/conf/fuego.conf
+
+if [ -z $NO_JENKINS ]; then
+ # Start the Jenkins service
+ systemctl daemon-reload
+ service jenkins restart
+
+ # provide permission to current user
+ [ -z $SUDO_USER ] && user=$USER || user=$SUDO_USER
+ usermod -a -G jenkins $user
+ chmod -R 775 $fuego_dir/fuego-rw
+
+ chown -R jenkins:jenkins $fuego_dir/fuego-ro
+ chown -R jenkins:jenkins $fuego_dir/fuego-rw
+ chown -R jenkins:jenkins $fuego_dir/fuego-core
+fi
+
+ # ==============================================================================
+ # Create Links to fuego folders
+ # ==============================================================================
+ ln -sf $fuego_dir/fuego-ro /
+ ln -sf $fuego_dir/fuego-rw /
+ ln -sf $fuego_dir/fuego-core /
+
+ # ==============================================================================
+ # Small guide
+ # ==============================================================================
+ echo "Logout and Login current user session to work properly"
+ echo "Run 'service netperf start' to start a netperf server"
+ echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
+ echo "Run 'ftc list-boards' to see the available boards"
+ echo "Run 'ftc list-tests' to see the available tests"
+ echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
+ echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
+ echo "Run 'ftc gen-report' to get results"
diff --git a/install-scripts/install-debian-common.sh b/install-scripts/install-debian-common.sh
new file mode 100755
index 0000000..8c6d565
--- /dev/null
+++ b/install-scripts/install-debian-common.sh
@@ -0,0 +1,83 @@
+#!/bin/bash
+# 2021 (c) Toshiba corp.
+#
+# Install Debian common steps
+set -e
+
+debian_version=${1:stretch}
+
+# netperf is in non-free
+echo deb http://deb.debian.org/debian $debian_version main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security $debian_version/updates main >> /etc/apt/sources.list
+
+# Fuego python dependencies
+# - python-lxml: ftc, loggen
+# - python-simplejson: ftc
+# - python-yaml: ftc
+# - python-openpyxl: ftc (also LTP)
+# - python-requests: ftc (also fuego_release_test)
+# - python-reportlab: ftc
+# - python-parsedatetime: ftc
+# - python-pexpect: ssh_exec (part of ttc)
+# - python-pip: to install filelock, flake8
+# - filelock: parser
+mkdir -p /usr/share/man/man1
+apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
+ python-lxml python-simplejson python-yaml python-openpyxl \
+ python-requests python-reportlab python-parsedatetime \
+ python-pexpect python-pip python-setuptools python-wheel
+pip install filelock
+
+# Fuego command dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ git sshpass openssh-client sudo net-tools wget curl lava-tool \
+ bash-completion iproute2
+
+# Default SDK for testing locally or on an x86 board
+apt-get -q=2 -V --no-install-recommends install \
+ build-essential cmake bison flex automake kmod libtool \
+ libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
+
+# Default test host dependencies
+# - iperf iperf3 netperf: used as servers on the host
+# - bzip2 bc: used for local tests by Functional.bzip2/bc
+# - python-matplotlib: Benchmark.iperf3 parser
+# - python-xmltodict: AGL tests
+# - flake8: Functional.fuego_lint
+# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
+# - iputils-ping - for /bin/ping command, used by some tests
+# FIXTHIS: install dependencies dynamically on the tests that need them
+apt-get -q=2 -V --no-install-recommends install \
+ iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
+pip install flake8
+
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# FIXTHIS: determine if these tools are really necessary
+# apt-get -q=2 -V --no-install-recommends install \
+# apt-utils python-paramiko \
+# xmlstarlet rsync \
+# inotify-tools gettext netpipe-tcp \
+# at minicom lzop bsdmainutils \
+# mc netcat openssh-server
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+if [ -n "$http_proxy" ]; then
+ sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
+ echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+fi
\ No newline at end of file
--git a/install-scripts/install-extras.sh b/install-scripts/install-extras.sh
new file mode 100755
index 0000000..f0290aa
--- /dev/null
+++ b/install-scripts/install-extras.sh
@@ -0,0 +1,69 @@
+#!/bin/bash
+# 2021 (c) Toshiba corp.
+#
+# Install extra
+set -e
+
+nojenkins=0
+[ "$1" = "--nojenkins" ] && nojenkins=1
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+[ -d /usr/local/src/ttc ] || git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+# as of ttc version 2.0.2, this should not be needed
+#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ apt-get -q=2 -V --no-install-recommends install jq
+ [ -d /usr/local/src/board-farm-rest-api ] || git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+ cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+ chmod a+x /usr/local/bin/ebf
+fi
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+[ -d /usr/local/src/serio ] || git clone https://github.com/frowand/serio.git /usr/local/src/serio
+if [ $nojenkins -eq 0 ]; then
+ cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+ if patch -d /usr/local/src/serio -p1 -N -s --dry-run </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch; then
+ patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
+ fi
+ if patch -d /usr/local/src/serio -p1 -N -s --dry-run </tmp/0002-Output-data-from-port-during-command-execution.patch; then
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
+ fi
+ chown -R jenkins /usr/local/src/serio
+fi
+cp /usr/local/src/serio/serio /usr/local/bin/
+ln -sf /usr/local/bin/serio /usr/local/bin/sercp
+ln -sf /usr/local/bin/serio /usr/local/bin/sersh
+
+[ -d /usr/local/src/serlogin ] || git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
+[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
+cp /usr/local/src/serlogin/serlogin /usr/local/bin
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+[ -d /usr/local/lib/fserver ] || git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -sf /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# ftc post installation
+# ==============================================================================
+ln -sf /fuego-core/scripts/ftc /usr/local/bin/
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+echo ". /etc/bash_completion" >> /root/.bashrc
+
+# ==============================================================================
+# Lava
+# ==============================================================================
+ln -sf /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -sf /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
\ No newline at end of file
diff --git a/install-scripts/install-jenkins.sh b/install-scripts/install-jenkins.sh
new file mode 100755
index 0000000..e0b549b
--- /dev/null
+++ b/install-scripts/install-jenkins.sh
@@ -0,0 +1,157 @@
+#!/bin/bash
+# 2021 (c) Toshiba corp.
+#
+# Install Jenkins specific
+set -e
+
+JENKINS_PORT=${1:-8090}
+JENKINS_VERSION=${2:-2.249.3}
+JENKINS_SHA=${3:-534014c007edbb533a1833fe6f2dc115faf3faa2}
+uid=${4:-}
+gid=${5:-}
+
+JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
+export JENKINS_UC=https://updates.jenkins.io
+export REF=/var/lib/jenkins/plugins
+export JENKINS_HOME=/var/lib/jenkins
+export JENKINS_PORT=$JENKINS_PORT
+
+# Jenkins dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ default-jdk daemon psmisc adduser procps unzip
+pip install python-jenkins==1.4.0
+
+echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
+
+if [ -z $gid ]; then
+ getent group jenkins >/dev/null || groupadd jenkins
+else
+ getent group ${gid} >/dev/null || groupadd -g ${gid} jenkins
+fi
+
+if [ -z $uid ]; then
+ id jenkins >/dev/null 2>&1 || useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
+else
+ id jenkins >/dev/null 2>&1 || useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash jenkins
+fi
+
+wget -nv ${JENKINS_URL}
+echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
+
+# allow Jenkins to start and install plugins, as part of dpkg installation
+printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
+dpkg -i jenkins_${JENKINS_VERSION}_all.deb
+rm jenkins_${JENKINS_VERSION}_all.deb
+
+# ==============================================================================
+# Jenkins post installation
+# ==============================================================================
+source /etc/default/jenkins && \
+ JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
+ sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+source /etc/default/jenkins && \
+ JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
+ if [ -n "$http_proxy" ]; then \
+ PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
+ fi && \
+ sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
+
+sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
+
+# set up Jenkins plugins
+cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
+ /usr/local/bin/
+
+# start and stop jenkins to pre-populate some settings
+service jenkins start && \
+ sleep 30 && \
+ service jenkins stop
+
+# install plugins - these versions are for Jenkins version 2.164.2
+# Explicitly install script-security v1.68, otherwise
+# v1.74 will automatically be installed as a dependency of
+# the junit plugin. Make sure to install before junit plugin.
+# Explicitly install junit:1.27
+
+# install other plugins from Jenkins update center
+# NOTE: not sure all of these are needed, but keep list
+# as compatible as possible with 1.2.1 release for now
+# Do NOT change the order of the plugins, unless you know what
+# you are doing. Otherwise, the install-plugins.sh script will
+# install the wrong versions of other plugins based on dependency
+# information in the plugin files.
+
+#/usr/local/bin/install-plugins.sh \
+# script-security:1.68 \
+# structs:1.20 \
+# workflow-step-api:2.22 \
+# workflow-api:2.40 \
+# junit:1.27 \
+# scm-api:2.6.3 \
+# ant:1.9 \
+# antisamy-markup-formatter:1.5 \
+# bouncycastle-api:2.17 \
+# command-launcher:1.3 \
+# description-setter:1.10 \
+# display-url-api:2.3.1 \
+# external-monitor-job:1.7 \
+# greenballs:1.15 \
+# icon-shim:2.0.3 \
+# javadoc:1.5 \
+# jdk-tool:1.2 \
+# ldap:1.20 \
+# mailer:1.23 \
+# matrix-auth:2.3 \
+# matrix-project:1.14 \
+# pam-auth:1.5 \
+# pegdown-formatter:1.3 \
+# windows-slaves:1.4
+
+# install plugins for latest LTS Jenkins (currently 2.249.3)
+/usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+# jenkins should automatically unzip any plugins in the plugin dir
+# make the mod.js symlink well after flot is installed
+service jenkins start && sleep 30 && \
+ rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
+ ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+
+mkdir -p $JENKINS_HOME/userContent/docs
+ln -sf /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+
+cp frontend-install/config.xml $JENKINS_HOME/config.xml
+cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+
+chown -R jenkins:jenkins $JENKINS_HOME/
\ No newline at end of file
diff --git a/install-scripts/install-stretch.sh b/install-scripts/install-stretch.sh
new file mode 100755
index 0000000..376c027
--- /dev/null
+++ b/install-scripts/install-stretch.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+# 2021 (c) Toshiba corp.
+#
+# Install Debian version specific packages
+set -e
+
+#TODO: find alternate package for netperf in buster version
+apt-get -q=2 -V --no-install-recommends install netperf
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+apt-get -q=2 -V --no-install-recommends install \
+ bsdmainutils
\ No newline at end of file
diff --git a/install.sh b/install.sh
index a5a90b0..97376d4 100755
--- a/install.sh
+++ b/install.sh
@@ -1,13 +1,13 @@
#!/bin/bash
#
-# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
+# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<debian version>]
#
# $1 is the exit code after usage is shown
usage() {
exit_code="$1"
cat <<HERE
-Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
+Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<debian version>]
Create the docker image and container with the Fuego test distribution.
If no <image_name> is provided, the image will be named 'fuego'.
@@ -34,7 +34,7 @@ fi
priv=0
NOCACHE=""
-dockerfile="Dockerfile"
+nojenkins=""
POSITIONAL=()
while [[ $# -gt 0 ]] ; do
@@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
shift
;;
--nojenkins)
- dockerfile="Dockerfile.nojenkins"
+ nojenkins="--nojenkins"
shift
;;
*)
@@ -60,6 +60,7 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
image_name=${1:-fuego}
jenkins_port=${2:-8090}
+debian_version=${3:-stretch}
container_name="${image_name}-container"
@@ -79,13 +80,16 @@ fi
set -e
-source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
+source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${debian_version} ${nojenkins}
if [ "$priv" == "0" ]; then
fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
else
fuego-host-scripts/docker-create-usb-privileged-container.sh ${image_name} ${container_name}
fi
+[ "$nojenkins" == "--nojenkins" ] && val=0 || val=1
+sed -i "s/\(^jenkins_enabled=\).*/\1$val/" fuego-ro/conf/fuego.conf
+
# copy host's ttc.conf file (if present) into the fuego configuration directory
sudo /bin/sh -c "if [ -f /etc/ttc.conf -a ! -f fuego-ro/conf/ttc.conf ] ; then cp /etc/ttc.conf fuego-ro/conf/ttc.conf ; fi"
diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
new file mode 100755
index 0000000..9c7efda
--- /dev/null
+++ b/uninstall-fuego.sh
@@ -0,0 +1,52 @@
+#!/bin/bash
+
+# Remove Jenkins plugins
+[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
+
+# uninstall Jenkins
+service jenkins stop
+dpkg -P jenkins
+
+[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
+[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
+[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
+[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
+sed -i '/^JENKINS_PORT=/d' /etc/environment
+
+# uninstall ttc script
+[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
+
+# uninstall ebf script
+[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
+[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
+
+# uninstall serial config
+[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
+[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
+[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
+[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
+[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
+[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
+
+# uninstall fserver
+[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
+[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
+
+# uninstall ftc
+[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
+[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
+
+# remove lava target files
+[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
+[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
+
+# remove links
+[ -d /fuego-ro ] && rm /fuego-ro
+[ -d /fuego-rw ] && rm /fuego-rw
+[ -d /fuego-core ] && rm /fuego-core
+
+# Change owner permissions back
+[ -z $SUDO_USER ] && user=$USER || user=$SUDO_USER
+[ -d ./fuego-ro ] && chown -R $user:$user ./fuego-ro
+[ -d ./fuego-rw ] && chown -R $user:$user ./fuego-rw
+[ -d ./fuego-core ] && chown -R $user:$user ./fuego-core
--
2.20.1
The information contained in this e-mail message and in any
attachments/annexure/appendices is confidential to the
recipient and may contain privileged information.
If you are not the intended recipient, please notify the
sender and delete the message along with any
attachments/annexure/appendices. You should not disclose,
copy or otherwise use the information contained in the
message or any annexure. Any views expressed in this e-mail
are those of the individual sender except where the sender
specifically states them to be the views of
Toshiba Software India Pvt. Ltd. (TSIP),Bangalore.
Although this transmission and any attachments are believed to be
free of any virus or other defect that might affect any computer
system into which it is received and opened, it is the responsibility
of the recipient to ensure that it is virus free and no responsibility
is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or
damage arising in any way from its use.
^ permalink raw reply related [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-07-06 9:00 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-06 9:00 [Fuego] [PATCH v2 0/1] Use common scripts to install fuego on host or docker system venkata.pyla
2021-07-06 9:00 ` [Fuego] [PATCH v2 1/1] install: Refactor fuego installation scripts to use common scripts venkata.pyla
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.