* [Fuego] Use common script to install fuego on host or docker system
@ 2021-06-04 13:43 venkata.pyla
2021-06-04 14:25 ` Venkata.Pyla
` (2 more replies)
0 siblings, 3 replies; 8+ messages in thread
From: venkata.pyla @ 2021-06-04 13:43 UTC (permalink / raw)
To: Tim.Bird, tbird20d; +Cc: fuego
From: venkata pyla <venkata.pyla@toshiba-tsip.com>
Modified the install scripts to use common scripts for both types of
installation, this way it will help to make sure both the environments
are same and if any future modification will have single place of change.
Now, for installing fuego on docker system the 'install.sh' script will
use common script in Dockefile to install fuego related stuff.
sameway for installing fuego on host system it can use the same common
script to install fuego related stuff.
Below are the common scripts that are used in both the installation types
for each debian distribution
* install-stretch.sh
* install-buster.sh
so finally,
To install fuego on docker system use the below script with options
$ ./install.sh <options> image_name <port> <distribution> <suite>
e.g: ./install.sh debian_stretch 8090 debian stretch
or
./install.sh debian_buster 8090 debian buster
or
./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
To install fuego on host system use the below script with options
$ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
e.g: ./install-fuego.sh -d debian -s stretch -p 8090
or
./install-fuego.sh -d debian -s buster -p 8090
or
./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
Dockerfile | 303 ++---------------
Dockerfile.nojenkins | 151 ---------
README | 29 +-
entrypoint.sh | 7 +
fuego-host-scripts/docker-build-image.sh | 11 +-
install-buster.sh | 294 ++++++++++++++++
install-debian.sh | 410 ++++++++++++++---------
install-fuego.sh | 109 ++++++
install-stretch.sh | 288 ++++++++++++++++
install.sh | 12 +-
uninstall-fuego.sh | 47 +++
11 files changed, 1057 insertions(+), 604 deletions(-)
delete mode 100644 Dockerfile.nojenkins
create mode 100755 entrypoint.sh
create mode 100755 install-buster.sh
create mode 100755 install-fuego.sh
create mode 100755 install-stretch.sh
create mode 100644 uninstall-fuego.sh
diff --git a/Dockerfile b/Dockerfile
index 7f63798..ac57cb2 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -2,9 +2,11 @@
# WARNING: this Dockerfile assumes that the container will be created with
# several bind mounts (see docker-create-container.sh)
# ==============================================================================
-# FIXTHIS: build this as an extension of the nonjenkins image
-FROM debian:stretch-slim
+ARG SUITE=stretch
+ARG DIST=debian
+
+FROM ${DIST}:${SUITE}-slim
MAINTAINER tim.bird@sony.com
# ==============================================================================
@@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
ENV https_proxy ${HTTP_PROXY}
# ==============================================================================
-# Prepare basic image
+# Install fuego with the script
# ==============================================================================
-ARG DEBIAN_FRONTEND=noninteractive
-
-WORKDIR /
-RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-
-# Fuego python dependencies
-# - python-lxml: ftc, loggen
-# - python-simplejson: ftc
-# - python-yaml: ftc
-# - python-openpyxl: ftc (also LTP)
-# - python-requests: ftc (also fuego_release_test)
-# - python-reportlab: ftc
-# - python-parsedatetime: ftc
-# - python-pexpect: ssh_exec (part of ttc)
-# - python-pip: to install filelock, flake8
-# - filelock: parser
-RUN mkdir -p /usr/share/man/man1
-RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pexpect python-pip python-setuptools python-wheel
-RUN pip install filelock
-
-# Fuego command dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion iproute2
-
-# Default SDK for testing locally or on an x86 board
-RUN apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
-
-# Default test host dependencies
-# - iperf iperf3 netperf: used as servers on the host
-# - bzip2 bc: used for local tests by Functional.bzip2/bc
-# - python-matplotlib: Benchmark.iperf3 parser
-# - python-xmltodict: AGL tests
-# - flake8: Functional.fuego_lint
-# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
-# - iputils-ping - for /bin/ping command, used by some tests
-# FIXTHIS: install dependencies dynamically on the tests that need them
-RUN apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
- netpipe-tcp iputils-ping
-RUN pip install flake8
-
-# miscelaneous packages:
-# python-serial - used by serio
-# diffstat and vim - used by Tim
-# time - useful for timing command duration
-RUN apt-get -q=2 -V --no-install-recommends install \
- python-serial \
- diffstat \
- vim \
- time
-
-# install packages used by NuttX SDK
-RUN apt-get -q=2 -V --no-install-recommends install \
- genromfs
-
-# FIXTHIS: determine if these tools are really necessary
-#RUN apt-get -q=2 -V --no-install-recommends install \
-# apt-utils python-paramiko \
-# xmlstarlet rsync \
-# inotify-tools gettext netpipe-tcp \
-# at minicom lzop bsdmainutils \
-# mc netcat openssh-server
-
-# bsdmainutils provides hexdump, which is needed for netperf test
-RUN apt-get -q=2 -V --no-install-recommends install \
- bsdmainutils
-
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
-
-# ==============================================================================
-# Install Jenkins with the same UID/GID as the host user
-# ==============================================================================
+RUN mkdir /fuego
+COPY . /fuego/
+WORKDIR /fuego
-ARG user=jenkins
-ARG group=jenkins
-ARG uid=1000
-ARG gid=${uid}
ARG JENKINS_PORT=8090
-# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
-#ARG JENKINS_VERSION=2.164.2
-#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
-#ARG JENKINS_VERSION=2.235.3
-#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
-ARG JENKINS_VERSION=2.249.3
-ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
-
-ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
-ARG JENKINS_UC=https://updates.jenkins.io
-ARG REF=/var/lib/jenkins/plugins
+ARG NO_JENKINS=
+ARG SUITE=stretch
+ARG DIST=debian
+ARG uid=
+ARG gid=
ENV JENKINS_HOME=/var/lib/jenkins
ENV JENKINS_PORT=$JENKINS_PORT
+RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS
-# Jenkins dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- default-jdk daemon psmisc adduser procps unzip
-RUN pip install python-jenkins==1.4.0
-
-RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
-RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
-RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN wget -nv ${JENKINS_URL}
-RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
-# allow Jenkins to start and install plugins, as part of dpkg installation
-RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
-RUN rm jenkins_${JENKINS_VERSION}_all.deb
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-# as of ttc version 2.0.2, this should not be needed
-#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# get ebf script and helpers
-# ==============================================================================
-RUN apt-get -q=2 -V --no-install-recommends install jq
-RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
- cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
- chmod a+x /usr/local/bin/ebf
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-
-RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
-COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
- frontend-install/0002-Output-data-from-port-during-command-execution.patch \
- /tmp/
-RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
- patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
- chown -R jenkins /usr/local/src/serio ; \
- cp /usr/local/src/serio/serio /usr/local/bin/ ; \
- ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
- ln -s /usr/local/bin/serio /usr/local/bin/sersh'
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
- ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
-
-# ==============================================================================
-# Jenkins post installation
-# ==============================================================================
-
-RUN source /etc/default/jenkins && \
- JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
- sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
-
-RUN source /etc/default/jenkins && \
- JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
- if [ -n "$HTTP_PROXY" ]; then \
- PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
- PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
- JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
- fi && \
- sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
-
-RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
-
-# set up Jenkins plugins
-COPY frontend-install/install-plugins.sh \
- frontend-install/jenkins-support \
- frontend-install/clitest \
- /usr/local/bin/
-
-# start Jenkins onces to initialize directories?
-RUN service jenkins start && \
- sleep 30 && \
- service jenkins stop
-
-# install plugins - these versions are for Jenkins version 2.164.2
-# Explicitly install script-security v1.68, otherwise
-# v1.74 will automatically be installed as a dependency of
-# the junit plugin. Make sure to install before junit plugin.
-# Explicitly install junit:1.27
-
-# install other plugins from Jenkins update center
-# NOTE: not sure all of these are needed, but keep list
-# as compatible as possible with 1.2.1 release for now
-# Do NOT change the order of the plugins, unless you know what
-# you are doing. Otherwise, the install-plugins.sh script will
-# install the wrong versions of other plugins based on dependency
-# information in the plugin files.
-
-#RUN /usr/local/bin/install-plugins.sh \
-# script-security:1.68 \
-# structs:1.20 \
-# workflow-step-api:2.22 \
-# workflow-api:2.40 \
-# junit:1.27 \
-# scm-api:2.6.3 \
-# ant:1.9 \
-# antisamy-markup-formatter:1.5 \
-# bouncycastle-api:2.17 \
-# command-launcher:1.3 \
-# description-setter:1.10 \
-# display-url-api:2.3.1 \
-# external-monitor-job:1.7 \
-# greenballs:1.15 \
-# icon-shim:2.0.3 \
-# javadoc:1.5 \
-# jdk-tool:1.2 \
-# ldap:1.20 \
-# mailer:1.23 \
-# matrix-auth:2.3 \
-# matrix-project:1.14 \
-# pam-auth:1.5 \
-# pegdown-formatter:1.3 \
-# windows-slaves:1.4
-
-# install plugins for latest LTS Jenkins (currently 2.249.3)
-RUN /usr/local/bin/install-plugins.sh \
- script-security \
- structs \
- workflow-step-api \
- workflow-api \
- junit \
- scm-api \
- ant \
- antisamy-markup-formatter \
- bouncycastle-api \
- command-launcher \
- description-setter \
- display-url-api \
- external-monitor-job \
- greenballs \
- icon-shim \
- javadoc \
- jdk-tool \
- mailer \
- matrix-auth \
- matrix-project \
- pam-auth \
- pegdown-formatter \
- windows-slaves
+COPY entrypoint.sh /usr/bin/
-COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
-
-# jenkins should automatically unzip any plugins in the plugin dir
-# make a symlink for mod.js well after flot is installed
-RUN service jenkins start && sleep 30 && \
- rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
- ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-
-RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
-
-COPY frontend-install/config.xml $JENKINS_HOME/config.xml
-COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-
-RUN chown -R jenkins:jenkins $JENKINS_HOME/
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-
-RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
-COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-RUN echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
+# Delete the files that are copied for installation
+RUN rm -rf /fuego
+WORKDIR /
# ==============================================================================
# Setup startup command
# ==============================================================================
-
# FIXTHIS: when running multiple Fuego containers, or if the host is already
# running the netperf server, netperf will complain because the port is taken
-ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
+ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
deleted file mode 100644
index 56de98a..0000000
--- a/Dockerfile.nojenkins
+++ /dev/null
@@ -1,151 +0,0 @@
-# ==============================================================================
-# WARNING: this Dockerfile assumes that the container will be created with
-# several bind mounts (see docker-create-container.sh)
-# ==============================================================================
-# Usage:
-# host$ vi fuego-ro/conf/fuego.conf
-# jenkins_enabled=0
-# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
-# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
-# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
-# docker# ftc list-boards
-# docker, bbb, ...
-# docker# ftc list-tests
-# Functional.hello_world, ...
-# docker# ftc run-test -b docker -t Functional.hello_world
-# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
-
-FROM debian:stretch-slim
-MAINTAINER tim.bird@sony.com
-
-# ==============================================================================
-# Proxy variables
-# ==============================================================================
-
-ARG HTTP_PROXY
-ENV http_proxy ${HTTP_PROXY}
-ENV https_proxy ${HTTP_PROXY}
-
-# ==============================================================================
-# Prepare basic image
-# ==============================================================================
-
-ARG DEBIAN_FRONTEND=noninteractive
-
-WORKDIR /
-RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-
-# Fuego python dependencies
-# - python-lxml: ftc, loggen
-# - python-simplejson: ftc
-# - python-yaml: ftc
-# - python-openpyxl: ftc (also LTP)
-# - python-requests: ftc (also fuego_release_test)
-# - python-reportlab: ftc
-# - python-parsedatetime: ftc
-# - python-pexpect: ssh_exec (part of ttc)
-# - python-pip: to install filelock, flake8
-# - filelock: parser
-RUN mkdir -p /usr/share/man/man1
-RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pexpect python-pip python-setuptools python-wheel
-RUN pip install filelock
-
-# Fuego command dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion iproute2
-
-# Default SDK for testing locally or on an x86 board
-RUN apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
-
-# Default test host dependencies
-# - iperf iperf3 netperf: used as servers on the host
-# - bzip2 bc: used for local tests by Functional.bzip2/bc
-# - python-matplotlib: Benchmark.iperf3 parser
-# - python-xmltodict: AGL tests
-# - flake8: Functional.fuego_lint
-# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
-# - iputils-ping - for /bin/ping command, used by some tests
-# FIXTHIS: install dependencies dynamically on the tests that need them
-RUN apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
- netpipe-tcp iputils-ping
-RUN pip install flake8
-
-# miscelaneous packages:
-# python-serial - used by serio
-# diffstat and vim - used by Tim
-# time - useful for timing command duration
-RUN apt-get -q=2 -V --no-install-recommends install \
- python-serial \
- diffstat \
- vim \
- time
-
-# FIXTHIS: determine if these tools are really necessary
-# RUN apt-get -q=2 -V --no-install-recommends install \
-# apt-utils python-paramiko \
-# xmlstarlet rsync \
-# inotify-tools gettext netpipe-tcp \
-# at minicom lzop bsdmainutils \
-# mc netcat openssh-server
-
-# bsdmainutils provides hexdump, which is needed for netperf test
-RUN apt-get -q=2 -V --no-install-recommends install \
- bsdmainutils
-
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-# as of ttc version 2.0.2, this should not be needed
-#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-
-RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
-COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-RUN echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-
-# ==============================================================================
-# Setup startup command
-# ==============================================================================
-
-# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
-ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
diff --git a/README b/README
index 882d84e..ce42e1e 100644
--- a/README
+++ b/README
@@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins
by passing the corresponding parameters to install.sh.
./install.sh fuego-8082 8082
+To run fuego on different distribution versions like Debian(stretch, buster, etc.)
+ pass appropriate arguments to the install.sh script
+ ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
+
+By default install.sh script will install fuego on Debian stretch version
+ to install fuego on Debian buster version pass below options
+ ./install.sh fuego_buster 8090 debian buster
+
+[Note] It is currently verified on Debian versions (stretch and buster),
+and yet to provide install scripts for other distributions
+
+Installing Fuego to a host system
+-------------------------------------------------------------------------------
+Run install-fuego.sh to install fuego on host system, by default it assume the
+host system is Debian stretch
+ ./install-fuego.sh
+
+ to install on Debian buster version
+ ./install-fuego.sh -d debian -s buster -p 8090
+
+ to install without jenkins pass --nojenkins option to the script
+ ./install-fuego.sh --nojenkins
+
Running
-------------------------------------------------------------------------------
To start your Fuego container, issue the following command top directory:
@@ -85,9 +108,9 @@ Updating the Jenkins version
-------------------------------------------------------------------------------
Modify the version and checksum
- $ vi Dockerfile
- ARG JENKINS_VERSION=2.32.1
- ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
+ $ vi install-debian.sh
+ JENKINS_VERSION=2.32.1
+ JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
Re-build the flot plugin
$ sudo apt-get install maven
diff --git a/entrypoint.sh b/entrypoint.sh
new file mode 100755
index 0000000..6eac895
--- /dev/null
+++ b/entrypoint.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+[ "$1" != "--nojenkins" ] && service jenkins start
+
+#TODO: netperf not present in buster, remove this condition once alternative found
+[ -f /etc/init.d/netperf ] && service netperf start
+iperf3 -V -s -D -f M
diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
index 5823357..6889c47 100755
--- a/fuego-host-scripts/docker-build-image.sh
+++ b/fuego-host-scripts/docker-build-image.sh
@@ -2,9 +2,10 @@
# $1 - name for the docker image (default: fuego)
# $2 - port for jenkins (default: 8090)
# $3 - Dockerfile or Dockerfile.nojenkins
+# $4 - Debian code name on which to install (default: stretch)
#
# Example:
-# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins
+# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins
#
if [ "$1" = "--no-cache" ]; then
NO_CACHE=--no-cache
@@ -13,7 +14,10 @@ fi
DOCKERIMAGE=${1:-fuego}
JENKINS_PORT=${2:-8090}
-DOCKERFILE=${3:-Dockerfile}
+DISTRIBUTION=${3:-debian}
+SUITE=${4:-stretch}
+NO_JENKINS=${5:""}
+DOCKERFILE="Dockerfile"
if [ "$(id -u)" == "0" ]; then
JENKINS_UID=$(id -u $SUDO_USER)
@@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
--build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
+ --build-arg DIST=$DISTRIBUTION \
+ --build-arg SUITE=$SUITE \
+ --build-arg NO_JENKINS=$NO_JENKINS \
--build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
diff --git a/install-buster.sh b/install-buster.sh
new file mode 100755
index 0000000..f4ce65b
--- /dev/null
+++ b/install-buster.sh
@@ -0,0 +1,294 @@
+#!/bin/bash
+# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
+#
+# Usage:
+# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
+#
+usage(){
+
+cat <<HERE
+Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
+
+Install fuego on the host Debian filesystem.
+
+options:
+ --help Show usage help
+ --nojenkins Install Fuego without Jenkins
+HERE
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+nojenkins=0
+
+POSITIONAL=()
+while [[ $# -gt 0 ]] ; do
+ case $1 in
+ --nojenkins)
+ nojenkins=1
+ shift
+ ;;
+ --help | -h)
+ usage
+ exit 0
+ ;;
+ *)
+ POSITIONAL+=("$1") # save argument for later
+ shift
+ esac
+done
+set -- "${POSITIONAL[@]}" # restore positional arguments
+
+port=${2:-8090}
+
+# ==============================================================================
+# Install Fuego dependencies
+# ==============================================================================
+
+mkdir -p /usr/share/man/man1
+
+# netperf is in non-free
+echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list
+
+apt-get update -q=2
+
+# Fuego python dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ python-lxml python-simplejson python-yaml python-openpyxl \
+ python-requests python-reportlab python-parsedatetime \
+ python-pexpect python-pip python-setuptools python-wheel
+pip install filelock
+
+# Fuego command dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ git sshpass openssh-client sudo net-tools wget curl lava-tool \
+ bash-completion iproute2
+
+# Default SDK for testing locally or on an x86 board
+apt-get -q=2 -V --no-install-recommends install \
+ build-essential cmake bison flex kmod automake libtool \
+ libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
+
+# Default test host dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
+pip install flake8
+
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+#apt-get -q=2 -V --no-install-recommends install \
+# bsdmainutils
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+
+# TODO: check if proxy settings can set outside docker container
+if [ -n "$http_proxy" ]; then
+ sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
+ echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+fi
+
+# ==============================================================================
+# Install Jenkins
+# ==============================================================================
+
+if [ $nojenkins -eq 0 ]; then
+ #JENKINS_VERSION=2.164.2
+ #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
+ #JENKINS_VERSION=2.249.3
+ #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
+ JENKINS_VERSION=2.164.3
+ JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
+ JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
+ export JENKINS_UC=https://updates.jenkins.io
+ REF=/var/lib/jenkins/plugins
+ JENKINS_HOME=/var/lib/jenkins
+ JENKINS_PORT=$port
+
+ # Jenkins dependencies
+ apt-get -q=2 -V --no-install-recommends install \
+ default-jdk daemon psmisc adduser procps unzip
+ pip install python-jenkins==1.4.0
+
+ echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
+ export JENKINS_PORT=$port
+
+ groupadd jenkins
+ useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
+
+ wget -nv ${JENKINS_URL}
+ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
+
+ # allow Jenkins to start and install plugins, as part of dpkg installation
+ printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
+ dpkg -i jenkins_${JENKINS_VERSION}_all.deb
+ rm jenkins_${JENKINS_VERSION}_all.deb
+
+ # fix for Jenkins Java version check error
+ JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
+ sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
+
+ source /etc/default/jenkins && \
+ JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
+ sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+ source /etc/default/jenkins && \
+ JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
+ if [ -n "$http_proxy" ]; then \
+ PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
+ fi && \
+ sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
+
+ sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
+
+ # set up Jenkins plugins
+ cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
+ /usr/local/bin/
+
+ mkdir -p $JENKINS_HOME/userContent/docs
+ cp frontend-install/config.xml $JENKINS_HOME/config.xml
+ ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+ cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ chown -R jenkins:jenkins $JENKINS_HOME/
+
+ # start and stop jenkins to pre-populate some settings
+ service jenkins start && \
+ sleep 30 && \
+ service jenkins stop
+
+ # install other plugins from Jenkins update center
+ # NOTE: not sure all of these are needed, but keep list
+ # compatible with 1.2.1 release for now
+ #/usr/local/bin/install-plugins.sh \
+ # ant:1.9 \
+ # antisamy-markup-formatter:1.5 \
+ # bouncycastle-api:2.17 \
+ # command-launcher:1.3 \
+ # description-setter:1.10 \
+ # display-url-api:2.3.1 \
+ # external-monitor-job:1.7 \
+ # greenballs:1.15 \
+ # icon-shim:2.0.3 \
+ # javadoc:1.5 \
+ # jdk-tool:1.2 \
+ # junit:1.27 \
+ # ldap:1.20 \
+ # mailer:1.23 \
+ # matrix-auth:2.3 \
+ # matrix-project:1.14 \
+ # pam-auth:1.4 \
+ # pegdown-formatter:1.3 \
+ # structs:1.17 \
+ # windows-slaves:1.4
+
+ # install plugins for latest LTS Jenkins (currently 2.249.3)
+ /usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+ cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+ # make the mod.js symlink well after flot is installed
+ service jenkins start && sleep 30 && \
+ rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
+ ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+
+ chown -R jenkins:jenkins $JENKINS_HOME/
+fi
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+else
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+fi
+cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
+
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+apt-get -q=2 -V --no-install-recommends install jq
+git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+chmod a+x /usr/local/bin/ebf
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# ftc post installation
+# ==============================================================================
+ln -s /fuego-core/scripts/ftc /usr/local/bin/
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+echo ". /etc/bash_completion" >> /root/.bashrc
+
+# ==============================================================================
+# Lava
+# ==============================================================================
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
diff --git a/install-debian.sh b/install-debian.sh
index ccca032..2d3ffd3 100755
--- a/install-debian.sh
+++ b/install-debian.sh
@@ -2,102 +2,163 @@
# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
#
# Usage:
-# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
+# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]
#
-if [ -n "$1" ]; then
- if [ "$1" = "--help" -o "$1" = "-h" ]; then
- cat <<HERE
-Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
-
-Installs fuego on the host Debian filesystem.
-
-options:
- --help Show usage help
- --nojenkins Install Fuego without Jenkins
-HERE
- exit 0
- fi
-fi
+
+usage(){
+ echo ""
+ echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
+ echo "Install fuego on Debian filesystem."
+ echo ""
+ echo "options:"
+ echo " --help | -h Show usage help"
+ echo " --suite | -s <suite> Specify Debian suite"
+ echo " --port | -p <port> Jenkins port"
+ echo " --nojenkins Install Fuego without Jenkins"
+ echo ""
+}
if [[ $EUID -ne 0 ]]; then
echo "Sorry, you need root permissions"
exit 1
fi
-if [ "$1" = "--nojenkins" ]; then
- nojenkins=1
+nojenkins=0
+port=8090
+suite=stretch
+user=jenkins
+group=jenkins
+uid=1000
+gid=1000
+
+while [ "$1" != "" ]; do
+ case $1 in
+ -s | --suite )
+ shift
+ suite=$1
+ ;;
+ -p | --port )
+ shift
+ port=$1
+ ;;
+ -u | --uid )
+ shift
+ uid=$1
+ ;;
+ -g | --gid )
+ shift
+ gid=$1
+ ;;
+ --nojenkins )
+ nojenkins=1
+ ;;
+ -h | --help )
+ usage
+ exit 0
+ ;;
+ * )
+ break
+ esac
shift
-else
- nojenkins=0
- port=${1:-8090}
-fi
+done
# ==============================================================================
# Install Fuego dependencies
# ==============================================================================
# netperf is in non-free
-echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-
-apt-get update -q=2
+echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list
# Fuego python dependencies
-apt-get -q=2 -V --no-install-recommends install \
+# - python-lxml: ftc, loggen
+# - python-simplejson: ftc
+# - python-yaml: ftc
+# - python-openpyxl: ftc (also LTP)
+# - python-requests: ftc (also fuego_release_test)
+# - python-reportlab: ftc
+# - python-parsedatetime: ftc
+# - python-pexpect: ssh_exec (part of ttc)
+# - python-pip: to install filelock, flake8
+# - filelock: parser
+mkdir -p /usr/share/man/man1
+apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
python-lxml python-simplejson python-yaml python-openpyxl \
python-requests python-reportlab python-parsedatetime \
- python-pip
+ python-pexpect python-pip python-setuptools python-wheel
pip install filelock
# Fuego command dependencies
apt-get -q=2 -V --no-install-recommends install \
git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion
+ bash-completion iproute2
# Default SDK for testing locally or on an x86 board
apt-get -q=2 -V --no-install-recommends install \
build-essential cmake bison flex automake kmod libtool \
libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
# Default test host dependencies
+# - iperf iperf3 netperf: used as servers on the host
+# - bzip2 bc: used for local tests by Functional.bzip2/bc
+# - python-matplotlib: Benchmark.iperf3 parser
+# - python-xmltodict: AGL tests
+# - flake8: Functional.fuego_lint
+# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
+# - iputils-ping - for /bin/ping command, used by some tests
+# FIXTHIS: install dependencies dynamically on the tests that need them
apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
-pip install flake8 bsdmainutils
+ iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
-echo "dash dash/sh boolean false" | debconf-set-selections
-DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
-if [ -n "$http_proxy" ]; then
- sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
- echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+#TODO: find alternate package for netperf in buster version
+if [ $suite != "buster" ];then
+ apt-get -q=2 -V --no-install-recommends install netperf
fi
+pip install flake8
-# ==============================================================================
-# Clone fuego and fuego-core
-# ==============================================================================
-if [ ! -d "/fuego" ]; then
- cd /
- git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
- ln -s /fuego/fuego-ro /fuego-ro
- ln -s /fuego/fuego-rw /fuego-rw
- cd fuego
- git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
- ln -s /fuego/fuego-core /fuego-core
-fi
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# FIXTHIS: determine if these tools are really necessary
+#RUN apt-get -q=2 -V --no-install-recommends install \
+# apt-utils python-paramiko \
+# xmlstarlet rsync \
+# inotify-tools gettext netpipe-tcp \
+# at minicom lzop bsdmainutils \
+# mc netcat openssh-server
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+apt-get -q=2 -V --no-install-recommends install \
+ bsdmainutils
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
# ==============================================================================
# Install Jenkins
# ==============================================================================
-
if [ $nojenkins -eq 0 ]; then
#JENKINS_VERSION=2.164.2
#JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
JENKINS_VERSION=2.249.3
JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
- JENKINS_UC=https://updates.jenkins.io
- REF=/var/lib/jenkins/plugins
- JENKINS_HOME=/var/lib/jenkins
+ export JENKINS_UC=https://updates.jenkins.io
+ export REF=/var/lib/jenkins/plugins
+ export JENKINS_HOME=/var/lib/jenkins
JENKINS_PORT=$port
# Jenkins dependencies
@@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
pip install python-jenkins==1.4.0
echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
- groupadd jenkins
- useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
- cd
- wget ${JENKINS_URL}
+ export JENKINS_PORT=$port
+
+ getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
+ useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
+
+ wget -nv ${JENKINS_URL}
echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
# allow Jenkins to start and install plugins, as part of dpkg installation
@@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
dpkg -i jenkins_${JENKINS_VERSION}_all.deb
rm jenkins_${JENKINS_VERSION}_all.deb
- # update ownership
- chown -R jenkins:jenkins /fuego
- chown -R jenkins:jenkins /fuego-ro
- chown -R jenkins:jenkins /fuego-rw
- chown -R jenkins:jenkins /fuego-core
+fi
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+# as of ttc version 2.0.2, this should not be needed
+#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ apt-get -q=2 -V --no-install-recommends install jq
+ git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+ cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+ chmod a+x /usr/local/bin/ebf
+fi
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+git clone https://github.com/frowand/serio.git /usr/local/src/serio
+if [ $nojenkins -eq 0 ]; then
+ cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+ patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
+ chown -R jenkins /usr/local/src/serio
+fi
+cp /usr/local/src/serio/serio /usr/local/bin/
+ln -s /usr/local/bin/serio /usr/local/bin/sercp
+ln -s /usr/local/bin/serio /usr/local/bin/sersh
+
+git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
+[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
+cp /usr/local/src/serlogin/serlogin /usr/local/bin
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# Jenkins post installation
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+
+ # Fix jenkins issue, Java version check fails when java version is 11
+ JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-\]*\\\).*\\\".*/\\\1\\\2\/p;')"
+ sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
source /etc/default/jenkins && \
JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
@@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
- cp /fuego/frontend-install/install-plugins.sh \
- /fuego/frontend-install/jenkins-support \
- /fuego/frontend-install/clitest \
+ # set up Jenkins plugins
+ cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
/usr/local/bin/
- cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
- ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
- mkdir $JENKINS_HOME/userContent/docs
- cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
- jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
- sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
- chown -R jenkins:jenkins $JENKINS_HOME/
-
# start and stop jenkins to pre-populate some settings
service jenkins start && \
sleep 30 && \
service jenkins stop
+ # install plugins - these versions are for Jenkins version 2.164.2
+ # Explicitly install script-security v1.68, otherwise
+ # v1.74 will automatically be installed as a dependency of
+ # the junit plugin. Make sure to install before junit plugin.
+ # Explicitly install junit:1.27
+
# install other plugins from Jenkins update center
# NOTE: not sure all of these are needed, but keep list
- # compatible with 1.2.1 release for now
- #/usr/local/bin/install-plugins.sh \
- # ant:1.9 \
- # antisamy-markup-formatter:1.5 \
- # bouncycastle-api:2.17 \
- # command-launcher:1.3 \
- # description-setter:1.10 \
- # display-url-api:2.3.1 \
- # external-monitor-job:1.7 \
- # greenballs:1.15 \
- # icon-shim:2.0.3 \
- # javadoc:1.5 \
- # jdk-tool:1.2 \
- # junit:1.27 \
- # ldap:1.20 \
- # mailer:1.23 \
- # matrix-auth:2.3 \
- # matrix-project:1.14 \
- # pam-auth:1.4 \
- # pegdown-formatter:1.3 \
- # structs:1.17 \
- # windows-slaves:1.4
-
- # install plugins for latest LTS Jenkins (currently 2.249.3)
- /usr/local/bin/install-plugins.sh \
- script-security \
- structs \
- workflow-step-api \
- workflow-api \
- junit \
- scm-api \
- ant \
- antisamy-markup-formatter \
- bouncycastle-api \
- command-launcher \
- description-setter \
- display-url-api \
- external-monitor-job \
- greenballs \
- icon-shim \
- javadoc \
- jdk-tool \
- mailer \
- matrix-auth \
- matrix-project \
- pam-auth \
- pegdown-formatter \
- windows-slaves
-
- cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+ # as compatible as possible with 1.2.1 release for now
+ # Do NOT change the order of the plugins, unless you know what
+ # you are doing. Otherwise, the install-plugins.sh script will
+ # install the wrong versions of other plugins based on dependency
+ # information in the plugin files.
+ #/usr/local/bin/install-plugins.sh \
+ # script-security:1.68 \
+ # structs:1.20 \
+ # workflow-step-api:2.22 \
+ # workflow-api:2.40 \
+ # junit:1.27 \
+ # scm-api:2.6.3 \
+ # ant:1.9 \
+ # antisamy-markup-formatter:1.5 \
+ # bouncycastle-api:2.17 \
+ # command-launcher:1.3 \
+ # description-setter:1.10 \
+ # display-url-api:2.3.1 \
+ # external-monitor-job:1.7 \
+ # greenballs:1.15 \
+ # icon-shim:2.0.3 \
+ # javadoc:1.5 \
+ # jdk-tool:1.2 \
+ # ldap:1.20 \
+ # mailer:1.23 \
+ # matrix-auth:2.3 \
+ # matrix-project:1.14 \
+ # pam-auth:1.5 \
+ # pegdown-formatter:1.3 \
+ # windows-slaves:1.4
+
+ # install plugins for latest LTS Jenkins (currently 2.249.3)
+ /usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+ cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+ # jenkins should automatically unzip any plugins in the plugin dir
# make the mod.js symlink well after flot is installed
service jenkins start && sleep 30 && \
rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
- chown -R jenkins:jenkins $JENKINS_HOME/
-else
- sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
-fi
+ mkdir -p $JENKINS_HOME/userContent/docs
+ ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-/usr/local/src/ttc/install.sh /usr/local/bin
-perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+ cp frontend-install/config.xml $JENKINS_HOME/config.xml
+ cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-if [ $nojenkins -eq 0 ]; then
- /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
- /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-else
- /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
- /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+ chown -R jenkins:jenkins $JENKINS_HOME/
fi
-# ==============================================================================
-# fserver
-# ==============================================================================
-git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
-ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
-
# ==============================================================================
# ftc post installation
# ==============================================================================
ln -s /fuego-core/scripts/ftc /usr/local/bin/
-cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
echo ". /etc/bash_completion" >> /root/.bashrc
# ==============================================================================
@@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
# ==============================================================================
ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-
-# ==============================================================================
-# Small guide
-# ==============================================================================
-echo "Run 'service netperf start' to start a netperf server"
-echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
-echo "Run 'ftc list-boards' to see the available boards"
-echo "Run 'ftc list-tests' to see the available tests"
-echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
-echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
-echo "Run 'ftc gen-report' to get results"
diff --git a/install-fuego.sh b/install-fuego.sh
new file mode 100755
index 0000000..7b3e010
--- /dev/null
+++ b/install-fuego.sh
@@ -0,0 +1,109 @@
+#!/bin/bash
+# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
+#
+# Usage:
+# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]
+#
+
+usage(){
+ echo ""
+ echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
+ echo "Install fuego on Debian filesystem."
+ echo ""
+ echo "options:"
+ echo " --help | -h Show usage help"
+ echo " --dist | -d <distribution> Specify distribution"
+ echo " --suite | -s <suite> Specify suite"
+ echo " --port | -p <port> Jenkins port"
+ echo " --nojenkins Install Fuego without Jenkins"
+ echo ""
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+nojenkins=
+port=8090
+dist=debian
+suite=stretch
+fuego_dir=$(pwd)
+
+while [ "$1" != "" ]; do
+ case $1 in
+ -d | --dist )
+ shift
+ dist=$1
+ ;;
+ -s | --suite )
+ shift
+ suite=$1
+ ;;
+ -p | --port )
+ shift
+ port=$1
+ ;;
+ --nojenkins )
+ nojenkins="--nojenkins"
+ ;;
+ -h | --help )
+ usage
+ exit 0
+ ;;
+ * )
+ break
+ esac
+ shift
+done
+
+# ==============================================================================
+# Clone fuego-core
+# ==============================================================================
+if [ ! -f "fuego-core/scripts/ftc" ]; then
+ # set fuego-core branch to same as current fuego branch
+ # get current git branch
+ set -o noglob
+ while IFS=" " read -r part1 part2 ; do
+ if [ $part1 = "*" ] ; then
+ branch=$part2
+ fi
+ done < <(git branch)
+ set +o noglob
+ git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
+fi
+
+# ==============================================================================
+# Install fuego on host system
+# ==============================================================================
+./install-$dist.sh -s $suite -p $port $nojenkins
+
+systemctl daemon-reload
+service jenkins restart
+
+# provide permission to current user
+usermod -aG jenkins $USER
+chmod -R 775 $fuego_dir/fuego-rw
+
+chown -R jenkins:jenkins $fuego_dir/fuego-ro
+chown -R jenkins:jenkins $fuego_dir/fuego-rw
+chown -R jenkins:jenkins $fuego_dir/fuego-core
+
+# ==============================================================================
+# Create Links to fuego folders
+# ==============================================================================
+ln -s $fuego_dir/fuego-ro /
+ln -s $fuego_dir/fuego-rw /
+ln -s $fuego_dir/fuego-core /
+
+# ==============================================================================
+# Small guide
+# ==============================================================================
+echo "Logout and Login current user session to work properly"
+echo "Run 'service netperf start' to start a netperf server"
+echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
+echo "Run 'ftc list-boards' to see the available boards"
+echo "Run 'ftc list-tests' to see the available tests"
+echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
+echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
+echo "Run 'ftc gen-report' to get results"
diff --git a/install-stretch.sh b/install-stretch.sh
new file mode 100755
index 0000000..4f6f724
--- /dev/null
+++ b/install-stretch.sh
@@ -0,0 +1,288 @@
+#!/bin/bash
+# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
+#
+# Usage:
+# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
+#
+usage(){
+
+cat <<HERE
+Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
+
+Install fuego on the host Debian filesystem.
+
+options:
+ --help Show usage help
+ --nojenkins Install Fuego without Jenkins
+HERE
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+nojenkins=0
+
+POSITIONAL=()
+while [[ $# -gt 0 ]] ; do
+ case $1 in
+ --nojenkins)
+ nojenkins=1
+ shift
+ ;;
+ --help | -h)
+ usage
+ exit 0
+ ;;
+ *)
+ POSITIONAL+=("$1") # save argument for later
+ shift
+ esac
+done
+set -- "${POSITIONAL[@]}" # restore positional arguments
+
+port=${2:-8090}
+
+# ==============================================================================
+# Install Fuego dependencies
+# ==============================================================================
+
+mkdir -p /usr/share/man/man1
+
+# netperf is in non-free
+echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
+
+apt-get update -q=2
+
+# Fuego python dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ python-lxml python-simplejson python-yaml python-openpyxl \
+ python-requests python-reportlab python-parsedatetime \
+ python-pexpect python-pip python-setuptools python-wheel
+pip install filelock
+
+# Fuego command dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ git sshpass openssh-client sudo net-tools wget curl lava-tool \
+ bash-completion iproute2
+
+# Default SDK for testing locally or on an x86 board
+apt-get -q=2 -V --no-install-recommends install \
+ build-essential cmake bison flex kmod automake libtool \
+ libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
+
+# Default test host dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
+pip install flake8
+
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+apt-get -q=2 -V --no-install-recommends install \
+ bsdmainutils
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+
+# TODO: check if proxy settings can set outside docker container
+#if [ -n "$http_proxy" ]; then
+# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
+# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+#fi
+
+# ==============================================================================
+# Install Jenkins
+# ==============================================================================
+
+if [ $nojenkins -eq 0 ]; then
+ #JENKINS_VERSION=2.164.2
+ #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
+ JENKINS_VERSION=2.249.3
+ JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
+ JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
+ export JENKINS_UC=https://updates.jenkins.io
+ REF=/var/lib/jenkins/plugins
+ JENKINS_HOME=/var/lib/jenkins
+ JENKINS_PORT=$port
+
+ # Jenkins dependencies
+ apt-get -q=2 -V --no-install-recommends install \
+ default-jdk daemon psmisc adduser procps unzip
+ pip install python-jenkins==1.4.0
+
+ echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
+ export JENKINS_PORT=$port
+
+ groupadd jenkins
+ useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
+
+ wget -nv ${JENKINS_URL}
+ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
+
+ # allow Jenkins to start and install plugins, as part of dpkg installation
+ printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
+ dpkg -i jenkins_${JENKINS_VERSION}_all.deb
+ rm jenkins_${JENKINS_VERSION}_all.deb
+
+ source /etc/default/jenkins && \
+ JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
+ sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+ source /etc/default/jenkins && \
+ JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
+ if [ -n "$http_proxy" ]; then \
+ PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
+ fi && \
+ sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
+
+ sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
+
+ # set up Jenkins plugins
+ cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
+ /usr/local/bin/
+
+ mkdir -p $JENKINS_HOME/userContent/docs
+ cp frontend-install/config.xml $JENKINS_HOME/config.xml
+ ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+ cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ chown -R jenkins:jenkins $JENKINS_HOME/
+
+ # start and stop jenkins to pre-populate some settings
+ service jenkins start && \
+ sleep 30 && \
+ service jenkins stop
+
+ # install other plugins from Jenkins update center
+ # NOTE: not sure all of these are needed, but keep list
+ # compatible with 1.2.1 release for now
+ #/usr/local/bin/install-plugins.sh \
+ # ant:1.9 \
+ # antisamy-markup-formatter:1.5 \
+ # bouncycastle-api:2.17 \
+ # command-launcher:1.3 \
+ # description-setter:1.10 \
+ # display-url-api:2.3.1 \
+ # external-monitor-job:1.7 \
+ # greenballs:1.15 \
+ # icon-shim:2.0.3 \
+ # javadoc:1.5 \
+ # jdk-tool:1.2 \
+ # junit:1.27 \
+ # ldap:1.20 \
+ # mailer:1.23 \
+ # matrix-auth:2.3 \
+ # matrix-project:1.14 \
+ # pam-auth:1.4 \
+ # pegdown-formatter:1.3 \
+ # structs:1.17 \
+ # windows-slaves:1.4
+
+ # install plugins for latest LTS Jenkins (currently 2.249.3)
+ /usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+ cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+ # make the mod.js symlink well after flot is installed
+ service jenkins start && sleep 30 && \
+ rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
+ ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+
+ chown -R jenkins:jenkins $JENKINS_HOME/
+fi
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+else
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+fi
+cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
+
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+apt-get -q=2 -V --no-install-recommends install jq
+git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+chmod a+x /usr/local/bin/ebf
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# ftc post installation
+# ==============================================================================
+ln -s /fuego-core/scripts/ftc /usr/local/bin/
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+echo ". /etc/bash_completion" >> /root/.bashrc
+
+# ==============================================================================
+# Lava
+# ==============================================================================
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
diff --git a/install.sh b/install.sh
index a5a90b0..259c887 100755
--- a/install.sh
+++ b/install.sh
@@ -1,13 +1,13 @@
#!/bin/bash
#
-# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
+# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
#
# $1 is the exit code after usage is shown
usage() {
exit_code="$1"
cat <<HERE
-Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
+Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
Create the docker image and container with the Fuego test distribution.
If no <image_name> is provided, the image will be named 'fuego'.
@@ -34,7 +34,7 @@ fi
priv=0
NOCACHE=""
-dockerfile="Dockerfile"
+NOJENKINS=""
POSITIONAL=()
while [[ $# -gt 0 ]] ; do
@@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
shift
;;
--nojenkins)
- dockerfile="Dockerfile.nojenkins"
+ NOJENKINS="--nojenkins"
shift
;;
*)
@@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
image_name=${1:-fuego}
jenkins_port=${2:-8090}
+distribution=${3:-debian}
+suite=${4:-stretch}
container_name="${image_name}-container"
@@ -79,7 +81,7 @@ fi
set -e
-source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
+source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
if [ "$priv" == "0" ]; then
fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
else
diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
new file mode 100644
index 0000000..b899e7c
--- /dev/null
+++ b/uninstall-fuego.sh
@@ -0,0 +1,47 @@
+#!/bin/bash
+
+# Remove Jenkins plugins
+[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
+
+# uninstall Jenkins
+service jenkins stop
+dpkg -P jenkins
+
+[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
+[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
+[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
+[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
+sed -i '/^JENKINS_PORT=/d' /etc/environment
+
+# uninstall ttc script
+[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
+
+# uninstall ebf script
+[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
+[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
+
+# uninstall serial config
+[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
+[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
+[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
+[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
+[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
+[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
+
+# uninstall fserver
+[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
+[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
+
+# uninstall ftc
+[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
+[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
+
+# remove lava target files
+[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
+[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
+
+# remove links
+[ -d /fuego-ro ] && rm /fuego-ro
+[ -d /fuego-rw ] && rm /fuego-rw
+[ -d /fuego-core ] && rm /fuego-core
+
--
2.20.1
The information contained in this e-mail message and in any
attachments/annexure/appendices is confidential to the
recipient and may contain privileged information.
If you are not the intended recipient, please notify the
sender and delete the message along with any
attachments/annexure/appendices. You should not disclose,
copy or otherwise use the information contained in the
message or any annexure. Any views expressed in this e-mail
are those of the individual sender except where the sender
specifically states them to be the views of
Toshiba Software India Pvt. Ltd. (TSIP),Bangalore.
Although this transmission and any attachments are believed to be
free of any virus or other defect that might affect any computer
system into which it is received and opened, it is the responsibility
of the recipient to ensure that it is virus free and no responsibility
is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or
damage arising in any way from its use.
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-04 13:43 [Fuego] Use common script to install fuego on host or docker system venkata.pyla
@ 2021-06-04 14:25 ` Venkata.Pyla
2021-06-09 3:35 ` Tim.Bird
2021-06-15 0:08 ` Tim.Bird
2 siblings, 0 replies; 8+ messages in thread
From: Venkata.Pyla @ 2021-06-04 14:25 UTC (permalink / raw)
To: Venkata.Pyla, Tim.Bird, tbird20d; +Cc: fuego
Hi Tim & All
I have done some work on install scripts where it uses a common script inside the two installation types we have (docker or host system).
I think some more improvement is required like merge install-<suite>.sh scripts in to one and etc, and I will continue to do it.
Meanwhile if you have any suggestions to this current implementation or something please let me know.
I have also created a [PR](https://bitbucket.org/fuegotest/fuego/pull-requests/4/use-common-script-to-install-fuego-on-host) to review the changes better.
Thanks
Venkata.
-----Original Message-----
From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
Sent: 04 June 2021 19:14
To: Tim.Bird@sony.com; tbird20d@yahoo.com
Cc: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com>; fuego@lists.linuxfoundation.org; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>
Subject: [Fuego] Use common script to install fuego on host or docker system
From: venkata pyla <venkata.pyla@toshiba-tsip.com>
Modified the install scripts to use common scripts for both types of
installation, this way it will help to make sure both the environments
are same and if any future modification will have single place of change.
Now, for installing fuego on docker system the 'install.sh' script will
use common script in Dockefile to install fuego related stuff.
sameway for installing fuego on host system it can use the same common
script to install fuego related stuff.
Below are the common scripts that are used in both the installation types
for each debian distribution
* install-stretch.sh
* install-buster.sh
so finally,
To install fuego on docker system use the below script with options
$ ./install.sh <options> image_name <port> <distribution> <suite>
e.g: ./install.sh debian_stretch 8090 debian stretch
or
./install.sh debian_buster 8090 debian buster
or
./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
To install fuego on host system use the below script with options
$ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
e.g: ./install-fuego.sh -d debian -s stretch -p 8090
or
./install-fuego.sh -d debian -s buster -p 8090
or
./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
---
Dockerfile | 303 ++---------------
Dockerfile.nojenkins | 151 ---------
README | 29 +-
entrypoint.sh | 7 +
fuego-host-scripts/docker-build-image.sh | 11 +-
install-buster.sh | 294 ++++++++++++++++
install-debian.sh | 410 ++++++++++++++---------
install-fuego.sh | 109 ++++++
install-stretch.sh | 288 ++++++++++++++++
install.sh | 12 +-
uninstall-fuego.sh | 47 +++
11 files changed, 1057 insertions(+), 604 deletions(-)
delete mode 100644 Dockerfile.nojenkins
create mode 100755 entrypoint.sh
create mode 100755 install-buster.sh
create mode 100755 install-fuego.sh
create mode 100755 install-stretch.sh
create mode 100644 uninstall-fuego.sh
diff --git a/Dockerfile b/Dockerfile
index 7f63798..ac57cb2 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -2,9 +2,11 @@
# WARNING: this Dockerfile assumes that the container will be created with
# several bind mounts (see docker-create-container.sh)
# ==============================================================================
-# FIXTHIS: build this as an extension of the nonjenkins image
-FROM debian:stretch-slim
+ARG SUITE=stretch
+ARG DIST=debian
+
+FROM ${DIST}:${SUITE}-slim
MAINTAINER tim.bird@sony.com
# ==============================================================================
@@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
ENV https_proxy ${HTTP_PROXY}
# ==============================================================================
-# Prepare basic image
+# Install fuego with the script
# ==============================================================================
-ARG DEBIAN_FRONTEND=noninteractive
-
-WORKDIR /
-RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-
-# Fuego python dependencies
-# - python-lxml: ftc, loggen
-# - python-simplejson: ftc
-# - python-yaml: ftc
-# - python-openpyxl: ftc (also LTP)
-# - python-requests: ftc (also fuego_release_test)
-# - python-reportlab: ftc
-# - python-parsedatetime: ftc
-# - python-pexpect: ssh_exec (part of ttc)
-# - python-pip: to install filelock, flake8
-# - filelock: parser
-RUN mkdir -p /usr/share/man/man1
-RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pexpect python-pip python-setuptools python-wheel
-RUN pip install filelock
-
-# Fuego command dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion iproute2
-
-# Default SDK for testing locally or on an x86 board
-RUN apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
-
-# Default test host dependencies
-# - iperf iperf3 netperf: used as servers on the host
-# - bzip2 bc: used for local tests by Functional.bzip2/bc
-# - python-matplotlib: Benchmark.iperf3 parser
-# - python-xmltodict: AGL tests
-# - flake8: Functional.fuego_lint
-# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
-# - iputils-ping - for /bin/ping command, used by some tests
-# FIXTHIS: install dependencies dynamically on the tests that need them
-RUN apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
- netpipe-tcp iputils-ping
-RUN pip install flake8
-
-# miscelaneous packages:
-# python-serial - used by serio
-# diffstat and vim - used by Tim
-# time - useful for timing command duration
-RUN apt-get -q=2 -V --no-install-recommends install \
- python-serial \
- diffstat \
- vim \
- time
-
-# install packages used by NuttX SDK
-RUN apt-get -q=2 -V --no-install-recommends install \
- genromfs
-
-# FIXTHIS: determine if these tools are really necessary
-#RUN apt-get -q=2 -V --no-install-recommends install \
-# apt-utils python-paramiko \
-# xmlstarlet rsync \
-# inotify-tools gettext netpipe-tcp \
-# at minicom lzop bsdmainutils \
-# mc netcat openssh-server
-
-# bsdmainutils provides hexdump, which is needed for netperf test
-RUN apt-get -q=2 -V --no-install-recommends install \
- bsdmainutils
-
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
-
-# ==============================================================================
-# Install Jenkins with the same UID/GID as the host user
-# ==============================================================================
+RUN mkdir /fuego
+COPY . /fuego/
+WORKDIR /fuego
-ARG user=jenkins
-ARG group=jenkins
-ARG uid=1000
-ARG gid=${uid}
ARG JENKINS_PORT=8090
-# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
-#ARG JENKINS_VERSION=2.164.2
-#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
-#ARG JENKINS_VERSION=2.235.3
-#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
-ARG JENKINS_VERSION=2.249.3
-ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
-
-ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
-ARG JENKINS_UC=https://updates.jenkins.io
-ARG REF=/var/lib/jenkins/plugins
+ARG NO_JENKINS=
+ARG SUITE=stretch
+ARG DIST=debian
+ARG uid=
+ARG gid=
ENV JENKINS_HOME=/var/lib/jenkins
ENV JENKINS_PORT=$JENKINS_PORT
+RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS
-# Jenkins dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- default-jdk daemon psmisc adduser procps unzip
-RUN pip install python-jenkins==1.4.0
-
-RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
-RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
-RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN wget -nv ${JENKINS_URL}
-RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
-# allow Jenkins to start and install plugins, as part of dpkg installation
-RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
-RUN rm jenkins_${JENKINS_VERSION}_all.deb
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-# as of ttc version 2.0.2, this should not be needed
-#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# get ebf script and helpers
-# ==============================================================================
-RUN apt-get -q=2 -V --no-install-recommends install jq
-RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
- cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
- chmod a+x /usr/local/bin/ebf
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-
-RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
-COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
- frontend-install/0002-Output-data-from-port-during-command-execution.patch \
- /tmp/
-RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
- patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
- chown -R jenkins /usr/local/src/serio ; \
- cp /usr/local/src/serio/serio /usr/local/bin/ ; \
- ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
- ln -s /usr/local/bin/serio /usr/local/bin/sersh'
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
- ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
-
-# ==============================================================================
-# Jenkins post installation
-# ==============================================================================
-
-RUN source /etc/default/jenkins && \
- JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
- sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
-
-RUN source /etc/default/jenkins && \
- JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
- if [ -n "$HTTP_PROXY" ]; then \
- PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
- PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
- JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
- fi && \
- sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
-
-RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
-
-# set up Jenkins plugins
-COPY frontend-install/install-plugins.sh \
- frontend-install/jenkins-support \
- frontend-install/clitest \
- /usr/local/bin/
-
-# start Jenkins onces to initialize directories?
-RUN service jenkins start && \
- sleep 30 && \
- service jenkins stop
-
-# install plugins - these versions are for Jenkins version 2.164.2
-# Explicitly install script-security v1.68, otherwise
-# v1.74 will automatically be installed as a dependency of
-# the junit plugin. Make sure to install before junit plugin.
-# Explicitly install junit:1.27
-
-# install other plugins from Jenkins update center
-# NOTE: not sure all of these are needed, but keep list
-# as compatible as possible with 1.2.1 release for now
-# Do NOT change the order of the plugins, unless you know what
-# you are doing. Otherwise, the install-plugins.sh script will
-# install the wrong versions of other plugins based on dependency
-# information in the plugin files.
-
-#RUN /usr/local/bin/install-plugins.sh \
-# script-security:1.68 \
-# structs:1.20 \
-# workflow-step-api:2.22 \
-# workflow-api:2.40 \
-# junit:1.27 \
-# scm-api:2.6.3 \
-# ant:1.9 \
-# antisamy-markup-formatter:1.5 \
-# bouncycastle-api:2.17 \
-# command-launcher:1.3 \
-# description-setter:1.10 \
-# display-url-api:2.3.1 \
-# external-monitor-job:1.7 \
-# greenballs:1.15 \
-# icon-shim:2.0.3 \
-# javadoc:1.5 \
-# jdk-tool:1.2 \
-# ldap:1.20 \
-# mailer:1.23 \
-# matrix-auth:2.3 \
-# matrix-project:1.14 \
-# pam-auth:1.5 \
-# pegdown-formatter:1.3 \
-# windows-slaves:1.4
-
-# install plugins for latest LTS Jenkins (currently 2.249.3)
-RUN /usr/local/bin/install-plugins.sh \
- script-security \
- structs \
- workflow-step-api \
- workflow-api \
- junit \
- scm-api \
- ant \
- antisamy-markup-formatter \
- bouncycastle-api \
- command-launcher \
- description-setter \
- display-url-api \
- external-monitor-job \
- greenballs \
- icon-shim \
- javadoc \
- jdk-tool \
- mailer \
- matrix-auth \
- matrix-project \
- pam-auth \
- pegdown-formatter \
- windows-slaves
+COPY entrypoint.sh /usr/bin/
-COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
-
-# jenkins should automatically unzip any plugins in the plugin dir
-# make a symlink for mod.js well after flot is installed
-RUN service jenkins start && sleep 30 && \
- rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
- ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-
-RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
-
-COPY frontend-install/config.xml $JENKINS_HOME/config.xml
-COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-
-RUN chown -R jenkins:jenkins $JENKINS_HOME/
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-
-RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
-COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-RUN echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
+# Delete the files that are copied for installation
+RUN rm -rf /fuego
+WORKDIR /
# ==============================================================================
# Setup startup command
# ==============================================================================
-
# FIXTHIS: when running multiple Fuego containers, or if the host is already
# running the netperf server, netperf will complain because the port is taken
-ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
+ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
deleted file mode 100644
index 56de98a..0000000
--- a/Dockerfile.nojenkins
+++ /dev/null
@@ -1,151 +0,0 @@
-# ==============================================================================
-# WARNING: this Dockerfile assumes that the container will be created with
-# several bind mounts (see docker-create-container.sh)
-# ==============================================================================
-# Usage:
-# host$ vi fuego-ro/conf/fuego.conf
-# jenkins_enabled=0
-# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
-# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
-# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
-# docker# ftc list-boards
-# docker, bbb, ...
-# docker# ftc list-tests
-# Functional.hello_world, ...
-# docker# ftc run-test -b docker -t Functional.hello_world
-# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
-
-FROM debian:stretch-slim
-MAINTAINER tim.bird@sony.com
-
-# ==============================================================================
-# Proxy variables
-# ==============================================================================
-
-ARG HTTP_PROXY
-ENV http_proxy ${HTTP_PROXY}
-ENV https_proxy ${HTTP_PROXY}
-
-# ==============================================================================
-# Prepare basic image
-# ==============================================================================
-
-ARG DEBIAN_FRONTEND=noninteractive
-
-WORKDIR /
-RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-
-# Fuego python dependencies
-# - python-lxml: ftc, loggen
-# - python-simplejson: ftc
-# - python-yaml: ftc
-# - python-openpyxl: ftc (also LTP)
-# - python-requests: ftc (also fuego_release_test)
-# - python-reportlab: ftc
-# - python-parsedatetime: ftc
-# - python-pexpect: ssh_exec (part of ttc)
-# - python-pip: to install filelock, flake8
-# - filelock: parser
-RUN mkdir -p /usr/share/man/man1
-RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
- python-lxml python-simplejson python-yaml python-openpyxl \
- python-requests python-reportlab python-parsedatetime \
- python-pexpect python-pip python-setuptools python-wheel
-RUN pip install filelock
-
-# Fuego command dependencies
-RUN apt-get -q=2 -V --no-install-recommends install \
- git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion iproute2
-
-# Default SDK for testing locally or on an x86 board
-RUN apt-get -q=2 -V --no-install-recommends install \
- build-essential cmake bison flex automake kmod libtool \
- libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
-
-# Default test host dependencies
-# - iperf iperf3 netperf: used as servers on the host
-# - bzip2 bc: used for local tests by Functional.bzip2/bc
-# - python-matplotlib: Benchmark.iperf3 parser
-# - python-xmltodict: AGL tests
-# - flake8: Functional.fuego_lint
-# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
-# - iputils-ping - for /bin/ping command, used by some tests
-# FIXTHIS: install dependencies dynamically on the tests that need them
-RUN apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
- netpipe-tcp iputils-ping
-RUN pip install flake8
-
-# miscelaneous packages:
-# python-serial - used by serio
-# diffstat and vim - used by Tim
-# time - useful for timing command duration
-RUN apt-get -q=2 -V --no-install-recommends install \
- python-serial \
- diffstat \
- vim \
- time
-
-# FIXTHIS: determine if these tools are really necessary
-# RUN apt-get -q=2 -V --no-install-recommends install \
-# apt-utils python-paramiko \
-# xmlstarlet rsync \
-# inotify-tools gettext netpipe-tcp \
-# at minicom lzop bsdmainutils \
-# mc netcat openssh-server
-
-# bsdmainutils provides hexdump, which is needed for netperf test
-RUN apt-get -q=2 -V --no-install-recommends install \
- bsdmainutils
-
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
-
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-# as of ttc version 2.0.2, this should not be needed
-#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
-
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-
-# ==============================================================================
-# fserver
-# ==============================================================================
-
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
-
-# ==============================================================================
-# ftc post installation
-# ==============================================================================
-
-RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
-COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
-RUN echo ". /etc/bash_completion" >> /root/.bashrc
-
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-
-# ==============================================================================
-# Setup startup command
-# ==============================================================================
-
-# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
-ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
diff --git a/README b/README
index 882d84e..ce42e1e 100644
--- a/README
+++ b/README
@@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins
by passing the corresponding parameters to install.sh.
./install.sh fuego-8082 8082
+To run fuego on different distribution versions like Debian(stretch, buster, etc.)
+ pass appropriate arguments to the install.sh script
+ ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
+
+By default install.sh script will install fuego on Debian stretch version
+ to install fuego on Debian buster version pass below options
+ ./install.sh fuego_buster 8090 debian buster
+
+[Note] It is currently verified on Debian versions (stretch and buster),
+and yet to provide install scripts for other distributions
+
+Installing Fuego to a host system
+-------------------------------------------------------------------------------
+Run install-fuego.sh to install fuego on host system, by default it assume the
+host system is Debian stretch
+ ./install-fuego.sh
+
+ to install on Debian buster version
+ ./install-fuego.sh -d debian -s buster -p 8090
+
+ to install without jenkins pass --nojenkins option to the script
+ ./install-fuego.sh --nojenkins
+
Running
-------------------------------------------------------------------------------
To start your Fuego container, issue the following command top directory:
@@ -85,9 +108,9 @@ Updating the Jenkins version
-------------------------------------------------------------------------------
Modify the version and checksum
- $ vi Dockerfile
- ARG JENKINS_VERSION=2.32.1
- ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
+ $ vi install-debian.sh
+ JENKINS_VERSION=2.32.1
+ JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
Re-build the flot plugin
$ sudo apt-get install maven
diff --git a/entrypoint.sh b/entrypoint.sh
new file mode 100755
index 0000000..6eac895
--- /dev/null
+++ b/entrypoint.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+[ "$1" != "--nojenkins" ] && service jenkins start
+
+#TODO: netperf not present in buster, remove this condition once alternative found
+[ -f /etc/init.d/netperf ] && service netperf start
+iperf3 -V -s -D -f M
diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
index 5823357..6889c47 100755
--- a/fuego-host-scripts/docker-build-image.sh
+++ b/fuego-host-scripts/docker-build-image.sh
@@ -2,9 +2,10 @@
# $1 - name for the docker image (default: fuego)
# $2 - port for jenkins (default: 8090)
# $3 - Dockerfile or Dockerfile.nojenkins
+# $4 - Debian code name on which to install (default: stretch)
#
# Example:
-# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins
+# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins
#
if [ "$1" = "--no-cache" ]; then
NO_CACHE=--no-cache
@@ -13,7 +14,10 @@ fi
DOCKERIMAGE=${1:-fuego}
JENKINS_PORT=${2:-8090}
-DOCKERFILE=${3:-Dockerfile}
+DISTRIBUTION=${3:-debian}
+SUITE=${4:-stretch}
+NO_JENKINS=${5:""}
+DOCKERFILE="Dockerfile"
if [ "$(id -u)" == "0" ]; then
JENKINS_UID=$(id -u $SUDO_USER)
@@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
--build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
+ --build-arg DIST=$DISTRIBUTION \
+ --build-arg SUITE=$SUITE \
+ --build-arg NO_JENKINS=$NO_JENKINS \
--build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
diff --git a/install-buster.sh b/install-buster.sh
new file mode 100755
index 0000000..f4ce65b
--- /dev/null
+++ b/install-buster.sh
@@ -0,0 +1,294 @@
+#!/bin/bash
+# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
+#
+# Usage:
+# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
+#
+usage(){
+
+cat <<HERE
+Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
+
+Install fuego on the host Debian filesystem.
+
+options:
+ --help Show usage help
+ --nojenkins Install Fuego without Jenkins
+HERE
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+nojenkins=0
+
+POSITIONAL=()
+while [[ $# -gt 0 ]] ; do
+ case $1 in
+ --nojenkins)
+ nojenkins=1
+ shift
+ ;;
+ --help | -h)
+ usage
+ exit 0
+ ;;
+ *)
+ POSITIONAL+=("$1") # save argument for later
+ shift
+ esac
+done
+set -- "${POSITIONAL[@]}" # restore positional arguments
+
+port=${2:-8090}
+
+# ==============================================================================
+# Install Fuego dependencies
+# ==============================================================================
+
+mkdir -p /usr/share/man/man1
+
+# netperf is in non-free
+echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list
+
+apt-get update -q=2
+
+# Fuego python dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ python-lxml python-simplejson python-yaml python-openpyxl \
+ python-requests python-reportlab python-parsedatetime \
+ python-pexpect python-pip python-setuptools python-wheel
+pip install filelock
+
+# Fuego command dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ git sshpass openssh-client sudo net-tools wget curl lava-tool \
+ bash-completion iproute2
+
+# Default SDK for testing locally or on an x86 board
+apt-get -q=2 -V --no-install-recommends install \
+ build-essential cmake bison flex kmod automake libtool \
+ libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
+
+# Default test host dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
+pip install flake8
+
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+#apt-get -q=2 -V --no-install-recommends install \
+# bsdmainutils
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+
+# TODO: check if proxy settings can set outside docker container
+if [ -n "$http_proxy" ]; then
+ sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
+ echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+fi
+
+# ==============================================================================
+# Install Jenkins
+# ==============================================================================
+
+if [ $nojenkins -eq 0 ]; then
+ #JENKINS_VERSION=2.164.2
+ #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
+ #JENKINS_VERSION=2.249.3
+ #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
+ JENKINS_VERSION=2.164.3
+ JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
+ JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
+ export JENKINS_UC=https://updates.jenkins.io
+ REF=/var/lib/jenkins/plugins
+ JENKINS_HOME=/var/lib/jenkins
+ JENKINS_PORT=$port
+
+ # Jenkins dependencies
+ apt-get -q=2 -V --no-install-recommends install \
+ default-jdk daemon psmisc adduser procps unzip
+ pip install python-jenkins==1.4.0
+
+ echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
+ export JENKINS_PORT=$port
+
+ groupadd jenkins
+ useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
+
+ wget -nv ${JENKINS_URL}
+ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
+
+ # allow Jenkins to start and install plugins, as part of dpkg installation
+ printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
+ dpkg -i jenkins_${JENKINS_VERSION}_all.deb
+ rm jenkins_${JENKINS_VERSION}_all.deb
+
+ # fix for Jenkins Java version check error
+ JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
+ sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
+
+ source /etc/default/jenkins && \
+ JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
+ sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+ source /etc/default/jenkins && \
+ JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
+ if [ -n "$http_proxy" ]; then \
+ PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
+ fi && \
+ sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
+
+ sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
+
+ # set up Jenkins plugins
+ cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
+ /usr/local/bin/
+
+ mkdir -p $JENKINS_HOME/userContent/docs
+ cp frontend-install/config.xml $JENKINS_HOME/config.xml
+ ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+ cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ chown -R jenkins:jenkins $JENKINS_HOME/
+
+ # start and stop jenkins to pre-populate some settings
+ service jenkins start && \
+ sleep 30 && \
+ service jenkins stop
+
+ # install other plugins from Jenkins update center
+ # NOTE: not sure all of these are needed, but keep list
+ # compatible with 1.2.1 release for now
+ #/usr/local/bin/install-plugins.sh \
+ # ant:1.9 \
+ # antisamy-markup-formatter:1.5 \
+ # bouncycastle-api:2.17 \
+ # command-launcher:1.3 \
+ # description-setter:1.10 \
+ # display-url-api:2.3.1 \
+ # external-monitor-job:1.7 \
+ # greenballs:1.15 \
+ # icon-shim:2.0.3 \
+ # javadoc:1.5 \
+ # jdk-tool:1.2 \
+ # junit:1.27 \
+ # ldap:1.20 \
+ # mailer:1.23 \
+ # matrix-auth:2.3 \
+ # matrix-project:1.14 \
+ # pam-auth:1.4 \
+ # pegdown-formatter:1.3 \
+ # structs:1.17 \
+ # windows-slaves:1.4
+
+ # install plugins for latest LTS Jenkins (currently 2.249.3)
+ /usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+ cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+ # make the mod.js symlink well after flot is installed
+ service jenkins start && sleep 30 && \
+ rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
+ ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+
+ chown -R jenkins:jenkins $JENKINS_HOME/
+fi
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+else
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+fi
+cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
+
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+apt-get -q=2 -V --no-install-recommends install jq
+git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+chmod a+x /usr/local/bin/ebf
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# ftc post installation
+# ==============================================================================
+ln -s /fuego-core/scripts/ftc /usr/local/bin/
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+echo ". /etc/bash_completion" >> /root/.bashrc
+
+# ==============================================================================
+# Lava
+# ==============================================================================
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
diff --git a/install-debian.sh b/install-debian.sh
index ccca032..2d3ffd3 100755
--- a/install-debian.sh
+++ b/install-debian.sh
@@ -2,102 +2,163 @@
# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
#
# Usage:
-# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
+# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]
#
-if [ -n "$1" ]; then
- if [ "$1" = "--help" -o "$1" = "-h" ]; then
- cat <<HERE
-Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
-
-Installs fuego on the host Debian filesystem.
-
-options:
- --help Show usage help
- --nojenkins Install Fuego without Jenkins
-HERE
- exit 0
- fi
-fi
+
+usage(){
+ echo ""
+ echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
+ echo "Install fuego on Debian filesystem."
+ echo ""
+ echo "options:"
+ echo " --help | -h Show usage help"
+ echo " --suite | -s <suite> Specify Debian suite"
+ echo " --port | -p <port> Jenkins port"
+ echo " --nojenkins Install Fuego without Jenkins"
+ echo ""
+}
if [[ $EUID -ne 0 ]]; then
echo "Sorry, you need root permissions"
exit 1
fi
-if [ "$1" = "--nojenkins" ]; then
- nojenkins=1
+nojenkins=0
+port=8090
+suite=stretch
+user=jenkins
+group=jenkins
+uid=1000
+gid=1000
+
+while [ "$1" != "" ]; do
+ case $1 in
+ -s | --suite )
+ shift
+ suite=$1
+ ;;
+ -p | --port )
+ shift
+ port=$1
+ ;;
+ -u | --uid )
+ shift
+ uid=$1
+ ;;
+ -g | --gid )
+ shift
+ gid=$1
+ ;;
+ --nojenkins )
+ nojenkins=1
+ ;;
+ -h | --help )
+ usage
+ exit 0
+ ;;
+ * )
+ break
+ esac
shift
-else
- nojenkins=0
- port=${1:-8090}
-fi
+done
# ==============================================================================
# Install Fuego dependencies
# ==============================================================================
# netperf is in non-free
-echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
-echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
-
-apt-get update -q=2
+echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list
# Fuego python dependencies
-apt-get -q=2 -V --no-install-recommends install \
+# - python-lxml: ftc, loggen
+# - python-simplejson: ftc
+# - python-yaml: ftc
+# - python-openpyxl: ftc (also LTP)
+# - python-requests: ftc (also fuego_release_test)
+# - python-reportlab: ftc
+# - python-parsedatetime: ftc
+# - python-pexpect: ssh_exec (part of ttc)
+# - python-pip: to install filelock, flake8
+# - filelock: parser
+mkdir -p /usr/share/man/man1
+apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
python-lxml python-simplejson python-yaml python-openpyxl \
python-requests python-reportlab python-parsedatetime \
- python-pip
+ python-pexpect python-pip python-setuptools python-wheel
pip install filelock
# Fuego command dependencies
apt-get -q=2 -V --no-install-recommends install \
git sshpass openssh-client sudo net-tools wget curl lava-tool \
- bash-completion
+ bash-completion iproute2
# Default SDK for testing locally or on an x86 board
apt-get -q=2 -V --no-install-recommends install \
build-essential cmake bison flex automake kmod libtool \
libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
- libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
# Default test host dependencies
+# - iperf iperf3 netperf: used as servers on the host
+# - bzip2 bc: used for local tests by Functional.bzip2/bc
+# - python-matplotlib: Benchmark.iperf3 parser
+# - python-xmltodict: AGL tests
+# - flake8: Functional.fuego_lint
+# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
+# - iputils-ping - for /bin/ping command, used by some tests
+# FIXTHIS: install dependencies dynamically on the tests that need them
apt-get -q=2 -V --no-install-recommends install \
- iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
-pip install flake8 bsdmainutils
+ iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
-echo "dash dash/sh boolean false" | debconf-set-selections
-DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
-if [ -n "$http_proxy" ]; then
- sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
- echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+#TODO: find alternate package for netperf in buster version
+if [ $suite != "buster" ];then
+ apt-get -q=2 -V --no-install-recommends install netperf
fi
+pip install flake8
-# ==============================================================================
-# Clone fuego and fuego-core
-# ==============================================================================
-if [ ! -d "/fuego" ]; then
- cd /
- git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
- ln -s /fuego/fuego-ro /fuego-ro
- ln -s /fuego/fuego-rw /fuego-rw
- cd fuego
- git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
- ln -s /fuego/fuego-core /fuego-core
-fi
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# FIXTHIS: determine if these tools are really necessary
+#RUN apt-get -q=2 -V --no-install-recommends install \
+# apt-utils python-paramiko \
+# xmlstarlet rsync \
+# inotify-tools gettext netpipe-tcp \
+# at minicom lzop bsdmainutils \
+# mc netcat openssh-server
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+apt-get -q=2 -V --no-install-recommends install \
+ bsdmainutils
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
# ==============================================================================
# Install Jenkins
# ==============================================================================
-
if [ $nojenkins -eq 0 ]; then
#JENKINS_VERSION=2.164.2
#JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
JENKINS_VERSION=2.249.3
JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
- JENKINS_UC=https://updates.jenkins.io
- REF=/var/lib/jenkins/plugins
- JENKINS_HOME=/var/lib/jenkins
+ export JENKINS_UC=https://updates.jenkins.io
+ export REF=/var/lib/jenkins/plugins
+ export JENKINS_HOME=/var/lib/jenkins
JENKINS_PORT=$port
# Jenkins dependencies
@@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
pip install python-jenkins==1.4.0
echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
- groupadd jenkins
- useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
- cd
- wget ${JENKINS_URL}
+ export JENKINS_PORT=$port
+
+ getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
+ useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
+
+ wget -nv ${JENKINS_URL}
echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
# allow Jenkins to start and install plugins, as part of dpkg installation
@@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
dpkg -i jenkins_${JENKINS_VERSION}_all.deb
rm jenkins_${JENKINS_VERSION}_all.deb
- # update ownership
- chown -R jenkins:jenkins /fuego
- chown -R jenkins:jenkins /fuego-ro
- chown -R jenkins:jenkins /fuego-rw
- chown -R jenkins:jenkins /fuego-core
+fi
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+# as of ttc version 2.0.2, this should not be needed
+#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ apt-get -q=2 -V --no-install-recommends install jq
+ git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+ cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+ chmod a+x /usr/local/bin/ebf
+fi
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+git clone https://github.com/frowand/serio.git /usr/local/src/serio
+if [ $nojenkins -eq 0 ]; then
+ cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+ patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
+ chown -R jenkins /usr/local/src/serio
+fi
+cp /usr/local/src/serio/serio /usr/local/bin/
+ln -s /usr/local/bin/serio /usr/local/bin/sercp
+ln -s /usr/local/bin/serio /usr/local/bin/sersh
+
+git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
+[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
+cp /usr/local/src/serlogin/serlogin /usr/local/bin
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# Jenkins post installation
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+
+ # Fix jenkins issue, Java version check fails when java version is 11
+ JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-\]*\\\).*\\\".*/\\\1\\\2\/p;')"
+ sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
source /etc/default/jenkins && \
JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
@@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
- cp /fuego/frontend-install/install-plugins.sh \
- /fuego/frontend-install/jenkins-support \
- /fuego/frontend-install/clitest \
+ # set up Jenkins plugins
+ cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
/usr/local/bin/
- cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
- ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
- mkdir $JENKINS_HOME/userContent/docs
- cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
- jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
- sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
- chown -R jenkins:jenkins $JENKINS_HOME/
-
# start and stop jenkins to pre-populate some settings
service jenkins start && \
sleep 30 && \
service jenkins stop
+ # install plugins - these versions are for Jenkins version 2.164.2
+ # Explicitly install script-security v1.68, otherwise
+ # v1.74 will automatically be installed as a dependency of
+ # the junit plugin. Make sure to install before junit plugin.
+ # Explicitly install junit:1.27
+
# install other plugins from Jenkins update center
# NOTE: not sure all of these are needed, but keep list
- # compatible with 1.2.1 release for now
- #/usr/local/bin/install-plugins.sh \
- # ant:1.9 \
- # antisamy-markup-formatter:1.5 \
- # bouncycastle-api:2.17 \
- # command-launcher:1.3 \
- # description-setter:1.10 \
- # display-url-api:2.3.1 \
- # external-monitor-job:1.7 \
- # greenballs:1.15 \
- # icon-shim:2.0.3 \
- # javadoc:1.5 \
- # jdk-tool:1.2 \
- # junit:1.27 \
- # ldap:1.20 \
- # mailer:1.23 \
- # matrix-auth:2.3 \
- # matrix-project:1.14 \
- # pam-auth:1.4 \
- # pegdown-formatter:1.3 \
- # structs:1.17 \
- # windows-slaves:1.4
-
- # install plugins for latest LTS Jenkins (currently 2.249.3)
- /usr/local/bin/install-plugins.sh \
- script-security \
- structs \
- workflow-step-api \
- workflow-api \
- junit \
- scm-api \
- ant \
- antisamy-markup-formatter \
- bouncycastle-api \
- command-launcher \
- description-setter \
- display-url-api \
- external-monitor-job \
- greenballs \
- icon-shim \
- javadoc \
- jdk-tool \
- mailer \
- matrix-auth \
- matrix-project \
- pam-auth \
- pegdown-formatter \
- windows-slaves
-
- cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+ # as compatible as possible with 1.2.1 release for now
+ # Do NOT change the order of the plugins, unless you know what
+ # you are doing. Otherwise, the install-plugins.sh script will
+ # install the wrong versions of other plugins based on dependency
+ # information in the plugin files.
+ #/usr/local/bin/install-plugins.sh \
+ # script-security:1.68 \
+ # structs:1.20 \
+ # workflow-step-api:2.22 \
+ # workflow-api:2.40 \
+ # junit:1.27 \
+ # scm-api:2.6.3 \
+ # ant:1.9 \
+ # antisamy-markup-formatter:1.5 \
+ # bouncycastle-api:2.17 \
+ # command-launcher:1.3 \
+ # description-setter:1.10 \
+ # display-url-api:2.3.1 \
+ # external-monitor-job:1.7 \
+ # greenballs:1.15 \
+ # icon-shim:2.0.3 \
+ # javadoc:1.5 \
+ # jdk-tool:1.2 \
+ # ldap:1.20 \
+ # mailer:1.23 \
+ # matrix-auth:2.3 \
+ # matrix-project:1.14 \
+ # pam-auth:1.5 \
+ # pegdown-formatter:1.3 \
+ # windows-slaves:1.4
+
+ # install plugins for latest LTS Jenkins (currently 2.249.3)
+ /usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+ cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+ # jenkins should automatically unzip any plugins in the plugin dir
# make the mod.js symlink well after flot is installed
service jenkins start && sleep 30 && \
rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
- chown -R jenkins:jenkins $JENKINS_HOME/
-else
- sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
-fi
+ mkdir -p $JENKINS_HOME/userContent/docs
+ ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
-# ==============================================================================
-# get ttc script and helpers
-# ==============================================================================
-git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-/usr/local/src/ttc/install.sh /usr/local/bin
-perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+ cp frontend-install/config.xml $JENKINS_HOME/config.xml
+ cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
-# ==============================================================================
-# Serial Config
-# ==============================================================================
-if [ $nojenkins -eq 0 ]; then
- /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
- /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
-else
- /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
- /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+ chown -R jenkins:jenkins $JENKINS_HOME/
fi
-# ==============================================================================
-# fserver
-# ==============================================================================
-git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
-ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
-
# ==============================================================================
# ftc post installation
# ==============================================================================
ln -s /fuego-core/scripts/ftc /usr/local/bin/
-cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
echo ". /etc/bash_completion" >> /root/.bashrc
# ==============================================================================
@@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
# ==============================================================================
ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-
-# ==============================================================================
-# Small guide
-# ==============================================================================
-echo "Run 'service netperf start' to start a netperf server"
-echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
-echo "Run 'ftc list-boards' to see the available boards"
-echo "Run 'ftc list-tests' to see the available tests"
-echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
-echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
-echo "Run 'ftc gen-report' to get results"
diff --git a/install-fuego.sh b/install-fuego.sh
new file mode 100755
index 0000000..7b3e010
--- /dev/null
+++ b/install-fuego.sh
@@ -0,0 +1,109 @@
+#!/bin/bash
+# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
+#
+# Usage:
+# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]
+#
+
+usage(){
+ echo ""
+ echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
+ echo "Install fuego on Debian filesystem."
+ echo ""
+ echo "options:"
+ echo " --help | -h Show usage help"
+ echo " --dist | -d <distribution> Specify distribution"
+ echo " --suite | -s <suite> Specify suite"
+ echo " --port | -p <port> Jenkins port"
+ echo " --nojenkins Install Fuego without Jenkins"
+ echo ""
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+nojenkins=
+port=8090
+dist=debian
+suite=stretch
+fuego_dir=$(pwd)
+
+while [ "$1" != "" ]; do
+ case $1 in
+ -d | --dist )
+ shift
+ dist=$1
+ ;;
+ -s | --suite )
+ shift
+ suite=$1
+ ;;
+ -p | --port )
+ shift
+ port=$1
+ ;;
+ --nojenkins )
+ nojenkins="--nojenkins"
+ ;;
+ -h | --help )
+ usage
+ exit 0
+ ;;
+ * )
+ break
+ esac
+ shift
+done
+
+# ==============================================================================
+# Clone fuego-core
+# ==============================================================================
+if [ ! -f "fuego-core/scripts/ftc" ]; then
+ # set fuego-core branch to same as current fuego branch
+ # get current git branch
+ set -o noglob
+ while IFS=" " read -r part1 part2 ; do
+ if [ $part1 = "*" ] ; then
+ branch=$part2
+ fi
+ done < <(git branch)
+ set +o noglob
+ git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
+fi
+
+# ==============================================================================
+# Install fuego on host system
+# ==============================================================================
+./install-$dist.sh -s $suite -p $port $nojenkins
+
+systemctl daemon-reload
+service jenkins restart
+
+# provide permission to current user
+usermod -aG jenkins $USER
+chmod -R 775 $fuego_dir/fuego-rw
+
+chown -R jenkins:jenkins $fuego_dir/fuego-ro
+chown -R jenkins:jenkins $fuego_dir/fuego-rw
+chown -R jenkins:jenkins $fuego_dir/fuego-core
+
+# ==============================================================================
+# Create Links to fuego folders
+# ==============================================================================
+ln -s $fuego_dir/fuego-ro /
+ln -s $fuego_dir/fuego-rw /
+ln -s $fuego_dir/fuego-core /
+
+# ==============================================================================
+# Small guide
+# ==============================================================================
+echo "Logout and Login current user session to work properly"
+echo "Run 'service netperf start' to start a netperf server"
+echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
+echo "Run 'ftc list-boards' to see the available boards"
+echo "Run 'ftc list-tests' to see the available tests"
+echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
+echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
+echo "Run 'ftc gen-report' to get results"
diff --git a/install-stretch.sh b/install-stretch.sh
new file mode 100755
index 0000000..4f6f724
--- /dev/null
+++ b/install-stretch.sh
@@ -0,0 +1,288 @@
+#!/bin/bash
+# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
+#
+# Usage:
+# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
+#
+usage(){
+
+cat <<HERE
+Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
+
+Install fuego on the host Debian filesystem.
+
+options:
+ --help Show usage help
+ --nojenkins Install Fuego without Jenkins
+HERE
+}
+
+if [[ $EUID -ne 0 ]]; then
+ echo "Sorry, you need root permissions"
+ exit 1
+fi
+
+nojenkins=0
+
+POSITIONAL=()
+while [[ $# -gt 0 ]] ; do
+ case $1 in
+ --nojenkins)
+ nojenkins=1
+ shift
+ ;;
+ --help | -h)
+ usage
+ exit 0
+ ;;
+ *)
+ POSITIONAL+=("$1") # save argument for later
+ shift
+ esac
+done
+set -- "${POSITIONAL[@]}" # restore positional arguments
+
+port=${2:-8090}
+
+# ==============================================================================
+# Install Fuego dependencies
+# ==============================================================================
+
+mkdir -p /usr/share/man/man1
+
+# netperf is in non-free
+echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
+echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
+
+apt-get update -q=2
+
+# Fuego python dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ python-lxml python-simplejson python-yaml python-openpyxl \
+ python-requests python-reportlab python-parsedatetime \
+ python-pexpect python-pip python-setuptools python-wheel
+pip install filelock
+
+# Fuego command dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ git sshpass openssh-client sudo net-tools wget curl lava-tool \
+ bash-completion iproute2
+
+# Default SDK for testing locally or on an x86 board
+apt-get -q=2 -V --no-install-recommends install \
+ build-essential cmake bison flex kmod automake libtool \
+ libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
+ libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
+
+# Default test host dependencies
+apt-get -q=2 -V --no-install-recommends install \
+ iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
+ netpipe-tcp iputils-ping
+pip install flake8
+
+# miscelaneous packages:
+# python-serial - used by serio
+# diffstat and vim - used by Tim
+# time - useful for timing command duration
+apt-get -q=2 -V --no-install-recommends install \
+ python-serial \
+ diffstat \
+ vim \
+ time
+
+# install packages used by NuttX SDK
+apt-get -q=2 -V --no-install-recommends install \
+ genromfs
+
+# bsdmainutils provides hexdump, which is needed for netperf test
+apt-get -q=2 -V --no-install-recommends install \
+ bsdmainutils
+
+echo "dash dash/sh boolean false" | debconf-set-selections
+DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+
+# TODO: check if proxy settings can set outside docker container
+#if [ -n "$http_proxy" ]; then
+# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
+# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
+#fi
+
+# ==============================================================================
+# Install Jenkins
+# ==============================================================================
+
+if [ $nojenkins -eq 0 ]; then
+ #JENKINS_VERSION=2.164.2
+ #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
+ JENKINS_VERSION=2.249.3
+ JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
+ JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
+ export JENKINS_UC=https://updates.jenkins.io
+ REF=/var/lib/jenkins/plugins
+ JENKINS_HOME=/var/lib/jenkins
+ JENKINS_PORT=$port
+
+ # Jenkins dependencies
+ apt-get -q=2 -V --no-install-recommends install \
+ default-jdk daemon psmisc adduser procps unzip
+ pip install python-jenkins==1.4.0
+
+ echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
+ export JENKINS_PORT=$port
+
+ groupadd jenkins
+ useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
+
+ wget -nv ${JENKINS_URL}
+ echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
+
+ # allow Jenkins to start and install plugins, as part of dpkg installation
+ printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
+ dpkg -i jenkins_${JENKINS_VERSION}_all.deb
+ rm jenkins_${JENKINS_VERSION}_all.deb
+
+ source /etc/default/jenkins && \
+ JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
+ sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+ source /etc/default/jenkins && \
+ JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
+ if [ -n "$http_proxy" ]; then \
+ PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
+ PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
+ JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
+ fi && \
+ sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
+
+ sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
+
+ # set up Jenkins plugins
+ cp frontend-install/install-plugins.sh \
+ frontend-install/jenkins-support \
+ frontend-install/clitest \
+ /usr/local/bin/
+
+ mkdir -p $JENKINS_HOME/userContent/docs
+ cp frontend-install/config.xml $JENKINS_HOME/config.xml
+ ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+ cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+ chown -R jenkins:jenkins $JENKINS_HOME/
+
+ # start and stop jenkins to pre-populate some settings
+ service jenkins start && \
+ sleep 30 && \
+ service jenkins stop
+
+ # install other plugins from Jenkins update center
+ # NOTE: not sure all of these are needed, but keep list
+ # compatible with 1.2.1 release for now
+ #/usr/local/bin/install-plugins.sh \
+ # ant:1.9 \
+ # antisamy-markup-formatter:1.5 \
+ # bouncycastle-api:2.17 \
+ # command-launcher:1.3 \
+ # description-setter:1.10 \
+ # display-url-api:2.3.1 \
+ # external-monitor-job:1.7 \
+ # greenballs:1.15 \
+ # icon-shim:2.0.3 \
+ # javadoc:1.5 \
+ # jdk-tool:1.2 \
+ # junit:1.27 \
+ # ldap:1.20 \
+ # mailer:1.23 \
+ # matrix-auth:2.3 \
+ # matrix-project:1.14 \
+ # pam-auth:1.4 \
+ # pegdown-formatter:1.3 \
+ # structs:1.17 \
+ # windows-slaves:1.4
+
+ # install plugins for latest LTS Jenkins (currently 2.249.3)
+ /usr/local/bin/install-plugins.sh \
+ script-security \
+ structs \
+ workflow-step-api \
+ workflow-api \
+ junit \
+ scm-api \
+ ant \
+ antisamy-markup-formatter \
+ bouncycastle-api \
+ command-launcher \
+ description-setter \
+ display-url-api \
+ external-monitor-job \
+ greenballs \
+ icon-shim \
+ javadoc \
+ jdk-tool \
+ mailer \
+ matrix-auth \
+ matrix-project \
+ pam-auth \
+ pegdown-formatter \
+ windows-slaves
+
+ cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
+
+ # make the mod.js symlink well after flot is installed
+ service jenkins start && sleep 30 && \
+ rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
+ ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+
+ chown -R jenkins:jenkins $JENKINS_HOME/
+fi
+
+# ==============================================================================
+# get ttc script and helpers
+# ==============================================================================
+git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
+
+# ==============================================================================
+# Serial Config
+# ==============================================================================
+if [ $nojenkins -eq 0 ]; then
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+else
+ /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+ /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+fi
+cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
+ frontend-install/0002-Output-data-from-port-during-command-execution.patch \
+ /tmp/
+/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
+ patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
+
+
+# ==============================================================================
+# get ebf script and helpers
+# ==============================================================================
+apt-get -q=2 -V --no-install-recommends install jq
+git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
+cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
+chmod a+x /usr/local/bin/ebf
+
+# ==============================================================================
+# fserver
+# ==============================================================================
+git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
+ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
+
+# ==============================================================================
+# ftc post installation
+# ==============================================================================
+ln -s /fuego-core/scripts/ftc /usr/local/bin/
+cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
+echo ". /etc/bash_completion" >> /root/.bashrc
+
+# ==============================================================================
+# Lava
+# ==============================================================================
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
diff --git a/install.sh b/install.sh
index a5a90b0..259c887 100755
--- a/install.sh
+++ b/install.sh
@@ -1,13 +1,13 @@
#!/bin/bash
#
-# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
+# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
#
# $1 is the exit code after usage is shown
usage() {
exit_code="$1"
cat <<HERE
-Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
+Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
Create the docker image and container with the Fuego test distribution.
If no <image_name> is provided, the image will be named 'fuego'.
@@ -34,7 +34,7 @@ fi
priv=0
NOCACHE=""
-dockerfile="Dockerfile"
+NOJENKINS=""
POSITIONAL=()
while [[ $# -gt 0 ]] ; do
@@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
shift
;;
--nojenkins)
- dockerfile="Dockerfile.nojenkins"
+ NOJENKINS="--nojenkins"
shift
;;
*)
@@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
image_name=${1:-fuego}
jenkins_port=${2:-8090}
+distribution=${3:-debian}
+suite=${4:-stretch}
container_name="${image_name}-container"
@@ -79,7 +81,7 @@ fi
set -e
-source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
+source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
if [ "$priv" == "0" ]; then
fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
else
diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
new file mode 100644
index 0000000..b899e7c
--- /dev/null
+++ b/uninstall-fuego.sh
@@ -0,0 +1,47 @@
+#!/bin/bash
+
+# Remove Jenkins plugins
+[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
+
+# uninstall Jenkins
+service jenkins stop
+dpkg -P jenkins
+
+[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
+[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
+[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
+[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
+sed -i '/^JENKINS_PORT=/d' /etc/environment
+
+# uninstall ttc script
+[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
+
+# uninstall ebf script
+[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
+[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
+
+# uninstall serial config
+[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
+[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
+[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
+[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
+[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
+[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
+
+# uninstall fserver
+[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
+[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
+
+# uninstall ftc
+[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
+[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
+
+# remove lava target files
+[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
+[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
+
+# remove links
+[ -d /fuego-ro ] && rm /fuego-ro
+[ -d /fuego-rw ] && rm /fuego-rw
+[ -d /fuego-core ] && rm /fuego-core
+
--
2.20.1
The information contained in this e-mail message and in any
attachments/annexure/appendices is confidential to the
recipient and may contain privileged information.
If you are not the intended recipient, please notify the
sender and delete the message along with any
attachments/annexure/appendices. You should not disclose,
copy or otherwise use the information contained in the
message or any annexure. Any views expressed in this e-mail
are those of the individual sender except where the sender
specifically states them to be the views of
Toshiba Software India Pvt. Ltd. (TSIP),Bangalore.
Although this transmission and any attachments are believed to be
free of any virus or other defect that might affect any computer
system into which it is received and opened, it is the responsibility
of the recipient to ensure that it is virus free and no responsibility
is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or
damage arising in any way from its use.
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-04 13:43 [Fuego] Use common script to install fuego on host or docker system venkata.pyla
2021-06-04 14:25 ` Venkata.Pyla
@ 2021-06-09 3:35 ` Tim.Bird
2021-06-15 0:08 ` Tim.Bird
2 siblings, 0 replies; 8+ messages in thread
From: Tim.Bird @ 2021-06-09 3:35 UTC (permalink / raw)
To: venkata.pyla, tbird20d; +Cc: fuego
Sorry for the slow response. This looks like a nice refactoring. I did a quick look,
and saw a few things that might be problems, but I haven't had time yet to
do a more thorough review. This refactoring of the install scripts is something
I've wanted to do for a while, so I very much appreciate this work.
I'll try to take a closer look, and get back to you with feedback, this week.
Thanks,
-- Tim
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> Sent: Friday, June 4, 2021 7:44 AM
> To: Bird, Tim <Tim.Bird@sony.com>; tbird20d@yahoo.com
> Cc: venkata pyla <venkata.pyla@toshiba-tsip.com>; fuego@lists.linuxfoundation.org; daniel.sangorrin@toshiba.co.jp
> Subject: [Fuego] Use common script to install fuego on host or docker system
>
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
> Modified the install scripts to use common scripts for both types of
> installation, this way it will help to make sure both the environments
> are same and if any future modification will have single place of change.
>
> Now, for installing fuego on docker system the 'install.sh' script will
> use common script in Dockefile to install fuego related stuff.
>
> sameway for installing fuego on host system it can use the same common
> script to install fuego related stuff.
>
> Below are the common scripts that are used in both the installation types
> for each debian distribution
> * install-stretch.sh
> * install-buster.sh
>
> so finally,
> To install fuego on docker system use the below script with options
> $ ./install.sh <options> image_name <port> <distribution> <suite>
> e.g: ./install.sh debian_stretch 8090 debian stretch
> or
> ./install.sh debian_buster 8090 debian buster
> or
> ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
>
> To install fuego on host system use the below script with options
> $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
> e.g: ./install-fuego.sh -d debian -s stretch -p 8090
> or
> ./install-fuego.sh -d debian -s buster -p 8090
> or
> ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> Dockerfile | 303 ++---------------
> Dockerfile.nojenkins | 151 ---------
> README | 29 +-
> entrypoint.sh | 7 +
> fuego-host-scripts/docker-build-image.sh | 11 +-
> install-buster.sh | 294 ++++++++++++++++
> install-debian.sh | 410 ++++++++++++++---------
> install-fuego.sh | 109 ++++++
> install-stretch.sh | 288 ++++++++++++++++
> install.sh | 12 +-
> uninstall-fuego.sh | 47 +++
> 11 files changed, 1057 insertions(+), 604 deletions(-)
> delete mode 100644 Dockerfile.nojenkins
> create mode 100755 entrypoint.sh
> create mode 100755 install-buster.sh
> create mode 100755 install-fuego.sh
> create mode 100755 install-stretch.sh
> create mode 100644 uninstall-fuego.sh
>
> diff --git a/Dockerfile b/Dockerfile
> index 7f63798..ac57cb2 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -2,9 +2,11 @@
> # WARNING: this Dockerfile assumes that the container will be created with
> # several bind mounts (see docker-create-container.sh)
> # ==============================================================================
> -# FIXTHIS: build this as an extension of the nonjenkins image
>
> -FROM debian:stretch-slim
> +ARG SUITE=stretch
> +ARG DIST=debian
> +
> +FROM ${DIST}:${SUITE}-slim
> MAINTAINER tim.bird@sony.com
>
> # ==============================================================================
> @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
> ENV https_proxy ${HTTP_PROXY}
>
> # ==============================================================================
> -# Prepare basic image
> +# Install fuego with the script
> # ==============================================================================
>
> -ARG DEBIAN_FRONTEND=noninteractive
> -
> -WORKDIR /
> -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -
> -# Fuego python dependencies
> -# - python-lxml: ftc, loggen
> -# - python-simplejson: ftc
> -# - python-yaml: ftc
> -# - python-openpyxl: ftc (also LTP)
> -# - python-requests: ftc (also fuego_release_test)
> -# - python-reportlab: ftc
> -# - python-parsedatetime: ftc
> -# - python-pexpect: ssh_exec (part of ttc)
> -# - python-pip: to install filelock, flake8
> -# - filelock: parser
> -RUN mkdir -p /usr/share/man/man1
> -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> - python-lxml python-simplejson python-yaml python-openpyxl \
> - python-requests python-reportlab python-parsedatetime \
> - python-pexpect python-pip python-setuptools python-wheel
> -RUN pip install filelock
> -
> -# Fuego command dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion iproute2
> -
> -# Default SDK for testing locally or on an x86 board
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - build-essential cmake bison flex automake kmod libtool \
> - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> -
> -# Default test host dependencies
> -# - iperf iperf3 netperf: used as servers on the host
> -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> -# - python-matplotlib: Benchmark.iperf3 parser
> -# - python-xmltodict: AGL tests
> -# - flake8: Functional.fuego_lint
> -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> -# - iputils-ping - for /bin/ping command, used by some tests
> -# FIXTHIS: install dependencies dynamically on the tests that need them
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> - netpipe-tcp iputils-ping
> -RUN pip install flake8
> -
> -# miscelaneous packages:
> -# python-serial - used by serio
> -# diffstat and vim - used by Tim
> -# time - useful for timing command duration
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - python-serial \
> - diffstat \
> - vim \
> - time
> -
> -# install packages used by NuttX SDK
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - genromfs
> -
> -# FIXTHIS: determine if these tools are really necessary
> -#RUN apt-get -q=2 -V --no-install-recommends install \
> -# apt-utils python-paramiko \
> -# xmlstarlet rsync \
> -# inotify-tools gettext netpipe-tcp \
> -# at minicom lzop bsdmainutils \
> -# mc netcat openssh-server
> -
> -# bsdmainutils provides hexdump, which is needed for netperf test
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - bsdmainutils
> -
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> -
> -# ==============================================================================
> -# Install Jenkins with the same UID/GID as the host user
> -# ==============================================================================
> +RUN mkdir /fuego
> +COPY . /fuego/
> +WORKDIR /fuego
>
> -ARG user=jenkins
> -ARG group=jenkins
> -ARG uid=1000
> -ARG gid=${uid}
> ARG JENKINS_PORT=8090
> -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
> -#ARG JENKINS_VERSION=2.164.2
> -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> -#ARG JENKINS_VERSION=2.235.3
> -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
> -ARG JENKINS_VERSION=2.249.3
> -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> -
> -ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> -ARG JENKINS_UC=https://updates.jenkins.io
> -ARG REF=/var/lib/jenkins/plugins
> +ARG NO_JENKINS=
> +ARG SUITE=stretch
> +ARG DIST=debian
> +ARG uid=
> +ARG gid=
> ENV JENKINS_HOME=/var/lib/jenkins
> ENV JENKINS_PORT=$JENKINS_PORT
> +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS
>
> -# Jenkins dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - default-jdk daemon psmisc adduser procps unzip
> -RUN pip install python-jenkins==1.4.0
> -
> -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> -RUN wget -nv ${JENKINS_URL}
> -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> -# allow Jenkins to start and install plugins, as part of dpkg installation
> -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> -
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -# as of ttc version 2.0.2, this should not be needed
> -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> -
> -# ==============================================================================
> -# get ebf script and helpers
> -# ==============================================================================
> -RUN apt-get -q=2 -V --no-install-recommends install jq
> -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
> - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
> - chmod a+x /usr/local/bin/ebf
> -
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -
> -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
> -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> - frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> - /tmp/
> -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
> - chown -R jenkins /usr/local/src/serio ; \
> - cp /usr/local/src/serio/serio /usr/local/bin/ ; \
> - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
> - ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp
> /usr/local/src/serlogin/serlogin /usr/local/bin'
> -
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
> - ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
> -
> -# ==============================================================================
> -# Jenkins post installation
> -# ==============================================================================
> -
> -RUN source /etc/default/jenkins && \
> - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> -
> -RUN source /etc/default/jenkins && \
> - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> - if [ -n "$HTTP_PROXY" ]; then \
> - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> - fi && \
> - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> -
> -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> -
> -# set up Jenkins plugins
> -COPY frontend-install/install-plugins.sh \
> - frontend-install/jenkins-support \
> - frontend-install/clitest \
> - /usr/local/bin/
> -
> -# start Jenkins onces to initialize directories?
> -RUN service jenkins start && \
> - sleep 30 && \
> - service jenkins stop
> -
> -# install plugins - these versions are for Jenkins version 2.164.2
> -# Explicitly install script-security v1.68, otherwise
> -# v1.74 will automatically be installed as a dependency of
> -# the junit plugin. Make sure to install before junit plugin.
> -# Explicitly install junit:1.27
> -
> -# install other plugins from Jenkins update center
> -# NOTE: not sure all of these are needed, but keep list
> -# as compatible as possible with 1.2.1 release for now
> -# Do NOT change the order of the plugins, unless you know what
> -# you are doing. Otherwise, the install-plugins.sh script will
> -# install the wrong versions of other plugins based on dependency
> -# information in the plugin files.
> -
> -#RUN /usr/local/bin/install-plugins.sh \
> -# script-security:1.68 \
> -# structs:1.20 \
> -# workflow-step-api:2.22 \
> -# workflow-api:2.40 \
> -# junit:1.27 \
> -# scm-api:2.6.3 \
> -# ant:1.9 \
> -# antisamy-markup-formatter:1.5 \
> -# bouncycastle-api:2.17 \
> -# command-launcher:1.3 \
> -# description-setter:1.10 \
> -# display-url-api:2.3.1 \
> -# external-monitor-job:1.7 \
> -# greenballs:1.15 \
> -# icon-shim:2.0.3 \
> -# javadoc:1.5 \
> -# jdk-tool:1.2 \
> -# ldap:1.20 \
> -# mailer:1.23 \
> -# matrix-auth:2.3 \
> -# matrix-project:1.14 \
> -# pam-auth:1.5 \
> -# pegdown-formatter:1.3 \
> -# windows-slaves:1.4
> -
> -# install plugins for latest LTS Jenkins (currently 2.249.3)
> -RUN /usr/local/bin/install-plugins.sh \
> - script-security \
> - structs \
> - workflow-step-api \
> - workflow-api \
> - junit \
> - scm-api \
> - ant \
> - antisamy-markup-formatter \
> - bouncycastle-api \
> - command-launcher \
> - description-setter \
> - display-url-api \
> - external-monitor-job \
> - greenballs \
> - icon-shim \
> - javadoc \
> - jdk-tool \
> - mailer \
> - matrix-auth \
> - matrix-project \
> - pam-auth \
> - pegdown-formatter \
> - windows-slaves
> +COPY entrypoint.sh /usr/bin/
>
> -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> -
> -# jenkins should automatically unzip any plugins in the plugin dir
> -# make a symlink for mod.js well after flot is installed
> -RUN service jenkins start && sleep 30 && \
> - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> -
> -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> -
> -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> -RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> -
> -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> -
> -# ==============================================================================
> -# ftc post installation
> -# ==============================================================================
> -
> -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> -
> -# ==============================================================================
> -# Lava
> -# ==============================================================================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> +# Delete the files that are copied for installation
> +RUN rm -rf /fuego
> +WORKDIR /
>
> # ==============================================================================
> # Setup startup command
> # ==============================================================================
> -
> # FIXTHIS: when running multiple Fuego containers, or if the host is already
> # running the netperf server, netperf will complain because the port is taken
> -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
> +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
> diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
> deleted file mode 100644
> index 56de98a..0000000
> --- a/Dockerfile.nojenkins
> +++ /dev/null
> @@ -1,151 +0,0 @@
> -# ==============================================================================
> -# WARNING: this Dockerfile assumes that the container will be created with
> -# several bind mounts (see docker-create-container.sh)
> -# ==============================================================================
> -# Usage:
> -# host$ vi fuego-ro/conf/fuego.conf
> -# jenkins_enabled=0
> -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
> -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
> -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
> -# docker# ftc list-boards
> -# docker, bbb, ...
> -# docker# ftc list-tests
> -# Functional.hello_world, ...
> -# docker# ftc run-test -b docker -t Functional.hello_world
> -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
> -
> -FROM debian:stretch-slim
> -MAINTAINER tim.bird@sony.com
> -
> -# ==============================================================================
> -# Proxy variables
> -# ==============================================================================
> -
> -ARG HTTP_PROXY
> -ENV http_proxy ${HTTP_PROXY}
> -ENV https_proxy ${HTTP_PROXY}
> -
> -# ==============================================================================
> -# Prepare basic image
> -# ==============================================================================
> -
> -ARG DEBIAN_FRONTEND=noninteractive
> -
> -WORKDIR /
> -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -
> -# Fuego python dependencies
> -# - python-lxml: ftc, loggen
> -# - python-simplejson: ftc
> -# - python-yaml: ftc
> -# - python-openpyxl: ftc (also LTP)
> -# - python-requests: ftc (also fuego_release_test)
> -# - python-reportlab: ftc
> -# - python-parsedatetime: ftc
> -# - python-pexpect: ssh_exec (part of ttc)
> -# - python-pip: to install filelock, flake8
> -# - filelock: parser
> -RUN mkdir -p /usr/share/man/man1
> -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> - python-lxml python-simplejson python-yaml python-openpyxl \
> - python-requests python-reportlab python-parsedatetime \
> - python-pexpect python-pip python-setuptools python-wheel
> -RUN pip install filelock
> -
> -# Fuego command dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion iproute2
> -
> -# Default SDK for testing locally or on an x86 board
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - build-essential cmake bison flex automake kmod libtool \
> - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> -
> -# Default test host dependencies
> -# - iperf iperf3 netperf: used as servers on the host
> -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> -# - python-matplotlib: Benchmark.iperf3 parser
> -# - python-xmltodict: AGL tests
> -# - flake8: Functional.fuego_lint
> -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> -# - iputils-ping - for /bin/ping command, used by some tests
> -# FIXTHIS: install dependencies dynamically on the tests that need them
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> - netpipe-tcp iputils-ping
> -RUN pip install flake8
> -
> -# miscelaneous packages:
> -# python-serial - used by serio
> -# diffstat and vim - used by Tim
> -# time - useful for timing command duration
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - python-serial \
> - diffstat \
> - vim \
> - time
> -
> -# FIXTHIS: determine if these tools are really necessary
> -# RUN apt-get -q=2 -V --no-install-recommends install \
> -# apt-utils python-paramiko \
> -# xmlstarlet rsync \
> -# inotify-tools gettext netpipe-tcp \
> -# at minicom lzop bsdmainutils \
> -# mc netcat openssh-server
> -
> -# bsdmainutils provides hexdump, which is needed for netperf test
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - bsdmainutils
> -
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> -
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -# as of ttc version 2.0.2, this should not be needed
> -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> -
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s
> /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> -
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server'
> -
> -# ==============================================================================
> -# ftc post installation
> -# ==============================================================================
> -
> -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> -
> -# ==============================================================================
> -# Lava
> -# ==============================================================================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -
> -# ==============================================================================
> -# Setup startup command
> -# ==============================================================================
> -
> -# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
> -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
> diff --git a/README b/README
> index 882d84e..ce42e1e 100644
> --- a/README
> +++ b/README
> @@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins
> by passing the corresponding parameters to install.sh.
> ./install.sh fuego-8082 8082
>
> +To run fuego on different distribution versions like Debian(stretch, buster, etc.)
> + pass appropriate arguments to the install.sh script
> + ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
> +
> +By default install.sh script will install fuego on Debian stretch version
> + to install fuego on Debian buster version pass below options
> + ./install.sh fuego_buster 8090 debian buster
> +
> +[Note] It is currently verified on Debian versions (stretch and buster),
> +and yet to provide install scripts for other distributions
> +
> +Installing Fuego to a host system
> +-------------------------------------------------------------------------------
> +Run install-fuego.sh to install fuego on host system, by default it assume the
> +host system is Debian stretch
> + ./install-fuego.sh
> +
> + to install on Debian buster version
> + ./install-fuego.sh -d debian -s buster -p 8090
> +
> + to install without jenkins pass --nojenkins option to the script
> + ./install-fuego.sh --nojenkins
> +
> Running
> -------------------------------------------------------------------------------
> To start your Fuego container, issue the following command top directory:
> @@ -85,9 +108,9 @@ Updating the Jenkins version
> -------------------------------------------------------------------------------
>
> Modify the version and checksum
> - $ vi Dockerfile
> - ARG JENKINS_VERSION=2.32.1
> - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> + $ vi install-debian.sh
> + JENKINS_VERSION=2.32.1
> + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
>
> Re-build the flot plugin
> $ sudo apt-get install maven
> diff --git a/entrypoint.sh b/entrypoint.sh
> new file mode 100755
> index 0000000..6eac895
> --- /dev/null
> +++ b/entrypoint.sh
> @@ -0,0 +1,7 @@
> +#!/bin/bash
> +
> +[ "$1" != "--nojenkins" ] && service jenkins start
> +
> +#TODO: netperf not present in buster, remove this condition once alternative found
> +[ -f /etc/init.d/netperf ] && service netperf start
> +iperf3 -V -s -D -f M
> diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
> index 5823357..6889c47 100755
> --- a/fuego-host-scripts/docker-build-image.sh
> +++ b/fuego-host-scripts/docker-build-image.sh
> @@ -2,9 +2,10 @@
> # $1 - name for the docker image (default: fuego)
> # $2 - port for jenkins (default: 8090)
> # $3 - Dockerfile or Dockerfile.nojenkins
> +# $4 - Debian code name on which to install (default: stretch)
> #
> # Example:
> -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins
> +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins
> #
> if [ "$1" = "--no-cache" ]; then
> NO_CACHE=--no-cache
> @@ -13,7 +14,10 @@ fi
>
> DOCKERIMAGE=${1:-fuego}
> JENKINS_PORT=${2:-8090}
> -DOCKERFILE=${3:-Dockerfile}
> +DISTRIBUTION=${3:-debian}
> +SUITE=${4:-stretch}
> +NO_JENKINS=${5:""}
> +DOCKERFILE="Dockerfile"
>
> if [ "$(id -u)" == "0" ]; then
> JENKINS_UID=$(id -u $SUDO_USER)
> @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
>
> sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
> --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
> + --build-arg DIST=$DISTRIBUTION \
> + --build-arg SUITE=$SUITE \
> + --build-arg NO_JENKINS=$NO_JENKINS \
> --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
> diff --git a/install-buster.sh b/install-buster.sh
> new file mode 100755
> index 0000000..f4ce65b
> --- /dev/null
> +++ b/install-buster.sh
> @@ -0,0 +1,294 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> +#
> +usage(){
> +
> +cat <<HERE
> +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> +
> +Install fuego on the host Debian filesystem.
> +
> +options:
> + --help Show usage help
> + --nojenkins Install Fuego without Jenkins
> +HERE
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=0
> +
> +POSITIONAL=()
> +while [[ $# -gt 0 ]] ; do
> + case $1 in
> + --nojenkins)
> + nojenkins=1
> + shift
> + ;;
> + --help | -h)
> + usage
> + exit 0
> + ;;
> + *)
> + POSITIONAL+=("$1") # save argument for later
> + shift
> + esac
> +done
> +set -- "${POSITIONAL[@]}" # restore positional arguments
> +
> +port=${2:-8090}
> +
> +# ==============================================================================
> +# Install Fuego dependencies
> +# ==============================================================================
> +
> +mkdir -p /usr/share/man/man1
> +
> +# netperf is in non-free
> +echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list
> +
> +apt-get update -q=2
> +
> +# Fuego python dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + python-lxml python-simplejson python-yaml python-openpyxl \
> + python-requests python-reportlab python-parsedatetime \
> + python-pexpect python-pip python-setuptools python-wheel
> +pip install filelock
> +
> +# Fuego command dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> + bash-completion iproute2
> +
> +# Default SDK for testing locally or on an x86 board
> +apt-get -q=2 -V --no-install-recommends install \
> + build-essential cmake bison flex kmod automake libtool \
> + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> +
> +# Default test host dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
> +pip install flake8
> +
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +#apt-get -q=2 -V --no-install-recommends install \
> +# bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +
> +# TODO: check if proxy settings can set outside docker container
> +if [ -n "$http_proxy" ]; then
> + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +fi
> +
> +# ==============================================================================
> +# Install Jenkins
> +# ==============================================================================
> +
> +if [ $nojenkins -eq 0 ]; then
> + #JENKINS_VERSION=2.164.2
> + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> + #JENKINS_VERSION=2.249.3
> + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> + JENKINS_VERSION=2.164.3
> + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
> + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> + export JENKINS_UC=https://updates.jenkins.io
> + REF=/var/lib/jenkins/plugins
> + JENKINS_HOME=/var/lib/jenkins
> + JENKINS_PORT=$port
> +
> + # Jenkins dependencies
> + apt-get -q=2 -V --no-install-recommends install \
> + default-jdk daemon psmisc adduser procps unzip
> + pip install python-jenkins==1.4.0
> +
> + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> + export JENKINS_PORT=$port
> +
> + groupadd jenkins
> + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> +
> + wget -nv ${JENKINS_URL}
> + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> +
> + # allow Jenkins to start and install plugins, as part of dpkg installation
> + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> + rm jenkins_${JENKINS_VERSION}_all.deb
> +
> + # fix for Jenkins Java version check error
> + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version
> \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
> + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
> +
> + source /etc/default/jenkins && \
> + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> +
> + source /etc/default/jenkins && \
> + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> + if [ -n "$http_proxy" ]; then \
> + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> + fi && \
> + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> +
> + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> +
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> + /usr/local/bin/
> +
> + mkdir -p $JENKINS_HOME/userContent/docs
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +
> + # start and stop jenkins to pre-populate some settings
> + service jenkins start && \
> + sleep 30 && \
> + service jenkins stop
> +
> + # install other plugins from Jenkins update center
> + # NOTE: not sure all of these are needed, but keep list
> + # compatible with 1.2.1 release for now
> + #/usr/local/bin/install-plugins.sh \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # junit:1.27 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.4 \
> + # pegdown-formatter:1.3 \
> + # structs:1.17 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # make the mod.js symlink well after flot is installed
> + service jenkins start && sleep 30 && \
> + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> +
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> +else
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> +fi
> +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> +
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +apt-get -q=2 -V --no-install-recommends install jq
> +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> +chmod a+x /usr/local/bin/ebf
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# ftc post installation
> +# ==============================================================================
> +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +echo ". /etc/bash_completion" >> /root/.bashrc
> +
> +# ==============================================================================
> +# Lava
> +# ==============================================================================
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/install-debian.sh b/install-debian.sh
> index ccca032..2d3ffd3 100755
> --- a/install-debian.sh
> +++ b/install-debian.sh
> @@ -2,102 +2,163 @@
> # 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> #
> # Usage:
> -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> #
> -if [ -n "$1" ]; then
> - if [ "$1" = "--help" -o "$1" = "-h" ]; then
> - cat <<HERE
> -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> -
> -Installs fuego on the host Debian filesystem.
> -
> -options:
> - --help Show usage help
> - --nojenkins Install Fuego without Jenkins
> -HERE
> - exit 0
> - fi
> -fi
> +
> +usage(){
> + echo ""
> + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> + echo "Install fuego on Debian filesystem."
> + echo ""
> + echo "options:"
> + echo " --help | -h Show usage help"
> + echo " --suite | -s <suite> Specify Debian suite"
> + echo " --port | -p <port> Jenkins port"
> + echo " --nojenkins Install Fuego without Jenkins"
> + echo ""
> +}
>
> if [[ $EUID -ne 0 ]]; then
> echo "Sorry, you need root permissions"
> exit 1
> fi
>
> -if [ "$1" = "--nojenkins" ]; then
> - nojenkins=1
> +nojenkins=0
> +port=8090
> +suite=stretch
> +user=jenkins
> +group=jenkins
> +uid=1000
> +gid=1000
> +
> +while [ "$1" != "" ]; do
> + case $1 in
> + -s | --suite )
> + shift
> + suite=$1
> + ;;
> + -p | --port )
> + shift
> + port=$1
> + ;;
> + -u | --uid )
> + shift
> + uid=$1
> + ;;
> + -g | --gid )
> + shift
> + gid=$1
> + ;;
> + --nojenkins )
> + nojenkins=1
> + ;;
> + -h | --help )
> + usage
> + exit 0
> + ;;
> + * )
> + break
> + esac
> shift
> -else
> - nojenkins=0
> - port=${1:-8090}
> -fi
> +done
>
> # ==============================================================================
> # Install Fuego dependencies
> # ==============================================================================
>
> # netperf is in non-free
> -echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -
> -apt-get update -q=2
> +echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list
>
> # Fuego python dependencies
> -apt-get -q=2 -V --no-install-recommends install \
> +# - python-lxml: ftc, loggen
> +# - python-simplejson: ftc
> +# - python-yaml: ftc
> +# - python-openpyxl: ftc (also LTP)
> +# - python-requests: ftc (also fuego_release_test)
> +# - python-reportlab: ftc
> +# - python-parsedatetime: ftc
> +# - python-pexpect: ssh_exec (part of ttc)
> +# - python-pip: to install filelock, flake8
> +# - filelock: parser
> +mkdir -p /usr/share/man/man1
> +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> python-lxml python-simplejson python-yaml python-openpyxl \
> python-requests python-reportlab python-parsedatetime \
> - python-pip
> + python-pexpect python-pip python-setuptools python-wheel
> pip install filelock
>
> # Fuego command dependencies
> apt-get -q=2 -V --no-install-recommends install \
> git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion
> + bash-completion iproute2
>
> # Default SDK for testing locally or on an x86 board
> apt-get -q=2 -V --no-install-recommends install \
> build-essential cmake bison flex automake kmod libtool \
> libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
>
> # Default test host dependencies
> +# - iperf iperf3 netperf: used as servers on the host
> +# - bzip2 bc: used for local tests by Functional.bzip2/bc
> +# - python-matplotlib: Benchmark.iperf3 parser
> +# - python-xmltodict: AGL tests
> +# - flake8: Functional.fuego_lint
> +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> +# - iputils-ping - for /bin/ping command, used by some tests
> +# FIXTHIS: install dependencies dynamically on the tests that need them
> apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
> -pip install flake8 bsdmainutils
> + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
>
> -echo "dash dash/sh boolean false" | debconf-set-selections
> -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> -if [ -n "$http_proxy" ]; then
> - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +#TODO: find alternate package for netperf in buster version
> +if [ $suite != "buster" ];then
> + apt-get -q=2 -V --no-install-recommends install netperf
> fi
> +pip install flake8
>
> -# ==============================================================================
> -# Clone fuego and fuego-core
> -# ==============================================================================
> -if [ ! -d "/fuego" ]; then
> - cd /
> - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
> - ln -s /fuego/fuego-ro /fuego-ro
> - ln -s /fuego/fuego-rw /fuego-rw
> - cd fuego
> - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
> - ln -s /fuego/fuego-core /fuego-core
> -fi
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# FIXTHIS: determine if these tools are really necessary
> +#RUN apt-get -q=2 -V --no-install-recommends install \
> +# apt-utils python-paramiko \
> +# xmlstarlet rsync \
> +# inotify-tools gettext netpipe-tcp \
> +# at minicom lzop bsdmainutils \
> +# mc netcat openssh-server
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +apt-get -q=2 -V --no-install-recommends install \
> + bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
>
> # ==============================================================================
> # Install Jenkins
> # ==============================================================================
> -
> if [ $nojenkins -eq 0 ]; then
> #JENKINS_VERSION=2.164.2
> #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> JENKINS_VERSION=2.249.3
> JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> - JENKINS_UC=https://updates.jenkins.io
> - REF=/var/lib/jenkins/plugins
> - JENKINS_HOME=/var/lib/jenkins
> + export JENKINS_UC=https://updates.jenkins.io
> + export REF=/var/lib/jenkins/plugins
> + export JENKINS_HOME=/var/lib/jenkins
> JENKINS_PORT=$port
>
> # Jenkins dependencies
> @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
> pip install python-jenkins==1.4.0
>
> echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> - groupadd jenkins
> - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> - cd
> - wget ${JENKINS_URL}
> + export JENKINS_PORT=$port
> +
> + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> +
> + wget -nv ${JENKINS_URL}
> echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
>
> # allow Jenkins to start and install plugins, as part of dpkg installation
> @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
> dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> rm jenkins_${JENKINS_VERSION}_all.deb
>
> - # update ownership
> - chown -R jenkins:jenkins /fuego
> - chown -R jenkins:jenkins /fuego-ro
> - chown -R jenkins:jenkins /fuego-rw
> - chown -R jenkins:jenkins /fuego-core
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +# as of ttc version 2.0.2, this should not be needed
> +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + apt-get -q=2 -V --no-install-recommends install jq
> + git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> + chmod a+x /usr/local/bin/ebf
> +fi
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +git clone https://github.com/frowand/serio.git /usr/local/src/serio
> +if [ $nojenkins -eq 0 ]; then
> + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
> + chown -R jenkins /usr/local/src/serio
> +fi
> +cp /usr/local/src/serio/serio /usr/local/bin/
> +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> +
> +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
> +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
> +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# Jenkins post installation
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> +
> + # Fix jenkins issue, Java version check fails when java version is 11
> + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-
> \]*\\\).*\\\".*/\\\1\\\2\/p;')"
> + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
>
> source /etc/default/jenkins && \
> JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
>
> sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
>
> - cp /fuego/frontend-install/install-plugins.sh \
> - /fuego/frontend-install/jenkins-support \
> - /fuego/frontend-install/clitest \
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> /usr/local/bin/
>
> - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
> - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> - mkdir $JENKINS_HOME/userContent/docs
> - cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> - jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> - sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> - chown -R jenkins:jenkins $JENKINS_HOME/
> -
> # start and stop jenkins to pre-populate some settings
> service jenkins start && \
> sleep 30 && \
> service jenkins stop
>
> + # install plugins - these versions are for Jenkins version 2.164.2
> + # Explicitly install script-security v1.68, otherwise
> + # v1.74 will automatically be installed as a dependency of
> + # the junit plugin. Make sure to install before junit plugin.
> + # Explicitly install junit:1.27
> +
> # install other plugins from Jenkins update center
> # NOTE: not sure all of these are needed, but keep list
> - # compatible with 1.2.1 release for now
> - #/usr/local/bin/install-plugins.sh \
> - # ant:1.9 \
> - # antisamy-markup-formatter:1.5 \
> - # bouncycastle-api:2.17 \
> - # command-launcher:1.3 \
> - # description-setter:1.10 \
> - # display-url-api:2.3.1 \
> - # external-monitor-job:1.7 \
> - # greenballs:1.15 \
> - # icon-shim:2.0.3 \
> - # javadoc:1.5 \
> - # jdk-tool:1.2 \
> - # junit:1.27 \
> - # ldap:1.20 \
> - # mailer:1.23 \
> - # matrix-auth:2.3 \
> - # matrix-project:1.14 \
> - # pam-auth:1.4 \
> - # pegdown-formatter:1.3 \
> - # structs:1.17 \
> - # windows-slaves:1.4
> -
> - # install plugins for latest LTS Jenkins (currently 2.249.3)
> - /usr/local/bin/install-plugins.sh \
> - script-security \
> - structs \
> - workflow-step-api \
> - workflow-api \
> - junit \
> - scm-api \
> - ant \
> - antisamy-markup-formatter \
> - bouncycastle-api \
> - command-launcher \
> - description-setter \
> - display-url-api \
> - external-monitor-job \
> - greenballs \
> - icon-shim \
> - javadoc \
> - jdk-tool \
> - mailer \
> - matrix-auth \
> - matrix-project \
> - pam-auth \
> - pegdown-formatter \
> - windows-slaves
> -
> - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> + # as compatible as possible with 1.2.1 release for now
> + # Do NOT change the order of the plugins, unless you know what
> + # you are doing. Otherwise, the install-plugins.sh script will
> + # install the wrong versions of other plugins based on dependency
> + # information in the plugin files.
>
> + #/usr/local/bin/install-plugins.sh \
> + # script-security:1.68 \
> + # structs:1.20 \
> + # workflow-step-api:2.22 \
> + # workflow-api:2.40 \
> + # junit:1.27 \
> + # scm-api:2.6.3 \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.5 \
> + # pegdown-formatter:1.3 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # jenkins should automatically unzip any plugins in the plugin dir
> # make the mod.js symlink well after flot is installed
> service jenkins start && sleep 30 && \
> rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
>
> - chown -R jenkins:jenkins $JENKINS_HOME/
> -else
> - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
> -fi
> + mkdir -p $JENKINS_HOME/userContent/docs
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
>
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -/usr/local/src/ttc/install.sh /usr/local/bin
> -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
>
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -if [ $nojenkins -eq 0 ]; then
> - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> -else
> - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> + chown -R jenkins:jenkins $JENKINS_HOME/
> fi
>
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> -ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> -
> # ==============================================================================
> # ftc post installation
> # ==============================================================================
> ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> echo ". /etc/bash_completion" >> /root/.bashrc
>
> # ==============================================================================
> @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
> # ==============================================================================
> ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -
> -# ==============================================================================
> -# Small guide
> -# ==============================================================================
> -echo "Run 'service netperf start' to start a netperf server"
> -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> -echo "Run 'ftc list-boards' to see the available boards"
> -echo "Run 'ftc list-tests' to see the available tests"
> -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> -echo "Run 'ftc gen-report' to get results"
> diff --git a/install-fuego.sh b/install-fuego.sh
> new file mode 100755
> index 0000000..7b3e010
> --- /dev/null
> +++ b/install-fuego.sh
> @@ -0,0 +1,109 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> +#
> +
> +usage(){
> + echo ""
> + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> + echo "Install fuego on Debian filesystem."
> + echo ""
> + echo "options:"
> + echo " --help | -h Show usage help"
> + echo " --dist | -d <distribution> Specify distribution"
> + echo " --suite | -s <suite> Specify suite"
> + echo " --port | -p <port> Jenkins port"
> + echo " --nojenkins Install Fuego without Jenkins"
> + echo ""
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=
> +port=8090
> +dist=debian
> +suite=stretch
> +fuego_dir=$(pwd)
> +
> +while [ "$1" != "" ]; do
> + case $1 in
> + -d | --dist )
> + shift
> + dist=$1
> + ;;
> + -s | --suite )
> + shift
> + suite=$1
> + ;;
> + -p | --port )
> + shift
> + port=$1
> + ;;
> + --nojenkins )
> + nojenkins="--nojenkins"
> + ;;
> + -h | --help )
> + usage
> + exit 0
> + ;;
> + * )
> + break
> + esac
> + shift
> +done
> +
> +# ==============================================================================
> +# Clone fuego-core
> +# ==============================================================================
> +if [ ! -f "fuego-core/scripts/ftc" ]; then
> + # set fuego-core branch to same as current fuego branch
> + # get current git branch
> + set -o noglob
> + while IFS=" " read -r part1 part2 ; do
> + if [ $part1 = "*" ] ; then
> + branch=$part2
> + fi
> + done < <(git branch)
> + set +o noglob
> + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
> +fi
> +
> +# ==============================================================================
> +# Install fuego on host system
> +# ==============================================================================
> +./install-$dist.sh -s $suite -p $port $nojenkins
> +
> +systemctl daemon-reload
> +service jenkins restart
> +
> +# provide permission to current user
> +usermod -aG jenkins $USER
> +chmod -R 775 $fuego_dir/fuego-rw
> +
> +chown -R jenkins:jenkins $fuego_dir/fuego-ro
> +chown -R jenkins:jenkins $fuego_dir/fuego-rw
> +chown -R jenkins:jenkins $fuego_dir/fuego-core
> +
> +# ==============================================================================
> +# Create Links to fuego folders
> +# ==============================================================================
> +ln -s $fuego_dir/fuego-ro /
> +ln -s $fuego_dir/fuego-rw /
> +ln -s $fuego_dir/fuego-core /
> +
> +# ==============================================================================
> +# Small guide
> +# ==============================================================================
> +echo "Logout and Login current user session to work properly"
> +echo "Run 'service netperf start' to start a netperf server"
> +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> +echo "Run 'ftc list-boards' to see the available boards"
> +echo "Run 'ftc list-tests' to see the available tests"
> +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> +echo "Run 'ftc gen-report' to get results"
> diff --git a/install-stretch.sh b/install-stretch.sh
> new file mode 100755
> index 0000000..4f6f724
> --- /dev/null
> +++ b/install-stretch.sh
> @@ -0,0 +1,288 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> +#
> +usage(){
> +
> +cat <<HERE
> +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> +
> +Install fuego on the host Debian filesystem.
> +
> +options:
> + --help Show usage help
> + --nojenkins Install Fuego without Jenkins
> +HERE
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=0
> +
> +POSITIONAL=()
> +while [[ $# -gt 0 ]] ; do
> + case $1 in
> + --nojenkins)
> + nojenkins=1
> + shift
> + ;;
> + --help | -h)
> + usage
> + exit 0
> + ;;
> + *)
> + POSITIONAL+=("$1") # save argument for later
> + shift
> + esac
> +done
> +set -- "${POSITIONAL[@]}" # restore positional arguments
> +
> +port=${2:-8090}
> +
> +# ==============================================================================
> +# Install Fuego dependencies
> +# ==============================================================================
> +
> +mkdir -p /usr/share/man/man1
> +
> +# netperf is in non-free
> +echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> +
> +apt-get update -q=2
> +
> +# Fuego python dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + python-lxml python-simplejson python-yaml python-openpyxl \
> + python-requests python-reportlab python-parsedatetime \
> + python-pexpect python-pip python-setuptools python-wheel
> +pip install filelock
> +
> +# Fuego command dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> + bash-completion iproute2
> +
> +# Default SDK for testing locally or on an x86 board
> +apt-get -q=2 -V --no-install-recommends install \
> + build-essential cmake bison flex kmod automake libtool \
> + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> +
> +# Default test host dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
> +pip install flake8
> +
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +apt-get -q=2 -V --no-install-recommends install \
> + bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +
> +# TODO: check if proxy settings can set outside docker container
> +#if [ -n "$http_proxy" ]; then
> +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +#fi
> +
> +# ==============================================================================
> +# Install Jenkins
> +# ==============================================================================
> +
> +if [ $nojenkins -eq 0 ]; then
> + #JENKINS_VERSION=2.164.2
> + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> + JENKINS_VERSION=2.249.3
> + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> + export JENKINS_UC=https://updates.jenkins.io
> + REF=/var/lib/jenkins/plugins
> + JENKINS_HOME=/var/lib/jenkins
> + JENKINS_PORT=$port
> +
> + # Jenkins dependencies
> + apt-get -q=2 -V --no-install-recommends install \
> + default-jdk daemon psmisc adduser procps unzip
> + pip install python-jenkins==1.4.0
> +
> + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> + export JENKINS_PORT=$port
> +
> + groupadd jenkins
> + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> +
> + wget -nv ${JENKINS_URL}
> + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> +
> + # allow Jenkins to start and install plugins, as part of dpkg installation
> + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> + rm jenkins_${JENKINS_VERSION}_all.deb
> +
> + source /etc/default/jenkins && \
> + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> +
> + source /etc/default/jenkins && \
> + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> + if [ -n "$http_proxy" ]; then \
> + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> + fi && \
> + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> +
> + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> +
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> + /usr/local/bin/
> +
> + mkdir -p $JENKINS_HOME/userContent/docs
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +
> + # start and stop jenkins to pre-populate some settings
> + service jenkins start && \
> + sleep 30 && \
> + service jenkins stop
> +
> + # install other plugins from Jenkins update center
> + # NOTE: not sure all of these are needed, but keep list
> + # compatible with 1.2.1 release for now
> + #/usr/local/bin/install-plugins.sh \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # junit:1.27 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.4 \
> + # pegdown-formatter:1.3 \
> + # structs:1.17 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # make the mod.js symlink well after flot is installed
> + service jenkins start && sleep 30 && \
> + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> +
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> +else
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> +fi
> +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> +
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +apt-get -q=2 -V --no-install-recommends install jq
> +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> +chmod a+x /usr/local/bin/ebf
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# ftc post installation
> +# ==============================================================================
> +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +echo ". /etc/bash_completion" >> /root/.bashrc
> +
> +# ==============================================================================
> +# Lava
> +# ==============================================================================
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/install.sh b/install.sh
> index a5a90b0..259c887 100755
> --- a/install.sh
> +++ b/install.sh
> @@ -1,13 +1,13 @@
> #!/bin/bash
> #
> -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
> #
>
> # $1 is the exit code after usage is shown
> usage() {
> exit_code="$1"
> cat <<HERE
> -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
>
> Create the docker image and container with the Fuego test distribution.
> If no <image_name> is provided, the image will be named 'fuego'.
> @@ -34,7 +34,7 @@ fi
>
> priv=0
> NOCACHE=""
> -dockerfile="Dockerfile"
> +NOJENKINS=""
>
> POSITIONAL=()
> while [[ $# -gt 0 ]] ; do
> @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
> shift
> ;;
> --nojenkins)
> - dockerfile="Dockerfile.nojenkins"
> + NOJENKINS="--nojenkins"
> shift
> ;;
> *)
> @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
>
> image_name=${1:-fuego}
> jenkins_port=${2:-8090}
> +distribution=${3:-debian}
> +suite=${4:-stretch}
>
> container_name="${image_name}-container"
>
> @@ -79,7 +81,7 @@ fi
>
> set -e
>
> -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
> +source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
> if [ "$priv" == "0" ]; then
> fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
> else
> diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
> new file mode 100644
> index 0000000..b899e7c
> --- /dev/null
> +++ b/uninstall-fuego.sh
> @@ -0,0 +1,47 @@
> +#!/bin/bash
> +
> +# Remove Jenkins plugins
> +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
> +
> +# uninstall Jenkins
> +service jenkins stop
> +dpkg -P jenkins
> +
> +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
> +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
> +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
> +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
> +sed -i '/^JENKINS_PORT=/d' /etc/environment
> +
> +# uninstall ttc script
> +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
> +
> +# uninstall ebf script
> +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
> +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
> +
> +# uninstall serial config
> +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
> +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
> +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
> +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
> +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
> +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
> +
> +# uninstall fserver
> +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
> +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
> +
> +# uninstall ftc
> +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
> +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
> +
> +# remove lava target files
> +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
> +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
> +
> +# remove links
> +[ -d /fuego-ro ] && rm /fuego-ro
> +[ -d /fuego-rw ] && rm /fuego-rw
> +[ -d /fuego-core ] && rm /fuego-core
> +
> --
> 2.20.1
>
> The information contained in this e-mail message and in any
> attachments/annexure/appendices is confidential to the
> recipient and may contain privileged information.
> If you are not the intended recipient, please notify the
> sender and delete the message along with any
> attachments/annexure/appendices. You should not disclose,
> copy or otherwise use the information contained in the
> message or any annexure. Any views expressed in this e-mail
> are those of the individual sender except where the sender
> specifically states them to be the views of
> Toshiba Software India Pvt. Ltd. (TSIP),Bangalore.
>
> Although this transmission and any attachments are believed to be
> free of any virus or other defect that might affect any computer
> system into which it is received and opened, it is the responsibility
> of the recipient to ensure that it is virus free and no responsibility
> is accepted by Toshiba Embedded Software India Pvt. Ltd, for any loss or
> damage arising in any way from its use.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-04 13:43 [Fuego] Use common script to install fuego on host or docker system venkata.pyla
2021-06-04 14:25 ` Venkata.Pyla
2021-06-09 3:35 ` Tim.Bird
@ 2021-06-15 0:08 ` Tim.Bird
2021-06-15 3:46 ` Venkata.Pyla
2021-06-17 7:01 ` Venkata.Pyla
2 siblings, 2 replies; 8+ messages in thread
From: Tim.Bird @ 2021-06-15 0:08 UTC (permalink / raw)
To: venkata.pyla, tbird20d; +Cc: fuego
Please see comments inline below.
I tried to apply this patch from the e-mail list using 'git am', but got a failure.
That made it a hard to experiment with and view the changes in
context.
What version of the code is this patch based on?
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> Subject: [Fuego] Use common script to install fuego on host or docker system
>
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
> Modified the install scripts to use common scripts for both types of
> installation, this way it will help to make sure both the environments
> are same and if any future modification will have single place of change.
>
> Now, for installing fuego on docker system the 'install.sh' script will
> use common script in Dockefile to install fuego related stuff.
>
> sameway for installing fuego on host system it can use the same common
> script to install fuego related stuff.
>
> Below are the common scripts that are used in both the installation types
> for each debian distribution
> * install-stretch.sh
> * install-buster.sh
>
> so finally,
> To install fuego on docker system use the below script with options
> $ ./install.sh <options> image_name <port> <distribution> <suite>
> e.g: ./install.sh debian_stretch 8090 debian stretch
> or
> ./install.sh debian_buster 8090 debian buster
> or
> ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
This is a bit wordy, but OK.
If we don't support any other distributions besides Debian at the moment,
then I'm not sure we need the distribution argument. Do you have any
non-debian distribution install scripts ready to submit soon?
I'm not sure I follow calling the Debian release name the "<suite>".
>
> To install fuego on host system use the below script with options
> $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
"suite" seems like an odd name. Should this be distro "version" or "release"?
> e.g: ./install-fuego.sh -d debian -s stretch -p 8090
> or
> ./install-fuego.sh -d debian -s buster -p 8090
> or
> ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> Dockerfile | 303 ++---------------
> Dockerfile.nojenkins | 151 ---------
> README | 29 +-
> entrypoint.sh | 7 +
> fuego-host-scripts/docker-build-image.sh | 11 +-
> install-buster.sh | 294 ++++++++++++++++
> install-debian.sh | 410 ++++++++++++++---------
> install-fuego.sh | 109 ++++++
> install-stretch.sh | 288 ++++++++++++++++
Is install-debian.sh the common parts for a Debian installation, with
buster and stretch having the parts that are unique to each of
those distros respectively? If we're going to refactor this, I
want to eliminate any duplication, so that package names are
not used in multiple places. This has been a headache to
maintain.
It appears there is lots of duplication between install-buster.sh and
install-stretch.sh. Can we possible put this duplication into an
"install-debian-common.sh", and reserve install-buster.sh and
install-stretch.sh for only the parts unique to those distros?
> install.sh | 12 +-
> uninstall-fuego.sh | 47 +++
Ooo - I like the sound of this uninstall script.
> 11 files changed, 1057 insertions(+), 604 deletions(-)
> delete mode 100644 Dockerfile.nojenkins
> create mode 100755 entrypoint.sh
> create mode 100755 install-buster.sh
> create mode 100755 install-fuego.sh
> create mode 100755 install-stretch.sh
> create mode 100644 uninstall-fuego.sh
>
> diff --git a/Dockerfile b/Dockerfile
> index 7f63798..ac57cb2 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -2,9 +2,11 @@
> # WARNING: this Dockerfile assumes that the container will be created with
> # several bind mounts (see docker-create-container.sh)
> # ==============================================================================
> -# FIXTHIS: build this as an extension of the nonjenkins image
>
> -FROM debian:stretch-slim
> +ARG SUITE=stretch
> +ARG DIST=debian
> +
> +FROM ${DIST}:${SUITE}-slim
> MAINTAINER tim.bird@sony.com
>
> # ==============================================================================
> @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
> ENV https_proxy ${HTTP_PROXY}
>
> # ==============================================================================
> -# Prepare basic image
> +# Install fuego with the script
> # ==============================================================================
>
> -ARG DEBIAN_FRONTEND=noninteractive
> -
> -WORKDIR /
> -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -
> -# Fuego python dependencies
> -# - python-lxml: ftc, loggen
> -# - python-simplejson: ftc
> -# - python-yaml: ftc
> -# - python-openpyxl: ftc (also LTP)
> -# - python-requests: ftc (also fuego_release_test)
> -# - python-reportlab: ftc
> -# - python-parsedatetime: ftc
> -# - python-pexpect: ssh_exec (part of ttc)
> -# - python-pip: to install filelock, flake8
> -# - filelock: parser
> -RUN mkdir -p /usr/share/man/man1
> -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> - python-lxml python-simplejson python-yaml python-openpyxl \
> - python-requests python-reportlab python-parsedatetime \
> - python-pexpect python-pip python-setuptools python-wheel
> -RUN pip install filelock
> -
> -# Fuego command dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion iproute2
> -
> -# Default SDK for testing locally or on an x86 board
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - build-essential cmake bison flex automake kmod libtool \
> - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> -
> -# Default test host dependencies
> -# - iperf iperf3 netperf: used as servers on the host
> -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> -# - python-matplotlib: Benchmark.iperf3 parser
> -# - python-xmltodict: AGL tests
> -# - flake8: Functional.fuego_lint
> -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> -# - iputils-ping - for /bin/ping command, used by some tests
> -# FIXTHIS: install dependencies dynamically on the tests that need them
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> - netpipe-tcp iputils-ping
> -RUN pip install flake8
> -
> -# miscelaneous packages:
> -# python-serial - used by serio
> -# diffstat and vim - used by Tim
> -# time - useful for timing command duration
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - python-serial \
> - diffstat \
> - vim \
> - time
> -
> -# install packages used by NuttX SDK
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - genromfs
> -
> -# FIXTHIS: determine if these tools are really necessary
> -#RUN apt-get -q=2 -V --no-install-recommends install \
> -# apt-utils python-paramiko \
> -# xmlstarlet rsync \
> -# inotify-tools gettext netpipe-tcp \
> -# at minicom lzop bsdmainutils \
> -# mc netcat openssh-server
> -
> -# bsdmainutils provides hexdump, which is needed for netperf test
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - bsdmainutils
> -
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> -
> -# ==============================================================================
> -# Install Jenkins with the same UID/GID as the host user
> -# ==============================================================================
> +RUN mkdir /fuego
> +COPY . /fuego/
> +WORKDIR /fuego
>
> -ARG user=jenkins
> -ARG group=jenkins
> -ARG uid=1000
> -ARG gid=${uid}
> ARG JENKINS_PORT=8090
> -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
> -#ARG JENKINS_VERSION=2.164.2
> -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> -#ARG JENKINS_VERSION=2.235.3
> -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
> -ARG JENKINS_VERSION=2.249.3
> -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> -
> -ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> -ARG JENKINS_UC=https://updates.jenkins.io
> -ARG REF=/var/lib/jenkins/plugins
> +ARG NO_JENKINS=
> +ARG SUITE=stretch
> +ARG DIST=debian
> +ARG uid=
> +ARG gid=
> ENV JENKINS_HOME=/var/lib/jenkins
> ENV JENKINS_PORT=$JENKINS_PORT
> +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS
One question about this overall approach. Does this retain the ability to cache
operations, so that if something goes wrong with an install, you don't have to
start over from scratch. When building with docker, with multiple steps, it
caches the image at each step. When rebuilding, or when an error occurs,
this can save a lot of time. It also can introduce subtle bugs, but that's
a separate issue.
This setup seems like it performs the bulk of the install as a single docker
step, is that right?
> -# Jenkins dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - default-jdk daemon psmisc adduser procps unzip
> -RUN pip install python-jenkins==1.4.0
> -
> -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> -RUN wget -nv ${JENKINS_URL}
> -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> -# allow Jenkins to start and install plugins, as part of dpkg installation
> -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> -
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -# as of ttc version 2.0.2, this should not be needed
> -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> -
> -# ==============================================================================
> -# get ebf script and helpers
> -# ==============================================================================
> -RUN apt-get -q=2 -V --no-install-recommends install jq
> -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
> - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
> - chmod a+x /usr/local/bin/ebf
> -
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -
> -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
> -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> - frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> - /tmp/
> -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
> - chown -R jenkins /usr/local/src/serio ; \
> - cp /usr/local/src/serio/serio /usr/local/bin/ ; \
> - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
> - ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp
> /usr/local/src/serlogin/serlogin /usr/local/bin'
> -
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
> - ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
> -
> -# ==============================================================================
> -# Jenkins post installation
> -# ==============================================================================
> -
> -RUN source /etc/default/jenkins && \
> - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> -
> -RUN source /etc/default/jenkins && \
> - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> - if [ -n "$HTTP_PROXY" ]; then \
> - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> - fi && \
> - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> -
> -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> -
> -# set up Jenkins plugins
> -COPY frontend-install/install-plugins.sh \
> - frontend-install/jenkins-support \
> - frontend-install/clitest \
> - /usr/local/bin/
> -
> -# start Jenkins onces to initialize directories?
> -RUN service jenkins start && \
> - sleep 30 && \
> - service jenkins stop
> -
> -# install plugins - these versions are for Jenkins version 2.164.2
> -# Explicitly install script-security v1.68, otherwise
> -# v1.74 will automatically be installed as a dependency of
> -# the junit plugin. Make sure to install before junit plugin.
> -# Explicitly install junit:1.27
> -
> -# install other plugins from Jenkins update center
> -# NOTE: not sure all of these are needed, but keep list
> -# as compatible as possible with 1.2.1 release for now
> -# Do NOT change the order of the plugins, unless you know what
> -# you are doing. Otherwise, the install-plugins.sh script will
> -# install the wrong versions of other plugins based on dependency
> -# information in the plugin files.
> -
> -#RUN /usr/local/bin/install-plugins.sh \
> -# script-security:1.68 \
> -# structs:1.20 \
> -# workflow-step-api:2.22 \
> -# workflow-api:2.40 \
> -# junit:1.27 \
> -# scm-api:2.6.3 \
> -# ant:1.9 \
> -# antisamy-markup-formatter:1.5 \
> -# bouncycastle-api:2.17 \
> -# command-launcher:1.3 \
> -# description-setter:1.10 \
> -# display-url-api:2.3.1 \
> -# external-monitor-job:1.7 \
> -# greenballs:1.15 \
> -# icon-shim:2.0.3 \
> -# javadoc:1.5 \
> -# jdk-tool:1.2 \
> -# ldap:1.20 \
> -# mailer:1.23 \
> -# matrix-auth:2.3 \
> -# matrix-project:1.14 \
> -# pam-auth:1.5 \
> -# pegdown-formatter:1.3 \
> -# windows-slaves:1.4
> -
> -# install plugins for latest LTS Jenkins (currently 2.249.3)
> -RUN /usr/local/bin/install-plugins.sh \
> - script-security \
> - structs \
> - workflow-step-api \
> - workflow-api \
> - junit \
> - scm-api \
> - ant \
> - antisamy-markup-formatter \
> - bouncycastle-api \
> - command-launcher \
> - description-setter \
> - display-url-api \
> - external-monitor-job \
> - greenballs \
> - icon-shim \
> - javadoc \
> - jdk-tool \
> - mailer \
> - matrix-auth \
> - matrix-project \
> - pam-auth \
> - pegdown-formatter \
> - windows-slaves
> +COPY entrypoint.sh /usr/bin/
>
> -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> -
> -# jenkins should automatically unzip any plugins in the plugin dir
> -# make a symlink for mod.js well after flot is installed
> -RUN service jenkins start && sleep 30 && \
> - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> -
> -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> -
> -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> -RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> -
> -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> -
> -# ==============================================================================
> -# ftc post installation
> -# ==============================================================================
> -
> -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> -
> -# ==============================================================================
> -# Lava
> -# ==============================================================================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> +# Delete the files that are copied for installation
> +RUN rm -rf /fuego
I'm not sure what this is doing. It looks scary. Please explain.
> +WORKDIR /
>
> # ==============================================================================
> # Setup startup command
> # ==============================================================================
> -
> # FIXTHIS: when running multiple Fuego containers, or if the host is already
> # running the netperf server, netperf will complain because the port is taken
> -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
> +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
Isolating this to a separate script is nice!
> diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
> deleted file mode 100644
> index 56de98a..0000000
> --- a/Dockerfile.nojenkins
> +++ /dev/null
Getting rid of this separate Dockerfile is nice.
> @@ -1,151 +0,0 @@
> -# ==============================================================================
> -# WARNING: this Dockerfile assumes that the container will be created with
> -# several bind mounts (see docker-create-container.sh)
> -# ==============================================================================
> -# Usage:
> -# host$ vi fuego-ro/conf/fuego.conf
> -# jenkins_enabled=0
> -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
> -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
> -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
> -# docker# ftc list-boards
> -# docker, bbb, ...
> -# docker# ftc list-tests
> -# Functional.hello_world, ...
> -# docker# ftc run-test -b docker -t Functional.hello_world
> -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
> -
> -FROM debian:stretch-slim
> -MAINTAINER tim.bird@sony.com
> -
> -# ==============================================================================
> -# Proxy variables
> -# ==============================================================================
> -
> -ARG HTTP_PROXY
> -ENV http_proxy ${HTTP_PROXY}
> -ENV https_proxy ${HTTP_PROXY}
> -
> -# ==============================================================================
> -# Prepare basic image
> -# ==============================================================================
> -
> -ARG DEBIAN_FRONTEND=noninteractive
> -
> -WORKDIR /
> -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -
> -# Fuego python dependencies
> -# - python-lxml: ftc, loggen
> -# - python-simplejson: ftc
> -# - python-yaml: ftc
> -# - python-openpyxl: ftc (also LTP)
> -# - python-requests: ftc (also fuego_release_test)
> -# - python-reportlab: ftc
> -# - python-parsedatetime: ftc
> -# - python-pexpect: ssh_exec (part of ttc)
> -# - python-pip: to install filelock, flake8
> -# - filelock: parser
> -RUN mkdir -p /usr/share/man/man1
> -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> - python-lxml python-simplejson python-yaml python-openpyxl \
> - python-requests python-reportlab python-parsedatetime \
> - python-pexpect python-pip python-setuptools python-wheel
> -RUN pip install filelock
> -
> -# Fuego command dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion iproute2
> -
> -# Default SDK for testing locally or on an x86 board
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - build-essential cmake bison flex automake kmod libtool \
> - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> -
> -# Default test host dependencies
> -# - iperf iperf3 netperf: used as servers on the host
> -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> -# - python-matplotlib: Benchmark.iperf3 parser
> -# - python-xmltodict: AGL tests
> -# - flake8: Functional.fuego_lint
> -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> -# - iputils-ping - for /bin/ping command, used by some tests
> -# FIXTHIS: install dependencies dynamically on the tests that need them
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> - netpipe-tcp iputils-ping
> -RUN pip install flake8
> -
> -# miscelaneous packages:
> -# python-serial - used by serio
> -# diffstat and vim - used by Tim
> -# time - useful for timing command duration
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - python-serial \
> - diffstat \
> - vim \
> - time
> -
> -# FIXTHIS: determine if these tools are really necessary
> -# RUN apt-get -q=2 -V --no-install-recommends install \
> -# apt-utils python-paramiko \
> -# xmlstarlet rsync \
> -# inotify-tools gettext netpipe-tcp \
> -# at minicom lzop bsdmainutils \
> -# mc netcat openssh-server
> -
> -# bsdmainutils provides hexdump, which is needed for netperf test
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - bsdmainutils
> -
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> -
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -# as of ttc version 2.0.2, this should not be needed
> -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> -
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s
> /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> -
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server'
> -
> -# ==============================================================================
> -# ftc post installation
> -# ==============================================================================
> -
> -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> -
> -# ==============================================================================
> -# Lava
> -# ==============================================================================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -
> -# ==============================================================================
> -# Setup startup command
> -# ==============================================================================
> -
> -# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
> -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
> diff --git a/README b/README
> index 882d84e..ce42e1e 100644
> --- a/README
> +++ b/README
> @@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins
> by passing the corresponding parameters to install.sh.
> ./install.sh fuego-8082 8082
>
> +To run fuego on different distribution versions like Debian(stretch, buster, etc.)
> + pass appropriate arguments to the install.sh script
> + ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
should probably be:
./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
> +
> +By default install.sh script will install fuego on Debian stretch version
> + to install fuego on Debian buster version pass below options
> + ./install.sh fuego_buster 8090 debian buster
> +
> +[Note] It is currently verified on Debian versions (stretch and buster),
> +and yet to provide install scripts for other distributions
> +
> +Installing Fuego to a host system
> +-------------------------------------------------------------------------------
> +Run install-fuego.sh to install fuego on host system, by default it assume the
> +host system is Debian stretch
> + ./install-fuego.sh
> +
> + to install on Debian buster version
> + ./install-fuego.sh -d debian -s buster -p 8090
> +
> + to install without jenkins pass --nojenkins option to the script
> + ./install-fuego.sh --nojenkins
> +
> Running
> -------------------------------------------------------------------------------
> To start your Fuego container, issue the following command top directory:
> @@ -85,9 +108,9 @@ Updating the Jenkins version
> -------------------------------------------------------------------------------
>
> Modify the version and checksum
> - $ vi Dockerfile
> - ARG JENKINS_VERSION=2.32.1
> - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> + $ vi install-debian.sh
> + JENKINS_VERSION=2.32.1
> + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
>
> Re-build the flot plugin
> $ sudo apt-get install maven
> diff --git a/entrypoint.sh b/entrypoint.sh
> new file mode 100755
> index 0000000..6eac895
> --- /dev/null
> +++ b/entrypoint.sh
> @@ -0,0 +1,7 @@
> +#!/bin/bash
> +
> +[ "$1" != "--nojenkins" ] && service jenkins start
> +
> +#TODO: netperf not present in buster, remove this condition once alternative found
> +[ -f /etc/init.d/netperf ] && service netperf start
> +iperf3 -V -s -D -f M
> diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
> index 5823357..6889c47 100755
> --- a/fuego-host-scripts/docker-build-image.sh
> +++ b/fuego-host-scripts/docker-build-image.sh
> @@ -2,9 +2,10 @@
> # $1 - name for the docker image (default: fuego)
> # $2 - port for jenkins (default: 8090)
> # $3 - Dockerfile or Dockerfile.nojenkins
> +# $4 - Debian code name on which to install (default: stretch)
> #
> # Example:
> -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins
> +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins
> #
> if [ "$1" = "--no-cache" ]; then
> NO_CACHE=--no-cache
> @@ -13,7 +14,10 @@ fi
>
> DOCKERIMAGE=${1:-fuego}
> JENKINS_PORT=${2:-8090}
> -DOCKERFILE=${3:-Dockerfile}
> +DISTRIBUTION=${3:-debian}
> +SUITE=${4:-stretch}
> +NO_JENKINS=${5:""}
> +DOCKERFILE="Dockerfile"
>
> if [ "$(id -u)" == "0" ]; then
> JENKINS_UID=$(id -u $SUDO_USER)
> @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
>
> sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
> --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
> + --build-arg DIST=$DISTRIBUTION \
> + --build-arg SUITE=$SUITE \
> + --build-arg NO_JENKINS=$NO_JENKINS \
> --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
> diff --git a/install-buster.sh b/install-buster.sh
> new file mode 100755
> index 0000000..f4ce65b
> --- /dev/null
> +++ b/install-buster.sh
> @@ -0,0 +1,294 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> +#
> +usage(){
> +
> +cat <<HERE
> +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> +
> +Install fuego on the host Debian filesystem.
> +
> +options:
> + --help Show usage help
> + --nojenkins Install Fuego without Jenkins
> +HERE
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=0
> +
> +POSITIONAL=()
> +while [[ $# -gt 0 ]] ; do
> + case $1 in
> + --nojenkins)
> + nojenkins=1
> + shift
> + ;;
> + --help | -h)
> + usage
> + exit 0
> + ;;
> + *)
> + POSITIONAL+=("$1") # save argument for later
> + shift
> + esac
> +done
> +set -- "${POSITIONAL[@]}" # restore positional arguments
> +
> +port=${2:-8090}
> +
> +# ==============================================================================
> +# Install Fuego dependencies
> +# ==============================================================================
> +
> +mkdir -p /usr/share/man/man1
> +
> +# netperf is in non-free
> +echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list
> +
> +apt-get update -q=2
> +
> +# Fuego python dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + python-lxml python-simplejson python-yaml python-openpyxl \
> + python-requests python-reportlab python-parsedatetime \
> + python-pexpect python-pip python-setuptools python-wheel
> +pip install filelock
> +
Can we put this stuff that is common between install-buster.sh and install-stretch.sh
into an "install-debian-common.sh"?
> +# Fuego command dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> + bash-completion iproute2
> +
> +# Default SDK for testing locally or on an x86 board
> +apt-get -q=2 -V --no-install-recommends install \
> + build-essential cmake bison flex kmod automake libtool \
> + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> +
> +# Default test host dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
> +pip install flake8
> +
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +#apt-get -q=2 -V --no-install-recommends install \
> +# bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +
> +# TODO: check if proxy settings can set outside docker container
> +if [ -n "$http_proxy" ]; then
> + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +fi
This doesn't seem specific to Debian (/etc/wgetrc and /etc/environment).
Should this be moved to an 'install-common.sh' script?
> +
> +# ==============================================================================
> +# Install Jenkins
> +# ==============================================================================
> +
> +if [ $nojenkins -eq 0 ]; then
> + #JENKINS_VERSION=2.164.2
> + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> + #JENKINS_VERSION=2.249.3
> + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> + JENKINS_VERSION=2.164.3
> + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
> + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> + export JENKINS_UC=https://updates.jenkins.io
> + REF=/var/lib/jenkins/plugins
> + JENKINS_HOME=/var/lib/jenkins
> + JENKINS_PORT=$port
> +
> + # Jenkins dependencies
> + apt-get -q=2 -V --no-install-recommends install \
> + default-jdk daemon psmisc adduser procps unzip
> + pip install python-jenkins==1.4.0
> +
> + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> + export JENKINS_PORT=$port
> +
> + groupadd jenkins
> + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> +
> + wget -nv ${JENKINS_URL}
> + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> +
> + # allow Jenkins to start and install plugins, as part of dpkg installation
> + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> + rm jenkins_${JENKINS_VERSION}_all.deb
> +
> + # fix for Jenkins Java version check error
> + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version
> \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
> + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
> +
> + source /etc/default/jenkins && \
> + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> +
> + source /etc/default/jenkins && \
> + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> + if [ -n "$http_proxy" ]; then \
> + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> + fi && \
> + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> +
> + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/Jenkins
The above looks like there's a lot of non-Debian-specific stuff intermingled with Debian-specific stuff.
I'm not sure what other distributions are intended to be supported, but wouldn't the user and
group manipulation be the same for those?
> +
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> + /usr/local/bin/
> +
> + mkdir -p $JENKINS_HOME/userContent/docs
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +
> + # start and stop jenkins to pre-populate some settings
> + service jenkins start && \
> + sleep 30 && \
> + service jenkins stop
> +
> + # install other plugins from Jenkins update center
> + # NOTE: not sure all of these are needed, but keep list
> + # compatible with 1.2.1 release for now
> + #/usr/local/bin/install-plugins.sh \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # junit:1.27 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.4 \
> + # pegdown-formatter:1.3 \
> + # structs:1.17 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # make the mod.js symlink well after flot is installed
> + service jenkins start && sleep 30 && \
> + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> +
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
The above and following are definitely not unique to Debian, or a specific version of Debian.
They should be put into an "install-common.sh"
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> +else
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> +fi
> +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> +
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +apt-get -q=2 -V --no-install-recommends install jq
> +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> +chmod a+x /usr/local/bin/ebf
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# ftc post installation
> +# ==============================================================================
> +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +echo ". /etc/bash_completion" >> /root/.bashrc
> +
> +# ==============================================================================
> +# Lava
> +# ==============================================================================
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/install-debian.sh b/install-debian.sh
> index ccca032..2d3ffd3 100755
> --- a/install-debian.sh
> +++ b/install-debian.sh
> @@ -2,102 +2,163 @@
> # 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> #
> # Usage:
> -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> #
> -if [ -n "$1" ]; then
> - if [ "$1" = "--help" -o "$1" = "-h" ]; then
> - cat <<HERE
> -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> -
> -Installs fuego on the host Debian filesystem.
> -
> -options:
> - --help Show usage help
> - --nojenkins Install Fuego without Jenkins
> -HERE
> - exit 0
> - fi
> -fi
> +
> +usage(){
> + echo ""
> + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> + echo "Install fuego on Debian filesystem."
> + echo ""
> + echo "options:"
> + echo " --help | -h Show usage help"
> + echo " --suite | -s <suite> Specify Debian suite"
> + echo " --port | -p <port> Jenkins port"
> + echo " --nojenkins Install Fuego without Jenkins"
> + echo ""
> +}
>
> if [[ $EUID -ne 0 ]]; then
> echo "Sorry, you need root permissions"
> exit 1
> fi
>
> -if [ "$1" = "--nojenkins" ]; then
> - nojenkins=1
> +nojenkins=0
> +port=8090
> +suite=stretch
> +user=jenkins
> +group=jenkins
> +uid=1000
> +gid=1000
> +
> +while [ "$1" != "" ]; do
> + case $1 in
> + -s | --suite )
> + shift
> + suite=$1
> + ;;
> + -p | --port )
> + shift
> + port=$1
> + ;;
> + -u | --uid )
> + shift
> + uid=$1
> + ;;
> + -g | --gid )
> + shift
> + gid=$1
> + ;;
> + --nojenkins )
> + nojenkins=1
> + ;;
> + -h | --help )
> + usage
> + exit 0
> + ;;
> + * )
> + break
> + esac
> shift
> -else
> - nojenkins=0
> - port=${1:-8090}
> -fi
> +done
>
> # ==============================================================================
> # Install Fuego dependencies
> # ==============================================================================
>
> # netperf is in non-free
> -echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -
> -apt-get update -q=2
> +echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list
>
> # Fuego python dependencies
> -apt-get -q=2 -V --no-install-recommends install \
> +# - python-lxml: ftc, loggen
> +# - python-simplejson: ftc
> +# - python-yaml: ftc
> +# - python-openpyxl: ftc (also LTP)
> +# - python-requests: ftc (also fuego_release_test)
> +# - python-reportlab: ftc
> +# - python-parsedatetime: ftc
> +# - python-pexpect: ssh_exec (part of ttc)
> +# - python-pip: to install filelock, flake8
> +# - filelock: parser
> +mkdir -p /usr/share/man/man1
> +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> python-lxml python-simplejson python-yaml python-openpyxl \
> python-requests python-reportlab python-parsedatetime \
> - python-pip
> + python-pexpect python-pip python-setuptools python-wheel
> pip install filelock
>
> # Fuego command dependencies
> apt-get -q=2 -V --no-install-recommends install \
> git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion
> + bash-completion iproute2
>
> # Default SDK for testing locally or on an x86 board
> apt-get -q=2 -V --no-install-recommends install \
> build-essential cmake bison flex automake kmod libtool \
> libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
>
> # Default test host dependencies
> +# - iperf iperf3 netperf: used as servers on the host
> +# - bzip2 bc: used for local tests by Functional.bzip2/bc
> +# - python-matplotlib: Benchmark.iperf3 parser
> +# - python-xmltodict: AGL tests
> +# - flake8: Functional.fuego_lint
> +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> +# - iputils-ping - for /bin/ping command, used by some tests
> +# FIXTHIS: install dependencies dynamically on the tests that need them
> apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
> -pip install flake8 bsdmainutils
> + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
>
> -echo "dash dash/sh boolean false" | debconf-set-selections
> -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> -if [ -n "$http_proxy" ]; then
> - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +#TODO: find alternate package for netperf in buster version
> +if [ $suite != "buster" ];then
> + apt-get -q=2 -V --no-install-recommends install netperf
> fi
> +pip install flake8
>
> -# ==============================================================================
> -# Clone fuego and fuego-core
> -# ==============================================================================
> -if [ ! -d "/fuego" ]; then
> - cd /
> - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
> - ln -s /fuego/fuego-ro /fuego-ro
> - ln -s /fuego/fuego-rw /fuego-rw
> - cd fuego
> - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
> - ln -s /fuego/fuego-core /fuego-core
> -fi
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# FIXTHIS: determine if these tools are really necessary
> +#RUN apt-get -q=2 -V --no-install-recommends install \
> +# apt-utils python-paramiko \
> +# xmlstarlet rsync \
> +# inotify-tools gettext netpipe-tcp \
> +# at minicom lzop bsdmainutils \
> +# mc netcat openssh-server
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +apt-get -q=2 -V --no-install-recommends install \
> + bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
>
> # ==============================================================================
> # Install Jenkins
> # ==============================================================================
> -
> if [ $nojenkins -eq 0 ]; then
> #JENKINS_VERSION=2.164.2
> #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> JENKINS_VERSION=2.249.3
> JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> - JENKINS_UC=https://updates.jenkins.io
> - REF=/var/lib/jenkins/plugins
> - JENKINS_HOME=/var/lib/jenkins
> + export JENKINS_UC=https://updates.jenkins.io
> + export REF=/var/lib/jenkins/plugins
> + export JENKINS_HOME=/var/lib/jenkins
> JENKINS_PORT=$port
>
> # Jenkins dependencies
> @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
> pip install python-jenkins==1.4.0
>
> echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> - groupadd jenkins
> - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> - cd
> - wget ${JENKINS_URL}
> + export JENKINS_PORT=$port
> +
> + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> +
> + wget -nv ${JENKINS_URL}
> echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
>
> # allow Jenkins to start and install plugins, as part of dpkg installation
> @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
> dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> rm jenkins_${JENKINS_VERSION}_all.deb
>
> - # update ownership
> - chown -R jenkins:jenkins /fuego
> - chown -R jenkins:jenkins /fuego-ro
> - chown -R jenkins:jenkins /fuego-rw
> - chown -R jenkins:jenkins /fuego-core
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +# as of ttc version 2.0.2, this should not be needed
> +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + apt-get -q=2 -V --no-install-recommends install jq
> + git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> + chmod a+x /usr/local/bin/ebf
> +fi
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +git clone https://github.com/frowand/serio.git /usr/local/src/serio
> +if [ $nojenkins -eq 0 ]; then
> + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
> + chown -R jenkins /usr/local/src/serio
> +fi
> +cp /usr/local/src/serio/serio /usr/local/bin/
> +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> +
> +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
> +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
> +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# Jenkins post installation
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> +
> + # Fix jenkins issue, Java version check fails when java version is 11
> + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-
> \]*\\\).*\\\".*/\\\1\\\2\/p;')"
> + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
>
> source /etc/default/jenkins && \
> JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
>
> sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
>
> - cp /fuego/frontend-install/install-plugins.sh \
> - /fuego/frontend-install/jenkins-support \
> - /fuego/frontend-install/clitest \
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> /usr/local/bin/
>
> - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
> - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> - mkdir $JENKINS_HOME/userContent/docs
> - cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> - jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> - sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> - chown -R jenkins:jenkins $JENKINS_HOME/
> -
> # start and stop jenkins to pre-populate some settings
> service jenkins start && \
> sleep 30 && \
> service jenkins stop
>
> + # install plugins - these versions are for Jenkins version 2.164.2
> + # Explicitly install script-security v1.68, otherwise
> + # v1.74 will automatically be installed as a dependency of
> + # the junit plugin. Make sure to install before junit plugin.
> + # Explicitly install junit:1.27
> +
> # install other plugins from Jenkins update center
> # NOTE: not sure all of these are needed, but keep list
> - # compatible with 1.2.1 release for now
> - #/usr/local/bin/install-plugins.sh \
> - # ant:1.9 \
> - # antisamy-markup-formatter:1.5 \
> - # bouncycastle-api:2.17 \
> - # command-launcher:1.3 \
> - # description-setter:1.10 \
> - # display-url-api:2.3.1 \
> - # external-monitor-job:1.7 \
> - # greenballs:1.15 \
> - # icon-shim:2.0.3 \
> - # javadoc:1.5 \
> - # jdk-tool:1.2 \
> - # junit:1.27 \
> - # ldap:1.20 \
> - # mailer:1.23 \
> - # matrix-auth:2.3 \
> - # matrix-project:1.14 \
> - # pam-auth:1.4 \
> - # pegdown-formatter:1.3 \
> - # structs:1.17 \
> - # windows-slaves:1.4
> -
> - # install plugins for latest LTS Jenkins (currently 2.249.3)
> - /usr/local/bin/install-plugins.sh \
> - script-security \
> - structs \
> - workflow-step-api \
> - workflow-api \
> - junit \
> - scm-api \
> - ant \
> - antisamy-markup-formatter \
> - bouncycastle-api \
> - command-launcher \
> - description-setter \
> - display-url-api \
> - external-monitor-job \
> - greenballs \
> - icon-shim \
> - javadoc \
> - jdk-tool \
> - mailer \
> - matrix-auth \
> - matrix-project \
> - pam-auth \
> - pegdown-formatter \
> - windows-slaves
> -
> - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> + # as compatible as possible with 1.2.1 release for now
> + # Do NOT change the order of the plugins, unless you know what
> + # you are doing. Otherwise, the install-plugins.sh script will
> + # install the wrong versions of other plugins based on dependency
> + # information in the plugin files.
>
> + #/usr/local/bin/install-plugins.sh \
> + # script-security:1.68 \
> + # structs:1.20 \
> + # workflow-step-api:2.22 \
> + # workflow-api:2.40 \
> + # junit:1.27 \
> + # scm-api:2.6.3 \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.5 \
> + # pegdown-formatter:1.3 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # jenkins should automatically unzip any plugins in the plugin dir
> # make the mod.js symlink well after flot is installed
> service jenkins start && sleep 30 && \
> rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
>
> - chown -R jenkins:jenkins $JENKINS_HOME/
> -else
> - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
> -fi
> + mkdir -p $JENKINS_HOME/userContent/docs
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
>
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -/usr/local/src/ttc/install.sh /usr/local/bin
> -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
>
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -if [ $nojenkins -eq 0 ]; then
> - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> -else
> - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> + chown -R jenkins:jenkins $JENKINS_HOME/
> fi
>
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> -ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> -
> # ==============================================================================
> # ftc post installation
> # ==============================================================================
> ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> echo ". /etc/bash_completion" >> /root/.bashrc
>
> # ==============================================================================
> @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
> # ==============================================================================
> ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -
> -# ==============================================================================
> -# Small guide
> -# ==============================================================================
> -echo "Run 'service netperf start' to start a netperf server"
> -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> -echo "Run 'ftc list-boards' to see the available boards"
> -echo "Run 'ftc list-tests' to see the available tests"
> -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> -echo "Run 'ftc gen-report' to get results"
> diff --git a/install-fuego.sh b/install-fuego.sh
> new file mode 100755
> index 0000000..7b3e010
> --- /dev/null
> +++ b/install-fuego.sh
> @@ -0,0 +1,109 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> +#
> +
> +usage(){
> + echo ""
> + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> + echo "Install fuego on Debian filesystem."
> + echo ""
> + echo "options:"
> + echo " --help | -h Show usage help"
> + echo " --dist | -d <distribution> Specify distribution"
> + echo " --suite | -s <suite> Specify suite"
> + echo " --port | -p <port> Jenkins port"
> + echo " --nojenkins Install Fuego without Jenkins"
> + echo ""
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=
> +port=8090
> +dist=debian
> +suite=stretch
> +fuego_dir=$(pwd)
> +
> +while [ "$1" != "" ]; do
> + case $1 in
> + -d | --dist )
> + shift
> + dist=$1
> + ;;
> + -s | --suite )
> + shift
> + suite=$1
> + ;;
> + -p | --port )
> + shift
> + port=$1
> + ;;
> + --nojenkins )
> + nojenkins="--nojenkins"
> + ;;
> + -h | --help )
> + usage
> + exit 0
> + ;;
> + * )
> + break
> + esac
> + shift
> +done
> +
> +# ==============================================================================
> +# Clone fuego-core
> +# ==============================================================================
> +if [ ! -f "fuego-core/scripts/ftc" ]; then
> + # set fuego-core branch to same as current fuego branch
> + # get current git branch
> + set -o noglob
> + while IFS=" " read -r part1 part2 ; do
> + if [ $part1 = "*" ] ; then
> + branch=$part2
> + fi
> + done < <(git branch)
> + set +o noglob
> + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
> +fi
> +
> +# ==============================================================================
> +# Install fuego on host system
> +# ==============================================================================
> +./install-$dist.sh -s $suite -p $port $nojenkins
> +
> +systemctl daemon-reload
> +service jenkins restart
> +
> +# provide permission to current user
> +usermod -aG jenkins $USER
> +chmod -R 775 $fuego_dir/fuego-rw
> +
> +chown -R jenkins:jenkins $fuego_dir/fuego-ro
> +chown -R jenkins:jenkins $fuego_dir/fuego-rw
> +chown -R jenkins:jenkins $fuego_dir/fuego-core
> +
> +# ==============================================================================
> +# Create Links to fuego folders
> +# ==============================================================================
> +ln -s $fuego_dir/fuego-ro /
> +ln -s $fuego_dir/fuego-rw /
> +ln -s $fuego_dir/fuego-core /
> +
> +# ==============================================================================
> +# Small guide
> +# ==============================================================================
> +echo "Logout and Login current user session to work properly"
> +echo "Run 'service netperf start' to start a netperf server"
> +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> +echo "Run 'ftc list-boards' to see the available boards"
> +echo "Run 'ftc list-tests' to see the available tests"
> +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> +echo "Run 'ftc gen-report' to get results"
> diff --git a/install-stretch.sh b/install-stretch.sh
> new file mode 100755
> index 0000000..4f6f724
> --- /dev/null
> +++ b/install-stretch.sh
> @@ -0,0 +1,288 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> +#
> +usage(){
> +
> +cat <<HERE
> +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> +
> +Install fuego on the host Debian filesystem.
> +
> +options:
> + --help Show usage help
> + --nojenkins Install Fuego without Jenkins
> +HERE
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=0
> +
> +POSITIONAL=()
> +while [[ $# -gt 0 ]] ; do
> + case $1 in
> + --nojenkins)
> + nojenkins=1
> + shift
> + ;;
> + --help | -h)
> + usage
> + exit 0
> + ;;
> + *)
> + POSITIONAL+=("$1") # save argument for later
> + shift
> + esac
> +done
> +set -- "${POSITIONAL[@]}" # restore positional arguments
> +
> +port=${2:-8090}
> +
> +# ==============================================================================
> +# Install Fuego dependencies
> +# ==============================================================================
> +
> +mkdir -p /usr/share/man/man1
> +
> +# netperf is in non-free
> +echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> +
> +apt-get update -q=2
> +
> +# Fuego python dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + python-lxml python-simplejson python-yaml python-openpyxl \
> + python-requests python-reportlab python-parsedatetime \
> + python-pexpect python-pip python-setuptools python-wheel
> +pip install filelock
> +
> +# Fuego command dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> + bash-completion iproute2
> +
> +# Default SDK for testing locally or on an x86 board
> +apt-get -q=2 -V --no-install-recommends install \
> + build-essential cmake bison flex kmod automake libtool \
> + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> +
> +# Default test host dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
> +pip install flake8
> +
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +apt-get -q=2 -V --no-install-recommends install \
> + bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +
> +# TODO: check if proxy settings can set outside docker container
> +#if [ -n "$http_proxy" ]; then
> +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +#fi
> +
> +# ==============================================================================
> +# Install Jenkins
> +# ==============================================================================
> +
> +if [ $nojenkins -eq 0 ]; then
> + #JENKINS_VERSION=2.164.2
> + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> + JENKINS_VERSION=2.249.3
> + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> + export JENKINS_UC=https://updates.jenkins.io
> + REF=/var/lib/jenkins/plugins
> + JENKINS_HOME=/var/lib/jenkins
> + JENKINS_PORT=$port
> +
> + # Jenkins dependencies
> + apt-get -q=2 -V --no-install-recommends install \
> + default-jdk daemon psmisc adduser procps unzip
> + pip install python-jenkins==1.4.0
> +
> + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> + export JENKINS_PORT=$port
> +
> + groupadd jenkins
> + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> +
> + wget -nv ${JENKINS_URL}
> + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> +
> + # allow Jenkins to start and install plugins, as part of dpkg installation
> + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> + rm jenkins_${JENKINS_VERSION}_all.deb
> +
> + source /etc/default/jenkins && \
> + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> +
> + source /etc/default/jenkins && \
> + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> + if [ -n "$http_proxy" ]; then \
> + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> + fi && \
> + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> +
> + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> +
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> + /usr/local/bin/
> +
> + mkdir -p $JENKINS_HOME/userContent/docs
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +
> + # start and stop jenkins to pre-populate some settings
> + service jenkins start && \
> + sleep 30 && \
> + service jenkins stop
> +
> + # install other plugins from Jenkins update center
> + # NOTE: not sure all of these are needed, but keep list
> + # compatible with 1.2.1 release for now
> + #/usr/local/bin/install-plugins.sh \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # junit:1.27 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.4 \
> + # pegdown-formatter:1.3 \
> + # structs:1.17 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # make the mod.js symlink well after flot is installed
> + service jenkins start && sleep 30 && \
> + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> +
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> +else
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> +fi
> +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> +
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +apt-get -q=2 -V --no-install-recommends install jq
> +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> +chmod a+x /usr/local/bin/ebf
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# ftc post installation
> +# ==============================================================================
> +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +echo ". /etc/bash_completion" >> /root/.bashrc
> +
> +# ==============================================================================
> +# Lava
> +# ==============================================================================
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/install.sh b/install.sh
> index a5a90b0..259c887 100755
> --- a/install.sh
> +++ b/install.sh
> @@ -1,13 +1,13 @@
> #!/bin/bash
> #
> -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
> #
>
> # $1 is the exit code after usage is shown
> usage() {
> exit_code="$1"
> cat <<HERE
> -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
>
> Create the docker image and container with the Fuego test distribution.
> If no <image_name> is provided, the image will be named 'fuego'.
> @@ -34,7 +34,7 @@ fi
>
> priv=0
> NOCACHE=""
> -dockerfile="Dockerfile"
> +NOJENKINS=""
>
> POSITIONAL=()
> while [[ $# -gt 0 ]] ; do
> @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
> shift
> ;;
> --nojenkins)
> - dockerfile="Dockerfile.nojenkins"
> + NOJENKINS="--nojenkins"
> shift
> ;;
> *)
> @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
>
> image_name=${1:-fuego}
> jenkins_port=${2:-8090}
> +distribution=${3:-debian}
> +suite=${4:-stretch}
>
> container_name="${image_name}-container"
>
> @@ -79,7 +81,7 @@ fi
>
> set -e
>
> -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
> +source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
> if [ "$priv" == "0" ]; then
> fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
> else
> diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
> new file mode 100644
> index 0000000..b899e7c
> --- /dev/null
> +++ b/uninstall-fuego.sh
> @@ -0,0 +1,47 @@
> +#!/bin/bash
> +
> +# Remove Jenkins plugins
> +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
> +
> +# uninstall Jenkins
> +service jenkins stop
> +dpkg -P jenkins
> +
> +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
> +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
> +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
> +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
> +sed -i '/^JENKINS_PORT=/d' /etc/environment
> +
> +# uninstall ttc script
> +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
> +
> +# uninstall ebf script
> +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
> +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
> +
> +# uninstall serial config
> +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
> +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
> +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
> +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
> +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
> +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
> +
> +# uninstall fserver
> +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
> +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
> +
> +# uninstall ftc
> +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
> +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
> +
> +# remove lava target files
> +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
> +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
> +
> +# remove links
> +[ -d /fuego-ro ] && rm /fuego-ro
> +[ -d /fuego-rw ] && rm /fuego-rw
> +[ -d /fuego-core ] && rm /fuego-core
> +
> --
> 2.20.1
>
Overall - this looks like a good start. But I'd like to see the duplicate sections of install-buster.sh
and install-stretch.sh put into a single location "install-debian-common.sh". Also, I'd like to see
the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
Can you please re-do this with that refactoring in mind?
Thanks,
-- Tim
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-15 0:08 ` Tim.Bird
@ 2021-06-15 3:46 ` Venkata.Pyla
2021-06-16 17:47 ` Tim.Bird
2021-06-17 7:01 ` Venkata.Pyla
1 sibling, 1 reply; 8+ messages in thread
From: Venkata.Pyla @ 2021-06-15 3:46 UTC (permalink / raw)
To: Tim.Bird, tbird20d; +Cc: fuego
Hello Tim,
I sent patch based on the master branch of this repository https://bitbucket.org/fuegotest/fuego/src/master/
And the previous commit id was "3c813c717f1ebd7dc279ecdb0f288e4f9d30d6d7".
Also created a PR (https://bitbucket.org/fuegotest/fuego/pull-requests/4) to visualize the changes.
Please let me know if I need to send another patch with any specific branch or version?
Thanks,
Venkata.
-----Original Message-----
From: Tim.Bird@sony.com <Tim.Bird@sony.com>
Sent: 15 June 2021 05:39
To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com>; tbird20d@yahoo.com
Cc: fuego@lists.linuxfoundation.org; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>
Subject: RE: [Fuego] Use common script to install fuego on host or docker system
Please see comments inline below.
I tried to apply this patch from the e-mail list using 'git am', but got a failure.
That made it a hard to experiment with and view the changes in
context.
What version of the code is this patch based on?
> -----Original Message-----
> From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> Subject: [Fuego] Use common script to install fuego on host or docker system
>
> From: venkata pyla <venkata.pyla@toshiba-tsip.com>
>
> Modified the install scripts to use common scripts for both types of
> installation, this way it will help to make sure both the environments
> are same and if any future modification will have single place of change.
>
> Now, for installing fuego on docker system the 'install.sh' script will
> use common script in Dockefile to install fuego related stuff.
>
> sameway for installing fuego on host system it can use the same common
> script to install fuego related stuff.
>
> Below are the common scripts that are used in both the installation types
> for each debian distribution
> * install-stretch.sh
> * install-buster.sh
>
> so finally,
> To install fuego on docker system use the below script with options
> $ ./install.sh <options> image_name <port> <distribution> <suite>
> e.g: ./install.sh debian_stretch 8090 debian stretch
> or
> ./install.sh debian_buster 8090 debian buster
> or
> ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
This is a bit wordy, but OK.
If we don't support any other distributions besides Debian at the moment,
then I'm not sure we need the distribution argument. Do you have any
non-debian distribution install scripts ready to submit soon?
I'm not sure I follow calling the Debian release name the "<suite>".
>
> To install fuego on host system use the below script with options
> $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
"suite" seems like an odd name. Should this be distro "version" or "release"?
> e.g: ./install-fuego.sh -d debian -s stretch -p 8090
> or
> ./install-fuego.sh -d debian -s buster -p 8090
> or
> ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
>
> Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> ---
> Dockerfile | 303 ++---------------
> Dockerfile.nojenkins | 151 ---------
> README | 29 +-
> entrypoint.sh | 7 +
> fuego-host-scripts/docker-build-image.sh | 11 +-
> install-buster.sh | 294 ++++++++++++++++
> install-debian.sh | 410 ++++++++++++++---------
> install-fuego.sh | 109 ++++++
> install-stretch.sh | 288 ++++++++++++++++
Is install-debian.sh the common parts for a Debian installation, with
buster and stretch having the parts that are unique to each of
those distros respectively? If we're going to refactor this, I
want to eliminate any duplication, so that package names are
not used in multiple places. This has been a headache to
maintain.
It appears there is lots of duplication between install-buster.sh and
install-stretch.sh. Can we possible put this duplication into an
"install-debian-common.sh", and reserve install-buster.sh and
install-stretch.sh for only the parts unique to those distros?
> install.sh | 12 +-
> uninstall-fuego.sh | 47 +++
Ooo - I like the sound of this uninstall script.
> 11 files changed, 1057 insertions(+), 604 deletions(-)
> delete mode 100644 Dockerfile.nojenkins
> create mode 100755 entrypoint.sh
> create mode 100755 install-buster.sh
> create mode 100755 install-fuego.sh
> create mode 100755 install-stretch.sh
> create mode 100644 uninstall-fuego.sh
>
> diff --git a/Dockerfile b/Dockerfile
> index 7f63798..ac57cb2 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -2,9 +2,11 @@
> # WARNING: this Dockerfile assumes that the container will be created with
> # several bind mounts (see docker-create-container.sh)
> # ==============================================================================
> -# FIXTHIS: build this as an extension of the nonjenkins image
>
> -FROM debian:stretch-slim
> +ARG SUITE=stretch
> +ARG DIST=debian
> +
> +FROM ${DIST}:${SUITE}-slim
> MAINTAINER tim.bird@sony.com
>
> # ==============================================================================
> @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
> ENV https_proxy ${HTTP_PROXY}
>
> # ==============================================================================
> -# Prepare basic image
> +# Install fuego with the script
> # ==============================================================================
>
> -ARG DEBIAN_FRONTEND=noninteractive
> -
> -WORKDIR /
> -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -
> -# Fuego python dependencies
> -# - python-lxml: ftc, loggen
> -# - python-simplejson: ftc
> -# - python-yaml: ftc
> -# - python-openpyxl: ftc (also LTP)
> -# - python-requests: ftc (also fuego_release_test)
> -# - python-reportlab: ftc
> -# - python-parsedatetime: ftc
> -# - python-pexpect: ssh_exec (part of ttc)
> -# - python-pip: to install filelock, flake8
> -# - filelock: parser
> -RUN mkdir -p /usr/share/man/man1
> -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> - python-lxml python-simplejson python-yaml python-openpyxl \
> - python-requests python-reportlab python-parsedatetime \
> - python-pexpect python-pip python-setuptools python-wheel
> -RUN pip install filelock
> -
> -# Fuego command dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion iproute2
> -
> -# Default SDK for testing locally or on an x86 board
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - build-essential cmake bison flex automake kmod libtool \
> - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> -
> -# Default test host dependencies
> -# - iperf iperf3 netperf: used as servers on the host
> -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> -# - python-matplotlib: Benchmark.iperf3 parser
> -# - python-xmltodict: AGL tests
> -# - flake8: Functional.fuego_lint
> -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> -# - iputils-ping - for /bin/ping command, used by some tests
> -# FIXTHIS: install dependencies dynamically on the tests that need them
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> - netpipe-tcp iputils-ping
> -RUN pip install flake8
> -
> -# miscelaneous packages:
> -# python-serial - used by serio
> -# diffstat and vim - used by Tim
> -# time - useful for timing command duration
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - python-serial \
> - diffstat \
> - vim \
> - time
> -
> -# install packages used by NuttX SDK
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - genromfs
> -
> -# FIXTHIS: determine if these tools are really necessary
> -#RUN apt-get -q=2 -V --no-install-recommends install \
> -# apt-utils python-paramiko \
> -# xmlstarlet rsync \
> -# inotify-tools gettext netpipe-tcp \
> -# at minicom lzop bsdmainutils \
> -# mc netcat openssh-server
> -
> -# bsdmainutils provides hexdump, which is needed for netperf test
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - bsdmainutils
> -
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> -
> -# ==============================================================================
> -# Install Jenkins with the same UID/GID as the host user
> -# ==============================================================================
> +RUN mkdir /fuego
> +COPY . /fuego/
> +WORKDIR /fuego
>
> -ARG user=jenkins
> -ARG group=jenkins
> -ARG uid=1000
> -ARG gid=${uid}
> ARG JENKINS_PORT=8090
> -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
> -#ARG JENKINS_VERSION=2.164.2
> -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> -#ARG JENKINS_VERSION=2.235.3
> -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
> -ARG JENKINS_VERSION=2.249.3
> -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> -
> -ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> -ARG JENKINS_UC=https://updates.jenkins.io
> -ARG REF=/var/lib/jenkins/plugins
> +ARG NO_JENKINS=
> +ARG SUITE=stretch
> +ARG DIST=debian
> +ARG uid=
> +ARG gid=
> ENV JENKINS_HOME=/var/lib/jenkins
> ENV JENKINS_PORT=$JENKINS_PORT
> +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS
One question about this overall approach. Does this retain the ability to cache
operations, so that if something goes wrong with an install, you don't have to
start over from scratch. When building with docker, with multiple steps, it
caches the image at each step. When rebuilding, or when an error occurs,
this can save a lot of time. It also can introduce subtle bugs, but that's
a separate issue.
This setup seems like it performs the bulk of the install as a single docker
step, is that right?
> -# Jenkins dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - default-jdk daemon psmisc adduser procps unzip
> -RUN pip install python-jenkins==1.4.0
> -
> -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> -RUN wget -nv ${JENKINS_URL}
> -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> -# allow Jenkins to start and install plugins, as part of dpkg installation
> -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> -
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -# as of ttc version 2.0.2, this should not be needed
> -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> -
> -# ==============================================================================
> -# get ebf script and helpers
> -# ==============================================================================
> -RUN apt-get -q=2 -V --no-install-recommends install jq
> -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
> - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
> - chmod a+x /usr/local/bin/ebf
> -
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -
> -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
> -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> - frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> - /tmp/
> -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
> - chown -R jenkins /usr/local/src/serio ; \
> - cp /usr/local/src/serio/serio /usr/local/bin/ ; \
> - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
> - ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp
> /usr/local/src/serlogin/serlogin /usr/local/bin'
> -
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
> - ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
> -
> -# ==============================================================================
> -# Jenkins post installation
> -# ==============================================================================
> -
> -RUN source /etc/default/jenkins && \
> - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> -
> -RUN source /etc/default/jenkins && \
> - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> - if [ -n "$HTTP_PROXY" ]; then \
> - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> - fi && \
> - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> -
> -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> -
> -# set up Jenkins plugins
> -COPY frontend-install/install-plugins.sh \
> - frontend-install/jenkins-support \
> - frontend-install/clitest \
> - /usr/local/bin/
> -
> -# start Jenkins onces to initialize directories?
> -RUN service jenkins start && \
> - sleep 30 && \
> - service jenkins stop
> -
> -# install plugins - these versions are for Jenkins version 2.164.2
> -# Explicitly install script-security v1.68, otherwise
> -# v1.74 will automatically be installed as a dependency of
> -# the junit plugin. Make sure to install before junit plugin.
> -# Explicitly install junit:1.27
> -
> -# install other plugins from Jenkins update center
> -# NOTE: not sure all of these are needed, but keep list
> -# as compatible as possible with 1.2.1 release for now
> -# Do NOT change the order of the plugins, unless you know what
> -# you are doing. Otherwise, the install-plugins.sh script will
> -# install the wrong versions of other plugins based on dependency
> -# information in the plugin files.
> -
> -#RUN /usr/local/bin/install-plugins.sh \
> -# script-security:1.68 \
> -# structs:1.20 \
> -# workflow-step-api:2.22 \
> -# workflow-api:2.40 \
> -# junit:1.27 \
> -# scm-api:2.6.3 \
> -# ant:1.9 \
> -# antisamy-markup-formatter:1.5 \
> -# bouncycastle-api:2.17 \
> -# command-launcher:1.3 \
> -# description-setter:1.10 \
> -# display-url-api:2.3.1 \
> -# external-monitor-job:1.7 \
> -# greenballs:1.15 \
> -# icon-shim:2.0.3 \
> -# javadoc:1.5 \
> -# jdk-tool:1.2 \
> -# ldap:1.20 \
> -# mailer:1.23 \
> -# matrix-auth:2.3 \
> -# matrix-project:1.14 \
> -# pam-auth:1.5 \
> -# pegdown-formatter:1.3 \
> -# windows-slaves:1.4
> -
> -# install plugins for latest LTS Jenkins (currently 2.249.3)
> -RUN /usr/local/bin/install-plugins.sh \
> - script-security \
> - structs \
> - workflow-step-api \
> - workflow-api \
> - junit \
> - scm-api \
> - ant \
> - antisamy-markup-formatter \
> - bouncycastle-api \
> - command-launcher \
> - description-setter \
> - display-url-api \
> - external-monitor-job \
> - greenballs \
> - icon-shim \
> - javadoc \
> - jdk-tool \
> - mailer \
> - matrix-auth \
> - matrix-project \
> - pam-auth \
> - pegdown-formatter \
> - windows-slaves
> +COPY entrypoint.sh /usr/bin/
>
> -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> -
> -# jenkins should automatically unzip any plugins in the plugin dir
> -# make a symlink for mod.js well after flot is installed
> -RUN service jenkins start && sleep 30 && \
> - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> -
> -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> -
> -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> -RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> -
> -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> -
> -# ==============================================================================
> -# ftc post installation
> -# ==============================================================================
> -
> -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> -
> -# ==============================================================================
> -# Lava
> -# ==============================================================================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> +# Delete the files that are copied for installation
> +RUN rm -rf /fuego
I'm not sure what this is doing. It looks scary. Please explain.
> +WORKDIR /
>
> # ==============================================================================
> # Setup startup command
> # ==============================================================================
> -
> # FIXTHIS: when running multiple Fuego containers, or if the host is already
> # running the netperf server, netperf will complain because the port is taken
> -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
> +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
Isolating this to a separate script is nice!
> diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
> deleted file mode 100644
> index 56de98a..0000000
> --- a/Dockerfile.nojenkins
> +++ /dev/null
Getting rid of this separate Dockerfile is nice.
> @@ -1,151 +0,0 @@
> -# ==============================================================================
> -# WARNING: this Dockerfile assumes that the container will be created with
> -# several bind mounts (see docker-create-container.sh)
> -# ==============================================================================
> -# Usage:
> -# host$ vi fuego-ro/conf/fuego.conf
> -# jenkins_enabled=0
> -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
> -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
> -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
> -# docker# ftc list-boards
> -# docker, bbb, ...
> -# docker# ftc list-tests
> -# Functional.hello_world, ...
> -# docker# ftc run-test -b docker -t Functional.hello_world
> -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
> -
> -FROM debian:stretch-slim
> -MAINTAINER tim.bird@sony.com
> -
> -# ==============================================================================
> -# Proxy variables
> -# ==============================================================================
> -
> -ARG HTTP_PROXY
> -ENV http_proxy ${HTTP_PROXY}
> -ENV https_proxy ${HTTP_PROXY}
> -
> -# ==============================================================================
> -# Prepare basic image
> -# ==============================================================================
> -
> -ARG DEBIAN_FRONTEND=noninteractive
> -
> -WORKDIR /
> -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -
> -# Fuego python dependencies
> -# - python-lxml: ftc, loggen
> -# - python-simplejson: ftc
> -# - python-yaml: ftc
> -# - python-openpyxl: ftc (also LTP)
> -# - python-requests: ftc (also fuego_release_test)
> -# - python-reportlab: ftc
> -# - python-parsedatetime: ftc
> -# - python-pexpect: ssh_exec (part of ttc)
> -# - python-pip: to install filelock, flake8
> -# - filelock: parser
> -RUN mkdir -p /usr/share/man/man1
> -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> - python-lxml python-simplejson python-yaml python-openpyxl \
> - python-requests python-reportlab python-parsedatetime \
> - python-pexpect python-pip python-setuptools python-wheel
> -RUN pip install filelock
> -
> -# Fuego command dependencies
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion iproute2
> -
> -# Default SDK for testing locally or on an x86 board
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - build-essential cmake bison flex automake kmod libtool \
> - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> -
> -# Default test host dependencies
> -# - iperf iperf3 netperf: used as servers on the host
> -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> -# - python-matplotlib: Benchmark.iperf3 parser
> -# - python-xmltodict: AGL tests
> -# - flake8: Functional.fuego_lint
> -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> -# - iputils-ping - for /bin/ping command, used by some tests
> -# FIXTHIS: install dependencies dynamically on the tests that need them
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> - netpipe-tcp iputils-ping
> -RUN pip install flake8
> -
> -# miscelaneous packages:
> -# python-serial - used by serio
> -# diffstat and vim - used by Tim
> -# time - useful for timing command duration
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - python-serial \
> - diffstat \
> - vim \
> - time
> -
> -# FIXTHIS: determine if these tools are really necessary
> -# RUN apt-get -q=2 -V --no-install-recommends install \
> -# apt-utils python-paramiko \
> -# xmlstarlet rsync \
> -# inotify-tools gettext netpipe-tcp \
> -# at minicom lzop bsdmainutils \
> -# mc netcat openssh-server
> -
> -# bsdmainutils provides hexdump, which is needed for netperf test
> -RUN apt-get -q=2 -V --no-install-recommends install \
> - bsdmainutils
> -
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> -
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -# as of ttc version 2.0.2, this should not be needed
> -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> -
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s
> /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> -
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server'
> -
> -# ==============================================================================
> -# ftc post installation
> -# ==============================================================================
> -
> -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> -
> -# ==============================================================================
> -# Lava
> -# ==============================================================================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -
> -# ==============================================================================
> -# Setup startup command
> -# ==============================================================================
> -
> -# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
> -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
> diff --git a/README b/README
> index 882d84e..ce42e1e 100644
> --- a/README
> +++ b/README
> @@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins
> by passing the corresponding parameters to install.sh.
> ./install.sh fuego-8082 8082
>
> +To run fuego on different distribution versions like Debian(stretch, buster, etc.)
> + pass appropriate arguments to the install.sh script
> + ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
should probably be:
./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
> +
> +By default install.sh script will install fuego on Debian stretch version
> + to install fuego on Debian buster version pass below options
> + ./install.sh fuego_buster 8090 debian buster
> +
> +[Note] It is currently verified on Debian versions (stretch and buster),
> +and yet to provide install scripts for other distributions
> +
> +Installing Fuego to a host system
> +-------------------------------------------------------------------------------
> +Run install-fuego.sh to install fuego on host system, by default it assume the
> +host system is Debian stretch
> + ./install-fuego.sh
> +
> + to install on Debian buster version
> + ./install-fuego.sh -d debian -s buster -p 8090
> +
> + to install without jenkins pass --nojenkins option to the script
> + ./install-fuego.sh --nojenkins
> +
> Running
> -------------------------------------------------------------------------------
> To start your Fuego container, issue the following command top directory:
> @@ -85,9 +108,9 @@ Updating the Jenkins version
> -------------------------------------------------------------------------------
>
> Modify the version and checksum
> - $ vi Dockerfile
> - ARG JENKINS_VERSION=2.32.1
> - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> + $ vi install-debian.sh
> + JENKINS_VERSION=2.32.1
> + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
>
> Re-build the flot plugin
> $ sudo apt-get install maven
> diff --git a/entrypoint.sh b/entrypoint.sh
> new file mode 100755
> index 0000000..6eac895
> --- /dev/null
> +++ b/entrypoint.sh
> @@ -0,0 +1,7 @@
> +#!/bin/bash
> +
> +[ "$1" != "--nojenkins" ] && service jenkins start
> +
> +#TODO: netperf not present in buster, remove this condition once alternative found
> +[ -f /etc/init.d/netperf ] && service netperf start
> +iperf3 -V -s -D -f M
> diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
> index 5823357..6889c47 100755
> --- a/fuego-host-scripts/docker-build-image.sh
> +++ b/fuego-host-scripts/docker-build-image.sh
> @@ -2,9 +2,10 @@
> # $1 - name for the docker image (default: fuego)
> # $2 - port for jenkins (default: 8090)
> # $3 - Dockerfile or Dockerfile.nojenkins
> +# $4 - Debian code name on which to install (default: stretch)
> #
> # Example:
> -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins
> +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins
> #
> if [ "$1" = "--no-cache" ]; then
> NO_CACHE=--no-cache
> @@ -13,7 +14,10 @@ fi
>
> DOCKERIMAGE=${1:-fuego}
> JENKINS_PORT=${2:-8090}
> -DOCKERFILE=${3:-Dockerfile}
> +DISTRIBUTION=${3:-debian}
> +SUITE=${4:-stretch}
> +NO_JENKINS=${5:""}
> +DOCKERFILE="Dockerfile"
>
> if [ "$(id -u)" == "0" ]; then
> JENKINS_UID=$(id -u $SUDO_USER)
> @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
>
> sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
> --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
> + --build-arg DIST=$DISTRIBUTION \
> + --build-arg SUITE=$SUITE \
> + --build-arg NO_JENKINS=$NO_JENKINS \
> --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
> diff --git a/install-buster.sh b/install-buster.sh
> new file mode 100755
> index 0000000..f4ce65b
> --- /dev/null
> +++ b/install-buster.sh
> @@ -0,0 +1,294 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> +#
> +usage(){
> +
> +cat <<HERE
> +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> +
> +Install fuego on the host Debian filesystem.
> +
> +options:
> + --help Show usage help
> + --nojenkins Install Fuego without Jenkins
> +HERE
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=0
> +
> +POSITIONAL=()
> +while [[ $# -gt 0 ]] ; do
> + case $1 in
> + --nojenkins)
> + nojenkins=1
> + shift
> + ;;
> + --help | -h)
> + usage
> + exit 0
> + ;;
> + *)
> + POSITIONAL+=("$1") # save argument for later
> + shift
> + esac
> +done
> +set -- "${POSITIONAL[@]}" # restore positional arguments
> +
> +port=${2:-8090}
> +
> +# ==============================================================================
> +# Install Fuego dependencies
> +# ==============================================================================
> +
> +mkdir -p /usr/share/man/man1
> +
> +# netperf is in non-free
> +echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list
> +
> +apt-get update -q=2
> +
> +# Fuego python dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + python-lxml python-simplejson python-yaml python-openpyxl \
> + python-requests python-reportlab python-parsedatetime \
> + python-pexpect python-pip python-setuptools python-wheel
> +pip install filelock
> +
Can we put this stuff that is common between install-buster.sh and install-stretch.sh
into an "install-debian-common.sh"?
> +# Fuego command dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> + bash-completion iproute2
> +
> +# Default SDK for testing locally or on an x86 board
> +apt-get -q=2 -V --no-install-recommends install \
> + build-essential cmake bison flex kmod automake libtool \
> + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> +
> +# Default test host dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
> +pip install flake8
> +
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +#apt-get -q=2 -V --no-install-recommends install \
> +# bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +
> +# TODO: check if proxy settings can set outside docker container
> +if [ -n "$http_proxy" ]; then
> + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +fi
This doesn't seem specific to Debian (/etc/wgetrc and /etc/environment).
Should this be moved to an 'install-common.sh' script?
> +
> +# ==============================================================================
> +# Install Jenkins
> +# ==============================================================================
> +
> +if [ $nojenkins -eq 0 ]; then
> + #JENKINS_VERSION=2.164.2
> + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> + #JENKINS_VERSION=2.249.3
> + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> + JENKINS_VERSION=2.164.3
> + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
> + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> + export JENKINS_UC=https://updates.jenkins.io
> + REF=/var/lib/jenkins/plugins
> + JENKINS_HOME=/var/lib/jenkins
> + JENKINS_PORT=$port
> +
> + # Jenkins dependencies
> + apt-get -q=2 -V --no-install-recommends install \
> + default-jdk daemon psmisc adduser procps unzip
> + pip install python-jenkins==1.4.0
> +
> + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> + export JENKINS_PORT=$port
> +
> + groupadd jenkins
> + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> +
> + wget -nv ${JENKINS_URL}
> + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> +
> + # allow Jenkins to start and install plugins, as part of dpkg installation
> + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> + rm jenkins_${JENKINS_VERSION}_all.deb
> +
> + # fix for Jenkins Java version check error
> + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version
> \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
> + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
> +
> + source /etc/default/jenkins && \
> + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> +
> + source /etc/default/jenkins && \
> + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> + if [ -n "$http_proxy" ]; then \
> + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> + fi && \
> + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> +
> + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/Jenkins
The above looks like there's a lot of non-Debian-specific stuff intermingled with Debian-specific stuff.
I'm not sure what other distributions are intended to be supported, but wouldn't the user and
group manipulation be the same for those?
> +
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> + /usr/local/bin/
> +
> + mkdir -p $JENKINS_HOME/userContent/docs
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +
> + # start and stop jenkins to pre-populate some settings
> + service jenkins start && \
> + sleep 30 && \
> + service jenkins stop
> +
> + # install other plugins from Jenkins update center
> + # NOTE: not sure all of these are needed, but keep list
> + # compatible with 1.2.1 release for now
> + #/usr/local/bin/install-plugins.sh \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # junit:1.27 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.4 \
> + # pegdown-formatter:1.3 \
> + # structs:1.17 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # make the mod.js symlink well after flot is installed
> + service jenkins start && sleep 30 && \
> + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> +
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
The above and following are definitely not unique to Debian, or a specific version of Debian.
They should be put into an "install-common.sh"
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> +else
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> +fi
> +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> +
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +apt-get -q=2 -V --no-install-recommends install jq
> +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> +chmod a+x /usr/local/bin/ebf
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# ftc post installation
> +# ==============================================================================
> +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +echo ". /etc/bash_completion" >> /root/.bashrc
> +
> +# ==============================================================================
> +# Lava
> +# ==============================================================================
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/install-debian.sh b/install-debian.sh
> index ccca032..2d3ffd3 100755
> --- a/install-debian.sh
> +++ b/install-debian.sh
> @@ -2,102 +2,163 @@
> # 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> #
> # Usage:
> -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> #
> -if [ -n "$1" ]; then
> - if [ "$1" = "--help" -o "$1" = "-h" ]; then
> - cat <<HERE
> -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> -
> -Installs fuego on the host Debian filesystem.
> -
> -options:
> - --help Show usage help
> - --nojenkins Install Fuego without Jenkins
> -HERE
> - exit 0
> - fi
> -fi
> +
> +usage(){
> + echo ""
> + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> + echo "Install fuego on Debian filesystem."
> + echo ""
> + echo "options:"
> + echo " --help | -h Show usage help"
> + echo " --suite | -s <suite> Specify Debian suite"
> + echo " --port | -p <port> Jenkins port"
> + echo " --nojenkins Install Fuego without Jenkins"
> + echo ""
> +}
>
> if [[ $EUID -ne 0 ]]; then
> echo "Sorry, you need root permissions"
> exit 1
> fi
>
> -if [ "$1" = "--nojenkins" ]; then
> - nojenkins=1
> +nojenkins=0
> +port=8090
> +suite=stretch
> +user=jenkins
> +group=jenkins
> +uid=1000
> +gid=1000
> +
> +while [ "$1" != "" ]; do
> + case $1 in
> + -s | --suite )
> + shift
> + suite=$1
> + ;;
> + -p | --port )
> + shift
> + port=$1
> + ;;
> + -u | --uid )
> + shift
> + uid=$1
> + ;;
> + -g | --gid )
> + shift
> + gid=$1
> + ;;
> + --nojenkins )
> + nojenkins=1
> + ;;
> + -h | --help )
> + usage
> + exit 0
> + ;;
> + * )
> + break
> + esac
> shift
> -else
> - nojenkins=0
> - port=${1:-8090}
> -fi
> +done
>
> # ==============================================================================
> # Install Fuego dependencies
> # ==============================================================================
>
> # netperf is in non-free
> -echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> -echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> -
> -apt-get update -q=2
> +echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list
>
> # Fuego python dependencies
> -apt-get -q=2 -V --no-install-recommends install \
> +# - python-lxml: ftc, loggen
> +# - python-simplejson: ftc
> +# - python-yaml: ftc
> +# - python-openpyxl: ftc (also LTP)
> +# - python-requests: ftc (also fuego_release_test)
> +# - python-reportlab: ftc
> +# - python-parsedatetime: ftc
> +# - python-pexpect: ssh_exec (part of ttc)
> +# - python-pip: to install filelock, flake8
> +# - filelock: parser
> +mkdir -p /usr/share/man/man1
> +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> python-lxml python-simplejson python-yaml python-openpyxl \
> python-requests python-reportlab python-parsedatetime \
> - python-pip
> + python-pexpect python-pip python-setuptools python-wheel
> pip install filelock
>
> # Fuego command dependencies
> apt-get -q=2 -V --no-install-recommends install \
> git sshpass openssh-client sudo net-tools wget curl lava-tool \
> - bash-completion
> + bash-completion iproute2
>
> # Default SDK for testing locally or on an x86 board
> apt-get -q=2 -V --no-install-recommends install \
> build-essential cmake bison flex automake kmod libtool \
> libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
>
> # Default test host dependencies
> +# - iperf iperf3 netperf: used as servers on the host
> +# - bzip2 bc: used for local tests by Functional.bzip2/bc
> +# - python-matplotlib: Benchmark.iperf3 parser
> +# - python-xmltodict: AGL tests
> +# - flake8: Functional.fuego_lint
> +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> +# - iputils-ping - for /bin/ping command, used by some tests
> +# FIXTHIS: install dependencies dynamically on the tests that need them
> apt-get -q=2 -V --no-install-recommends install \
> - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
> -pip install flake8 bsdmainutils
> + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
>
> -echo "dash dash/sh boolean false" | debconf-set-selections
> -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> -if [ -n "$http_proxy" ]; then
> - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +#TODO: find alternate package for netperf in buster version
> +if [ $suite != "buster" ];then
> + apt-get -q=2 -V --no-install-recommends install netperf
> fi
> +pip install flake8
>
> -# ==============================================================================
> -# Clone fuego and fuego-core
> -# ==============================================================================
> -if [ ! -d "/fuego" ]; then
> - cd /
> - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
> - ln -s /fuego/fuego-ro /fuego-ro
> - ln -s /fuego/fuego-rw /fuego-rw
> - cd fuego
> - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
> - ln -s /fuego/fuego-core /fuego-core
> -fi
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# FIXTHIS: determine if these tools are really necessary
> +#RUN apt-get -q=2 -V --no-install-recommends install \
> +# apt-utils python-paramiko \
> +# xmlstarlet rsync \
> +# inotify-tools gettext netpipe-tcp \
> +# at minicom lzop bsdmainutils \
> +# mc netcat openssh-server
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +apt-get -q=2 -V --no-install-recommends install \
> + bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
>
> # ==============================================================================
> # Install Jenkins
> # ==============================================================================
> -
> if [ $nojenkins -eq 0 ]; then
> #JENKINS_VERSION=2.164.2
> #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> JENKINS_VERSION=2.249.3
> JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> - JENKINS_UC=https://updates.jenkins.io
> - REF=/var/lib/jenkins/plugins
> - JENKINS_HOME=/var/lib/jenkins
> + export JENKINS_UC=https://updates.jenkins.io
> + export REF=/var/lib/jenkins/plugins
> + export JENKINS_HOME=/var/lib/jenkins
> JENKINS_PORT=$port
>
> # Jenkins dependencies
> @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
> pip install python-jenkins==1.4.0
>
> echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> - groupadd jenkins
> - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> - cd
> - wget ${JENKINS_URL}
> + export JENKINS_PORT=$port
> +
> + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> +
> + wget -nv ${JENKINS_URL}
> echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
>
> # allow Jenkins to start and install plugins, as part of dpkg installation
> @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
> dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> rm jenkins_${JENKINS_VERSION}_all.deb
>
> - # update ownership
> - chown -R jenkins:jenkins /fuego
> - chown -R jenkins:jenkins /fuego-ro
> - chown -R jenkins:jenkins /fuego-rw
> - chown -R jenkins:jenkins /fuego-core
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +# as of ttc version 2.0.2, this should not be needed
> +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + apt-get -q=2 -V --no-install-recommends install jq
> + git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> + chmod a+x /usr/local/bin/ebf
> +fi
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +git clone https://github.com/frowand/serio.git /usr/local/src/serio
> +if [ $nojenkins -eq 0 ]; then
> + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
> + chown -R jenkins /usr/local/src/serio
> +fi
> +cp /usr/local/src/serio/serio /usr/local/bin/
> +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> +
> +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
> +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
> +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# Jenkins post installation
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> +
> + # Fix jenkins issue, Java version check fails when java version is 11
> + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-
> \]*\\\).*\\\".*/\\\1\\\2\/p;')"
> + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
>
> source /etc/default/jenkins && \
> JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
>
> sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
>
> - cp /fuego/frontend-install/install-plugins.sh \
> - /fuego/frontend-install/jenkins-support \
> - /fuego/frontend-install/clitest \
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> /usr/local/bin/
>
> - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
> - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> - mkdir $JENKINS_HOME/userContent/docs
> - cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> - jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> - sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> - chown -R jenkins:jenkins $JENKINS_HOME/
> -
> # start and stop jenkins to pre-populate some settings
> service jenkins start && \
> sleep 30 && \
> service jenkins stop
>
> + # install plugins - these versions are for Jenkins version 2.164.2
> + # Explicitly install script-security v1.68, otherwise
> + # v1.74 will automatically be installed as a dependency of
> + # the junit plugin. Make sure to install before junit plugin.
> + # Explicitly install junit:1.27
> +
> # install other plugins from Jenkins update center
> # NOTE: not sure all of these are needed, but keep list
> - # compatible with 1.2.1 release for now
> - #/usr/local/bin/install-plugins.sh \
> - # ant:1.9 \
> - # antisamy-markup-formatter:1.5 \
> - # bouncycastle-api:2.17 \
> - # command-launcher:1.3 \
> - # description-setter:1.10 \
> - # display-url-api:2.3.1 \
> - # external-monitor-job:1.7 \
> - # greenballs:1.15 \
> - # icon-shim:2.0.3 \
> - # javadoc:1.5 \
> - # jdk-tool:1.2 \
> - # junit:1.27 \
> - # ldap:1.20 \
> - # mailer:1.23 \
> - # matrix-auth:2.3 \
> - # matrix-project:1.14 \
> - # pam-auth:1.4 \
> - # pegdown-formatter:1.3 \
> - # structs:1.17 \
> - # windows-slaves:1.4
> -
> - # install plugins for latest LTS Jenkins (currently 2.249.3)
> - /usr/local/bin/install-plugins.sh \
> - script-security \
> - structs \
> - workflow-step-api \
> - workflow-api \
> - junit \
> - scm-api \
> - ant \
> - antisamy-markup-formatter \
> - bouncycastle-api \
> - command-launcher \
> - description-setter \
> - display-url-api \
> - external-monitor-job \
> - greenballs \
> - icon-shim \
> - javadoc \
> - jdk-tool \
> - mailer \
> - matrix-auth \
> - matrix-project \
> - pam-auth \
> - pegdown-formatter \
> - windows-slaves
> -
> - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> + # as compatible as possible with 1.2.1 release for now
> + # Do NOT change the order of the plugins, unless you know what
> + # you are doing. Otherwise, the install-plugins.sh script will
> + # install the wrong versions of other plugins based on dependency
> + # information in the plugin files.
>
> + #/usr/local/bin/install-plugins.sh \
> + # script-security:1.68 \
> + # structs:1.20 \
> + # workflow-step-api:2.22 \
> + # workflow-api:2.40 \
> + # junit:1.27 \
> + # scm-api:2.6.3 \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.5 \
> + # pegdown-formatter:1.3 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # jenkins should automatically unzip any plugins in the plugin dir
> # make the mod.js symlink well after flot is installed
> service jenkins start && sleep 30 && \
> rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
>
> - chown -R jenkins:jenkins $JENKINS_HOME/
> -else
> - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
> -fi
> + mkdir -p $JENKINS_HOME/userContent/docs
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
>
> -# ==============================================================================
> -# get ttc script and helpers
> -# ==============================================================================
> -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -/usr/local/src/ttc/install.sh /usr/local/bin
> -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
>
> -# ==============================================================================
> -# Serial Config
> -# ==============================================================================
> -if [ $nojenkins -eq 0 ]; then
> - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> -else
> - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> + chown -R jenkins:jenkins $JENKINS_HOME/
> fi
>
> -# ==============================================================================
> -# fserver
> -# ==============================================================================
> -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> -ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> -
> # ==============================================================================
> # ftc post installation
> # ==============================================================================
> ln -s /fuego-core/scripts/ftc /usr/local/bin/
> -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> echo ". /etc/bash_completion" >> /root/.bashrc
>
> # ==============================================================================
> @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
> # ==============================================================================
> ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -
> -# ==============================================================================
> -# Small guide
> -# ==============================================================================
> -echo "Run 'service netperf start' to start a netperf server"
> -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> -echo "Run 'ftc list-boards' to see the available boards"
> -echo "Run 'ftc list-tests' to see the available tests"
> -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> -echo "Run 'ftc gen-report' to get results"
> diff --git a/install-fuego.sh b/install-fuego.sh
> new file mode 100755
> index 0000000..7b3e010
> --- /dev/null
> +++ b/install-fuego.sh
> @@ -0,0 +1,109 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> +#
> +
> +usage(){
> + echo ""
> + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> + echo "Install fuego on Debian filesystem."
> + echo ""
> + echo "options:"
> + echo " --help | -h Show usage help"
> + echo " --dist | -d <distribution> Specify distribution"
> + echo " --suite | -s <suite> Specify suite"
> + echo " --port | -p <port> Jenkins port"
> + echo " --nojenkins Install Fuego without Jenkins"
> + echo ""
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=
> +port=8090
> +dist=debian
> +suite=stretch
> +fuego_dir=$(pwd)
> +
> +while [ "$1" != "" ]; do
> + case $1 in
> + -d | --dist )
> + shift
> + dist=$1
> + ;;
> + -s | --suite )
> + shift
> + suite=$1
> + ;;
> + -p | --port )
> + shift
> + port=$1
> + ;;
> + --nojenkins )
> + nojenkins="--nojenkins"
> + ;;
> + -h | --help )
> + usage
> + exit 0
> + ;;
> + * )
> + break
> + esac
> + shift
> +done
> +
> +# ==============================================================================
> +# Clone fuego-core
> +# ==============================================================================
> +if [ ! -f "fuego-core/scripts/ftc" ]; then
> + # set fuego-core branch to same as current fuego branch
> + # get current git branch
> + set -o noglob
> + while IFS=" " read -r part1 part2 ; do
> + if [ $part1 = "*" ] ; then
> + branch=$part2
> + fi
> + done < <(git branch)
> + set +o noglob
> + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
> +fi
> +
> +# ==============================================================================
> +# Install fuego on host system
> +# ==============================================================================
> +./install-$dist.sh -s $suite -p $port $nojenkins
> +
> +systemctl daemon-reload
> +service jenkins restart
> +
> +# provide permission to current user
> +usermod -aG jenkins $USER
> +chmod -R 775 $fuego_dir/fuego-rw
> +
> +chown -R jenkins:jenkins $fuego_dir/fuego-ro
> +chown -R jenkins:jenkins $fuego_dir/fuego-rw
> +chown -R jenkins:jenkins $fuego_dir/fuego-core
> +
> +# ==============================================================================
> +# Create Links to fuego folders
> +# ==============================================================================
> +ln -s $fuego_dir/fuego-ro /
> +ln -s $fuego_dir/fuego-rw /
> +ln -s $fuego_dir/fuego-core /
> +
> +# ==============================================================================
> +# Small guide
> +# ==============================================================================
> +echo "Logout and Login current user session to work properly"
> +echo "Run 'service netperf start' to start a netperf server"
> +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> +echo "Run 'ftc list-boards' to see the available boards"
> +echo "Run 'ftc list-tests' to see the available tests"
> +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> +echo "Run 'ftc gen-report' to get results"
> diff --git a/install-stretch.sh b/install-stretch.sh
> new file mode 100755
> index 0000000..4f6f724
> --- /dev/null
> +++ b/install-stretch.sh
> @@ -0,0 +1,288 @@
> +#!/bin/bash
> +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> +#
> +# Usage:
> +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> +#
> +usage(){
> +
> +cat <<HERE
> +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> +
> +Install fuego on the host Debian filesystem.
> +
> +options:
> + --help Show usage help
> + --nojenkins Install Fuego without Jenkins
> +HERE
> +}
> +
> +if [[ $EUID -ne 0 ]]; then
> + echo "Sorry, you need root permissions"
> + exit 1
> +fi
> +
> +nojenkins=0
> +
> +POSITIONAL=()
> +while [[ $# -gt 0 ]] ; do
> + case $1 in
> + --nojenkins)
> + nojenkins=1
> + shift
> + ;;
> + --help | -h)
> + usage
> + exit 0
> + ;;
> + *)
> + POSITIONAL+=("$1") # save argument for later
> + shift
> + esac
> +done
> +set -- "${POSITIONAL[@]}" # restore positional arguments
> +
> +port=${2:-8090}
> +
> +# ==============================================================================
> +# Install Fuego dependencies
> +# ==============================================================================
> +
> +mkdir -p /usr/share/man/man1
> +
> +# netperf is in non-free
> +echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> +echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> +
> +apt-get update -q=2
> +
> +# Fuego python dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + python-lxml python-simplejson python-yaml python-openpyxl \
> + python-requests python-reportlab python-parsedatetime \
> + python-pexpect python-pip python-setuptools python-wheel
> +pip install filelock
> +
> +# Fuego command dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> + bash-completion iproute2
> +
> +# Default SDK for testing locally or on an x86 board
> +apt-get -q=2 -V --no-install-recommends install \
> + build-essential cmake bison flex kmod automake libtool \
> + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> +
> +# Default test host dependencies
> +apt-get -q=2 -V --no-install-recommends install \
> + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> + netpipe-tcp iputils-ping
> +pip install flake8
> +
> +# miscelaneous packages:
> +# python-serial - used by serio
> +# diffstat and vim - used by Tim
> +# time - useful for timing command duration
> +apt-get -q=2 -V --no-install-recommends install \
> + python-serial \
> + diffstat \
> + vim \
> + time
> +
> +# install packages used by NuttX SDK
> +apt-get -q=2 -V --no-install-recommends install \
> + genromfs
> +
> +# bsdmainutils provides hexdump, which is needed for netperf test
> +apt-get -q=2 -V --no-install-recommends install \
> + bsdmainutils
> +
> +echo "dash dash/sh boolean false" | debconf-set-selections
> +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +
> +# TODO: check if proxy settings can set outside docker container
> +#if [ -n "$http_proxy" ]; then
> +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> +#fi
> +
> +# ==============================================================================
> +# Install Jenkins
> +# ==============================================================================
> +
> +if [ $nojenkins -eq 0 ]; then
> + #JENKINS_VERSION=2.164.2
> + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> + JENKINS_VERSION=2.249.3
> + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> + export JENKINS_UC=https://updates.jenkins.io
> + REF=/var/lib/jenkins/plugins
> + JENKINS_HOME=/var/lib/jenkins
> + JENKINS_PORT=$port
> +
> + # Jenkins dependencies
> + apt-get -q=2 -V --no-install-recommends install \
> + default-jdk daemon psmisc adduser procps unzip
> + pip install python-jenkins==1.4.0
> +
> + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> + export JENKINS_PORT=$port
> +
> + groupadd jenkins
> + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> +
> + wget -nv ${JENKINS_URL}
> + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> +
> + # allow Jenkins to start and install plugins, as part of dpkg installation
> + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> + rm jenkins_${JENKINS_VERSION}_all.deb
> +
> + source /etc/default/jenkins && \
> + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> +
> + source /etc/default/jenkins && \
> + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> + if [ -n "$http_proxy" ]; then \
> + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> + fi && \
> + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> +
> + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> +
> + # set up Jenkins plugins
> + cp frontend-install/install-plugins.sh \
> + frontend-install/jenkins-support \
> + frontend-install/clitest \
> + /usr/local/bin/
> +
> + mkdir -p $JENKINS_HOME/userContent/docs
> + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +
> + # start and stop jenkins to pre-populate some settings
> + service jenkins start && \
> + sleep 30 && \
> + service jenkins stop
> +
> + # install other plugins from Jenkins update center
> + # NOTE: not sure all of these are needed, but keep list
> + # compatible with 1.2.1 release for now
> + #/usr/local/bin/install-plugins.sh \
> + # ant:1.9 \
> + # antisamy-markup-formatter:1.5 \
> + # bouncycastle-api:2.17 \
> + # command-launcher:1.3 \
> + # description-setter:1.10 \
> + # display-url-api:2.3.1 \
> + # external-monitor-job:1.7 \
> + # greenballs:1.15 \
> + # icon-shim:2.0.3 \
> + # javadoc:1.5 \
> + # jdk-tool:1.2 \
> + # junit:1.27 \
> + # ldap:1.20 \
> + # mailer:1.23 \
> + # matrix-auth:2.3 \
> + # matrix-project:1.14 \
> + # pam-auth:1.4 \
> + # pegdown-formatter:1.3 \
> + # structs:1.17 \
> + # windows-slaves:1.4
> +
> + # install plugins for latest LTS Jenkins (currently 2.249.3)
> + /usr/local/bin/install-plugins.sh \
> + script-security \
> + structs \
> + workflow-step-api \
> + workflow-api \
> + junit \
> + scm-api \
> + ant \
> + antisamy-markup-formatter \
> + bouncycastle-api \
> + command-launcher \
> + description-setter \
> + display-url-api \
> + external-monitor-job \
> + greenballs \
> + icon-shim \
> + javadoc \
> + jdk-tool \
> + mailer \
> + matrix-auth \
> + matrix-project \
> + pam-auth \
> + pegdown-formatter \
> + windows-slaves
> +
> + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> +
> + # make the mod.js symlink well after flot is installed
> + service jenkins start && sleep 30 && \
> + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> +
> + chown -R jenkins:jenkins $JENKINS_HOME/
> +fi
> +
> +# ==============================================================================
> +# get ttc script and helpers
> +# ==============================================================================
> +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> +
> +# ==============================================================================
> +# Serial Config
> +# ==============================================================================
> +if [ $nojenkins -eq 0 ]; then
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> +else
> + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> /usr/local/bin'
> +fi
> +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> + /tmp/
> +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> +
> +
> +# ==============================================================================
> +# get ebf script and helpers
> +# ==============================================================================
> +apt-get -q=2 -V --no-install-recommends install jq
> +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> +chmod a+x /usr/local/bin/ebf
> +
> +# ==============================================================================
> +# fserver
> +# ==============================================================================
> +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> +
> +# ==============================================================================
> +# ftc post installation
> +# ==============================================================================
> +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> +echo ". /etc/bash_completion" >> /root/.bashrc
> +
> +# ==============================================================================
> +# Lava
> +# ==============================================================================
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/install.sh b/install.sh
> index a5a90b0..259c887 100755
> --- a/install.sh
> +++ b/install.sh
> @@ -1,13 +1,13 @@
> #!/bin/bash
> #
> -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
> #
>
> # $1 is the exit code after usage is shown
> usage() {
> exit_code="$1"
> cat <<HERE
> -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
>
> Create the docker image and container with the Fuego test distribution.
> If no <image_name> is provided, the image will be named 'fuego'.
> @@ -34,7 +34,7 @@ fi
>
> priv=0
> NOCACHE=""
> -dockerfile="Dockerfile"
> +NOJENKINS=""
>
> POSITIONAL=()
> while [[ $# -gt 0 ]] ; do
> @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
> shift
> ;;
> --nojenkins)
> - dockerfile="Dockerfile.nojenkins"
> + NOJENKINS="--nojenkins"
> shift
> ;;
> *)
> @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
>
> image_name=${1:-fuego}
> jenkins_port=${2:-8090}
> +distribution=${3:-debian}
> +suite=${4:-stretch}
>
> container_name="${image_name}-container"
>
> @@ -79,7 +81,7 @@ fi
>
> set -e
>
> -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
> +source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
> if [ "$priv" == "0" ]; then
> fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
> else
> diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
> new file mode 100644
> index 0000000..b899e7c
> --- /dev/null
> +++ b/uninstall-fuego.sh
> @@ -0,0 +1,47 @@
> +#!/bin/bash
> +
> +# Remove Jenkins plugins
> +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
> +
> +# uninstall Jenkins
> +service jenkins stop
> +dpkg -P jenkins
> +
> +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
> +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
> +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
> +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
> +sed -i '/^JENKINS_PORT=/d' /etc/environment
> +
> +# uninstall ttc script
> +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
> +
> +# uninstall ebf script
> +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
> +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
> +
> +# uninstall serial config
> +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
> +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
> +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
> +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
> +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
> +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
> +
> +# uninstall fserver
> +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
> +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
> +
> +# uninstall ftc
> +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
> +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
> +
> +# remove lava target files
> +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
> +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
> +
> +# remove links
> +[ -d /fuego-ro ] && rm /fuego-ro
> +[ -d /fuego-rw ] && rm /fuego-rw
> +[ -d /fuego-core ] && rm /fuego-core
> +
> --
> 2.20.1
>
Overall - this looks like a good start. But I'd like to see the duplicate sections of install-buster.sh
and install-stretch.sh put into a single location "install-debian-common.sh". Also, I'd like to see
the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
Can you please re-do this with that refactoring in mind?
Thanks,
-- Tim
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-15 3:46 ` Venkata.Pyla
@ 2021-06-16 17:47 ` Tim.Bird
0 siblings, 0 replies; 8+ messages in thread
From: Tim.Bird @ 2021-06-16 17:47 UTC (permalink / raw)
To: Venkata.Pyla, tbird20d; +Cc: fuego
Venkata,
Did you have any responses or comments on my other feedback for the patch?
-- Tim
> -----Original Message-----
> From: Venkata.Pyla@toshiba-tsip.com <Venkata.Pyla@toshiba-tsip.com>
>
> Hello Tim,
>
> I sent patch based on the master branch of this repository https://bitbucket.org/fuegotest/fuego/src/master/
> And the previous commit id was "3c813c717f1ebd7dc279ecdb0f288e4f9d30d6d7".
>
> Also created a PR (https://bitbucket.org/fuegotest/fuego/pull-requests/4) to visualize the changes.
>
> Please let me know if I need to send another patch with any specific branch or version?
>
>
> Thanks,
> Venkata.
> -----Original Message-----
> From: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Sent: 15 June 2021 05:39
> To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com>; tbird20d@yahoo.com
> Cc: fuego@lists.linuxfoundation.org; sangorrin daniel(サンゴリン ダニエル □ＳＷＣ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>
> Subject: RE: [Fuego] Use common script to install fuego on host or docker system
>
> Please see comments inline below.
>
> I tried to apply this patch from the e-mail list using 'git am', but got a failure.
> That made it a hard to experiment with and view the changes in
> context.
>
> What version of the code is this patch based on?
>
> > -----Original Message-----
> > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > Subject: [Fuego] Use common script to install fuego on host or docker system
> >
> > From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >
> > Modified the install scripts to use common scripts for both types of
> > installation, this way it will help to make sure both the environments
> > are same and if any future modification will have single place of change.
> >
> > Now, for installing fuego on docker system the 'install.sh' script will
> > use common script in Dockefile to install fuego related stuff.
> >
> > sameway for installing fuego on host system it can use the same common
> > script to install fuego related stuff.
> >
> > Below are the common scripts that are used in both the installation types
> > for each debian distribution
> > * install-stretch.sh
> > * install-buster.sh
> >
> > so finally,
> > To install fuego on docker system use the below script with options
> > $ ./install.sh <options> image_name <port> <distribution> <suite>
> > e.g: ./install.sh debian_stretch 8090 debian stretch
> > or
> > ./install.sh debian_buster 8090 debian buster
> > or
> > ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
> This is a bit wordy, but OK.
>
> If we don't support any other distributions besides Debian at the moment,
> then I'm not sure we need the distribution argument. Do you have any
> non-debian distribution install scripts ready to submit soon?
>
> I'm not sure I follow calling the Debian release name the "<suite>".
>
> >
> > To install fuego on host system use the below script with options
> > $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
> "suite" seems like an odd name. Should this be distro "version" or "release"?
>
> > e.g: ./install-fuego.sh -d debian -s stretch -p 8090
> > or
> > ./install-fuego.sh -d debian -s buster -p 8090
> > or
> > ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
> >
> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > ---
> > Dockerfile | 303 ++---------------
> > Dockerfile.nojenkins | 151 ---------
> > README | 29 +-
> > entrypoint.sh | 7 +
> > fuego-host-scripts/docker-build-image.sh | 11 +-
> > install-buster.sh | 294 ++++++++++++++++
> > install-debian.sh | 410 ++++++++++++++---------
> > install-fuego.sh | 109 ++++++
> > install-stretch.sh | 288 ++++++++++++++++
>
> Is install-debian.sh the common parts for a Debian installation, with
> buster and stretch having the parts that are unique to each of
> those distros respectively? If we're going to refactor this, I
> want to eliminate any duplication, so that package names are
> not used in multiple places. This has been a headache to
> maintain.
>
> It appears there is lots of duplication between install-buster.sh and
> install-stretch.sh. Can we possible put this duplication into an
> "install-debian-common.sh", and reserve install-buster.sh and
> install-stretch.sh for only the parts unique to those distros?
>
> > install.sh | 12 +-
> > uninstall-fuego.sh | 47 +++
>
> Ooo - I like the sound of this uninstall script.
>
> > 11 files changed, 1057 insertions(+), 604 deletions(-)
> > delete mode 100644 Dockerfile.nojenkins
> > create mode 100755 entrypoint.sh
> > create mode 100755 install-buster.sh
> > create mode 100755 install-fuego.sh
> > create mode 100755 install-stretch.sh
> > create mode 100644 uninstall-fuego.sh
> >
>
>
> > diff --git a/Dockerfile b/Dockerfile
> > index 7f63798..ac57cb2 100644
> > --- a/Dockerfile
> > +++ b/Dockerfile
> > @@ -2,9 +2,11 @@
> > # WARNING: this Dockerfile assumes that the container will be created with
> > # several bind mounts (see docker-create-container.sh)
> > # ==============================================================================
> > -# FIXTHIS: build this as an extension of the nonjenkins image
> >
> > -FROM debian:stretch-slim
> > +ARG SUITE=stretch
> > +ARG DIST=debian
> > +
> > +FROM ${DIST}:${SUITE}-slim
> > MAINTAINER tim.bird@sony.com
> >
> > # ==============================================================================
> > @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
> > ENV https_proxy ${HTTP_PROXY}
> >
> > # ==============================================================================
> > -# Prepare basic image
> > +# Install fuego with the script
> > # ==============================================================================
> >
> > -ARG DEBIAN_FRONTEND=noninteractive
> > -
> > -WORKDIR /
> > -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> > -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> > -
> > -# Fuego python dependencies
> > -# - python-lxml: ftc, loggen
> > -# - python-simplejson: ftc
> > -# - python-yaml: ftc
> > -# - python-openpyxl: ftc (also LTP)
> > -# - python-requests: ftc (also fuego_release_test)
> > -# - python-reportlab: ftc
> > -# - python-parsedatetime: ftc
> > -# - python-pexpect: ssh_exec (part of ttc)
> > -# - python-pip: to install filelock, flake8
> > -# - filelock: parser
> > -RUN mkdir -p /usr/share/man/man1
> > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> > - python-lxml python-simplejson python-yaml python-openpyxl \
> > - python-requests python-reportlab python-parsedatetime \
> > - python-pexpect python-pip python-setuptools python-wheel
> > -RUN pip install filelock
> > -
> > -# Fuego command dependencies
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > - bash-completion iproute2
> > -
> > -# Default SDK for testing locally or on an x86 board
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - build-essential cmake bison flex automake kmod libtool \
> > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > -
> > -# Default test host dependencies
> > -# - iperf iperf3 netperf: used as servers on the host
> > -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > -# - python-matplotlib: Benchmark.iperf3 parser
> > -# - python-xmltodict: AGL tests
> > -# - flake8: Functional.fuego_lint
> > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > -# - iputils-ping - for /bin/ping command, used by some tests
> > -# FIXTHIS: install dependencies dynamically on the tests that need them
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > - netpipe-tcp iputils-ping
> > -RUN pip install flake8
> > -
> > -# miscelaneous packages:
> > -# python-serial - used by serio
> > -# diffstat and vim - used by Tim
> > -# time - useful for timing command duration
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - python-serial \
> > - diffstat \
> > - vim \
> > - time
> > -
> > -# install packages used by NuttX SDK
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - genromfs
> > -
> > -# FIXTHIS: determine if these tools are really necessary
> > -#RUN apt-get -q=2 -V --no-install-recommends install \
> > -# apt-utils python-paramiko \
> > -# xmlstarlet rsync \
> > -# inotify-tools gettext netpipe-tcp \
> > -# at minicom lzop bsdmainutils \
> > -# mc netcat openssh-server
> > -
> > -# bsdmainutils provides hexdump, which is needed for netperf test
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - bsdmainutils
> > -
> > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> > -
> > -# ==============================================================================
> > -# Install Jenkins with the same UID/GID as the host user
> > -# ==============================================================================
> > +RUN mkdir /fuego
> > +COPY . /fuego/
> > +WORKDIR /fuego
> >
> > -ARG user=jenkins
> > -ARG group=jenkins
> > -ARG uid=1000
> > -ARG gid=${uid}
> > ARG JENKINS_PORT=8090
> > -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
> > -#ARG JENKINS_VERSION=2.164.2
> > -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > -#ARG JENKINS_VERSION=2.235.3
> > -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
> > -ARG JENKINS_VERSION=2.249.3
> > -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > -
> > -ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > -ARG JENKINS_UC=https://updates.jenkins.io
> > -ARG REF=/var/lib/jenkins/plugins
> > +ARG NO_JENKINS=
> > +ARG SUITE=stretch
> > +ARG DIST=debian
> > +ARG uid=
> > +ARG gid=
> > ENV JENKINS_HOME=/var/lib/jenkins
> > ENV JENKINS_PORT=$JENKINS_PORT
> > +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid $NO_JENKINS
>
> One question about this overall approach. Does this retain the ability to cache
> operations, so that if something goes wrong with an install, you don't have to
> start over from scratch. When building with docker, with multiple steps, it
> caches the image at each step. When rebuilding, or when an error occurs,
> this can save a lot of time. It also can introduce subtle bugs, but that's
> a separate issue.
>
> This setup seems like it performs the bulk of the install as a single docker
> step, is that right?
>
> > -# Jenkins dependencies
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - default-jdk daemon psmisc adduser procps unzip
> > -RUN pip install python-jenkins==1.4.0
> > -
> > -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> > -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> > -RUN wget -nv ${JENKINS_URL}
> > -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> > -# allow Jenkins to start and install plugins, as part of dpkg installation
> > -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> > -
> > -# ==============================================================================
> > -# get ttc script and helpers
> > -# ==============================================================================
> > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> > -# as of ttc version 2.0.2, this should not be needed
> > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> > -
> > -# ==============================================================================
> > -# get ebf script and helpers
> > -# ==============================================================================
> > -RUN apt-get -q=2 -V --no-install-recommends install jq
> > -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api ; \
> > - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
> > - chmod a+x /usr/local/bin/ebf
> > -
> > -# ==============================================================================
> > -# Serial Config
> > -# ==============================================================================
> > -
> > -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
> > -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> > - frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> > - /tmp/
> > -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> > - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ; \
> > - chown -R jenkins /usr/local/src/serio ; \
> > - cp /usr/local/src/serio/serio /usr/local/bin/ ; \
> > - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
> > - ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> cp
> > /usr/local/src/serlogin/serlogin /usr/local/bin'
> > -
> > -# ==============================================================================
> > -# fserver
> > -# ==============================================================================
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; \
> > - ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server'
> > -
> > -# ==============================================================================
> > -# Jenkins post installation
> > -# ==============================================================================
> > -
> > -RUN source /etc/default/jenkins && \
> > - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> > -
> > -RUN source /etc/default/jenkins && \
> > - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > - if [ -n "$HTTP_PROXY" ]; then \
> > - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > - JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > - fi && \
> > - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> > -
> > -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > -
> > -# set up Jenkins plugins
> > -COPY frontend-install/install-plugins.sh \
> > - frontend-install/jenkins-support \
> > - frontend-install/clitest \
> > - /usr/local/bin/
> > -
> > -# start Jenkins onces to initialize directories?
> > -RUN service jenkins start && \
> > - sleep 30 && \
> > - service jenkins stop
> > -
> > -# install plugins - these versions are for Jenkins version 2.164.2
> > -# Explicitly install script-security v1.68, otherwise
> > -# v1.74 will automatically be installed as a dependency of
> > -# the junit plugin. Make sure to install before junit plugin.
> > -# Explicitly install junit:1.27
> > -
> > -# install other plugins from Jenkins update center
> > -# NOTE: not sure all of these are needed, but keep list
> > -# as compatible as possible with 1.2.1 release for now
> > -# Do NOT change the order of the plugins, unless you know what
> > -# you are doing. Otherwise, the install-plugins.sh script will
> > -# install the wrong versions of other plugins based on dependency
> > -# information in the plugin files.
> > -
> > -#RUN /usr/local/bin/install-plugins.sh \
> > -# script-security:1.68 \
> > -# structs:1.20 \
> > -# workflow-step-api:2.22 \
> > -# workflow-api:2.40 \
> > -# junit:1.27 \
> > -# scm-api:2.6.3 \
> > -# ant:1.9 \
> > -# antisamy-markup-formatter:1.5 \
> > -# bouncycastle-api:2.17 \
> > -# command-launcher:1.3 \
> > -# description-setter:1.10 \
> > -# display-url-api:2.3.1 \
> > -# external-monitor-job:1.7 \
> > -# greenballs:1.15 \
> > -# icon-shim:2.0.3 \
> > -# javadoc:1.5 \
> > -# jdk-tool:1.2 \
> > -# ldap:1.20 \
> > -# mailer:1.23 \
> > -# matrix-auth:2.3 \
> > -# matrix-project:1.14 \
> > -# pam-auth:1.5 \
> > -# pegdown-formatter:1.3 \
> > -# windows-slaves:1.4
> > -
> > -# install plugins for latest LTS Jenkins (currently 2.249.3)
> > -RUN /usr/local/bin/install-plugins.sh \
> > - script-security \
> > - structs \
> > - workflow-step-api \
> > - workflow-api \
> > - junit \
> > - scm-api \
> > - ant \
> > - antisamy-markup-formatter \
> > - bouncycastle-api \
> > - command-launcher \
> > - description-setter \
> > - display-url-api \
> > - external-monitor-job \
> > - greenballs \
> > - icon-shim \
> > - javadoc \
> > - jdk-tool \
> > - mailer \
> > - matrix-auth \
> > - matrix-project \
> > - pam-auth \
> > - pegdown-formatter \
> > - windows-slaves
> > +COPY entrypoint.sh /usr/bin/
> >
> > -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> > -
> > -# jenkins should automatically unzip any plugins in the plugin dir
> > -# make a symlink for mod.js well after flot is installed
> > -RUN service jenkins start && sleep 30 && \
> > - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> > -
> > -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> > -
> > -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> > -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > -RUN sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > -
> > -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> > -
> > -# ==============================================================================
> > -# ftc post installation
> > -# ==============================================================================
> > -
> > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> > -
> > -# ==============================================================================
> > -# Lava
> > -# ==============================================================================
> > -
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > +# Delete the files that are copied for installation
> > +RUN rm -rf /fuego
>
> I'm not sure what this is doing. It looks scary. Please explain.
>
> > +WORKDIR /
> >
> > # ==============================================================================
> > # Setup startup command
> > # ==============================================================================
> > -
> > # FIXTHIS: when running multiple Fuego containers, or if the host is already
> > # running the netperf server, netperf will complain because the port is taken
> > -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D -f M && /bin/bash
> > +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
>
> Isolating this to a separate script is nice!
>
> > diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
> > deleted file mode 100644
> > index 56de98a..0000000
> > --- a/Dockerfile.nojenkins
> > +++ /dev/null
>
> Getting rid of this separate Dockerfile is nice.
>
> > @@ -1,151 +0,0 @@
> > -# ==============================================================================
> > -# WARNING: this Dockerfile assumes that the container will be created with
> > -# several bind mounts (see docker-create-container.sh)
> > -# ==============================================================================
> > -# Usage:
> > -# host$ vi fuego-ro/conf/fuego.conf
> > -# jenkins_enabled=0
> > -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090 Dockerfile.nojenkins
> > -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins fuego-nojenkins-container
> > -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-container
> > -# docker# ftc list-boards
> > -# docker, bbb, ...
> > -# docker# ftc list-tests
> > -# Functional.hello_world, ...
> > -# docker# ftc run-test -b docker -t Functional.hello_world
> > -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or equal than 1)
> > -
> > -FROM debian:stretch-slim
> > -MAINTAINER tim.bird@sony.com
> > -
> > -# ==============================================================================
> > -# Proxy variables
> > -# ==============================================================================
> > -
> > -ARG HTTP_PROXY
> > -ENV http_proxy ${HTTP_PROXY}
> > -ENV https_proxy ${HTTP_PROXY}
> > -
> > -# ==============================================================================
> > -# Prepare basic image
> > -# ==============================================================================
> > -
> > -ARG DEBIAN_FRONTEND=noninteractive
> > -
> > -WORKDIR /
> > -RUN echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> > -RUN echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> > -
> > -# Fuego python dependencies
> > -# - python-lxml: ftc, loggen
> > -# - python-simplejson: ftc
> > -# - python-yaml: ftc
> > -# - python-openpyxl: ftc (also LTP)
> > -# - python-requests: ftc (also fuego_release_test)
> > -# - python-reportlab: ftc
> > -# - python-parsedatetime: ftc
> > -# - python-pexpect: ssh_exec (part of ttc)
> > -# - python-pip: to install filelock, flake8
> > -# - filelock: parser
> > -RUN mkdir -p /usr/share/man/man1
> > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> > - python-lxml python-simplejson python-yaml python-openpyxl \
> > - python-requests python-reportlab python-parsedatetime \
> > - python-pexpect python-pip python-setuptools python-wheel
> > -RUN pip install filelock
> > -
> > -# Fuego command dependencies
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > - bash-completion iproute2
> > -
> > -# Default SDK for testing locally or on an x86 board
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - build-essential cmake bison flex automake kmod libtool \
> > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > -
> > -# Default test host dependencies
> > -# - iperf iperf3 netperf: used as servers on the host
> > -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > -# - python-matplotlib: Benchmark.iperf3 parser
> > -# - python-xmltodict: AGL tests
> > -# - flake8: Functional.fuego_lint
> > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > -# - iputils-ping - for /bin/ping command, used by some tests
> > -# FIXTHIS: install dependencies dynamically on the tests that need them
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > - netpipe-tcp iputils-ping
> > -RUN pip install flake8
> > -
> > -# miscelaneous packages:
> > -# python-serial - used by serio
> > -# diffstat and vim - used by Tim
> > -# time - useful for timing command duration
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - python-serial \
> > - diffstat \
> > - vim \
> > - time
> > -
> > -# FIXTHIS: determine if these tools are really necessary
> > -# RUN apt-get -q=2 -V --no-install-recommends install \
> > -# apt-utils python-paramiko \
> > -# xmlstarlet rsync \
> > -# inotify-tools gettext netpipe-tcp \
> > -# at minicom lzop bsdmainutils \
> > -# mc netcat openssh-server
> > -
> > -# bsdmainutils provides hexdump, which is needed for netperf test
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - bsdmainutils
> > -
> > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; dpkg-reconfigure dash'
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
> > -
> > -# ==============================================================================
> > -# get ttc script and helpers
> > -# ==============================================================================
> > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> > -# as of ttc version 2.0.2, this should not be needed
> > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> > -
> > -# ==============================================================================
> > -# Serial Config
> > -# ==============================================================================
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s
> > /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > -
> > -# ==============================================================================
> > -# fserver
> > -# ==============================================================================
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver ; ln -s
> /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server'
> > -
> > -# ==============================================================================
> > -# ftc post installation
> > -# ==============================================================================
> > -
> > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> > -
> > -# ==============================================================================
> > -# Lava
> > -# ==============================================================================
> > -
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > -
> > -# ==============================================================================
> > -# Setup startup command
> > -# ==============================================================================
> > -
> > -# FIXTHIS: when running multiple Fuego containers netperf will complain because the port is taken
> > -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
> > diff --git a/README b/README
> > index 882d84e..ce42e1e 100644
> > --- a/README
> > +++ b/README
> > @@ -44,6 +44,29 @@ You can change the name of the docker image and the port used by Jenkins
> > by passing the corresponding parameters to install.sh.
> > ./install.sh fuego-8082 8082
> >
> > +To run fuego on different distribution versions like Debian(stretch, buster, etc.)
> > + pass appropriate arguments to the install.sh script
> > + ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
>
> should probably be:
> ./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
>
> > +
> > +By default install.sh script will install fuego on Debian stretch version
> > + to install fuego on Debian buster version pass below options
> > + ./install.sh fuego_buster 8090 debian buster
> > +
> > +[Note] It is currently verified on Debian versions (stretch and buster),
> > +and yet to provide install scripts for other distributions
> > +
> > +Installing Fuego to a host system
> > +-------------------------------------------------------------------------------
> > +Run install-fuego.sh to install fuego on host system, by default it assume the
> > +host system is Debian stretch
> > + ./install-fuego.sh
> > +
> > + to install on Debian buster version
> > + ./install-fuego.sh -d debian -s buster -p 8090
> > +
> > + to install without jenkins pass --nojenkins option to the script
> > + ./install-fuego.sh --nojenkins
> > +
> > Running
> > -------------------------------------------------------------------------------
> > To start your Fuego container, issue the following command top directory:
> > @@ -85,9 +108,9 @@ Updating the Jenkins version
> > -------------------------------------------------------------------------------
> >
> > Modify the version and checksum
> > - $ vi Dockerfile
> > - ARG JENKINS_VERSION=2.32.1
> > - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> > + $ vi install-debian.sh
> > + JENKINS_VERSION=2.32.1
> > + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> >
> > Re-build the flot plugin
> > $ sudo apt-get install maven
> > diff --git a/entrypoint.sh b/entrypoint.sh
> > new file mode 100755
> > index 0000000..6eac895
> > --- /dev/null
> > +++ b/entrypoint.sh
> > @@ -0,0 +1,7 @@
> > +#!/bin/bash
> > +
> > +[ "$1" != "--nojenkins" ] && service jenkins start
> > +
> > +#TODO: netperf not present in buster, remove this condition once alternative found
> > +[ -f /etc/init.d/netperf ] && service netperf start
> > +iperf3 -V -s -D -f M
> > diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
> > index 5823357..6889c47 100755
> > --- a/fuego-host-scripts/docker-build-image.sh
> > +++ b/fuego-host-scripts/docker-build-image.sh
> > @@ -2,9 +2,10 @@
> > # $1 - name for the docker image (default: fuego)
> > # $2 - port for jenkins (default: 8090)
> > # $3 - Dockerfile or Dockerfile.nojenkins
> > +# $4 - Debian code name on which to install (default: stretch)
> > #
> > # Example:
> > -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 Dockerfile.nojenkins
> > +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082 debian stretch --nojenkins
> > #
> > if [ "$1" = "--no-cache" ]; then
> > NO_CACHE=--no-cache
> > @@ -13,7 +14,10 @@ fi
> >
> > DOCKERIMAGE=${1:-fuego}
> > JENKINS_PORT=${2:-8090}
> > -DOCKERFILE=${3:-Dockerfile}
> > +DISTRIBUTION=${3:-debian}
> > +SUITE=${4:-stretch}
> > +NO_JENKINS=${5:""}
> > +DOCKERFILE="Dockerfile"
> >
> > if [ "$(id -u)" == "0" ]; then
> > JENKINS_UID=$(id -u $SUDO_USER)
> > @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
> >
> > sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy \
> > --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
> > + --build-arg DIST=$DISTRIBUTION \
> > + --build-arg SUITE=$SUITE \
> > + --build-arg NO_JENKINS=$NO_JENKINS \
> > --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
> > diff --git a/install-buster.sh b/install-buster.sh
> > new file mode 100755
> > index 0000000..f4ce65b
> > --- /dev/null
> > +++ b/install-buster.sh
> > @@ -0,0 +1,294 @@
> > +#!/bin/bash
> > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > +#
> > +# Usage:
> > +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> > +#
> > +usage(){
> > +
> > +cat <<HERE
> > +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> > +
> > +Install fuego on the host Debian filesystem.
> > +
> > +options:
> > + --help Show usage help
> > + --nojenkins Install Fuego without Jenkins
> > +HERE
> > +}
> > +
> > +if [[ $EUID -ne 0 ]]; then
> > + echo "Sorry, you need root permissions"
> > + exit 1
> > +fi
> > +
> > +nojenkins=0
> > +
> > +POSITIONAL=()
> > +while [[ $# -gt 0 ]] ; do
> > + case $1 in
> > + --nojenkins)
> > + nojenkins=1
> > + shift
> > + ;;
> > + --help | -h)
> > + usage
> > + exit 0
> > + ;;
> > + *)
> > + POSITIONAL+=("$1") # save argument for later
> > + shift
> > + esac
> > +done
> > +set -- "${POSITIONAL[@]}" # restore positional arguments
> > +
> > +port=${2:-8090}
> > +
> > +# ==============================================================================
> > +# Install Fuego dependencies
> > +# ==============================================================================
> > +
> > +mkdir -p /usr/share/man/man1
> > +
> > +# netperf is in non-free
> > +echo deb http://deb.debian.org/debian buster main non-free > /etc/apt/sources.list
> > +echo deb http://security.debian.org/debian-security buster/updates main >> /etc/apt/sources.list
> > +
> > +apt-get update -q=2
> > +
> > +# Fuego python dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-lxml python-simplejson python-yaml python-openpyxl \
> > + python-requests python-reportlab python-parsedatetime \
> > + python-pexpect python-pip python-setuptools python-wheel
> > +pip install filelock
> > +
>
> Can we put this stuff that is common between install-buster.sh and install-stretch.sh
> into an "install-debian-common.sh"?
>
> > +# Fuego command dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > + bash-completion iproute2
> > +
> > +# Default SDK for testing locally or on an x86 board
> > +apt-get -q=2 -V --no-install-recommends install \
> > + build-essential cmake bison flex kmod automake libtool \
> > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > +
> > +# Default test host dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> > + netpipe-tcp iputils-ping
> > +pip install flake8
> > +
> > +# miscelaneous packages:
> > +# python-serial - used by serio
> > +# diffstat and vim - used by Tim
> > +# time - useful for timing command duration
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-serial \
> > + diffstat \
> > + vim \
> > + time
> > +
> > +# install packages used by NuttX SDK
> > +apt-get -q=2 -V --no-install-recommends install \
> > + genromfs
> > +
> > +# bsdmainutils provides hexdump, which is needed for netperf test
> > +#apt-get -q=2 -V --no-install-recommends install \
> > +# bsdmainutils
> > +
> > +echo "dash dash/sh boolean false" | debconf-set-selections
> > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > +
> > +# TODO: check if proxy settings can set outside docker container
> > +if [ -n "$http_proxy" ]; then
> > + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> > +fi
>
> This doesn't seem specific to Debian (/etc/wgetrc and /etc/environment).
> Should this be moved to an 'install-common.sh' script?
>
> > +
> > +# ==============================================================================
> > +# Install Jenkins
> > +# ==============================================================================
> > +
> > +if [ $nojenkins -eq 0 ]; then
> > + #JENKINS_VERSION=2.164.2
> > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > + #JENKINS_VERSION=2.249.3
> > + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > + JENKINS_VERSION=2.164.3
> > + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
> > + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > + export JENKINS_UC=https://updates.jenkins.io
> > + REF=/var/lib/jenkins/plugins
> > + JENKINS_HOME=/var/lib/jenkins
> > + JENKINS_PORT=$port
> > +
> > + # Jenkins dependencies
> > + apt-get -q=2 -V --no-install-recommends install \
> > + default-jdk daemon psmisc adduser procps unzip
> > + pip install python-jenkins==1.4.0
> > +
> > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > + export JENKINS_PORT=$port
> > +
> > + groupadd jenkins
> > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> > +
> > + wget -nv ${JENKINS_URL}
> > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> > +
> > + # allow Jenkins to start and install plugins, as part of dpkg installation
> > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > + rm jenkins_${JENKINS_VERSION}_all.deb
> > +
> > + # fix for Jenkins Java version check error
> > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version
> > \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
> > + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
> > +
> > + source /etc/default/jenkins && \
> > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> > +
> > + source /etc/default/jenkins && \
> > + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > + if [ -n "$http_proxy" ]; then \
> > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > + fi && \
> > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> > +
> > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/Jenkins
>
> The above looks like there's a lot of non-Debian-specific stuff intermingled with Debian-specific stuff.
> I'm not sure what other distributions are intended to be supported, but wouldn't the user and
> group manipulation be the same for those?
>
> > +
> > + # set up Jenkins plugins
> > + cp frontend-install/install-plugins.sh \
> > + frontend-install/jenkins-support \
> > + frontend-install/clitest \
> > + /usr/local/bin/
> > +
> > + mkdir -p $JENKINS_HOME/userContent/docs
> > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +
> > + # start and stop jenkins to pre-populate some settings
> > + service jenkins start && \
> > + sleep 30 && \
> > + service jenkins stop
> > +
> > + # install other plugins from Jenkins update center
> > + # NOTE: not sure all of these are needed, but keep list
> > + # compatible with 1.2.1 release for now
> > + #/usr/local/bin/install-plugins.sh \
> > + # ant:1.9 \
> > + # antisamy-markup-formatter:1.5 \
> > + # bouncycastle-api:2.17 \
> > + # command-launcher:1.3 \
> > + # description-setter:1.10 \
> > + # display-url-api:2.3.1 \
> > + # external-monitor-job:1.7 \
> > + # greenballs:1.15 \
> > + # icon-shim:2.0.3 \
> > + # javadoc:1.5 \
> > + # jdk-tool:1.2 \
> > + # junit:1.27 \
> > + # ldap:1.20 \
> > + # mailer:1.23 \
> > + # matrix-auth:2.3 \
> > + # matrix-project:1.14 \
> > + # pam-auth:1.4 \
> > + # pegdown-formatter:1.3 \
> > + # structs:1.17 \
> > + # windows-slaves:1.4
> > +
> > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > + /usr/local/bin/install-plugins.sh \
> > + script-security \
> > + structs \
> > + workflow-step-api \
> > + workflow-api \
> > + junit \
> > + scm-api \
> > + ant \
> > + antisamy-markup-formatter \
> > + bouncycastle-api \
> > + command-launcher \
> > + description-setter \
> > + display-url-api \
> > + external-monitor-job \
> > + greenballs \
> > + icon-shim \
> > + javadoc \
> > + jdk-tool \
> > + mailer \
> > + matrix-auth \
> > + matrix-project \
> > + pam-auth \
> > + pegdown-formatter \
> > + windows-slaves
> > +
> > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> > +
> > + # make the mod.js symlink well after flot is installed
> > + service jenkins start && sleep 30 && \
> > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> > +
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +fi
> > +
> > +# ==============================================================================
> > +# get ttc script and helpers
> > +# ==============================================================================
> > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > +/usr/local/src/ttc/install.sh /usr/local/bin
> > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> > +
> The above and following are definitely not unique to Debian, or a specific version of Debian.
> They should be put into an "install-common.sh"
>
> > +# ==============================================================================
> > +# Serial Config
> > +# ==============================================================================
> > +if [ $nojenkins -eq 0 ]; then
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > +else
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > +fi
> > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> > + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> > + /tmp/
> > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> > +
> > +
> > +# ==============================================================================
> > +# get ebf script and helpers
> > +# ==============================================================================
> > +apt-get -q=2 -V --no-install-recommends install jq
> > +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > +chmod a+x /usr/local/bin/ebf
> > +
> > +# ==============================================================================
> > +# fserver
> > +# ==============================================================================
> > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> > +
> > +# ==============================================================================
> > +# ftc post installation
> > +# ==============================================================================
> > +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > +echo ". /etc/bash_completion" >> /root/.bashrc
> > +
> > +# ==============================================================================
> > +# Lava
> > +# ==============================================================================
> > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > diff --git a/install-debian.sh b/install-debian.sh
> > index ccca032..2d3ffd3 100755
> > --- a/install-debian.sh
> > +++ b/install-debian.sh
> > @@ -2,102 +2,163 @@
> > # 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > #
> > # Usage:
> > -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> > +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> > #
> > -if [ -n "$1" ]; then
> > - if [ "$1" = "--help" -o "$1" = "-h" ]; then
> > - cat <<HERE
> > -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> > -
> > -Installs fuego on the host Debian filesystem.
> > -
> > -options:
> > - --help Show usage help
> > - --nojenkins Install Fuego without Jenkins
> > -HERE
> > - exit 0
> > - fi
> > -fi
> > +
> > +usage(){
> > + echo ""
> > + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> > + echo "Install fuego on Debian filesystem."
> > + echo ""
> > + echo "options:"
> > + echo " --help | -h Show usage help"
> > + echo " --suite | -s <suite> Specify Debian suite"
> > + echo " --port | -p <port> Jenkins port"
> > + echo " --nojenkins Install Fuego without Jenkins"
> > + echo ""
> > +}
> >
> > if [[ $EUID -ne 0 ]]; then
> > echo "Sorry, you need root permissions"
> > exit 1
> > fi
> >
> > -if [ "$1" = "--nojenkins" ]; then
> > - nojenkins=1
> > +nojenkins=0
> > +port=8090
> > +suite=stretch
> > +user=jenkins
> > +group=jenkins
> > +uid=1000
> > +gid=1000
> > +
> > +while [ "$1" != "" ]; do
> > + case $1 in
> > + -s | --suite )
> > + shift
> > + suite=$1
> > + ;;
> > + -p | --port )
> > + shift
> > + port=$1
> > + ;;
> > + -u | --uid )
> > + shift
> > + uid=$1
> > + ;;
> > + -g | --gid )
> > + shift
> > + gid=$1
> > + ;;
> > + --nojenkins )
> > + nojenkins=1
> > + ;;
> > + -h | --help )
> > + usage
> > + exit 0
> > + ;;
> > + * )
> > + break
> > + esac
> > shift
> > -else
> > - nojenkins=0
> > - port=${1:-8090}
> > -fi
> > +done
> >
> > # ==============================================================================
> > # Install Fuego dependencies
> > # ==============================================================================
> >
> > # netperf is in non-free
> > -echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> > -echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> > -
> > -apt-get update -q=2
> > +echo deb http://deb.debian.org/debian $suite main non-free > /etc/apt/sources.list
> > +echo deb http://security.debian.org/debian-security $suite/updates main >> /etc/apt/sources.list
> >
> > # Fuego python dependencies
> > -apt-get -q=2 -V --no-install-recommends install \
> > +# - python-lxml: ftc, loggen
> > +# - python-simplejson: ftc
> > +# - python-yaml: ftc
> > +# - python-openpyxl: ftc (also LTP)
> > +# - python-requests: ftc (also fuego_release_test)
> > +# - python-reportlab: ftc
> > +# - python-parsedatetime: ftc
> > +# - python-pexpect: ssh_exec (part of ttc)
> > +# - python-pip: to install filelock, flake8
> > +# - filelock: parser
> > +mkdir -p /usr/share/man/man1
> > +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> > python-lxml python-simplejson python-yaml python-openpyxl \
> > python-requests python-reportlab python-parsedatetime \
> > - python-pip
> > + python-pexpect python-pip python-setuptools python-wheel
> > pip install filelock
> >
> > # Fuego command dependencies
> > apt-get -q=2 -V --no-install-recommends install \
> > git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > - bash-completion
> > + bash-completion iproute2
> >
> > # Default SDK for testing locally or on an x86 board
> > apt-get -q=2 -V --no-install-recommends install \
> > build-essential cmake bison flex automake kmod libtool \
> > libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
> > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> >
> > # Default test host dependencies
> > +# - iperf iperf3 netperf: used as servers on the host
> > +# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > +# - python-matplotlib: Benchmark.iperf3 parser
> > +# - python-xmltodict: AGL tests
> > +# - flake8: Functional.fuego_lint
> > +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > +# - iputils-ping - for /bin/ping command, used by some tests
> > +# FIXTHIS: install dependencies dynamically on the tests that need them
> > apt-get -q=2 -V --no-install-recommends install \
> > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
> > -pip install flake8 bsdmainutils
> > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> > + netpipe-tcp iputils-ping
> >
> > -echo "dash dash/sh boolean false" | debconf-set-selections
> > -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > -if [ -n "$http_proxy" ]; then
> > - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> > +#TODO: find alternate package for netperf in buster version
> > +if [ $suite != "buster" ];then
> > + apt-get -q=2 -V --no-install-recommends install netperf
> > fi
> > +pip install flake8
> >
> > -# ==============================================================================
> > -# Clone fuego and fuego-core
> > -# ==============================================================================
> > -if [ ! -d "/fuego" ]; then
> > - cd /
> > - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego.git
> > - ln -s /fuego/fuego-ro /fuego-ro
> > - ln -s /fuego/fuego-rw /fuego-rw
> > - cd fuego
> > - git clone --branch next --depth=1 https://bitbucket.org/nirrognas/fuego-core.git
> > - ln -s /fuego/fuego-core /fuego-core
> > -fi
> > +# miscelaneous packages:
> > +# python-serial - used by serio
> > +# diffstat and vim - used by Tim
> > +# time - useful for timing command duration
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-serial \
> > + diffstat \
> > + vim \
> > + time
> > +
> > +# install packages used by NuttX SDK
> > +apt-get -q=2 -V --no-install-recommends install \
> > + genromfs
> > +
> > +# FIXTHIS: determine if these tools are really necessary
> > +#RUN apt-get -q=2 -V --no-install-recommends install \
> > +# apt-utils python-paramiko \
> > +# xmlstarlet rsync \
> > +# inotify-tools gettext netpipe-tcp \
> > +# at minicom lzop bsdmainutils \
> > +# mc netcat openssh-server
> > +
> > +# bsdmainutils provides hexdump, which is needed for netperf test
> > +apt-get -q=2 -V --no-install-recommends install \
> > + bsdmainutils
> > +
> > +echo "dash dash/sh boolean false" | debconf-set-selections
> > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> >
> > # ==============================================================================
> > # Install Jenkins
> > # ==============================================================================
> > -
> > if [ $nojenkins -eq 0 ]; then
> > #JENKINS_VERSION=2.164.2
> > #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > JENKINS_VERSION=2.249.3
> > JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > - JENKINS_UC=https://updates.jenkins.io
> > - REF=/var/lib/jenkins/plugins
> > - JENKINS_HOME=/var/lib/jenkins
> > + export JENKINS_UC=https://updates.jenkins.io
> > + export REF=/var/lib/jenkins/plugins
> > + export JENKINS_HOME=/var/lib/jenkins
> > JENKINS_PORT=$port
> >
> > # Jenkins dependencies
> > @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
> > pip install python-jenkins==1.4.0
> >
> > echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > - groupadd jenkins
> > - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> > - cd
> > - wget ${JENKINS_URL}
> > + export JENKINS_PORT=$port
> > +
> > + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> > + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
> > +
> > + wget -nv ${JENKINS_URL}
> > echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> >
> > # allow Jenkins to start and install plugins, as part of dpkg installation
> > @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
> > dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > rm jenkins_${JENKINS_VERSION}_all.deb
> >
> > - # update ownership
> > - chown -R jenkins:jenkins /fuego
> > - chown -R jenkins:jenkins /fuego-ro
> > - chown -R jenkins:jenkins /fuego-rw
> > - chown -R jenkins:jenkins /fuego-core
> > +fi
> > +
> > +# ==============================================================================
> > +# get ttc script and helpers
> > +# ==============================================================================
> > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > +/usr/local/src/ttc/install.sh /usr/local/bin
> > +# as of ttc version 2.0.2, this should not be needed
> > +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> > +
> > +# ==============================================================================
> > +# get ebf script and helpers
> > +# ==============================================================================
> > +if [ $nojenkins -eq 0 ]; then
> > + apt-get -q=2 -V --no-install-recommends install jq
> > + git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> > + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > + chmod a+x /usr/local/bin/ebf
> > +fi
> > +
> > +# ==============================================================================
> > +# Serial Config
> > +# ==============================================================================
> > +git clone https://github.com/frowand/serio.git /usr/local/src/serio
> > +if [ $nojenkins -eq 0 ]; then
> > + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> > + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> > + /tmp/
> > + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch
> > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch
> > + chown -R jenkins /usr/local/src/serio
> > +fi
> > +cp /usr/local/src/serio/serio /usr/local/bin/
> > +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> > +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> > +
> > +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
> > +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
> > +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> > +
> > +# ==============================================================================
> > +# fserver
> > +# ==============================================================================
> > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> > +
> > +# ==============================================================================
> > +# Jenkins post installation
> > +# ==============================================================================
> > +if [ $nojenkins -eq 0 ]; then
> > +
> > + # Fix jenkins issue, Java version check fails when java version is 11
> > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.* version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-
> > \]*\\\).*\\\".*/\\\1\\\2\/p;')"
> > + sed -i -e "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g" /etc/init.d/jenkins
> >
> > source /etc/default/jenkins && \
> > JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
> >
> > sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> >
> > - cp /fuego/frontend-install/install-plugins.sh \
> > - /fuego/frontend-install/jenkins-support \
> > - /fuego/frontend-install/clitest \
> > + # set up Jenkins plugins
> > + cp frontend-install/install-plugins.sh \
> > + frontend-install/jenkins-support \
> > + frontend-install/clitest \
> > /usr/local/bin/
> >
> > - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
> > - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > - mkdir $JENKINS_HOME/userContent/docs
> > - cp /fuego/docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> > - jenkins cp /fuego/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > - sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > - chown -R jenkins:jenkins $JENKINS_HOME/
> > -
> > # start and stop jenkins to pre-populate some settings
> > service jenkins start && \
> > sleep 30 && \
> > service jenkins stop
> >
> > + # install plugins - these versions are for Jenkins version 2.164.2
> > + # Explicitly install script-security v1.68, otherwise
> > + # v1.74 will automatically be installed as a dependency of
> > + # the junit plugin. Make sure to install before junit plugin.
> > + # Explicitly install junit:1.27
> > +
> > # install other plugins from Jenkins update center
> > # NOTE: not sure all of these are needed, but keep list
> > - # compatible with 1.2.1 release for now
> > - #/usr/local/bin/install-plugins.sh \
> > - # ant:1.9 \
> > - # antisamy-markup-formatter:1.5 \
> > - # bouncycastle-api:2.17 \
> > - # command-launcher:1.3 \
> > - # description-setter:1.10 \
> > - # display-url-api:2.3.1 \
> > - # external-monitor-job:1.7 \
> > - # greenballs:1.15 \
> > - # icon-shim:2.0.3 \
> > - # javadoc:1.5 \
> > - # jdk-tool:1.2 \
> > - # junit:1.27 \
> > - # ldap:1.20 \
> > - # mailer:1.23 \
> > - # matrix-auth:2.3 \
> > - # matrix-project:1.14 \
> > - # pam-auth:1.4 \
> > - # pegdown-formatter:1.3 \
> > - # structs:1.17 \
> > - # windows-slaves:1.4
> > -
> > - # install plugins for latest LTS Jenkins (currently 2.249.3)
> > - /usr/local/bin/install-plugins.sh \
> > - script-security \
> > - structs \
> > - workflow-step-api \
> > - workflow-api \
> > - junit \
> > - scm-api \
> > - ant \
> > - antisamy-markup-formatter \
> > - bouncycastle-api \
> > - command-launcher \
> > - description-setter \
> > - display-url-api \
> > - external-monitor-job \
> > - greenballs \
> > - icon-shim \
> > - javadoc \
> > - jdk-tool \
> > - mailer \
> > - matrix-auth \
> > - matrix-project \
> > - pam-auth \
> > - pegdown-formatter \
> > - windows-slaves
> > -
> > - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> > + # as compatible as possible with 1.2.1 release for now
> > + # Do NOT change the order of the plugins, unless you know what
> > + # you are doing. Otherwise, the install-plugins.sh script will
> > + # install the wrong versions of other plugins based on dependency
> > + # information in the plugin files.
> >
> > + #/usr/local/bin/install-plugins.sh \
> > + # script-security:1.68 \
> > + # structs:1.20 \
> > + # workflow-step-api:2.22 \
> > + # workflow-api:2.40 \
> > + # junit:1.27 \
> > + # scm-api:2.6.3 \
> > + # ant:1.9 \
> > + # antisamy-markup-formatter:1.5 \
> > + # bouncycastle-api:2.17 \
> > + # command-launcher:1.3 \
> > + # description-setter:1.10 \
> > + # display-url-api:2.3.1 \
> > + # external-monitor-job:1.7 \
> > + # greenballs:1.15 \
> > + # icon-shim:2.0.3 \
> > + # javadoc:1.5 \
> > + # jdk-tool:1.2 \
> > + # ldap:1.20 \
> > + # mailer:1.23 \
> > + # matrix-auth:2.3 \
> > + # matrix-project:1.14 \
> > + # pam-auth:1.5 \
> > + # pegdown-formatter:1.3 \
> > + # windows-slaves:1.4
> > +
> > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > + /usr/local/bin/install-plugins.sh \
> > + script-security \
> > + structs \
> > + workflow-step-api \
> > + workflow-api \
> > + junit \
> > + scm-api \
> > + ant \
> > + antisamy-markup-formatter \
> > + bouncycastle-api \
> > + command-launcher \
> > + description-setter \
> > + display-url-api \
> > + external-monitor-job \
> > + greenballs \
> > + icon-shim \
> > + javadoc \
> > + jdk-tool \
> > + mailer \
> > + matrix-auth \
> > + matrix-project \
> > + pam-auth \
> > + pegdown-formatter \
> > + windows-slaves
> > +
> > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> > +
> > + # jenkins should automatically unzip any plugins in the plugin dir
> > # make the mod.js symlink well after flot is installed
> > service jenkins start && sleep 30 && \
> > rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> >
> > - chown -R jenkins:jenkins $JENKINS_HOME/
> > -else
> > - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-ro/conf/fuego.conf
> > -fi
> > + mkdir -p $JENKINS_HOME/userContent/docs
> > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> >
> > -# ==============================================================================
> > -# get ttc script and helpers
> > -# ==============================================================================
> > -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > -/usr/local/src/ttc/install.sh /usr/local/bin
> > -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> >
> > -# ==============================================================================
> > -# Serial Config
> > -# ==============================================================================
> > -if [ $nojenkins -eq 0 ]; then
> > - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > -else
> > - /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > fi
> >
> > -# ==============================================================================
> > -# fserver
> > -# ==============================================================================
> > -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > -ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> > -
> > # ==============================================================================
> > # ftc post installation
> > # ==============================================================================
> > ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > echo ". /etc/bash_completion" >> /root/.bashrc
> >
> > # ==============================================================================
> > @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
> > # ==============================================================================
> > ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > -
> > -# ==============================================================================
> > -# Small guide
> > -# ==============================================================================
> > -echo "Run 'service netperf start' to start a netperf server"
> > -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > -echo "Run 'ftc list-boards' to see the available boards"
> > -echo "Run 'ftc list-tests' to see the available tests"
> > -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> > -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> > -echo "Run 'ftc gen-report' to get results"
> > diff --git a/install-fuego.sh b/install-fuego.sh
> > new file mode 100755
> > index 0000000..7b3e010
> > --- /dev/null
> > +++ b/install-fuego.sh
> > @@ -0,0 +1,109 @@
> > +#!/bin/bash
> > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > +#
> > +# Usage:
> > +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]
> > +#
> > +
> > +usage(){
> > + echo ""
> > + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p <jenkins port>] [--nojenkins]"
> > + echo "Install fuego on Debian filesystem."
> > + echo ""
> > + echo "options:"
> > + echo " --help | -h Show usage help"
> > + echo " --dist | -d <distribution> Specify distribution"
> > + echo " --suite | -s <suite> Specify suite"
> > + echo " --port | -p <port> Jenkins port"
> > + echo " --nojenkins Install Fuego without Jenkins"
> > + echo ""
> > +}
> > +
> > +if [[ $EUID -ne 0 ]]; then
> > + echo "Sorry, you need root permissions"
> > + exit 1
> > +fi
> > +
> > +nojenkins=
> > +port=8090
> > +dist=debian
> > +suite=stretch
> > +fuego_dir=$(pwd)
> > +
> > +while [ "$1" != "" ]; do
> > + case $1 in
> > + -d | --dist )
> > + shift
> > + dist=$1
> > + ;;
> > + -s | --suite )
> > + shift
> > + suite=$1
> > + ;;
> > + -p | --port )
> > + shift
> > + port=$1
> > + ;;
> > + --nojenkins )
> > + nojenkins="--nojenkins"
> > + ;;
> > + -h | --help )
> > + usage
> > + exit 0
> > + ;;
> > + * )
> > + break
> > + esac
> > + shift
> > +done
> > +
> > +# ==============================================================================
> > +# Clone fuego-core
> > +# ==============================================================================
> > +if [ ! -f "fuego-core/scripts/ftc" ]; then
> > + # set fuego-core branch to same as current fuego branch
> > + # get current git branch
> > + set -o noglob
> > + while IFS=" " read -r part1 part2 ; do
> > + if [ $part1 = "*" ] ; then
> > + branch=$part2
> > + fi
> > + done < <(git branch)
> > + set +o noglob
> > + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
> > +fi
> > +
> > +# ==============================================================================
> > +# Install fuego on host system
> > +# ==============================================================================
> > +./install-$dist.sh -s $suite -p $port $nojenkins
> > +
> > +systemctl daemon-reload
> > +service jenkins restart
> > +
> > +# provide permission to current user
> > +usermod -aG jenkins $USER
> > +chmod -R 775 $fuego_dir/fuego-rw
> > +
> > +chown -R jenkins:jenkins $fuego_dir/fuego-ro
> > +chown -R jenkins:jenkins $fuego_dir/fuego-rw
> > +chown -R jenkins:jenkins $fuego_dir/fuego-core
> > +
> > +# ==============================================================================
> > +# Create Links to fuego folders
> > +# ==============================================================================
> > +ln -s $fuego_dir/fuego-ro /
> > +ln -s $fuego_dir/fuego-rw /
> > +ln -s $fuego_dir/fuego-core /
> > +
> > +# ==============================================================================
> > +# Small guide
> > +# ==============================================================================
> > +echo "Logout and Login current user session to work properly"
> > +echo "Run 'service netperf start' to start a netperf server"
> > +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > +echo "Run 'ftc list-boards' to see the available boards"
> > +echo "Run 'ftc list-tests' to see the available tests"
> > +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello world"
> > +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run Dhrystone"
> > +echo "Run 'ftc gen-report' to get results"
> > diff --git a/install-stretch.sh b/install-stretch.sh
> > new file mode 100755
> > index 0000000..4f6f724
> > --- /dev/null
> > +++ b/install-stretch.sh
> > @@ -0,0 +1,288 @@
> > +#!/bin/bash
> > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > +#
> > +# Usage:
> > +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> > +#
> > +usage(){
> > +
> > +cat <<HERE
> > +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> > +
> > +Install fuego on the host Debian filesystem.
> > +
> > +options:
> > + --help Show usage help
> > + --nojenkins Install Fuego without Jenkins
> > +HERE
> > +}
> > +
> > +if [[ $EUID -ne 0 ]]; then
> > + echo "Sorry, you need root permissions"
> > + exit 1
> > +fi
> > +
> > +nojenkins=0
> > +
> > +POSITIONAL=()
> > +while [[ $# -gt 0 ]] ; do
> > + case $1 in
> > + --nojenkins)
> > + nojenkins=1
> > + shift
> > + ;;
> > + --help | -h)
> > + usage
> > + exit 0
> > + ;;
> > + *)
> > + POSITIONAL+=("$1") # save argument for later
> > + shift
> > + esac
> > +done
> > +set -- "${POSITIONAL[@]}" # restore positional arguments
> > +
> > +port=${2:-8090}
> > +
> > +# ==============================================================================
> > +# Install Fuego dependencies
> > +# ==============================================================================
> > +
> > +mkdir -p /usr/share/man/man1
> > +
> > +# netperf is in non-free
> > +echo deb http://deb.debian.org/debian stretch main non-free > /etc/apt/sources.list
> > +echo deb http://security.debian.org/debian-security stretch/updates main >> /etc/apt/sources.list
> > +
> > +apt-get update -q=2
> > +
> > +# Fuego python dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-lxml python-simplejson python-yaml python-openpyxl \
> > + python-requests python-reportlab python-parsedatetime \
> > + python-pexpect python-pip python-setuptools python-wheel
> > +pip install filelock
> > +
> > +# Fuego command dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > + bash-completion iproute2
> > +
> > +# Default SDK for testing locally or on an x86 board
> > +apt-get -q=2 -V --no-install-recommends install \
> > + build-essential cmake bison flex kmod automake libtool \
> > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > +
> > +# Default test host dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > + netpipe-tcp iputils-ping
> > +pip install flake8
> > +
> > +# miscelaneous packages:
> > +# python-serial - used by serio
> > +# diffstat and vim - used by Tim
> > +# time - useful for timing command duration
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-serial \
> > + diffstat \
> > + vim \
> > + time
> > +
> > +# install packages used by NuttX SDK
> > +apt-get -q=2 -V --no-install-recommends install \
> > + genromfs
> > +
> > +# bsdmainutils provides hexdump, which is needed for netperf test
> > +apt-get -q=2 -V --no-install-recommends install \
> > + bsdmainutils
> > +
> > +echo "dash dash/sh boolean false" | debconf-set-selections
> > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > +
> > +# TODO: check if proxy settings can set outside docker container
> > +#if [ -n "$http_proxy" ]; then
> > +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >> /etc/environment
> > +#fi
> > +
> > +# ==============================================================================
> > +# Install Jenkins
> > +# ==============================================================================
> > +
> > +if [ $nojenkins -eq 0 ]; then
> > + #JENKINS_VERSION=2.164.2
> > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > + JENKINS_VERSION=2.249.3
> > + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > + JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > + export JENKINS_UC=https://updates.jenkins.io
> > + REF=/var/lib/jenkins/plugins
> > + JENKINS_HOME=/var/lib/jenkins
> > + JENKINS_PORT=$port
> > +
> > + # Jenkins dependencies
> > + apt-get -q=2 -V --no-install-recommends install \
> > + default-jdk daemon psmisc adduser procps unzip
> > + pip install python-jenkins==1.4.0
> > +
> > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > + export JENKINS_PORT=$port
> > +
> > + groupadd jenkins
> > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash jenkins
> > +
> > + wget -nv ${JENKINS_URL}
> > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
> > +
> > + # allow Jenkins to start and install plugins, as part of dpkg installation
> > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > + rm jenkins_${JENKINS_VERSION}_all.deb
> > +
> > + source /etc/default/jenkins && \
> > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > + sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
> > +
> > + source /etc/default/jenkins && \
> > + JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > + if [ -n "$http_proxy" ]; then \
> > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > + JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > + fi && \
> > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
> > +
> > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > +
> > + # set up Jenkins plugins
> > + cp frontend-install/install-plugins.sh \
> > + frontend-install/jenkins-support \
> > + frontend-install/clitest \
> > + /usr/local/bin/
> > +
> > + mkdir -p $JENKINS_HOME/userContent/docs
> > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + sed -i -e "s#8080#$JENKINS_PORT#g" $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +
> > + # start and stop jenkins to pre-populate some settings
> > + service jenkins start && \
> > + sleep 30 && \
> > + service jenkins stop
> > +
> > + # install other plugins from Jenkins update center
> > + # NOTE: not sure all of these are needed, but keep list
> > + # compatible with 1.2.1 release for now
> > + #/usr/local/bin/install-plugins.sh \
> > + # ant:1.9 \
> > + # antisamy-markup-formatter:1.5 \
> > + # bouncycastle-api:2.17 \
> > + # command-launcher:1.3 \
> > + # description-setter:1.10 \
> > + # display-url-api:2.3.1 \
> > + # external-monitor-job:1.7 \
> > + # greenballs:1.15 \
> > + # icon-shim:2.0.3 \
> > + # javadoc:1.5 \
> > + # jdk-tool:1.2 \
> > + # junit:1.27 \
> > + # ldap:1.20 \
> > + # mailer:1.23 \
> > + # matrix-auth:2.3 \
> > + # matrix-project:1.14 \
> > + # pam-auth:1.4 \
> > + # pegdown-formatter:1.3 \
> > + # structs:1.17 \
> > + # windows-slaves:1.4
> > +
> > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > + /usr/local/bin/install-plugins.sh \
> > + script-security \
> > + structs \
> > + workflow-step-api \
> > + workflow-api \
> > + junit \
> > + scm-api \
> > + ant \
> > + antisamy-markup-formatter \
> > + bouncycastle-api \
> > + command-launcher \
> > + description-setter \
> > + display-url-api \
> > + external-monitor-job \
> > + greenballs \
> > + icon-shim \
> > + javadoc \
> > + jdk-tool \
> > + mailer \
> > + matrix-auth \
> > + matrix-project \
> > + pam-auth \
> > + pegdown-formatter \
> > + windows-slaves
> > +
> > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi $JENKINS_HOME/plugins/flot.jpi
> > +
> > + # make the mod.js symlink well after flot is installed
> > + service jenkins start && sleep 30 && \
> > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > + ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> > +
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +fi
> > +
> > +# ==============================================================================
> > +# get ttc script and helpers
> > +# ==============================================================================
> > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > +/usr/local/src/ttc/install.sh /usr/local/bin
> > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
> > +
> > +# ==============================================================================
> > +# Serial Config
> > +# ==============================================================================
> > +if [ $nojenkins -eq 0 ]; then
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > +else
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > +fi
> > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-name.patch \
> > + frontend-install/0002-Output-data-from-port-during-command-execution.patch \
> > + /tmp/
> > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-serial-device-with-in-name.patch ; \
> > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-during-command-execution.patch ;'
> > +
> > +
> > +# ==============================================================================
> > +# get ebf script and helpers
> > +# ==============================================================================
> > +apt-get -q=2 -V --no-install-recommends install jq
> > +git clone https://github.com/TimesysGit/board-farm-rest-api.git /usr/local/src/board-farm-rest-api
> > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > +chmod a+x /usr/local/bin/ebf
> > +
> > +# ==============================================================================
> > +# fserver
> > +# ==============================================================================
> > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > +ln -s /usr/local/lib/fserver/start_local_bg_server /usr/local/bin/start_local_bg_server
> > +
> > +# ==============================================================================
> > +# ftc post installation
> > +# ==============================================================================
> > +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > +echo ". /etc/bash_completion" >> /root/.bashrc
> > +
> > +# ==============================================================================
> > +# Lava
> > +# ==============================================================================
> > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > diff --git a/install.sh b/install.sh
> > index a5a90b0..259c887 100755
> > --- a/install.sh
> > +++ b/install.sh
> > @@ -1,13 +1,13 @@
> > #!/bin/bash
> > #
> > -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> > +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
> > #
> >
> > # $1 is the exit code after usage is shown
> > usage() {
> > exit_code="$1"
> > cat <<HERE
> > -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>]
> > +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>] [<port>] [<distribution>] [<suit>]
> >
> > Create the docker image and container with the Fuego test distribution.
> > If no <image_name> is provided, the image will be named 'fuego'.
> > @@ -34,7 +34,7 @@ fi
> >
> > priv=0
> > NOCACHE=""
> > -dockerfile="Dockerfile"
> > +NOJENKINS=""
> >
> > POSITIONAL=()
> > while [[ $# -gt 0 ]] ; do
> > @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
> > shift
> > ;;
> > --nojenkins)
> > - dockerfile="Dockerfile.nojenkins"
> > + NOJENKINS="--nojenkins"
> > shift
> > ;;
> > *)
> > @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional arguments
> >
> > image_name=${1:-fuego}
> > jenkins_port=${2:-8090}
> > +distribution=${3:-debian}
> > +suite=${4:-stretch}
> >
> > container_name="${image_name}-container"
> >
> > @@ -79,7 +81,7 @@ fi
> >
> > set -e
> >
> > -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${dockerfile}
> > +source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
> > if [ "$priv" == "0" ]; then
> > fuego-host-scripts/docker-create-container.sh ${image_name} ${container_name}
> > else
> > diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
> > new file mode 100644
> > index 0000000..b899e7c
> > --- /dev/null
> > +++ b/uninstall-fuego.sh
> > @@ -0,0 +1,47 @@
> > +#!/bin/bash
> > +
> > +# Remove Jenkins plugins
> > +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf /usr/share/jenkins/ref/plugins
> > +
> > +# uninstall Jenkins
> > +service jenkins stop
> > +dpkg -P jenkins
> > +
> > +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-plugins.sh
> > +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
> > +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
> > +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
> > +sed -i '/^JENKINS_PORT=/d' /etc/environment
> > +
> > +# uninstall ttc script
> > +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
> > +
> > +# uninstall ebf script
> > +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-farm-rest-api
> > +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
> > +
> > +# uninstall serial config
> > +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
> > +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
> > +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
> > +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
> > +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
> > +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
> > +
> > +# uninstall fserver
> > +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
> > +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf /usr/local/bin/start_local_bg_server
> > +
> > +# uninstall ftc
> > +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
> > +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
> > +
> > +# remove lava target files
> > +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-lava-target-setup
> > +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf /usr/local/bin/fuego-lava-target-teardown
> > +
> > +# remove links
> > +[ -d /fuego-ro ] && rm /fuego-ro
> > +[ -d /fuego-rw ] && rm /fuego-rw
> > +[ -d /fuego-core ] && rm /fuego-core
> > +
> > --
> > 2.20.1
> >
>
> Overall - this looks like a good start. But I'd like to see the duplicate sections of install-buster.sh
> and install-stretch.sh put into a single location "install-debian-common.sh". Also, I'd like to see
> the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
>
> Can you please re-do this with that refactoring in mind?
>
> Thanks,
> -- Tim
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-15 0:08 ` Tim.Bird
2021-06-15 3:46 ` Venkata.Pyla
@ 2021-06-17 7:01 ` Venkata.Pyla
2021-06-17 20:53 ` Tim.Bird
1 sibling, 1 reply; 8+ messages in thread
From: Venkata.Pyla @ 2021-06-17 7:01 UTC (permalink / raw)
To: Tim.Bird, tbird20d; +Cc: fuego
Hi Tim,
Thanks for the feedback, I have commented in line.
I also copied your feedback and my answers here as the mail is longer to find them, also you can see in the inline in the below mail to see the context.
> If we don't support any other distributions besides Debian at the moment,
> then I'm not sure we need the distribution argument. Do you have any
> non-debian distribution install scripts ready to submit soon?
No I don’t have any plan for other distributions coming soon, I kept the distribution argument
Because in case if we have any future plan to support other distributions we can easily adapt.
I think we can remove the distribution argument if we don’t have any plan to support in near future,
I will remove this argument in my next patch if you have no other opinion.
.
.
.
> Is install-debian.sh the common parts for a Debian installation, with
> buster and stretch having the parts that are unique to each of
> those distros respectively? If we're going to refactor this, I
> want to eliminate any duplication, so that package names are
> not used in multiple places. This has been a headache to
> maintain.
>
> It appears there is lots of duplication between install-buster.sh and
> install-stretch.sh. Can we possible put this duplication into an
> "install-debian-common.sh", and reserve install-buster.sh and
> install-stretch.sh for only the parts unique to those distros?
Yes, that is a good idea, even I got the similar feedback from our internal review,
I will do this refactoring in my next patch.
.
.
.
> Ooo - I like the sound of this uninstall script.
Sorry do you have anything in mind, not required or required this uninstall script?
I have created it because it helps me in testing the for uninstalling and installing the fuego multiple times.
The uninstall script may also need lot of improvement, but I just created a basic script to solve my current problem.
I may further improve based on the feedback or the issues if any.
.
.
.
> One question about this overall approach. Does this retain the ability to cache
> operations, so that if something goes wrong with an install, you don't have to
> start over from scratch. When building with docker, with multiple steps, it
> caches the image at each step. When rebuilding, or when an error occurs,
> this can save a lot of time. It also can introduce subtle bugs, but that's
> a separate issue.
>
> This setup seems like it performs the bulk of the install as a single docker
> step, is that right?
Yes, it will not create the cache for each step in the script, this will be a disadvantage with this common script approach.
.
.
.
> I'm not sure what this is doing. It looks scary. Please explain.
Because, I had COPY the fuego current directory to the Docker image for installation purpose and removing it after installation,
and this is required because the common script "install-*.sh" requires some files to copy inside docker to a specified location
for e.g:
cp frontend-install/install-plugins.sh \
frontend-install/jenkins-support \
frontend-install/clitest \
/usr/local/bin/
And in the previous approach this was done using COPY command which will directly copy from host machine to docker location.
May be I could have used mounting directory inside docker workdir to avoid copying and later removing it, maybe I faced some problem with this approach and so later chosen to copy it inside docker image.
I have done this work little long ago so I have not remembered why I have chosen the copy it inside docker approach, Let me work on again and understand why have done this and change it to mounting directory instead copy and remove.
Please let me know if you have other ideas.
.
.
.
> should probably be:
> ./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
Thanks for suggestion,
maybe if I remove the distribution it would be like this?
./install.sh [[<image name>] [[<Jenkins port>] [<suite>]]]
.
.
.
> Overall - this looks like a good start. But I'd like to see the duplicate sections of
> install-buster.sh
> and install-stretch.sh put into a single location "install-debian-common.sh".
> Also, I'd like to see
> the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
>
> Can you please re-do this with that refactoring in mind?
Thank you for the feedback, I will work on these refactoring and send you another patch.
Thanks
Venkata.
> -----Original Message-----
> From: Tim.Bird@sony.com <Tim.Bird@sony.com>
> Sent: 15 June 2021 05:39
> To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com>;
> tbird20d@yahoo.com
> Cc: fuego@lists.linuxfoundation.org; sangorrin daniel(サンゴリン ダニエル □ＳＷ
> Ｃ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>
> Subject: RE: [Fuego] Use common script to install fuego on host or docker
> system
>
> Please see comments inline below.
>
> I tried to apply this patch from the e-mail list using 'git am', but got a failure.
> That made it a hard to experiment with and view the changes in
> context.
>
> What version of the code is this patch based on?
>
> > -----Original Message-----
> > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > Subject: [Fuego] Use common script to install fuego on host or docker system
> >
> > From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> >
> > Modified the install scripts to use common scripts for both types of
> > installation, this way it will help to make sure both the environments
> > are same and if any future modification will have single place of change.
> >
> > Now, for installing fuego on docker system the 'install.sh' script will
> > use common script in Dockefile to install fuego related stuff.
> >
> > sameway for installing fuego on host system it can use the same common
> > script to install fuego related stuff.
> >
> > Below are the common scripts that are used in both the installation types
> > for each debian distribution
> > * install-stretch.sh
> > * install-buster.sh
> >
> > so finally,
> > To install fuego on docker system use the below script with options
> > $ ./install.sh <options> image_name <port> <distribution> <suite>
> > e.g: ./install.sh debian_stretch 8090 debian stretch
> > or
> > ./install.sh debian_buster 8090 debian buster
> > or
> > ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
> This is a bit wordy, but OK.
>
> If we don't support any other distributions besides Debian at the moment,
> then I'm not sure we need the distribution argument. Do you have any
> non-debian distribution install scripts ready to submit soon?
No I don’t have any plan for other distributions coming soon, I kept the distribution argument
Because in case if we have any future plan to support other distributions we can easily adapt.
I think we can remove the distribution argument if we don’t have any plan to support.
I will remove this argument in my next patch, please let me know if you have any other opinion.
>
> I'm not sure I follow calling the Debian release name the "<suite>".
>
> >
> > To install fuego on host system use the below script with options
> > $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
> "suite" seems like an odd name. Should this be distro "version" or "release"?
>
> > e.g: ./install-fuego.sh -d debian -s stretch -p 8090
> > or
> > ./install-fuego.sh -d debian -s buster -p 8090
> > or
> > ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
> >
> > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > ---
> > Dockerfile | 303 ++---------------
> > Dockerfile.nojenkins | 151 ---------
> > README | 29 +-
> > entrypoint.sh | 7 +
> > fuego-host-scripts/docker-build-image.sh | 11 +-
> > install-buster.sh | 294 ++++++++++++++++
> > install-debian.sh | 410 ++++++++++++++---------
> > install-fuego.sh | 109 ++++++
> > install-stretch.sh | 288 ++++++++++++++++
>
> Is install-debian.sh the common parts for a Debian installation, with
> buster and stretch having the parts that are unique to each of
> those distros respectively? If we're going to refactor this, I
> want to eliminate any duplication, so that package names are
> not used in multiple places. This has been a headache to
> maintain.
>
> It appears there is lots of duplication between install-buster.sh and
> install-stretch.sh. Can we possible put this duplication into an
> "install-debian-common.sh", and reserve install-buster.sh and
> install-stretch.sh for only the parts unique to those distros?
Yes, that is a good idea, even I got the same feedback from our internal review,
I will do this refactoring in my next patch.
>
> > install.sh | 12 +-
> > uninstall-fuego.sh | 47 +++
>
> Ooo - I like the sound of this uninstall script.
Sorry do you have anything in mind, not required or required this uninstall script
I have created it because it helps me in testing the environment uninstalling and installing the fuego multiple times.
The uninstall script may also need lot of improvement, but I just created a basic script to solve my current problem.
I may further improve based on the feedback or the issues if any.
>
> > 11 files changed, 1057 insertions(+), 604 deletions(-)
> > delete mode 100644 Dockerfile.nojenkins
> > create mode 100755 entrypoint.sh
> > create mode 100755 install-buster.sh
> > create mode 100755 install-fuego.sh
> > create mode 100755 install-stretch.sh
> > create mode 100644 uninstall-fuego.sh
> >
>
>
> > diff --git a/Dockerfile b/Dockerfile
> > index 7f63798..ac57cb2 100644
> > --- a/Dockerfile
> > +++ b/Dockerfile
> > @@ -2,9 +2,11 @@
> > # WARNING: this Dockerfile assumes that the container will be created with
> > # several bind mounts (see docker-create-container.sh)
> > #
> ================================================================
> ==============
> > -# FIXTHIS: build this as an extension of the nonjenkins image
> >
> > -FROM debian:stretch-slim
> > +ARG SUITE=stretch
> > +ARG DIST=debian
> > +
> > +FROM ${DIST}:${SUITE}-slim
> > MAINTAINER tim.bird@sony.com
> >
> > #
> ================================================================
> ==============
> > @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
> > ENV https_proxy ${HTTP_PROXY}
> >
> > #
> ================================================================
> ==============
> > -# Prepare basic image
> > +# Install fuego with the script
> > #
> ================================================================
> ==============
> >
> > -ARG DEBIAN_FRONTEND=noninteractive
> > -
> > -WORKDIR /
> > -RUN echo deb http://deb.debian.org/debian stretch main non-free >
> /etc/apt/sources.list
> > -RUN echo deb http://security.debian.org/debian-security stretch/updates
> main >> /etc/apt/sources.list
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy
> "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> > -
> > -# Fuego python dependencies
> > -# - python-lxml: ftc, loggen
> > -# - python-simplejson: ftc
> > -# - python-yaml: ftc
> > -# - python-openpyxl: ftc (also LTP)
> > -# - python-requests: ftc (also fuego_release_test)
> > -# - python-reportlab: ftc
> > -# - python-parsedatetime: ftc
> > -# - python-pexpect: ssh_exec (part of ttc)
> > -# - python-pip: to install filelock, flake8
> > -# - filelock: parser
> > -RUN mkdir -p /usr/share/man/man1
> > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install
> \
> > - python-lxml python-simplejson python-yaml python-openpyxl \
> > - python-requests python-reportlab python-parsedatetime \
> > - python-pexpect python-pip python-setuptools python-wheel
> > -RUN pip install filelock
> > -
> > -# Fuego command dependencies
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > - bash-completion iproute2
> > -
> > -# Default SDK for testing locally or on an x86 board
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - build-essential cmake bison flex automake kmod libtool \
> > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > -
> > -# Default test host dependencies
> > -# - iperf iperf3 netperf: used as servers on the host
> > -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > -# - python-matplotlib: Benchmark.iperf3 parser
> > -# - python-xmltodict: AGL tests
> > -# - flake8: Functional.fuego_lint
> > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > -# - iputils-ping - for /bin/ping command, used by some tests
> > -# FIXTHIS: install dependencies dynamically on the tests that need them
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > - netpipe-tcp iputils-ping
> > -RUN pip install flake8
> > -
> > -# miscelaneous packages:
> > -# python-serial - used by serio
> > -# diffstat and vim - used by Tim
> > -# time - useful for timing command duration
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - python-serial \
> > - diffstat \
> > - vim \
> > - time
> > -
> > -# install packages used by NuttX SDK
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - genromfs
> > -
> > -# FIXTHIS: determine if these tools are really necessary
> > -#RUN apt-get -q=2 -V --no-install-recommends install \
> > -# apt-utils python-paramiko \
> > -# xmlstarlet rsync \
> > -# inotify-tools gettext netpipe-tcp \
> > -# at minicom lzop bsdmainutils \
> > -# mc netcat openssh-server
> > -
> > -# bsdmainutils provides hexdump, which is needed for netperf test
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - bsdmainutils
> > -
> > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections
> ; dpkg-reconfigure dash'
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e
> "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment;
> fi
> > -
> > -#
> ================================================================
> ==============
> > -# Install Jenkins with the same UID/GID as the host user
> > -#
> ================================================================
> ==============
> > +RUN mkdir /fuego
> > +COPY . /fuego/
> > +WORKDIR /fuego
> >
> > -ARG user=jenkins
> > -ARG group=jenkins
> > -ARG uid=1000
> > -ARG gid=${uid}
> > ARG JENKINS_PORT=8090
> > -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
> > -#ARG JENKINS_VERSION=2.164.2
> > -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > -#ARG JENKINS_VERSION=2.235.3
> > -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
> > -ARG JENKINS_VERSION=2.249.3
> > -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > -
> > -ARG JENKINS_URL=https://pkg.jenkins.io/debian-
> stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > -ARG JENKINS_UC=https://updates.jenkins.io
> > -ARG REF=/var/lib/jenkins/plugins
> > +ARG NO_JENKINS=
> > +ARG SUITE=stretch
> > +ARG DIST=debian
> > +ARG uid=
> > +ARG gid=
> > ENV JENKINS_HOME=/var/lib/jenkins
> > ENV JENKINS_PORT=$JENKINS_PORT
> > +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid
> $NO_JENKINS
>
> One question about this overall approach. Does this retain the ability to cache
> operations, so that if something goes wrong with an install, you don't have to
> start over from scratch. When building with docker, with multiple steps, it
> caches the image at each step. When rebuilding, or when an error occurs,
> this can save a lot of time. It also can introduce subtle bugs, but that's
> a separate issue.
>
> This setup seems like it performs the bulk of the install as a single docker
> step, is that right?
Yes, it will not create the cache for each step in the installation script, this will be a disadvantage with this common script approach.
>
> > -# Jenkins dependencies
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - default-jdk daemon psmisc adduser procps unzip
> > -RUN pip install python-jenkins==1.4.0
> > -
> > -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> > -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> /bin/bash ${user}
> > -RUN wget -nv ${JENKINS_URL}
> > -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
> -c -
> > -# allow Jenkins to start and install plugins, as part of dpkg installation
> > -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> > -
> > -#
> ================================================================
> ==============
> > -# get ttc script and helpers
> > -#
> ================================================================
> ==============
> > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> > -# as of ttc version 2.0.2, this should not be needed
> > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> > -
> > -#
> ================================================================
> ==============
> > -# get ebf script and helpers
> > -#
> ================================================================
> ==============
> > -RUN apt-get -q=2 -V --no-install-recommends install jq
> > -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git
> /usr/local/src/board-farm-rest-api ; \
> > - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
> > - chmod a+x /usr/local/bin/ebf
> > -
> > -#
> ================================================================
> ==============
> > -# Serial Config
> > -#
> ================================================================
> ==============
> > -
> > -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
> > -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> name.patch \
> > - frontend-install/0002-Output-data-from-port-during-command-
> execution.patch \
> > - /tmp/
> > -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-
> parsing-for-serial-device-with-in-name.patch ; \
> > - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> during-command-execution.patch ; \
> > - chown -R jenkins /usr/local/src/serio ; \
> > - cp /usr/local/src/serio/serio /usr/local/bin/ ; \
> > - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
> > - ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp
> > /usr/local/src/serlogin/serlogin /usr/local/bin'
> > -
> > -#
> ================================================================
> ==============
> > -# fserver
> > -#
> ================================================================
> ==============
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git
> /usr/local/lib/fserver ; \
> > - ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server'
> > -
> > -#
> ================================================================
> ==============
> > -# Jenkins post installation
> > -#
> ================================================================
> ==============
> > -
> > -RUN source /etc/default/jenkins && \
> > - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> /etc/default/jenkins
> > -
> > -RUN source /etc/default/jenkins && \
> > - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > - if [ -n "$HTTP_PROXY" ]; then \
> > - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > - JAVA_ARGS="$JAVA_ARGS -
> Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > - fi && \
> > - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> /etc/default/jenkins;
> > -
> > -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > -
> > -# set up Jenkins plugins
> > -COPY frontend-install/install-plugins.sh \
> > - frontend-install/jenkins-support \
> > - frontend-install/clitest \
> > - /usr/local/bin/
> > -
> > -# start Jenkins onces to initialize directories?
> > -RUN service jenkins start && \
> > - sleep 30 && \
> > - service jenkins stop
> > -
> > -# install plugins - these versions are for Jenkins version 2.164.2
> > -# Explicitly install script-security v1.68, otherwise
> > -# v1.74 will automatically be installed as a dependency of
> > -# the junit plugin. Make sure to install before junit plugin.
> > -# Explicitly install junit:1.27
> > -
> > -# install other plugins from Jenkins update center
> > -# NOTE: not sure all of these are needed, but keep list
> > -# as compatible as possible with 1.2.1 release for now
> > -# Do NOT change the order of the plugins, unless you know what
> > -# you are doing. Otherwise, the install-plugins.sh script will
> > -# install the wrong versions of other plugins based on dependency
> > -# information in the plugin files.
> > -
> > -#RUN /usr/local/bin/install-plugins.sh \
> > -# script-security:1.68 \
> > -# structs:1.20 \
> > -# workflow-step-api:2.22 \
> > -# workflow-api:2.40 \
> > -# junit:1.27 \
> > -# scm-api:2.6.3 \
> > -# ant:1.9 \
> > -# antisamy-markup-formatter:1.5 \
> > -# bouncycastle-api:2.17 \
> > -# command-launcher:1.3 \
> > -# description-setter:1.10 \
> > -# display-url-api:2.3.1 \
> > -# external-monitor-job:1.7 \
> > -# greenballs:1.15 \
> > -# icon-shim:2.0.3 \
> > -# javadoc:1.5 \
> > -# jdk-tool:1.2 \
> > -# ldap:1.20 \
> > -# mailer:1.23 \
> > -# matrix-auth:2.3 \
> > -# matrix-project:1.14 \
> > -# pam-auth:1.5 \
> > -# pegdown-formatter:1.3 \
> > -# windows-slaves:1.4
> > -
> > -# install plugins for latest LTS Jenkins (currently 2.249.3)
> > -RUN /usr/local/bin/install-plugins.sh \
> > - script-security \
> > - structs \
> > - workflow-step-api \
> > - workflow-api \
> > - junit \
> > - scm-api \
> > - ant \
> > - antisamy-markup-formatter \
> > - bouncycastle-api \
> > - command-launcher \
> > - description-setter \
> > - display-url-api \
> > - external-monitor-job \
> > - greenballs \
> > - icon-shim \
> > - javadoc \
> > - jdk-tool \
> > - mailer \
> > - matrix-auth \
> > - matrix-project \
> > - pam-auth \
> > - pegdown-formatter \
> > - windows-slaves
> > +COPY entrypoint.sh /usr/bin/
> >
> > -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi
> $JENKINS_HOME/plugins/flot.jpi
> > -
> > -# jenkins should automatically unzip any plugins in the plugin dir
> > -# make a symlink for mod.js well after flot is installed
> > -RUN service jenkins start && sleep 30 && \
> > - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> > -
> > -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> docs.pdf
> > -
> > -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> > -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > -RUN sed -i -e "s#8080#$JENKINS_PORT#g"
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > -
> > -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> > -
> > -#
> ================================================================
> ==============
> > -# ftc post installation
> > -#
> ================================================================
> ==============
> > -
> > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> > -
> > -#
> ================================================================
> ==============
> > -# Lava
> > -#
> ================================================================
> ==============
> > -
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > +# Delete the files that are copied for installation
> > +RUN rm -rf /fuego
>
> I'm not sure what this is doing. It looks scary. Please explain.
Because, I had COPY the fuego current directory to the Docker image for installation purpose and removing it after installation,
and this is required because the common script "install-*.sh" requires some files to copy inside docker to a specified location
for e.g:
cp frontend-install/install-plugins.sh \
frontend-install/jenkins-support \
frontend-install/clitest \
/usr/local/bin/
And in the previous approach this was done using COPY command which will directly copy from host machine to docker location.
May be I could have used mounting directory inside docker workdir to avoid copying and later removing it, maybe I faced some problem with this approach and so later chosen to copy it inside docker image.
I have done this work little long so I have not remembered why I have chosen the copy it inside docker approach, Let me work on again and understand why have done this and change it to mounting directory instead copy and remove.
>
> > +WORKDIR /
> >
> > #
> ================================================================
> ==============
> > # Setup startup command
> > #
> ================================================================
> ==============
> > -
> > # FIXTHIS: when running multiple Fuego containers, or if the host is already
> > # running the netperf server, netperf will complain because the port is taken
> > -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D
> -f M && /bin/bash
> > +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
>
> Isolating this to a separate script is nice!
>
> > diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
> > deleted file mode 100644
> > index 56de98a..0000000
> > --- a/Dockerfile.nojenkins
> > +++ /dev/null
>
> Getting rid of this separate Dockerfile is nice.
>
> > @@ -1,151 +0,0 @@
> > -#
> ================================================================
> ==============
> > -# WARNING: this Dockerfile assumes that the container will be created with
> > -# several bind mounts (see docker-create-container.sh)
> > -#
> ================================================================
> ==============
> > -# Usage:
> > -# host$ vi fuego-ro/conf/fuego.conf
> > -# jenkins_enabled=0
> > -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090
> Dockerfile.nojenkins
> > -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins
> fuego-nojenkins-container
> > -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-
> container
> > -# docker# ftc list-boards
> > -# docker, bbb, ...
> > -# docker# ftc list-tests
> > -# Functional.hello_world, ...
> > -# docker# ftc run-test -b docker -t Functional.hello_world
> > -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or
> equal than 1)
> > -
> > -FROM debian:stretch-slim
> > -MAINTAINER tim.bird@sony.com
> > -
> > -#
> ================================================================
> ==============
> > -# Proxy variables
> > -#
> ================================================================
> ==============
> > -
> > -ARG HTTP_PROXY
> > -ENV http_proxy ${HTTP_PROXY}
> > -ENV https_proxy ${HTTP_PROXY}
> > -
> > -#
> ================================================================
> ==============
> > -# Prepare basic image
> > -#
> ================================================================
> ==============
> > -
> > -ARG DEBIAN_FRONTEND=noninteractive
> > -
> > -WORKDIR /
> > -RUN echo deb http://deb.debian.org/debian stretch main non-free >
> /etc/apt/sources.list
> > -RUN echo deb http://security.debian.org/debian-security stretch/updates
> main >> /etc/apt/sources.list
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy
> "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> > -
> > -# Fuego python dependencies
> > -# - python-lxml: ftc, loggen
> > -# - python-simplejson: ftc
> > -# - python-yaml: ftc
> > -# - python-openpyxl: ftc (also LTP)
> > -# - python-requests: ftc (also fuego_release_test)
> > -# - python-reportlab: ftc
> > -# - python-parsedatetime: ftc
> > -# - python-pexpect: ssh_exec (part of ttc)
> > -# - python-pip: to install filelock, flake8
> > -# - filelock: parser
> > -RUN mkdir -p /usr/share/man/man1
> > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install
> \
> > - python-lxml python-simplejson python-yaml python-openpyxl \
> > - python-requests python-reportlab python-parsedatetime \
> > - python-pexpect python-pip python-setuptools python-wheel
> > -RUN pip install filelock
> > -
> > -# Fuego command dependencies
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > - bash-completion iproute2
> > -
> > -# Default SDK for testing locally or on an x86 board
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - build-essential cmake bison flex automake kmod libtool \
> > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > -
> > -# Default test host dependencies
> > -# - iperf iperf3 netperf: used as servers on the host
> > -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > -# - python-matplotlib: Benchmark.iperf3 parser
> > -# - python-xmltodict: AGL tests
> > -# - flake8: Functional.fuego_lint
> > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > -# - iputils-ping - for /bin/ping command, used by some tests
> > -# FIXTHIS: install dependencies dynamically on the tests that need them
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > - netpipe-tcp iputils-ping
> > -RUN pip install flake8
> > -
> > -# miscelaneous packages:
> > -# python-serial - used by serio
> > -# diffstat and vim - used by Tim
> > -# time - useful for timing command duration
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - python-serial \
> > - diffstat \
> > - vim \
> > - time
> > -
> > -# FIXTHIS: determine if these tools are really necessary
> > -# RUN apt-get -q=2 -V --no-install-recommends install \
> > -# apt-utils python-paramiko \
> > -# xmlstarlet rsync \
> > -# inotify-tools gettext netpipe-tcp \
> > -# at minicom lzop bsdmainutils \
> > -# mc netcat openssh-server
> > -
> > -# bsdmainutils provides hexdump, which is needed for netperf test
> > -RUN apt-get -q=2 -V --no-install-recommends install \
> > - bsdmainutils
> > -
> > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections
> ; dpkg-reconfigure dash'
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e
> "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment;
> fi
> > -
> > -#
> ================================================================
> ==============
> > -# get ttc script and helpers
> > -#
> ================================================================
> ==============
> > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> > -# as of ttc version 2.0.2, this should not be needed
> > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> > -
> > -#
> ================================================================
> ==============
> > -# Serial Config
> > -#
> ================================================================
> ==============
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s
> > /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> /usr/local/bin/sersh'
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > -
> > -#
> ================================================================
> ==============
> > -# fserver
> > -#
> ================================================================
> ==============
> > -
> > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git
> /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server'
> > -
> > -#
> ================================================================
> ==============
> > -# ftc post installation
> > -#
> ================================================================
> ==============
> > -
> > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> > -
> > -#
> ================================================================
> ==============
> > -# Lava
> > -#
> ================================================================
> ==============
> > -
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > -
> > -#
> ================================================================
> ==============
> > -# Setup startup command
> > -#
> ================================================================
> ==============
> > -
> > -# FIXTHIS: when running multiple Fuego containers netperf will complain
> because the port is taken
> > -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
> > diff --git a/README b/README
> > index 882d84e..ce42e1e 100644
> > --- a/README
> > +++ b/README
> > @@ -44,6 +44,29 @@ You can change the name of the docker image and the
> port used by Jenkins
> > by passing the corresponding parameters to install.sh.
> > ./install.sh fuego-8082 8082
> >
> > +To run fuego on different distribution versions like Debian(stretch, buster,
> etc.)
> > + pass appropriate arguments to the install.sh script
> > + ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
>
> should probably be:
> ./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
Thanks for suggestion, may be if I remove the distribution then it will be like this
./install.sh [[<image name>] [[<Jenkins port>] [<suite>]]]
> > +
> > +By default install.sh script will install fuego on Debian stretch version
> > + to install fuego on Debian buster version pass below options
> > + ./install.sh fuego_buster 8090 debian buster
> > +
> > +[Note] It is currently verified on Debian versions (stretch and buster),
> > +and yet to provide install scripts for other distributions
> > +
> > +Installing Fuego to a host system
> > +-------------------------------------------------------------------------------
> > +Run install-fuego.sh to install fuego on host system, by default it assume the
> > +host system is Debian stretch
> > + ./install-fuego.sh
> > +
> > + to install on Debian buster version
> > + ./install-fuego.sh -d debian -s buster -p 8090
> > +
> > + to install without jenkins pass --nojenkins option to the script
> > + ./install-fuego.sh --nojenkins
> > +
> > Running
> > -------------------------------------------------------------------------------
> > To start your Fuego container, issue the following command top directory:
> > @@ -85,9 +108,9 @@ Updating the Jenkins version
> > -------------------------------------------------------------------------------
> >
> > Modify the version and checksum
> > - $ vi Dockerfile
> > - ARG JENKINS_VERSION=2.32.1
> > - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> > + $ vi install-debian.sh
> > + JENKINS_VERSION=2.32.1
> > + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> >
> > Re-build the flot plugin
> > $ sudo apt-get install maven
> > diff --git a/entrypoint.sh b/entrypoint.sh
> > new file mode 100755
> > index 0000000..6eac895
> > --- /dev/null
> > +++ b/entrypoint.sh
> > @@ -0,0 +1,7 @@
> > +#!/bin/bash
> > +
> > +[ "$1" != "--nojenkins" ] && service jenkins start
> > +
> > +#TODO: netperf not present in buster, remove this condition once
> alternative found
> > +[ -f /etc/init.d/netperf ] && service netperf start
> > +iperf3 -V -s -D -f M
> > diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> scripts/docker-build-image.sh
> > index 5823357..6889c47 100755
> > --- a/fuego-host-scripts/docker-build-image.sh
> > +++ b/fuego-host-scripts/docker-build-image.sh
> > @@ -2,9 +2,10 @@
> > # $1 - name for the docker image (default: fuego)
> > # $2 - port for jenkins (default: 8090)
> > # $3 - Dockerfile or Dockerfile.nojenkins
> > +# $4 - Debian code name on which to install (default: stretch)
> > #
> > # Example:
> > -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082
> Dockerfile.nojenkins
> > +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082
> debian stretch --nojenkins
> > #
> > if [ "$1" = "--no-cache" ]; then
> > NO_CACHE=--no-cache
> > @@ -13,7 +14,10 @@ fi
> >
> > DOCKERIMAGE=${1:-fuego}
> > JENKINS_PORT=${2:-8090}
> > -DOCKERFILE=${3:-Dockerfile}
> > +DISTRIBUTION=${3:-debian}
> > +SUITE=${4:-stretch}
> > +NO_JENKINS=${5:""}
> > +DOCKERFILE="Dockerfile"
> >
> > if [ "$(id -u)" == "0" ]; then
> > JENKINS_UID=$(id -u $SUDO_USER)
> > @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
> >
> > sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg
> HTTP_PROXY=$http_proxy \
> > --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
> > + --build-arg DIST=$DISTRIBUTION \
> > + --build-arg SUITE=$SUITE \
> > + --build-arg NO_JENKINS=$NO_JENKINS \
> > --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
> > diff --git a/install-buster.sh b/install-buster.sh
> > new file mode 100755
> > index 0000000..f4ce65b
> > --- /dev/null
> > +++ b/install-buster.sh
> > @@ -0,0 +1,294 @@
> > +#!/bin/bash
> > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > +#
> > +# Usage:
> > +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> > +#
> > +usage(){
> > +
> > +cat <<HERE
> > +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> > +
> > +Install fuego on the host Debian filesystem.
> > +
> > +options:
> > + --help Show usage help
> > + --nojenkins Install Fuego without Jenkins
> > +HERE
> > +}
> > +
> > +if [[ $EUID -ne 0 ]]; then
> > + echo "Sorry, you need root permissions"
> > + exit 1
> > +fi
> > +
> > +nojenkins=0
> > +
> > +POSITIONAL=()
> > +while [[ $# -gt 0 ]] ; do
> > + case $1 in
> > + --nojenkins)
> > + nojenkins=1
> > + shift
> > + ;;
> > + --help | -h)
> > + usage
> > + exit 0
> > + ;;
> > + *)
> > + POSITIONAL+=("$1") # save argument for later
> > + shift
> > + esac
> > +done
> > +set -- "${POSITIONAL[@]}" # restore positional arguments
> > +
> > +port=${2:-8090}
> > +
> > +#
> ================================================================
> ==============
> > +# Install Fuego dependencies
> > +#
> ================================================================
> ==============
> > +
> > +mkdir -p /usr/share/man/man1
> > +
> > +# netperf is in non-free
> > +echo deb http://deb.debian.org/debian buster main non-free >
> /etc/apt/sources.list
> > +echo deb http://security.debian.org/debian-security buster/updates main >>
> /etc/apt/sources.list
> > +
> > +apt-get update -q=2
> > +
> > +# Fuego python dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-lxml python-simplejson python-yaml python-openpyxl \
> > + python-requests python-reportlab python-parsedatetime \
> > + python-pexpect python-pip python-setuptools python-wheel
> > +pip install filelock
> > +
>
> Can we put this stuff that is common between install-buster.sh and install-
> stretch.sh
> into an "install-debian-common.sh"?
Yes, I will change this in my next patch.
Thanks for suggesting it.
>
> > +# Fuego command dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > + bash-completion iproute2
> > +
> > +# Default SDK for testing locally or on an x86 board
> > +apt-get -q=2 -V --no-install-recommends install \
> > + build-essential cmake bison flex kmod automake libtool \
> > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > +
> > +# Default test host dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> > + netpipe-tcp iputils-ping
> > +pip install flake8
> > +
> > +# miscelaneous packages:
> > +# python-serial - used by serio
> > +# diffstat and vim - used by Tim
> > +# time - useful for timing command duration
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-serial \
> > + diffstat \
> > + vim \
> > + time
> > +
> > +# install packages used by NuttX SDK
> > +apt-get -q=2 -V --no-install-recommends install \
> > + genromfs
> > +
> > +# bsdmainutils provides hexdump, which is needed for netperf test
> > +#apt-get -q=2 -V --no-install-recommends install \
> > +# bsdmainutils
> > +
> > +echo "dash dash/sh boolean false" | debconf-set-selections
> > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > +
> > +# TODO: check if proxy settings can set outside docker container
> > +if [ -n "$http_proxy" ]; then
> > + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >>
> /etc/environment
> > +fi
>
> This doesn't seem specific to Debian (/etc/wgetrc and /etc/environment).
> Should this be moved to an 'install-common.sh' script?
Yes, this one too I will do in my next work.
>
> > +
> > +#
> ================================================================
> ==============
> > +# Install Jenkins
> > +#
> ================================================================
> ==============
> > +
> > +if [ $nojenkins -eq 0 ]; then
> > + #JENKINS_VERSION=2.164.2
> > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > + #JENKINS_VERSION=2.249.3
> > + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > + JENKINS_VERSION=2.164.3
> > + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
> > + JENKINS_URL=https://pkg.jenkins.io/debian-
> stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > + export JENKINS_UC=https://updates.jenkins.io
> > + REF=/var/lib/jenkins/plugins
> > + JENKINS_HOME=/var/lib/jenkins
> > + JENKINS_PORT=$port
> > +
> > + # Jenkins dependencies
> > + apt-get -q=2 -V --no-install-recommends install \
> > + default-jdk daemon psmisc adduser procps unzip
> > + pip install python-jenkins==1.4.0
> > +
> > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > + export JENKINS_PORT=$port
> > +
> > + groupadd jenkins
> > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash
> jenkins
> > +
> > + wget -nv ${JENKINS_URL}
> > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> sha1sum -c -
> > +
> > + # allow Jenkins to start and install plugins, as part of dpkg installation
> > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > + rm jenkins_${JENKINS_VERSION}_all.deb
> > +
> > + # fix for Jenkins Java version check error
> > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.*
> version
> > \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
> > + sed -i -e
> "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g"
> /etc/init.d/jenkins
> > +
> > + source /etc/default/jenkins && \
> > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > + sed -i -e
> "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> /etc/default/jenkins
> > +
> > + source /etc/default/jenkins && \
> > + JAVA_ARGS="$JAVA_ARGS -
> Djenkins.install.runSetupWizard=false -
> > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > + if [ -n "$http_proxy" ]; then \
> > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://'
> | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' |
> sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > + JAVA_ARGS="$JAVA_ARGS -
> Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > + fi && \
> > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> /etc/default/jenkins;
> > +
> > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/Jenkins
>
> The above looks like there's a lot of non-Debian-specific stuff intermingled with
> Debian-specific stuff.
> I'm not sure what other distributions are intended to be supported, but wouldn't
> the user and
> group manipulation be the same for those?
>
> > +
> > + # set up Jenkins plugins
> > + cp frontend-install/install-plugins.sh \
> > + frontend-install/jenkins-support \
> > + frontend-install/clitest \
> > + /usr/local/bin/
> > +
> > + mkdir -p $JENKINS_HOME/userContent/docs
> > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> docs.pdf
> > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + sed -i -e "s#8080#$JENKINS_PORT#g"
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +
> > + # start and stop jenkins to pre-populate some settings
> > + service jenkins start && \
> > + sleep 30 && \
> > + service jenkins stop
> > +
> > + # install other plugins from Jenkins update center
> > + # NOTE: not sure all of these are needed, but keep list
> > + # compatible with 1.2.1 release for now
> > + #/usr/local/bin/install-plugins.sh \
> > + # ant:1.9 \
> > + # antisamy-markup-formatter:1.5 \
> > + # bouncycastle-api:2.17 \
> > + # command-launcher:1.3 \
> > + # description-setter:1.10 \
> > + # display-url-api:2.3.1 \
> > + # external-monitor-job:1.7 \
> > + # greenballs:1.15 \
> > + # icon-shim:2.0.3 \
> > + # javadoc:1.5 \
> > + # jdk-tool:1.2 \
> > + # junit:1.27 \
> > + # ldap:1.20 \
> > + # mailer:1.23 \
> > + # matrix-auth:2.3 \
> > + # matrix-project:1.14 \
> > + # pam-auth:1.4 \
> > + # pegdown-formatter:1.3 \
> > + # structs:1.17 \
> > + # windows-slaves:1.4
> > +
> > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > + /usr/local/bin/install-plugins.sh \
> > + script-security \
> > + structs \
> > + workflow-step-api \
> > + workflow-api \
> > + junit \
> > + scm-api \
> > + ant \
> > + antisamy-markup-formatter \
> > + bouncycastle-api \
> > + command-launcher \
> > + description-setter \
> > + display-url-api \
> > + external-monitor-job \
> > + greenballs \
> > + icon-shim \
> > + javadoc \
> > + jdk-tool \
> > + mailer \
> > + matrix-auth \
> > + matrix-project \
> > + pam-auth \
> > + pegdown-formatter \
> > + windows-slaves
> > +
> > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> $JENKINS_HOME/plugins/flot.jpi
> > +
> > + # make the mod.js symlink well after flot is installed
> > + service jenkins start && sleep 30 && \
> > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > + ln -s /fuego-core/scripts/mod.js
> $JENKINS_HOME/plugins/flot/flot/mod.js
> > +
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +fi
> > +
> > +#
> ================================================================
> ==============
> > +# get ttc script and helpers
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > +/usr/local/src/ttc/install.sh /usr/local/bin
> > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> > +
> The above and following are definitely not unique to Debian, or a specific
> version of Debian.
> They should be put into an "install-common.sh"
>
> > +#
> ================================================================
> ==============
> > +# Serial Config
> > +#
> ================================================================
> ==============
> > +if [ $nojenkins -eq 0 ]; then
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > +else
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > +fi
> > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> name.patch \
> > + frontend-install/0002-Output-data-from-port-during-command-
> execution.patch \
> > + /tmp/
> > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-
> for-serial-device-with-in-name.patch ; \
> > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> during-command-execution.patch ;'
> > +
> > +
> > +#
> ================================================================
> ==============
> > +# get ebf script and helpers
> > +#
> ================================================================
> ==============
> > +apt-get -q=2 -V --no-install-recommends install jq
> > +git clone https://github.com/TimesysGit/board-farm-rest-api.git
> /usr/local/src/board-farm-rest-api
> > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > +chmod a+x /usr/local/bin/ebf
> > +
> > +#
> ================================================================
> ==============
> > +# fserver
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > +ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server
> > +
> > +#
> ================================================================
> ==============
> > +# ftc post installation
> > +#
> ================================================================
> ==============
> > +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > +echo ". /etc/bash_completion" >> /root/.bashrc
> > +
> > +#
> ================================================================
> ==============
> > +# Lava
> > +#
> ================================================================
> ==============
> > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > diff --git a/install-debian.sh b/install-debian.sh
> > index ccca032..2d3ffd3 100755
> > --- a/install-debian.sh
> > +++ b/install-debian.sh
> > @@ -2,102 +2,163 @@
> > # 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > #
> > # Usage:
> > -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> > +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--
> nojenkins]
> > #
> > -if [ -n "$1" ]; then
> > - if [ "$1" = "--help" -o "$1" = "-h" ]; then
> > - cat <<HERE
> > -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> > -
> > -Installs fuego on the host Debian filesystem.
> > -
> > -options:
> > - --help Show usage help
> > - --nojenkins Install Fuego without Jenkins
> > -HERE
> > - exit 0
> > - fi
> > -fi
> > +
> > +usage(){
> > + echo ""
> > + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins
> port>] [--nojenkins]"
> > + echo "Install fuego on Debian filesystem."
> > + echo ""
> > + echo "options:"
> > + echo " --help | -h Show usage help"
> > + echo " --suite | -s <suite> Specify Debian suite"
> > + echo " --port | -p <port> Jenkins port"
> > + echo " --nojenkins Install Fuego without Jenkins"
> > + echo ""
> > +}
> >
> > if [[ $EUID -ne 0 ]]; then
> > echo "Sorry, you need root permissions"
> > exit 1
> > fi
> >
> > -if [ "$1" = "--nojenkins" ]; then
> > - nojenkins=1
> > +nojenkins=0
> > +port=8090
> > +suite=stretch
> > +user=jenkins
> > +group=jenkins
> > +uid=1000
> > +gid=1000
> > +
> > +while [ "$1" != "" ]; do
> > + case $1 in
> > + -s | --suite )
> > + shift
> > + suite=$1
> > + ;;
> > + -p | --port )
> > + shift
> > + port=$1
> > + ;;
> > + -u | --uid )
> > + shift
> > + uid=$1
> > + ;;
> > + -g | --gid )
> > + shift
> > + gid=$1
> > + ;;
> > + --nojenkins )
> > + nojenkins=1
> > + ;;
> > + -h | --help )
> > + usage
> > + exit 0
> > + ;;
> > + * )
> > + break
> > + esac
> > shift
> > -else
> > - nojenkins=0
> > - port=${1:-8090}
> > -fi
> > +done
> >
> > #
> ================================================================
> ==============
> > # Install Fuego dependencies
> > #
> ================================================================
> ==============
> >
> > # netperf is in non-free
> > -echo deb http://deb.debian.org/debian stretch main non-free >
> /etc/apt/sources.list
> > -echo deb http://security.debian.org/debian-security stretch/updates main >>
> /etc/apt/sources.list
> > -
> > -apt-get update -q=2
> > +echo deb http://deb.debian.org/debian $suite main non-free >
> /etc/apt/sources.list
> > +echo deb http://security.debian.org/debian-security $suite/updates main >>
> /etc/apt/sources.list
> >
> > # Fuego python dependencies
> > -apt-get -q=2 -V --no-install-recommends install \
> > +# - python-lxml: ftc, loggen
> > +# - python-simplejson: ftc
> > +# - python-yaml: ftc
> > +# - python-openpyxl: ftc (also LTP)
> > +# - python-requests: ftc (also fuego_release_test)
> > +# - python-reportlab: ftc
> > +# - python-parsedatetime: ftc
> > +# - python-pexpect: ssh_exec (part of ttc)
> > +# - python-pip: to install filelock, flake8
> > +# - filelock: parser
> > +mkdir -p /usr/share/man/man1
> > +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> > python-lxml python-simplejson python-yaml python-openpyxl \
> > python-requests python-reportlab python-parsedatetime \
> > - python-pip
> > + python-pexpect python-pip python-setuptools python-wheel
> > pip install filelock
> >
> > # Fuego command dependencies
> > apt-get -q=2 -V --no-install-recommends install \
> > git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > - bash-completion
> > + bash-completion iproute2
> >
> > # Default SDK for testing locally or on an x86 board
> > apt-get -q=2 -V --no-install-recommends install \
> > build-essential cmake bison flex automake kmod libtool \
> > libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
> > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> >
> > # Default test host dependencies
> > +# - iperf iperf3 netperf: used as servers on the host
> > +# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > +# - python-matplotlib: Benchmark.iperf3 parser
> > +# - python-xmltodict: AGL tests
> > +# - flake8: Functional.fuego_lint
> > +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > +# - iputils-ping - for /bin/ping command, used by some tests
> > +# FIXTHIS: install dependencies dynamically on the tests that need them
> > apt-get -q=2 -V --no-install-recommends install \
> > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
> > -pip install flake8 bsdmainutils
> > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> > + netpipe-tcp iputils-ping
> >
> > -echo "dash dash/sh boolean false" | debconf-set-selections
> > -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > -if [ -n "$http_proxy" ]; then
> > - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >>
> /etc/environment
> > +#TODO: find alternate package for netperf in buster version
> > +if [ $suite != "buster" ];then
> > + apt-get -q=2 -V --no-install-recommends install netperf
> > fi
> > +pip install flake8
> >
> > -#
> ================================================================
> ==============
> > -# Clone fuego and fuego-core
> > -#
> ================================================================
> ==============
> > -if [ ! -d "/fuego" ]; then
> > - cd /
> > - git clone --branch next --depth=1
> https://bitbucket.org/nirrognas/fuego.git
> > - ln -s /fuego/fuego-ro /fuego-ro
> > - ln -s /fuego/fuego-rw /fuego-rw
> > - cd fuego
> > - git clone --branch next --depth=1
> https://bitbucket.org/nirrognas/fuego-core.git
> > - ln -s /fuego/fuego-core /fuego-core
> > -fi
> > +# miscelaneous packages:
> > +# python-serial - used by serio
> > +# diffstat and vim - used by Tim
> > +# time - useful for timing command duration
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-serial \
> > + diffstat \
> > + vim \
> > + time
> > +
> > +# install packages used by NuttX SDK
> > +apt-get -q=2 -V --no-install-recommends install \
> > + genromfs
> > +
> > +# FIXTHIS: determine if these tools are really necessary
> > +#RUN apt-get -q=2 -V --no-install-recommends install \
> > +# apt-utils python-paramiko \
> > +# xmlstarlet rsync \
> > +# inotify-tools gettext netpipe-tcp \
> > +# at minicom lzop bsdmainutils \
> > +# mc netcat openssh-server
> > +
> > +# bsdmainutils provides hexdump, which is needed for netperf test
> > +apt-get -q=2 -V --no-install-recommends install \
> > + bsdmainutils
> > +
> > +echo "dash dash/sh boolean false" | debconf-set-selections
> > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> >
> > #
> ================================================================
> ==============
> > # Install Jenkins
> > #
> ================================================================
> ==============
> > -
> > if [ $nojenkins -eq 0 ]; then
> > #JENKINS_VERSION=2.164.2
> > #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > JENKINS_VERSION=2.249.3
> > JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > JENKINS_URL=https://pkg.jenkins.io/debian-
> stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > - JENKINS_UC=https://updates.jenkins.io
> > - REF=/var/lib/jenkins/plugins
> > - JENKINS_HOME=/var/lib/jenkins
> > + export JENKINS_UC=https://updates.jenkins.io
> > + export REF=/var/lib/jenkins/plugins
> > + export JENKINS_HOME=/var/lib/jenkins
> > JENKINS_PORT=$port
> >
> > # Jenkins dependencies
> > @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
> > pip install python-jenkins==1.4.0
> >
> > echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > - groupadd jenkins
> > - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash
> jenkins
> > - cd
> > - wget ${JENKINS_URL}
> > + export JENKINS_PORT=$port
> > +
> > + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> > + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> /bin/bash ${user}
> > +
> > + wget -nv ${JENKINS_URL}
> > echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> sha1sum -c -
> >
> > # allow Jenkins to start and install plugins, as part of dpkg installation
> > @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
> > dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > rm jenkins_${JENKINS_VERSION}_all.deb
> >
> > - # update ownership
> > - chown -R jenkins:jenkins /fuego
> > - chown -R jenkins:jenkins /fuego-ro
> > - chown -R jenkins:jenkins /fuego-rw
> > - chown -R jenkins:jenkins /fuego-core
> > +fi
> > +
> > +#
> ================================================================
> ==============
> > +# get ttc script and helpers
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > +/usr/local/src/ttc/install.sh /usr/local/bin
> > +# as of ttc version 2.0.2, this should not be needed
> > +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> > +
> > +#
> ================================================================
> ==============
> > +# get ebf script and helpers
> > +#
> ================================================================
> ==============
> > +if [ $nojenkins -eq 0 ]; then
> > + apt-get -q=2 -V --no-install-recommends install jq
> > + git clone https://github.com/TimesysGit/board-farm-rest-api.git
> /usr/local/src/board-farm-rest-api
> > + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > + chmod a+x /usr/local/bin/ebf
> > +fi
> > +
> > +#
> ================================================================
> ==============
> > +# Serial Config
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/frowand/serio.git /usr/local/src/serio
> > +if [ $nojenkins -eq 0 ]; then
> > + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> name.patch \
> > + frontend-install/0002-Output-data-from-port-during-command-
> execution.patch \
> > + /tmp/
> > + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-
> serial-device-with-in-name.patch
> > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> during-command-execution.patch
> > + chown -R jenkins /usr/local/src/serio
> > +fi
> > +cp /usr/local/src/serio/serio /usr/local/bin/
> > +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> > +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> > +
> > +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
> > +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
> > +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> > +
> > +#
> ================================================================
> ==============
> > +# fserver
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > +ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server
> > +
> > +#
> ================================================================
> ==============
> > +# Jenkins post installation
> > +#
> ================================================================
> ==============
> > +if [ $nojenkins -eq 0 ]; then
> > +
> > + # Fix jenkins issue, Java version check fails when java version is 11
> > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.*
> version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-
> > \]*\\\).*\\\".*/\\\1\\\2\/p;')"
> > + sed -i -e
> "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g"
> /etc/init.d/jenkins
> >
> > source /etc/default/jenkins && \
> > JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
> >
> > sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> >
> > - cp /fuego/frontend-install/install-plugins.sh \
> > - /fuego/frontend-install/jenkins-support \
> > - /fuego/frontend-install/clitest \
> > + # set up Jenkins plugins
> > + cp frontend-install/install-plugins.sh \
> > + frontend-install/jenkins-support \
> > + frontend-install/clitest \
> > /usr/local/bin/
> >
> > - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
> > - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > - mkdir $JENKINS_HOME/userContent/docs
> > - cp /fuego/docs/fuego-docs.pdf
> $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> > - jenkins cp /fuego/frontend-
> install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > - sed -i -e "s#8080#$JENKINS_PORT#g"
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > - chown -R jenkins:jenkins $JENKINS_HOME/
> > -
> > # start and stop jenkins to pre-populate some settings
> > service jenkins start && \
> > sleep 30 && \
> > service jenkins stop
> >
> > + # install plugins - these versions are for Jenkins version 2.164.2
> > + # Explicitly install script-security v1.68, otherwise
> > + # v1.74 will automatically be installed as a dependency of
> > + # the junit plugin. Make sure to install before junit plugin.
> > + # Explicitly install junit:1.27
> > +
> > # install other plugins from Jenkins update center
> > # NOTE: not sure all of these are needed, but keep list
> > - # compatible with 1.2.1 release for now
> > - #/usr/local/bin/install-plugins.sh \
> > - # ant:1.9 \
> > - # antisamy-markup-formatter:1.5 \
> > - # bouncycastle-api:2.17 \
> > - # command-launcher:1.3 \
> > - # description-setter:1.10 \
> > - # display-url-api:2.3.1 \
> > - # external-monitor-job:1.7 \
> > - # greenballs:1.15 \
> > - # icon-shim:2.0.3 \
> > - # javadoc:1.5 \
> > - # jdk-tool:1.2 \
> > - # junit:1.27 \
> > - # ldap:1.20 \
> > - # mailer:1.23 \
> > - # matrix-auth:2.3 \
> > - # matrix-project:1.14 \
> > - # pam-auth:1.4 \
> > - # pegdown-formatter:1.3 \
> > - # structs:1.17 \
> > - # windows-slaves:1.4
> > -
> > - # install plugins for latest LTS Jenkins (currently 2.249.3)
> > - /usr/local/bin/install-plugins.sh \
> > - script-security \
> > - structs \
> > - workflow-step-api \
> > - workflow-api \
> > - junit \
> > - scm-api \
> > - ant \
> > - antisamy-markup-formatter \
> > - bouncycastle-api \
> > - command-launcher \
> > - description-setter \
> > - display-url-api \
> > - external-monitor-job \
> > - greenballs \
> > - icon-shim \
> > - javadoc \
> > - jdk-tool \
> > - mailer \
> > - matrix-auth \
> > - matrix-project \
> > - pam-auth \
> > - pegdown-formatter \
> > - windows-slaves
> > -
> > - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> $JENKINS_HOME/plugins/flot.jpi
> > + # as compatible as possible with 1.2.1 release for now
> > + # Do NOT change the order of the plugins, unless you know what
> > + # you are doing. Otherwise, the install-plugins.sh script will
> > + # install the wrong versions of other plugins based on dependency
> > + # information in the plugin files.
> >
> > + #/usr/local/bin/install-plugins.sh \
> > + # script-security:1.68 \
> > + # structs:1.20 \
> > + # workflow-step-api:2.22 \
> > + # workflow-api:2.40 \
> > + # junit:1.27 \
> > + # scm-api:2.6.3 \
> > + # ant:1.9 \
> > + # antisamy-markup-formatter:1.5 \
> > + # bouncycastle-api:2.17 \
> > + # command-launcher:1.3 \
> > + # description-setter:1.10 \
> > + # display-url-api:2.3.1 \
> > + # external-monitor-job:1.7 \
> > + # greenballs:1.15 \
> > + # icon-shim:2.0.3 \
> > + # javadoc:1.5 \
> > + # jdk-tool:1.2 \
> > + # ldap:1.20 \
> > + # mailer:1.23 \
> > + # matrix-auth:2.3 \
> > + # matrix-project:1.14 \
> > + # pam-auth:1.5 \
> > + # pegdown-formatter:1.3 \
> > + # windows-slaves:1.4
> > +
> > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > + /usr/local/bin/install-plugins.sh \
> > + script-security \
> > + structs \
> > + workflow-step-api \
> > + workflow-api \
> > + junit \
> > + scm-api \
> > + ant \
> > + antisamy-markup-formatter \
> > + bouncycastle-api \
> > + command-launcher \
> > + description-setter \
> > + display-url-api \
> > + external-monitor-job \
> > + greenballs \
> > + icon-shim \
> > + javadoc \
> > + jdk-tool \
> > + mailer \
> > + matrix-auth \
> > + matrix-project \
> > + pam-auth \
> > + pegdown-formatter \
> > + windows-slaves
> > +
> > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> $JENKINS_HOME/plugins/flot.jpi
> > +
> > + # jenkins should automatically unzip any plugins in the plugin dir
> > # make the mod.js symlink well after flot is installed
> > service jenkins start && sleep 30 && \
> > rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > ln -s /fuego-core/scripts/mod.js
> $JENKINS_HOME/plugins/flot/flot/mod.js
> >
> > - chown -R jenkins:jenkins $JENKINS_HOME/
> > -else
> > - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-
> ro/conf/fuego.conf
> > -fi
> > + mkdir -p $JENKINS_HOME/userContent/docs
> > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> docs.pdf
> >
> > -#
> ================================================================
> ==============
> > -# get ttc script and helpers
> > -#
> ================================================================
> ==============
> > -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > -/usr/local/src/ttc/install.sh /usr/local/bin
> > -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + sed -i -e "s#8080#$JENKINS_PORT#g"
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> >
> > -#
> ================================================================
> ==============
> > -# Serial Config
> > -#
> ================================================================
> ==============
> > -if [ $nojenkins -eq 0 ]; then
> > - /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > -else
> > - /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> /usr/local/bin/sersh'
> > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > fi
> >
> > -#
> ================================================================
> ==============
> > -# fserver
> > -#
> ================================================================
> ==============
> > -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > -ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server
> > -
> > #
> ================================================================
> ==============
> > # ftc post installation
> > #
> ================================================================
> ==============
> > ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > echo ". /etc/bash_completion" >> /root/.bashrc
> >
> > #
> ================================================================
> ==============
> > @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
> > #
> ================================================================
> ==============
> > ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > -
> > -#
> ================================================================
> ==============
> > -# Small guide
> > -#
> ================================================================
> ==============
> > -echo "Run 'service netperf start' to start a netperf server"
> > -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > -echo "Run 'ftc list-boards' to see the available boards"
> > -echo "Run 'ftc list-tests' to see the available tests"
> > -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> world"
> > -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> Dhrystone"
> > -echo "Run 'ftc gen-report' to get results"
> > diff --git a/install-fuego.sh b/install-fuego.sh
> > new file mode 100755
> > index 0000000..7b3e010
> > --- /dev/null
> > +++ b/install-fuego.sh
> > @@ -0,0 +1,109 @@
> > +#!/bin/bash
> > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > +#
> > +# Usage:
> > +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p
> <jenkins port>] [--nojenkins]
> > +#
> > +
> > +usage(){
> > + echo ""
> > + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s
> <suite>] [-p <jenkins port>] [--nojenkins]"
> > + echo "Install fuego on Debian filesystem."
> > + echo ""
> > + echo "options:"
> > + echo " --help | -h Show usage help"
> > + echo " --dist | -d <distribution> Specify distribution"
> > + echo " --suite | -s <suite> Specify suite"
> > + echo " --port | -p <port> Jenkins port"
> > + echo " --nojenkins Install Fuego without Jenkins"
> > + echo ""
> > +}
> > +
> > +if [[ $EUID -ne 0 ]]; then
> > + echo "Sorry, you need root permissions"
> > + exit 1
> > +fi
> > +
> > +nojenkins=
> > +port=8090
> > +dist=debian
> > +suite=stretch
> > +fuego_dir=$(pwd)
> > +
> > +while [ "$1" != "" ]; do
> > + case $1 in
> > + -d | --dist )
> > + shift
> > + dist=$1
> > + ;;
> > + -s | --suite )
> > + shift
> > + suite=$1
> > + ;;
> > + -p | --port )
> > + shift
> > + port=$1
> > + ;;
> > + --nojenkins )
> > + nojenkins="--nojenkins"
> > + ;;
> > + -h | --help )
> > + usage
> > + exit 0
> > + ;;
> > + * )
> > + break
> > + esac
> > + shift
> > +done
> > +
> > +#
> ================================================================
> ==============
> > +# Clone fuego-core
> > +#
> ================================================================
> ==============
> > +if [ ! -f "fuego-core/scripts/ftc" ]; then
> > + # set fuego-core branch to same as current fuego branch
> > + # get current git branch
> > + set -o noglob
> > + while IFS=" " read -r part1 part2 ; do
> > + if [ $part1 = "*" ] ; then
> > + branch=$part2
> > + fi
> > + done < <(git branch)
> > + set +o noglob
> > + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
> > +fi
> > +
> > +#
> ================================================================
> ==============
> > +# Install fuego on host system
> > +#
> ================================================================
> ==============
> > +./install-$dist.sh -s $suite -p $port $nojenkins
> > +
> > +systemctl daemon-reload
> > +service jenkins restart
> > +
> > +# provide permission to current user
> > +usermod -aG jenkins $USER
> > +chmod -R 775 $fuego_dir/fuego-rw
> > +
> > +chown -R jenkins:jenkins $fuego_dir/fuego-ro
> > +chown -R jenkins:jenkins $fuego_dir/fuego-rw
> > +chown -R jenkins:jenkins $fuego_dir/fuego-core
> > +
> > +#
> ================================================================
> ==============
> > +# Create Links to fuego folders
> > +#
> ================================================================
> ==============
> > +ln -s $fuego_dir/fuego-ro /
> > +ln -s $fuego_dir/fuego-rw /
> > +ln -s $fuego_dir/fuego-core /
> > +
> > +#
> ================================================================
> ==============
> > +# Small guide
> > +#
> ================================================================
> ==============
> > +echo "Logout and Login current user session to work properly"
> > +echo "Run 'service netperf start' to start a netperf server"
> > +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > +echo "Run 'ftc list-boards' to see the available boards"
> > +echo "Run 'ftc list-tests' to see the available tests"
> > +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> world"
> > +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> Dhrystone"
> > +echo "Run 'ftc gen-report' to get results"
> > diff --git a/install-stretch.sh b/install-stretch.sh
> > new file mode 100755
> > index 0000000..4f6f724
> > --- /dev/null
> > +++ b/install-stretch.sh
> > @@ -0,0 +1,288 @@
> > +#!/bin/bash
> > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > +#
> > +# Usage:
> > +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> > +#
> > +usage(){
> > +
> > +cat <<HERE
> > +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> > +
> > +Install fuego on the host Debian filesystem.
> > +
> > +options:
> > + --help Show usage help
> > + --nojenkins Install Fuego without Jenkins
> > +HERE
> > +}
> > +
> > +if [[ $EUID -ne 0 ]]; then
> > + echo "Sorry, you need root permissions"
> > + exit 1
> > +fi
> > +
> > +nojenkins=0
> > +
> > +POSITIONAL=()
> > +while [[ $# -gt 0 ]] ; do
> > + case $1 in
> > + --nojenkins)
> > + nojenkins=1
> > + shift
> > + ;;
> > + --help | -h)
> > + usage
> > + exit 0
> > + ;;
> > + *)
> > + POSITIONAL+=("$1") # save argument for later
> > + shift
> > + esac
> > +done
> > +set -- "${POSITIONAL[@]}" # restore positional arguments
> > +
> > +port=${2:-8090}
> > +
> > +#
> ================================================================
> ==============
> > +# Install Fuego dependencies
> > +#
> ================================================================
> ==============
> > +
> > +mkdir -p /usr/share/man/man1
> > +
> > +# netperf is in non-free
> > +echo deb http://deb.debian.org/debian stretch main non-free >
> /etc/apt/sources.list
> > +echo deb http://security.debian.org/debian-security stretch/updates main >>
> /etc/apt/sources.list
> > +
> > +apt-get update -q=2
> > +
> > +# Fuego python dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-lxml python-simplejson python-yaml python-openpyxl \
> > + python-requests python-reportlab python-parsedatetime \
> > + python-pexpect python-pip python-setuptools python-wheel
> > +pip install filelock
> > +
> > +# Fuego command dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > + bash-completion iproute2
> > +
> > +# Default SDK for testing locally or on an x86 board
> > +apt-get -q=2 -V --no-install-recommends install \
> > + build-essential cmake bison flex kmod automake libtool \
> > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > +
> > +# Default test host dependencies
> > +apt-get -q=2 -V --no-install-recommends install \
> > + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > + netpipe-tcp iputils-ping
> > +pip install flake8
> > +
> > +# miscelaneous packages:
> > +# python-serial - used by serio
> > +# diffstat and vim - used by Tim
> > +# time - useful for timing command duration
> > +apt-get -q=2 -V --no-install-recommends install \
> > + python-serial \
> > + diffstat \
> > + vim \
> > + time
> > +
> > +# install packages used by NuttX SDK
> > +apt-get -q=2 -V --no-install-recommends install \
> > + genromfs
> > +
> > +# bsdmainutils provides hexdump, which is needed for netperf test
> > +apt-get -q=2 -V --no-install-recommends install \
> > + bsdmainutils
> > +
> > +echo "dash dash/sh boolean false" | debconf-set-selections
> > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > +
> > +# TODO: check if proxy settings can set outside docker container
> > +#if [ -n "$http_proxy" ]; then
> > +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >>
> /etc/environment
> > +#fi
> > +
> > +#
> ================================================================
> ==============
> > +# Install Jenkins
> > +#
> ================================================================
> ==============
> > +
> > +if [ $nojenkins -eq 0 ]; then
> > + #JENKINS_VERSION=2.164.2
> > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > + JENKINS_VERSION=2.249.3
> > + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > + JENKINS_URL=https://pkg.jenkins.io/debian-
> stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > + export JENKINS_UC=https://updates.jenkins.io
> > + REF=/var/lib/jenkins/plugins
> > + JENKINS_HOME=/var/lib/jenkins
> > + JENKINS_PORT=$port
> > +
> > + # Jenkins dependencies
> > + apt-get -q=2 -V --no-install-recommends install \
> > + default-jdk daemon psmisc adduser procps unzip
> > + pip install python-jenkins==1.4.0
> > +
> > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > + export JENKINS_PORT=$port
> > +
> > + groupadd jenkins
> > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash
> jenkins
> > +
> > + wget -nv ${JENKINS_URL}
> > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> sha1sum -c -
> > +
> > + # allow Jenkins to start and install plugins, as part of dpkg installation
> > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > + rm jenkins_${JENKINS_VERSION}_all.deb
> > +
> > + source /etc/default/jenkins && \
> > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > + sed -i -e
> "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> /etc/default/jenkins
> > +
> > + source /etc/default/jenkins && \
> > + JAVA_ARGS="$JAVA_ARGS -
> Djenkins.install.runSetupWizard=false -
> > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > + if [ -n "$http_proxy" ]; then \
> > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://'
> | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' |
> sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > + JAVA_ARGS="$JAVA_ARGS -
> Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > + fi && \
> > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> /etc/default/jenkins;
> > +
> > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > +
> > + # set up Jenkins plugins
> > + cp frontend-install/install-plugins.sh \
> > + frontend-install/jenkins-support \
> > + frontend-install/clitest \
> > + /usr/local/bin/
> > +
> > + mkdir -p $JENKINS_HOME/userContent/docs
> > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> docs.pdf
> > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + sed -i -e "s#8080#$JENKINS_PORT#g"
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +
> > + # start and stop jenkins to pre-populate some settings
> > + service jenkins start && \
> > + sleep 30 && \
> > + service jenkins stop
> > +
> > + # install other plugins from Jenkins update center
> > + # NOTE: not sure all of these are needed, but keep list
> > + # compatible with 1.2.1 release for now
> > + #/usr/local/bin/install-plugins.sh \
> > + # ant:1.9 \
> > + # antisamy-markup-formatter:1.5 \
> > + # bouncycastle-api:2.17 \
> > + # command-launcher:1.3 \
> > + # description-setter:1.10 \
> > + # display-url-api:2.3.1 \
> > + # external-monitor-job:1.7 \
> > + # greenballs:1.15 \
> > + # icon-shim:2.0.3 \
> > + # javadoc:1.5 \
> > + # jdk-tool:1.2 \
> > + # junit:1.27 \
> > + # ldap:1.20 \
> > + # mailer:1.23 \
> > + # matrix-auth:2.3 \
> > + # matrix-project:1.14 \
> > + # pam-auth:1.4 \
> > + # pegdown-formatter:1.3 \
> > + # structs:1.17 \
> > + # windows-slaves:1.4
> > +
> > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > + /usr/local/bin/install-plugins.sh \
> > + script-security \
> > + structs \
> > + workflow-step-api \
> > + workflow-api \
> > + junit \
> > + scm-api \
> > + ant \
> > + antisamy-markup-formatter \
> > + bouncycastle-api \
> > + command-launcher \
> > + description-setter \
> > + display-url-api \
> > + external-monitor-job \
> > + greenballs \
> > + icon-shim \
> > + javadoc \
> > + jdk-tool \
> > + mailer \
> > + matrix-auth \
> > + matrix-project \
> > + pam-auth \
> > + pegdown-formatter \
> > + windows-slaves
> > +
> > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> $JENKINS_HOME/plugins/flot.jpi
> > +
> > + # make the mod.js symlink well after flot is installed
> > + service jenkins start && sleep 30 && \
> > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > + ln -s /fuego-core/scripts/mod.js
> $JENKINS_HOME/plugins/flot/flot/mod.js
> > +
> > + chown -R jenkins:jenkins $JENKINS_HOME/
> > +fi
> > +
> > +#
> ================================================================
> ==============
> > +# get ttc script and helpers
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > +/usr/local/src/ttc/install.sh /usr/local/bin
> > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> > +
> > +#
> ================================================================
> ==============
> > +# Serial Config
> > +#
> ================================================================
> ==============
> > +if [ $nojenkins -eq 0 ]; then
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > +else
> > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> /usr/local/bin/sersh'
> > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > /usr/local/bin'
> > +fi
> > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> name.patch \
> > + frontend-install/0002-Output-data-from-port-during-command-
> execution.patch \
> > + /tmp/
> > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-
> for-serial-device-with-in-name.patch ; \
> > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> during-command-execution.patch ;'
> > +
> > +
> > +#
> ================================================================
> ==============
> > +# get ebf script and helpers
> > +#
> ================================================================
> ==============
> > +apt-get -q=2 -V --no-install-recommends install jq
> > +git clone https://github.com/TimesysGit/board-farm-rest-api.git
> /usr/local/src/board-farm-rest-api
> > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > +chmod a+x /usr/local/bin/ebf
> > +
> > +#
> ================================================================
> ==============
> > +# fserver
> > +#
> ================================================================
> ==============
> > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > +ln -s /usr/local/lib/fserver/start_local_bg_server
> /usr/local/bin/start_local_bg_server
> > +
> > +#
> ================================================================
> ==============
> > +# ftc post installation
> > +#
> ================================================================
> ==============
> > +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > +echo ". /etc/bash_completion" >> /root/.bashrc
> > +
> > +#
> ================================================================
> ==============
> > +# Lava
> > +#
> ================================================================
> ==============
> > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > diff --git a/install.sh b/install.sh
> > index a5a90b0..259c887 100755
> > --- a/install.sh
> > +++ b/install.sh
> > @@ -1,13 +1,13 @@
> > #!/bin/bash
> > #
> > -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> [<port>]
> > +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> [<port>] [<distribution>] [<suit>]
> > #
> >
> > # $1 is the exit code after usage is shown
> > usage() {
> > exit_code="$1"
> > cat <<HERE
> > -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> [<port>]
> > +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> [<port>] [<distribution>] [<suit>]
> >
> > Create the docker image and container with the Fuego test distribution.
> > If no <image_name> is provided, the image will be named 'fuego'.
> > @@ -34,7 +34,7 @@ fi
> >
> > priv=0
> > NOCACHE=""
> > -dockerfile="Dockerfile"
> > +NOJENKINS=""
> >
> > POSITIONAL=()
> > while [[ $# -gt 0 ]] ; do
> > @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
> > shift
> > ;;
> > --nojenkins)
> > - dockerfile="Dockerfile.nojenkins"
> > + NOJENKINS="--nojenkins"
> > shift
> > ;;
> > *)
> > @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional
> arguments
> >
> > image_name=${1:-fuego}
> > jenkins_port=${2:-8090}
> > +distribution=${3:-debian}
> > +suite=${4:-stretch}
> >
> > container_name="${image_name}-container"
> >
> > @@ -79,7 +81,7 @@ fi
> >
> > set -e
> >
> > -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name}
> ${jenkins_port} ${dockerfile}
> > +source fuego-host-scripts/docker-build-image.sh $NOCACHE
> ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
> > if [ "$priv" == "0" ]; then
> > fuego-host-scripts/docker-create-container.sh ${image_name}
> ${container_name}
> > else
> > diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
> > new file mode 100644
> > index 0000000..b899e7c
> > --- /dev/null
> > +++ b/uninstall-fuego.sh
> > @@ -0,0 +1,47 @@
> > +#!/bin/bash
> > +
> > +# Remove Jenkins plugins
> > +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf
> /usr/share/jenkins/ref/plugins
> > +
> > +# uninstall Jenkins
> > +service jenkins stop
> > +dpkg -P jenkins
> > +
> > +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-
> plugins.sh
> > +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
> > +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
> > +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
> > +sed -i '/^JENKINS_PORT=/d' /etc/environment
> > +
> > +# uninstall ttc script
> > +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
> > +
> > +# uninstall ebf script
> > +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-
> farm-rest-api
> > +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
> > +
> > +# uninstall serial config
> > +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
> > +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
> > +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
> > +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
> > +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
> > +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
> > +
> > +# uninstall fserver
> > +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
> > +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf
> /usr/local/bin/start_local_bg_server
> > +
> > +# uninstall ftc
> > +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
> > +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
> > +
> > +# remove lava target files
> > +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-
> lava-target-setup
> > +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf
> /usr/local/bin/fuego-lava-target-teardown
> > +
> > +# remove links
> > +[ -d /fuego-ro ] && rm /fuego-ro
> > +[ -d /fuego-rw ] && rm /fuego-rw
> > +[ -d /fuego-core ] && rm /fuego-core
> > +
> > --
> > 2.20.1
> >
>
> Overall - this looks like a good start. But I'd like to see the duplicate sections of
> install-buster.sh
> and install-stretch.sh put into a single location "install-debian-common.sh".
> Also, I'd like to see
> the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
>
> Can you please re-do this with that refactoring in mind?
Thank you for the feedback, I will work on these refactoring and send you another patch.
>
> Thanks,
> -- Tim
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Fuego] Use common script to install fuego on host or docker system
2021-06-17 7:01 ` Venkata.Pyla
@ 2021-06-17 20:53 ` Tim.Bird
0 siblings, 0 replies; 8+ messages in thread
From: Tim.Bird @ 2021-06-17 20:53 UTC (permalink / raw)
To: Venkata.Pyla, tbird20d; +Cc: fuego
> -----Original Message-----
> From: Venkata.Pyla@toshiba-tsip.com <Venkata.Pyla@toshiba-tsip.com>
>
> Hi Tim,
>
> Thanks for the feedback, I have commented in line.
>
> I also copied your feedback and my answers here as the mail is longer to find them, also you can see in the inline in the below mail to see
> the context.
Sounds good.
>
> > If we don't support any other distributions besides Debian at the moment,
> > then I'm not sure we need the distribution argument. Do you have any
> > non-debian distribution install scripts ready to submit soon?
> No I don’t have any plan for other distributions coming soon, I kept the distribution argument
> Because in case if we have any future plan to support other distributions we can easily adapt.
> I think we can remove the distribution argument if we don’t have any plan to support in near future,
> I will remove this argument in my next patch if you have no other opinion.
Let's remove it.
>
> .
> .
> .
> > Is install-debian.sh the common parts for a Debian installation, with
> > buster and stretch having the parts that are unique to each of
> > those distros respectively? If we're going to refactor this, I
> > want to eliminate any duplication, so that package names are
> > not used in multiple places. This has been a headache to
> > maintain.
> >
> > It appears there is lots of duplication between install-buster.sh and
> > install-stretch.sh. Can we possible put this duplication into an
> > "install-debian-common.sh", and reserve install-buster.sh and
> > install-stretch.sh for only the parts unique to those distros?
> Yes, that is a good idea, even I got the similar feedback from our internal review,
> I will do this refactoring in my next patch.
>
> .
> .
> .
> > Ooo - I like the sound of this uninstall script.
> Sorry do you have anything in mind, not required or required this uninstall script?
> I have created it because it helps me in testing the for uninstalling and installing the fuego multiple times.
> The uninstall script may also need lot of improvement, but I just created a basic script to solve my current problem.
> I may further improve based on the feedback or the issues if any.
I don't have any additional input. I just like the overall idea.
It would normally only be useful for end users doing a native (not containerized) install.
> .
> > One question about this overall approach. Does this retain the ability to cache
> > operations, so that if something goes wrong with an install, you don't have to
> > start over from scratch. When building with docker, with multiple steps, it
> > caches the image at each step. When rebuilding, or when an error occurs,
> > this can save a lot of time. It also can introduce subtle bugs, but that's
> > a separate issue.
> >
> > This setup seems like it performs the bulk of the install as a single docker
> > step, is that right?
> Yes, it will not create the cache for each step in the script, this will be a
> disadvantage with this common script approach.
>
How about having the Docker file call the separate scripts, instead of the scripts
calling each other? This is kind of a middle ground.
The different steps would be:
- install-debian-common.sh - all the common Debian packages
- install-buster.sh - the buster-specific Debian packages
or
- install-stretch.sh - the stretch-specific Debian packages
- install-extras.sh (for ttc, serio, ebf(board farm rest api), labcontrol, fserver, and anything else we get directly from git)
- install-jenkins.sh (stuff for Jenkins, called if --nojenkins was not specified)
There could be a simple script:
install-native.sh that would mirror the 4 steps in the Docker file.
and install-container.sh could do the docker-create, etc.
> .
> .
> > I'm not sure what this is doing. It looks scary. Please explain.
> Because, I had COPY the fuego current directory to the Docker image for installation purpose and removing it after installation,
> and this is required because the common script "install-*.sh" requires some files to copy inside docker to a specified location
> for e.g:
> cp frontend-install/install-plugins.sh \
> frontend-install/jenkins-support \
> frontend-install/clitest \
> /usr/local/bin/
>
> And in the previous approach this was done using COPY command which will directly copy from host machine to docker location.
>
> May be I could have used mounting directory inside docker workdir to avoid copying and later removing it, maybe I faced some problem
> with this approach and so later chosen to copy it inside docker image.
> I have done this work little long ago so I have not remembered why I have chosen the copy it inside docker approach, Let me work on again
> and understand why have done this and change it to mounting directory instead copy and remove.
> Please let me know if you have other ideas.
OK - that's fine. I just missed that.
>
>
> .
> .
> .
> > should probably be:
> > ./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
> Thanks for suggestion,
> maybe if I remove the distribution it would be like this?
> ./install.sh [[<image name>] [[<Jenkins port>] [<suite>]]]
Sounds good.
>
>
> .
> .
> .
> > Overall - this looks like a good start. But I'd like to see the duplicate sections of
> > install-buster.sh
> > and install-stretch.sh put into a single location "install-debian-common.sh".
> > Also, I'd like to see
> > the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
> >
> > Can you please re-do this with that refactoring in mind?
> Thank you for the feedback, I will work on these refactoring and send you another patch.
Sounds good. Thanks,
-- Tim
>
> Thanks
> Venkata.
>
>
> > -----Original Message-----
> > From: Tim.Bird@sony.com <Tim.Bird@sony.com>
> > Sent: 15 June 2021 05:39
> > To: pyla venkata(ＴＳＩＰ) <Venkata.Pyla@toshiba-tsip.com>;
> > tbird20d@yahoo.com
> > Cc: fuego@lists.linuxfoundation.org; sangorrin daniel(サンゴリン ダニエル □ＳＷ
> > Ｃ◯ＡＣＴ) <daniel.sangorrin@toshiba.co.jp>
> > Subject: RE: [Fuego] Use common script to install fuego on host or docker
> > system
> >
> > Please see comments inline below.
> >
> > I tried to apply this patch from the e-mail list using 'git am', but got a failure.
> > That made it a hard to experiment with and view the changes in
> > context.
> >
> > What version of the code is this patch based on?
> >
> > > -----Original Message-----
> > > From: venkata.pyla@toshiba-tsip.com <venkata.pyla@toshiba-tsip.com>
> > > Subject: [Fuego] Use common script to install fuego on host or docker system
> > >
> > > From: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > >
> > > Modified the install scripts to use common scripts for both types of
> > > installation, this way it will help to make sure both the environments
> > > are same and if any future modification will have single place of change.
> > >
> > > Now, for installing fuego on docker system the 'install.sh' script will
> > > use common script in Dockefile to install fuego related stuff.
> > >
> > > sameway for installing fuego on host system it can use the same common
> > > script to install fuego related stuff.
> > >
> > > Below are the common scripts that are used in both the installation types
> > > for each debian distribution
> > > * install-stretch.sh
> > > * install-buster.sh
> > >
> > > so finally,
> > > To install fuego on docker system use the below script with options
> > > $ ./install.sh <options> image_name <port> <distribution> <suite>
> > > e.g: ./install.sh debian_stretch 8090 debian stretch
> > > or
> > > ./install.sh debian_buster 8090 debian buster
> > > or
> > > ./install.sh --nojenkins debian_buster_nojenkins 8090 debian buster
> > This is a bit wordy, but OK.
> >
> > If we don't support any other distributions besides Debian at the moment,
> > then I'm not sure we need the distribution argument. Do you have any
> > non-debian distribution install scripts ready to submit soon?
> No I don’t have any plan for other distributions coming soon, I kept the distribution argument
> Because in case if we have any future plan to support other distributions we can easily adapt.
> I think we can remove the distribution argument if we don’t have any plan to support.
> I will remove this argument in my next patch, please let me know if you have any other opinion.
>
> >
> > I'm not sure I follow calling the Debian release name the "<suite>".
> >
> > >
> > > To install fuego on host system use the below script with options
> > > $ ./install-fuego.sh <options> <-d distribution> <-s suite> <-p port>
> > "suite" seems like an odd name. Should this be distro "version" or "release"?
> >
> > > e.g: ./install-fuego.sh -d debian -s stretch -p 8090
> > > or
> > > ./install-fuego.sh -d debian -s buster -p 8090
> > > or
> > > ./install-fuego.sh -d debian -s buster -p 8090 --nojenkins
> > >
> > > Signed-off-by: venkata pyla <venkata.pyla@toshiba-tsip.com>
> > > ---
> > > Dockerfile | 303 ++---------------
> > > Dockerfile.nojenkins | 151 ---------
> > > README | 29 +-
> > > entrypoint.sh | 7 +
> > > fuego-host-scripts/docker-build-image.sh | 11 +-
> > > install-buster.sh | 294 ++++++++++++++++
> > > install-debian.sh | 410 ++++++++++++++---------
> > > install-fuego.sh | 109 ++++++
> > > install-stretch.sh | 288 ++++++++++++++++
> >
> > Is install-debian.sh the common parts for a Debian installation, with
> > buster and stretch having the parts that are unique to each of
> > those distros respectively? If we're going to refactor this, I
> > want to eliminate any duplication, so that package names are
> > not used in multiple places. This has been a headache to
> > maintain.
> >
> > It appears there is lots of duplication between install-buster.sh and
> > install-stretch.sh. Can we possible put this duplication into an
> > "install-debian-common.sh", and reserve install-buster.sh and
> > install-stretch.sh for only the parts unique to those distros?
> Yes, that is a good idea, even I got the same feedback from our internal review,
> I will do this refactoring in my next patch.
>
> >
> > > install.sh | 12 +-
> > > uninstall-fuego.sh | 47 +++
> >
> > Ooo - I like the sound of this uninstall script.
> Sorry do you have anything in mind, not required or required this uninstall script
> I have created it because it helps me in testing the environment uninstalling and installing the fuego multiple times.
> The uninstall script may also need lot of improvement, but I just created a basic script to solve my current problem.
> I may further improve based on the feedback or the issues if any.
>
> >
> > > 11 files changed, 1057 insertions(+), 604 deletions(-)
> > > delete mode 100644 Dockerfile.nojenkins
> > > create mode 100755 entrypoint.sh
> > > create mode 100755 install-buster.sh
> > > create mode 100755 install-fuego.sh
> > > create mode 100755 install-stretch.sh
> > > create mode 100644 uninstall-fuego.sh
> > >
> >
> >
> > > diff --git a/Dockerfile b/Dockerfile
> > > index 7f63798..ac57cb2 100644
> > > --- a/Dockerfile
> > > +++ b/Dockerfile
> > > @@ -2,9 +2,11 @@
> > > # WARNING: this Dockerfile assumes that the container will be created with
> > > # several bind mounts (see docker-create-container.sh)
> > > #
> > ================================================================
> > ==============
> > > -# FIXTHIS: build this as an extension of the nonjenkins image
> > >
> > > -FROM debian:stretch-slim
> > > +ARG SUITE=stretch
> > > +ARG DIST=debian
> > > +
> > > +FROM ${DIST}:${SUITE}-slim
> > > MAINTAINER tim.bird@sony.com
> > >
> > > #
> > ================================================================
> > ==============
> > > @@ -16,299 +18,32 @@ ENV http_proxy ${HTTP_PROXY}
> > > ENV https_proxy ${HTTP_PROXY}
> > >
> > > #
> > ================================================================
> > ==============
> > > -# Prepare basic image
> > > +# Install fuego with the script
> > > #
> > ================================================================
> > ==============
> > >
> > > -ARG DEBIAN_FRONTEND=noninteractive
> > > -
> > > -WORKDIR /
> > > -RUN echo deb http://deb.debian.org/debian stretch main non-free >
> > /etc/apt/sources.list
> > > -RUN echo deb http://security.debian.org/debian-security stretch/updates
> > main >> /etc/apt/sources.list
> > > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy
> > "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> > > -
> > > -# Fuego python dependencies
> > > -# - python-lxml: ftc, loggen
> > > -# - python-simplejson: ftc
> > > -# - python-yaml: ftc
> > > -# - python-openpyxl: ftc (also LTP)
> > > -# - python-requests: ftc (also fuego_release_test)
> > > -# - python-reportlab: ftc
> > > -# - python-parsedatetime: ftc
> > > -# - python-pexpect: ssh_exec (part of ttc)
> > > -# - python-pip: to install filelock, flake8
> > > -# - filelock: parser
> > > -RUN mkdir -p /usr/share/man/man1
> > > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install
> > \
> > > - python-lxml python-simplejson python-yaml python-openpyxl \
> > > - python-requests python-reportlab python-parsedatetime \
> > > - python-pexpect python-pip python-setuptools python-wheel
> > > -RUN pip install filelock
> > > -
> > > -# Fuego command dependencies
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > > - bash-completion iproute2
> > > -
> > > -# Default SDK for testing locally or on an x86 board
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - build-essential cmake bison flex automake kmod libtool \
> > > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > > -
> > > -# Default test host dependencies
> > > -# - iperf iperf3 netperf: used as servers on the host
> > > -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > > -# - python-matplotlib: Benchmark.iperf3 parser
> > > -# - python-xmltodict: AGL tests
> > > -# - flake8: Functional.fuego_lint
> > > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > > -# - iputils-ping - for /bin/ping command, used by some tests
> > > -# FIXTHIS: install dependencies dynamically on the tests that need them
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > > - netpipe-tcp iputils-ping
> > > -RUN pip install flake8
> > > -
> > > -# miscelaneous packages:
> > > -# python-serial - used by serio
> > > -# diffstat and vim - used by Tim
> > > -# time - useful for timing command duration
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - python-serial \
> > > - diffstat \
> > > - vim \
> > > - time
> > > -
> > > -# install packages used by NuttX SDK
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - genromfs
> > > -
> > > -# FIXTHIS: determine if these tools are really necessary
> > > -#RUN apt-get -q=2 -V --no-install-recommends install \
> > > -# apt-utils python-paramiko \
> > > -# xmlstarlet rsync \
> > > -# inotify-tools gettext netpipe-tcp \
> > > -# at minicom lzop bsdmainutils \
> > > -# mc netcat openssh-server
> > > -
> > > -# bsdmainutils provides hexdump, which is needed for netperf test
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - bsdmainutils
> > > -
> > > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections
> > ; dpkg-reconfigure dash'
> > > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> > > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e
> > "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment;
> > fi
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Install Jenkins with the same UID/GID as the host user
> > > -#
> > ================================================================
> > ==============
> > > +RUN mkdir /fuego
> > > +COPY . /fuego/
> > > +WORKDIR /fuego
> > >
> > > -ARG user=jenkins
> > > -ARG group=jenkins
> > > -ARG uid=1000
> > > -ARG gid=${uid}
> > > ARG JENKINS_PORT=8090
> > > -# see https://updates.jenkins.io/download/war/ for latest Jenkins versions
> > > -#ARG JENKINS_VERSION=2.164.2
> > > -#ARG JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > > -#ARG JENKINS_VERSION=2.235.3
> > > -#ARG JENKINS_SHA=89a37ed1b3c8c14ff94cc66c3f2254f281b60794
> > > -ARG JENKINS_VERSION=2.249.3
> > > -ARG JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > > -
> > > -ARG JENKINS_URL=https://pkg.jenkins.io/debian-
> > stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > > -ARG JENKINS_UC=https://updates.jenkins.io
> > > -ARG REF=/var/lib/jenkins/plugins
> > > +ARG NO_JENKINS=
> > > +ARG SUITE=stretch
> > > +ARG DIST=debian
> > > +ARG uid=
> > > +ARG gid=
> > > ENV JENKINS_HOME=/var/lib/jenkins
> > > ENV JENKINS_PORT=$JENKINS_PORT
> > > +RUN ./install-$DIST.sh -s $SUITE -p $JENKINS_PORT -u $uid -g $gid
> > $NO_JENKINS
> >
> > One question about this overall approach. Does this retain the ability to cache
> > operations, so that if something goes wrong with an install, you don't have to
> > start over from scratch. When building with docker, with multiple steps, it
> > caches the image at each step. When rebuilding, or when an error occurs,
> > this can save a lot of time. It also can introduce subtle bugs, but that's
> > a separate issue.
> >
> > This setup seems like it performs the bulk of the install as a single docker
> > step, is that right?
> Yes, it will not create the cache for each step in the installation script, this will be a disadvantage with this common script approach.
>
> >
> > > -# Jenkins dependencies
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - default-jdk daemon psmisc adduser procps unzip
> > > -RUN pip install python-jenkins==1.4.0
> > > -
> > > -RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > > -RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> > > -RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> > /bin/bash ${user}
> > > -RUN wget -nv ${JENKINS_URL}
> > > -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
> > -c -
> > > -# allow Jenkins to start and install plugins, as part of dpkg installation
> > > -RUN printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > > -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > > -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# get ttc script and helpers
> > > -#
> > ================================================================
> > ==============
> > > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > > -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> > > -# as of ttc version 2.0.2, this should not be needed
> > > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> > /usr/local/bin/ttc
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# get ebf script and helpers
> > > -#
> > ================================================================
> > ==============
> > > -RUN apt-get -q=2 -V --no-install-recommends install jq
> > > -RUN git clone https://github.com/TimesysGit/board-farm-rest-api.git
> > /usr/local/src/board-farm-rest-api ; \
> > > - cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf ; \
> > > - chmod a+x /usr/local/bin/ebf
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Serial Config
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio
> > > -COPY frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> > name.patch \
> > > - frontend-install/0002-Output-data-from-port-during-command-
> > execution.patch \
> > > - /tmp/
> > > -RUN /bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-
> > parsing-for-serial-device-with-in-name.patch ; \
> > > - patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> > during-command-execution.patch ; \
> > > - chown -R jenkins /usr/local/src/serio ; \
> > > - cp /usr/local/src/serio/serio /usr/local/bin/ ; \
> > > - ln -s /usr/local/bin/serio /usr/local/bin/sercp ; \
> > > - ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > > -
> > > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ; cp
> > > /usr/local/src/serlogin/serlogin /usr/local/bin'
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# fserver
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git
> > /usr/local/lib/fserver ; \
> > > - ln -s /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server'
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Jenkins post installation
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN source /etc/default/jenkins && \
> > > - JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > > - sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> > /etc/default/jenkins
> > > -
> > > -RUN source /etc/default/jenkins && \
> > > - JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false -
> > > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > > - if [ -n "$HTTP_PROXY" ]; then \
> > > - PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> > 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > > - PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> > 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > > - JAVA_ARGS="$JAVA_ARGS -
> > Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > > - fi && \
> > > - sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> > /etc/default/jenkins;
> > > -
> > > -RUN sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > > -
> > > -# set up Jenkins plugins
> > > -COPY frontend-install/install-plugins.sh \
> > > - frontend-install/jenkins-support \
> > > - frontend-install/clitest \
> > > - /usr/local/bin/
> > > -
> > > -# start Jenkins onces to initialize directories?
> > > -RUN service jenkins start && \
> > > - sleep 30 && \
> > > - service jenkins stop
> > > -
> > > -# install plugins - these versions are for Jenkins version 2.164.2
> > > -# Explicitly install script-security v1.68, otherwise
> > > -# v1.74 will automatically be installed as a dependency of
> > > -# the junit plugin. Make sure to install before junit plugin.
> > > -# Explicitly install junit:1.27
> > > -
> > > -# install other plugins from Jenkins update center
> > > -# NOTE: not sure all of these are needed, but keep list
> > > -# as compatible as possible with 1.2.1 release for now
> > > -# Do NOT change the order of the plugins, unless you know what
> > > -# you are doing. Otherwise, the install-plugins.sh script will
> > > -# install the wrong versions of other plugins based on dependency
> > > -# information in the plugin files.
> > > -
> > > -#RUN /usr/local/bin/install-plugins.sh \
> > > -# script-security:1.68 \
> > > -# structs:1.20 \
> > > -# workflow-step-api:2.22 \
> > > -# workflow-api:2.40 \
> > > -# junit:1.27 \
> > > -# scm-api:2.6.3 \
> > > -# ant:1.9 \
> > > -# antisamy-markup-formatter:1.5 \
> > > -# bouncycastle-api:2.17 \
> > > -# command-launcher:1.3 \
> > > -# description-setter:1.10 \
> > > -# display-url-api:2.3.1 \
> > > -# external-monitor-job:1.7 \
> > > -# greenballs:1.15 \
> > > -# icon-shim:2.0.3 \
> > > -# javadoc:1.5 \
> > > -# jdk-tool:1.2 \
> > > -# ldap:1.20 \
> > > -# mailer:1.23 \
> > > -# matrix-auth:2.3 \
> > > -# matrix-project:1.14 \
> > > -# pam-auth:1.5 \
> > > -# pegdown-formatter:1.3 \
> > > -# windows-slaves:1.4
> > > -
> > > -# install plugins for latest LTS Jenkins (currently 2.249.3)
> > > -RUN /usr/local/bin/install-plugins.sh \
> > > - script-security \
> > > - structs \
> > > - workflow-step-api \
> > > - workflow-api \
> > > - junit \
> > > - scm-api \
> > > - ant \
> > > - antisamy-markup-formatter \
> > > - bouncycastle-api \
> > > - command-launcher \
> > > - description-setter \
> > > - display-url-api \
> > > - external-monitor-job \
> > > - greenballs \
> > > - icon-shim \
> > > - javadoc \
> > > - jdk-tool \
> > > - mailer \
> > > - matrix-auth \
> > > - matrix-project \
> > > - pam-auth \
> > > - pegdown-formatter \
> > > - windows-slaves
> > > +COPY entrypoint.sh /usr/bin/
> > >
> > > -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi
> > $JENKINS_HOME/plugins/flot.jpi
> > > -
> > > -# jenkins should automatically unzip any plugins in the plugin dir
> > > -# make a symlink for mod.js well after flot is installed
> > > -RUN service jenkins start && sleep 30 && \
> > > - rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > > - ln -s /fuego-core/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
> > > -
> > > -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > > -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> > docs.pdf
> > > -
> > > -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> > > -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > -RUN sed -i -e "s#8080#$JENKINS_PORT#g"
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > -
> > > -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# ftc post installation
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > > -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Lava
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > > +# Delete the files that are copied for installation
> > > +RUN rm -rf /fuego
> >
> > I'm not sure what this is doing. It looks scary. Please explain.
> Because, I had COPY the fuego current directory to the Docker image for installation purpose and removing it after installation,
> and this is required because the common script "install-*.sh" requires some files to copy inside docker to a specified location
> for e.g:
> cp frontend-install/install-plugins.sh \
> frontend-install/jenkins-support \
> frontend-install/clitest \
> /usr/local/bin/
>
> And in the previous approach this was done using COPY command which will directly copy from host machine to docker location.
>
> May be I could have used mounting directory inside docker workdir to avoid copying and later removing it, maybe I faced some problem
> with this approach and so later chosen to copy it inside docker image.
> I have done this work little long so I have not remembered why I have chosen the copy it inside docker approach, Let me work on again and
> understand why have done this and change it to mounting directory instead copy and remove.
>
> >
> > > +WORKDIR /
> > >
> > > #
> > ================================================================
> > ==============
> > > # Setup startup command
> > > #
> > ================================================================
> > ==============
> > > -
> > > # FIXTHIS: when running multiple Fuego containers, or if the host is already
> > > # running the netperf server, netperf will complain because the port is taken
> > > -ENTRYPOINT service jenkins start && service netperf start && iperf3 -V -s -D
> > -f M && /bin/bash
> > > +ENTRYPOINT entrypoint.sh $NO_JENKINS && /bin/bash
> >
> > Isolating this to a separate script is nice!
> >
> > > diff --git a/Dockerfile.nojenkins b/Dockerfile.nojenkins
> > > deleted file mode 100644
> > > index 56de98a..0000000
> > > --- a/Dockerfile.nojenkins
> > > +++ /dev/null
> >
> > Getting rid of this separate Dockerfile is nice.
> >
> > > @@ -1,151 +0,0 @@
> > > -#
> > ================================================================
> > ==============
> > > -# WARNING: this Dockerfile assumes that the container will be created with
> > > -# several bind mounts (see docker-create-container.sh)
> > > -#
> > ================================================================
> > ==============
> > > -# Usage:
> > > -# host$ vi fuego-ro/conf/fuego.conf
> > > -# jenkins_enabled=0
> > > -# host$ ./fuego-host-scripts/docker-build-image.sh fuego-nojenkins 8090
> > Dockerfile.nojenkins
> > > -# host$ ./fuego-host-scripts/docker-create-container.sh fuego-nojenkins
> > fuego-nojenkins-container
> > > -# host$ ./fuego-host-scripts/docker-start-container.sh fuego-nojenkins-
> > container
> > > -# docker# ftc list-boards
> > > -# docker, bbb, ...
> > > -# docker# ftc list-tests
> > > -# Functional.hello_world, ...
> > > -# docker# ftc run-test -b docker -t Functional.hello_world
> > > -# log_compare: pattern 'SUCCESS' found 1 times (expected greater or
> > equal than 1)
> > > -
> > > -FROM debian:stretch-slim
> > > -MAINTAINER tim.bird@sony.com
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Proxy variables
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -ARG HTTP_PROXY
> > > -ENV http_proxy ${HTTP_PROXY}
> > > -ENV https_proxy ${HTTP_PROXY}
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Prepare basic image
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -ARG DEBIAN_FRONTEND=noninteractive
> > > -
> > > -WORKDIR /
> > > -RUN echo deb http://deb.debian.org/debian stretch main non-free >
> > /etc/apt/sources.list
> > > -RUN echo deb http://security.debian.org/debian-security stretch/updates
> > main >> /etc/apt/sources.list
> > > -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy
> > "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> > > -
> > > -# Fuego python dependencies
> > > -# - python-lxml: ftc, loggen
> > > -# - python-simplejson: ftc
> > > -# - python-yaml: ftc
> > > -# - python-openpyxl: ftc (also LTP)
> > > -# - python-requests: ftc (also fuego_release_test)
> > > -# - python-reportlab: ftc
> > > -# - python-parsedatetime: ftc
> > > -# - python-pexpect: ssh_exec (part of ttc)
> > > -# - python-pip: to install filelock, flake8
> > > -# - filelock: parser
> > > -RUN mkdir -p /usr/share/man/man1
> > > -RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install
> > \
> > > - python-lxml python-simplejson python-yaml python-openpyxl \
> > > - python-requests python-reportlab python-parsedatetime \
> > > - python-pexpect python-pip python-setuptools python-wheel
> > > -RUN pip install filelock
> > > -
> > > -# Fuego command dependencies
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > > - bash-completion iproute2
> > > -
> > > -# Default SDK for testing locally or on an x86 board
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - build-essential cmake bison flex automake kmod libtool \
> > > - libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > > - libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > > -
> > > -# Default test host dependencies
> > > -# - iperf iperf3 netperf: used as servers on the host
> > > -# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > > -# - python-matplotlib: Benchmark.iperf3 parser
> > > -# - python-xmltodict: AGL tests
> > > -# - flake8: Functional.fuego_lint
> > > -# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > > -# - iputils-ping - for /bin/ping command, used by some tests
> > > -# FIXTHIS: install dependencies dynamically on the tests that need them
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > > - netpipe-tcp iputils-ping
> > > -RUN pip install flake8
> > > -
> > > -# miscelaneous packages:
> > > -# python-serial - used by serio
> > > -# diffstat and vim - used by Tim
> > > -# time - useful for timing command duration
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - python-serial \
> > > - diffstat \
> > > - vim \
> > > - time
> > > -
> > > -# FIXTHIS: determine if these tools are really necessary
> > > -# RUN apt-get -q=2 -V --no-install-recommends install \
> > > -# apt-utils python-paramiko \
> > > -# xmlstarlet rsync \
> > > -# inotify-tools gettext netpipe-tcp \
> > > -# at minicom lzop bsdmainutils \
> > > -# mc netcat openssh-server
> > > -
> > > -# bsdmainutils provides hexdump, which is needed for netperf test
> > > -RUN apt-get -q=2 -V --no-install-recommends install \
> > > - bsdmainutils
> > > -
> > > -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections
> > ; dpkg-reconfigure dash'
> > > -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> > > -RUN if [ -n "$HTTP_PROXY" ]; then echo -e
> > "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment;
> > fi
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# get ttc script and helpers
> > > -#
> > ================================================================
> > ==============
> > > -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > > -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> > > -# as of ttc version 2.0.2, this should not be needed
> > > -#RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> > /usr/local/bin/ttc
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Serial Config
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s
> > > /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sersh'
> > > -
> > > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > > /usr/local/bin'
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# fserver
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN /bin/bash -c 'git clone https://github.com/tbird20d/fserver.git
> > /usr/local/lib/fserver ; ln -s /usr/local/lib/fserver/start_local_bg_server
> > > /usr/local/bin/start_local_bg_server'
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# ftc post installation
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > > -COPY fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > > -RUN echo ". /etc/bash_completion" >> /root/.bashrc
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Lava
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > > -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Setup startup command
> > > -#
> > ================================================================
> > ==============
> > > -
> > > -# FIXTHIS: when running multiple Fuego containers netperf will complain
> > because the port is taken
> > > -ENTRYPOINT service netperf start && iperf3 -V -s -D -f M && /bin/bash
> > > diff --git a/README b/README
> > > index 882d84e..ce42e1e 100644
> > > --- a/README
> > > +++ b/README
> > > @@ -44,6 +44,29 @@ You can change the name of the docker image and the
> > port used by Jenkins
> > > by passing the corresponding parameters to install.sh.
> > > ./install.sh fuego-8082 8082
> > >
> > > +To run fuego on different distribution versions like Debian(stretch, buster,
> > etc.)
> > > + pass appropriate arguments to the install.sh script
> > > + ./install.sh <image name> <Jenkins port> <Distribution name> <suite>
> >
> > should probably be:
> > ./install.sh [[<image name>] [[<Jenkins port>] [<Distribution name> <suite>]]]
> Thanks for suggestion, may be if I remove the distribution then it will be like this
> ./install.sh [[<image name>] [[<Jenkins port>] [<suite>]]]
>
> > > +
> > > +By default install.sh script will install fuego on Debian stretch version
> > > + to install fuego on Debian buster version pass below options
> > > + ./install.sh fuego_buster 8090 debian buster
> > > +
> > > +[Note] It is currently verified on Debian versions (stretch and buster),
> > > +and yet to provide install scripts for other distributions
> > > +
> > > +Installing Fuego to a host system
> > > +-------------------------------------------------------------------------------
> > > +Run install-fuego.sh to install fuego on host system, by default it assume the
> > > +host system is Debian stretch
> > > + ./install-fuego.sh
> > > +
> > > + to install on Debian buster version
> > > + ./install-fuego.sh -d debian -s buster -p 8090
> > > +
> > > + to install without jenkins pass --nojenkins option to the script
> > > + ./install-fuego.sh --nojenkins
> > > +
> > > Running
> > > -------------------------------------------------------------------------------
> > > To start your Fuego container, issue the following command top directory:
> > > @@ -85,9 +108,9 @@ Updating the Jenkins version
> > > -------------------------------------------------------------------------------
> > >
> > > Modify the version and checksum
> > > - $ vi Dockerfile
> > > - ARG JENKINS_VERSION=2.32.1
> > > - ARG JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> > > + $ vi install-debian.sh
> > > + JENKINS_VERSION=2.32.1
> > > + JENKINS_SHA=3cbe9b6d4d82c975642ea4409e21e087
> > >
> > > Re-build the flot plugin
> > > $ sudo apt-get install maven
> > > diff --git a/entrypoint.sh b/entrypoint.sh
> > > new file mode 100755
> > > index 0000000..6eac895
> > > --- /dev/null
> > > +++ b/entrypoint.sh
> > > @@ -0,0 +1,7 @@
> > > +#!/bin/bash
> > > +
> > > +[ "$1" != "--nojenkins" ] && service jenkins start
> > > +
> > > +#TODO: netperf not present in buster, remove this condition once
> > alternative found
> > > +[ -f /etc/init.d/netperf ] && service netperf start
> > > +iperf3 -V -s -D -f M
> > > diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> > scripts/docker-build-image.sh
> > > index 5823357..6889c47 100755
> > > --- a/fuego-host-scripts/docker-build-image.sh
> > > +++ b/fuego-host-scripts/docker-build-image.sh
> > > @@ -2,9 +2,10 @@
> > > # $1 - name for the docker image (default: fuego)
> > > # $2 - port for jenkins (default: 8090)
> > > # $3 - Dockerfile or Dockerfile.nojenkins
> > > +# $4 - Debian code name on which to install (default: stretch)
> > > #
> > > # Example:
> > > -# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082
> > Dockerfile.nojenkins
> > > +# ./fuego-host-scripts/docker-build-image.sh --no-cache myfuegoimg 8082
> > debian stretch --nojenkins
> > > #
> > > if [ "$1" = "--no-cache" ]; then
> > > NO_CACHE=--no-cache
> > > @@ -13,7 +14,10 @@ fi
> > >
> > > DOCKERIMAGE=${1:-fuego}
> > > JENKINS_PORT=${2:-8090}
> > > -DOCKERFILE=${3:-Dockerfile}
> > > +DISTRIBUTION=${3:-debian}
> > > +SUITE=${4:-stretch}
> > > +NO_JENKINS=${5:""}
> > > +DOCKERFILE="Dockerfile"
> > >
> > > if [ "$(id -u)" == "0" ]; then
> > > JENKINS_UID=$(id -u $SUDO_USER)
> > > @@ -27,4 +31,7 @@ echo "Using Port $JENKINS_PORT"
> > >
> > > sudo docker build ${NO_CACHE} -t ${DOCKERIMAGE} --build-arg
> > HTTP_PROXY=$http_proxy \
> > > --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID \
> > > + --build-arg DIST=$DISTRIBUTION \
> > > + --build-arg SUITE=$SUITE \
> > > + --build-arg NO_JENKINS=$NO_JENKINS \
> > > --build-arg JENKINS_PORT=$JENKINS_PORT -f $DOCKERFILE .
> > > diff --git a/install-buster.sh b/install-buster.sh
> > > new file mode 100755
> > > index 0000000..f4ce65b
> > > --- /dev/null
> > > +++ b/install-buster.sh
> > > @@ -0,0 +1,294 @@
> > > +#!/bin/bash
> > > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > > +#
> > > +# Usage:
> > > +# $ sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> > > +#
> > > +usage(){
> > > +
> > > +cat <<HERE
> > > +Usage: sudo ./install-buster.sh [--help|-h] [--nojenkins] [<port>]
> > > +
> > > +Install fuego on the host Debian filesystem.
> > > +
> > > +options:
> > > + --help Show usage help
> > > + --nojenkins Install Fuego without Jenkins
> > > +HERE
> > > +}
> > > +
> > > +if [[ $EUID -ne 0 ]]; then
> > > + echo "Sorry, you need root permissions"
> > > + exit 1
> > > +fi
> > > +
> > > +nojenkins=0
> > > +
> > > +POSITIONAL=()
> > > +while [[ $# -gt 0 ]] ; do
> > > + case $1 in
> > > + --nojenkins)
> > > + nojenkins=1
> > > + shift
> > > + ;;
> > > + --help | -h)
> > > + usage
> > > + exit 0
> > > + ;;
> > > + *)
> > > + POSITIONAL+=("$1") # save argument for later
> > > + shift
> > > + esac
> > > +done
> > > +set -- "${POSITIONAL[@]}" # restore positional arguments
> > > +
> > > +port=${2:-8090}
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Install Fuego dependencies
> > > +#
> > ================================================================
> > ==============
> > > +
> > > +mkdir -p /usr/share/man/man1
> > > +
> > > +# netperf is in non-free
> > > +echo deb http://deb.debian.org/debian buster main non-free >
> > /etc/apt/sources.list
> > > +echo deb http://security.debian.org/debian-security buster/updates main >>
> > /etc/apt/sources.list
> > > +
> > > +apt-get update -q=2
> > > +
> > > +# Fuego python dependencies
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + python-lxml python-simplejson python-yaml python-openpyxl \
> > > + python-requests python-reportlab python-parsedatetime \
> > > + python-pexpect python-pip python-setuptools python-wheel
> > > +pip install filelock
> > > +
> >
> > Can we put this stuff that is common between install-buster.sh and install-
> > stretch.sh
> > into an "install-debian-common.sh"?
> Yes, I will change this in my next patch.
> Thanks for suggesting it.
>
> >
> > > +# Fuego command dependencies
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > > + bash-completion iproute2
> > > +
> > > +# Default SDK for testing locally or on an x86 board
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + build-essential cmake bison flex kmod automake libtool \
> > > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > > +
> > > +# Default test host dependencies
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> > > + netpipe-tcp iputils-ping
> > > +pip install flake8
> > > +
> > > +# miscelaneous packages:
> > > +# python-serial - used by serio
> > > +# diffstat and vim - used by Tim
> > > +# time - useful for timing command duration
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + python-serial \
> > > + diffstat \
> > > + vim \
> > > + time
> > > +
> > > +# install packages used by NuttX SDK
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + genromfs
> > > +
> > > +# bsdmainutils provides hexdump, which is needed for netperf test
> > > +#apt-get -q=2 -V --no-install-recommends install \
> > > +# bsdmainutils
> > > +
> > > +echo "dash dash/sh boolean false" | debconf-set-selections
> > > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > > +
> > > +# TODO: check if proxy settings can set outside docker container
> > > +if [ -n "$http_proxy" ]; then
> > > + sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > > + echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >>
> > /etc/environment
> > > +fi
> >
> > This doesn't seem specific to Debian (/etc/wgetrc and /etc/environment).
> > Should this be moved to an 'install-common.sh' script?
> Yes, this one too I will do in my next work.
>
> >
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Install Jenkins
> > > +#
> > ================================================================
> > ==============
> > > +
> > > +if [ $nojenkins -eq 0 ]; then
> > > + #JENKINS_VERSION=2.164.2
> > > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > > + #JENKINS_VERSION=2.249.3
> > > + #JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > > + JENKINS_VERSION=2.164.3
> > > + JENKINS_SHA=27f14f0ccca207660765205317295a5c29f4dd53
> > > + JENKINS_URL=https://pkg.jenkins.io/debian-
> > stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > > + export JENKINS_UC=https://updates.jenkins.io
> > > + REF=/var/lib/jenkins/plugins
> > > + JENKINS_HOME=/var/lib/jenkins
> > > + JENKINS_PORT=$port
> > > +
> > > + # Jenkins dependencies
> > > + apt-get -q=2 -V --no-install-recommends install \
> > > + default-jdk daemon psmisc adduser procps unzip
> > > + pip install python-jenkins==1.4.0
> > > +
> > > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > > + export JENKINS_PORT=$port
> > > +
> > > + groupadd jenkins
> > > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash
> > jenkins
> > > +
> > > + wget -nv ${JENKINS_URL}
> > > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> > sha1sum -c -
> > > +
> > > + # allow Jenkins to start and install plugins, as part of dpkg installation
> > > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > > + rm jenkins_${JENKINS_VERSION}_all.deb
> > > +
> > > + # fix for Jenkins Java version check error
> > > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.*
> > version
> > > \"\\\(.*\\\)\\\.\\\(.*\\\)\\\..\*\\\..\*\".\*\/\\\1\\\2\/p;\'\)"
> > > + sed -i -e
> > "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g"
> > /etc/init.d/jenkins
> > > +
> > > + source /etc/default/jenkins && \
> > > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > > + sed -i -e
> > "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> > /etc/default/jenkins
> > > +
> > > + source /etc/default/jenkins && \
> > > + JAVA_ARGS="$JAVA_ARGS -
> > Djenkins.install.runSetupWizard=false -
> > > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > > + if [ -n "$http_proxy" ]; then \
> > > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://'
> > | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' |
> > sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > > + JAVA_ARGS="$JAVA_ARGS -
> > Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > > + fi && \
> > > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> > /etc/default/jenkins;
> > > +
> > > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/Jenkins
> >
> > The above looks like there's a lot of non-Debian-specific stuff intermingled with
> > Debian-specific stuff.
> > I'm not sure what other distributions are intended to be supported, but wouldn't
> > the user and
> > group manipulation be the same for those?
> >
> > > +
> > > + # set up Jenkins plugins
> > > + cp frontend-install/install-plugins.sh \
> > > + frontend-install/jenkins-support \
> > > + frontend-install/clitest \
> > > + /usr/local/bin/
> > > +
> > > + mkdir -p $JENKINS_HOME/userContent/docs
> > > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> > docs.pdf
> > > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > + sed -i -e "s#8080#$JENKINS_PORT#g"
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > + chown -R jenkins:jenkins $JENKINS_HOME/
> > > +
> > > + # start and stop jenkins to pre-populate some settings
> > > + service jenkins start && \
> > > + sleep 30 && \
> > > + service jenkins stop
> > > +
> > > + # install other plugins from Jenkins update center
> > > + # NOTE: not sure all of these are needed, but keep list
> > > + # compatible with 1.2.1 release for now
> > > + #/usr/local/bin/install-plugins.sh \
> > > + # ant:1.9 \
> > > + # antisamy-markup-formatter:1.5 \
> > > + # bouncycastle-api:2.17 \
> > > + # command-launcher:1.3 \
> > > + # description-setter:1.10 \
> > > + # display-url-api:2.3.1 \
> > > + # external-monitor-job:1.7 \
> > > + # greenballs:1.15 \
> > > + # icon-shim:2.0.3 \
> > > + # javadoc:1.5 \
> > > + # jdk-tool:1.2 \
> > > + # junit:1.27 \
> > > + # ldap:1.20 \
> > > + # mailer:1.23 \
> > > + # matrix-auth:2.3 \
> > > + # matrix-project:1.14 \
> > > + # pam-auth:1.4 \
> > > + # pegdown-formatter:1.3 \
> > > + # structs:1.17 \
> > > + # windows-slaves:1.4
> > > +
> > > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > > + /usr/local/bin/install-plugins.sh \
> > > + script-security \
> > > + structs \
> > > + workflow-step-api \
> > > + workflow-api \
> > > + junit \
> > > + scm-api \
> > > + ant \
> > > + antisamy-markup-formatter \
> > > + bouncycastle-api \
> > > + command-launcher \
> > > + description-setter \
> > > + display-url-api \
> > > + external-monitor-job \
> > > + greenballs \
> > > + icon-shim \
> > > + javadoc \
> > > + jdk-tool \
> > > + mailer \
> > > + matrix-auth \
> > > + matrix-project \
> > > + pam-auth \
> > > + pegdown-formatter \
> > > + windows-slaves
> > > +
> > > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> > $JENKINS_HOME/plugins/flot.jpi
> > > +
> > > + # make the mod.js symlink well after flot is installed
> > > + service jenkins start && sleep 30 && \
> > > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > > + ln -s /fuego-core/scripts/mod.js
> > $JENKINS_HOME/plugins/flot/flot/mod.js
> > > +
> > > + chown -R jenkins:jenkins $JENKINS_HOME/
> > > +fi
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# get ttc script and helpers
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > > +/usr/local/src/ttc/install.sh /usr/local/bin
> > > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> > /usr/local/bin/ttc
> > > +
> > The above and following are definitely not unique to Debian, or a specific
> > version of Debian.
> > They should be put into an "install-common.sh"
> >
> > > +#
> > ================================================================
> > ==============
> > > +# Serial Config
> > > +#
> > ================================================================
> > ==============
> > > +if [ $nojenkins -eq 0 ]; then
> > > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > > +else
> > > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sersh'
> > > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > > /usr/local/bin'
> > > +fi
> > > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> > name.patch \
> > > + frontend-install/0002-Output-data-from-port-during-command-
> > execution.patch \
> > > + /tmp/
> > > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-
> > for-serial-device-with-in-name.patch ; \
> > > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> > during-command-execution.patch ;'
> > > +
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# get ebf script and helpers
> > > +#
> > ================================================================
> > ==============
> > > +apt-get -q=2 -V --no-install-recommends install jq
> > > +git clone https://github.com/TimesysGit/board-farm-rest-api.git
> > /usr/local/src/board-farm-rest-api
> > > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > > +chmod a+x /usr/local/bin/ebf
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# fserver
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > > +ln -s /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# ftc post installation
> > > +#
> > ================================================================
> > ==============
> > > +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > > +echo ". /etc/bash_completion" >> /root/.bashrc
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Lava
> > > +#
> > ================================================================
> > ==============
> > > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > > diff --git a/install-debian.sh b/install-debian.sh
> > > index ccca032..2d3ffd3 100755
> > > --- a/install-debian.sh
> > > +++ b/install-debian.sh
> > > @@ -2,102 +2,163 @@
> > > # 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > > #
> > > # Usage:
> > > -# $ sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> > > +# $ sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins port>] [--
> > nojenkins]
> > > #
> > > -if [ -n "$1" ]; then
> > > - if [ "$1" = "--help" -o "$1" = "-h" ]; then
> > > - cat <<HERE
> > > -Usage: sudo ./install-debian.sh [--help|-h] [--nojenkins] [<port>]
> > > -
> > > -Installs fuego on the host Debian filesystem.
> > > -
> > > -options:
> > > - --help Show usage help
> > > - --nojenkins Install Fuego without Jenkins
> > > -HERE
> > > - exit 0
> > > - fi
> > > -fi
> > > +
> > > +usage(){
> > > + echo ""
> > > + echo "Usage: sudo ./install-debian.sh [--help|-h] [-s <suite>] [-p <jenkins
> > port>] [--nojenkins]"
> > > + echo "Install fuego on Debian filesystem."
> > > + echo ""
> > > + echo "options:"
> > > + echo " --help | -h Show usage help"
> > > + echo " --suite | -s <suite> Specify Debian suite"
> > > + echo " --port | -p <port> Jenkins port"
> > > + echo " --nojenkins Install Fuego without Jenkins"
> > > + echo ""
> > > +}
> > >
> > > if [[ $EUID -ne 0 ]]; then
> > > echo "Sorry, you need root permissions"
> > > exit 1
> > > fi
> > >
> > > -if [ "$1" = "--nojenkins" ]; then
> > > - nojenkins=1
> > > +nojenkins=0
> > > +port=8090
> > > +suite=stretch
> > > +user=jenkins
> > > +group=jenkins
> > > +uid=1000
> > > +gid=1000
> > > +
> > > +while [ "$1" != "" ]; do
> > > + case $1 in
> > > + -s | --suite )
> > > + shift
> > > + suite=$1
> > > + ;;
> > > + -p | --port )
> > > + shift
> > > + port=$1
> > > + ;;
> > > + -u | --uid )
> > > + shift
> > > + uid=$1
> > > + ;;
> > > + -g | --gid )
> > > + shift
> > > + gid=$1
> > > + ;;
> > > + --nojenkins )
> > > + nojenkins=1
> > > + ;;
> > > + -h | --help )
> > > + usage
> > > + exit 0
> > > + ;;
> > > + * )
> > > + break
> > > + esac
> > > shift
> > > -else
> > > - nojenkins=0
> > > - port=${1:-8090}
> > > -fi
> > > +done
> > >
> > > #
> > ================================================================
> > ==============
> > > # Install Fuego dependencies
> > > #
> > ================================================================
> > ==============
> > >
> > > # netperf is in non-free
> > > -echo deb http://deb.debian.org/debian stretch main non-free >
> > /etc/apt/sources.list
> > > -echo deb http://security.debian.org/debian-security stretch/updates main >>
> > /etc/apt/sources.list
> > > -
> > > -apt-get update -q=2
> > > +echo deb http://deb.debian.org/debian $suite main non-free >
> > /etc/apt/sources.list
> > > +echo deb http://security.debian.org/debian-security $suite/updates main >>
> > /etc/apt/sources.list
> > >
> > > # Fuego python dependencies
> > > -apt-get -q=2 -V --no-install-recommends install \
> > > +# - python-lxml: ftc, loggen
> > > +# - python-simplejson: ftc
> > > +# - python-yaml: ftc
> > > +# - python-openpyxl: ftc (also LTP)
> > > +# - python-requests: ftc (also fuego_release_test)
> > > +# - python-reportlab: ftc
> > > +# - python-parsedatetime: ftc
> > > +# - python-pexpect: ssh_exec (part of ttc)
> > > +# - python-pip: to install filelock, flake8
> > > +# - filelock: parser
> > > +mkdir -p /usr/share/man/man1
> > > +apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
> > > python-lxml python-simplejson python-yaml python-openpyxl \
> > > python-requests python-reportlab python-parsedatetime \
> > > - python-pip
> > > + python-pexpect python-pip python-setuptools python-wheel
> > > pip install filelock
> > >
> > > # Fuego command dependencies
> > > apt-get -q=2 -V --no-install-recommends install \
> > > git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > > - bash-completion
> > > + bash-completion iproute2
> > >
> > > # Default SDK for testing locally or on an x86 board
> > > apt-get -q=2 -V --no-install-recommends install \
> > > build-essential cmake bison flex automake kmod libtool \
> > > libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > > - libxmuu-dev libglib2.0-dev libaio-dev u-boot-tools pkg-config
> > > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > >
> > > # Default test host dependencies
> > > +# - iperf iperf3 netperf: used as servers on the host
> > > +# - bzip2 bc: used for local tests by Functional.bzip2/bc
> > > +# - python-matplotlib: Benchmark.iperf3 parser
> > > +# - python-xmltodict: AGL tests
> > > +# - flake8: Functional.fuego_lint
> > > +# - netpipe-tcp - used by Benchmark.netpipe (provides the netpipe server)
> > > +# - iputils-ping - for /bin/ping command, used by some tests
> > > +# FIXTHIS: install dependencies dynamically on the tests that need them
> > > apt-get -q=2 -V --no-install-recommends install \
> > > - iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict
> > > -pip install flake8 bsdmainutils
> > > + iperf iperf3 bzip2 bc python-matplotlib python-xmltodict \
> > > + netpipe-tcp iputils-ping
> > >
> > > -echo "dash dash/sh boolean false" | debconf-set-selections
> > > -DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > > -if [ -n "$http_proxy" ]; then
> > > - sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > > - echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >>
> > /etc/environment
> > > +#TODO: find alternate package for netperf in buster version
> > > +if [ $suite != "buster" ];then
> > > + apt-get -q=2 -V --no-install-recommends install netperf
> > > fi
> > > +pip install flake8
> > >
> > > -#
> > ================================================================
> > ==============
> > > -# Clone fuego and fuego-core
> > > -#
> > ================================================================
> > ==============
> > > -if [ ! -d "/fuego" ]; then
> > > - cd /
> > > - git clone --branch next --depth=1
> > https://bitbucket.org/nirrognas/fuego.git
> > > - ln -s /fuego/fuego-ro /fuego-ro
> > > - ln -s /fuego/fuego-rw /fuego-rw
> > > - cd fuego
> > > - git clone --branch next --depth=1
> > https://bitbucket.org/nirrognas/fuego-core.git
> > > - ln -s /fuego/fuego-core /fuego-core
> > > -fi
> > > +# miscelaneous packages:
> > > +# python-serial - used by serio
> > > +# diffstat and vim - used by Tim
> > > +# time - useful for timing command duration
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + python-serial \
> > > + diffstat \
> > > + vim \
> > > + time
> > > +
> > > +# install packages used by NuttX SDK
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + genromfs
> > > +
> > > +# FIXTHIS: determine if these tools are really necessary
> > > +#RUN apt-get -q=2 -V --no-install-recommends install \
> > > +# apt-utils python-paramiko \
> > > +# xmlstarlet rsync \
> > > +# inotify-tools gettext netpipe-tcp \
> > > +# at minicom lzop bsdmainutils \
> > > +# mc netcat openssh-server
> > > +
> > > +# bsdmainutils provides hexdump, which is needed for netperf test
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + bsdmainutils
> > > +
> > > +echo "dash dash/sh boolean false" | debconf-set-selections
> > > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > >
> > > #
> > ================================================================
> > ==============
> > > # Install Jenkins
> > > #
> > ================================================================
> > ==============
> > > -
> > > if [ $nojenkins -eq 0 ]; then
> > > #JENKINS_VERSION=2.164.2
> > > #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > > JENKINS_VERSION=2.249.3
> > > JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > > JENKINS_URL=https://pkg.jenkins.io/debian-
> > stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > > - JENKINS_UC=https://updates.jenkins.io
> > > - REF=/var/lib/jenkins/plugins
> > > - JENKINS_HOME=/var/lib/jenkins
> > > + export JENKINS_UC=https://updates.jenkins.io
> > > + export REF=/var/lib/jenkins/plugins
> > > + export JENKINS_HOME=/var/lib/jenkins
> > > JENKINS_PORT=$port
> > >
> > > # Jenkins dependencies
> > > @@ -106,10 +167,12 @@ if [ $nojenkins -eq 0 ]; then
> > > pip install python-jenkins==1.4.0
> > >
> > > echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > > - groupadd jenkins
> > > - useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash
> > jenkins
> > > - cd
> > > - wget ${JENKINS_URL}
> > > + export JENKINS_PORT=$port
> > > +
> > > + getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
> > > + useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> > /bin/bash ${user}
> > > +
> > > + wget -nv ${JENKINS_URL}
> > > echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> > sha1sum -c -
> > >
> > > # allow Jenkins to start and install plugins, as part of dpkg installation
> > > @@ -117,11 +180,60 @@ if [ $nojenkins -eq 0 ]; then
> > > dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > > rm jenkins_${JENKINS_VERSION}_all.deb
> > >
> > > - # update ownership
> > > - chown -R jenkins:jenkins /fuego
> > > - chown -R jenkins:jenkins /fuego-ro
> > > - chown -R jenkins:jenkins /fuego-rw
> > > - chown -R jenkins:jenkins /fuego-core
> > > +fi
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# get ttc script and helpers
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > > +/usr/local/src/ttc/install.sh /usr/local/bin
> > > +# as of ttc version 2.0.2, this should not be needed
> > > +#perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> > /usr/local/bin/ttc
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# get ebf script and helpers
> > > +#
> > ================================================================
> > ==============
> > > +if [ $nojenkins -eq 0 ]; then
> > > + apt-get -q=2 -V --no-install-recommends install jq
> > > + git clone https://github.com/TimesysGit/board-farm-rest-api.git
> > /usr/local/src/board-farm-rest-api
> > > + cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > > + chmod a+x /usr/local/bin/ebf
> > > +fi
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Serial Config
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/frowand/serio.git /usr/local/src/serio
> > > +if [ $nojenkins -eq 0 ]; then
> > > + cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> > name.patch \
> > > + frontend-install/0002-Output-data-from-port-during-command-
> > execution.patch \
> > > + /tmp/
> > > + patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-for-
> > serial-device-with-in-name.patch
> > > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> > during-command-execution.patch
> > > + chown -R jenkins /usr/local/src/serio
> > > +fi
> > > +cp /usr/local/src/serio/serio /usr/local/bin/
> > > +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> > > +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> > > +
> > > +git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin
> > > +[ $nojenkins -eq 0 ] && chown -R jenkins /usr/local/src/serlogin
> > > +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# fserver
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > > +ln -s /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Jenkins post installation
> > > +#
> > ================================================================
> > ==============
> > > +if [ $nojenkins -eq 0 ]; then
> > > +
> > > + # Fix jenkins issue, Java version check fails when java version is 11
> > > + JAVA_VERSION_CHECK="\$\(\$JAVA -version 2\>\&1 \| sed -n \';s/.*
> > version \"\\\(\[^.-\]*\\\)\\\.\\\(\[^.-
> > > \]*\\\).*\\\".*/\\\1\\\2\/p;')"
> > > + sed -i -e
> > "s#^JAVA_VERSION=.*#JAVA_VERSION\=\"${JAVA_VERSION_CHECK}\"#g"
> > /etc/init.d/jenkins
> > >
> > > source /etc/default/jenkins && \
> > > JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > > @@ -138,116 +250,107 @@ if [ $nojenkins -eq 0 ]; then
> > >
> > > sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > >
> > > - cp /fuego/frontend-install/install-plugins.sh \
> > > - /fuego/frontend-install/jenkins-support \
> > > - /fuego/frontend-install/clitest \
> > > + # set up Jenkins plugins
> > > + cp frontend-install/install-plugins.sh \
> > > + frontend-install/jenkins-support \
> > > + frontend-install/clitest \
> > > /usr/local/bin/
> > >
> > > - cp /fuego/frontend-install/config.xml $JENKINS_HOME/config.xml
> > > - ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > > - mkdir $JENKINS_HOME/userContent/docs
> > > - cp /fuego/docs/fuego-docs.pdf
> > $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> > > - jenkins cp /fuego/frontend-
> > install/jenkins.model.JenkinsLocationConfiguration.xml
> > > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > - sed -i -e "s#8080#$JENKINS_PORT#g"
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > - chown -R jenkins:jenkins $JENKINS_HOME/
> > > -
> > > # start and stop jenkins to pre-populate some settings
> > > service jenkins start && \
> > > sleep 30 && \
> > > service jenkins stop
> > >
> > > + # install plugins - these versions are for Jenkins version 2.164.2
> > > + # Explicitly install script-security v1.68, otherwise
> > > + # v1.74 will automatically be installed as a dependency of
> > > + # the junit plugin. Make sure to install before junit plugin.
> > > + # Explicitly install junit:1.27
> > > +
> > > # install other plugins from Jenkins update center
> > > # NOTE: not sure all of these are needed, but keep list
> > > - # compatible with 1.2.1 release for now
> > > - #/usr/local/bin/install-plugins.sh \
> > > - # ant:1.9 \
> > > - # antisamy-markup-formatter:1.5 \
> > > - # bouncycastle-api:2.17 \
> > > - # command-launcher:1.3 \
> > > - # description-setter:1.10 \
> > > - # display-url-api:2.3.1 \
> > > - # external-monitor-job:1.7 \
> > > - # greenballs:1.15 \
> > > - # icon-shim:2.0.3 \
> > > - # javadoc:1.5 \
> > > - # jdk-tool:1.2 \
> > > - # junit:1.27 \
> > > - # ldap:1.20 \
> > > - # mailer:1.23 \
> > > - # matrix-auth:2.3 \
> > > - # matrix-project:1.14 \
> > > - # pam-auth:1.4 \
> > > - # pegdown-formatter:1.3 \
> > > - # structs:1.17 \
> > > - # windows-slaves:1.4
> > > -
> > > - # install plugins for latest LTS Jenkins (currently 2.249.3)
> > > - /usr/local/bin/install-plugins.sh \
> > > - script-security \
> > > - structs \
> > > - workflow-step-api \
> > > - workflow-api \
> > > - junit \
> > > - scm-api \
> > > - ant \
> > > - antisamy-markup-formatter \
> > > - bouncycastle-api \
> > > - command-launcher \
> > > - description-setter \
> > > - display-url-api \
> > > - external-monitor-job \
> > > - greenballs \
> > > - icon-shim \
> > > - javadoc \
> > > - jdk-tool \
> > > - mailer \
> > > - matrix-auth \
> > > - matrix-project \
> > > - pam-auth \
> > > - pegdown-formatter \
> > > - windows-slaves
> > > -
> > > - cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> > $JENKINS_HOME/plugins/flot.jpi
> > > + # as compatible as possible with 1.2.1 release for now
> > > + # Do NOT change the order of the plugins, unless you know what
> > > + # you are doing. Otherwise, the install-plugins.sh script will
> > > + # install the wrong versions of other plugins based on dependency
> > > + # information in the plugin files.
> > >
> > > + #/usr/local/bin/install-plugins.sh \
> > > + # script-security:1.68 \
> > > + # structs:1.20 \
> > > + # workflow-step-api:2.22 \
> > > + # workflow-api:2.40 \
> > > + # junit:1.27 \
> > > + # scm-api:2.6.3 \
> > > + # ant:1.9 \
> > > + # antisamy-markup-formatter:1.5 \
> > > + # bouncycastle-api:2.17 \
> > > + # command-launcher:1.3 \
> > > + # description-setter:1.10 \
> > > + # display-url-api:2.3.1 \
> > > + # external-monitor-job:1.7 \
> > > + # greenballs:1.15 \
> > > + # icon-shim:2.0.3 \
> > > + # javadoc:1.5 \
> > > + # jdk-tool:1.2 \
> > > + # ldap:1.20 \
> > > + # mailer:1.23 \
> > > + # matrix-auth:2.3 \
> > > + # matrix-project:1.14 \
> > > + # pam-auth:1.5 \
> > > + # pegdown-formatter:1.3 \
> > > + # windows-slaves:1.4
> > > +
> > > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > > + /usr/local/bin/install-plugins.sh \
> > > + script-security \
> > > + structs \
> > > + workflow-step-api \
> > > + workflow-api \
> > > + junit \
> > > + scm-api \
> > > + ant \
> > > + antisamy-markup-formatter \
> > > + bouncycastle-api \
> > > + command-launcher \
> > > + description-setter \
> > > + display-url-api \
> > > + external-monitor-job \
> > > + greenballs \
> > > + icon-shim \
> > > + javadoc \
> > > + jdk-tool \
> > > + mailer \
> > > + matrix-auth \
> > > + matrix-project \
> > > + pam-auth \
> > > + pegdown-formatter \
> > > + windows-slaves
> > > +
> > > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> > $JENKINS_HOME/plugins/flot.jpi
> > > +
> > > + # jenkins should automatically unzip any plugins in the plugin dir
> > > # make the mod.js symlink well after flot is installed
> > > service jenkins start && sleep 30 && \
> > > rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > > ln -s /fuego-core/scripts/mod.js
> > $JENKINS_HOME/plugins/flot/flot/mod.js
> > >
> > > - chown -R jenkins:jenkins $JENKINS_HOME/
> > > -else
> > > - sed -i -e 's/jenkins_enabled=1/jenkins_enabled=0/g' /fuego-
> > ro/conf/fuego.conf
> > > -fi
> > > + mkdir -p $JENKINS_HOME/userContent/docs
> > > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> > docs.pdf
> > >
> > > -#
> > ================================================================
> > ==============
> > > -# get ttc script and helpers
> > > -#
> > ================================================================
> > ==============
> > > -git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > > -/usr/local/src/ttc/install.sh /usr/local/bin
> > > -perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> > /usr/local/bin/ttc
> > > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > + sed -i -e "s#8080#$JENKINS_PORT#g"
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > >
> > > -#
> > ================================================================
> > ==============
> > > -# Serial Config
> > > -#
> > ================================================================
> > ==============
> > > -if [ $nojenkins -eq 0 ]; then
> > > - /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > > -else
> > > - /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sersh'
> > > - /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > > /usr/local/bin'
> > > + chown -R jenkins:jenkins $JENKINS_HOME/
> > > fi
> > >
> > > -#
> > ================================================================
> > ==============
> > > -# fserver
> > > -#
> > ================================================================
> > ==============
> > > -git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > > -ln -s /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server
> > > -
> > > #
> > ================================================================
> > ==============
> > > # ftc post installation
> > > #
> > ================================================================
> > ==============
> > > ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > > -cp /fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > > echo ". /etc/bash_completion" >> /root/.bashrc
> > >
> > > #
> > ================================================================
> > ==============
> > > @@ -255,14 +358,3 @@ echo ". /etc/bash_completion" >> /root/.bashrc
> > > #
> > ================================================================
> > ==============
> > > ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > > ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > > -
> > > -#
> > ================================================================
> > ==============
> > > -# Small guide
> > > -#
> > ================================================================
> > ==============
> > > -echo "Run 'service netperf start' to start a netperf server"
> > > -echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > > -echo "Run 'ftc list-boards' to see the available boards"
> > > -echo "Run 'ftc list-tests' to see the available tests"
> > > -echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> > world"
> > > -echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> > Dhrystone"
> > > -echo "Run 'ftc gen-report' to get results"
> > > diff --git a/install-fuego.sh b/install-fuego.sh
> > > new file mode 100755
> > > index 0000000..7b3e010
> > > --- /dev/null
> > > +++ b/install-fuego.sh
> > > @@ -0,0 +1,109 @@
> > > +#!/bin/bash
> > > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > > +#
> > > +# Usage:
> > > +# $ sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s <suite>] [-p
> > <jenkins port>] [--nojenkins]
> > > +#
> > > +
> > > +usage(){
> > > + echo ""
> > > + echo "Usage: sudo ./install-fuego.sh [--help|-h] [-d <distribution>] [-s
> > <suite>] [-p <jenkins port>] [--nojenkins]"
> > > + echo "Install fuego on Debian filesystem."
> > > + echo ""
> > > + echo "options:"
> > > + echo " --help | -h Show usage help"
> > > + echo " --dist | -d <distribution> Specify distribution"
> > > + echo " --suite | -s <suite> Specify suite"
> > > + echo " --port | -p <port> Jenkins port"
> > > + echo " --nojenkins Install Fuego without Jenkins"
> > > + echo ""
> > > +}
> > > +
> > > +if [[ $EUID -ne 0 ]]; then
> > > + echo "Sorry, you need root permissions"
> > > + exit 1
> > > +fi
> > > +
> > > +nojenkins=
> > > +port=8090
> > > +dist=debian
> > > +suite=stretch
> > > +fuego_dir=$(pwd)
> > > +
> > > +while [ "$1" != "" ]; do
> > > + case $1 in
> > > + -d | --dist )
> > > + shift
> > > + dist=$1
> > > + ;;
> > > + -s | --suite )
> > > + shift
> > > + suite=$1
> > > + ;;
> > > + -p | --port )
> > > + shift
> > > + port=$1
> > > + ;;
> > > + --nojenkins )
> > > + nojenkins="--nojenkins"
> > > + ;;
> > > + -h | --help )
> > > + usage
> > > + exit 0
> > > + ;;
> > > + * )
> > > + break
> > > + esac
> > > + shift
> > > +done
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Clone fuego-core
> > > +#
> > ================================================================
> > ==============
> > > +if [ ! -f "fuego-core/scripts/ftc" ]; then
> > > + # set fuego-core branch to same as current fuego branch
> > > + # get current git branch
> > > + set -o noglob
> > > + while IFS=" " read -r part1 part2 ; do
> > > + if [ $part1 = "*" ] ; then
> > > + branch=$part2
> > > + fi
> > > + done < <(git branch)
> > > + set +o noglob
> > > + git clone -b $branch https://bitbucket.org/fuegotest/fuego-core.git
> > > +fi
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Install fuego on host system
> > > +#
> > ================================================================
> > ==============
> > > +./install-$dist.sh -s $suite -p $port $nojenkins
> > > +
> > > +systemctl daemon-reload
> > > +service jenkins restart
> > > +
> > > +# provide permission to current user
> > > +usermod -aG jenkins $USER
> > > +chmod -R 775 $fuego_dir/fuego-rw
> > > +
> > > +chown -R jenkins:jenkins $fuego_dir/fuego-ro
> > > +chown -R jenkins:jenkins $fuego_dir/fuego-rw
> > > +chown -R jenkins:jenkins $fuego_dir/fuego-core
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Create Links to fuego folders
> > > +#
> > ================================================================
> > ==============
> > > +ln -s $fuego_dir/fuego-ro /
> > > +ln -s $fuego_dir/fuego-rw /
> > > +ln -s $fuego_dir/fuego-core /
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Small guide
> > > +#
> > ================================================================
> > ==============
> > > +echo "Logout and Login current user session to work properly"
> > > +echo "Run 'service netperf start' to start a netperf server"
> > > +echo "Run 'iperf3 -V -s -D -f M' to start an iperf3 server"
> > > +echo "Run 'ftc list-boards' to see the available boards"
> > > +echo "Run 'ftc list-tests' to see the available tests"
> > > +echo "Run 'ftc run-test -b local -t Functional.hello_world' to run a hello
> > world"
> > > +echo "Run 'ftc run-test -b local -t Benchmark.Dhrystone -s 500M' to run
> > Dhrystone"
> > > +echo "Run 'ftc gen-report' to get results"
> > > diff --git a/install-stretch.sh b/install-stretch.sh
> > > new file mode 100755
> > > index 0000000..4f6f724
> > > --- /dev/null
> > > +++ b/install-stretch.sh
> > > @@ -0,0 +1,288 @@
> > > +#!/bin/bash
> > > +# 2019 (c) Toshiba corp. <daniel.sangorrin@toshiba.co.jp>
> > > +#
> > > +# Usage:
> > > +# $ sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> > > +#
> > > +usage(){
> > > +
> > > +cat <<HERE
> > > +Usage: sudo ./install-<debian-suit>.sh [--help|-h] [--nojenkins] [<port>]
> > > +
> > > +Install fuego on the host Debian filesystem.
> > > +
> > > +options:
> > > + --help Show usage help
> > > + --nojenkins Install Fuego without Jenkins
> > > +HERE
> > > +}
> > > +
> > > +if [[ $EUID -ne 0 ]]; then
> > > + echo "Sorry, you need root permissions"
> > > + exit 1
> > > +fi
> > > +
> > > +nojenkins=0
> > > +
> > > +POSITIONAL=()
> > > +while [[ $# -gt 0 ]] ; do
> > > + case $1 in
> > > + --nojenkins)
> > > + nojenkins=1
> > > + shift
> > > + ;;
> > > + --help | -h)
> > > + usage
> > > + exit 0
> > > + ;;
> > > + *)
> > > + POSITIONAL+=("$1") # save argument for later
> > > + shift
> > > + esac
> > > +done
> > > +set -- "${POSITIONAL[@]}" # restore positional arguments
> > > +
> > > +port=${2:-8090}
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Install Fuego dependencies
> > > +#
> > ================================================================
> > ==============
> > > +
> > > +mkdir -p /usr/share/man/man1
> > > +
> > > +# netperf is in non-free
> > > +echo deb http://deb.debian.org/debian stretch main non-free >
> > /etc/apt/sources.list
> > > +echo deb http://security.debian.org/debian-security stretch/updates main >>
> > /etc/apt/sources.list
> > > +
> > > +apt-get update -q=2
> > > +
> > > +# Fuego python dependencies
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + python-lxml python-simplejson python-yaml python-openpyxl \
> > > + python-requests python-reportlab python-parsedatetime \
> > > + python-pexpect python-pip python-setuptools python-wheel
> > > +pip install filelock
> > > +
> > > +# Fuego command dependencies
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + git sshpass openssh-client sudo net-tools wget curl lava-tool \
> > > + bash-completion iproute2
> > > +
> > > +# Default SDK for testing locally or on an x86 board
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + build-essential cmake bison flex kmod automake libtool \
> > > + libelf-dev libssl-dev libsdl1.2-dev libcairo2-dev libxmu-dev \
> > > + libxmuu-dev libglib2.0-dev libaio-dev pkg-config rsync u-boot-tools
> > > +
> > > +# Default test host dependencies
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + iperf iperf3 netperf bzip2 bc python-matplotlib python-xmltodict \
> > > + netpipe-tcp iputils-ping
> > > +pip install flake8
> > > +
> > > +# miscelaneous packages:
> > > +# python-serial - used by serio
> > > +# diffstat and vim - used by Tim
> > > +# time - useful for timing command duration
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + python-serial \
> > > + diffstat \
> > > + vim \
> > > + time
> > > +
> > > +# install packages used by NuttX SDK
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + genromfs
> > > +
> > > +# bsdmainutils provides hexdump, which is needed for netperf test
> > > +apt-get -q=2 -V --no-install-recommends install \
> > > + bsdmainutils
> > > +
> > > +echo "dash dash/sh boolean false" | debconf-set-selections
> > > +DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> > > +
> > > +# TODO: check if proxy settings can set outside docker container
> > > +#if [ -n "$http_proxy" ]; then
> > > +# sed -i -e 's/#use_proxy = on/use_proxy = on/g' /etc/wgetrc
> > > +# echo -e "http_proxy=$http_proxy\nhttps_proxy=$https_proxy" >>
> > /etc/environment
> > > +#fi
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Install Jenkins
> > > +#
> > ================================================================
> > ==============
> > > +
> > > +if [ $nojenkins -eq 0 ]; then
> > > + #JENKINS_VERSION=2.164.2
> > > + #JENKINS_SHA=4536f43f61b1fca6c58bd91040fa09304eea96ab
> > > + JENKINS_VERSION=2.249.3
> > > + JENKINS_SHA=534014c007edbb533a1833fe6f2dc115faf3faa2
> > > + JENKINS_URL=https://pkg.jenkins.io/debian-
> > stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> > > + export JENKINS_UC=https://updates.jenkins.io
> > > + REF=/var/lib/jenkins/plugins
> > > + JENKINS_HOME=/var/lib/jenkins
> > > + JENKINS_PORT=$port
> > > +
> > > + # Jenkins dependencies
> > > + apt-get -q=2 -V --no-install-recommends install \
> > > + default-jdk daemon psmisc adduser procps unzip
> > > + pip install python-jenkins==1.4.0
> > > +
> > > + echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
> > > + export JENKINS_PORT=$port
> > > +
> > > + groupadd jenkins
> > > + useradd -l -m -d "${JENKINS_HOME}" -g jenkins -G sudo -s /bin/bash
> > jenkins
> > > +
> > > + wget -nv ${JENKINS_URL}
> > > + echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> > sha1sum -c -
> > > +
> > > + # allow Jenkins to start and install plugins, as part of dpkg installation
> > > + printf "#!/bin/sh\nexit 0" > /usr/sbin/policy-rc.d
> > > + dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> > > + rm jenkins_${JENKINS_VERSION}_all.deb
> > > +
> > > + source /etc/default/jenkins && \
> > > + JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> > > + sed -i -e
> > "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> > /etc/default/jenkins
> > > +
> > > + source /etc/default/jenkins && \
> > > + JAVA_ARGS="$JAVA_ARGS -
> > Djenkins.install.runSetupWizard=false -
> > > Dhudson.model.DirectoryBrowserSupport.allowSymlinkEscape=true" && \
> > > + if [ -n "$http_proxy" ]; then \
> > > + PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://'
> > | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> > > + PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' |
> > sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> > > + JAVA_ARGS="$JAVA_ARGS -
> > Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> > > Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> > > + fi && \
> > > + sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> > /etc/default/jenkins;
> > > +
> > > + sed -i -e "s#8080#$JENKINS_PORT#g" /etc/default/jenkins
> > > +
> > > + # set up Jenkins plugins
> > > + cp frontend-install/install-plugins.sh \
> > > + frontend-install/jenkins-support \
> > > + frontend-install/clitest \
> > > + /usr/local/bin/
> > > +
> > > + mkdir -p $JENKINS_HOME/userContent/docs
> > > + cp frontend-install/config.xml $JENKINS_HOME/config.xml
> > > + ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> > > + cp docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> > docs.pdf
> > > + cp frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> > > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > + sed -i -e "s#8080#$JENKINS_PORT#g"
> > $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> > > + chown -R jenkins:jenkins $JENKINS_HOME/
> > > +
> > > + # start and stop jenkins to pre-populate some settings
> > > + service jenkins start && \
> > > + sleep 30 && \
> > > + service jenkins stop
> > > +
> > > + # install other plugins from Jenkins update center
> > > + # NOTE: not sure all of these are needed, but keep list
> > > + # compatible with 1.2.1 release for now
> > > + #/usr/local/bin/install-plugins.sh \
> > > + # ant:1.9 \
> > > + # antisamy-markup-formatter:1.5 \
> > > + # bouncycastle-api:2.17 \
> > > + # command-launcher:1.3 \
> > > + # description-setter:1.10 \
> > > + # display-url-api:2.3.1 \
> > > + # external-monitor-job:1.7 \
> > > + # greenballs:1.15 \
> > > + # icon-shim:2.0.3 \
> > > + # javadoc:1.5 \
> > > + # jdk-tool:1.2 \
> > > + # junit:1.27 \
> > > + # ldap:1.20 \
> > > + # mailer:1.23 \
> > > + # matrix-auth:2.3 \
> > > + # matrix-project:1.14 \
> > > + # pam-auth:1.4 \
> > > + # pegdown-formatter:1.3 \
> > > + # structs:1.17 \
> > > + # windows-slaves:1.4
> > > +
> > > + # install plugins for latest LTS Jenkins (currently 2.249.3)
> > > + /usr/local/bin/install-plugins.sh \
> > > + script-security \
> > > + structs \
> > > + workflow-step-api \
> > > + workflow-api \
> > > + junit \
> > > + scm-api \
> > > + ant \
> > > + antisamy-markup-formatter \
> > > + bouncycastle-api \
> > > + command-launcher \
> > > + description-setter \
> > > + display-url-api \
> > > + external-monitor-job \
> > > + greenballs \
> > > + icon-shim \
> > > + javadoc \
> > > + jdk-tool \
> > > + mailer \
> > > + matrix-auth \
> > > + matrix-project \
> > > + pam-auth \
> > > + pegdown-formatter \
> > > + windows-slaves
> > > +
> > > + cp frontend-install/plugins/flot-plotter-plugin/flot.hpi
> > $JENKINS_HOME/plugins/flot.jpi
> > > +
> > > + # make the mod.js symlink well after flot is installed
> > > + service jenkins start && sleep 30 && \
> > > + rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> > > + ln -s /fuego-core/scripts/mod.js
> > $JENKINS_HOME/plugins/flot/flot/mod.js
> > > +
> > > + chown -R jenkins:jenkins $JENKINS_HOME/
> > > +fi
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# get ttc script and helpers
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> > > +/usr/local/src/ttc/install.sh /usr/local/bin
> > > +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> > /usr/local/bin/ttc
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Serial Config
> > > +#
> > ================================================================
> > ==============
> > > +if [ $nojenkins -eq 0 ]; then
> > > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; chown -R jenkins /usr/local/src/serio ; cp
> > > /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> > > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; chown -R jenkins /usr/local/src/serlogin ;
> > > cp /usr/local/src/serlogin/serlogin /usr/local/bin'
> > > +else
> > > + /bin/bash -c 'git clone https://github.com/frowand/serio.git
> > /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -
> > > s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio
> > /usr/local/bin/sersh'
> > > + /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> > /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin
> > > /usr/local/bin'
> > > +fi
> > > +cp frontend-install/0001-Fix-host-parsing-for-serial-device-with-in-
> > name.patch \
> > > + frontend-install/0002-Output-data-from-port-during-command-
> > execution.patch \
> > > + /tmp/
> > > +/bin/bash -c 'patch -d /usr/local/src/serio -p1 </tmp/0001-Fix-host-parsing-
> > for-serial-device-with-in-name.patch ; \
> > > + patch -d /usr/local/src/serio -p1 </tmp/0002-Output-data-from-port-
> > during-command-execution.patch ;'
> > > +
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# get ebf script and helpers
> > > +#
> > ================================================================
> > ==============
> > > +apt-get -q=2 -V --no-install-recommends install jq
> > > +git clone https://github.com/TimesysGit/board-farm-rest-api.git
> > /usr/local/src/board-farm-rest-api
> > > +cp /usr/local/src/board-farm-rest-api/cli/ebf /usr/local/bin/ebf
> > > +chmod a+x /usr/local/bin/ebf
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# fserver
> > > +#
> > ================================================================
> > ==============
> > > +git clone https://github.com/tbird20d/fserver.git /usr/local/lib/fserver
> > > +ln -s /usr/local/lib/fserver/start_local_bg_server
> > /usr/local/bin/start_local_bg_server
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# ftc post installation
> > > +#
> > ================================================================
> > ==============
> > > +ln -s /fuego-core/scripts/ftc /usr/local/bin/
> > > +cp fuego-core/scripts/ftc_completion.sh /etc/bash_completion.d/ftc
> > > +echo ". /etc/bash_completion" >> /root/.bashrc
> > > +
> > > +#
> > ================================================================
> > ==============
> > > +# Lava
> > > +#
> > ================================================================
> > ==============
> > > +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> > > +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> > > diff --git a/install.sh b/install.sh
> > > index a5a90b0..259c887 100755
> > > --- a/install.sh
> > > +++ b/install.sh
> > > @@ -1,13 +1,13 @@
> > > #!/bin/bash
> > > #
> > > -# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> > [<port>]
> > > +# install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> > [<port>] [<distribution>] [<suit>]
> > > #
> > >
> > > # $1 is the exit code after usage is shown
> > > usage() {
> > > exit_code="$1"
> > > cat <<HERE
> > > -Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> > [<port>]
> > > +Usage: install.sh [--help] [--priv] [--no-cache] [--nojenkins] [<image_name>]
> > [<port>] [<distribution>] [<suit>]
> > >
> > > Create the docker image and container with the Fuego test distribution.
> > > If no <image_name> is provided, the image will be named 'fuego'.
> > > @@ -34,7 +34,7 @@ fi
> > >
> > > priv=0
> > > NOCACHE=""
> > > -dockerfile="Dockerfile"
> > > +NOJENKINS=""
> > >
> > > POSITIONAL=()
> > > while [[ $# -gt 0 ]] ; do
> > > @@ -48,7 +48,7 @@ while [[ $# -gt 0 ]] ; do
> > > shift
> > > ;;
> > > --nojenkins)
> > > - dockerfile="Dockerfile.nojenkins"
> > > + NOJENKINS="--nojenkins"
> > > shift
> > > ;;
> > > *)
> > > @@ -60,6 +60,8 @@ set -- "${POSITIONAL[@]}" # restore positional
> > arguments
> > >
> > > image_name=${1:-fuego}
> > > jenkins_port=${2:-8090}
> > > +distribution=${3:-debian}
> > > +suite=${4:-stretch}
> > >
> > > container_name="${image_name}-container"
> > >
> > > @@ -79,7 +81,7 @@ fi
> > >
> > > set -e
> > >
> > > -source fuego-host-scripts/docker-build-image.sh $NOCACHE ${image_name}
> > ${jenkins_port} ${dockerfile}
> > > +source fuego-host-scripts/docker-build-image.sh $NOCACHE
> > ${image_name} ${jenkins_port} ${distribution} ${suite} ${NOJENKINS}
> > > if [ "$priv" == "0" ]; then
> > > fuego-host-scripts/docker-create-container.sh ${image_name}
> > ${container_name}
> > > else
> > > diff --git a/uninstall-fuego.sh b/uninstall-fuego.sh
> > > new file mode 100644
> > > index 0000000..b899e7c
> > > --- /dev/null
> > > +++ b/uninstall-fuego.sh
> > > @@ -0,0 +1,47 @@
> > > +#!/bin/bash
> > > +
> > > +# Remove Jenkins plugins
> > > +[ -d /usr/share/jenkins/ref/plugins/ ] && rm -rf
> > /usr/share/jenkins/ref/plugins
> > > +
> > > +# uninstall Jenkins
> > > +service jenkins stop
> > > +dpkg -P jenkins
> > > +
> > > +[ -f /usr/local/bin/install-plugins.sh ] && rm -f /usr/local/bin/install-
> > plugins.sh
> > > +[ -f /usr/local/bin/jenkins-support ] && rm -rf /usr/local/bin/jenkins-support
> > > +[ -f /usr/local/bin/clitest ] && rm -rf /usr/local/bin/clitest
> > > +[ -d /var/lib/jenkins ] && rm -rf /var/lib/jenkins
> > > +sed -i '/^JENKINS_PORT=/d' /etc/environment
> > > +
> > > +# uninstall ttc script
> > > +[ -d /usr/local/src/ttc ] && rm -rf /usr/local/src/ttc
> > > +
> > > +# uninstall ebf script
> > > +[ -d /usr/local/src/board-farm-rest-api ] && rm -rf /usr/local/src/board-
> > farm-rest-api
> > > +[ -f /usr/local/bin/ebf ] && rm -f /usr/local/bin/ebf
> > > +
> > > +# uninstall serial config
> > > +[ -d /usr/local/src/serio ] && rm -rf /usr/local/src/serio
> > > +[ -f /usr/local/bin/serio ] && rm -rf /usr/local/bin/serio
> > > +[ -f /usr/local/bin/sercp ] && rm -rf /usr/local/bin/sercp
> > > +[ -f /usr/local/bin/sersh ] && rm -rf /usr/local/bin/sersh
> > > +[ -d /usr/local/src/serlogin ] && rm -rf /usr/local/src/serlogin
> > > +[ -f /usr/local/bin/serlogin ] && rm -rf /usr/local/bin/serlogin
> > > +
> > > +# uninstall fserver
> > > +[ -d /usr/local/lib/fserver ] && rm -f /usr/local/bin/fserver
> > > +[ -f /usr/local/bin/start_local_bg_server ] && rm -rf
> > /usr/local/bin/start_local_bg_server
> > > +
> > > +# uninstall ftc
> > > +[ -f /usr/local/bin/ftc ] && rm -rf /usr/local/bin/ftc
> > > +[ -f /etc/bash_completion.d/ftc ] && rm -rf /etc/bash_completion.d/ftc
> > > +
> > > +# remove lava target files
> > > +[ -f /usr/local/bin/fuego-lava-target-setup ] && rm -rf /usr/local/bin/fuego-
> > lava-target-setup
> > > +[ -f /usr/local/bin/fuego-lava-target-teardown ] && rm -rf
> > /usr/local/bin/fuego-lava-target-teardown
> > > +
> > > +# remove links
> > > +[ -d /fuego-ro ] && rm /fuego-ro
> > > +[ -d /fuego-rw ] && rm /fuego-rw
> > > +[ -d /fuego-core ] && rm /fuego-core
> > > +
> > > --
> > > 2.20.1
> > >
> >
> > Overall - this looks like a good start. But I'd like to see the duplicate sections of
> > install-buster.sh
> > and install-stretch.sh put into a single location "install-debian-common.sh".
> > Also, I'd like to see
> > the non-debian stuff (ttc, serio, ebf, etc.) put into "install-common.sh".
> >
> > Can you please re-do this with that refactoring in mind?
> Thank you for the feedback, I will work on these refactoring and send you another patch.
>
> >
> > Thanks,
> > -- Tim
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-06-17 20:53 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-04 13:43 [Fuego] Use common script to install fuego on host or docker system venkata.pyla
2021-06-04 14:25 ` Venkata.Pyla
2021-06-09 3:35 ` Tim.Bird
2021-06-15 0:08 ` Tim.Bird
2021-06-15 3:46 ` Venkata.Pyla
2021-06-16 17:47 ` Tim.Bird
2021-06-17 7:01 ` Venkata.Pyla
2021-06-17 20:53 ` Tim.Bird
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.