[Fuego] [PATCH 0/9] Docker Improvements

[Fuego] [PATCH 0/9] Docker Improvements

[Guilherme Campos Camargo]

Hello, everyone

This series of patches implements a few improvements on the Docker
infrastructure with the ultimate goal of allowing an image to be shared
amoung different users.

One of the benefits of this approach is that it allows the
build-from-scratch to be optional, providing the user much faster set-up
times.

The changes involve, in short:

    * Reorganization of the Dockerfile aiming to reduce the
    number of layers present in the resulting image (by reducing the
    number of RUN/COPY statements) and to reduce cache misses by moving
    components that change less frequently to the beginning of the
    Dockerfile.

    * Remove persistenct of proxy and other configurations that might
    result in conflicts to the user of the image (that's not using it at
    the same host where the image was built).

We tried to provide an in-depth reasoning for each one of the proposed
changes in the commit messages and we'll be glad to hear your inputs
about them.

Thanks

--
Guilherme

ProFUSION Embedded Systems


Guilherme Campos Camargo (9):
  Remove unused variables from docker build/create/run scripts
  Refactor preinst and apt-get/pip install
  Use curl for downloading Jenkins
  Move configuration and jenkins plugin installation to separate scripts
  Remove jenkins user created during build time with host's uid/gid
  Cleanup proxy
  Move dpkg-reconfigure dash to first RUN instruction
  Refactor plugin installation script
  Set depth to "one" and download just a single-branch on git clones

 Dockerfile                                         | 215 +++++++++++----------
 .../apt/sources/fuego-debian-jessie.list           |   2 +
 frontend-install/entrypoint.sh                     |  13 ++
 frontend-install/{ => setup/jenkins}/config.xml    |   0
 .../jenkins.model.JenkinsLocationConfiguration.xml |   0
 .../setup/jenkins/map-jenkins-uid-to-host.sh       |  12 ++
 .../plugins/flot-plotter-plugin/.gitignore         |   0
 .../jenkins}/plugins/flot-plotter-plugin/flot.hpi  | Bin
 .../jenkins}/plugins/flot-plotter-plugin/pom.xml   |   0
 .../src/main/java/flotile/ChartAction.java         |   0
 .../src/main/java/flotile/FlotPublisher.java       |   0
 .../flotile/ChartAction/floatingBox.jelly          |   0
 .../src/main/resources/index.jelly                 |   0
 .../src/main/webapp/flot/.copyarea.db              |   0
 .../src/main/webapp/flot/API.txt                   |   0
 .../src/main/webapp/flot/FAQ.txt                   |   0
 .../src/main/webapp/flot/LICENSE.txt               |   0
 .../src/main/webapp/flot/PLUGINS.txt               |   0
 .../src/main/webapp/flot/README.txt                |   0
 .../src/main/webapp/flot/base64.js                 |   0
 .../src/main/webapp/flot/canvas2image.js           |   0
 .../src/main/webapp/flot/excanvas.min.js           |   0
 .../main/webapp/flot/jquery.colorhelpers.min.js    |   0
 .../main/webapp/flot/jquery.flot.crosshair.min.js  |   0
 .../webapp/flot/jquery.flot.fillbetween.min.js     |   0
 .../src/main/webapp/flot/jquery.flot.image.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.min.js        |   0
 .../main/webapp/flot/jquery.flot.navigate.min.js   |   0
 .../src/main/webapp/flot/jquery.flot.pie.min.js    |   0
 .../src/main/webapp/flot/jquery.flot.resize.min.js |   0
 .../main/webapp/flot/jquery.flot.selection.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.stack.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.symbol.min.js |   0
 .../main/webapp/flot/jquery.flot.threshold.min.js  |   0
 .../src/main/webapp/flot/jquery.min.js             |   0
 .../src/main/webapp/flot/mod.js                    |   0
 .../src/main/webapp/flot/my.css                    |   0
 .../src/main/webapp/help-globalConfig.html         |   0
 frontend-install/setup/jenkins/plugins/install.sh  |  43 +++++
 .../setup/jenkins/set-java-args-proxy.sh           |   7 +
 frontend-install/setup/jenkins/setup.sh            |  11 ++
 frontend-install/setup/lava/setup.sh               |  11 ++
 frontend-install/setup/serial/setup-serio.sh       |   8 +
 frontend-install/setup/serial/setup-serlogin.sh    |   6 +
 frontend-install/setup/serial/setup-ttc.sh         |   7 +
 fuego-host-scripts/docker-build-image.sh           |  21 +-
 fuego-host-scripts/docker-create-container.sh      |   2 +
 .../docker-create-usb-privileged-container.sh      |   2 +
 fuego-host-scripts/docker-remove-container.sh      |   8 -
 fuego-host-scripts/docker-remove-image.sh          |   9 +-
 fuego-host-scripts/docker-start-container.sh       |   8 -
 fuego-host-scripts/jenkins-map-uid.sh              |  18 ++
 52 files changed, 258 insertions(+), 145 deletions(-)
 create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
 create mode 100755 frontend-install/entrypoint.sh
 rename frontend-install/{ => setup/jenkins}/config.xml (100%)
 rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
 create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/.gitignore (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/flot.hpi (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/pom.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/index.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html (100%)
 create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
 create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
 create mode 100755 frontend-install/setup/jenkins/setup.sh
 create mode 100755 frontend-install/setup/lava/setup.sh
 create mode 100755 frontend-install/setup/serial/setup-serio.sh
 create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
 create mode 100755 frontend-install/setup/serial/setup-ttc.sh
 create mode 100755 fuego-host-scripts/jenkins-map-uid.sh

-- 
2.15.1

[Fuego] [PATCH 1/9] Remove unused variables from docker build/create/run scripts

[Guilherme Campos Camargo]

The DIR variable is only being used by the CREATE scripts. There's no
reason to keep it in the BUILD/REMOVE/START scripts as well.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 fuego-host-scripts/docker-build-image.sh      | 9 +--------
 fuego-host-scripts/docker-remove-container.sh | 8 --------
 fuego-host-scripts/docker-remove-image.sh     | 9 +--------
 fuego-host-scripts/docker-start-container.sh  | 8 --------
 4 files changed, 2 insertions(+), 32 deletions(-)

diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
index a197dc5..7c11b11 100755
--- a/fuego-host-scripts/docker-build-image.sh
+++ b/fuego-host-scripts/docker-build-image.sh
@@ -1,12 +1,5 @@
+#!/bin/bash
 # $1 - name for the docker image (default: fuego)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERIMAGE=${1:-fuego}
 
 if [ "$(id -u)" == "0" ]; then
diff --git a/fuego-host-scripts/docker-remove-container.sh b/fuego-host-scripts/docker-remove-container.sh
index 2aae6a9..00c7ac9 100755
--- a/fuego-host-scripts/docker-remove-container.sh
+++ b/fuego-host-scripts/docker-remove-container.sh
@@ -1,13 +1,5 @@
 #!/bin/bash
 # $1 - name for the docker container (default: fuego-container)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERCONTAINER=${1:-fuego-container}
 
 sudo docker rm ${DOCKERCONTAINER}
diff --git a/fuego-host-scripts/docker-remove-image.sh b/fuego-host-scripts/docker-remove-image.sh
index 91dbda7..4c24684 100755
--- a/fuego-host-scripts/docker-remove-image.sh
+++ b/fuego-host-scripts/docker-remove-image.sh
@@ -1,12 +1,5 @@
+#!/bin/bash
 # $1 - name for the docker image (default: fuego)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERIMAGE=${1:-fuego}
 
 sudo docker rmi ${DOCKERIMAGE}
diff --git a/fuego-host-scripts/docker-start-container.sh b/fuego-host-scripts/docker-start-container.sh
index d9de85a..d0cee1a 100755
--- a/fuego-host-scripts/docker-start-container.sh
+++ b/fuego-host-scripts/docker-start-container.sh
@@ -1,13 +1,5 @@
 #!/bin/bash
 # $1 - name for the docker container (default: fuego-container)
-SOURCE="${BASH_SOURCE[0]}"
-while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a symlink
-  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-  SOURCE="$(readlink "$SOURCE")"
-  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a relative symlink, we need to resolve it relative to the path where the symlink file was located
-done
-DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
-
 DOCKERCONTAINER=${1:-fuego-container}
 
 echo "Starting Fuego container (${DOCKERCONTAINER})"
-- 
2.15.1

[Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

[Guilherme Campos Camargo]

All base-image related apt-get commands (including pre-install config,
as well as the package install) have been put together in the same RUN
instruction.

The same has been done with the LAVA-related apt-get commands on this
patch. They can't be merged because the requirements are using a
specific key that needs to be downloaded with curl (that's installed in
the previous step. Additionally, LAVA apt-get commands have been moved
up in the Dockerfile, allowing that RUN instruction to be invalidated
fewer times, because it does not depend on the files that have been
copied from the host.

We have also moved apt sources to an external .list file, that's copied
from the host to the container during build.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile                                         | 116 ++++++++++++++++-----
 .../apt/sources/fuego-debian-jessie.list           |   2 +
 2 files changed, 92 insertions(+), 26 deletions(-)
 create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list

diff --git a/Dockerfile b/Dockerfile
index f34fe4b..b8d78e6 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -19,25 +19,96 @@ ENV https_proxy ${HTTP_PROXY}
 # ==============================================================================
 
 WORKDIR /
-RUN echo deb http://httpredir.debian.org/debian jessie main non-free > /etc/apt/sources.list
-RUN echo deb http://httpredir.debian.org/debian jessie-updates main non-free >> /etc/apt/sources.list
-RUN echo deb http://security.debian.org/ jessie/updates main >> /etc/apt/sources.list
-RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install \
-	apt-utils daemon gcc make cmake python-paramiko python-lxml python-simplejson \
-	python-matplotlib python-serial python-yaml python-openpyxl python-requests \
-	libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf \
-	netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \
-	inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev \
-	time python-pip python-xmltodict at minicom lzop bsdmainutils u-boot-tools \
-	mc netcat lava-tool openssh-server python-parsedatetime \
-	libsdl1.2-dev libcairo2-dev libxmu-dev libxmuu-dev
-
-RUN pip install python-jenkins==0.4.14
-RUN pip install filelock
-RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'
-RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
-RUN if [ -n "$HTTP_PROXY" ]; then echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; fi
+COPY frontend-install/apt/sources/fuego-debian-jessie.list \
+        /etc/apt/sources.list.d/fuego-debian-jessie.list
+RUN if [ -n "$HTTP_PROXY" ]; then \
+        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \
+    fi && \
+    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get -yV install \
+        apt-utils \
+        at \
+        autoconf \
+        automake \
+        bc \
+        bsdmainutils \
+        bzip2 \
+        cmake \
+        curl \
+        daemon \
+        diffstat \
+        g++ \
+        gcc \
+        gettext \
+        git \
+        inotify-tools \
+        iperf \
+        lava-tool \
+        libaio-dev \
+        libcairo2-dev \
+        libglib2.0-dev \
+        libsdl1.2-dev \
+        libtool \
+        libxmu-dev \
+        libxmuu-dev \
+        lzop \
+        make \
+        mc \
+        minicom \
+        net-tools \
+        netcat \
+        netperf \
+        netpipe-tcp \
+        openjdk-7-jdk \
+        openjdk-7-jre \
+        openssh-server \
+        pkg-config \
+        python-lxml \
+        python-matplotlib \
+        python-openpyxl \
+        python-paramiko \
+        python-parsedatetime \
+        python-pip \
+        python-requests \
+        python-serial \
+        python-simplejson \
+        python-xmltodict \
+        python-yaml \
+        rsync \
+        sshpass \
+        sudo \
+        time \
+        u-boot-tools \
+        vim \
+        wget \
+        xmlstarlet && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
+    if [ -n "$HTTP_PROXY" ]; then \
+        echo "use_proxy = on" >> /etc/wgetrc; \
+        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \
+    fi
+
+RUN pip install \
+        filelock \
+        python-jenkins==0.4.14
+
+# TODO: Move toolchain-related instalation steps to a derivate image, like fuego:${version}-arhmhf
+RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \
+    curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \
+    dpkg --add-architecture armhf && \
+    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get -yV install \
+        binutils-arm-linux-gnueabihf \
+        cpp-arm-linux-gnueabihf \
+        crossbuild-essential-armhf \
+        gcc-arm-linux-gnueabihf && \
+    rm -rf /var/lib/apt/lists/*
+
+RUN pip install \
+        filelock \
+        python-jenkins==0.4.14
 
 # ==============================================================================
 # Install Jenkins with the same UID/GID as the host user
@@ -125,13 +196,6 @@ RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
 #RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
 #RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
 
-RUN echo "deb http://emdebian.org/tools/debian/ jessie main" > /etc/apt/sources.list.d/crosstools.list
-RUN dpkg --add-architecture armhf
-RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add -
-RUN DEBIAN_FRONTEND=noninteractive apt-get update
-RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf cpp-arm-linux-gnueabihf gcc-arm-linux-gnueabihf binutils-arm-linux-gnueabihf
-
-
 # ==============================================================================
 # Setup startup command
 # ==============================================================================
diff --git a/frontend-install/apt/sources/fuego-debian-jessie.list b/frontend-install/apt/sources/fuego-debian-jessie.list
new file mode 100644
index 0000000..d224f15
--- /dev/null
+++ b/frontend-install/apt/sources/fuego-debian-jessie.list
@@ -0,0 +1,2 @@
+deb http://httpredir.debian.org/debian jessie main non-free
+deb http://httpredir.debian.org/debian jessie-updates main non-free
-- 
2.15.1

[Fuego] [PATCH 3/9] Use curl for downloading Jenkins

[Guilherme Campos Camargo]

As curl had been used previously in the Dockerfile, let's make it our
preferred tool for downloading resources in your scripts.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Dockerfile b/Dockerfile
index b8d78e6..3dbae0f 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -125,7 +125,7 @@ ENV JENKINS_HOME=/var/lib/jenkins
 
 RUN groupadd -g ${gid} ${group} \
 	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN wget -nv ${JENKINS_URL}
+RUN curl -L -O ${JENKINS_URL}
 RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
 RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
 RUN rm jenkins_${JENKINS_VERSION}_all.deb
-- 
2.15.1

[Fuego] [PATCH 4/9] Move configuration and jenkins plugin installation to separate scripts

[Guilherme Campos Camargo]

In order to simplify the structure of the Dockerfile and optimize build
times.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile                                         |  96 ++++++---------------
 frontend-install/{ => setup/jenkins}/config.xml    |   0
 .../jenkins.model.JenkinsLocationConfiguration.xml |   0
 .../plugins/flot-plotter-plugin/.gitignore         |   0
 .../jenkins}/plugins/flot-plotter-plugin/flot.hpi  | Bin
 .../jenkins}/plugins/flot-plotter-plugin/pom.xml   |   0
 .../src/main/java/flotile/ChartAction.java         |   0
 .../src/main/java/flotile/FlotPublisher.java       |   0
 .../flotile/ChartAction/floatingBox.jelly          |   0
 .../src/main/resources/index.jelly                 |   0
 .../src/main/webapp/flot/.copyarea.db              |   0
 .../src/main/webapp/flot/API.txt                   |   0
 .../src/main/webapp/flot/FAQ.txt                   |   0
 .../src/main/webapp/flot/LICENSE.txt               |   0
 .../src/main/webapp/flot/PLUGINS.txt               |   0
 .../src/main/webapp/flot/README.txt                |   0
 .../src/main/webapp/flot/base64.js                 |   0
 .../src/main/webapp/flot/canvas2image.js           |   0
 .../src/main/webapp/flot/excanvas.min.js           |   0
 .../main/webapp/flot/jquery.colorhelpers.min.js    |   0
 .../main/webapp/flot/jquery.flot.crosshair.min.js  |   0
 .../webapp/flot/jquery.flot.fillbetween.min.js     |   0
 .../src/main/webapp/flot/jquery.flot.image.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.min.js        |   0
 .../main/webapp/flot/jquery.flot.navigate.min.js   |   0
 .../src/main/webapp/flot/jquery.flot.pie.min.js    |   0
 .../src/main/webapp/flot/jquery.flot.resize.min.js |   0
 .../main/webapp/flot/jquery.flot.selection.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.stack.min.js  |   0
 .../src/main/webapp/flot/jquery.flot.symbol.min.js |   0
 .../main/webapp/flot/jquery.flot.threshold.min.js  |   0
 .../src/main/webapp/flot/jquery.min.js             |   0
 .../src/main/webapp/flot/mod.js                    |   0
 .../src/main/webapp/flot/my.css                    |   0
 .../src/main/webapp/help-globalConfig.html         |   0
 frontend-install/setup/jenkins/plugins/install.sh  |  21 +++++
 frontend-install/setup/jenkins/setup.sh            |  26 ++++++
 frontend-install/setup/lava/setup.sh               |  11 +++
 frontend-install/setup/serial/setup-serio.sh       |   8 ++
 frontend-install/setup/serial/setup-serlogin.sh    |   6 ++
 frontend-install/setup/serial/setup-ttc.sh         |   7 ++
 41 files changed, 105 insertions(+), 70 deletions(-)
 rename frontend-install/{ => setup/jenkins}/config.xml (100%)
 rename frontend-install/{ => setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/.gitignore (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/flot.hpi (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/pom.xml (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/resources/index.jelly (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css (100%)
 rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html (100%)
 create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
 create mode 100755 frontend-install/setup/jenkins/setup.sh
 create mode 100755 frontend-install/setup/lava/setup.sh
 create mode 100755 frontend-install/setup/serial/setup-serio.sh
 create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
 create mode 100755 frontend-install/setup/serial/setup-ttc.sh

diff --git a/Dockerfile b/Dockerfile
index 3dbae0f..883ac21 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -106,95 +106,51 @@ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li
         gcc-arm-linux-gnueabihf && \
     rm -rf /var/lib/apt/lists/*
 
-RUN pip install \
-        filelock \
-        python-jenkins==0.4.14
-
 # ==============================================================================
-# Install Jenkins with the same UID/GID as the host user
+# Download and Install Jenkins
 # ==============================================================================
 
-ARG user=jenkins
-ARG group=jenkins
-ARG uid=1000
-ARG gid=${uid}
+ENV uid=1000
+ENV gid=${uid}
 ARG JENKINS_VERSION=2.32.1
 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
 ENV JENKINS_HOME=/var/lib/jenkins
 
-RUN groupadd -g ${gid} ${group} \
-	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-RUN curl -L -O ${JENKINS_URL}
-RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
-RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
-RUN rm jenkins_${JENKINS_VERSION}_all.deb
+RUN groupadd -g ${gid} ${group} && \
+    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} && \
+    curl -L -O ${JENKINS_URL} && \
+    echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \
+    dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
+    rm jenkins_${JENKINS_VERSION}_all.deb
 
 # ==============================================================================
-# get ttc script and helpers
+# Post installation
 # ==============================================================================
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
-RUN /usr/local/src/ttc/install.sh /usr/local/bin
-RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" /usr/local/bin/ttc
 
-# ==============================================================================
-# Serial Config
-# ==============================================================================
+COPY frontend-install/setup/jenkins /setup/jenkins
+WORKDIR /setup/jenkins
+RUN ./setup.sh
 
-RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git /usr/local/src/serio ;  chown -R jenkins /usr/local/src/serio ; cp /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
+WORKDIR /setup/jenkins/plugins
+RUN ./install.sh
 
-RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin ;  chown -R jenkins /usr/local/src/serlogin ; cp /usr/local/src/serlogin/serlogin /usr/local/bin'
+COPY frontend-install/setup/serial /setup/serial
+WORKDIR /setup
 
-# ==============================================================================
-# Post installation
-# ==============================================================================
+RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
+    ./serial/setup-ttc.sh
 
-RUN source /etc/default/jenkins && \
-	JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
-	sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
-
-RUN source /etc/default/jenkins && \
-	JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false" && \
-	if [ -n "$HTTP_PROXY" ]; then \
-		PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
-		PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
-		JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
-	fi && \
-	sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins;
-
-COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi /tmp
-
-RUN service jenkins start && \
-	sleep 30 && \
-	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter && \
-	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter && \
-    sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin /tmp/flot.hpi && \
-    sleep 10
-
-# Let Jenkins install flot before making the symlink
-RUN service jenkins restart && sleep 30 && \
-    rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
-    ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-
-RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
+RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \
+    ./serial/setup-serio.sh
 
-RUN ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
-COPY frontend-install/config.xml $JENKINS_HOME/config.xml
-COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
+RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \
+    ./serial/setup-serlogin.sh
 
-RUN chown -R jenkins:jenkins $JENKINS_HOME/
+COPY frontend-install/setup/lava /setup/lava
+RUN ./lava/setup.sh
 
-# ==============================================================================
-# Lava
-# ==============================================================================
-
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
-RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
-# CONVENIENCE HACKS
-# not mounted, yet
-#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
-#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
+COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
 
 # ==============================================================================
 # Setup startup command
diff --git a/frontend-install/config.xml b/frontend-install/setup/jenkins/config.xml
similarity index 100%
rename from frontend-install/config.xml
rename to frontend-install/setup/jenkins/config.xml
diff --git a/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml b/frontend-install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml
similarity index 100%
rename from frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
rename to frontend-install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml
diff --git a/frontend-install/plugins/flot-plotter-plugin/.gitignore b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/.gitignore
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore
diff --git a/frontend-install/plugins/flot-plotter-plugin/flot.hpi b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/flot.hpi
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi
diff --git a/frontend-install/plugins/flot-plotter-plugin/pom.xml b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/pom.xml
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/ChartAction.java
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/java/flotile/FlotPublisher.java
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/resources/index.jelly b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/index.jelly
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/resources/index.jelly
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/resources/index.jelly
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/.copyarea.db
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/API.txt
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/FAQ.txt
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/LICENSE.txt
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/PLUGINS.txt
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/README.txt
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/base64.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/canvas2image.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/excanvas.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.image.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.pie.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.resize.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.selection.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.stack.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/jquery.min.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/mod.js
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/flot/my.css
diff --git a/frontend-install/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html b/frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html
similarity index 100%
rename from frontend-install/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html
rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help-globalConfig.html
diff --git a/frontend-install/setup/jenkins/plugins/install.sh b/frontend-install/setup/jenkins/plugins/install.sh
new file mode 100755
index 0000000..0db8bf2
--- /dev/null
+++ b/frontend-install/setup/jenkins/plugins/install.sh
@@ -0,0 +1,21 @@
+#!/bin/bash
+
+set -e
+
+service jenkins start
+sleep 30
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter
+sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi
+sleep 10
+
+# Let Jenkins install flot before making the symlink
+service jenkins restart
+sleep 30
+rm $JENKINS_HOME/plugins/flot/flot/mod.js
+ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
+ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
+ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
+
+chown -R jenkins:jenkins $JENKINS_HOME/
+
diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-install/setup/jenkins/setup.sh
new file mode 100755
index 0000000..c550862
--- /dev/null
+++ b/frontend-install/setup/jenkins/setup.sh
@@ -0,0 +1,26 @@
+#!/bin/bash
+
+set -e
+
+user=jenkins
+group=jenkins
+uid=1000
+gid=${uid}
+
+# groupadd -g ${gid} ${group}
+# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
+
+cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}
+
+source /etc/default/jenkins
+JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
+sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+
+JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
+if [ -n "$HTTP_PROXY" ]; then
+	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/');
+	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/');
+	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}";
+fi
+sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins
+
diff --git a/frontend-install/setup/lava/setup.sh b/frontend-install/setup/lava/setup.sh
new file mode 100755
index 0000000..86df49f
--- /dev/null
+++ b/frontend-install/setup/lava/setup.sh
@@ -0,0 +1,11 @@
+#!/bin/bash
+
+set -e
+
+# CONVENIENCE HACKS
+# not mounted, yet
+#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
+#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
+
+ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
+ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
diff --git a/frontend-install/setup/serial/setup-serio.sh b/frontend-install/setup/serial/setup-serio.sh
new file mode 100755
index 0000000..7e25a96
--- /dev/null
+++ b/frontend-install/setup/serial/setup-serio.sh
@@ -0,0 +1,8 @@
+#!/bin/bash
+
+set -e
+
+chown -R jenkins:jenkins /usr/local/src/serio
+cp /usr/local/src/serio/serio /usr/local/bin/
+ln -s /usr/local/bin/serio /usr/local/bin/sercp
+ln -s /usr/local/bin/serio /usr/local/bin/sersh
diff --git a/frontend-install/setup/serial/setup-serlogin.sh b/frontend-install/setup/serial/setup-serlogin.sh
new file mode 100755
index 0000000..0de17ea
--- /dev/null
+++ b/frontend-install/setup/serial/setup-serlogin.sh
@@ -0,0 +1,6 @@
+#!/bin/bash
+
+set -e
+
+chown -R jenkins:jenkins /usr/local/src/serlogin
+cp /usr/local/src/serlogin/serlogin /usr/local/bin
diff --git a/frontend-install/setup/serial/setup-ttc.sh b/frontend-install/setup/serial/setup-ttc.sh
new file mode 100755
index 0000000..d32b73e
--- /dev/null
+++ b/frontend-install/setup/serial/setup-ttc.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+set -e
+
+/usr/local/src/ttc/install.sh /usr/local/bin
+perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" \
+    /usr/local/bin/ttc
-- 
2.15.1

[Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid

[Guilherme Campos Camargo]

Prior to this patch, the fuego-host-scripts/docker-build-image.sh script
was calling docker build passing two `build-arg`s: uid/gid. According to
the comments in the code, it seems that the intention was to make sure
that the jenkins user (needed by Jenkins) would have the same uid/gid as
the user of the host that has called the install script.

Given that the uid/gid of the jenkins user was being set at build time,
the generated image would be configured only for the machine where it's
built, what would prevent jenkins to change files in the /fuego-rw
mountpoint in different hosts.

One approach for allowing jenkins to write to fuego-rw, is to chown
/fuego-rw and it's pre-existing subdirectories during execution time,
what can be done in a docker entrypoint script. That approach per se,
solves the problem, by allowing jenkins to write to the mount point, but
generates a side-effect: the user on the host machine will be able to
inspect, but will not be able to delete/modify the files created by
jenkins (unless it forces it as root).

Another approach would be to change the uid and the gid of the jenkins
user in the first execution, (first call to docker-container-start.sh),
also through the entrypoint.  This would solve the problem of the access
from both sides, but has the downside of the need of running a recursive
`chown` in all jenkins files in `/var` (including cache), possibly
taking a long time to be accomplished.

Since the need to edit/remove the files from fuego-rw (without sudo)
depends on the user, our suggestion is to use the entrypoint for
`chowning` only the pre-existing directories from fuego-rw and to
provide the user a script for mapping the jenkins' uid/gid with user's
uid/gid only if the user needs to.

On this patch we have created the entrypoint.sh script, and the uid/gid
mapping script.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile                                             | 10 ++++------
 frontend-install/entrypoint.sh                         | 13 +++++++++++++
 .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12 ++++++++++++
 frontend-install/setup/jenkins/setup.sh                |  8 --------
 fuego-host-scripts/docker-build-image.sh               | 10 +---------
 fuego-host-scripts/jenkins-map-uid.sh                  | 18 ++++++++++++++++++
 6 files changed, 48 insertions(+), 23 deletions(-)
 create mode 100755 frontend-install/entrypoint.sh
 create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
 create mode 100755 fuego-host-scripts/jenkins-map-uid.sh

diff --git a/Dockerfile b/Dockerfile
index 883ac21..493414e 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.li
 # Download and Install Jenkins
 # ==============================================================================
 
-ENV uid=1000
-ENV gid=${uid}
 ARG JENKINS_VERSION=2.32.1
 ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
 ARG JENKINS_URL=https://pkg.jenkins.io/debian-stable/binary/jenkins_${JENKINS_VERSION}_all.deb
 ENV JENKINS_HOME=/var/lib/jenkins
 
-RUN groupadd -g ${gid} ${group} && \
-    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user} && \
-    curl -L -O ${JENKINS_URL} && \
+RUN curl -L -O ${JENKINS_URL} && \
     echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c - && \
     dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
     rm jenkins_${JENKINS_VERSION}_all.deb
@@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-docs.pdf
 # Setup startup command
 # ==============================================================================
 
-ENTRYPOINT service jenkins start && service netperf start && /bin/bash
+WORKDIR /
+COPY frontend-install/entrypoint.sh /
+ENTRYPOINT ["/entrypoint.sh"]
diff --git a/frontend-install/entrypoint.sh b/frontend-install/entrypoint.sh
new file mode 100755
index 0000000..281b0fe
--- /dev/null
+++ b/frontend-install/entrypoint.sh
@@ -0,0 +1,13 @@
+#!/bin/bash
+set -e
+
+service jenkins stop >> /dev/null
+
+mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+chown jenkins:jenkins \
+    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
+
+service jenkins start
+service netperf start
+
+exec /bin/bash
diff --git a/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh b/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
new file mode 100755
index 0000000..002f165
--- /dev/null
+++ b/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
@@ -0,0 +1,12 @@
+#!/bin/bash
+set -e
+
+service jenkins stop
+
+usermod -u "${jenkins_uid}" jenkins
+groupmod -g "${jenkins_gid}" jenkins
+
+chown -R "${jenkins_uid}":"${jenkins_gid}" \
+    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
+
+service jenkins start
diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-install/setup/jenkins/setup.sh
index c550862..11ad371 100755
--- a/frontend-install/setup/jenkins/setup.sh
+++ b/frontend-install/setup/jenkins/setup.sh
@@ -2,14 +2,6 @@
 
 set -e
 
-user=jenkins
-group=jenkins
-uid=1000
-gid=${uid}
-
-# groupadd -g ${gid} ${group}
-# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
-
 cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}
 
 source /etc/default/jenkins
diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
index 7c11b11..6550a98 100755
--- a/fuego-host-scripts/docker-build-image.sh
+++ b/fuego-host-scripts/docker-build-image.sh
@@ -2,12 +2,4 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-if [ "$(id -u)" == "0" ]; then
-	JENKINS_UID=$(id -u $SUDO_USER)
-	JENKINS_GID=$(id -g $SUDO_USER)
-else
-	JENKINS_UID=$(id -u $USER)
-	JENKINS_GID=$(id -g $USER)
-fi
-
-sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg gid=$JENKINS_GID .
+sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .
diff --git a/fuego-host-scripts/jenkins-map-uid.sh b/fuego-host-scripts/jenkins-map-uid.sh
new file mode 100755
index 0000000..fad619e
--- /dev/null
+++ b/fuego-host-scripts/jenkins-map-uid.sh
@@ -0,0 +1,18 @@
+#!/bin/bash
+# $1 - name for the docker container (default: fuego-container)
+DOCKERCONTAINER=${1:-fuego-container}
+
+if [ "${UID}" == "0" ]; then
+	uid=$(id -u "${SUDO_USER}")
+	gid=$(id -g "${SUDO_USER}")
+else
+	uid="${UID}"
+	gid=$(id -g "${USER}")
+fi
+
+sudo docker start "${DOCKERCONTAINER}" || \
+  echo "Please create Fuego docker container via docker-create-container.sh script"
+
+sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
+    "${DOCKERCONTAINER}" \
+    /setup/jenkins/map-jenkins-uid-to-host.sh
-- 
2.15.1

[Fuego] [PATCH 6/9] Cleanup proxy

[Guilherme Campos Camargo]

Prior to this patch, http_proxy was being set and saved during build
time according to the http_proxy values of the host.

This approach has at least two shortcomings, that are explained below:

1. Any other user that wanted to use a prebuilt image in their host
would be unable to do so, because the proxy configuration is matching
the host of the builder and not the host of the user.

2. A user of the image would be able to read the proxy configuration
that had been set during build from `/etc/default/jenkins` or the
apt-get config file. The proxy config may easily contain sensitive
information as the proxy user/password that were set on the host that
built the image.

In order to avoid the problems pointed out above, we chose to:

1. Remove the ARG/ENV instantiations of http_proxy from the Dockerfile
and use only the builtin docker ARG http_proxy. This will prevent the
proxy variable to be available in a running container and in the image
history.

2. Instead of using the apt-get global config file, we're just passing
http_proxy as a local environment variable to the RUN instruction in
which apt-get is being executed.

3. Jenkins /etc/default/jenkins JAVA_ARGS (that contain the proxy
information) is now being updated by a script that's been source by
/etc/default/jenkins itself. This means that, whenever `service jenkins`
is called, that script will run and append the proxy configurations to
the JAVA_ARGS at runtime, not replacing the JAVA_ARGS that are stored in
the file.

4. http_proxy variables are being passed as arguments to `docker create`
as well. By doing that, we make sure that the entrypoint will be running
with the http_proxy values of the user.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile                                            | 19 ++-----------------
 frontend-install/setup/jenkins/set-java-args-proxy.sh |  7 +++++++
 frontend-install/setup/jenkins/setup.sh               | 15 ++++-----------
 fuego-host-scripts/docker-build-image.sh              |  4 +++-
 fuego-host-scripts/docker-create-container.sh         |  2 ++
 .../docker-create-usb-privileged-container.sh         |  2 ++
 6 files changed, 20 insertions(+), 29 deletions(-)
 create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh

diff --git a/Dockerfile b/Dockerfile
index 493414e..c39454d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -6,14 +6,6 @@
 FROM debian:jessie
 MAINTAINER tim.bird@sony.com
 
-# ==============================================================================
-# Proxy variables
-# ==============================================================================
-
-ARG HTTP_PROXY
-ENV http_proxy ${HTTP_PROXY}
-ENV https_proxy ${HTTP_PROXY}
-
 # ==============================================================================
 # Prepare basic image
 # ==============================================================================
@@ -21,10 +13,7 @@ ENV https_proxy ${HTTP_PROXY}
 WORKDIR /
 COPY frontend-install/apt/sources/fuego-debian-jessie.list \
         /etc/apt/sources.list.d/fuego-debian-jessie.list
-RUN if [ -n "$HTTP_PROXY" ]; then \
-        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; \
-    fi && \
-    DEBIAN_FRONTEND=noninteractive apt-get update && \
+RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
     apt-get -yV install \
         apt-utils \
         at \
@@ -84,11 +73,7 @@ RUN if [ -n "$HTTP_PROXY" ]; then \
         xmlstarlet && \
     rm -rf /var/lib/apt/lists/*
 
-RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
-    if [ -n "$HTTP_PROXY" ]; then \
-        echo "use_proxy = on" >> /etc/wgetrc; \
-        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >> /etc/environment; \
-    fi
+RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
 
 RUN pip install \
         filelock \
diff --git a/frontend-install/setup/jenkins/set-java-args-proxy.sh b/frontend-install/setup/jenkins/set-java-args-proxy.sh
new file mode 100755
index 0000000..9b04f40
--- /dev/null
+++ b/frontend-install/setup/jenkins/set-java-args-proxy.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+if [ -n "${http_proxy}" ]; then
+	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/');
+	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/');
+	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}";
+fi
diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-install/setup/jenkins/setup.sh
index 11ad371..b95fa91 100755
--- a/frontend-install/setup/jenkins/setup.sh
+++ b/frontend-install/setup/jenkins/setup.sh
@@ -2,17 +2,10 @@
 
 set -e
 
-cp config.xml jenkins.model.JenkinsLocationConfiguration.xml ${JENKINS_HOME}
+cp config.xml jenkins.model.JenkinsLocationConfiguration.xml "${JENKINS_HOME}"
 
-source /etc/default/jenkins
-JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
-sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g" /etc/default/jenkins
+echo 'JENKINS_ARGS="${JENKINS_ARGS} --prefix=/fuego"' >> /etc/default/jenkins
+echo 'JAVA_ARGS="${JAVA_ARGS} -Djenkins.install.runSetupWizard=false"' >> /etc/default/jenkins
 
-JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
-if [ -n "$HTTP_PROXY" ]; then
-	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\1/');
-	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E 's/\///g' | sed -E 's/(.*):(.*)/\2/');
-	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -Dhttps.proxyPort=${PROXYPORT}";
-fi
-sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g" /etc/default/jenkins
+echo "source /setup/jenkins/set-java-args-proxy.sh" >> /etc/default/jenkins
 
diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-scripts/docker-build-image.sh
index 6550a98..d276ee0 100755
--- a/fuego-host-scripts/docker-build-image.sh
+++ b/fuego-host-scripts/docker-build-image.sh
@@ -2,4 +2,6 @@
 # $1 - name for the docker image (default: fuego)
 DOCKERIMAGE=${1:-fuego}
 
-sudo docker build -t ${DOCKERIMAGE} --build-arg HTTP_PROXY=$http_proxy .
+sudo docker build -t ${DOCKERIMAGE} \
+    --build-arg http_proxy=${http_proxy} \
+    --build-arg https_proxy=${https_proxy} .
diff --git a/fuego-host-scripts/docker-create-container.sh b/fuego-host-scripts/docker-create-container.sh
index 20f4b59..9e3d61a 100755
--- a/fuego-host-scripts/docker-create-container.sh
+++ b/fuego-host-scripts/docker-create-container.sh
@@ -22,5 +22,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
     -v $DIR/../../fuego-core:/fuego-core:ro \
+    -e http_proxy=${http_proxy} \
+    -e https_proxy=${https_proxy:-$http_proxy} \
     --net="host" ${DOCKERIMAGE} || \
     echo "Could not create fuego-container. See error messages."
diff --git a/fuego-host-scripts/docker-create-usb-privileged-container.sh b/fuego-host-scripts/docker-create-usb-privileged-container.sh
index b3a55c4..2431214 100755
--- a/fuego-host-scripts/docker-create-usb-privileged-container.sh
+++ b/fuego-host-scripts/docker-create-usb-privileged-container.sh
@@ -28,5 +28,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
     -v $DIR/../../fuego-core:/fuego-core:ro \
+    -e http_proxy=${http_proxy} \
+    -e https_proxy=${https_proxy:-$http_proxy} \
     --net="host" ${DOCKERIMAGE} || \
     echo "Could not create fuego-container. See error messages."
-- 
2.15.1

[Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN instruction

[Guilherme Campos Camargo]

DEBIAN_FRONTEND=noninteractive has been explicitely set as an ARG in the
beginning of the Dockerfile. That environment variable is set only
during build time, not being exported with the container image.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/Dockerfile b/Dockerfile
index c39454d..499dd52 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -10,10 +10,11 @@ MAINTAINER tim.bird@sony.com
 # Prepare basic image
 # ==============================================================================
 
-WORKDIR /
+ARG DEBIAN_FRONTEND=noninteractive
+
 COPY frontend-install/apt/sources/fuego-debian-jessie.list \
         /etc/apt/sources.list.d/fuego-debian-jessie.list
-RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
+RUN apt-get update && \
     apt-get -yV install \
         apt-utils \
         at \
@@ -71,9 +72,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
         vim \
         wget \
         xmlstarlet && \
-    rm -rf /var/lib/apt/lists/*
-
-RUN echo dash dash/sh boolean false | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
+    rm -rf /var/lib/apt/lists/* && \
+    echo dash dash/sh boolean false | debconf-set-selections && \
+    dpkg-reconfigure dash
 
 RUN pip install \
         filelock \
@@ -83,7 +84,7 @@ RUN pip install \
 RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list && \
     curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | apt-key add - && \
     dpkg --add-architecture armhf && \
-    DEBIAN_FRONTEND=noninteractive apt-get update && \
+    apt-get update && \
     apt-get -yV install \
         binutils-arm-linux-gnueabihf \
         cpp-arm-linux-gnueabihf \
-- 
2.15.1

[Fuego] [PATCH 8/9] Refactor plugin installation script

[Guilherme Campos Camargo]

On this patch we're trying to remove the explicit sleeps from the plugin
installation script, that are necessary for waiting for Jenkins
start/restart and the installation of the plugins.

With this change we hope that faster hosts (or hosts with faster
networks) will be able to finish Jenkins plugins installation sooner.

We also took the chance to remove the call to sudo, that are not
necessary for using jenkins-cli.

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 frontend-install/setup/jenkins/plugins/install.sh | 44 +++++++++++++++++------
 1 file changed, 33 insertions(+), 11 deletions(-)

diff --git a/frontend-install/setup/jenkins/plugins/install.sh b/frontend-install/setup/jenkins/plugins/install.sh
index 0db8bf2..37e8948 100755
--- a/frontend-install/setup/jenkins/plugins/install.sh
+++ b/frontend-install/setup/jenkins/plugins/install.sh
@@ -2,20 +2,42 @@
 
 set -e
 
+function install_jenkins_plugin {
+    echo "Waiting for Jenkins update center..."
+    while ! ret=$(java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin "$1" 2>/dev/null); do
+        sleep "$2"
+    done
+    echo "${ret}"
+}
+
+function wait_for_jenkins {
+    echo "Waiting for Jenkins..."
+    while ! curl --output /dev/null --silent --head --fail http://localhost:8080/fuego/ ; do
+        sleep "$1"
+    done
+}
+
+readonly sleep_time_s=1
+readonly required_plugins="
+   description-setter
+   pegdown-formatter
+   flot-plotter-plugin/flot.hpi
+   "
+
 service jenkins start
-sleep 30
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin description-setter
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter
-sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi
-sleep 10
+wait_for_jenkins ${sleep_time_s}
+
+for plugin in ${required_plugins}; do
+    install_jenkins_plugin "${plugin}" "${sleep_time_s}"
+done
 
 # Let Jenkins install flot before making the symlink
 service jenkins restart
-sleep 30
-rm $JENKINS_HOME/plugins/flot/flot/mod.js
-ln -s /fuego-core/engine/scripts/mod.js $JENKINS_HOME/plugins/flot/flot/mod.js
-ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
-ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
+wait_for_jenkins 1
 
-chown -R jenkins:jenkins $JENKINS_HOME/
+rm "$JENKINS_HOME/plugins/flot/flot/mod.js"
+ln -s /fuego-core/engine/scripts/mod.js "$JENKINS_HOME/plugins/flot/flot/mod.js"
+ln -s /fuego-rw/logs "$JENKINS_HOME/userContent/fuego.logs"
+ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
 
+chown -R jenkins:jenkins "$JENKINS_HOME/"
-- 
2.15.1

[Fuego] [PATCH 9/9] Set depth to "one" and download just a single-branch on git clones

[Guilherme Campos Camargo]

In order to improve git clone speeds

Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
---
 Dockerfile | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)

diff --git a/Dockerfile b/Dockerfile
index 499dd52..0eeeca0 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -120,13 +120,16 @@ RUN ./install.sh
 COPY frontend-install/setup/serial /setup/serial
 WORKDIR /setup
 
-RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
+RUN git clone https://github.com/tbird20d/ttc.git \
+        --depth 1 --single-branch /usr/local/src/ttc && \
     ./serial/setup-ttc.sh
 
-RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio && \
+RUN git clone https://github.com/frowand/serio.git \
+        --depth 1 --single-branch /usr/local/src/serio && \
     ./serial/setup-serio.sh
 
-RUN git clone https://github.com/tbird20d/serlogin.git /usr/local/src/serlogin && \
+RUN git clone https://github.com/tbird20d/serlogin.git \
+        --depth 1 --single-branch /usr/local/src/serlogin && \
     ./serial/setup-serlogin.sh
 
 COPY frontend-install/setup/lava /setup/lava
-- 
2.15.1

Re: [Fuego] [PATCH 1/9] Remove unused variables from docker build/create/run scripts

[Tim.Bird]

> -----Original Message-----
> From: Guilherme Campos Camargo
> 
> The DIR variable is only being used by the CREATE scripts. There's no
> reason to keep it in the BUILD/REMOVE/START scripts as well.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  fuego-host-scripts/docker-build-image.sh      | 9 +--------
>  fuego-host-scripts/docker-remove-container.sh | 8 --------
>  fuego-host-scripts/docker-remove-image.sh     | 9 +--------
>  fuego-host-scripts/docker-start-container.sh  | 8 --------
>  4 files changed, 2 insertions(+), 32 deletions(-)
> 
> diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> scripts/docker-build-image.sh
> index a197dc5..7c11b11 100755
> --- a/fuego-host-scripts/docker-build-image.sh
> +++ b/fuego-host-scripts/docker-build-image.sh
> @@ -1,12 +1,5 @@
> +#!/bin/bash
>  # $1 - name for the docker image (default: fuego)
> -SOURCE="${BASH_SOURCE[0]}"
> -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a
> symlink
> -  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -  SOURCE="$(readlink "$SOURCE")"
> -  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a
> relative symlink, we need to resolve it relative to the path where the symlink
> file was located
> -done
> -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -
>  DOCKERIMAGE=${1:-fuego}
> 
>  if [ "$(id -u)" == "0" ]; then
> diff --git a/fuego-host-scripts/docker-remove-container.sh b/fuego-host-
> scripts/docker-remove-container.sh
> index 2aae6a9..00c7ac9 100755
> --- a/fuego-host-scripts/docker-remove-container.sh
> +++ b/fuego-host-scripts/docker-remove-container.sh
> @@ -1,13 +1,5 @@
>  #!/bin/bash
>  # $1 - name for the docker container (default: fuego-container)
> -SOURCE="${BASH_SOURCE[0]}"
> -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a
> symlink
> -  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -  SOURCE="$(readlink "$SOURCE")"
> -  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a
> relative symlink, we need to resolve it relative to the path where the symlink
> file was located
> -done
> -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -
>  DOCKERCONTAINER=${1:-fuego-container}
> 
>  sudo docker rm ${DOCKERCONTAINER}
> diff --git a/fuego-host-scripts/docker-remove-image.sh b/fuego-host-
> scripts/docker-remove-image.sh
> index 91dbda7..4c24684 100755
> --- a/fuego-host-scripts/docker-remove-image.sh
> +++ b/fuego-host-scripts/docker-remove-image.sh
> @@ -1,12 +1,5 @@
> +#!/bin/bash
>  # $1 - name for the docker image (default: fuego)
> -SOURCE="${BASH_SOURCE[0]}"
> -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a
> symlink
> -  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -  SOURCE="$(readlink "$SOURCE")"
> -  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a
> relative symlink, we need to resolve it relative to the path where the symlink
> file was located
> -done
> -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -
>  DOCKERIMAGE=${1:-fuego}
> 
>  sudo docker rmi ${DOCKERIMAGE}
> diff --git a/fuego-host-scripts/docker-start-container.sh b/fuego-host-
> scripts/docker-start-container.sh
> index d9de85a..d0cee1a 100755
> --- a/fuego-host-scripts/docker-start-container.sh
> +++ b/fuego-host-scripts/docker-start-container.sh
> @@ -1,13 +1,5 @@
>  #!/bin/bash
>  # $1 - name for the docker container (default: fuego-container)
> -SOURCE="${BASH_SOURCE[0]}"
> -while [ -h "$SOURCE" ]; do # resolve $SOURCE until the file is no longer a
> symlink
> -  DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -  SOURCE="$(readlink "$SOURCE")"
> -  [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE" # if $SOURCE was a
> relative symlink, we need to resolve it relative to the path where the symlink
> file was located
> -done
> -DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
> -
>  DOCKERCONTAINER=${1:-fuego-container}
> 
>  echo "Starting Fuego container (${DOCKERCONTAINER})"
> --
> 2.15.1


Nice cleanup.  Thanks.
Applied.
 -- Tim

Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

[Tim.Bird]

> -----Original Message-----
> From: Guilherme Campos Camargo
> All base-image related apt-get commands (including pre-install config,
> as well as the package install) have been put together in the same RUN
> instruction.
> 
> The same has been done with the LAVA-related apt-get commands on this
Do you mean Fuego-related?

> patch. They can't be merged because the requirements are using a
> specific key that needs to be downloaded with curl (that's installed in
> the previous step. Additionally, LAVA apt-get commands have been moved
Another reference to LAVA?

> up in the Dockerfile, allowing that RUN instruction to be invalidated
> fewer times, because it does not depend on the files that have been
> copied from the host.
> 
> We have also moved apt sources to an external .list file, that's copied
> from the host to the container during build.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile                                         | 116 ++++++++++++++++-----
>  .../apt/sources/fuego-debian-jessie.list           |   2 +
>  2 files changed, 92 insertions(+), 26 deletions(-)
>  create mode 100644 frontend-install/apt/sources/fuego-debian-jessie.list
> 
> diff --git a/Dockerfile b/Dockerfile
> index f34fe4b..b8d78e6 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -19,25 +19,96 @@ ENV https_proxy ${HTTP_PROXY}
>  #
> ==========================================================
> ====================
> 
>  WORKDIR /
> -RUN echo deb http://httpredir.debian.org/debian jessie main non-free >
> /etc/apt/sources.list
> -RUN echo deb http://httpredir.debian.org/debian jessie-updates main non-
> free >> /etc/apt/sources.list
> -RUN echo deb http://security.debian.org/ jessie/updates main >>
> /etc/apt/sources.list
> -RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy
> "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> -RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV
> install \
> -	apt-utils daemon gcc make cmake python-paramiko python-lxml
> python-simplejson \
> -	python-matplotlib python-serial python-yaml python-openpyxl
> python-requests \
> -	libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-
> jdk iperf \
> -	netperf netpipe-tcp sshpass wget git diffstat sudo net-tools vim curl \
> -	inotify-tools g++ bzip2 bc libaio-dev gettext pkg-config libglib2.0-dev
> \
> -	time python-pip python-xmltodict at minicom lzop bsdmainutils u-
> boot-tools \
> -	mc netcat lava-tool openssh-server python-parsedatetime \
> -	libsdl1.2-dev libcairo2-dev libxmu-dev libxmuu-dev
> -
> -RUN pip install python-jenkins==0.4.14
> -RUN pip install filelock
> -RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-
> selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'
> -RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc; fi
> -RUN if [ -n "$HTTP_PROXY" ]; then echo -e
> "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>
> /etc/environment; fi
> +COPY frontend-install/apt/sources/fuego-debian-jessie.list \
> +        /etc/apt/sources.list.d/fuego-debian-jessie.list
> +RUN if [ -n "$HTTP_PROXY" ]; then \
> +        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' >
> /etc/apt/apt.conf.d/80proxy; \
> +    fi && \
> +    DEBIAN_FRONTEND=noninteractive apt-get update && \
> +    apt-get -yV install \
> +        apt-utils \
> +        at \
> +        autoconf \
> +        automake \
> +        bc \
> +        bsdmainutils \
> +        bzip2 \
> +        cmake \
> +        curl \
> +        daemon \
> +        diffstat \
> +        g++ \
> +        gcc \
> +        gettext \
> +        git \
> +        inotify-tools \
> +        iperf \
> +        lava-tool \
> +        libaio-dev \
> +        libcairo2-dev \
> +        libglib2.0-dev \
> +        libsdl1.2-dev \
> +        libtool \
> +        libxmu-dev \
> +        libxmuu-dev \
Most of the above libs are also toolchain packages, for x86_64 (the docker virtual board)
and would benefit from being moved to a separate image creation step or derivative
image.

> +        lzop \
> +        make \
> +        mc \
> +        minicom \
> +        net-tools \
> +        netcat \
> +        netperf \
> +        netpipe-tcp \
> +        openjdk-7-jdk \
> +        openjdk-7-jre \
> +        openssh-server \
> +        pkg-config \
> +        python-lxml \
> +        python-matplotlib \
> +        python-openpyxl \
> +        python-paramiko \
> +        python-parsedatetime \
> +        python-pip \
> +        python-requests \
> +        python-serial \
> +        python-simplejson \
> +        python-xmltodict \
> +        python-yaml \
> +        rsync \
> +        sshpass \
> +        sudo \
> +        time \
> +        u-boot-tools \
> +        vim \
> +        wget \
> +        xmlstarlet && \
> +    rm -rf /var/lib/apt/lists/*
> +
> +RUN echo dash dash/sh boolean false | debconf-set-selections ;
> DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
> +    if [ -n "$HTTP_PROXY" ]; then \
> +        echo "use_proxy = on" >> /etc/wgetrc; \
> +        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>
> /etc/environment; \
> +    fi
> +
> +RUN pip install \
> +        filelock \
> +        python-jenkins==0.4.14
> +
> +# TODO: Move toolchain-related instalation steps to a derivate image, like
> fuego:${version}-arhmhf
> +RUN echo deb http://emdebian.org/tools/debian/ jessie main >
> /etc/apt/sources.list.d/crosstools.list && \
> +    curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key
> | apt-key add - && \
> +    dpkg --add-architecture armhf && \
> +    DEBIAN_FRONTEND=noninteractive apt-get update && \
> +    apt-get -yV install \
> +        binutils-arm-linux-gnueabihf \
> +        cpp-arm-linux-gnueabihf \
> +        crossbuild-essential-armhf \
> +        gcc-arm-linux-gnueabihf && \
> +    rm -rf /var/lib/apt/lists/*
> +
> +RUN pip install \
> +        filelock \
> +        python-jenkins==0.4.14
> 
>  #
> ==========================================================
> ====================
>  # Install Jenkins with the same UID/GID as the host user
> @@ -125,13 +196,6 @@ RUN ln -s /fuego-ro/scripts/fuego-lava-target-
> teardown /usr/local/bin
>  #RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
>  #RUN echo "fuego-create-jobs --board raspberrypi3 --testplan
> testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
> 
> -RUN echo "deb http://emdebian.org/tools/debian/ jessie main" >
> /etc/apt/sources.list.d/crosstools.list
> -RUN dpkg --add-architecture armhf
> -RUN curl http://emdebian.org/tools/debian/emdebian-toolchain-
> archive.key | sudo apt-key add -
> -RUN DEBIAN_FRONTEND=noninteractive apt-get update
> -RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-
> essential-armhf cpp-arm-linux-gnueabihf gcc-arm-linux-gnueabihf binutils-
> arm-linux-gnueabihf
> -
> -
>  #
> ==========================================================
> ====================
>  # Setup startup command
>  #
> ==========================================================
> ====================
> diff --git a/frontend-install/apt/sources/fuego-debian-jessie.list b/frontend-
> install/apt/sources/fuego-debian-jessie.list
> new file mode 100644
> index 0000000..d224f15
> --- /dev/null
> +++ b/frontend-install/apt/sources/fuego-debian-jessie.list
> @@ -0,0 +1,2 @@
> +deb http://httpredir.debian.org/debian jessie main non-free
> +deb http://httpredir.debian.org/debian jessie-updates main non-free
> --
> 2.15.1

On the whole, it looks OK.  I've applied it as is (with, what I believe are
a few commit description errors).
 -- Tim

Re: [Fuego] [PATCH 3/9] Use curl for downloading Jenkins

[Tim.Bird]

OK - looks good.  Applied.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Guilherme Campos
> Camargo
> Sent: Friday, January 26, 2018 9:36 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 3/9] Use curl for downloading Jenkins
> 
> As curl had been used previously in the Dockerfile, let's make it our
> preferred tool for downloading resources in your scripts.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/Dockerfile b/Dockerfile
> index b8d78e6..3dbae0f 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -125,7 +125,7 @@ ENV JENKINS_HOME=/var/lib/jenkins
> 
>  RUN groupadd -g ${gid} ${group} \
>  	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -
> s /bin/bash ${user}
> -RUN wget -nv ${JENKINS_URL}
> +RUN curl -L -O ${JENKINS_URL}
>  RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> sha1sum -c -
>  RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
>  RUN rm jenkins_${JENKINS_VERSION}_all.deb
> --
> 2.15.1
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

[Gustavo Sverzut Barbieri]

On Tue, Jan 30, 2018 at 5:22 PM,  <Tim.Bird@sony.com> wrote:
>> -----Original Message-----
>> From: Guilherme Campos Camargo
>> All base-image related apt-get commands (including pre-install config,
>> as well as the package install) have been put together in the same RUN
>> instruction.
>>
>> The same has been done with the LAVA-related apt-get commands on this
> Do you mean Fuego-related?
>
>> patch. They can't be merged because the requirements are using a
>> specific key that needs to be downloaded with curl (that's installed in
>> the previous step. Additionally, LAVA apt-get commands have been moved
> Another reference to LAVA?

ops, and I missed those in my review as well.

...
>> apt-get update && \
>> +    apt-get -yV install \
>> +        apt-utils \
>> +        at \
>> +        autoconf \
>> +        automake \
>> +        bc \
>> +        bsdmainutils \
>> +        bzip2 \
>> +        cmake \
>> +        curl \
>> +        daemon \
>> +        diffstat \
>> +        g++ \
>> +        gcc \
>> +        gettext \
>> +        git \
>> +        inotify-tools \
>> +        iperf \
>> +        lava-tool \
>> +        libaio-dev \
>> +        libcairo2-dev \
>> +        libglib2.0-dev \
>> +        libsdl1.2-dev \
>> +        libtool \
>> +        libxmu-dev \
>> +        libxmuu-dev \
> Most of the above libs are also toolchain packages, for x86_64 (the docker virtual board)
> and would benefit from being moved to a separate image creation step or derivative
> image.

you mean those *-dev, cmake, autoconf, automake, gcc, g++, gettext and
libtool? indeed makes sense to move these to separate image once we
split the armhf.


-- 
Gustavo Sverzut Barbieri
http://profusion.mobi embedded systems
--------------------------------------
MSN, GTalk, FaceTime: barbieri@gmail.com
Skype: gsbarbieri
Mobile: +55 (16) 99354-9890

Re: [Fuego] [PATCH 2/9] Refactor preinst and apt-get/pip install

[Tim.Bird]

> -----Original Message-----
> From: Gustavo Sverzut 
> 
> On Tue, Jan 30, 2018 at 5:22 PM,  <Tim.Bird@sony.com> wrote:
> >> -----Original Message-----
> >> From: Guilherme Campos Camargo
> >> All base-image related apt-get commands (including pre-install config,
> >> as well as the package install) have been put together in the same RUN
> >> instruction.
> >>
> >> The same has been done with the LAVA-related apt-get commands on
> this
> > Do you mean Fuego-related?
> >
> >> patch. They can't be merged because the requirements are using a
> >> specific key that needs to be downloaded with curl (that's installed in
> >> the previous step. Additionally, LAVA apt-get commands have been
> moved
> > Another reference to LAVA?
> 
> ops, and I missed those in my review as well.

No problem.  It's not like people look at commit messages a lot after
the review ;-)

> 
> ...
> >> apt-get update && \
> >> +    apt-get -yV install \
> >> +        apt-utils \
> >> +        at \
> >> +        autoconf \
> >> +        automake \
> >> +        bc \
> >> +        bsdmainutils \
> >> +        bzip2 \
> >> +        cmake \
> >> +        curl \
> >> +        daemon \
> >> +        diffstat \
> >> +        g++ \
> >> +        gcc \
> >> +        gettext \
> >> +        git \
> >> +        inotify-tools \
> >> +        iperf \
> >> +        lava-tool \
> >> +        libaio-dev \
> >> +        libcairo2-dev \
> >> +        libglib2.0-dev \
> >> +        libsdl1.2-dev \
> >> +        libtool \
> >> +        libxmu-dev \
> >> +        libxmuu-dev \
> > Most of the above libs are also toolchain packages, for x86_64 (the docker
> virtual board)
> > and would benefit from being moved to a separate image creation step or
> derivative
> > image.
> 
> you mean those *-dev, cmake, autoconf, automake, gcc, g++, gettext and
> libtool? indeed makes sense to move these to separate image once we
> split the armhf.

Just the lib*-dev.  and gcc and g++.  cmake, autoconf, automake and libtool
are non-architecture-specific parts of the toolchain, which would be used
by any SDK used in the container, I believe.  I'm not sure about gettext.
I think it provides libraries or headers that would get compiled into binaries
that go to the target, so likely it is x86_64-specific. 

The only things that Fuego intends to compile (right now), are things for the
target boards.  That's not an absolute hard and fast rule.  I could imagine
some test program having both a host-side and target-side component
that needed to be compiled.  But I don't think we have any of those at the
moment.  ... (I just checked the Functional.commonAPI_* tests, which I
thought might have both host and target compiled programs, but they
apparently don't.  So I'm pretty sure this is correct.)

In any event, the general rule would be that the SDK installation step
should have the packages needed for compilation of programs intended
to run on the target.  docker is a special target, and we default
to including the SDK for that included in the base image.  But it would
still be good to keep these in a separate list.
 -- Tim

Re: [Fuego] [PATCH 4/9] Move configuration and jenkins plugin installation to separate scripts

[Bird, Timothy]

> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Guilherme Campos
> Camargo
> Sent: Friday, January 26, 2018 9:36 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 4/9] Move configuration and jenkins plugin
> installation to separate scripts
> 
> In order to simplify the structure of the Dockerfile and optimize build
> times.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile                                         |  96 ++++++---------------
>  frontend-install/{ => setup/jenkins}/config.xml    |   0
>  .../jenkins.model.JenkinsLocationConfiguration.xml |   0
>  .../plugins/flot-plotter-plugin/.gitignore         |   0
>  .../jenkins}/plugins/flot-plotter-plugin/flot.hpi  | Bin
>  .../jenkins}/plugins/flot-plotter-plugin/pom.xml   |   0
>  .../src/main/java/flotile/ChartAction.java         |   0
>  .../src/main/java/flotile/FlotPublisher.java       |   0
>  .../flotile/ChartAction/floatingBox.jelly          |   0
>  .../src/main/resources/index.jelly                 |   0
>  .../src/main/webapp/flot/.copyarea.db              |   0
>  .../src/main/webapp/flot/API.txt                   |   0
>  .../src/main/webapp/flot/FAQ.txt                   |   0
>  .../src/main/webapp/flot/LICENSE.txt               |   0
>  .../src/main/webapp/flot/PLUGINS.txt               |   0
>  .../src/main/webapp/flot/README.txt                |   0
>  .../src/main/webapp/flot/base64.js                 |   0
>  .../src/main/webapp/flot/canvas2image.js           |   0
>  .../src/main/webapp/flot/excanvas.min.js           |   0
>  .../main/webapp/flot/jquery.colorhelpers.min.js    |   0
>  .../main/webapp/flot/jquery.flot.crosshair.min.js  |   0
>  .../webapp/flot/jquery.flot.fillbetween.min.js     |   0
>  .../src/main/webapp/flot/jquery.flot.image.min.js  |   0
>  .../src/main/webapp/flot/jquery.flot.min.js        |   0
>  .../main/webapp/flot/jquery.flot.navigate.min.js   |   0
>  .../src/main/webapp/flot/jquery.flot.pie.min.js    |   0
>  .../src/main/webapp/flot/jquery.flot.resize.min.js |   0
>  .../main/webapp/flot/jquery.flot.selection.min.js  |   0
>  .../src/main/webapp/flot/jquery.flot.stack.min.js  |   0
>  .../src/main/webapp/flot/jquery.flot.symbol.min.js |   0
>  .../main/webapp/flot/jquery.flot.threshold.min.js  |   0
>  .../src/main/webapp/flot/jquery.min.js             |   0
>  .../src/main/webapp/flot/mod.js                    |   0
>  .../src/main/webapp/flot/my.css                    |   0
>  .../src/main/webapp/help-globalConfig.html         |   0
>  frontend-install/setup/jenkins/plugins/install.sh  |  21 +++++
>  frontend-install/setup/jenkins/setup.sh            |  26 ++++++
>  frontend-install/setup/lava/setup.sh               |  11 +++
>  frontend-install/setup/serial/setup-serio.sh       |   8 ++
>  frontend-install/setup/serial/setup-serlogin.sh    |   6 ++
>  frontend-install/setup/serial/setup-ttc.sh         |   7 ++

In general, I like the idea of refactoring this,  but not this specific
refactoring.  I'll keep your scripts,
but maybe with renames and some merges.  (serio and serlogin
setup don't need to be separate).
ttc is not related to serial.  the lava, serial and apt dirs are
not really related to 'frontend-install', which is itself
a euphemism for 'jenkins'.

I'm applying this patch, in case subsequent patches in this series
depend on the Dockerfile changes here, but I plan to refactor
the frontend-install directory again after I apply the rest of
this series.

>  41 files changed, 105 insertions(+), 70 deletions(-)
>  rename frontend-install/{ => setup/jenkins}/config.xml (100%)
>  rename frontend-install/{ =>
> setup/jenkins}/jenkins.model.JenkinsLocationConfiguration.xml (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/.gitignore (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/flot.hpi (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/pom.xml (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/java/flotile/ChartAction.java (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/java/flotile/FlotPublisher.java (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/resources/index.jelly (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/.copyarea.db (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/API.txt (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/FAQ.txt (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/LICENSE.txt (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/PLUGINS.txt (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/README.txt (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/base64.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/canvas2image.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/excanvas.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.colorhelpers.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.image.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.navigate.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.pie.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.resize.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.selection.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.stack.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.symbol.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.threshold.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.min.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/mod.js (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/flot/my.css (100%)
>  rename frontend-install/{ => setup/jenkins}/plugins/flot-plotter-
> plugin/src/main/webapp/help-globalConfig.html (100%)
>  create mode 100755 frontend-install/setup/jenkins/plugins/install.sh
>  create mode 100755 frontend-install/setup/jenkins/setup.sh
>  create mode 100755 frontend-install/setup/lava/setup.sh
>  create mode 100755 frontend-install/setup/serial/setup-serio.sh
>  create mode 100755 frontend-install/setup/serial/setup-serlogin.sh
>  create mode 100755 frontend-install/setup/serial/setup-ttc.sh
> 
> diff --git a/Dockerfile b/Dockerfile
> index 3dbae0f..883ac21 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -106,95 +106,51 @@ RUN echo deb http://emdebian.org/tools/debian/
> jessie main > /etc/apt/sources.li
>          gcc-arm-linux-gnueabihf && \
>      rm -rf /var/lib/apt/lists/*
> 
> -RUN pip install \
> -        filelock \
> -        python-jenkins==0.4.14
> -
>  #
> ==========================================================
> ====================
> -# Install Jenkins with the same UID/GID as the host user
> +# Download and Install Jenkins
>  #
> ==========================================================
> ====================
> 
> -ARG user=jenkins
> -ARG group=jenkins
> -ARG uid=1000
> -ARG gid=${uid}
> +ENV uid=1000
> +ENV gid=${uid}

Do these environment variables persist in the created container?
If so, these names are a bit short.  They don't show up in 
the shell environment inside the container do they?

>  ARG JENKINS_VERSION=2.32.1
>  ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
>  ARG JENKINS_URL=https://pkg.jenkins.io/debian-
> stable/binary/jenkins_${JENKINS_VERSION}_all.deb
>  ENV JENKINS_HOME=/var/lib/jenkins
> 
> -RUN groupadd -g ${gid} ${group} \
> -	&& useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -
> s /bin/bash ${user}
> -RUN curl -L -O ${JENKINS_URL}
> -RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" |
> sha1sum -c -
> -RUN dpkg -i jenkins_${JENKINS_VERSION}_all.deb
> -RUN rm jenkins_${JENKINS_VERSION}_all.deb
> +RUN groupadd -g ${gid} ${group} && \
> +    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> /bin/bash ${user} && \
> +    curl -L -O ${JENKINS_URL} && \
> +    echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
> -c - && \
> +    dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
> +    rm jenkins_${JENKINS_VERSION}_all.deb
> 
>  #
> ==========================================================
> ====================
> -# get ttc script and helpers
> +# Post installation
>  #
> ==========================================================
> ====================
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc
> -RUN /usr/local/src/ttc/install.sh /usr/local/bin
> -RUN perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#"
> /usr/local/bin/ttc
> 
> -#
> ==========================================================
> ====================
> -# Serial Config
> -#
> ==========================================================
> ====================
> +COPY frontend-install/setup/jenkins /setup/jenkins
> +WORKDIR /setup/jenkins
> +RUN ./setup.sh
> 
> -RUN /bin/bash -c 'git clone https://github.com/frowand/serio.git
> /usr/local/src/serio ;  chown -R jenkins /usr/local/src/serio ; cp
> /usr/local/src/serio/serio /usr/local/bin/ ; ln -s /usr/local/bin/serio
> /usr/local/bin/sercp ; ln -s /usr/local/bin/serio /usr/local/bin/sersh'
> +WORKDIR /setup/jenkins/plugins
> +RUN ./install.sh
> 
> -RUN /bin/bash -c 'git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin ;  chown -R jenkins /usr/local/src/serlogin ; cp
> /usr/local/src/serlogin/serlogin /usr/local/bin'
> +COPY frontend-install/setup/serial /setup/serial
> +WORKDIR /setup
> 
> -#
> ==========================================================
> ====================
> -# Post installation
> -#
> ==========================================================
> ====================
> +RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
> +    ./serial/setup-ttc.sh
> 
> -RUN source /etc/default/jenkins && \
> -	JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego" && \
> -	sed -i -e
> "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> /etc/default/jenkins
> -
> -RUN source /etc/default/jenkins && \
> -	JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
> && \
> -	if [ -n "$HTTP_PROXY" ]; then \
> -		PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed
> -E 's/\///g' | sed -E 's/(.*):(.*)/\1/') && \
> -		PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\2/') && \
> -		JAVA_ARGS="$JAVA_ARGS -
> Dhttp.proxyHost="${PROXYSERVER}" -Dhttp.proxyPort="${PROXYPORT}" -
> Dhttps.proxyHost="${PROXYSERVER}" -Dhttps.proxyPort="${PROXYPORT}; \
> -	fi && \
> -	sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> /etc/default/jenkins;
> -
> -COPY frontend-install/plugins/flot-plotter-plugin/flot.hpi /tmp
> -
> -RUN service jenkins start && \
> -	sleep 30 && \
> -	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-
> cli.jar -s http://localhost:8080/fuego install-plugin description-setter && \
> -	sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-
> cli.jar -s http://localhost:8080/fuego install-plugin pegdown-formatter && \
> -    sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -
> s http://localhost:8080/fuego install-plugin /tmp/flot.hpi && \
> -    sleep 10
> -
> -# Let Jenkins install flot before making the symlink
> -RUN service jenkins restart && sleep 30 && \
> -    rm $JENKINS_HOME/plugins/flot/flot/mod.js && \
> -    ln -s /fuego-core/engine/scripts/mod.js
> $JENKINS_HOME/plugins/flot/flot/mod.js
> -
> -RUN ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> -COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> docs.pdf
> +RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio &&
> \
> +    ./serial/setup-serio.sh
> 
> -RUN ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
> -COPY frontend-install/config.xml $JENKINS_HOME/config.xml
> -COPY frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> $JENKINS_HOME/jenkins.model.JenkinsLocationConfiguration.xml
> +RUN git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin && \
> +    ./serial/setup-serlogin.sh
> 
> -RUN chown -R jenkins:jenkins $JENKINS_HOME/
> +COPY frontend-install/setup/lava /setup/lava
> +RUN ./lava/setup.sh
> 
> -#
> ==========================================================
> ====================
> -# Lava
> -#
> ==========================================================
> ====================
> -
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> -RUN ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> -# CONVENIENCE HACKS
> -# not mounted, yet
> -#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
> -#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan
> testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
> +COPY docs/fuego-docs.pdf $JENKINS_HOME/userContent/docs/fuego-
> docs.pdf
> 
>  #
> ==========================================================
> ====================
>  # Setup startup command
> diff --git a/frontend-install/config.xml b/frontend-
> install/setup/jenkins/config.xml
> similarity index 100%
> rename from frontend-install/config.xml
> rename to frontend-install/setup/jenkins/config.xml
> diff --git a/frontend-install/jenkins.model.JenkinsLocationConfiguration.xml
> b/frontend-
> install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml
> similarity index 100%
> rename from frontend-
> install/jenkins.model.JenkinsLocationConfiguration.xml
> rename to frontend-
> install/setup/jenkins/jenkins.model.JenkinsLocationConfiguration.xml
> diff --git a/frontend-install/plugins/flot-plotter-plugin/.gitignore b/frontend-
> install/setup/jenkins/plugins/flot-plotter-plugin/.gitignore
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-plugin/.gitignore
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/.gitignore
> diff --git a/frontend-install/plugins/flot-plotter-plugin/flot.hpi b/frontend-
> install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-plugin/flot.hpi
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-plugin/flot.hpi
> diff --git a/frontend-install/plugins/flot-plotter-plugin/pom.xml b/frontend-
> install/setup/jenkins/plugins/flot-plotter-plugin/pom.xml
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-plugin/pom.xml
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/pom.xml
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/java/flotile/ChartAction.java b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/java/flotile/ChartAction.java
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/java/flotile/ChartAction.java
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/java/flotile/ChartAction.java
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/java/flotile/FlotPublisher.java b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/java/flotile/FlotPublisher.java
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/java/flotile/FlotPublisher.java
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/java/flotile/FlotPublisher.java
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/resources/flotile/ChartAction/floatingBox.jelly
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/resources/index.jelly b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/resources/index.jelly
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/resources/index.jelly
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/resources/index.jelly
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/.copyarea.db b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/.copyarea.db
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/.copyarea.db
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/.copyarea.db
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/API.txt b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/API.txt
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/API.txt
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/API.txt
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/FAQ.txt b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/FAQ.txt
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/FAQ.txt
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/FAQ.txt
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/LICENSE.txt b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/LICENSE.txt
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/LICENSE.txt
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/LICENSE.txt
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/PLUGINS.txt b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/PLUGINS.txt
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/PLUGINS.txt
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/PLUGINS.txt
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/README.txt b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/README.txt
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/README.txt
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/README.txt
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/base64.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/base64.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/base64.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/base64.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/canvas2image.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/canvas2image.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/canvas2image.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/canvas2image.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/excanvas.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/excanvas.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/excanvas.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/excanvas.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.colorhelpers.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.colorhelpers.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.crosshair.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.fillbetween.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.image.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.image.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.image.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.image.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.navigate.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.navigate.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.pie.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.pie.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.pie.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.pie.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.resize.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.resize.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.resize.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.resize.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.selection.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.selection.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.selection.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.selection.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.stack.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.stack.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.stack.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.stack.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.symbol.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.symbol.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.threshold.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.flot.threshold.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.min.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.min.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.min.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/jquery.min.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/mod.js b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/mod.js
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/mod.js
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/mod.js
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/my.css b/frontend-
> install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/my.css
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/flot/my.css
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/flot/my.css
> diff --git a/frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/help-globalConfig.html b/frontend-
> install/setup/jenkins/plugins/flot-plotter-plugin/src/main/webapp/help-
> globalConfig.html
> similarity index 100%
> rename from frontend-install/plugins/flot-plotter-
> plugin/src/main/webapp/help-globalConfig.html
> rename to frontend-install/setup/jenkins/plugins/flot-plotter-
> plugin/src/main/webapp/help-globalConfig.html
> diff --git a/frontend-install/setup/jenkins/plugins/install.sh b/frontend-
> install/setup/jenkins/plugins/install.sh
> new file mode 100755
> index 0000000..0db8bf2
> --- /dev/null
> +++ b/frontend-install/setup/jenkins/plugins/install.sh
> @@ -0,0 +1,21 @@
> +#!/bin/bash
> +
> +set -e
> +
> +service jenkins start
> +sleep 30
> +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin description-setter
> +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin pegdown-formatter
> +sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi
> +sleep 10
> +
> +# Let Jenkins install flot before making the symlink
> +service jenkins restart
> +sleep 30
> +rm $JENKINS_HOME/plugins/flot/flot/mod.js
> +ln -s /fuego-core/engine/scripts/mod.js
> $JENKINS_HOME/plugins/flot/flot/mod.js
> +ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> +ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
> +
> +chown -R jenkins:jenkins $JENKINS_HOME/
> +
> diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-
> install/setup/jenkins/setup.sh
> new file mode 100755
> index 0000000..c550862
> --- /dev/null
> +++ b/frontend-install/setup/jenkins/setup.sh
> @@ -0,0 +1,26 @@
> +#!/bin/bash
> +
> +set -e
> +
> +user=jenkins
> +group=jenkins
> +uid=1000
> +gid=${uid}
> +
> +# groupadd -g ${gid} ${group}
> +# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> /bin/bash ${user}
> +
> +cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
> ${JENKINS_HOME}
> +
> +source /etc/default/jenkins
> +JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
> +sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> /etc/default/jenkins
> +
> +JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
> +if [ -n "$HTTP_PROXY" ]; then
> +	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\1/');
> +	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\2/');
> +	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -
> Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -
> Dhttps.proxyPort=${PROXYPORT}";
> +fi
> +sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> /etc/default/jenkins
> +
> diff --git a/frontend-install/setup/lava/setup.sh b/frontend-
> install/setup/lava/setup.sh
> new file mode 100755
> index 0000000..86df49f
> --- /dev/null
> +++ b/frontend-install/setup/lava/setup.sh
> @@ -0,0 +1,11 @@
> +#!/bin/bash
> +
> +set -e
> +
> +# CONVENIENCE HACKS
> +# not mounted, yet
> +#RUN echo "fuego-create-node --board raspberrypi3" >> /root/firststart.sh
> +#RUN echo "fuego-create-jobs --board raspberrypi3 --testplan
> testplan_docker --distrib nosyslogd.dist" >> /root/firststart.sh
> +
> +ln -s /fuego-ro/scripts/fuego-lava-target-setup /usr/local/bin
> +ln -s /fuego-ro/scripts/fuego-lava-target-teardown /usr/local/bin
> diff --git a/frontend-install/setup/serial/setup-serio.sh b/frontend-
> install/setup/serial/setup-serio.sh
> new file mode 100755
> index 0000000..7e25a96
> --- /dev/null
> +++ b/frontend-install/setup/serial/setup-serio.sh
> @@ -0,0 +1,8 @@
> +#!/bin/bash
> +
> +set -e
> +
> +chown -R jenkins:jenkins /usr/local/src/serio
> +cp /usr/local/src/serio/serio /usr/local/bin/
> +ln -s /usr/local/bin/serio /usr/local/bin/sercp
> +ln -s /usr/local/bin/serio /usr/local/bin/sersh
> diff --git a/frontend-install/setup/serial/setup-serlogin.sh b/frontend-
> install/setup/serial/setup-serlogin.sh
> new file mode 100755
> index 0000000..0de17ea
> --- /dev/null
> +++ b/frontend-install/setup/serial/setup-serlogin.sh
> @@ -0,0 +1,6 @@
> +#!/bin/bash
> +
> +set -e
> +
> +chown -R jenkins:jenkins /usr/local/src/serlogin
> +cp /usr/local/src/serlogin/serlogin /usr/local/bin
> diff --git a/frontend-install/setup/serial/setup-ttc.sh b/frontend-
> install/setup/serial/setup-ttc.sh
> new file mode 100755
> index 0000000..d32b73e
> --- /dev/null
> +++ b/frontend-install/setup/serial/setup-ttc.sh
> @@ -0,0 +1,7 @@
> +#!/bin/bash
> +
> +set -e
> +
> +/usr/local/src/ttc/install.sh /usr/local/bin
> +perl -p -i -e "s#config_dir = \"/etc\"#config_dir = \"/fuego-ro/conf\"#" \
> +    /usr/local/bin/ttc
> --
> 2.15.1


OK - applied, but with caveats mentioned above.
 -- Tim

Re: [Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid

[Tim.Bird]

> -----Original Message-----
> From: Guilherme Campos Camargo
> 
> Prior to this patch, the fuego-host-scripts/docker-build-image.sh script
> was calling docker build passing two `build-arg`s: uid/gid. According to
> the comments in the code, it seems that the intention was to make sure
> that the jenkins user (needed by Jenkins) would have the same uid/gid as
> the user of the host that has called the install script.
> 
> Given that the uid/gid of the jenkins user was being set at build time,
> the generated image would be configured only for the machine where it's
> built, what would prevent jenkins to change files in the /fuego-rw
> mountpoint in different hosts.

That's correct, and changing this is a nice step forward to making the 
container usable on other machines than the one it was built on.

> 
> One approach for allowing jenkins to write to fuego-rw, is to chown
> /fuego-rw and it's pre-existing subdirectories during execution time,
> what can be done in a docker entrypoint script. That approach per se,
> solves the problem, by allowing jenkins to write to the mount point, but
> generates a side-effect: the user on the host machine will be able to
> inspect, but will not be able to delete/modify the files created by
> jenkins (unless it forces it as root).
> 
> Another approach would be to change the uid and the gid of the jenkins
> user in the first execution, (first call to docker-container-start.sh),
> also through the entrypoint.  This would solve the problem of the access
> from both sides, but has the downside of the need of running a recursive
> `chown` in all jenkins files in `/var` (including cache), possibly
> taking a long time to be accomplished.

I'd much rather do this.  This amounts to a one-time container installation
overhead on a new host, for the Fuego user.

I modify files in fuego-ro and fuego-rw all the time, from the host side.
This is my normal workflow, and I'd like to see it supported.

> 
> Since the need to edit/remove the files from fuego-rw (without sudo)
> depends on the user, our suggestion is to use the entrypoint for
> `chowning` only the pre-existing directories from fuego-rw and to
> provide the user a script for mapping the jenkins' uid/gid with user's
> uid/gid only if the user needs to.
> 
> On this patch we have created the entrypoint.sh script, and the uid/gid
> mapping script.

At this point I'm confused.  Do both approaches have an entrypoint.sh script?

Could you give an example of the steps required to install the container
in both your scenarios, including the arguments required for the scripts,
and the resulting permissions and ownership both inside and outside
the container, for the volumes?

I'm not sure when map-jenkins-uid-to-host.sh and
jenkins-map-uid.sh are run, and there seem to be no instructions
for using them, or a human-readable description of what they do.

Thanks,
 -- Tim

> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile                                             | 10 ++++------
>  frontend-install/entrypoint.sh                         | 13 +++++++++++++
>  .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12 ++++++++++++
>  frontend-install/setup/jenkins/setup.sh                |  8 --------
>  fuego-host-scripts/docker-build-image.sh               | 10 +---------
>  fuego-host-scripts/jenkins-map-uid.sh                  | 18 ++++++++++++++++++
>  6 files changed, 48 insertions(+), 23 deletions(-)
>  create mode 100755 frontend-install/entrypoint.sh
>  create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-
> host.sh
>  create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
> 
> diff --git a/Dockerfile b/Dockerfile
> index 883ac21..493414e 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/
> jessie main > /etc/apt/sources.li
>  # Download and Install Jenkins
>  #
> ==========================================================
> ====================
> 
> -ENV uid=1000
> -ENV gid=${uid}
>  ARG JENKINS_VERSION=2.32.1
>  ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
>  ARG JENKINS_URL=https://pkg.jenkins.io/debian-
> stable/binary/jenkins_${JENKINS_VERSION}_all.deb
>  ENV JENKINS_HOME=/var/lib/jenkins
> 
> -RUN groupadd -g ${gid} ${group} && \
> -    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> /bin/bash ${user} && \
> -    curl -L -O ${JENKINS_URL} && \
> +RUN curl -L -O ${JENKINS_URL} && \
>      echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
> -c - && \
>      dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
>      rm jenkins_${JENKINS_VERSION}_all.deb
> @@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf
> $JENKINS_HOME/userContent/docs/fuego-docs.pdf
>  # Setup startup command
>  #
> ==========================================================
> ====================
> 
> -ENTRYPOINT service jenkins start && service netperf start && /bin/bash
> +WORKDIR /
> +COPY frontend-install/entrypoint.sh /
> +ENTRYPOINT ["/entrypoint.sh"]
> diff --git a/frontend-install/entrypoint.sh b/frontend-install/entrypoint.sh
> new file mode 100755
> index 0000000..281b0fe
> --- /dev/null
> +++ b/frontend-install/entrypoint.sh
> @@ -0,0 +1,13 @@
> +#!/bin/bash
> +set -e
> +
> +service jenkins stop >> /dev/null
> +
> +mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
> +chown jenkins:jenkins \
> +    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
> +
> +service jenkins start
> +service netperf start
> +
> +exec /bin/bash
> diff --git a/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
> b/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
> new file mode 100755
> index 0000000..002f165
> --- /dev/null
> +++ b/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh

Who calls this?  When, and with what args?

> @@ -0,0 +1,12 @@
> +#!/bin/bash
> +set -e
> +
> +service jenkins stop
> +
> +usermod -u "${jenkins_uid}" jenkins
> +groupmod -g "${jenkins_gid}" jenkins
> +
> +chown -R "${jenkins_uid}":"${jenkins_gid}" \
> +    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
> +
> +service jenkins start
> diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-
> install/setup/jenkins/setup.sh
> index c550862..11ad371 100755
> --- a/frontend-install/setup/jenkins/setup.sh
> +++ b/frontend-install/setup/jenkins/setup.sh
> @@ -2,14 +2,6 @@
> 
>  set -e
> 
> -user=jenkins
> -group=jenkins
> -uid=1000
> -gid=${uid}
> -
> -# groupadd -g ${gid} ${group}
> -# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> /bin/bash ${user}
> -
>  cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
> ${JENKINS_HOME}
> 
>  source /etc/default/jenkins
> diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> scripts/docker-build-image.sh
> index 7c11b11..6550a98 100755
> --- a/fuego-host-scripts/docker-build-image.sh
> +++ b/fuego-host-scripts/docker-build-image.sh
> @@ -2,12 +2,4 @@
>  # $1 - name for the docker image (default: fuego)
>  DOCKERIMAGE=${1:-fuego}
> 
> -if [ "$(id -u)" == "0" ]; then
> -	JENKINS_UID=$(id -u $SUDO_USER)
> -	JENKINS_GID=$(id -g $SUDO_USER)
> -else
> -	JENKINS_UID=$(id -u $USER)
> -	JENKINS_GID=$(id -g $USER)
> -fi
> -
> -sudo docker build -t ${DOCKERIMAGE} --build-arg
> HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg
> gid=$JENKINS_GID .
> +sudo docker build -t ${DOCKERIMAGE} --build-arg
> HTTP_PROXY=$http_proxy .
> diff --git a/fuego-host-scripts/jenkins-map-uid.sh b/fuego-host-
> scripts/jenkins-map-uid.sh
> new file mode 100755
> index 0000000..fad619e
> --- /dev/null
> +++ b/fuego-host-scripts/jenkins-map-uid.sh

Who calls this?  When, and with what args?

> @@ -0,0 +1,18 @@
> +#!/bin/bash
> +# $1 - name for the docker container (default: fuego-container)
> +DOCKERCONTAINER=${1:-fuego-container}
> +
> +if [ "${UID}" == "0" ]; then
> +	uid=$(id -u "${SUDO_USER}")
> +	gid=$(id -g "${SUDO_USER}")
> +else
> +	uid="${UID}"
> +	gid=$(id -g "${USER}")
> +fi
> +
> +sudo docker start "${DOCKERCONTAINER}" || \
> +  echo "Please create Fuego docker container via docker-create-
> container.sh script"
> +
> +sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
> +    "${DOCKERCONTAINER}" \
> +    /setup/jenkins/map-jenkins-uid-to-host.sh
> --
> 2.15.1


I need to understand this better before I apply it.
 -- Tim

Re: [Fuego] [PATCH 6/9] Cleanup proxy

[Tim.Bird]

OK - I had to modify this because I skipped patch 5/9, but
I applied it.

I'm not running this from behind a proxy, so I'll have to find someone to test this.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Guilherme Campos
> Camargo
> Sent: Friday, January 26, 2018 9:36 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 6/9] Cleanup proxy
> 
> Prior to this patch, http_proxy was being set and saved during build
> time according to the http_proxy values of the host.
> 
> This approach has at least two shortcomings, that are explained below:
> 
> 1. Any other user that wanted to use a prebuilt image in their host
> would be unable to do so, because the proxy configuration is matching
> the host of the builder and not the host of the user.
> 
> 2. A user of the image would be able to read the proxy configuration
> that had been set during build from `/etc/default/jenkins` or the
> apt-get config file. The proxy config may easily contain sensitive
> information as the proxy user/password that were set on the host that
> built the image.
> 
> In order to avoid the problems pointed out above, we chose to:
> 
> 1. Remove the ARG/ENV instantiations of http_proxy from the Dockerfile
> and use only the builtin docker ARG http_proxy. This will prevent the
> proxy variable to be available in a running container and in the image
> history.
> 
> 2. Instead of using the apt-get global config file, we're just passing
> http_proxy as a local environment variable to the RUN instruction in
> which apt-get is being executed.
> 
> 3. Jenkins /etc/default/jenkins JAVA_ARGS (that contain the proxy
> information) is now being updated by a script that's been source by
> /etc/default/jenkins itself. This means that, whenever `service jenkins`
> is called, that script will run and append the proxy configurations to
> the JAVA_ARGS at runtime, not replacing the JAVA_ARGS that are stored in
> the file.
> 
> 4. http_proxy variables are being passed as arguments to `docker create`
> as well. By doing that, we make sure that the entrypoint will be running
> with the http_proxy values of the user.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile                                            | 19 ++-----------------
>  frontend-install/setup/jenkins/set-java-args-proxy.sh |  7 +++++++
>  frontend-install/setup/jenkins/setup.sh               | 15 ++++-----------
>  fuego-host-scripts/docker-build-image.sh              |  4 +++-
>  fuego-host-scripts/docker-create-container.sh         |  2 ++
>  .../docker-create-usb-privileged-container.sh         |  2 ++
>  6 files changed, 20 insertions(+), 29 deletions(-)
>  create mode 100755 frontend-install/setup/jenkins/set-java-args-proxy.sh
> 
> diff --git a/Dockerfile b/Dockerfile
> index 493414e..c39454d 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -6,14 +6,6 @@
>  FROM debian:jessie
>  MAINTAINER tim.bird@sony.com
> 
> -#
> ==========================================================
> ====================
> -# Proxy variables
> -#
> ==========================================================
> ====================
> -
> -ARG HTTP_PROXY
> -ENV http_proxy ${HTTP_PROXY}
> -ENV https_proxy ${HTTP_PROXY}
> -
>  #
> ==========================================================
> ====================
>  # Prepare basic image
>  #
> ==========================================================
> ====================
> @@ -21,10 +13,7 @@ ENV https_proxy ${HTTP_PROXY}
>  WORKDIR /
>  COPY frontend-install/apt/sources/fuego-debian-jessie.list \
>          /etc/apt/sources.list.d/fuego-debian-jessie.list
> -RUN if [ -n "$HTTP_PROXY" ]; then \
> -        echo 'Acquire::http::proxy "'$HTTP_PROXY'";' >
> /etc/apt/apt.conf.d/80proxy; \
> -    fi && \
> -    DEBIAN_FRONTEND=noninteractive apt-get update && \
> +RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
>      apt-get -yV install \
>          apt-utils \
>          at \
> @@ -84,11 +73,7 @@ RUN if [ -n "$HTTP_PROXY" ]; then \
>          xmlstarlet && \
>      rm -rf /var/lib/apt/lists/*
> 
> -RUN echo dash dash/sh boolean false | debconf-set-selections ;
> DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash && \
> -    if [ -n "$HTTP_PROXY" ]; then \
> -        echo "use_proxy = on" >> /etc/wgetrc; \
> -        echo -e "http_proxy=$HTTP_PROXY\nhttps_proxy=$HTTP_PROXY" >>
> /etc/environment; \
> -    fi
> +RUN echo dash dash/sh boolean false | debconf-set-selections ;
> DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> 
>  RUN pip install \
>          filelock \
> diff --git a/frontend-install/setup/jenkins/set-java-args-proxy.sh
> b/frontend-install/setup/jenkins/set-java-args-proxy.sh
> new file mode 100755
> index 0000000..9b04f40
> --- /dev/null
> +++ b/frontend-install/setup/jenkins/set-java-args-proxy.sh
> @@ -0,0 +1,7 @@
> +#!/bin/bash
> +
> +if [ -n "${http_proxy}" ]; then
> +	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\1/');
> +	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\2/');
> +	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -
> Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -
> Dhttps.proxyPort=${PROXYPORT}";
> +fi
> diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-
> install/setup/jenkins/setup.sh
> index 11ad371..b95fa91 100755
> --- a/frontend-install/setup/jenkins/setup.sh
> +++ b/frontend-install/setup/jenkins/setup.sh
> @@ -2,17 +2,10 @@
> 
>  set -e
> 
> -cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
> ${JENKINS_HOME}
> +cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
> "${JENKINS_HOME}"
> 
> -source /etc/default/jenkins
> -JENKINS_ARGS="$JENKINS_ARGS --prefix=/fuego"
> -sed -i -e "s#JENKINS_ARGS.*#JENKINS_ARGS\=\"${JENKINS_ARGS}\"#g"
> /etc/default/jenkins
> +echo 'JENKINS_ARGS="${JENKINS_ARGS} --prefix=/fuego"' >>
> /etc/default/jenkins
> +echo 'JAVA_ARGS="${JAVA_ARGS} -
> Djenkins.install.runSetupWizard=false"' >> /etc/default/jenkins
> 
> -JAVA_ARGS="$JAVA_ARGS -Djenkins.install.runSetupWizard=false"
> -if [ -n "$HTTP_PROXY" ]; then
> -	PROXYSERVER=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\1/');
> -	PROXYPORT=$(echo $http_proxy | sed -E 's/^http://' | sed -E
> 's/\///g' | sed -E 's/(.*):(.*)/\2/');
> -	JAVA_ARGS="$JAVA_ARGS -Dhttp.proxyHost=${PROXYSERVER} -
> Dhttp.proxyPort=${PROXYPORT} -Dhttps.proxyHost=${PROXYSERVER} -
> Dhttps.proxyPort=${PROXYPORT}";
> -fi
> -sed -i -e "s#^JAVA_ARGS.*#JAVA_ARGS\=\"${JAVA_ARGS}\"#g"
> /etc/default/jenkins
> +echo "source /setup/jenkins/set-java-args-proxy.sh" >>
> /etc/default/jenkins
> 
> diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> scripts/docker-build-image.sh
> index 6550a98..d276ee0 100755
> --- a/fuego-host-scripts/docker-build-image.sh
> +++ b/fuego-host-scripts/docker-build-image.sh
> @@ -2,4 +2,6 @@
>  # $1 - name for the docker image (default: fuego)
>  DOCKERIMAGE=${1:-fuego}
> 
> -sudo docker build -t ${DOCKERIMAGE} --build-arg
> HTTP_PROXY=$http_proxy .
> +sudo docker build -t ${DOCKERIMAGE} \
> +    --build-arg http_proxy=${http_proxy} \
> +    --build-arg https_proxy=${https_proxy} .
> diff --git a/fuego-host-scripts/docker-create-container.sh b/fuego-host-
> scripts/docker-create-container.sh
> index 20f4b59..9e3d61a 100755
> --- a/fuego-host-scripts/docker-create-container.sh
> +++ b/fuego-host-scripts/docker-create-container.sh
> @@ -22,5 +22,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
>      -v $DIR/../fuego-rw:/fuego-rw \
>      -v $DIR/../fuego-ro:/fuego-ro:ro \
>      -v $DIR/../../fuego-core:/fuego-core:ro \
> +    -e http_proxy=${http_proxy} \
> +    -e https_proxy=${https_proxy:-$http_proxy} \
>      --net="host" ${DOCKERIMAGE} || \
>      echo "Could not create fuego-container. See error messages."
> diff --git a/fuego-host-scripts/docker-create-usb-privileged-container.sh
> b/fuego-host-scripts/docker-create-usb-privileged-container.sh
> index b3a55c4..2431214 100755
> --- a/fuego-host-scripts/docker-create-usb-privileged-container.sh
> +++ b/fuego-host-scripts/docker-create-usb-privileged-container.sh
> @@ -28,5 +28,7 @@ sudo docker create -it --name ${DOCKERCONTAINER} \
>      -v $DIR/../fuego-rw:/fuego-rw \
>      -v $DIR/../fuego-ro:/fuego-ro:ro \
>      -v $DIR/../../fuego-core:/fuego-core:ro \
> +    -e http_proxy=${http_proxy} \
> +    -e https_proxy=${https_proxy:-$http_proxy} \
>      --net="host" ${DOCKERIMAGE} || \
>      echo "Could not create fuego-container. See error messages."
> --
> 2.15.1
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN instruction

[Tim.Bird]

Looks good.  Applied.
 -- Tim


> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Guilherme Campos
> Camargo
> Sent: Friday, January 26, 2018 9:36 AM
> To: fuego@lists.linuxfoundation.org
> Subject: [Fuego] [PATCH 7/9] Move dpkg-reconfigure dash to first RUN
> instruction
> 
> DEBIAN_FRONTEND=noninteractive has been explicitely set as an ARG in the
> beginning of the Dockerfile. That environment variable is set only
> during build time, not being exported with the container image.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile | 13 +++++++------
>  1 file changed, 7 insertions(+), 6 deletions(-)
> 
> diff --git a/Dockerfile b/Dockerfile
> index c39454d..499dd52 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -10,10 +10,11 @@ MAINTAINER tim.bird@sony.com
>  # Prepare basic image
>  #
> ==========================================================
> ====================
> 
> -WORKDIR /
> +ARG DEBIAN_FRONTEND=noninteractive
> +
>  COPY frontend-install/apt/sources/fuego-debian-jessie.list \
>          /etc/apt/sources.list.d/fuego-debian-jessie.list
> -RUN DEBIAN_FRONTEND=noninteractive apt-get update && \
> +RUN apt-get update && \
>      apt-get -yV install \
>          apt-utils \
>          at \
> @@ -71,9 +72,9 @@ RUN DEBIAN_FRONTEND=noninteractive apt-get
> update && \
>          vim \
>          wget \
>          xmlstarlet && \
> -    rm -rf /var/lib/apt/lists/*
> -
> -RUN echo dash dash/sh boolean false | debconf-set-selections ;
> DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash
> +    rm -rf /var/lib/apt/lists/* && \
> +    echo dash dash/sh boolean false | debconf-set-selections && \
> +    dpkg-reconfigure dash
> 
>  RUN pip install \
>          filelock \
> @@ -83,7 +84,7 @@ RUN pip install \
>  RUN echo deb http://emdebian.org/tools/debian/ jessie main >
> /etc/apt/sources.list.d/crosstools.list && \
>      curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key |
> apt-key add - && \
>      dpkg --add-architecture armhf && \
> -    DEBIAN_FRONTEND=noninteractive apt-get update && \
> +    apt-get update && \
>      apt-get -yV install \
>          binutils-arm-linux-gnueabihf \
>          cpp-arm-linux-gnueabihf \
> --
> 2.15.1
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 8/9] Refactor plugin installation script

[Tim.Bird]

OK - applied.

Thanks.
 -- Tim


> -----Original Message-----
> From: Guilherme Campos Camargo on Friday, January 26, 2018 9:36 AM
> On this patch we're trying to remove the explicit sleeps from the plugin
> installation script, that are necessary for waiting for Jenkins
> start/restart and the installation of the plugins.
> 
> With this change we hope that faster hosts (or hosts with faster
> networks) will be able to finish Jenkins plugins installation sooner.
> 
> We also took the chance to remove the call to sudo, that are not
> necessary for using jenkins-cli.
> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  frontend-install/setup/jenkins/plugins/install.sh | 44 +++++++++++++++++-
> -----
>  1 file changed, 33 insertions(+), 11 deletions(-)
> 
> diff --git a/frontend-install/setup/jenkins/plugins/install.sh b/frontend-
> install/setup/jenkins/plugins/install.sh
> index 0db8bf2..37e8948 100755
> --- a/frontend-install/setup/jenkins/plugins/install.sh
> +++ b/frontend-install/setup/jenkins/plugins/install.sh
> @@ -2,20 +2,42 @@
> 
>  set -e
> 
> +function install_jenkins_plugin {
> +    echo "Waiting for Jenkins update center..."
> +    while ! ret=$(java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin "$1" 2>/dev/null); do
> +        sleep "$2"
> +    done
> +    echo "${ret}"
> +}
> +
> +function wait_for_jenkins {
> +    echo "Waiting for Jenkins..."
> +    while ! curl --output /dev/null --silent --head --fail
> http://localhost:8080/fuego/ ; do
> +        sleep "$1"
> +    done
> +}
> +
> +readonly sleep_time_s=1
> +readonly required_plugins="
> +   description-setter
> +   pegdown-formatter
> +   flot-plotter-plugin/flot.hpi
> +   "
> +
>  service jenkins start
> -sleep 30
> -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin description-setter
> -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin pegdown-formatter
> -sudo -u jenkins java -jar /var/cache/jenkins/war/WEB-INF/jenkins-cli.jar -s
> http://localhost:8080/fuego install-plugin flot-plotter-plugin/flot.hpi
> -sleep 10
> +wait_for_jenkins ${sleep_time_s}
> +
> +for plugin in ${required_plugins}; do
> +    install_jenkins_plugin "${plugin}" "${sleep_time_s}"
> +done
> 
>  # Let Jenkins install flot before making the symlink
>  service jenkins restart
> -sleep 30
> -rm $JENKINS_HOME/plugins/flot/flot/mod.js
> -ln -s /fuego-core/engine/scripts/mod.js
> $JENKINS_HOME/plugins/flot/flot/mod.js
> -ln -s /fuego-rw/logs $JENKINS_HOME/userContent/fuego.logs
> -ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
> +wait_for_jenkins 1
> 
> -chown -R jenkins:jenkins $JENKINS_HOME/
> +rm "$JENKINS_HOME/plugins/flot/flot/mod.js"
> +ln -s /fuego-core/engine/scripts/mod.js
> "$JENKINS_HOME/plugins/flot/flot/mod.js"
> +ln -s /fuego-rw/logs "$JENKINS_HOME/userContent/fuego.logs"
> +ln -s /fuego-core/engine/scripts/ftc /usr/local/bin/
> 
> +chown -R jenkins:jenkins "$JENKINS_HOME/"
> --
> 2.15.1

Re: [Fuego] [PATCH 9/9] Set depth to "one" and download just a single-branch on git clones

[Tim.Bird]

> -----Original Message-----
> From: Guilherme Campos Camargo on Friday, January 26, 2018 9:36 AM
> In order to improve git clone speeds
For repositories this small, this does not speed up the clone operation
appreciably.  Since it makes the clone commands a bit ugly, I don't
think the tradeoff is worth it.

I tested this on my machine (with a relatively fast network), and the
speed for all these clones, without the extra args, was under 1 
second each.  The time difference with the extra args was less than
100 milliseconds.

Do these take longer for you?  These are relatively small repositories.
-- Tim

> 
> Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> ---
>  Dockerfile | 9 ++++++---
>  1 file changed, 6 insertions(+), 3 deletions(-)
> 
> diff --git a/Dockerfile b/Dockerfile
> index 499dd52..0eeeca0 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -120,13 +120,16 @@ RUN ./install.sh
>  COPY frontend-install/setup/serial /setup/serial
>  WORKDIR /setup
> 
> -RUN git clone https://github.com/tbird20d/ttc.git /usr/local/src/ttc && \
> +RUN git clone https://github.com/tbird20d/ttc.git \
> +        --depth 1 --single-branch /usr/local/src/ttc && \
>      ./serial/setup-ttc.sh
> 
> -RUN git clone https://github.com/frowand/serio.git /usr/local/src/serio &&
> \
> +RUN git clone https://github.com/frowand/serio.git \
> +        --depth 1 --single-branch /usr/local/src/serio && \
>      ./serial/setup-serio.sh
> 
> -RUN git clone https://github.com/tbird20d/serlogin.git
> /usr/local/src/serlogin && \
> +RUN git clone https://github.com/tbird20d/serlogin.git \
> +        --depth 1 --single-branch /usr/local/src/serlogin && \
>      ./serial/setup-serlogin.sh
> 
>  COPY frontend-install/setup/lava /setup/lava
> --
> 2.15.1
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego

Re: [Fuego] [PATCH 5/9] Remove jenkins user created during build time with host's uid/gid

[Guilherme Camargo]

[-- Attachment #1: Type: text/plain, Size: 12068 bytes --]

Hello, Tim

Thanks for your input.

On Tue, Jan 30, 2018 at 6:39 PM, <Tim.Bird@sony.com> wrote:

> > -----Original Message-----
> > From: Guilherme Campos Camargo
> >
> > Prior to this patch, the fuego-host-scripts/docker-build-image.sh script
> > was calling docker build passing two `build-arg`s: uid/gid. According to
> > the comments in the code, it seems that the intention was to make sure
> > that the jenkins user (needed by Jenkins) would have the same uid/gid as
> > the user of the host that has called the install script.
> >
> > Given that the uid/gid of the jenkins user was being set at build time,
> > the generated image would be configured only for the machine where it's
> > built, what would prevent jenkins to change files in the /fuego-rw
> > mountpoint in different hosts.
>
> That's correct, and changing this is a nice step forward to making the
> container usable on other machines than the one it was built on.
>
> >
> > One approach for allowing jenkins to write to fuego-rw, is to chown
> > /fuego-rw and it's pre-existing subdirectories during execution time,
> > what can be done in a docker entrypoint script. That approach per se,
> > solves the problem, by allowing jenkins to write to the mount point, but
> > generates a side-effect: the user on the host machine will be able to
> > inspect, but will not be able to delete/modify the files created by
> > jenkins (unless it forces it as root).
> >
> > Another approach would be to change the uid and the gid of the jenkins
> > user in the first execution, (first call to docker-container-start.sh),
> > also through the entrypoint.  This would solve the problem of the access
> > from both sides, but has the downside of the need of running a recursive
> > `chown` in all jenkins files in `/var` (including cache), possibly
> > taking a long time to be accomplished.
>
> I'd much rather do this.  This amounts to a one-time container installation
> overhead on a new host, for the Fuego user.
>
> I modify files in fuego-ro and fuego-rw all the time, from the host side.
> This is my normal workflow, and I'd like to see it supported.
>
> >
> > Since the need to edit/remove the files from fuego-rw (without sudo)
> > depends on the user, our suggestion is to use the entrypoint for
> > `chowning` only the pre-existing directories from fuego-rw and to
> > provide the user a script for mapping the jenkins' uid/gid with user's
> > uid/gid only if the user needs to.
> >
> > On this patch we have created the entrypoint.sh script, and the uid/gid
> > mapping script.
>
> At this point I'm confused.  Do both approaches have an entrypoint.sh
> script?


> Could you give an example of the steps required to install the container
> in both your scenarios, including the arguments required for the scripts,
> and the resulting permissions and ownership both inside and outside
> the container, for the volumes?
>

With both approaches, there would be no change compared to the instructions
that
are given in Fuego's Quick Start guide.​

$ ./install.sh
$ ./fuego-host-scripts/create-container.sh
$ ./fuego-host-scripts/start-container.sh

We agree with you, and decided to move to the second approach
(set the uid/gid at the first start through the entrypoint).

Explaining:

With the first approach (the one that was implemented in the first version
of
my patch), the uid and the gid of the Jenkins user, inside the container,
would
be different than the uid/gid of the user that's running the container from
the
host. In order to allow Jenkins to write into fuego-rw, we were changing the
ownership of fuego-rw, fuego-rw/boards, fuego-rw/logs and
fuego-rw/buildzone to
the Jenkins user inside the container (not recursively). This would allow
jenkins to write to those directories and the change would propagate to the
outside, not allowing the user to modify the contents of fuego-rw anymore.

Apparently there was a misconception from our side, that most users would
not use fuego-rw to write, but only to read, and that Jenkins would be the
only
one (most of times) to write into that directory.

In order to allow those users to write into that directory as well, we had
added a new script (fuego-host-scripts/jenkins-map-uid.sh), that mapped
Jenkin's uid/gid to the host user's only on-demand.

In order to run that script the user would simply call without any arguments
$ ./fuego-host-scripts/jenkins-map-uid.sh

On docker, the only way to run a script inside the container is to have that
script copied to the container. Therefore,
fuego-hosts-scripts/jenkins-map-uid.sh was just calling that other script
(setup/jenkins/map-jenkins-uid-to-host.sh) that's been copied during Docker
build. "map-jenkins-uid-to-host.sh" is the one that would actually change
jenkins uid and chown the nedeed directories.

With the second approach (the one that we have implemented in our fixup),
the uid and the gid of the Jenkins user, inside the container, would be the
same as the
user that's running the container from the host. What means that, if the
owner
of the fuego-rw directory is the same user that's running the container,
fuego-rw will be accessible from both sides seamlessly.

The downside of this approach is that the create-container.sh step would
take
longer, due to the recursive chowns that are needed after replacing Jenkins'
uid/gid with the user's.

However, as you said, given that most of users will actually need to write
to
fuego-rw, the second approach is clearly the best, as you agree. So I'm
sending
a new patch, implementing the second approach, as a replacement for this
one.

I'm rebasing over the new 'next' branch as well.


> I'm not sure when map-jenkins-uid-to-host.sh and
> jenkins-map-uid.sh are run, and there seem to be no instructions
> for using them, or a human-readable description of what they do.
>
> Thanks,
>  -- Tim
>
> >
> > Signed-off-by: Guilherme Campos Camargo <guicc@profusion.mobi>
> > ---
> >  Dockerfile                                             | 10 ++++------
> >  frontend-install/entrypoint.sh                         | 13
> +++++++++++++
> >  .../setup/jenkins/map-jenkins-uid-to-host.sh           | 12
> ++++++++++++
> >  frontend-install/setup/jenkins/setup.sh                |  8 --------
> >  fuego-host-scripts/docker-build-image.sh               | 10 +---------
> >  fuego-host-scripts/jenkins-map-uid.sh                  | 18
> ++++++++++++++++++
> >  6 files changed, 48 insertions(+), 23 deletions(-)
> >  create mode 100755 frontend-install/entrypoint.sh
> >  create mode 100755 frontend-install/setup/jenkins/map-jenkins-uid-to-
> > host.sh
> >  create mode 100755 fuego-host-scripts/jenkins-map-uid.sh
> >
> > diff --git a/Dockerfile b/Dockerfile
> > index 883ac21..493414e 100644
> > --- a/Dockerfile
> > +++ b/Dockerfile
> > @@ -110,16 +110,12 @@ RUN echo deb http://emdebian.org/tools/debian/
> > jessie main > /etc/apt/sources.li
> >  # Download and Install Jenkins
> >  #
> > ==========================================================
> > ====================
> >
> > -ENV uid=1000
> > -ENV gid=${uid}
> >  ARG JENKINS_VERSION=2.32.1
> >  ARG JENKINS_SHA=bfc226aabe2bb089623772950c4cc13aee613af1
> >  ARG JENKINS_URL=https://pkg.jenkins.io/debian-
> > stable/binary/jenkins_${JENKINS_VERSION}_all.deb
> >  ENV JENKINS_HOME=/var/lib/jenkins
> >
> > -RUN groupadd -g ${gid} ${group} && \
> > -    useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> > /bin/bash ${user} && \
> > -    curl -L -O ${JENKINS_URL} && \
> > +RUN curl -L -O ${JENKINS_URL} && \
> >      echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum
> > -c - && \
> >      dpkg -i jenkins_${JENKINS_VERSION}_all.deb && \
> >      rm jenkins_${JENKINS_VERSION}_all.deb
> > @@ -156,4 +152,6 @@ COPY docs/fuego-docs.pdf
> > $JENKINS_HOME/userContent/docs/fuego-docs.pdf
> >  # Setup startup command
> >  #
> > ==========================================================
> > ====================
> >
> > -ENTRYPOINT service jenkins start && service netperf start && /bin/bash
> > +WORKDIR /
> > +COPY frontend-install/entrypoint.sh /
> > +ENTRYPOINT ["/entrypoint.sh"]
> > diff --git a/frontend-install/entrypoint.sh
> b/frontend-install/entrypoint.sh
> > new file mode 100755
> > index 0000000..281b0fe
> > --- /dev/null
> > +++ b/frontend-install/entrypoint.sh
> > @@ -0,0 +1,13 @@
> > +#!/bin/bash
> > +set -e
> > +
> > +service jenkins stop >> /dev/null
> > +
> > +mkdir -p fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
> > +chown jenkins:jenkins \
> > +    fuego-rw fuego-rw/boards /fuego-rw/logs /fuego-rw/buildzone
> > +
> > +service jenkins start
> > +service netperf start
> > +
> > +exec /bin/bash
> > diff --git a/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
> > b/frontend-install/setup/jenkins/map-jenkins-uid-to-host.sh
> > new file mode 100755
> > index 0000000..002f165
> > --- /dev/null
> > +++ b/frontend-install/
> ​​
> setup/jenkins/map-jenkins-uid-to-host.sh
>
> Who calls this?  When, and with what args?
>
> > @@ -0,0 +1,12 @@
> > +#!/bin/bash
> > +set -e
> > +
> > +service jenkins stop
> > +
> > +usermod -u "${jenkins_uid}" jenkins
> > +groupmod -g "${jenkins_gid}" jenkins
> > +
> > +chown -R "${jenkins_uid}":"${jenkins_gid}" \
> > +    /var/lib/jenkins /var/cache/jenkins /var/log/jenkins /fuego-rw
> > +
> > +service jenkins start
> > diff --git a/frontend-install/setup/jenkins/setup.sh b/frontend-
> > install/setup/jenkins/setup.sh
> > index c550862..11ad371 100755
> > --- a/frontend-install/setup/jenkins/setup.sh
> > +++ b/frontend-install/setup/jenkins/setup.sh
> > @@ -2,14 +2,6 @@
> >
> >  set -e
> >
> > -user=jenkins
> > -group=jenkins
> > -uid=1000
> > -gid=${uid}
> > -
> > -# groupadd -g ${gid} ${group}
> > -# useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s
> > /bin/bash ${user}
> > -
> >  cp config.xml jenkins.model.JenkinsLocationConfiguration.xml
> > ${JENKINS_HOME}
> >
> >  source /etc/default/jenkins
> > diff --git a/fuego-host-scripts/docker-build-image.sh b/fuego-host-
> > scripts/docker-build-image.sh
> > index 7c11b11..6550a98 100755
> > --- a/fuego-host-scripts/docker-build-image.sh
> > +++ b/fuego-host-scripts/docker-build-image.sh
> > @@ -2,12 +2,4 @@
> >  # $1 - name for the docker image (default: fuego)
> >  DOCKERIMAGE=${1:-fuego}
> >
> > -if [ "$(id -u)" == "0" ]; then
> > -     JENKINS_UID=$(id -u $SUDO_USER)
> > -     JENKINS_GID=$(id -g $SUDO_USER)
> > -else
> > -     JENKINS_UID=$(id -u $USER)
> > -     JENKINS_GID=$(id -g $USER)
> > -fi
> > -
> > -sudo docker build -t ${DOCKERIMAGE} --build-arg
> > HTTP_PROXY=$http_proxy --build-arg uid=$JENKINS_UID --build-arg
> > gid=$JENKINS_GID .
> > +sudo docker build -t ${DOCKERIMAGE} --build-arg
> > HTTP_PROXY=$http_proxy .
> > diff --git a/fuego-host-scripts/jenkins-map-uid.sh b/fuego-host-
> > scripts/jenkins-map-uid.sh
> > new file mode 100755
> > index 0000000..fad619e
> > --- /dev/null
> > +++ b/fuego-host-scripts/jenkins-map-uid.sh
>
> Who calls this?  When, and with what args?
>
> > @@ -0,0 +1,18 @@
> > +#!/bin/bash
> > +# $1 - name for the docker container (default: fuego-container)
> > +DOCKERCONTAINER=${1:-fuego-container}
> > +
> > +if [ "${UID}" == "0" ]; then
> > +     uid=$(id -u "${SUDO_USER}")
> > +     gid=$(id -g "${SUDO_USER}")
> > +else
> > +     uid="${UID}"
> > +     gid=$(id -g "${USER}")
> > +fi
> > +
> > +sudo docker start "${DOCKERCONTAINER}" || \
> > +  echo "Please create Fuego docker container via docker-create-
> > container.sh script"
> > +
> > +sudo docker exec -e jenkins_uid="${uid}" -e jenkins_gid="${gid}" \
> > +    "${DOCKERCONTAINER}" \
> > +    /setup/jenkins/map-jenkins-uid-to-host.sh
> > --
> > 2.15.1
>
>
> I need to understand this better before I apply it.
>  -- Tim
>
>

[-- Attachment #2: Type: text/html, Size: 19309 bytes --]