* [Fuego] Proxy settings and gitignore for Fuego
@ 2016-10-11 7:46 Daniel Sangorrin
2016-10-11 7:46 ` [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy Daniel Sangorrin
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Daniel Sangorrin @ 2016-10-11 7:46 UTC (permalink / raw)
To: fuego; +Cc: dmitry.cherkasov
Hi all,
I'm sending a couple of patches that I just wrote while
trying the latest Fuego dockerfile.
The first one is for using Fuego behind a proxy. From the user's
point of view nothing changes: the proxy's information is
automatically extracted from the environment variable 'http_proxy'
by docker-build.sh.
TODO: Could someone check that it works when http_proxy is empty?
The second patch is just a .gitignore that contains a few files
and folder names that, as far as I know, are not supposed to be
committed. I guess more entries will need to be added.
[PATCH 1/2] proxy: add support for environments behind a proxy
[PATCH 2/2] gitignore: add a gitignore file
Best regards,
Daniel Sangorrin
IoT Technology center, TOSHIBA
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy
2016-10-11 7:46 [Fuego] Proxy settings and gitignore for Fuego Daniel Sangorrin
@ 2016-10-11 7:46 ` Daniel Sangorrin
2016-10-18 17:24 ` Dmitry Cherkasov
2016-10-11 7:46 ` [Fuego] [PATCH 2/2] gitignore: add a gitignore file Daniel Sangorrin
2016-10-11 18:09 ` [Fuego] Proxy settings and gitignore for Fuego Bird, Timothy
2 siblings, 1 reply; 6+ messages in thread
From: Daniel Sangorrin @ 2016-10-11 7:46 UTC (permalink / raw)
To: fuego; +Cc: dmitry.cherkasov
This uses docker builder's --build-arg in order to pass the
value of http_proxy to the Dockerfile. The toolchain install
has been moved to the Dockerfile to make proxy settings
simple. I also added a slight reformatting of the Dockerfile.
In particular, the variable defining the fuego-core commit
has been moved to the head of the Dockerfile.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
Dockerfile | 30 +++++++++-------------
frontend-install/jenkins-updates/constants.sh | 9 ++++---
fuego-host-scripts/docker-build.sh | 2 +-
.../install-arm-linux-gnueabihf-toolchain.sh | 7 -----
4 files changed, 18 insertions(+), 30 deletions(-)
delete mode 100755 fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
diff --git a/Dockerfile b/Dockerfile
index 7e03944..7ad3f0c 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,13 +1,14 @@
# ==============================================================================
# WARNING: containter created from this image should be run with userdata mounted at /userdata inside docker fs
# ==============================================================================
-
FROM debian:jessie
MAINTAINER dmitrii.cherkasov@cogentembedded.com
# ==============================================================================
# Influential environment variables
# ==============================================================================
+ARG HTTP_PROXY
+ENV INST_FUEGO_CORE_GIT_REVISION ab2aee1c9266e6553c57b68699246f394d31062a
ENV INST_FUEGO_ENGINE_PATH /home/jenkins
ENV INST_FUEGO_FRONTEND_PATH /var/lib/jenkins
# URL_PREFIX sets Jenkins URL --prefix note: no trailing "/" at the end!
@@ -19,30 +20,32 @@ ENV URL_PREFIX /fuego
WORKDIR /fuego-install
RUN dpkg --add-architecture i386
RUN echo deb http://ftp.us.debian.org/debian jessie main non-free >> /etc/apt/sources.list
+RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install apt-utils daemon gcc make python-paramiko python-lxml python-simplejson python-matplotlib libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf netperf netpipe-tcp texlive-latex-base sshpass wget git sudo net-tools vim openssh-server curl inotify-tools
RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure dash'
COPY frontend-install/jenkins_1.509.2_all.deb /fuego-install/
RUN dpkg -i /fuego-install/jenkins_1.509.2_all.deb
-RUN /bin/bash -c 'wget -nv "http://downloads.sourceforge.net/project/getfo/texml/texml-2.0.2/texml-2.0.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=1398789654&use_mirror=sunet" -O texml.tar.gz ; tar xvf texml.tar.gz; cd texml-2.0.2 ; python setup.py install; cd -'
+RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc && echo 'http_proxy =' $HTTP_PROXY >> /etc/wgetrc; fi
+RUN /bin/bash -c 'wget -nv "http://downloads.sourceforge.net/project/getfo/texml/texml-2.0.2/texml-2.0.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=1398789654&use_mirror=jaist" -O texml.tar.gz ; tar xvf texml.tar.gz; cd texml-2.0.2 ; python setup.py install; cd -'
RUN echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
# ==============================================================================
# Install debian armhf cross toolchain
# ==============================================================================
-
-COPY fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh /fuego-install/
-RUN bash /fuego-install/install-arm-linux-gnueabihf-toolchain.sh
+RUN echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list
+RUN dpkg --add-architecture armhf
+RUN if [ -n "$HTTP_PROXY" ]; then curl --proxy $HTTP_PROXY http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add -; else curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add -; fi
+RUN DEBIAN_FRONTEND=noninteractive apt-get update
+RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf
# ==============================================================================
# get Fuego core via git
# ==============================================================================
-ENV INST_FUEGO_CORE_GIT_REVISION ab2aee1c9266e6553c57b68699246f394d31062a
-
RUN mkdir -p /home/jenkins
+RUN if [ -n "$HTTP_PROXY" ]; then git config --global http.proxy $HTTP_PROXY; fi
RUN git clone https://cogentembedded@bitbucket.org/cogentembedded/fuego-core.git $INST_FUEGO_ENGINE_PATH/fuego && cd $INST_FUEGO_ENGINE_PATH/fuego && git reset --hard $INST_FUEGO_CORE_GIT_REVISION && cd /fuego-install
RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/engine/* $INST_FUEGO_ENGINE_PATH/
RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/jobs $INST_FUEGO_FRONTEND_PATH/jobs
-
COPY docs $INST_FUEGO_FRONTEND_PATH/userContent/docs/
# ==============================================================================
@@ -53,12 +56,10 @@ COPY fuego-scripts/maintain_config_link.sh /usr/local/bin/
# ==============================================================================
# Init userdata
# ==============================================================================
-
RUN ln -s /userdata/buildzone $INST_FUEGO_ENGINE_PATH/buildzone
RUN ln -s /userdata/work $INST_FUEGO_ENGINE_PATH/work
-RUN ln -s /userdata/logs $INST_FUEGO_ENGINE_PATH/logs
+RUN ln -s /userdata/logs $INST_FUEGO_ENGINE_PATH/logs
RUN ln -s /userdata/logs $INST_FUEGO_FRONTEND_PATH/logs
-
RUN ln -s /userdata/conf/boards $INST_FUEGO_ENGINE_PATH/overlays/boards
RUN ln -s /userdata/conf/config.xml $INST_FUEGO_FRONTEND_PATH/config.xml
RUN ln -s /userdata/conf/tools.sh $INST_FUEGO_ENGINE_PATH/scripts/tools.sh
@@ -67,10 +68,8 @@ RUN ln -s /userdata/conf/tools.sh $INST_FUEGO_ENGINE_PATH/scripts/tools.sh
# ==============================================================================
# Initialize Jenkins plugin configs
# ==============================================================================
-
RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/plugins-conf/scriptler $INST_FUEGO_FRONTEND_PATH/
RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/plugins-conf/sidebar-link.xml $INST_FUEGO_FRONTEND_PATH/
-
COPY frontend-install/jenkins.cfg /etc/default/jenkins
COPY fuego-scripts/subsitute_jen_url_prefix.sh /fuego-install/
RUN /fuego-install/subsitute_jen_url_prefix.sh /etc/default/jenkins
@@ -78,7 +77,6 @@ RUN /fuego-install/subsitute_jen_url_prefix.sh /etc/default/jenkins
# ==============================================================================
# Install Jenkins UI updates
# ==============================================================================
-
RUN chown -R jenkins $INST_FUEGO_ENGINE_PATH $INST_FUEGO_FRONTEND_PATH /var/cache/jenkins /etc/default/jenkins
COPY frontend-install/plugins $INST_FUEGO_FRONTEND_PATH/
COPY frontend-install/jenkins-updates /fuego-install/jenkins-updates
@@ -87,13 +85,11 @@ WORKDIR /fuego-install/jenkins-updates
RUN echo "installing custom UI updates"
RUN /etc/init.d/jenkins start && ./updates.sh
RUN ln -s $INST_FUEGO_ENGINE_PATH/logs $INST_FUEGO_FRONTEND_PATH/userContent/fuego.logs
-
RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/jobs/tests.info $INST_FUEGO_FRONTEND_PATH/userContent/tests.info
# ==============================================================================
# Setup daemons config
# ==============================================================================
-
COPY container-cfg/sshd_config /etc/ssh/sshd_config
COPY fuego-scripts/user-setup.sh /fuego-install/
RUN /fuego-install/user-setup.sh
@@ -101,10 +97,8 @@ RUN /fuego-install/user-setup.sh
# ==============================================================================
# Clear workspace
# ==============================================================================
-
WORKDIR /home/jenkins
-
# ==============================================================================
# Setup startup command
# ==============================================================================
diff --git a/frontend-install/jenkins-updates/constants.sh b/frontend-install/jenkins-updates/constants.sh
index 2e440ec..7016641 100755
--- a/frontend-install/jenkins-updates/constants.sh
+++ b/frontend-install/jenkins-updates/constants.sh
@@ -14,7 +14,7 @@ RSYNC_ARGS="-ahq --stats"
# $1 - path to fix recursively
fix_permissions() {
- chown -fRH jenkins "$1"
+ chown -fRH jenkins "$1"
}
output_errors_only() {
@@ -28,9 +28,10 @@ default_output() {
}
sync_fetch_jenkins_cli() {
- until wget -nv ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar
+ until wget -v --no-proxy ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar
do
- echo "Retrying wget -nv ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar"
- sleep 1
+ echo "Retrying wget -v --no-proxy ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar"
+ service jenkins restart
+ sleep 10
done
}
diff --git a/fuego-host-scripts/docker-build.sh b/fuego-host-scripts/docker-build.sh
index 7aeb386..656d201 100755
--- a/fuego-host-scripts/docker-build.sh
+++ b/fuego-host-scripts/docker-build.sh
@@ -1 +1 @@
-sudo docker build -t fuego .
+sudo docker build -t fuego --build-arg HTTP_PROXY=$http_proxy .
diff --git a/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh b/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
deleted file mode 100755
index 8e3f92f..0000000
--- a/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
+++ /dev/null
@@ -1,7 +0,0 @@
-#!/bin/bash
-
-echo deb http://emdebian.org/tools/debian/ jessie main > /etc/apt/sources.list.d/crosstools.list
-dpkg --add-architecture armhf
-curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo apt-key add -
-DEBIAN_FRONTEND=noninteractive apt-get update
-DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf
--
2.7.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Fuego] [PATCH 2/2] gitignore: add a gitignore file
2016-10-11 7:46 [Fuego] Proxy settings and gitignore for Fuego Daniel Sangorrin
2016-10-11 7:46 ` [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy Daniel Sangorrin
@ 2016-10-11 7:46 ` Daniel Sangorrin
2016-10-11 18:09 ` [Fuego] Proxy settings and gitignore for Fuego Bird, Timothy
2 siblings, 0 replies; 6+ messages in thread
From: Daniel Sangorrin @ 2016-10-11 7:46 UTC (permalink / raw)
To: fuego; +Cc: dmitry.cherkasov
Ignore autogenerated files and directories.
Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
---
.gitignore | 4 ++++
1 file changed, 4 insertions(+)
create mode 100644 .gitignore
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..804e061
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,4 @@
+last_fuego_container.id
+userdata/buildzone/
+userdata/logs/
+userdata/work/
--
2.7.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [Fuego] Proxy settings and gitignore for Fuego
2016-10-11 7:46 [Fuego] Proxy settings and gitignore for Fuego Daniel Sangorrin
2016-10-11 7:46 ` [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy Daniel Sangorrin
2016-10-11 7:46 ` [Fuego] [PATCH 2/2] gitignore: add a gitignore file Daniel Sangorrin
@ 2016-10-11 18:09 ` Bird, Timothy
2 siblings, 0 replies; 6+ messages in thread
From: Bird, Timothy @ 2016-10-11 18:09 UTC (permalink / raw)
To: Daniel Sangorrin, fuego; +Cc: dmitry.cherkasov
> -----Original Message-----
> From: Daniel Sangorrin [mailto:daniel.sangorrin@toshiba.co.jp]
> Hi all,
>
> I'm sending a couple of patches that I just wrote while
> trying the latest Fuego dockerfile.
>
> The first one is for using Fuego behind a proxy. From the user's
> point of view nothing changes: the proxy's information is
> automatically extracted from the environment variable 'http_proxy'
> by docker-build.sh.
Sounds good.
>
> TODO: Could someone check that it works when http_proxy is empty?
>
> The second patch is just a .gitignore that contains a few files
> and folder names that, as far as I know, are not supposed to be
> committed. I guess more entries will need to be added.
Thankyou, thankyou, thankyou. That will save a lot of git grief.
I'm pretty busy at ELCE this week, but I'll try to find some time
to test these and get them committed. Realistically, however,
it might not be until next week.
-- Tim
> [PATCH 1/2] proxy: add support for environments behind a proxy
> [PATCH 2/2] gitignore: add a gitignore file
>
> Best regards,
> Daniel Sangorrin
> IoT Technology center, TOSHIBA
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy
2016-10-11 7:46 ` [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy Daniel Sangorrin
@ 2016-10-18 17:24 ` Dmitry Cherkasov
2016-10-19 0:14 ` Daniel Sangorrin
0 siblings, 1 reply; 6+ messages in thread
From: Dmitry Cherkasov @ 2016-10-18 17:24 UTC (permalink / raw)
To: Daniel Sangorrin; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 11127 bytes --]
Hi Daniel. Thanks for the patches.
I am in process of reviewing & testing them.
One minor note:
> In particular, the variable defining the fuego-core commit has been moved
to the head of the Dockerfile.
that variable definition was put in that place for a reason. Namely, it
allows to skip all that lengthy debian package install process when
updating just version of fuego-core commit.
On Tue, Oct 11, 2016 at 10:46 AM, Daniel Sangorrin <
daniel.sangorrin@toshiba.co.jp> wrote:
> This uses docker builder's --build-arg in order to pass the
> value of http_proxy to the Dockerfile. The toolchain install
> has been moved to the Dockerfile to make proxy settings
> simple. I also added a slight reformatting of the Dockerfile.
> In particular, the variable defining the fuego-core commit
> has been moved to the head of the Dockerfile.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp>
> ---
> Dockerfile | 30
> +++++++++-------------
> frontend-install/jenkins-updates/constants.sh | 9 ++++---
> fuego-host-scripts/docker-build.sh | 2 +-
> .../install-arm-linux-gnueabihf-toolchain.sh | 7 -----
> 4 files changed, 18 insertions(+), 30 deletions(-)
> delete mode 100755 fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
>
> diff --git a/Dockerfile b/Dockerfile
> index 7e03944..7ad3f0c 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -1,13 +1,14 @@
> # ============================================================
> ==================
> # WARNING: containter created from this image should be run with userdata
> mounted at /userdata inside docker fs
> # ============================================================
> ==================
> -
> FROM debian:jessie
> MAINTAINER dmitrii.cherkasov@cogentembedded.com
>
> # ============================================================
> ==================
> # Influential environment variables
> # ============================================================
> ==================
> +ARG HTTP_PROXY
> +ENV INST_FUEGO_CORE_GIT_REVISION ab2aee1c9266e6553c57b68699246f394d31062a
> ENV INST_FUEGO_ENGINE_PATH /home/jenkins
> ENV INST_FUEGO_FRONTEND_PATH /var/lib/jenkins
> # URL_PREFIX sets Jenkins URL --prefix note: no trailing "/" at the end!
> @@ -19,30 +20,32 @@ ENV URL_PREFIX /fuego
> WORKDIR /fuego-install
> RUN dpkg --add-architecture i386
> RUN echo deb http://ftp.us.debian.org/debian jessie main non-free >>
> /etc/apt/sources.list
> +RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy
> "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install
> apt-utils daemon gcc make python-paramiko python-lxml python-simplejson
> python-matplotlib libtool xmlstarlet autoconf automake rsync openjdk-7-jre
> openjdk-7-jdk iperf netperf netpipe-tcp texlive-latex-base sshpass wget git
> sudo net-tools vim openssh-server curl inotify-tools
> RUN /bin/bash -c 'echo "dash dash/sh boolean false" |
> debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-reconfigure
> dash'
> COPY frontend-install/jenkins_1.509.2_all.deb /fuego-install/
> RUN dpkg -i /fuego-install/jenkins_1.509.2_all.deb
> -RUN /bin/bash -c 'wget -nv "http://downloads.sourceforge.
> net/project/getfo/texml/texml-2.0.2/texml-2.0.2.tar.gz?r=
> http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=
> 1398789654&use_mirror=sunet" -O texml.tar.gz ; tar xvf texml.tar.gz; cd
> texml-2.0.2 ; python setup.py install; cd -'
> +RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc &&
> echo 'http_proxy =' $HTTP_PROXY >> /etc/wgetrc; fi
> +RUN /bin/bash -c 'wget -nv "http://downloads.sourceforge.
> net/project/getfo/texml/texml-2.0.2/texml-2.0.2.tar.gz?r=
> http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=
> 1398789654&use_mirror=jaist" -O texml.tar.gz ; tar xvf texml.tar.gz; cd
> texml-2.0.2 ; python setup.py install; cd -'
> RUN echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
>
> # ============================================================
> ==================
> # Install debian armhf cross toolchain
> # ============================================================
> ==================
> -
> -COPY fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
> /fuego-install/
> -RUN bash /fuego-install/install-arm-linux-gnueabihf-toolchain.sh
> +RUN echo deb http://emdebian.org/tools/debian/ jessie main >
> /etc/apt/sources.list.d/crosstools.list
> +RUN dpkg --add-architecture armhf
> +RUN if [ -n "$HTTP_PROXY" ]; then curl --proxy $HTTP_PROXY
> http://emdebian.org/tools/debian/emdebian-toolchain-archive.key | sudo
> apt-key add -; else curl http://emdebian.org/tools/
> debian/emdebian-toolchain-archive.key | sudo apt-key add -; fi
> +RUN DEBIAN_FRONTEND=noninteractive apt-get update
> +RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install
> crossbuild-essential-armhf
>
> # ============================================================
> ==================
> # get Fuego core via git
> # ============================================================
> ==================
> -ENV INST_FUEGO_CORE_GIT_REVISION ab2aee1c9266e6553c57b68699246f394d31062a
> -
> RUN mkdir -p /home/jenkins
> +RUN if [ -n "$HTTP_PROXY" ]; then git config --global http.proxy
> $HTTP_PROXY; fi
> RUN git clone https://cogentembedded@bitbucket.org/cogentembedded/
> fuego-core.git $INST_FUEGO_ENGINE_PATH/fuego && cd
> $INST_FUEGO_ENGINE_PATH/fuego && git reset --hard
> $INST_FUEGO_CORE_GIT_REVISION && cd /fuego-install
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/engine/* $INST_FUEGO_ENGINE_PATH/
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/jobs
> $INST_FUEGO_FRONTEND_PATH/jobs
> -
> COPY docs $INST_FUEGO_FRONTEND_PATH/userContent/docs/
>
> # ============================================================
> ==================
> @@ -53,12 +56,10 @@ COPY fuego-scripts/maintain_config_link.sh
> /usr/local/bin/
> # ============================================================
> ==================
> # Init userdata
> # ============================================================
> ==================
> -
> RUN ln -s /userdata/buildzone $INST_FUEGO_ENGINE_PATH/buildzone
> RUN ln -s /userdata/work $INST_FUEGO_ENGINE_PATH/work
> -RUN ln -s /userdata/logs $INST_FUEGO_ENGINE_PATH/logs
> +RUN ln -s /userdata/logs $INST_FUEGO_ENGINE_PATH/logs
> RUN ln -s /userdata/logs $INST_FUEGO_FRONTEND_PATH/logs
> -
> RUN ln -s /userdata/conf/boards $INST_FUEGO_ENGINE_PATH/overlays/boards
> RUN ln -s /userdata/conf/config.xml $INST_FUEGO_FRONTEND_PATH/config.xml
> RUN ln -s /userdata/conf/tools.sh $INST_FUEGO_ENGINE_PATH/
> scripts/tools.sh
> @@ -67,10 +68,8 @@ RUN ln -s /userdata/conf/tools.sh
> $INST_FUEGO_ENGINE_PATH/scripts/tools.sh
> # ============================================================
> ==================
> # Initialize Jenkins plugin configs
> # ============================================================
> ==================
> -
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/plugins-conf/scriptler
> $INST_FUEGO_FRONTEND_PATH/
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/plugins-conf/sidebar-link.xml
> $INST_FUEGO_FRONTEND_PATH/
> -
> COPY frontend-install/jenkins.cfg /etc/default/jenkins
> COPY fuego-scripts/subsitute_jen_url_prefix.sh /fuego-install/
> RUN /fuego-install/subsitute_jen_url_prefix.sh /etc/default/jenkins
> @@ -78,7 +77,6 @@ RUN /fuego-install/subsitute_jen_url_prefix.sh
> /etc/default/jenkins
> # ============================================================
> ==================
> # Install Jenkins UI updates
> # ============================================================
> ==================
> -
> RUN chown -R jenkins $INST_FUEGO_ENGINE_PATH $INST_FUEGO_FRONTEND_PATH
> /var/cache/jenkins /etc/default/jenkins
> COPY frontend-install/plugins $INST_FUEGO_FRONTEND_PATH/
> COPY frontend-install/jenkins-updates /fuego-install/jenkins-updates
> @@ -87,13 +85,11 @@ WORKDIR /fuego-install/jenkins-updates
> RUN echo "installing custom UI updates"
> RUN /etc/init.d/jenkins start && ./updates.sh
> RUN ln -s $INST_FUEGO_ENGINE_PATH/logs $INST_FUEGO_FRONTEND_PATH/
> userContent/fuego.logs
> -
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/jobs/tests.info
> $INST_FUEGO_FRONTEND_PATH/userContent/tests.info
>
> # ============================================================
> ==================
> # Setup daemons config
> # ============================================================
> ==================
> -
> COPY container-cfg/sshd_config /etc/ssh/sshd_config
> COPY fuego-scripts/user-setup.sh /fuego-install/
> RUN /fuego-install/user-setup.sh
> @@ -101,10 +97,8 @@ RUN /fuego-install/user-setup.sh
> # ============================================================
> ==================
> # Clear workspace
> # ============================================================
> ==================
> -
> WORKDIR /home/jenkins
>
> -
> # ============================================================
> ==================
> # Setup startup command
> # ============================================================
> ==================
> diff --git a/frontend-install/jenkins-updates/constants.sh
> b/frontend-install/jenkins-updates/constants.sh
> index 2e440ec..7016641 100755
> --- a/frontend-install/jenkins-updates/constants.sh
> +++ b/frontend-install/jenkins-updates/constants.sh
> @@ -14,7 +14,7 @@ RSYNC_ARGS="-ahq --stats"
>
> # $1 - path to fix recursively
> fix_permissions() {
> - chown -fRH jenkins "$1"
> + chown -fRH jenkins "$1"
> }
>
> output_errors_only() {
> @@ -28,9 +28,10 @@ default_output() {
> }
>
> sync_fetch_jenkins_cli() {
> - until wget -nv ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar
> + until wget -v --no-proxy ${JEN_URL}/jnlpJars/jenkins-cli.jar -O
> jenkins-cli.jar
> do
> - echo "Retrying wget -nv ${JEN_URL}/jnlpJars/jenkins-cli.jar -O
> jenkins-cli.jar"
> - sleep 1
> + echo "Retrying wget -v --no-proxy ${JEN_URL}/jnlpJars/jenkins-cli.jar
> -O jenkins-cli.jar"
> + service jenkins restart
> + sleep 10
> done
> }
> diff --git a/fuego-host-scripts/docker-build.sh
> b/fuego-host-scripts/docker-build.sh
> index 7aeb386..656d201 100755
> --- a/fuego-host-scripts/docker-build.sh
> +++ b/fuego-host-scripts/docker-build.sh
> @@ -1 +1 @@
> -sudo docker build -t fuego .
> +sudo docker build -t fuego --build-arg HTTP_PROXY=$http_proxy .
> diff --git a/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
> b/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
> deleted file mode 100755
> index 8e3f92f..0000000
> --- a/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -#!/bin/bash
> -
> -echo deb http://emdebian.org/tools/debian/ jessie main >
> /etc/apt/sources.list.d/crosstools.list
> -dpkg --add-architecture armhf
> -curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key |
> sudo apt-key add -
> -DEBIAN_FRONTEND=noninteractive apt-get update
> -DEBIAN_FRONTEND=noninteractive apt-get -yV install
> crossbuild-essential-armhf
> --
> 2.7.4
>
>
>
[-- Attachment #2: Type: text/html, Size: 14381 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy
2016-10-18 17:24 ` Dmitry Cherkasov
@ 2016-10-19 0:14 ` Daniel Sangorrin
0 siblings, 0 replies; 6+ messages in thread
From: Daniel Sangorrin @ 2016-10-19 0:14 UTC (permalink / raw)
To: 'Dmitry Cherkasov'; +Cc: fuego
Hi Dmitry,
> -----Original Message-----
> From: Dmitry Cherkasov [mailto:dmitry.cherkasov@cogentembedded.com]
> Sent: Wednesday, October 19, 2016 2:24 AM
> To: Daniel Sangorrin
> Cc: fuego@lists.linuxfoundation.org; Tim.Bird@am.sony.com
> Subject: Re: [PATCH 1/2] proxy: add support for environments behind a proxy
>
> Hi Daniel. Thanks for the patches.
>
> I am in process of reviewing & testing them.
Thanks a lot for your review
> One minor note:
>
> > In particular, the variable defining the fuego-core commit has been moved to the head of the Dockerfile.
>
> that variable definition was put in that place for a reason. Namely, it allows to skip all that lengthy debian package install process when
> updating just version of fuego-core commit.
Oh, I see. I didn't know that it was there for that reason. Your feedback is very important. Please feel free to put it back where it was.
By the way, is the e-mail address in the Dockerfile correct?
Best regards,
Daniel
> On Tue, Oct 11, 2016 at 10:46 AM, Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp <mailto:daniel.sangorrin@toshiba.co.jp> > wrote:
>
>
> This uses docker builder's --build-arg in order to pass the
> value of http_proxy to the Dockerfile. The toolchain install
> has been moved to the Dockerfile to make proxy settings
> simple. I also added a slight reformatting of the Dockerfile.
> In particular, the variable defining the fuego-core commit
> has been moved to the head of the Dockerfile.
>
> Signed-off-by: Daniel Sangorrin <daniel.sangorrin@toshiba.co.jp <mailto:daniel.sangorrin@toshiba.co.jp> >
> ---
> Dockerfile | 30 +++++++++-------------
> frontend-install/jenkins-updates/constants.sh | 9 ++++---
> fuego-host-scripts/docker-build.sh | 2 +-
> .../install-arm-linux-gnueabihf-toolchain.sh | 7 -----
> 4 files changed, 18 insertions(+), 30 deletions(-)
> delete mode 100755 fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
>
> diff --git a/Dockerfile b/Dockerfile
> index 7e03944..7ad3f0c 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -1,13 +1,14 @@
> # ==============================================================================
> # WARNING: containter created from this image should be run with userdata mounted at /userdata inside docker fs
> # ==============================================================================
> -
> FROM debian:jessie
> MAINTAINER dmitrii.cherkasov@cogentembedded.com <mailto:dmitrii.cherkasov@cogentembedded.com>
>
> # ==============================================================================
> # Influential environment variables
> # ==============================================================================
> +ARG HTTP_PROXY
> +ENV INST_FUEGO_CORE_GIT_REVISION ab2aee1c9266e6553c57b68699246f394d31062a
> ENV INST_FUEGO_ENGINE_PATH /home/jenkins
> ENV INST_FUEGO_FRONTEND_PATH /var/lib/jenkins
> # URL_PREFIX sets Jenkins URL --prefix note: no trailing "/" at the end!
> @@ -19,30 +20,32 @@ ENV URL_PREFIX /fuego
> WORKDIR /fuego-install
> RUN dpkg --add-architecture i386
> RUN echo deb http://ftp.us.debian.org/debian <http://ftp.us.debian.org/debian> jessie main non-free >>
> /etc/apt/sources.list
> +RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' > /etc/apt/apt.conf.d/80proxy; fi
> RUN DEBIAN_FRONTEND=noninteractive apt-get update && apt-get -yV install apt-utils daemon gcc make python-paramiko
> python-lxml python-simplejson python-matplotlib libtool xmlstarlet autoconf automake rsync openjdk-7-jre openjdk-7-jdk iperf
> netperf netpipe-tcp texlive-latex-base sshpass wget git sudo net-tools vim openssh-server curl inotify-tools
> RUN /bin/bash -c 'echo "dash dash/sh boolean false" | debconf-set-selections ; DEBIAN_FRONTEND=noninteractive dpkg-
> reconfigure dash'
> COPY frontend-install/jenkins_1.509.2_all.deb /fuego-install/
> RUN dpkg -i /fuego-install/jenkins_1.509.2_all.deb
> -RUN /bin/bash -c 'wget -nv "http://downloads.sourceforge.net/project/getfo/texml/texml-2.0.2/texml-
> 2.0.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=1398789654&use_mirror=sunet
> <http://downloads.sourceforge.net/project/getfo/texml/texml-2.0.2/texml-
> 2.0.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=1398789654&use_mirror=sunet> " -O texml.tar.gz ; tar
> xvf texml.tar.gz; cd texml-2.0.2 ; python setup.py install; cd -'
> +RUN if [ -n "$HTTP_PROXY" ]; then echo "use_proxy = on" >> /etc/wgetrc && echo 'http_proxy =' $HTTP_PROXY >>
> /etc/wgetrc; fi
> +RUN /bin/bash -c 'wget -nv "http://downloads.sourceforge.net/project/getfo/texml/texml-2.0.2/texml-
> 2.0.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=1398789654&use_mirror=jaist
> <http://downloads.sourceforge.net/project/getfo/texml/texml-2.0.2/texml-
> 2.0.2.tar.gz?r=http%3A%2F%2Fsourceforge.net%2Fprojects%2Fgetfo%2F&ts=1398789654&use_mirror=jaist> " -O texml.tar.gz ; tar
> xvf texml.tar.gz; cd texml-2.0.2 ; python setup.py install; cd -'
> RUN echo "PermitRootLogin yes" >> /etc/ssh/sshd_config
>
> # ==============================================================================
> # Install debian armhf cross toolchain
> # ==============================================================================
> -
> -COPY fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh /fuego-install/
> -RUN bash /fuego-install/install-arm-linux-gnueabihf-toolchain.sh
> +RUN echo deb http://emdebian.org/tools/debian/ <http://emdebian.org/tools/debian/> jessie main >
> /etc/apt/sources.list.d/crosstools.list
> +RUN dpkg --add-architecture armhf
> +RUN if [ -n "$HTTP_PROXY" ]; then curl --proxy $HTTP_PROXY http://emdebian.org/tools/debian/emdebian-toolchain-
> archive.key <http://emdebian.org/tools/debian/emdebian-toolchain-archive.key> | sudo apt-key add -; else curl
> http://emdebian.org/tools/debian/emdebian-toolchain-archive.key <http://emdebian.org/tools/debian/emdebian-toolchain-
> archive.key> | sudo apt-key add -; fi
> +RUN DEBIAN_FRONTEND=noninteractive apt-get update
> +RUN DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf
>
> # ==============================================================================
> # get Fuego core via git
> # ==============================================================================
> -ENV INST_FUEGO_CORE_GIT_REVISION ab2aee1c9266e6553c57b68699246f394d31062a
> -
> RUN mkdir -p /home/jenkins
> +RUN if [ -n "$HTTP_PROXY" ]; then git config --global http.proxy $HTTP_PROXY; fi
> RUN git clone https://cogentembedded@bitbucket.org/cogentembedded/fuego-core.git
> <https://cogentembedded@bitbucket.org/cogentembedded/fuego-core.git> $INST_FUEGO_ENGINE_PATH/fuego && cd
> $INST_FUEGO_ENGINE_PATH/fuego && git reset --hard $INST_FUEGO_CORE_GIT_REVISION && cd /fuego-install
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/engine/* $INST_FUEGO_ENGINE_PATH/
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/jobs $INST_FUEGO_FRONTEND_PATH/jobs
> -
> COPY docs $INST_FUEGO_FRONTEND_PATH/userContent/docs/
>
> # ==============================================================================
> @@ -53,12 +56,10 @@ COPY fuego-scripts/maintain_config_link.sh /usr/local/bin/
> # ==============================================================================
> # Init userdata
> # ==============================================================================
> -
> RUN ln -s /userdata/buildzone $INST_FUEGO_ENGINE_PATH/buildzone
> RUN ln -s /userdata/work $INST_FUEGO_ENGINE_PATH/work
> -RUN ln -s /userdata/logs $INST_FUEGO_ENGINE_PATH/logs
> +RUN ln -s /userdata/logs $INST_FUEGO_ENGINE_PATH/logs
> RUN ln -s /userdata/logs $INST_FUEGO_FRONTEND_PATH/logs
> -
> RUN ln -s /userdata/conf/boards $INST_FUEGO_ENGINE_PATH/overlays/boards
> RUN ln -s /userdata/conf/config.xml $INST_FUEGO_FRONTEND_PATH/config.xml
> RUN ln -s /userdata/conf/tools.sh $INST_FUEGO_ENGINE_PATH/scripts/tools.sh
> @@ -67,10 +68,8 @@ RUN ln -s /userdata/conf/tools.sh $INST_FUEGO_ENGINE_PATH/scripts/tools.sh
> # ==============================================================================
> # Initialize Jenkins plugin configs
> # ==============================================================================
> -
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/plugins-conf/scriptler $INST_FUEGO_FRONTEND_PATH/
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/plugins-conf/sidebar-link.xml $INST_FUEGO_FRONTEND_PATH/
> -
> COPY frontend-install/jenkins.cfg /etc/default/jenkins
> COPY fuego-scripts/subsitute_jen_url_prefix.sh /fuego-install/
> RUN /fuego-install/subsitute_jen_url_prefix.sh /etc/default/jenkins
> @@ -78,7 +77,6 @@ RUN /fuego-install/subsitute_jen_url_prefix.sh /etc/default/jenkins
> # ==============================================================================
> # Install Jenkins UI updates
> # ==============================================================================
> -
> RUN chown -R jenkins $INST_FUEGO_ENGINE_PATH $INST_FUEGO_FRONTEND_PATH /var/cache/jenkins
> /etc/default/jenkins
> COPY frontend-install/plugins $INST_FUEGO_FRONTEND_PATH/
> COPY frontend-install/jenkins-updates /fuego-install/jenkins-updates
> @@ -87,13 +85,11 @@ WORKDIR /fuego-install/jenkins-updates
> RUN echo "installing custom UI updates"
> RUN /etc/init.d/jenkins start && ./updates.sh
> RUN ln -s $INST_FUEGO_ENGINE_PATH/logs $INST_FUEGO_FRONTEND_PATH/userContent/fuego.logs
> -
> RUN ln -s $INST_FUEGO_ENGINE_PATH/fuego/jobs/tests.info <http://tests.info>
> $INST_FUEGO_FRONTEND_PATH/userContent/tests.info <http://tests.info>
>
> # ==============================================================================
> # Setup daemons config
> # ==============================================================================
> -
> COPY container-cfg/sshd_config /etc/ssh/sshd_config
> COPY fuego-scripts/user-setup.sh /fuego-install/
> RUN /fuego-install/user-setup.sh
> @@ -101,10 +97,8 @@ RUN /fuego-install/user-setup.sh
> # ==============================================================================
> # Clear workspace
> # ==============================================================================
> -
> WORKDIR /home/jenkins
>
> -
> # ==============================================================================
> # Setup startup command
> # ==============================================================================
> diff --git a/frontend-install/jenkins-updates/constants.sh b/frontend-install/jenkins-updates/constants.sh
> index 2e440ec..7016641 100755
> --- a/frontend-install/jenkins-updates/constants.sh
> +++ b/frontend-install/jenkins-updates/constants.sh
> @@ -14,7 +14,7 @@ RSYNC_ARGS="-ahq --stats"
>
> # $1 - path to fix recursively
> fix_permissions() {
> - chown -fRH jenkins "$1"
> + chown -fRH jenkins "$1"
> }
>
> output_errors_only() {
> @@ -28,9 +28,10 @@ default_output() {
> }
>
> sync_fetch_jenkins_cli() {
> - until wget -nv ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar
> + until wget -v --no-proxy ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar
> do
> - echo "Retrying wget -nv ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar"
> - sleep 1
> + echo "Retrying wget -v --no-proxy ${JEN_URL}/jnlpJars/jenkins-cli.jar -O jenkins-cli.jar"
> + service jenkins restart
> + sleep 10
> done
> }
> diff --git a/fuego-host-scripts/docker-build.sh b/fuego-host-scripts/docker-build.sh
> index 7aeb386..656d201 100755
> --- a/fuego-host-scripts/docker-build.sh
> +++ b/fuego-host-scripts/docker-build.sh
> @@ -1 +1 @@
> -sudo docker build -t fuego .
> +sudo docker build -t fuego --build-arg HTTP_PROXY=$http_proxy .
> diff --git a/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh b/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
> deleted file mode 100755
> index 8e3f92f..0000000
> --- a/fuego-scripts/install-arm-linux-gnueabihf-toolchain.sh
> +++ /dev/null
> @@ -1,7 +0,0 @@
> -#!/bin/bash
> -
> -echo deb http://emdebian.org/tools/debian/ <http://emdebian.org/tools/debian/> jessie main >
> /etc/apt/sources.list.d/crosstools.list
> -dpkg --add-architecture armhf
> -curl http://emdebian.org/tools/debian/emdebian-toolchain-archive.key <http://emdebian.org/tools/debian/emdebian-
> toolchain-archive.key> | sudo apt-key add -
> -DEBIAN_FRONTEND=noninteractive apt-get update
> -DEBIAN_FRONTEND=noninteractive apt-get -yV install crossbuild-essential-armhf
> --
> 2.7.4
>
>
>
>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2016-10-19 0:14 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-11 7:46 [Fuego] Proxy settings and gitignore for Fuego Daniel Sangorrin
2016-10-11 7:46 ` [Fuego] [PATCH 1/2] proxy: add support for environments behind a proxy Daniel Sangorrin
2016-10-18 17:24 ` Dmitry Cherkasov
2016-10-19 0:14 ` Daniel Sangorrin
2016-10-11 7:46 ` [Fuego] [PATCH 2/2] gitignore: add a gitignore file Daniel Sangorrin
2016-10-11 18:09 ` [Fuego] Proxy settings and gitignore for Fuego Bird, Timothy
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.