[Fuego] [PATCH 1/2] fuego-host-scripts: Have docker executable and libltdl7 in container

[Fuego] [PATCH 1/2] fuego-host-scripts: Have docker executable and libltdl7 in container

[Laszlo Sitzer]

This is one step towards fixing Functional.fuego_release_test which is
currently broken.

In order to start fixing it, I made /usr/bin/docker available in the
container by bind-mounting it. To be able to execute the docker
command-line client libltdl7 had to be installed into the container.

Signed-off-by: Laszlo Sitzer <laszlo.sitzer@native-instruments.de>
---
 Dockerfile                                    | 5 +++++
 fuego-host-scripts/docker-create-container.sh | 1 +
 2 files changed, 6 insertions(+)

diff --git a/Dockerfile b/Dockerfile
index f42b6c7..e6b7af4 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -69,6 +69,11 @@ RUN apt-get -q=2 -V --no-install-recommends install \
     netpipe-tcp iputils-ping
 RUN pip install flake8
 
+# Fuego release test dependencies
+# - libltdl7 - needed for /usr/bin/docker
+RUN apt-get -q=2 -V --no-install-recommends install \
+        libltdl7
+
 # miscelaneous packages:
 # python-serial - used by serio
 # diffstat and vim - used by Tim
diff --git a/fuego-host-scripts/docker-create-container.sh b/fuego-host-scripts/docker-create-container.sh
index c122cd9..d7844e6 100755
--- a/fuego-host-scripts/docker-create-container.sh
+++ b/fuego-host-scripts/docker-create-container.sh
@@ -24,6 +24,7 @@ fi
 
 sudo docker create -it --name ${DOCKERCONTAINER} \
     -v /var/run/docker.sock:/var/run/docker.sock \
+    -v /usr/bin/docker:/usr/bin/docker \
     -v /boot:/boot:ro \
     -v $DIR/../fuego-rw:/fuego-rw \
     -v $DIR/../fuego-ro:/fuego-ro:ro \
-- 
2.27.0

[Fuego] [PATCH 2/2] Dockerfile: Add missing dependencies for release test

[Laszlo Sitzer]

Also allow the jenkins user in the container to use sudo without a
password. This is needed to automate interaction with sudo which
is used in fuego_test.sh of the release test and in the fuego-host-scripts.

Signed-off-by: Laszlo Sitzer <laszlo.sitzer@native-instruments.de>
---
 Dockerfile | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)

diff --git a/Dockerfile b/Dockerfile
index e6b7af4..f6ddacf 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -37,12 +37,20 @@ RUN if [ -n "$HTTP_PROXY" ]; then echo 'Acquire::http::proxy "'$HTTP_PROXY'";' >
 # - python-pexpect: ssh_exec (part of ttc)
 # - python-pip: to install filelock, flake8
 # - filelock: parser
+# - python3-pip: fuego_release_test
+# - python3-pexpect: fuego_release_test
+# - python3-pil: fuego_release_test
+# - docker: fuego_release_test
+# - selenium: fuego_release_test
 RUN mkdir -p /usr/share/man/man1
 RUN apt-get update -q=2 && apt-get -q=2 -V --no-install-recommends install \
 	python-lxml python-simplejson python-yaml python-openpyxl \
 	python-requests python-reportlab python-parsedatetime \
-	python-pexpect python-pip python-setuptools python-wheel
+	python-pexpect python-pip python-setuptools python-wheel \
+	python3-pip python3-pexpect python3-pil
 RUN pip install filelock
+RUN pip3 install docker=4.10
+RUN pip3 install selenium==2.141.0
 
 # Fuego command dependencies
 RUN apt-get -q=2 -V --no-install-recommends install \
@@ -122,6 +130,11 @@ RUN pip install python-jenkins==1.4.0
 RUN echo -e "JENKINS_PORT=$JENKINS_PORT" >> /etc/environment
 RUN getent group ${gid} >/dev/null || groupadd -g ${gid} ${group}
 RUN useradd -l -m -d "${JENKINS_HOME}" -u ${uid} -g ${gid} -G sudo -s /bin/bash ${user}
+
+# For 'Functional.fuego_release_test'
+#  - requires jenkins user to use sudo without a password
+RUN echo "${user} ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/no-passwd
+
 RUN wget -nv ${JENKINS_URL}
 RUN echo "${JENKINS_SHA} jenkins_${JENKINS_VERSION}_all.deb" | sha1sum -c -
 # allow Jenkins to start and install plugins, as part of dpkg installation
-- 
2.27.0

Re: [Fuego] [PATCH 1/2] fuego-host-scripts: Have docker executable and libltdl7 in container

[Bird, Tim]

> -----Original Message-----
> From:  Laszlo Sitzer
> 
> This is one step towards fixing Functional.fuego_release_test which is
> currently broken.
> 
> In order to start fixing it, I made /usr/bin/docker available in the
> container by bind-mounting it. To be able to execute the docker
> command-line client libltdl7 had to be installed into the container.
> 
> Signed-off-by: Laszlo Sitzer <laszlo.sitzer@native-instruments.de>
> ---
>  Dockerfile                                    | 5 +++++
>  fuego-host-scripts/docker-create-container.sh | 1 +
>  2 files changed, 6 insertions(+)
> 
> diff --git a/Dockerfile b/Dockerfile
> index f42b6c7..e6b7af4 100644
> --- a/Dockerfile
> +++ b/Dockerfile
> @@ -69,6 +69,11 @@ RUN apt-get -q=2 -V --no-install-recommends install \
>      netpipe-tcp iputils-ping
>  RUN pip install flake8
> 
> +# Fuego release test dependencies
> +# - libltdl7 - needed for /usr/bin/docker
> +RUN apt-get -q=2 -V --no-install-recommends install \
> +        libltdl7
> +
>  # miscelaneous packages:
>  # python-serial - used by serio
>  # diffstat and vim - used by Tim
> diff --git a/fuego-host-scripts/docker-create-container.sh b/fuego-host-scripts/docker-create-container.sh
> index c122cd9..d7844e6 100755
> --- a/fuego-host-scripts/docker-create-container.sh
> +++ b/fuego-host-scripts/docker-create-container.sh
> @@ -24,6 +24,7 @@ fi
> 
>  sudo docker create -it --name ${DOCKERCONTAINER} \
>      -v /var/run/docker.sock:/var/run/docker.sock \
> +    -v /usr/bin/docker:/usr/bin/docker \
>      -v /boot:/boot:ro \
>      -v $DIR/../fuego-rw:/fuego-rw \
>      -v $DIR/../fuego-ro:/fuego-ro:ro \
> --
> 2.27.0

I think this is not needed because docker is installed locally in the container
in Dockerfile.test.  I believe you said to ignore your changes related to
Dockerfile, after you found Dockerfile.test.  So I'm dropping this patch.
Let me know if that's not right.
 -- Tim

Re: [Fuego] [PATCH 1/2] fuego-host-scripts: Have docker executable and libltdl7 in container

[daniel.sangorrin]

Hi

> -----Original Message-----
> From: Fuego <fuego-bounces@lists.linuxfoundation.org> On Behalf Of Bird, Tim
> Sent: Thursday, July 16, 2020 5:38 AM
> To: Laszlo Sitzer <laszlo.sitzer@native-instruments.de>; fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] [PATCH 1/2] fuego-host-scripts: Have docker executable and libltdl7 in container
> 
> 
> 
> > -----Original Message-----
> > From:  Laszlo Sitzer
> >
> > This is one step towards fixing Functional.fuego_release_test which is
> > currently broken.
> >
> > In order to start fixing it, I made /usr/bin/docker available in the
> > container by bind-mounting it. To be able to execute the docker
> > command-line client libltdl7 had to be installed into the container.
> >
> > Signed-off-by: Laszlo Sitzer <laszlo.sitzer@native-instruments.de>
> > ---
> >  Dockerfile                                    | 5 +++++
> >  fuego-host-scripts/docker-create-container.sh | 1 +
> >  2 files changed, 6 insertions(+)
> >
> > diff --git a/Dockerfile b/Dockerfile
> > index f42b6c7..e6b7af4 100644
> > --- a/Dockerfile
> > +++ b/Dockerfile
> > @@ -69,6 +69,11 @@ RUN apt-get -q=2 -V --no-install-recommends install \
> >      netpipe-tcp iputils-ping
> >  RUN pip install flake8
> >
> > +# Fuego release test dependencies
> > +# - libltdl7 - needed for /usr/bin/docker RUN apt-get -q=2 -V
> > +--no-install-recommends install \
> > +        libltdl7
> > +
> >  # miscelaneous packages:
> >  # python-serial - used by serio
> >  # diffstat and vim - used by Tim
> > diff --git a/fuego-host-scripts/docker-create-container.sh
> > b/fuego-host-scripts/docker-create-container.sh
> > index c122cd9..d7844e6 100755
> > --- a/fuego-host-scripts/docker-create-container.sh
> > +++ b/fuego-host-scripts/docker-create-container.sh
> > @@ -24,6 +24,7 @@ fi
> >
> >  sudo docker create -it --name ${DOCKERCONTAINER} \
> >      -v /var/run/docker.sock:/var/run/docker.sock \
> > +    -v /usr/bin/docker:/usr/bin/docker \
> >      -v /boot:/boot:ro \
> >      -v $DIR/../fuego-rw:/fuego-rw \
> >      -v $DIR/../fuego-ro:/fuego-ro:ro \

Related to the volume vs bind mount discussion. Since Docker 17.06 there is such distinction, so technically instead of -v we should be using --mount. However, Docker has kept back compatibility.
https://docs.docker.com/storage/bind-mounts/

Daniel


> > --
> > 2.27.0
> 
> I think this is not needed because docker is installed locally in the container in Dockerfile.test.  I believe you said to ignore your changes
> related to Dockerfile, after you found Dockerfile.test.  So I'm dropping this patch.
> Let me know if that's not right.
>  -- Tim
> 
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego