* [Fuego] release test issues
@ 2018-02-21 14:48 Tim.Bird
2018-02-21 17:23 ` Guilherme Camargo
0 siblings, 1 reply; 15+ messages in thread
From: Tim.Bird @ 2018-02-21 14:48 UTC (permalink / raw)
To: guicc; +Cc: fuego
> -----Original Message-----
> From: Guilherme Camargo
> Tim, please sync the repo again (
> https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test> ).
> I've force-pushed some changes.
>
> After that, you can just run `./build_and_run -c up`.
My container doesn't start Jenkins. I'm not sure if this is
a cascade error from some initial problem, but here is
some information about my situation.
I removed my repository, and did a new 'git clone' of the
fuego-release-test repository. It doesn't appear to me to
be updated recently:
$ git branch -v
* master 9d82c80 Initial commit
$ git log
commit 9d82c80d66a4de24ab97b38a16c8536849fea5bc
Author: Guilherme Campos Camargo <guicc@profusion.mobi>
Date: Tue Feb 6 02:13:20 2018 -0200
Initial commit
This repository has been created to store the resources needed for the
generation of a Docker image, based on fuego-base, that has all the
tools needed for the automation of the tests that are required for
validating a new Fuego release with Fuego itself.
...
I see that there are other branches with more recent commits (the
'ppa', and the 'fuego-base' branch. Should I be using one of these?
Here's the error I saw while trying build_and_run.sh:
-----
Downloading/unpacking pexpect
Downloading/unpacking ptyprocess>=0.5 (from pexpect)
Downloading ptyprocess-0.5.2-py2.py3-none-any.whl
Installing collected packages: pexpect, ptyprocess
Successfully installed pexpect ptyprocess
Cleaning up...
---> fd97e25fac02
Removing intermediate container 5a3a3638cb21
Step 4/7 : RUN echo "jenkins ALL = (root) NOPASSWD: /usr/bin/docker, /bin/sh" >> /etc/sudoers
---> Running in da0a1d824354
---> fb6468918ab5
Removing intermediate container da0a1d824354
Step 5/7 : COPY fuego-ro /fuego-ro
---> bbe089e90e5f
Removing intermediate container 6a4b20b92a4e
Step 6/7 : COPY tests /fuego-core/engine/tests
---> 57c7374dd097
Removing intermediate container aa6b4c80255c
Step 7/7 : COPY fuego-rt-entrypoint.sh /
---> 9ef063ab4a5b
Removing intermediate container 32a90a4f3abb
Successfully built 9ef063ab4a5b
Successfully tagged fuego-rt:latest
Error response from daemon: No such container: fuego-rt-container
6f513d49f117300672e1ba8782f0411448ffdf8e5258dd689797895cd79d8068
Waiting for Jenkins on http://localhost:8090/fuego/...
Creating job fuego-test.default.Functional.fuegotest
fuego-rt-container
[FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
[ ok ] Starting network benchmark server.
root@6f513d49f117:/#
-----
And here is some investigation:
-----
root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
INFO: Winstone shutdown successfully
Feb 21, 2018 2:13:22 PM winstone.Logger logInternal
SEVERE: Container startup failed
java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
at winstone.HostGroup.initHost(HostGroup.java:66)
at winstone.HostGroup.<init>(HostGroup.java:45)
at winstone.Launcher.<init>(Launcher.java:143)
at winstone.Launcher.main(Launcher.java:352)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at Main._main(Main.java:264)
at Main.main(Main.java:112)
root@6f513d49f117:/# su - jenkins
jenkins@6f513d49f117:~$ cat /var/cache/jenkins/war/META-INF/MANIFEST.MF
cat: /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission denied
jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins /var/cache/jenkins/war /var/cache/jenkins/war/META-INF /var/cache/jenkins/war/META-INF/MANIFEST.MF
ls: cannot access /var/cache/jenkins/war: Permission denied
ls: cannot access /var/cache/jenkins/war/META-INF: Permission denied
ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission denied
drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
jenkins@6f513d49f117:~$ exit
logout
root@6f513d49f117:/# ls -lad /var /var/cache /var/cache/jenkins /var/cache/jenkins/war /var/cache/jenkins/war/META-INF /var/cache/jenkins/war/META-INF/MANIFEST.MF
drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war
drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war/META-INF
-rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56 /var/cache/jenkins/war/META-INF/MANIFEST.MF
root@6f513d49f117:/# su - jenkins
jenkins@6f513d49f117:~$ id
uid=1000(jenkins) gid=987(jenkins) groups=987(jenkins)
jenkins@6f513d49f117:~$
-----
For reasons that escape me, it looks like these permissions are not
allowing jenkins in the container to access the 'war' directory:
drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
-----
root@6f513d49f117:/var/cache/jenkins# chmod o+rx .
root@6f513d49f117:/var/cache/jenkins# ls -la
total 12
drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 .
drwxr-xr-x 17 root root 4096 Feb 21 07:12 ..
drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 war
root@6f513d49f117:/var/cache/jenkins# su - jenkins
jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins /var/cache/jenkins/war /var/cache/jenkins/war/META-INF /var/cache/jenkins/war/META-INF/MANIFEST.MF
drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war
drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war/META-INF
-rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56 /var/cache/jenkins/war/META-INF/MANIFEST.MF
-----
However, maybe it's not that simple:
-----
jenkins@6f513d49f117:~$ exit
logout
root@6f513d49f117:/var/cache/jenkins# exit
exit
1000072457@tlinux:~/work/fuego/release-test-2018-02-19/fuego-release-test$ ./build_and_run.sh up
[FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
[ ok ] Starting network benchmark server.
root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
INFO: Winstone shutdown successfully
Feb 21, 2018 2:38:41 PM winstone.Logger logInternal
SEVERE: Container startup failed
java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
at winstone.HostGroup.initHost(HostGroup.java:66)
at winstone.HostGroup.<init>(HostGroup.java:45)
at winstone.Launcher.<init>(Launcher.java:143)
at winstone.Launcher.main(Launcher.java:352)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at Main._main(Main.java:264)
at Main.main(Main.java:112)
root@6f513d49f117:/# su - jenkins
jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins /var/cache/jenkins/war /var/cache/jenkins/war/META-INF /var/cache/jenkins/war/META-INF/MANIFEST.MF
ls: cannot access /var/cache/jenkins/war: Permission denied
ls: cannot access /var/cache/jenkins/war/META-INF: Permission denied
ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission denied
drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
I'm running this on Ubuntu 14.04, with docker 17.05.0-ce
-----
OK - in summary:
- I think I might be running the wrong branch of fuego-release-test
- I have weird permission problems in my docker container, for the fuego user and group
that are preventing jenkins from starting
- the jenkins user id and group id don't match my local user's user id and group id
Any help would be appreciated.
-- Tim
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-21 14:48 [Fuego] release test issues Tim.Bird
@ 2018-02-21 17:23 ` Guilherme Camargo
2018-02-22 1:19 ` Tim.Bird
0 siblings, 1 reply; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-21 17:23 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 12179 bytes --]
Hello, Tim, thanks for trying it out again and thanks for your time
in debugging this issue and providing logs.
This problem is happening because we're not using fuego-host-scripts
in the derived fuego-release-test, what ends up resulting in the
jenkins' user not being properly mapped to the host's user.
That's my fault, I forgot that this would be required in the
derivate image as well.
I temporarily added that logic to build_and_run.sh (as a fixup commit)
so that you're able to test it out, please see the patch below:
diff --git a/build_and_run.sh b/build_and_run.sh
index e448ddf..0134759 100755
--- a/build_and_run.sh
+++ b/build_and_run.sh
@@ -50,6 +50,25 @@ fi
if [ "$1" = "up" ]; then
if [ -v clean_start ]; then
+ # FIXME: We should provide a simpler way of mapping jenkins uid
with
+ # hosts uid, that does not require the user (that's extending
+ # fuego-base) to have this logic in his docker build.
+ #
+ # Jenkins' uid/gid should probably be given as a docker-run
variable
+ # (as http_proxy/https_proxy, and set properly through fuego-base
+ # entrypoint.)
+ #
+ if [ "${UID}" == "0" ]; then
+ uid=$(id -u "${SUDO_USER}")
+ gid=$(id -g "${SUDO_USER}")
+ else
+ uid="${UID}"
+ gid=$(id -g "${USER}")
+ fi
+
+ sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
+ sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
+
docker build -t "${fuego_rt_image}" .
docker rm -f "${fuego_rt_container}"
docker run -dit --name ${fuego_rt_container} \
diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
index 0c95eae..96cdcc9 100644
--- a/fuego-ro/conf/fuego.conf
+++ b/fuego-ro/conf/fuego.conf
@@ -1,3 +1,3 @@
-docker_jenkins_uid=1000
-docker_jenkins_gid=987
+docker_jenkins_uid=HOST_UID
+docker_jenkins_gid=HOST_GID
As I mentioned in my FIXME message, its undesirable to require a derivate
image to contain specific build steps (as the one that's replacing the id
into fuego.conf),
so we might need to come up with a better approach to make this work.
The patch has already been pushed to
https://bitbucket.org/profusionmobi/fuego-release-test (master).
Can you please re-sync and test again?
Thank you
--
Guilherme
On Wed, Feb 21, 2018 at 11:48 AM, <Tim.Bird@sony.com> wrote:
> > -----Original Message-----
> > From: Guilherme Camargo
> > Tim, please sync the repo again (
> >
>
> https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> ).
> > I've force-pushed some changes.
> >
> > After that, you can just run `./build_and_run -c up`.
>
> My container doesn't start Jenkins. I'm not sure if this is
> a cascade error from some initial problem, but here is
> some information about my situation.
>
> I removed my repository, and did a new 'git clone' of the
> fuego-release-test repository. It doesn't appear to me to
> be updated recently:
>
> $ git branch -v
> * master 9d82c80 Initial commit
> $ git log
> commit 9d82c80d66a4de24ab97b38a16c8536849fea5bc
> Author: Guilherme Campos Camargo <guicc@profusion.mobi>
> Date: Tue Feb 6 02:13:20 2018 -0200
>
> Initial commit
>
> This repository has been created to store the resources needed for the
> generation of a Docker image, based on fuego-base, that has all the
> tools needed for the automation of the tests that are required for
> validating a new Fuego release with Fuego itself.
> ...
>
> I see that there are other branches with more recent commits (the
> 'ppa', and the 'fuego-base' branch. Should I be using one of these?
>
> Here's the error I saw while trying build_and_run.sh:
>
> -----
> Downloading/unpacking pexpect
> Downloading/unpacking ptyprocess>=0.5 (from pexpect)
> Downloading ptyprocess-0.5.2-py2.py3-none-any.whl
> Installing collected packages: pexpect, ptyprocess
> Successfully installed pexpect ptyprocess
> Cleaning up...
> ---> fd97e25fac02
> Removing intermediate container 5a3a3638cb21
> Step 4/7 : RUN echo "jenkins ALL = (root) NOPASSWD: /usr/bin/docker,
> /bin/sh" >> /etc/sudoers
> ---> Running in da0a1d824354
> ---> fb6468918ab5
> Removing intermediate container da0a1d824354
> Step 5/7 : COPY fuego-ro /fuego-ro
> ---> bbe089e90e5f
> Removing intermediate container 6a4b20b92a4e
> Step 6/7 : COPY tests /fuego-core/engine/tests
> ---> 57c7374dd097
> Removing intermediate container aa6b4c80255c
> Step 7/7 : COPY fuego-rt-entrypoint.sh /
> ---> 9ef063ab4a5b
> Removing intermediate container 32a90a4f3abb
> Successfully built 9ef063ab4a5b
> Successfully tagged fuego-rt:latest
> Error response from daemon: No such container: fuego-rt-container
> 6f513d49f117300672e1ba8782f0411448ffdf8e5258dd689797895cd79d8068
> Waiting for Jenkins on http://localhost:8090/fuego/...
> Creating job fuego-test.default.Functional.fuegotest
> fuego-rt-container
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root@6f513d49f117:/#
>
> -----
>
> And here is some investigation:
> -----
> root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 21, 2018 2:13:22 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at winstone.HostConfiguration.getWebRoot(HostConfiguration.
> java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> root@6f513d49f117:/# su - jenkins
> jenkins@6f513d49f117:~$ cat /var/cache/jenkins/war/META-INF/MANIFEST.MF
> cat: /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission denied
> jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> ls: cannot access /var/cache/jenkins/war: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission
> denied
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> jenkins@6f513d49f117:~$ exit
> logout
> root@6f513d49f117:/# ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> root@6f513d49f117:/# su - jenkins
> jenkins@6f513d49f117:~$ id
> uid=1000(jenkins) gid=987(jenkins) groups=987(jenkins)
> jenkins@6f513d49f117:~$
> -----
> For reasons that escape me, it looks like these permissions are not
> allowing jenkins in the container to access the 'war' directory:
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
>
> -----
> root@6f513d49f117:/var/cache/jenkins# chmod o+rx .
> root@6f513d49f117:/var/cache/jenkins# ls -la
> total 12
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 .
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 ..
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 war
> root@6f513d49f117:/var/cache/jenkins# su - jenkins
> jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> -----
> However, maybe it's not that simple:
> -----
>
> jenkins@6f513d49f117:~$ exit
> logout
> root@6f513d49f117:/var/cache/jenkins# exit
> exit
> 1000072457@tlinux:~/work/fuego/release-test-2018-02-19/fuego-release-test$
> ./build_and_run.sh up
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 21, 2018 2:38:41 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at winstone.HostConfiguration.getWebRoot(HostConfiguration.
> java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> root@6f513d49f117:/# su - jenkins
> jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> ls: cannot access /var/cache/jenkins/war: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission
> denied
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
>
> I'm running this on Ubuntu 14.04, with docker 17.05.0-ce
>
> -----
> OK - in summary:
>
> - I think I might be running the wrong branch of fuego-release-test
> - I have weird permission problems in my docker container, for the fuego
> user and group
> that are preventing jenkins from starting
> - the jenkins user id and group id don't match my local user's user id and
> group id
>
> Any help would be appreciated.
> -- Tim
>
>
>
[-- Attachment #2: Type: text/html, Size: 17486 bytes --]
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-21 17:23 ` Guilherme Camargo
@ 2018-02-22 1:19 ` Tim.Bird
2018-02-22 1:59 ` Guilherme Camargo
0 siblings, 1 reply; 15+ messages in thread
From: Tim.Bird @ 2018-02-22 1:19 UTC (permalink / raw)
To: guicc; +Cc: fuego
> -----Original Message-----
> From: Guilherme Camargo
>
> Hello, Tim, thanks for trying it out again and thanks for your time
> in debugging this issue and providing logs.
>
> This problem is happening because we're not using fuego-host-scripts
> in the derived fuego-release-test, what ends up resulting in the
> jenkins' user not being properly mapped to the host's user.
>
> That's my fault, I forgot that this would be required in the
> derivate image as well.
>
> I temporarily added that logic to build_and_run.sh (as a fixup commit)
>
> so that you're able to test it out, please see the patch below:
>
> diff --git a/build_and_run.sh b/build_and_run.sh
> index e448ddf..0134759 100755
> --- a/build_and_run.sh
> +++ b/build_and_run.sh
> @@ -50,6 +50,25 @@ fi
>
> if [ "$1" = "up" ]; then
> if [ -v clean_start ]; then
> + # FIXME: We should provide a simpler way of mapping jenkins uid with
> + # hosts uid, that does not require the user (that's extending
> + # fuego-base) to have this logic in his docker build.
> + #
> + # Jenkins' uid/gid should probably be given as a docker-run variable
> + # (as http_proxy/https_proxy, and set properly through fuego-base
> + # entrypoint.)
> + #
> + if [ "${UID}" == "0" ]; then
> + uid=$(id -u "${SUDO_USER}")
> + gid=$(id -g "${SUDO_USER}")
> + else
> + uid="${UID}"
> + gid=$(id -g "${USER}")
> + fi
> +
> + sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
> + sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
> +
> docker build -t "${fuego_rt_image}" .
> docker rm -f "${fuego_rt_container}"
> docker run -dit --name ${fuego_rt_container} \
> diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> index 0c95eae..96cdcc9 100644
> --- a/fuego-ro/conf/fuego.conf
> +++ b/fuego-ro/conf/fuego.conf
> @@ -1,3 +1,3 @@
> -docker_jenkins_uid=1000
> -docker_jenkins_gid=987
> +docker_jenkins_uid=HOST_UID
> +docker_jenkins_gid=HOST_GID
>
> As I mentioned in my FIXME message, its undesirable to require a derivate
> image to contain specific build steps (as the one that's replacing the id into
> fuego.conf),
> so we might need to come up with a better approach to make this work.
>
> The patch has already been pushed to
> <https://bitbucket.org/profusionmobi/fuego-release-test>
>
> https://bitbucket.org/profusionmobi/fuego-release-test (master).
> Can you please re-sync and test again?
The jenkins uid and gid in the container now match those of my host user,
but I'm getting the same start failure as before.
Even when I explicitly change permissions in the path:
/var/cache/jenkins/war/META-INF/MANIFEST.MF, I still
get the same (Permission denied) message shown in the log (as below).
I'm not sure what to try next.
-- Tim
>
> Thank you
>
> --
> Guilherme
>
>
> On Wed, Feb 21, 2018 at 11:48 AM, <Tim.Bird@sony.com
> <mailto:Tim.Bird@sony.com> > wrote:
>
>
> > -----Original Message-----
> > From: Guilherme Camargo
> > Tim, please sync the repo again (
> > <https://bitbucket.org/profusionmobi/fuego-release-test>
>
> https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test> > ).
> > I've force-pushed some changes.
> >
> > After that, you can just run `./build_and_run -c up`.
>
> My container doesn't start Jenkins. I'm not sure if this is
> a cascade error from some initial problem, but here is
> some information about my situation.
>
> I removed my repository, and did a new 'git clone' of the
> fuego-release-test repository. It doesn't appear to me to
> be updated recently:
>
> $ git branch -v
> * master 9d82c80 Initial commit
> $ git log
> commit 9d82c80d66a4de24ab97b38a16c8536849fea5bc
> Author: Guilherme Campos Camargo <guicc@profusion.mobi
> <mailto:guicc@profusion.mobi> >
> Date: Tue Feb 6 02:13:20 2018 -0200
>
> Initial commit
>
> This repository has been created to store the resources needed for
> the
> generation of a Docker image, based on fuego-base, that has all
> the
> tools needed for the automation of the tests that are required for
> validating a new Fuego release with Fuego itself.
> ...
>
> I see that there are other branches with more recent commits (the
> 'ppa', and the 'fuego-base' branch. Should I be using one of these?
>
> Here's the error I saw while trying build_and_run.sh:
>
> -----
> Downloading/unpacking pexpect
> Downloading/unpacking ptyprocess>=0.5 (from pexpect)
> Downloading ptyprocess-0.5.2-py2.py3-none-any.whl
> Installing collected packages: pexpect, ptyprocess
> Successfully installed pexpect ptyprocess
> Cleaning up...
> ---> fd97e25fac02
> Removing intermediate container 5a3a3638cb21
> Step 4/7 : RUN echo "jenkins ALL = (root) NOPASSWD:
> /usr/bin/docker, /bin/sh" >> /etc/sudoers
> ---> Running in da0a1d824354
> ---> fb6468918ab5
> Removing intermediate container da0a1d824354
> Step 5/7 : COPY fuego-ro /fuego-ro
> ---> bbe089e90e5f
> Removing intermediate container 6a4b20b92a4e
> Step 6/7 : COPY tests /fuego-core/engine/tests
> ---> 57c7374dd097
> Removing intermediate container aa6b4c80255c
> Step 7/7 : COPY fuego-rt-entrypoint.sh /
> ---> 9ef063ab4a5b
> Removing intermediate container 32a90a4f3abb
> Successfully built 9ef063ab4a5b
> Successfully tagged fuego-rt:latest
> Error response from daemon: No such container: fuego-rt-container
> 6f513d49f117300672e1ba8782f0411448ffdf8e5258dd689797895cd79d
> 8068
> Waiting for Jenkins on http://localhost:8090/fuego/...
> Creating job fuego-test.default.Functional.fuegotest
> fuego-rt-container
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root@6f513d49f117:/#
>
> -----
>
> And here is some investigation:
> -----
> root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 21, 2018 2:13:22 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> INF/MANIFEST.MF (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at
> winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> ava:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> root@6f513d49f117:/# su - jenkins
> jenkins@6f513d49f117:~$ cat /var/cache/jenkins/war/META-
> INF/MANIFEST.MF
> cat: /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission
> denied
> jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> ls: cannot access /var/cache/jenkins/war: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF: Permission
> denied
> ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF:
> Permission denied
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> jenkins@6f513d49f117:~$ exit
> logout
> root@6f513d49f117:/# ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> root@6f513d49f117:/# su - jenkins
> jenkins@6f513d49f117:~$ id
> uid=1000(jenkins) gid=987(jenkins) groups=987(jenkins)
> jenkins@6f513d49f117:~$
> -----
> For reasons that escape me, it looks like these permissions are not
> allowing jenkins in the container to access the 'war' directory:
> drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
>
> -----
> root@6f513d49f117:/var/cache/jenkins# chmod o+rx .
> root@6f513d49f117:/var/cache/jenkins# ls -la
> total 12
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 .
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 ..
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 war
> root@6f513d49f117:/var/cache/jenkins# su - jenkins
> jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> -----
> However, maybe it's not that simple:
> -----
>
> jenkins@6f513d49f117:~$ exit
> logout
> root@6f513d49f117:/var/cache/jenkins# exit
> exit
> 1000072457@tlinux:~/work/fuego/release-test-2018-02-19/fuego-
> release-test$ ./build_and_run.sh up
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 21, 2018 2:38:41 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> INF/MANIFEST.MF (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at
> winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> ava:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> root@6f513d49f117:/# su - jenkins
> jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> ls: cannot access /var/cache/jenkins/war: Permission denied
> ls: cannot access /var/cache/jenkins/war/META-INF: Permission
> denied
> ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF:
> Permission denied
> drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
>
> I'm running this on Ubuntu 14.04, with docker 17.05.0-ce
>
> -----
> OK - in summary:
>
> - I think I might be running the wrong branch of fuego-release-test
> - I have weird permission problems in my docker container, for the
> fuego user and group
> that are preventing jenkins from starting
> - the jenkins user id and group id don't match my local user's user id
> and group id
>
> Any help would be appreciated.
> -- Tim
>
>
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 1:19 ` Tim.Bird
@ 2018-02-22 1:59 ` Guilherme Camargo
2018-02-22 20:19 ` Tim.Bird
0 siblings, 1 reply; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-22 1:59 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 14962 bytes --]
On Wed, Feb 21, 2018 at 10:19 PM, <Tim.Bird@sony.com> wrote:
>
>
> > -----Original Message-----
> > From: Guilherme Camargo
> >
> > Hello, Tim, thanks for trying it out again and thanks for your time
> > in debugging this issue and providing logs.
> >
> > This problem is happening because we're not using fuego-host-scripts
> > in the derived fuego-release-test, what ends up resulting in the
> > jenkins' user not being properly mapped to the host's user.
> >
> > That's my fault, I forgot that this would be required in the
> > derivate image as well.
> >
> > I temporarily added that logic to build_and_run.sh (as a fixup commit)
> >
> > so that you're able to test it out, please see the patch below:
> >
> > diff --git a/build_and_run.sh b/build_and_run.sh
> > index e448ddf..0134759 100755
> > --- a/build_and_run.sh
> > +++ b/build_and_run.sh
> > @@ -50,6 +50,25 @@ fi
> >
> > if [ "$1" = "up" ]; then
> > if [ -v clean_start ]; then
> > + # FIXME: We should provide a simpler way of mapping jenkins uid
> with
> > + # hosts uid, that does not require the user (that's extending
> > + # fuego-base) to have this logic in his docker build.
> > + #
> > + # Jenkins' uid/gid should probably be given as a docker-run
> variable
> > + # (as http_proxy/https_proxy, and set properly through
> fuego-base
> > + # entrypoint.)
> > + #
> > + if [ "${UID}" == "0" ]; then
> > + uid=$(id -u "${SUDO_USER}")
> > + gid=$(id -g "${SUDO_USER}")
> > + else
> > + uid="${UID}"
> > + gid=$(id -g "${USER}")
> > + fi
> > +
> > + sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
> > + sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
> > +
> > docker build -t "${fuego_rt_image}" .
> > docker rm -f "${fuego_rt_container}"
> > docker run -dit --name ${fuego_rt_container} \
> > diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> > index 0c95eae..96cdcc9 100644
> > --- a/fuego-ro/conf/fuego.conf
> > +++ b/fuego-ro/conf/fuego.conf
> > @@ -1,3 +1,3 @@
> > -docker_jenkins_uid=1000
> > -docker_jenkins_gid=987
> > +docker_jenkins_uid=HOST_UID
> > +docker_jenkins_gid=HOST_GID
> >
> > As I mentioned in my FIXME message, its undesirable to require a derivate
> > image to contain specific build steps (as the one that's replacing the
> id into
> > fuego.conf),
> > so we might need to come up with a better approach to make this work.
> >
> > The patch has already been pushed to
> > <https://bitbucket.org/profusionmobi/fuego-release-test>
> >
> > https://bitbucket.org/profusionmobi/fuego-release-test (master).
> > Can you please re-sync and test again?
>
> The jenkins uid and gid in the container now match those of my host user,
> but I'm getting the same start failure as before.
>
> Even when I explicitly change permissions in the path:
> /var/cache/jenkins/war/META-INF/MANIFEST.MF, I still
> get the same (Permission denied) message shown in the log (as below).
>
> I'm not sure what to try next.
> -- Tim
>
>
Thanks for helping, Tim.
I'm taking a closer look to understand why this is happening.
>
> >
> > Thank you
> >
> > --
> > Guilherme
> >
> >
> > On Wed, Feb 21, 2018 at 11:48 AM, <Tim.Bird@sony.com
> > <mailto:Tim.Bird@sony.com> > wrote:
> >
> >
> > > -----Original Message-----
> > > From: Guilherme Camargo
> > > Tim, please sync the repo again (
> > > <https://bitbucket.org/profusionmobi/fuego-release-test>
> >
> > https://bitbucket.org/profusionmobi/fuego-release-test
> > > <https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> > ).
> > > I've force-pushed some changes.
> > >
> > > After that, you can just run `./build_and_run -c up`.
> >
> > My container doesn't start Jenkins. I'm not sure if this is
> > a cascade error from some initial problem, but here is
> > some information about my situation.
> >
> > I removed my repository, and did a new 'git clone' of the
> > fuego-release-test repository. It doesn't appear to me to
> > be updated recently:
> >
> > $ git branch -v
> > * master 9d82c80 Initial commit
> > $ git log
> > commit 9d82c80d66a4de24ab97b38a16c8536849fea5bc
> > Author: Guilherme Campos Camargo <guicc@profusion.mobi
> > <mailto:guicc@profusion.mobi> >
> > Date: Tue Feb 6 02:13:20 2018 -0200
> >
> > Initial commit
> >
> > This repository has been created to store the resources needed
> for
> > the
> > generation of a Docker image, based on fuego-base, that has all
> > the
> > tools needed for the automation of the tests that are required
> for
> > validating a new Fuego release with Fuego itself.
> > ...
> >
> > I see that there are other branches with more recent commits (the
> > 'ppa', and the 'fuego-base' branch. Should I be using one of
> these?
> >
> > Here's the error I saw while trying build_and_run.sh:
> >
> > -----
> > Downloading/unpacking pexpect
> > Downloading/unpacking ptyprocess>=0.5 (from pexpect)
> > Downloading ptyprocess-0.5.2-py2.py3-none-any.whl
> > Installing collected packages: pexpect, ptyprocess
> > Successfully installed pexpect ptyprocess
> > Cleaning up...
> > ---> fd97e25fac02
> > Removing intermediate container 5a3a3638cb21
> > Step 4/7 : RUN echo "jenkins ALL = (root) NOPASSWD:
> > /usr/bin/docker, /bin/sh" >> /etc/sudoers
> > ---> Running in da0a1d824354
> > ---> fb6468918ab5
> > Removing intermediate container da0a1d824354
> > Step 5/7 : COPY fuego-ro /fuego-ro
> > ---> bbe089e90e5f
> > Removing intermediate container 6a4b20b92a4e
> > Step 6/7 : COPY tests /fuego-core/engine/tests
> > ---> 57c7374dd097
> > Removing intermediate container aa6b4c80255c
> > Step 7/7 : COPY fuego-rt-entrypoint.sh /
> > ---> 9ef063ab4a5b
> > Removing intermediate container 32a90a4f3abb
> > Successfully built 9ef063ab4a5b
> > Successfully tagged fuego-rt:latest
> > Error response from daemon: No such container: fuego-rt-container
> > 6f513d49f117300672e1ba8782f0411448ffdf8e5258dd689797895cd79d
> > 8068
> > Waiting for Jenkins on http://localhost:8090/fuego/...
> > Creating job fuego-test.default.Functional.fuegotest
> > fuego-rt-container
> > [FAIL] Starting Jenkins Continuous Integration Server: jenkins
> failed!
> > [ ok ] Starting network benchmark server.
> > root@6f513d49f117:/#
> >
> > -----
> >
> > And here is some investigation:
> > -----
> > root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> > INFO: Winstone shutdown successfully
> > Feb 21, 2018 2:13:22 PM winstone.Logger logInternal
> > SEVERE: Container startup failed
> > java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF (Permission denied)
> > at java.io.FileOutputStream.open(Native Method)
> > at java.io.FileOutputStream.<init>(FileOutputStream.java:
> 221)
> > at java.io.FileOutputStream.<init>(FileOutputStream.java:
> 171)
> > at
> > winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> > at winstone.HostConfiguration.<
> init>(HostConfiguration.java:83)
> > at winstone.HostGroup.initHost(HostGroup.java:66)
> > at winstone.HostGroup.<init>(HostGroup.java:45)
> > at winstone.Launcher.<init>(Launcher.java:143)
> > at winstone.Launcher.main(Launcher.java:352)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> > ava:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> > sorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at Main._main(Main.java:264)
> > at Main.main(Main.java:112)
> >
> > root@6f513d49f117:/# su - jenkins
> > jenkins@6f513d49f117:~$ cat /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF
> > cat: /var/cache/jenkins/war/META-INF/MANIFEST.MF: Permission
> > denied
> > jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > ls: cannot access /var/cache/jenkins/war: Permission denied
> > ls: cannot access /var/cache/jenkins/war/META-INF: Permission
> > denied
> > ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF:
> > Permission denied
> > drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> > drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> > drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> > jenkins@6f513d49f117:~$ exit
> > logout
> > root@6f513d49f117:/# ls -lad /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> > drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> > drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins
> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12
> > /var/cache/jenkins/war
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> > /var/cache/jenkins/war/META-INF
> > -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > root@6f513d49f117:/# su - jenkins
> > jenkins@6f513d49f117:~$ id
> > uid=1000(jenkins) gid=987(jenkins) groups=987(jenkins)
> > jenkins@6f513d49f117:~$
> > -----
> > For reasons that escape me, it looks like these permissions are not
> > allowing jenkins in the container to access the 'war' directory:
> > drwxr-x--- 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins
> >
> > -----
> > root@6f513d49f117:/var/cache/jenkins# chmod o+rx .
> > root@6f513d49f117:/var/cache/jenkins# ls -la
> > total 12
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 .
> > drwxr-xr-x 17 root root 4096 Feb 21 07:12 ..
> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12 war
> > root@6f513d49f117:/var/cache/jenkins# su - jenkins
> > jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> > drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> /var/cache/jenkins
> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 21 07:12
> > /var/cache/jenkins/war
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12
> > /var/cache/jenkins/war/META-INF
> > -rw-r--r-- 1 jenkins jenkins 129535 Feb 15 18:56
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > -----
> > However, maybe it's not that simple:
> > -----
> >
> > jenkins@6f513d49f117:~$ exit
> > logout
> > root@6f513d49f117:/var/cache/jenkins# exit
> > exit
> > 1000072457@tlinux:~/work/fuego/release-test-2018-02-19/fuego-
> > release-test$ ./build_and_run.sh up
> > [FAIL] Starting Jenkins Continuous Integration Server: jenkins
> failed!
> > [ ok ] Starting network benchmark server.
> > root@6f513d49f117:/# tail -n 20 /var/log/jenkins/jenkins.log
> > INFO: Winstone shutdown successfully
> > Feb 21, 2018 2:38:41 PM winstone.Logger logInternal
> > SEVERE: Container startup failed
> > java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF (Permission denied)
> > at java.io.FileOutputStream.open(Native Method)
> > at java.io.FileOutputStream.<init>(FileOutputStream.java:
> 221)
> > at java.io.FileOutputStream.<init>(FileOutputStream.java:
> 171)
> > at
> > winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> > at winstone.HostConfiguration.<
> init>(HostConfiguration.java:83)
> > at winstone.HostGroup.initHost(HostGroup.java:66)
> > at winstone.HostGroup.<init>(HostGroup.java:45)
> > at winstone.Launcher.<init>(Launcher.java:143)
> > at winstone.Launcher.main(Launcher.java:352)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> > Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> > ava:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> > sorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at Main._main(Main.java:264)
> > at Main.main(Main.java:112)
> >
> > root@6f513d49f117:/# su - jenkins
> > jenkins@6f513d49f117:~$ ls -lad /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > ls: cannot access /var/cache/jenkins/war: Permission denied
> > ls: cannot access /var/cache/jenkins/war/META-INF: Permission
> > denied
> > ls: cannot access /var/cache/jenkins/war/META-INF/MANIFEST.MF:
> > Permission denied
> > drwxr-xr-x 32 root root 4096 Feb 21 07:12 /var
> > drwxr-xr-x 17 root root 4096 Feb 21 07:12 /var/cache
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 21 07:12 /var/cache/jenkins
> >
> > I'm running this on Ubuntu 14.04, with docker 17.05.0-ce
> >
> > -----
> > OK - in summary:
> >
> > - I think I might be running the wrong branch of fuego-release-test
> > - I have weird permission problems in my docker container, for the
> > fuego user and group
> > that are preventing jenkins from starting
> > - the jenkins user id and group id don't match my local user's
> user id
> > and group id
> >
> > Any help would be appreciated.
> > -- Tim
> >
> >
> >
> >
>
>
[-- Attachment #2: Type: text/html, Size: 20365 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 1:59 ` Guilherme Camargo
@ 2018-02-22 20:19 ` Tim.Bird
2018-02-22 21:02 ` Guilherme Camargo
0 siblings, 1 reply; 15+ messages in thread
From: Tim.Bird @ 2018-02-22 20:19 UTC (permalink / raw)
To: guicc; +Cc: fuego
> -----Original Message-----
> From: Guilherme Camargo
> On Wed, Feb 21, 2018 at 10:19 PM, <Tim.Bird@sony.com
> <mailto:Tim.Bird@sony.com> > wrote:
>
>
>
>
> > -----Original Message-----
> > From: Guilherme Camargo
> >
>
> > Hello, Tim, thanks for trying it out again and thanks for your time
> > in debugging this issue and providing logs.
> >
> > This problem is happening because we're not using fuego-host-
> scripts
> > in the derived fuego-release-test, what ends up resulting in the
> > jenkins' user not being properly mapped to the host's user.
> >
> > That's my fault, I forgot that this would be required in the
> > derivate image as well.
> >
> > I temporarily added that logic to build_and_run.sh (as a fixup
> commit)
> >
> > so that you're able to test it out, please see the patch below:
> >
> > diff --git a/build_and_run.sh b/build_and_run.sh
> > index e448ddf..0134759 100755
> > --- a/build_and_run.sh
> > +++ b/build_and_run.sh
> > @@ -50,6 +50,25 @@ fi
> >
> > if [ "$1" = "up" ]; then
> > if [ -v clean_start ]; then
> > + # FIXME: We should provide a simpler way of mapping jenkins
> uid with
> > + # hosts uid, that does not require the user (that's extending
> > + # fuego-base) to have this logic in his docker build.
> > + #
> > + # Jenkins' uid/gid should probably be given as a docker-run
> variable
> > + # (as http_proxy/https_proxy, and set properly through
> fuego-base
> > + # entrypoint.)
> > + #
> > + if [ "${UID}" == "0" ]; then
> > + uid=$(id -u "${SUDO_USER}")
> > + gid=$(id -g "${SUDO_USER}")
> > + else
> > + uid="${UID}"
> > + gid=$(id -g "${USER}")
> > + fi
> > +
> > + sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
> > + sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
> > +
> > docker build -t "${fuego_rt_image}" .
> > docker rm -f "${fuego_rt_container}"
> > docker run -dit --name ${fuego_rt_container} \
> > diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> > index 0c95eae..96cdcc9 100644
> > --- a/fuego-ro/conf/fuego.conf
> > +++ b/fuego-ro/conf/fuego.conf
> > @@ -1,3 +1,3 @@
> > -docker_jenkins_uid=1000
> > -docker_jenkins_gid=987
> > +docker_jenkins_uid=HOST_UID
> > +docker_jenkins_gid=HOST_GID
> >
> > As I mentioned in my FIXME message, its undesirable to require a
> derivate
> > image to contain specific build steps (as the one that's replacing the
> id into
> > fuego.conf),
> > so we might need to come up with a better approach to make this
> work.
> >
> > The patch has already been pushed to
> > <https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test> >
> >
> > https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test> (master).
> > Can you please re-sync and test again?
>
>
> The jenkins uid and gid in the container now match those of my host
> user,
> but I'm getting the same start failure as before.
>
> Even when I explicitly change permissions in the path:
> /var/cache/jenkins/war/META-INF/MANIFEST.MF, I still
> get the same (Permission denied) message shown in the log (as
> below).
>
> I'm not sure what to try next.
> -- Tim
>
>
>
>
> Thanks for helping, Tim.
>
> I'm taking a closer look to understand why this is happening.
I'm not sure what is the root cause of this, and fear it might be a docker bug.
I have some notes I made when I was first integrating ProFusion's
changes to Fuego to start from a base docker container (on Feb 1).
In my notes I describe the same permission problem with
/var/cache/jenkins/war/META-INF/MANIFEST.MF.
In my notes it says:
"changed permissions, and it worked"
then later
"changed permissions back, and it still works (docker fs bug??)"
But I may have done something else, unrelated to the permissions,
which I didn't take note of at the time.
However, I'm pretty sure the current issue has the same root cause
as what I was seeing before.
Hopefully that's helpful information.
-- Tim
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 20:19 ` Tim.Bird
@ 2018-02-22 21:02 ` Guilherme Camargo
2018-02-22 21:05 ` Tim.Bird
2018-02-22 21:27 ` Tim.Bird
0 siblings, 2 replies; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-22 21:02 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 5807 bytes --]
I made a few more modifications in the way that the uid/gid are set on Fuego
(
https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4efa55aaa345a319b1b8fd6
)
and also made the needed changes on fuego-release-test to match those.
Hopefully they're more robust now. I plan to send all the new changes by
e-mail
to the list, but I just wanted to check if you're able to run it locally, so
that you can get a feeling about this, before we move on.
Can you please try again with:
```
git clone https://bitbucket.org/profusionmobi/fuego-release-test
cd fuego-release-test
./build_and_run -c up
Then try to access fuego in localhost:8080/fuego/
```
And check if things go well this time?
Can you please also provide info about the distribution you're using along
with
the docker version? I'll try to find out if there's any similar issue
reported
by other users.
Thanks for your help, Tim.
On Thu, Feb 22, 2018 at 5:19 PM, <Tim.Bird@sony.com> wrote:
>
>
> > -----Original Message-----
> > From: Guilherme Camargo
> > On Wed, Feb 21, 2018 at 10:19 PM, <Tim.Bird@sony.com
> > <mailto:Tim.Bird@sony.com> > wrote:
> >
> >
> >
> >
> > > -----Original Message-----
> > > From: Guilherme Camargo
> > >
> >
> > > Hello, Tim, thanks for trying it out again and thanks for your
> time
> > > in debugging this issue and providing logs.
> > >
> > > This problem is happening because we're not using fuego-host-
> > scripts
> > > in the derived fuego-release-test, what ends up resulting in the
> > > jenkins' user not being properly mapped to the host's user.
> > >
> > > That's my fault, I forgot that this would be required in the
> > > derivate image as well.
> > >
> > > I temporarily added that logic to build_and_run.sh (as a fixup
> > commit)
> > >
> > > so that you're able to test it out, please see the patch below:
> > >
> > > diff --git a/build_and_run.sh b/build_and_run.sh
> > > index e448ddf..0134759 100755
> > > --- a/build_and_run.sh
> > > +++ b/build_and_run.sh
> > > @@ -50,6 +50,25 @@ fi
> > >
> > > if [ "$1" = "up" ]; then
> > > if [ -v clean_start ]; then
> > > + # FIXME: We should provide a simpler way of mapping
> jenkins
> > uid with
> > > + # hosts uid, that does not require the user (that's
> extending
> > > + # fuego-base) to have this logic in his docker build.
> > > + #
> > > + # Jenkins' uid/gid should probably be given as a
> docker-run
> > variable
> > > + # (as http_proxy/https_proxy, and set properly through
> > fuego-base
> > > + # entrypoint.)
> > > + #
> > > + if [ "${UID}" == "0" ]; then
> > > + uid=$(id -u "${SUDO_USER}")
> > > + gid=$(id -g "${SUDO_USER}")
> > > + else
> > > + uid="${UID}"
> > > + gid=$(id -g "${USER}")
> > > + fi
> > > +
> > > + sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
> > > + sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
> > > +
> > > docker build -t "${fuego_rt_image}" .
> > > docker rm -f "${fuego_rt_container}"
> > > docker run -dit --name ${fuego_rt_container} \
> > > diff --git a/fuego-ro/conf/fuego.conf b/fuego-ro/conf/fuego.conf
> > > index 0c95eae..96cdcc9 100644
> > > --- a/fuego-ro/conf/fuego.conf
> > > +++ b/fuego-ro/conf/fuego.conf
> > > @@ -1,3 +1,3 @@
> > > -docker_jenkins_uid=1000
> > > -docker_jenkins_gid=987
> > > +docker_jenkins_uid=HOST_UID
> > > +docker_jenkins_gid=HOST_GID
> > >
> > > As I mentioned in my FIXME message, its undesirable to require a
> > derivate
> > > image to contain specific build steps (as the one that's
> replacing the
> > id into
> > > fuego.conf),
> > > so we might need to come up with a better approach to make this
> > work.
> > >
> > > The patch has already been pushed to
> > > <https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> >
> > >
> > > https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> (master).
> > > Can you please re-sync and test again?
> >
> >
> > The jenkins uid and gid in the container now match those of my host
> > user,
> > but I'm getting the same start failure as before.
> >
> > Even when I explicitly change permissions in the path:
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF, I still
> > get the same (Permission denied) message shown in the log (as
> > below).
> >
> > I'm not sure what to try next.
> > -- Tim
> >
> >
> >
> >
> > Thanks for helping, Tim.
> >
> > I'm taking a closer look to understand why this is happening.
>
> I'm not sure what is the root cause of this, and fear it might be a docker
> bug.
> I have some notes I made when I was first integrating ProFusion's
> changes to Fuego to start from a base docker container (on Feb 1).
>
> In my notes I describe the same permission problem with
> /var/cache/jenkins/war/META-INF/MANIFEST.MF.
>
> In my notes it says:
> "changed permissions, and it worked"
> then later
> "changed permissions back, and it still works (docker fs bug??)"
>
> But I may have done something else, unrelated to the permissions,
> which I didn't take note of at the time.
> However, I'm pretty sure the current issue has the same root cause
> as what I was seeing before.
>
> Hopefully that's helpful information.
> -- Tim
>
[-- Attachment #2: Type: text/html, Size: 10489 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 21:02 ` Guilherme Camargo
@ 2018-02-22 21:05 ` Tim.Bird
2018-02-22 21:06 ` Guilherme Camargo
2018-02-22 21:27 ` Tim.Bird
1 sibling, 1 reply; 15+ messages in thread
From: Tim.Bird @ 2018-02-22 21:05 UTC (permalink / raw)
To: guicc; +Cc: fuego
Will do right now.
-- Tim
> -----Original Message-----
> From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> bounces@lists.linuxfoundation.org] On Behalf Of Guilherme Camargo
> Sent: Thursday, February 22, 2018 1:03 PM
> To: Bird, Timothy <Tim.Bird@sony.com>
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] release test issues
>
> I made a few more modifications in the way that the uid/gid are set on Fuego
> (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> fa55aaa345a319b1b8fd6)
> and also made the needed changes on fuego-release-test to match those.
> Hopefully they're more robust now. I plan to send all the new changes by e-
> mail
> to the list, but I just wanted to check if you're able to run it locally, so
> that you can get a feeling about this, before we move on.
>
>
> Can you please try again with:
>
>
> ```
> git clone https://bitbucket.org/profusionmobi/fuego-release-test
> cd fuego-release-test
> ./build_and_run -c up
>
>
> Then try to access fuego in localhost:8080/fuego/
>
>
> ```
>
>
> And check if things go well this time?
>
>
> Can you please also provide info about the distribution you're using along
> with
> the docker version? I'll try to find out if there's any similar issue reported
> by other users.
>
>
> Thanks for your help, Tim.
>
>
>
> On Thu, Feb 22, 2018 at 5:19 PM, <Tim.Bird@sony.com
> <mailto:Tim.Bird@sony.com> > wrote:
>
>
>
>
> > -----Original Message-----
> > From: Guilherme Camargo
> > On Wed, Feb 21, 2018 at 10:19 PM, <Tim.Bird@sony.com
> <mailto:Tim.Bird@sony.com>
> > <mailto:Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > >
> wrote:
> >
> >
> >
> >
> > > -----Original Message-----
> > > From: Guilherme Camargo
> > >
> >
>
> > > Hello, Tim, thanks for trying it out again and thanks for your
> time
> > > in debugging this issue and providing logs.
> > >
> > > This problem is happening because we're not using fuego-
> host-
> > scripts
> > > in the derived fuego-release-test, what ends up resulting in
> the
> > > jenkins' user not being properly mapped to the host's user.
> > >
> > > That's my fault, I forgot that this would be required in the
> > > derivate image as well.
> > >
> > > I temporarily added that logic to build_and_run.sh (as a fixup
> > commit)
> > >
> > > so that you're able to test it out, please see the patch below:
> > >
> > > diff --git a/build_and_run.sh b/build_and_run.sh
> > > index e448ddf..0134759 100755
> > > --- a/build_and_run.sh
> > > +++ b/build_and_run.sh
> > > @@ -50,6 +50,25 @@ fi
> > >
> > > if [ "$1" = "up" ]; then
> > > if [ -v clean_start ]; then
> > > + # FIXME: We should provide a simpler way of mapping
> jenkins
> > uid with
> > > + # hosts uid, that does not require the user (that's
> extending
> > > + # fuego-base) to have this logic in his docker build.
> > > + #
> > > + # Jenkins' uid/gid should probably be given as a docker-
> run
> > variable
> > > + # (as http_proxy/https_proxy, and set properly through
> > fuego-base
> > > + # entrypoint.)
> > > + #
> > > + if [ "${UID}" == "0" ]; then
> > > + uid=$(id -u "${SUDO_USER}")
> > > + gid=$(id -g "${SUDO_USER}")
> > > + else
> > > + uid="${UID}"
> > > + gid=$(id -g "${USER}")
> > > + fi
> > > +
> > > + sed -i "s/HOST_UID/${uid}/" ./fuego-ro/conf/fuego.conf
> > > + sed -i "s/HOST_GID/${gid}/" ./fuego-ro/conf/fuego.conf
> > > +
> > > docker build -t "${fuego_rt_image}" .
> > > docker rm -f "${fuego_rt_container}"
> > > docker run -dit --name ${fuego_rt_container} \
> > > diff --git a/fuego-ro/conf/fuego.conf b/fuego-
> ro/conf/fuego.conf
> > > index 0c95eae..96cdcc9 100644
> > > --- a/fuego-ro/conf/fuego.conf
> > > +++ b/fuego-ro/conf/fuego.conf
> > > @@ -1,3 +1,3 @@
> > > -docker_jenkins_uid=1000
> > > -docker_jenkins_gid=987
> > > +docker_jenkins_uid=HOST_UID
> > > +docker_jenkins_gid=HOST_GID
> > >
> > > As I mentioned in my FIXME message, its undesirable to
> require a
> > derivate
> > > image to contain specific build steps (as the one that's
> replacing the
> > id into
> > > fuego.conf),
> > > so we might need to come up with a better approach to make
> this
> > work.
> > >
> > > The patch has already been pushed to
> > > <https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test>
> > <https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test> > >
> > >
> > > https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test>
> > <https://bitbucket.org/profusionmobi/fuego-release-test
> <https://bitbucket.org/profusionmobi/fuego-release-test> > (master).
> > > Can you please re-sync and test again?
> >
> >
> > The jenkins uid and gid in the container now match those of my
> host
> > user,
> > but I'm getting the same start failure as before.
> >
> > Even when I explicitly change permissions in the path:
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF, I still
> > get the same (Permission denied) message shown in the log (as
> > below).
> >
> > I'm not sure what to try next.
> > -- Tim
> >
> >
> >
> >
> > Thanks for helping, Tim.
> >
> > I'm taking a closer look to understand why this is happening.
>
>
> I'm not sure what is the root cause of this, and fear it might be a
> docker bug.
> I have some notes I made when I was first integrating ProFusion's
> changes to Fuego to start from a base docker container (on Feb 1).
>
> In my notes I describe the same permission problem with
> /var/cache/jenkins/war/META-INF/MANIFEST.MF.
>
> In my notes it says:
> "changed permissions, and it worked"
> then later
> "changed permissions back, and it still works (docker fs bug??)"
>
> But I may have done something else, unrelated to the permissions,
> which I didn't take note of at the time.
> However, I'm pretty sure the current issue has the same root cause
> as what I was seeing before.
>
> Hopefully that's helpful information.
> -- Tim
>
>
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 21:05 ` Tim.Bird
@ 2018-02-22 21:06 ` Guilherme Camargo
0 siblings, 0 replies; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-22 21:06 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 8075 bytes --]
Great, thanks
On Thu, Feb 22, 2018 at 6:05 PM, <Tim.Bird@sony.com> wrote:
> Will do right now.
> -- Tim
>
>
> > -----Original Message-----
> > From: fuego-bounces@lists.linuxfoundation.org [mailto:fuego-
> > bounces@lists.linuxfoundation.org] On Behalf Of Guilherme Camargo
> > Sent: Thursday, February 22, 2018 1:03 PM
> > To: Bird, Timothy <Tim.Bird@sony.com>
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] release test issues
> >
> > I made a few more modifications in the way that the uid/gid are set on
> Fuego
> > (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> > fa55aaa345a319b1b8fd6)
> > and also made the needed changes on fuego-release-test to match those.
> > Hopefully they're more robust now. I plan to send all the new changes by
> e-
> > mail
> > to the list, but I just wanted to check if you're able to run it
> locally, so
> > that you can get a feeling about this, before we move on.
> >
> >
> > Can you please try again with:
> >
> >
> > ```
> > git clone https://bitbucket.org/profusionmobi/fuego-release-test
> > cd fuego-release-test
> > ./build_and_run -c up
> >
> >
> > Then try to access fuego in localhost:8080/fuego/
> >
> >
> > ```
> >
> >
> > And check if things go well this time?
> >
> >
> > Can you please also provide info about the distribution you're using
> along
> > with
> > the docker version? I'll try to find out if there's any similar issue
> reported
> > by other users.
> >
> >
> > Thanks for your help, Tim.
> >
> >
> >
> > On Thu, Feb 22, 2018 at 5:19 PM, <Tim.Bird@sony.com
> > <mailto:Tim.Bird@sony.com> > wrote:
> >
> >
> >
> >
> > > -----Original Message-----
> > > From: Guilherme Camargo
> > > On Wed, Feb 21, 2018 at 10:19 PM, <Tim.Bird@sony.com
> > <mailto:Tim.Bird@sony.com>
> > > <mailto:Tim.Bird@sony.com <mailto:Tim.Bird@sony.com> > >
> > wrote:
> > >
> > >
> > >
> > >
> > > > -----Original Message-----
> > > > From: Guilherme Camargo
> > > >
> > >
> >
> > > > Hello, Tim, thanks for trying it out again and thanks
> for your
> > time
> > > > in debugging this issue and providing logs.
> > > >
> > > > This problem is happening because we're not using fuego-
> > host-
> > > scripts
> > > > in the derived fuego-release-test, what ends up
> resulting in
> > the
> > > > jenkins' user not being properly mapped to the host's
> user.
> > > >
> > > > That's my fault, I forgot that this would be required in
> the
> > > > derivate image as well.
> > > >
> > > > I temporarily added that logic to build_and_run.sh (as a
> fixup
> > > commit)
> > > >
> > > > so that you're able to test it out, please see the patch
> below:
> > > >
> > > > diff --git a/build_and_run.sh b/build_and_run.sh
> > > > index e448ddf..0134759 100755
> > > > --- a/build_and_run.sh
> > > > +++ b/build_and_run.sh
> > > > @@ -50,6 +50,25 @@ fi
> > > >
> > > > if [ "$1" = "up" ]; then
> > > > if [ -v clean_start ]; then
> > > > + # FIXME: We should provide a simpler way of
> mapping
> > jenkins
> > > uid with
> > > > + # hosts uid, that does not require the user
> (that's
> > extending
> > > > + # fuego-base) to have this logic in his docker
> build.
> > > > + #
> > > > + # Jenkins' uid/gid should probably be given as
> a docker-
> > run
> > > variable
> > > > + # (as http_proxy/https_proxy, and set properly
> through
> > > fuego-base
> > > > + # entrypoint.)
> > > > + #
> > > > + if [ "${UID}" == "0" ]; then
> > > > + uid=$(id -u "${SUDO_USER}")
> > > > + gid=$(id -g "${SUDO_USER}")
> > > > + else
> > > > + uid="${UID}"
> > > > + gid=$(id -g "${USER}")
> > > > + fi
> > > > +
> > > > + sed -i "s/HOST_UID/${uid}/"
> ./fuego-ro/conf/fuego.conf
> > > > + sed -i "s/HOST_GID/${gid}/"
> ./fuego-ro/conf/fuego.conf
> > > > +
> > > > docker build -t "${fuego_rt_image}" .
> > > > docker rm -f "${fuego_rt_container}"
> > > > docker run -dit --name ${fuego_rt_container} \
> > > > diff --git a/fuego-ro/conf/fuego.conf b/fuego-
> > ro/conf/fuego.conf
> > > > index 0c95eae..96cdcc9 100644
> > > > --- a/fuego-ro/conf/fuego.conf
> > > > +++ b/fuego-ro/conf/fuego.conf
> > > > @@ -1,3 +1,3 @@
> > > > -docker_jenkins_uid=1000
> > > > -docker_jenkins_gid=987
> > > > +docker_jenkins_uid=HOST_UID
> > > > +docker_jenkins_gid=HOST_GID
> > > >
> > > > As I mentioned in my FIXME message, its undesirable to
> > require a
> > > derivate
> > > > image to contain specific build steps (as the one that's
> > replacing the
> > > id into
> > > > fuego.conf),
> > > > so we might need to come up with a better approach to
> make
> > this
> > > work.
> > > >
> > > > The patch has already been pushed to
> > > > <https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test>
> > > <https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> > >
> > > >
> > > > https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test>
> > > <https://bitbucket.org/profusionmobi/fuego-release-test
> > <https://bitbucket.org/profusionmobi/fuego-release-test> > (master).
> > > > Can you please re-sync and test again?
> > >
> > >
> > > The jenkins uid and gid in the container now match those
> of my
> > host
> > > user,
> > > but I'm getting the same start failure as before.
> > >
> > > Even when I explicitly change permissions in the path:
> > > /var/cache/jenkins/war/META-INF/MANIFEST.MF, I still
> > > get the same (Permission denied) message shown in the log
> (as
> > > below).
> > >
> > > I'm not sure what to try next.
> > > -- Tim
> > >
> > >
> > >
> > >
> > > Thanks for helping, Tim.
> > >
> > > I'm taking a closer look to understand why this is happening.
> >
> >
> > I'm not sure what is the root cause of this, and fear it might be a
> > docker bug.
> > I have some notes I made when I was first integrating ProFusion's
> > changes to Fuego to start from a base docker container (on Feb 1).
> >
> > In my notes I describe the same permission problem with
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF.
> >
> > In my notes it says:
> > "changed permissions, and it worked"
> > then later
> > "changed permissions back, and it still works (docker fs bug??)"
> >
> > But I may have done something else, unrelated to the permissions,
> > which I didn't take note of at the time.
> > However, I'm pretty sure the current issue has the same root cause
> > as what I was seeing before.
> >
> > Hopefully that's helpful information.
> > -- Tim
> >
> >
>
>
[-- Attachment #2: Type: text/html, Size: 12920 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 21:02 ` Guilherme Camargo
2018-02-22 21:05 ` Tim.Bird
@ 2018-02-22 21:27 ` Tim.Bird
2018-02-22 21:29 ` Guilherme Camargo
2018-02-22 21:31 ` Tim.Bird
1 sibling, 2 replies; 15+ messages in thread
From: Tim.Bird @ 2018-02-22 21:27 UTC (permalink / raw)
To: guicc; +Cc: fuego
> -----Original Message-----
> From: Guilherme Camargo
>
> I made a few more modifications in the way that the uid/gid are set on Fuego
> (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> fa55aaa345a319b1b8fd6)
> and also made the needed changes on fuego-release-test to match those.
> Hopefully they're more robust now. I plan to send all the new changes by e-
> mail
> to the list, but I just wanted to check if you're able to run it locally, so
> that you can get a feeling about this, before we move on.
>
>
> Can you please try again with:
>
>
> ```
> git clone https://bitbucket.org/profusionmobi/fuego-release-test
> cd fuego-release-test
> ./build_and_run -c up
Still no joy.
Step 7/7 : COPY fuego-rt-entrypoint.sh /
---> 4e39b71fb0d5
Removing intermediate container a5e9b9e4244d
Successfully built 4e39b71fb0d5
Successfully tagged fuego-rt:latest
fuego-rt-container
a8750b71dc87bc1ad61ed26954c9484e79376d019fbd563abf86e1fcec86c70b
Waiting for Jenkins...
Creating job fuego-test.default.Functional.fuegotest
fuego-rt-container
[FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
[ ok ] Starting network benchmark server.
root@a8750b71dc87:/# tail -n 20 /var/log/jenkins/jenkins.log
INFO: Winstone shutdown successfully
Feb 22, 2018 9:15:14 PM winstone.Logger logInternal
SEVERE: Container startup failed
java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
at winstone.HostGroup.initHost(HostGroup.java:66)
at winstone.HostGroup.<init>(HostGroup.java:45)
at winstone.Launcher.<init>(Launcher.java:143)
at winstone.Launcher.main(Launcher.java:352)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at Main._main(Main.java:264)
at Main.main(Main.java:112)
----
But check out this sequence of actions:
-----
root@a8750b71dc87:/# id jenkins
uid=2060932290(jenkins) gid=2060452353(jenkins) groups=2060452353(jenkins)
...
root@a8750b71dc87:/# ls -ld /var /var/cache /var/cache/jenkins /var/cache/jenkins/war /var/cache/jenkins/war/META-INF /var/cache/jenkins/war/META-INF/MANIFEST.MF
drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war/META-INF
-rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17 /var/cache/jenkins/war/META-INF/MANIFEST.MF
root@a8750b71dc87:/# su - jenkins
jenkins@a8750b71dc87:~$ ls -ld /var /var/cache /var/cache/jenkins /var/cache/jenkins/war /var/cache/jenkins/war/META-INF /var/cache/jenkins/war/META-INF/MANIFEST.MF
drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war/META-INF
-rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17 /var/cache/jenkins/war/META-INF/MANIFEST.MF
jenkins@a8750b71dc87:~$ cat /var/cache/jenkins/war/META-INF/MANIFEST.MF
Manifest-Version: 1.0
Jenkins-Version: 2.32.1
Implementation-Version: 2.32.1
Hudson-Version: 1.395
Build-Jdk: 1.7.0_21
Built-By: kohsuke
Created-By: Apache Maven 3.3.9
Main-Class: Main
Archiver-Version: Plexus Archiver
Name: images/48x48/red_anime.gif
SHA-256-Digest: 2ktTeqkcRPD3cWv0z9xPu6Hf3kMNxoh55qzYKVMdCwk=
... [lots of lines omitted]
Name: scripts/yui/assets/skins/sam/button.css
SHA-256-Digest: Yp390BkBCcwkhhCQ4Fqj06dTN5qg/zI1QbaR3Gkl6LY=
jenkins@a8750b71dc87:~$ exit
logout
root@a8750b71dc87:/# service jenkins start
[ ok ] Starting Jenkins Continuous Integration Server: jenkins.
root@a8750b71dc87:/#
-----
If the jenkins log message is correct, there's something really
weird going on here. The permissions look OK, and the 'jenkins'
user can access the MANIFEST.MF file, and a subsequent
attempt to start jenkins (as root) works. So I'm not sure what's going on.
-- Tim
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 21:27 ` Tim.Bird
@ 2018-02-22 21:29 ` Guilherme Camargo
2018-02-22 21:31 ` Tim.Bird
1 sibling, 0 replies; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-22 21:29 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 5390 bytes --]
I'm not sure either. It's very strange. Maybe something related with
selinux?
Can you please tell me what linux distribution you're using and the docker
version?
--
Guilherme
On Thu, Feb 22, 2018 at 6:27 PM, <Tim.Bird@sony.com> wrote:
>
>
> > -----Original Message-----
> > From: Guilherme Camargo
> >
> > I made a few more modifications in the way that the uid/gid are set on
> Fuego
> > (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> > fa55aaa345a319b1b8fd6)
> > and also made the needed changes on fuego-release-test to match those.
> > Hopefully they're more robust now. I plan to send all the new changes by
> e-
> > mail
> > to the list, but I just wanted to check if you're able to run it
> locally, so
> > that you can get a feeling about this, before we move on.
> >
> >
> > Can you please try again with:
> >
> >
> > ```
> > git clone https://bitbucket.org/profusionmobi/fuego-release-test
> > cd fuego-release-test
> > ./build_and_run -c up
>
> Still no joy.
>
> Step 7/7 : COPY fuego-rt-entrypoint.sh /
> ---> 4e39b71fb0d5
> Removing intermediate container a5e9b9e4244d
> Successfully built 4e39b71fb0d5
> Successfully tagged fuego-rt:latest
> fuego-rt-container
> a8750b71dc87bc1ad61ed26954c9484e79376d019fbd563abf86e1fcec86c70b
> Waiting for Jenkins...
> Creating job fuego-test.default.Functional.fuegotest
> fuego-rt-container
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root@a8750b71dc87:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 22, 2018 9:15:14 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-INF/MANIFEST.MF
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at winstone.HostConfiguration.getWebRoot(HostConfiguration.
> java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> ----
> But check out this sequence of actions:
>
> -----
> root@a8750b71dc87:/# id jenkins
> uid=2060932290(jenkins) gid=2060452353(jenkins) groups=2060452353(jenkins)
> ...
> root@a8750b71dc87:/# ls -ld /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
> drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> root@a8750b71dc87:/# su - jenkins
> jenkins@a8750b71dc87:~$ ls -ld /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
> drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> jenkins@a8750b71dc87:~$ cat /var/cache/jenkins/war/META-INF/MANIFEST.MF
> Manifest-Version: 1.0
> Jenkins-Version: 2.32.1
> Implementation-Version: 2.32.1
> Hudson-Version: 1.395
> Build-Jdk: 1.7.0_21
> Built-By: kohsuke
> Created-By: Apache Maven 3.3.9
> Main-Class: Main
> Archiver-Version: Plexus Archiver
>
> Name: images/48x48/red_anime.gif
> SHA-256-Digest: 2ktTeqkcRPD3cWv0z9xPu6Hf3kMNxoh55qzYKVMdCwk=
> ... [lots of lines omitted]
>
> Name: scripts/yui/assets/skins/sam/button.css
> SHA-256-Digest: Yp390BkBCcwkhhCQ4Fqj06dTN5qg/zI1QbaR3Gkl6LY=
>
> jenkins@a8750b71dc87:~$ exit
> logout
> root@a8750b71dc87:/# service jenkins start
> [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> root@a8750b71dc87:/#
> -----
> If the jenkins log message is correct, there's something really
> weird going on here. The permissions look OK, and the 'jenkins'
> user can access the MANIFEST.MF file, and a subsequent
> attempt to start jenkins (as root) works. So I'm not sure what's going on.
> -- Tim
>
>
[-- Attachment #2: Type: text/html, Size: 7098 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 21:27 ` Tim.Bird
2018-02-22 21:29 ` Guilherme Camargo
@ 2018-02-22 21:31 ` Tim.Bird
2018-02-23 16:23 ` Guilherme Camargo
1 sibling, 1 reply; 15+ messages in thread
From: Tim.Bird @ 2018-02-22 21:31 UTC (permalink / raw)
To: Tim.Bird, guicc; +Cc: fuego
But, just because Jenkins started, doesn't mean everything is working.
I got a Jenkins "Oops" when trying to build the
fuego-test.default.Functional.fuegotest job.
Here's the first part of the console log:
-----
A problem occurred while processing the request. Please check our bug tracker to see if a similar problem has already been reported. If it is already reported, please vote and put a comment on it to let us gauge the impact of the problem. If you think this is a new issue, please file a new issue. When you file an issue, make sure to add the entire stack trace, along with the version of Jenkins and relevant plugins. The users list might be also useful in understanding what has happened.
Stack trace
javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-2.32.1.jar!/hudson/model/Run/console.jelly:65:27: <j:whitespace> Failed to load the key: hudson.console.AnnotatedLargeText.consoleAnnotator
at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:103)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
at
> -----Original Message-----
> From: Tim Bird
> > -----Original Message-----
> > From: Guilherme Camargo
> >
> > I made a few more modifications in the way that the uid/gid are set on
> Fuego
> >
> (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> > fa55aaa345a319b1b8fd6)
> > and also made the needed changes on fuego-release-test to match those.
> > Hopefully they're more robust now. I plan to send all the new changes by
> e-
> > mail
> > to the list, but I just wanted to check if you're able to run it locally, so
> > that you can get a feeling about this, before we move on.
> >
> >
> > Can you please try again with:
> >
> >
> > ```
> > git clone https://bitbucket.org/profusionmobi/fuego-release-test
> > cd fuego-release-test
> > ./build_and_run -c up
>
> Still no joy.
>
> Step 7/7 : COPY fuego-rt-entrypoint.sh /
> ---> 4e39b71fb0d5
> Removing intermediate container a5e9b9e4244d
> Successfully built 4e39b71fb0d5
> Successfully tagged fuego-rt:latest
> fuego-rt-container
> a8750b71dc87bc1ad61ed26954c9484e79376d019fbd563abf86e1fcec86c70b
> Waiting for Jenkins...
> Creating job fuego-test.default.Functional.fuegotest
> fuego-rt-container
> [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> [ ok ] Starting network benchmark server.
> root@a8750b71dc87:/# tail -n 20 /var/log/jenkins/jenkins.log
> INFO: Winstone shutdown successfully
> Feb 22, 2018 9:15:14 PM winstone.Logger logInternal
> SEVERE: Container startup failed
> java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> INF/MANIFEST.MF (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at
> winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> at winstone.HostGroup.initHost(HostGroup.java:66)
> at winstone.HostGroup.<init>(HostGroup.java:45)
> at winstone.Launcher.<init>(Launcher.java:143)
> at winstone.Launcher.main(Launcher.java:352)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> ava:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> sorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at Main._main(Main.java:264)
> at Main.main(Main.java:112)
>
> ----
> But check out this sequence of actions:
>
> -----
> root@a8750b71dc87:/# id jenkins
> uid=2060932290(jenkins) gid=2060452353(jenkins)
> groups=2060452353(jenkins)
> ...
> root@a8750b71dc87:/# ls -ld /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
> drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> root@a8750b71dc87:/# su - jenkins
> jenkins@a8750b71dc87:~$ ls -ld /var /var/cache /var/cache/jenkins
> /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
> drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
> drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
> drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
> drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
> /var/cache/jenkins/war/META-INF
> -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
> /var/cache/jenkins/war/META-INF/MANIFEST.MF
> jenkins@a8750b71dc87:~$ cat /var/cache/jenkins/war/META-
> INF/MANIFEST.MF
> Manifest-Version: 1.0
> Jenkins-Version: 2.32.1
> Implementation-Version: 2.32.1
> Hudson-Version: 1.395
> Build-Jdk: 1.7.0_21
> Built-By: kohsuke
> Created-By: Apache Maven 3.3.9
> Main-Class: Main
> Archiver-Version: Plexus Archiver
>
> Name: images/48x48/red_anime.gif
> SHA-256-Digest: 2ktTeqkcRPD3cWv0z9xPu6Hf3kMNxoh55qzYKVMdCwk=
> ... [lots of lines omitted]
>
> Name: scripts/yui/assets/skins/sam/button.css
> SHA-256-Digest: Yp390BkBCcwkhhCQ4Fqj06dTN5qg/zI1QbaR3Gkl6LY=
>
> jenkins@a8750b71dc87:~$ exit
> logout
> root@a8750b71dc87:/# service jenkins start
> [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> root@a8750b71dc87:/#
> -----
> If the jenkins log message is correct, there's something really
> weird going on here. The permissions look OK, and the 'jenkins'
> user can access the MANIFEST.MF file, and a subsequent
> attempt to start jenkins (as root) works. So I'm not sure what's going on.
> -- Tim
>
> _______________________________________________
> Fuego mailing list
> Fuego@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/fuego
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-22 21:31 ` Tim.Bird
@ 2018-02-23 16:23 ` Guilherme Camargo
2018-02-26 18:30 ` Guilherme Camargo
0 siblings, 1 reply; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-23 16:23 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 8057 bytes --]
Hello, Tim
I think we're getting closer to the root cause.
It's still not very clear why, but there seems to be some sort of race
condition when, while running the fuego-release-test as a daemon we try to
execute fuego-rt-entrypoint.sh. For some reason, we might be trying to run
the
`ftc` commands while the uid/gid of jenkins have not been mapped yet in the
original entrypoint.sh.
With my newest change, we're calling `docker run` only once, guaranteeing
that:
- fuego-base "entrypoint.sh"
- fuego-release-test "fuego-rt-entrypoint.sh"
- and "/bin/bash"
are executed sequentially.
Would you please try again with the new commits that have been pushed to
master?
Steps are:
```
git clone https://bitbucket.org/profusionmobi/fuego-release-test
cd fuego-release-test
./build_and_run -c up
Then try to access fuego in localhost:8080/fuego/
You can also pass -p PORT to build_and_run and access fuego from that
location instead.
```
Thank you very much
--
Guilherme
On Thu, Feb 22, 2018 at 6:31 PM, <Tim.Bird@sony.com> wrote:
> But, just because Jenkins started, doesn't mean everything is working.
> I got a Jenkins "Oops" when trying to build the
> fuego-test.default.Functional.fuegotest job.
>
> Here's the first part of the console log:
> -----
> A problem occurred while processing the request. Please check our bug
> tracker to see if a similar problem has already been reported. If it is
> already reported, please vote and put a comment on it to let us gauge the
> impact of the problem. If you think this is a new issue, please file a new
> issue. When you file an issue, make sure to add the entire stack trace,
> along with the version of Jenkins and relevant plugins. The users list
> might be also useful in understanding what has happened.
>
> Stack trace
>
> javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException:
> jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-2.3
> 2.1.jar!/hudson/model/Run/console.jelly:65:27: <j:whitespace> Failed to
> load the key: hudson.console.AnnotatedLargeText.consoleAnnotator
> at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.j
> ava:103)
> at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
> at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
> at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
> at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
> at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
> at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
> at
> > -----Original Message-----
> > From: Tim Bird
> > > -----Original Message-----
> > > From: Guilherme Camargo
> > >
> > > I made a few more modifications in the way that the uid/gid are set on
> > Fuego
> > >
> > (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
> > > fa55aaa345a319b1b8fd6)
> > > and also made the needed changes on fuego-release-test to match those.
> > > Hopefully they're more robust now. I plan to send all the new changes
> by
> > e-
> > > mail
> > > to the list, but I just wanted to check if you're able to run it
> locally, so
> > > that you can get a feeling about this, before we move on.
> > >
> > >
> > > Can you please try again with:
> > >
> > >
> > > ```
> > > git clone https://bitbucket.org/profusionmobi/fuego-release-test
> > > cd fuego-release-test
> > > ./build_and_run -c up
> >
> > Still no joy.
> >
> > Step 7/7 : COPY fuego-rt-entrypoint.sh /
> > ---> 4e39b71fb0d5
> > Removing intermediate container a5e9b9e4244d
> > Successfully built 4e39b71fb0d5
> > Successfully tagged fuego-rt:latest
> > fuego-rt-container
> > a8750b71dc87bc1ad61ed26954c9484e79376d019fbd563abf86e1fcec86c70b
> > Waiting for Jenkins...
> > Creating job fuego-test.default.Functional.fuegotest
> > fuego-rt-container
> > [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
> > [ ok ] Starting network benchmark server.
> > root@a8750b71dc87:/# tail -n 20 /var/log/jenkins/jenkins.log
> > INFO: Winstone shutdown successfully
> > Feb 22, 2018 9:15:14 PM winstone.Logger logInternal
> > SEVERE: Container startup failed
> > java.io.FileNotFoundException: /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF (Permission denied)
> > at java.io.FileOutputStream.open(Native Method)
> > at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> > at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> > at
> > winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
> > at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
> > at winstone.HostGroup.initHost(HostGroup.java:66)
> > at winstone.HostGroup.<init>(HostGroup.java:45)
> > at winstone.Launcher.<init>(Launcher.java:143)
> > at winstone.Launcher.main(Launcher.java:352)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
> > ava:57)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
> > sorImpl.java:43)
> > at java.lang.reflect.Method.invoke(Method.java:606)
> > at Main._main(Main.java:264)
> > at Main.main(Main.java:112)
> >
> > ----
> > But check out this sequence of actions:
> >
> > -----
> > root@a8750b71dc87:/# id jenkins
> > uid=2060932290(jenkins) gid=2060452353(jenkins)
> > groups=2060452353(jenkins)
> > ...
> > root@a8750b71dc87:/# ls -ld /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
> > drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
> > drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
> > /var/cache/jenkins/war/META-INF
> > -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > root@a8750b71dc87:/# su - jenkins
> > jenkins@a8750b71dc87:~$ ls -ld /var /var/cache /var/cache/jenkins
> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
> > drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
> > drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
> > /var/cache/jenkins/war/META-INF
> > -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
> > jenkins@a8750b71dc87:~$ cat /var/cache/jenkins/war/META-
> > INF/MANIFEST.MF
> > Manifest-Version: 1.0
> > Jenkins-Version: 2.32.1
> > Implementation-Version: 2.32.1
> > Hudson-Version: 1.395
> > Build-Jdk: 1.7.0_21
> > Built-By: kohsuke
> > Created-By: Apache Maven 3.3.9
> > Main-Class: Main
> > Archiver-Version: Plexus Archiver
> >
> > Name: images/48x48/red_anime.gif
> > SHA-256-Digest: 2ktTeqkcRPD3cWv0z9xPu6Hf3kMNxoh55qzYKVMdCwk=
> > ... [lots of lines omitted]
> >
> > Name: scripts/yui/assets/skins/sam/button.css
> > SHA-256-Digest: Yp390BkBCcwkhhCQ4Fqj06dTN5qg/zI1QbaR3Gkl6LY=
> >
> > jenkins@a8750b71dc87:~$ exit
> > logout
> > root@a8750b71dc87:/# service jenkins start
> > [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
> > root@a8750b71dc87:/#
> > -----
> > If the jenkins log message is correct, there's something really
> > weird going on here. The permissions look OK, and the 'jenkins'
> > user can access the MANIFEST.MF file, and a subsequent
> > attempt to start jenkins (as root) works. So I'm not sure what's going
> on.
> > -- Tim
> >
> > _______________________________________________
> > Fuego mailing list
> > Fuego@lists.linuxfoundation.org
> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>
[-- Attachment #2: Type: text/html, Size: 16323 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-23 16:23 ` Guilherme Camargo
@ 2018-02-26 18:30 ` Guilherme Camargo
2018-02-26 18:52 ` Tim.Bird
0 siblings, 1 reply; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-26 18:30 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 8501 bytes --]
Hello, Tim
Hope everything is going well.
Did you have the chance to run the test again?
Thank you
--
Guilherme
On Fri, Feb 23, 2018 at 1:23 PM, Guilherme Camargo <guicc@profusion.mobi>
wrote:
> Hello, Tim
>
> I think we're getting closer to the root cause.
>
> It's still not very clear why, but there seems to be some sort of race
> condition when, while running the fuego-release-test as a daemon we try to
> execute fuego-rt-entrypoint.sh. For some reason, we might be trying to run
> the
> `ftc` commands while the uid/gid of jenkins have not been mapped yet in the
> original entrypoint.sh.
>
> With my newest change, we're calling `docker run` only once, guaranteeing
> that:
> - fuego-base "entrypoint.sh"
> - fuego-release-test "fuego-rt-entrypoint.sh"
> - and "/bin/bash"
> are executed sequentially.
>
> Would you please try again with the new commits that have been pushed to
> master?
>
> Steps are:
>
> ```
> git clone https://bitbucket.org/profusionmobi/fuego-release-test
> cd fuego-release-test
> ./build_and_run -c up
>
> Then try to access fuego in localhost:8080/fuego/
>
> You can also pass -p PORT to build_and_run and access fuego from that
> location instead.
>
> ```
>
> Thank you very much
>
> --
> Guilherme
>
>
> On Thu, Feb 22, 2018 at 6:31 PM, <Tim.Bird@sony.com> wrote:
>
>> But, just because Jenkins started, doesn't mean everything is working.
>> I got a Jenkins "Oops" when trying to build the
>> fuego-test.default.Functional.fuegotest job.
>>
>> Here's the first part of the console log:
>> -----
>> A problem occurred while processing the request. Please check our bug
>> tracker to see if a similar problem has already been reported. If it is
>> already reported, please vote and put a comment on it to let us gauge the
>> impact of the problem. If you think this is a new issue, please file a new
>> issue. When you file an issue, make sure to add the entire stack trace,
>> along with the version of Jenkins and relevant plugins. The users list
>> might be also useful in understanding what has happened.
>>
>> Stack trace
>>
>> javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException:
>> jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-2.3
>> 2.1.jar!/hudson/model/Run/console.jelly:65:27: <j:whitespace> Failed to
>> load the key: hudson.console.AnnotatedLargeText.consoleAnnotator
>> at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.j
>> ava:103)
>> at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
>> at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
>> at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
>> at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
>> at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
>> at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
>> at
>> > -----Original Message-----
>> > From: Tim Bird
>> > > -----Original Message-----
>> > > From: Guilherme Camargo
>> > >
>> > > I made a few more modifications in the way that the uid/gid are set on
>> > Fuego
>> > >
>> > (https://bitbucket.org/profusionmobi/fuego/commits/fe24e93ce191c461f4e
>> > > fa55aaa345a319b1b8fd6)
>> > > and also made the needed changes on fuego-release-test to match those.
>> > > Hopefully they're more robust now. I plan to send all the new changes
>> by
>> > e-
>> > > mail
>> > > to the list, but I just wanted to check if you're able to run it
>> locally, so
>> > > that you can get a feeling about this, before we move on.
>> > >
>> > >
>> > > Can you please try again with:
>> > >
>> > >
>> > > ```
>> > > git clone https://bitbucket.org/profusionmobi/fuego-release-test
>> > > cd fuego-release-test
>> > > ./build_and_run -c up
>> >
>> > Still no joy.
>> >
>> > Step 7/7 : COPY fuego-rt-entrypoint.sh /
>> > ---> 4e39b71fb0d5
>> > Removing intermediate container a5e9b9e4244d
>> > Successfully built 4e39b71fb0d5
>> > Successfully tagged fuego-rt:latest
>> > fuego-rt-container
>> > a8750b71dc87bc1ad61ed26954c9484e79376d019fbd563abf86e1fcec86c70b
>> > Waiting for Jenkins...
>> > Creating job fuego-test.default.Functional.fuegotest
>> > fuego-rt-container
>> > [FAIL] Starting Jenkins Continuous Integration Server: jenkins failed!
>> > [ ok ] Starting network benchmark server.
>> > root@a8750b71dc87:/# tail -n 20 /var/log/jenkins/jenkins.log
>> > INFO: Winstone shutdown successfully
>> > Feb 22, 2018 9:15:14 PM winstone.Logger logInternal
>> > SEVERE: Container startup failed
>> > java.io.FileNotFoundException: /var/cache/jenkins/war/META-
>> > INF/MANIFEST.MF (Permission denied)
>> > at java.io.FileOutputStream.open(Native Method)
>> > at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
>> > at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
>> > at
>> > winstone.HostConfiguration.getWebRoot(HostConfiguration.java:280)
>> > at winstone.HostConfiguration.<init>(HostConfiguration.java:83)
>> > at winstone.HostGroup.initHost(HostGroup.java:66)
>> > at winstone.HostGroup.<init>(HostGroup.java:45)
>> > at winstone.Launcher.<init>(Launcher.java:143)
>> > at winstone.Launcher.main(Launcher.java:352)
>> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> > at
>> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.j
>> > ava:57)
>> > at
>> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces
>> > sorImpl.java:43)
>> > at java.lang.reflect.Method.invoke(Method.java:606)
>> > at Main._main(Main.java:264)
>> > at Main.main(Main.java:112)
>> >
>> > ----
>> > But check out this sequence of actions:
>> >
>> > -----
>> > root@a8750b71dc87:/# id jenkins
>> > uid=2060932290(jenkins) gid=2060452353(jenkins)
>> > groups=2060452353(jenkins)
>> > ...
>> > root@a8750b71dc87:/# ls -ld /var /var/cache /var/cache/jenkins
>> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
>> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
>> > drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
>> > drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
>> > drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
>> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
>> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
>> > /var/cache/jenkins/war/META-INF
>> > -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
>> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
>> > root@a8750b71dc87:/# su - jenkins
>> > jenkins@a8750b71dc87:~$ ls -ld /var /var/cache /var/cache/jenkins
>> > /var/cache/jenkins/war /var/cache/jenkins/war/META-INF
>> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
>> > drwxr-xr-x 32 root root 4096 Feb 22 21:14 /var
>> > drwxr-xr-x 17 root root 4096 Feb 22 21:14 /var/cache
>> > drwxr-x--- 4 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins
>> > drwxr-xr-x 18 jenkins jenkins 4096 Feb 22 21:14 /var/cache/jenkins/war
>> > drwxr-xr-x 4 jenkins jenkins 4096 Feb 22 21:14
>> > /var/cache/jenkins/war/META-INF
>> > -rw-r--r-- 1 jenkins jenkins 129535 Feb 22 18:17
>> > /var/cache/jenkins/war/META-INF/MANIFEST.MF
>> > jenkins@a8750b71dc87:~$ cat /var/cache/jenkins/war/META-
>> > INF/MANIFEST.MF
>> > Manifest-Version: 1.0
>> > Jenkins-Version: 2.32.1
>> > Implementation-Version: 2.32.1
>> > Hudson-Version: 1.395
>> > Build-Jdk: 1.7.0_21
>> > Built-By: kohsuke
>> > Created-By: Apache Maven 3.3.9
>> > Main-Class: Main
>> > Archiver-Version: Plexus Archiver
>> >
>> > Name: images/48x48/red_anime.gif
>> > SHA-256-Digest: 2ktTeqkcRPD3cWv0z9xPu6Hf3kMNxoh55qzYKVMdCwk=
>> > ... [lots of lines omitted]
>> >
>> > Name: scripts/yui/assets/skins/sam/button.css
>> > SHA-256-Digest: Yp390BkBCcwkhhCQ4Fqj06dTN5qg/zI1QbaR3Gkl6LY=
>> >
>> > jenkins@a8750b71dc87:~$ exit
>> > logout
>> > root@a8750b71dc87:/# service jenkins start
>> > [ ok ] Starting Jenkins Continuous Integration Server: jenkins.
>> > root@a8750b71dc87:/#
>> > -----
>> > If the jenkins log message is correct, there's something really
>> > weird going on here. The permissions look OK, and the 'jenkins'
>> > user can access the MANIFEST.MF file, and a subsequent
>> > attempt to start jenkins (as root) works. So I'm not sure what's going
>> on.
>> > -- Tim
>> >
>> > _______________________________________________
>> > Fuego mailing list
>> > Fuego@lists.linuxfoundation.org
>> > https://lists.linuxfoundation.org/mailman/listinfo/fuego
>>
>
>
[-- Attachment #2: Type: text/html, Size: 17923 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-26 18:30 ` Guilherme Camargo
@ 2018-02-26 18:52 ` Tim.Bird
2018-02-26 21:03 ` Guilherme Camargo
0 siblings, 1 reply; 15+ messages in thread
From: Tim.Bird @ 2018-02-26 18:52 UTC (permalink / raw)
To: guicc; +Cc: fuego
> -----Original Message-----
> From: Guilherme Camargo [mailto:guicc@profusion.mobi]
> Sent: Monday, February 26, 2018 10:31 AM
> To: Bird, Timothy <Tim.Bird@sony.com>
> Cc: fuego@lists.linuxfoundation.org
> Subject: Re: [Fuego] release test issues
>
> Hello, Tim
>
> Hope everything is going well.
>
> Did you have the chance to run the test again?
I ran it just now...
Good news!! Jenkins starts in the container, and the fuego-test.default./Functional.fuegotest
job is created.
When I build that job in Jenkins, it immediately fails, and I get the following error message
in the console log (this is a copy paste from the Jenkins interface):
A problem occurred while processing the request. Please check our bug tracker to see if a similar problem has already been reported. If it is already reported, please vote and put a comment on it to let us gauge the impact of the problem. If you think this is a new issue, please file a new issue. When you file an issue, make sure to add the entire stack trace, along with the version of Jenkins and relevant plugins. The users list might be also useful in understanding what has happened.
Stack trace
javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException: jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-2.32.1.jar!/hudson/model/Run/console.jelly:65:27: <j:whitespace> Failed to load the key: hudson.console.AnnotatedLargeText.consoleAnnotator
at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(JellyFacet.java:103)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
at
... (lots more traceback).
------------
If I look at /var/lib/jenkins/jobs/fuego-test.default.Functional.fuegotest/builds/3/log, I see the following:
Started by user ^[[8mha:AAAAlh+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAzOEgYu/dLi1CL9vNKcHACFIKlWvwAAAA==^[[0manonymous
Building remotely on ^[[8mha:AAAAoB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAyuEgYR/eT83ILSktQi/bTS1PR83ZLU4hIADoGZQ8kAAAA=^[[0mfuego-testFATAL: Failed to load the key: hudson.model.Job.serverCookie
^[[8mha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=^[[0mjava.lang.Error: Failed to load the key: hudson.model.Job.serverCookie
at jenkins.security.HexStringConfidentialKey.get(HexStringConfidentialKey.java:62)
at hudson.model.Job.getCharacteristicEnvVars(Job.java:367)
at hudson.model.Job.getEnvironment(Job.java:395)
at hudson.model.AbstractProject.getEnvironment(AbstractProject.java:353)
at hudson.model.Run.getEnvironment(Run.java:2264)
at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:922)
at hudson.model.AbstractBuild$AbstractBuildExecution.decideWorkspace(AbstractBuild.java:478)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:518)
at hudson.model.Run.execute(Run.java:1728)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.FileNotFoundException: /var/lib/jenkins/secrets/hudson.model.Job.serverCookie (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at jenkins.security.DefaultConfidentialStore.store(DefaultConfidentialStore.java:79)
at jenkins.security.ConfidentialKey.store(ConfidentialKey.java:50)
at jenkins.security.HexStringConfidentialKey.get(HexStringConfidentialKey.java:54)
... 11 more
FATAL: Failed to load the key: hudson.model.Job.serverCookie
^[[8mha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=^[[0mjava.lang.Error: Failed to load the key: hudson.model.Job.serverCookie
at jenkins.security.HexStringConfidentialKey.get(HexStringConfidentialKey.java:62)
at hudson.model.Job.getCharacteristicEnvVars(Job.java:367)
at hudson.model.Job.getEnvironment(Job.java:395)
at hudson.model.AbstractProject.getEnvironment(AbstractProject.java:353)
at hudson.model.Run.getEnvironment(Run.java:2264)
at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.java:922)
at hudson.plugins.descriptionsetter.DescriptionSetterHelper.setDescription(DescriptionSetterHelper.java:53)
at hudson.plugins.descriptionsetter.DescriptionSetterPublisher.perform(DescriptionSetterPublisher.java:70)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.AbstractBuild$AbstractBuildExecution.performAllBuildSteps(AbstractBuild.java:720)
at hudson.model.Build$BuildExecution.post2(Build.java:185)
at hudson.model.AbstractBuild$AbstractBuildExecution.post(AbstractBuild.java:665)
at hudson.model.Run.execute(Run.java:1753)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:404)
Caused by: java.io.FileNotFoundException: /var/lib/jenkins/secrets/hudson.model.Job.serverCookie (Permission denied)
at java.io.FileOutputStream.open(Native Method)
at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
at jenkins.security.DefaultConfidentialStore.store(DefaultConfidentialStore.java:79)
at jenkins.security.ConfidentialKey.store(ConfidentialKey.java:50)
at jenkins.security.HexStringConfidentialKey.get(HexStringConfidentialKey.java:54)
... 16 more
Finished: FAILURE
----------
Let me know if there is more information I can provide.
BTW - I tried to copy the log file to /fuego-rw/transfer so I could extract it from the container,
but I can't find fuego-rw on my host that corresponds to this. Where would this directory
be located, in the release test configuration of the containers?
-- Tim
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [Fuego] release test issues
2018-02-26 18:52 ` Tim.Bird
@ 2018-02-26 21:03 ` Guilherme Camargo
0 siblings, 0 replies; 15+ messages in thread
From: Guilherme Camargo @ 2018-02-26 21:03 UTC (permalink / raw)
To: Bird, Timothy; +Cc: fuego
[-- Attachment #1: Type: text/plain, Size: 7906 bytes --]
Hello, Tim, good to know that Jenkins started now.
But, still, there seems to be some issue with permissions that for some
reason affects your system and does not affect either mine or my peers'.
Can you please share the logs that are stored in fuego-rw inside
the fuego-rt-container?
We're currently not mapping fuego-rw to the host, neither on
fuego-release-test
(the fuego that is testing), nor in the fuego that's being tested - what
means
that the user would need to use Jenkin's web interface to read the logs, at
least for now. But we could change that to allow mapping if you find it
necessary.
Thank you a lot
--
Guilherme
On Mon, Feb 26, 2018 at 3:52 PM, <Tim.Bird@sony.com> wrote:
>
>
> > -----Original Message-----
> > From: Guilherme Camargo [mailto:guicc@profusion.mobi]
> > Sent: Monday, February 26, 2018 10:31 AM
> > To: Bird, Timothy <Tim.Bird@sony.com>
> > Cc: fuego@lists.linuxfoundation.org
> > Subject: Re: [Fuego] release test issues
> >
> > Hello, Tim
> >
> > Hope everything is going well.
> >
> > Did you have the chance to run the test again?
>
> I ran it just now...
>
> Good news!! Jenkins starts in the container, and the fuego-test.default./
> Functional.fuegotest
> job is created.
>
> When I build that job in Jenkins, it immediately fails, and I get the
> following error message
> in the console log (this is a copy paste from the Jenkins interface):
>
> A problem occurred while processing the request. Please check our bug
> tracker to see if a similar problem has already been reported. If it is
> already reported, please vote and put a comment on it to let us gauge the
> impact of the problem. If you think this is a new issue, please file a new
> issue. When you file an issue, make sure to add the entire stack trace,
> along with the version of Jenkins and relevant plugins. The users list
> might be also useful in understanding what has happened.
>
> Stack trace
>
> javax.servlet.ServletException: org.apache.commons.jelly.JellyTagException:
> jar:file:/var/cache/jenkins/war/WEB-INF/lib/jenkins-core-
> 2.32.1.jar!/hudson/model/Run/console.jelly:65:27: <j:whitespace> Failed
> to load the key: hudson.console.AnnotatedLargeText.consoleAnnotator
> at org.kohsuke.stapler.jelly.JellyFacet$1.dispatch(
> JellyFacet.java:103)
> at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
> at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
> at org.kohsuke.stapler.MetaClass$10.dispatch(MetaClass.java:362)
> at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
> at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
> at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:236)
> at
> ... (lots more traceback).
>
> ------------
> If I look at /var/lib/jenkins/jobs/fuego-test.default.Functional.fuegotest/builds/3/log,
> I see the following:
>
> Started by user ^[[8mha:AAAAlh+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+
> vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3Rdl
> LBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHs
> LgAzOEgYu/dLi1CL9vNKcHACFIKlWvwAAAA==^[[0manonymous
> Building remotely on ^[[8mha:AAAAoB+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+
> vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3Rdl
> LBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHs
> LgAyuEgYR/eT83ILSktQi/bTS1PR83ZLU4hIADoGZQ8kAAAA=^[[0mfuego-testFATAL:
> Failed to load the key: hudson.model.Job.serverCookie
> ^[[8mha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/
> PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=^[[0mjava.lang.Error:
> Failed to load the key: hudson.model.Job.serverCookie
> at jenkins.security.HexStringConfidentialKey.get(
> HexStringConfidentialKey.java:62)
> at hudson.model.Job.getCharacteristicEnvVars(Job.java:367)
> at hudson.model.Job.getEnvironment(Job.java:395)
> at hudson.model.AbstractProject.getEnvironment(
> AbstractProject.java:353)
> at hudson.model.Run.getEnvironment(Run.java:2264)
> at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.
> java:922)
> at hudson.model.AbstractBuild$AbstractBuildExecution.
> decideWorkspace(AbstractBuild.java:478)
> at hudson.model.AbstractBuild$AbstractBuildExecution.run(
> AbstractBuild.java:518)
> at hudson.model.Run.execute(Run.java:1728)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(
> ResourceController.java:98)
> at hudson.model.Executor.run(Executor.java:404)
> Caused by: java.io.FileNotFoundException: /var/lib/jenkins/secrets/hudson.model.Job.serverCookie
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at jenkins.security.DefaultConfidentialStore.store(
> DefaultConfidentialStore.java:79)
> at jenkins.security.ConfidentialKey.store(ConfidentialKey.java:50)
> at jenkins.security.HexStringConfidentialKey.get(
> HexStringConfidentialKey.java:54)
> ... 11 more
> FATAL: Failed to load the key: hudson.model.Job.serverCookie
> ^[[8mha:AAAAWB+LCAAAAAAAAP9b85aBtbiIQSmjNKU4P08vOT+vOD8nVc8DzHWtSE4tKMnMz/
> PLL0ldFVf2c+b/lb5MDAwVRQxSaBqcITRIIQMEMIIUFgAAckCEiWAAAAA=^[[0mjava.lang.Error:
> Failed to load the key: hudson.model.Job.serverCookie
> at jenkins.security.HexStringConfidentialKey.get(
> HexStringConfidentialKey.java:62)
> at hudson.model.Job.getCharacteristicEnvVars(Job.java:367)
> at hudson.model.Job.getEnvironment(Job.java:395)
> at hudson.model.AbstractProject.getEnvironment(
> AbstractProject.java:353)
> at hudson.model.Run.getEnvironment(Run.java:2264)
> at hudson.model.AbstractBuild.getEnvironment(AbstractBuild.
> java:922)
> at hudson.plugins.descriptionsetter.DescriptionSetterHelper.
> setDescription(DescriptionSetterHelper.java:53)
> at hudson.plugins.descriptionsetter.DescriptionSetterPublisher.
> perform(DescriptionSetterPublisher.java:70)
> at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.
> java:20)
> at hudson.model.AbstractBuild$AbstractBuildExecution.
> perform(AbstractBuild.java:779)
> at hudson.model.AbstractBuild$AbstractBuildExecution.
> performAllBuildSteps(AbstractBuild.java:720)
> at hudson.model.Build$BuildExecution.post2(Build.java:185)
> at hudson.model.AbstractBuild$AbstractBuildExecution.post(
> AbstractBuild.java:665)
> at hudson.model.Run.execute(Run.java:1753)
> at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
> at hudson.model.ResourceController.execute(
> ResourceController.java:98)
> at hudson.model.Executor.run(Executor.java:404)
> Caused by: java.io.FileNotFoundException: /var/lib/jenkins/secrets/hudson.model.Job.serverCookie
> (Permission denied)
> at java.io.FileOutputStream.open(Native Method)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:221)
> at java.io.FileOutputStream.<init>(FileOutputStream.java:171)
> at jenkins.security.DefaultConfidentialStore.store(
> DefaultConfidentialStore.java:79)
> at jenkins.security.ConfidentialKey.store(ConfidentialKey.java:50)
> at jenkins.security.HexStringConfidentialKey.get(
> HexStringConfidentialKey.java:54)
> ... 16 more
> Finished: FAILURE
>
> ----------
> Let me know if there is more information I can provide.
>
> BTW - I tried to copy the log file to /fuego-rw/transfer so I could
> extract it from the container,
> but I can't find fuego-rw on my host that corresponds to this. Where
> would this directory
> be located, in the release test configuration of the containers?
> -- Tim
>
[-- Attachment #2: Type: text/html, Size: 11108 bytes --]
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2018-02-26 21:03 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-21 14:48 [Fuego] release test issues Tim.Bird
2018-02-21 17:23 ` Guilherme Camargo
2018-02-22 1:19 ` Tim.Bird
2018-02-22 1:59 ` Guilherme Camargo
2018-02-22 20:19 ` Tim.Bird
2018-02-22 21:02 ` Guilherme Camargo
2018-02-22 21:05 ` Tim.Bird
2018-02-22 21:06 ` Guilherme Camargo
2018-02-22 21:27 ` Tim.Bird
2018-02-22 21:29 ` Guilherme Camargo
2018-02-22 21:31 ` Tim.Bird
2018-02-23 16:23 ` Guilherme Camargo
2018-02-26 18:30 ` Guilherme Camargo
2018-02-26 18:52 ` Tim.Bird
2018-02-26 21:03 ` Guilherme Camargo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.