[OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Alex Kiernan]

Generate output for each contaminated path so it's visible in the main
log file, then enable rootfs_check_host_user_contaminated by default.

Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
---

Changes in v2:
- expose each contaminated path in the main log

 meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/meta/classes/rootfs-postcommands.bbclass b/meta/classes/rootfs-postcommands.bbclass
index 2f171836fa0c..8dde15fc3407 100644
--- a/meta/classes/rootfs-postcommands.bbclass
+++ b/meta/classes/rootfs-postcommands.bbclass
@@ -37,7 +37,7 @@ POSTINST_LOGFILE ?= "${localstatedir}/log/postinstall.log"
 SYSTEMD_DEFAULT_TARGET ?= '${@bb.utils.contains("IMAGE_FEATURES", "x11-base", "graphical.target", "multi-user.target", d)}'
 ROOTFS_POSTPROCESS_COMMAND += '${@bb.utils.contains("DISTRO_FEATURES", "systemd", "set_systemd_default_target; systemd_create_users;", "", d)}'
 
-ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile;'
+ROOTFS_POSTPROCESS_COMMAND += 'empty_var_volatile; rootfs_check_host_user_contaminated;'
 
 # Sort the user and group entries in /etc by ID in order to make the content
 # deterministic. Package installs are not deterministic, causing the ordering
@@ -311,10 +311,9 @@ rootfs_check_host_user_contaminated () {
 	find "${IMAGE_ROOTFS}" -wholename "${IMAGE_ROOTFS}/home" -prune \
 	    -user "$HOST_USER_UID" -o -group "$HOST_USER_GID" >"$contaminated"
 
-	if [ -s "$contaminated" ]; then
-		echo "WARNING: Paths in the rootfs are owned by the same user or group as the user running bitbake. See the logfile for the specific paths."
-		cat "$contaminated" | sed "s,^,  ,"
-	fi
+	sed -e "s,${IMAGE_ROOTFS},," $contaminated | while read line; do
+		bbwarn "Path in the rootfs is owned by the same user or group as the user running bitbake:" $line
+	done
 }
 
 # Make any absolute links in a sysroot relative
-- 
2.17.1

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Richard Purdie]

On Mon, 2020-06-22 at 11:57 +0100, Alex Kiernan wrote:
> Generate output for each contaminated path so it's visible in the main
> log file, then enable rootfs_check_host_user_contaminated by default.
> 
> Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> ---
> 
> Changes in v2:
> - expose each contaminated path in the main log
> 
>  meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)

Thanks, this is definitely more helpful. Some warnings in the test
build:

https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/2046

step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd
step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm.bak
step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/state
step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm

https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/2073

step3b: WARNING: core-image-sato-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/cache/fontconfig/CACHEDIR.TAG

which does raise more questions.

Its conceivable its even related to the pseudo issue I just sent a patch for...

Cheers,

Richard

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Alex Kiernan]

On Mon, Jun 22, 2020 at 6:34 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2020-06-22 at 11:57 +0100, Alex Kiernan wrote:
> > Generate output for each contaminated path so it's visible in the main
> > log file, then enable rootfs_check_host_user_contaminated by default.
> >
> > Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> > ---
> >
> > Changes in v2:
> > - expose each contaminated path in the main log
> >
> >  meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
> >  1 file changed, 4 insertions(+), 5 deletions(-)
>
> Thanks, this is definitely more helpful. Some warnings in the test
> build:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/2046
>
> step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd
> step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm.bak
> step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/state
> step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/2073
>
> step3b: WARNING: core-image-sato-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/cache/fontconfig/CACHEDIR.TAG
>
> which does raise more questions.
>

I'm having no luck reproducing either of these locally :|

> Its conceivable its even related to the pseudo issue I just sent a patch for...
>


-- 
Alex Kiernan

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Richard Purdie]

On Mon, 2020-06-22 at 19:53 +0100, Alex Kiernan wrote:
> On Mon, Jun 22, 2020 at 6:34 PM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Mon, 2020-06-22 at 11:57 +0100, Alex Kiernan wrote:
> > > Generate output for each contaminated path so it's visible in the main
> > > log file, then enable rootfs_check_host_user_contaminated by default.
> > > 
> > > Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> > > ---
> > > 
> > > Changes in v2:
> > > - expose each contaminated path in the main log
> > > 
> > >  meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
> > >  1 file changed, 4 insertions(+), 5 deletions(-)
> > 
> > Thanks, this is definitely more helpful. Some warnings in the test
> > build:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/2046
> > 
> > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd
> > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm.bak
> > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/state
> > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/2073
> > 
> > step3b: WARNING: core-image-sato-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/cache/fontconfig/CACHEDIR.TAG
> > 
> > which does raise more questions.
> > 
> 
> I'm having no luck reproducing either of these locally :|

They don't reproduce on the autobuilder either. We're probably going to
need to open a bug in bugzilla and enter these there listing the target
that caused it and the host it was on.

Hopefully we'll see a pattern emerge.

Or maybe the pseudo fix I posted will make it all go away. I can live
in hope right?!

Either way, I think this patch is going to be a longer journey until we
figure out what is happening.

Cheers,

Richard

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Richard Purdie]

On Mon, 2020-06-22 at 20:46 +0100, Richard Purdie via lists.openembedded.org wrote:
> On Mon, 2020-06-22 at 19:53 +0100, Alex Kiernan wrote:
> > On Mon, Jun 22, 2020 at 6:34 PM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > > On Mon, 2020-06-22 at 11:57 +0100, Alex Kiernan wrote:
> > > > Generate output for each contaminated path so it's visible in the main
> > > > log file, then enable rootfs_check_host_user_contaminated by default.
> > > > 
> > > > Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> > > > ---
> > > > 
> > > > Changes in v2:
> > > > - expose each contaminated path in the main log
> > > > 
> > > >  meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
> > > >  1 file changed, 4 insertions(+), 5 deletions(-)
> > > 
> > > Thanks, this is definitely more helpful. Some warnings in the test
> > > build:
> > > 
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/2046
> > > 
> > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd
> > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm.bak
> > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/state
> > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm
> > > 
> > > https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/2073
> > > 
> > > step3b: WARNING: core-image-sato-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/cache/fontconfig/CACHEDIR.TAG
> > > 
> > > which does raise more questions.
> > > 
> > 
> > I'm having no luck reproducing either of these locally :|
> 
> They don't reproduce on the autobuilder either. We're probably going to
> need to open a bug in bugzilla and enter these there listing the target
> that caused it and the host it was on.
> 
> Hopefully we'll see a pattern emerge.
> 
> Or maybe the pseudo fix I posted will make it all go away. I can live
> in hope right?!
> 
> Either way, I think this patch is going to be a longer journey until we
> figure out what is happening.

Last nights build also gave warnings:

https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/2055

step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /usr/bin/crontab
step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/spool/cron

which means the pseudo patch doesn't fix it.

Cheers,

Richard

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Alex Kiernan]

On Tue, Jun 23, 2020 at 9:20 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Mon, 2020-06-22 at 20:46 +0100, Richard Purdie via lists.openembedded.org wrote:
> > On Mon, 2020-06-22 at 19:53 +0100, Alex Kiernan wrote:
> > > On Mon, Jun 22, 2020 at 6:34 PM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > > On Mon, 2020-06-22 at 11:57 +0100, Alex Kiernan wrote:
> > > > > Generate output for each contaminated path so it's visible in the main
> > > > > log file, then enable rootfs_check_host_user_contaminated by default.
> > > > >
> > > > > Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> > > > > ---
> > > > >
> > > > > Changes in v2:
> > > > > - expose each contaminated path in the main log
> > > > >
> > > > >  meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
> > > > >  1 file changed, 4 insertions(+), 5 deletions(-)
> > > >
> > > > Thanks, this is definitely more helpful. Some warnings in the test
> > > > build:
> > > >
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/2046
> > > >
> > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd
> > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm.bak
> > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/state
> > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/lib/nfs/statd/sm
> > > >
> > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/2073
> > > >
> > > > step3b: WARNING: core-image-sato-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/cache/fontconfig/CACHEDIR.TAG
> > > >
> > > > which does raise more questions.
> > > >
> > >
> > > I'm having no luck reproducing either of these locally :|
> >
> > They don't reproduce on the autobuilder either. We're probably going to
> > need to open a bug in bugzilla and enter these there listing the target
> > that caused it and the host it was on.
> >
> > Hopefully we'll see a pattern emerge.
> >
> > Or maybe the pseudo fix I posted will make it all go away. I can live
> > in hope right?!
> >
> > Either way, I think this patch is going to be a longer journey until we
> > figure out what is happening.
>
> Last nights build also gave warnings:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/2055
>
> step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /usr/bin/crontab
> step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/spool/cron
>

Interestingly both of those have file permissions which aren't simple
ugo-rwx type things.

-- 
Alex Kiernan

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Richard Purdie]

On Tue, 2020-06-23 at 10:01 +0100, Alex Kiernan wrote:
> On Tue, Jun 23, 2020 at 9:20 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> > On Mon, 2020-06-22 at 20:46 +0100, Richard Purdie via
> > lists.openembedded.org wrote:
> > > On Mon, 2020-06-22 at 19:53 +0100, Alex Kiernan wrote:
> > > > On Mon, Jun 22, 2020 at 6:34 PM Richard Purdie
> > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > On Mon, 2020-06-22 at 11:57 +0100, Alex Kiernan wrote:
> > > > > > Generate output for each contaminated path so it's visible
> > > > > > in the main
> > > > > > log file, then enable rootfs_check_host_user_contaminated
> > > > > > by default.
> > > > > > 
> > > > > > Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com>
> > > > > > ---
> > > > > > 
> > > > > > Changes in v2:
> > > > > > - expose each contaminated path in the main log
> > > > > > 
> > > > > >  meta/classes/rootfs-postcommands.bbclass | 9 ++++-----
> > > > > >  1 file changed, 4 insertions(+), 5 deletions(-)
> > > > > 
> > > > > Thanks, this is definitely more helpful. Some warnings in the
> > > > > test
> > > > > build:
> > > > > 
> > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/50/builds/2046
> > > > > 
> > > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path
> > > > > in the rootfs is owned by the same user or group as the user
> > > > > running bitbake: /var/lib/nfs/statd
> > > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path
> > > > > in the rootfs is owned by the same user or group as the user
> > > > > running bitbake: /var/lib/nfs/statd/sm.bak
> > > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path
> > > > > in the rootfs is owned by the same user or group as the user
> > > > > running bitbake: /var/lib/nfs/statd/state
> > > > > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path
> > > > > in the rootfs is owned by the same user or group as the user
> > > > > running bitbake: /var/lib/nfs/statd/sm
> > > > > 
> > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/44/builds/2073
> > > > > 
> > > > > step3b: WARNING: core-image-sato-1.0-r0 do_rootfs: Path in
> > > > > the rootfs is owned by the same user or group as the user
> > > > > running bitbake: /var/cache/fontconfig/CACHEDIR.TAG
> > > > > 
> > > > > which does raise more questions.
> > > > > 
> > > > 
> > > > I'm having no luck reproducing either of these locally :|
> > > 
> > > They don't reproduce on the autobuilder either. We're probably
> > > going to
> > > need to open a bug in bugzilla and enter these there listing the
> > > target
> > > that caused it and the host it was on.
> > > 
> > > Hopefully we'll see a pattern emerge.
> > > 
> > > Or maybe the pseudo fix I posted will make it all go away. I can
> > > live
> > > in hope right?!
> > > 
> > > Either way, I think this patch is going to be a longer journey
> > > until we
> > > figure out what is happening.
> > 
> > Last nights build also gave warnings:
> > 
> > https://autobuilder.yoctoproject.org/typhoon/#/builders/64/builds/2055
> > 
> > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the
> > rootfs is owned by the same user or group as the user running
> > bitbake: /usr/bin/crontab
> > step1b: WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the
> > rootfs is owned by the same user or group as the user running
> > bitbake: /var/spool/cron
> > 
> 
> Interestingly both of those have file permissions which aren't simple
> ugo-rwx type things.

Looking at the pattern, I did have a thought which I've not researched
yet.

Is it possible these are dynamically generated group/users in the
rootfs which happen to collide with the host user/group?

You'd think the adduser/addgroup calls would happen in a deterministic
order as the rootfs is build but perhaps it doesn't work out like that.

Are each of the above groups of files, specifically created users at
rootfs time?

Cheers,

Richard

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Richard Purdie]

On Tue, 2020-06-23 at 11:01 +0100, Richard Purdie wrote:
> Looking at the pattern, I did have a thought which I've not
> researched
> yet.
> 
> Is it possible these are dynamically generated group/users in the
> rootfs which happen to collide with the host user/group?
> 
> You'd think the adduser/addgroup calls would happen in a
> deterministic
> order as the rootfs is build but perhaps it doesn't work out like
> that.
> 
> Are each of the above groups of files, specifically created users at
> rootfs time?

At this point this patchset is driving me a bit crazy. I've had it in
master-next, we keep seeing warnings and its hard to know whether the
warnings are real ones or ones from this patch.

I'm worried that things are not deterministic and this meant I really
needed to investigate.

I added some fixes and more debug to the patch, then ran it on the
autobuilder again. An example failure is:

https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/2113

WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/spool/cron total 8 drwxrwx--- 2 0 995 4096 Oct 31 2019 . drwxr-xr-x 4 0 0 4096 Jul 3 12:27 ..
WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /usr/bin/crontab -rwxr-sr-x 1 0 995 51544 Oct 31 2019 /home/pokybuild/yocto-worker/musl-qemux86-64/build/build/tmp/work/qemux86_64-poky-linux-musl/core-image-sato-sdk/1.0-r0/rootfs//usr/bin/crontab
WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: /etc/passwd: root:x:0:0:root:/home/root:/bin/sh daemon:x:1:1:daemon:/usr/sbin:/bin/sh bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh sync:x:4:65534:sync:/bin:/bin/sync games:x:5:60:games:/usr/games:/bin/sh man:x:6:12:man:/var/cache/man:/bin/sh lp:x:7:7:lp:/var/spool/lpd:/bin/sh mail:x:8:8:mail:/var/mail:/bin/sh news:x:9:9:news:/var/spool/news:/bin/sh uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh proxy:x:13:13:proxy:/bin:/bin/sh www-data:x:33:33:www-data:/var/www:/bin/sh backup:x:34:34:backup:/var/backups:/bin/sh list:x:38:38:Mailing List Manager:/var/list:/bin/sh irc:x:39:39:ircd:/var/run/ircd:/bin/sh gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/bin/sh nobody:x:65534:65534:nobody:/nonexistent:/bin/sh messagebus:x:999:999::/var/lib/dbus:/bin/false rpc:x:998:998::/:/bin/false rpcuser:x:997:997::/var/lib/nfs:/bin/false avahi:x:996:996::/run/avahi-daemon:/bin/false sshd:x:995:994::/var/run/sshd:/bin/false pulse:x:994:993::/var/run/pulse:/bin/false
WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: /etc/group: root:x:0: daemon:x:1: bin:x:2: sys:x:3: adm:x:4: tty:x:5: disk:x:6: lp:x:7: mail:x:8: news:x:9: uucp:x:10: man:x:12: proxy:x:13: kmem:x:15: input:x:19: dialout:x:20: fax:x:21: voice:x:22: cdrom:x:24: floppy:x:25: tape:x:26: sudo:x:27: audio:x:29:pulse dip:x:30: www-data:x:33: backup:x:34: operator:x:37: list:x:38: irc:x:39: src:x:40: gnats:x:41: shadow:x:42: utmp:x:43: video:x:44: sasl:x:45: plugdev:x:46: kvm:x:47: staff:x:50: games:x:60: shutdown:x:70: users:x:100: nogroup:x:65534: messagebus:x:999: rpc:x:998: rpcuser:x:997: avahi:x:996: crontab:x:995: tracing:x:1000: sshd:x:994: pulse:x:993:pulse

On that centos7 worker, the build UID is 6000, the GID is 995.

As we can see from the dump of /etc/group, crontab is GID 995 and
crontab files owned by that group are fine. So the warning is a false
one and my theory about the cause is confirmed.

We'd only see the issue where the dynamicly assigned UIDs/GIDs conflict
with the host running the build which explains the intermittent nature.

I don't know how to fix this. I'm going to batch together the fixes we
have and send those, then drop this patch from master-next until we can
figure out something better to use in testing.

Cheers,

Richard

Re: [OE-Core][PATCH v2] rootfs-postcommands: Enable rootfs_check_host_user_contaminated

[Alex Kiernan]

On Fri, Jul 3, 2020 at 2:14 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2020-06-23 at 11:01 +0100, Richard Purdie wrote:
> > Looking at the pattern, I did have a thought which I've not
> > researched
> > yet.
> >
> > Is it possible these are dynamically generated group/users in the
> > rootfs which happen to collide with the host user/group?
> >
> > You'd think the adduser/addgroup calls would happen in a
> > deterministic
> > order as the rootfs is build but perhaps it doesn't work out like
> > that.
> >
> > Are each of the above groups of files, specifically created users at
> > rootfs time?
>
> At this point this patchset is driving me a bit crazy. I've had it in
> master-next, we keep seeing warnings and its hard to know whether the
> warnings are real ones or ones from this patch.
>
> I'm worried that things are not deterministic and this meant I really
> needed to investigate.
>
> I added some fixes and more debug to the patch,

Oh, that's the /home stuff... I looked at that, thought it looked
wrong and totally forgot to go back to it :|

> then ran it on the
> autobuilder again. An example failure is:
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/45/builds/2113
>
> WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /var/spool/cron total 8 drwxrwx--- 2 0 995 4096 Oct 31 2019 . drwxr-xr-x 4 0 0 4096 Jul 3 12:27 ..
> WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: Path in the rootfs is owned by the same user or group as the user running bitbake: /usr/bin/crontab -rwxr-sr-x 1 0 995 51544 Oct 31 2019 /home/pokybuild/yocto-worker/musl-qemux86-64/build/build/tmp/work/qemux86_64-poky-linux-musl/core-image-sato-sdk/1.0-r0/rootfs//usr/bin/crontab
> WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: /etc/passwd: root:x:0:0:root:/home/root:/bin/sh daemon:x:1:1:daemon:/usr/sbin:/bin/sh bin:x:2:2:bin:/bin:/bin/sh sys:x:3:3:sys:/dev:/bin/sh sync:x:4:65534:sync:/bin:/bin/sync games:x:5:60:games:/usr/games:/bin/sh man:x:6:12:man:/var/cache/man:/bin/sh lp:x:7:7:lp:/var/spool/lpd:/bin/sh mail:x:8:8:mail:/var/mail:/bin/sh news:x:9:9:news:/var/spool/news:/bin/sh uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh proxy:x:13:13:proxy:/bin:/bin/sh www-data:x:33:33:www-data:/var/www:/bin/sh backup:x:34:34:backup:/var/backups:/bin/sh list:x:38:38:Mailing List Manager:/var/list:/bin/sh irc:x:39:39:ircd:/var/run/ircd:/bin/sh gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/bin/sh nobody:x:65534:65534:nobody:/nonexistent:/bin/sh messagebus:x:999:999::/var/lib/dbus:/bin/false rpc:x:998:998::/:/bin/false rpcuser:x:997:997::/var/lib/nfs:/bin/false avahi:x:996:996::/run/avahi-daemon:/bin/false sshd:x:995:994::/var/run/sshd:/bin/false pulse:x:994:993::/var/run/pulse:/bin/false
> WARNING: core-image-sato-sdk-1.0-r0 do_rootfs: /etc/group: root:x:0: daemon:x:1: bin:x:2: sys:x:3: adm:x:4: tty:x:5: disk:x:6: lp:x:7: mail:x:8: news:x:9: uucp:x:10: man:x:12: proxy:x:13: kmem:x:15: input:x:19: dialout:x:20: fax:x:21: voice:x:22: cdrom:x:24: floppy:x:25: tape:x:26: sudo:x:27: audio:x:29:pulse dip:x:30: www-data:x:33: backup:x:34: operator:x:37: list:x:38: irc:x:39: src:x:40: gnats:x:41: shadow:x:42: utmp:x:43: video:x:44: sasl:x:45: plugdev:x:46: kvm:x:47: staff:x:50: games:x:60: shutdown:x:70: users:x:100: nogroup:x:65534: messagebus:x:999: rpc:x:998: rpcuser:x:997: avahi:x:996: crontab:x:995: tracing:x:1000: sshd:x:994: pulse:x:993:pulse
>
> On that centos7 worker, the build UID is 6000, the GID is 995.
>
> As we can see from the dump of /etc/group, crontab is GID 995 and
> crontab files owned by that group are fine. So the warning is a false
> one and my theory about the cause is confirmed.
>
> We'd only see the issue where the dynamicly assigned UIDs/GIDs conflict
> with the host running the build which explains the intermittent nature.
>
> I don't know how to fix this.

No, I can't think of anything off the top of my head either. I keep
coming back to AB configuration, which you can't rely on outside of
that environment.

> I'm going to batch together the fixes we
> have and send those, then drop this patch from master-next until we can
> figure out something better to use in testing.
>

I'll keep thinking about it...

-- 
Alex Kiernan