KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Marek Belisko]

Hi,

I'm bumping my project based on zeus to dunfell. I've update all
layers and in one of my recipes I'm seeing following issue (not see on
zeus at all):
WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png
ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function
in exec_python_func() autogenerated:

The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
     0001:
 *** 0002:sstate_report_unihash(d)
     0003:
File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass',
lineno: 840, function: sstate_report_unihash
     0836:    report_unihash = getattr(bb.parse.siggen, 'report_unihash', None)
     0837:
     0838:    if report_unihash:
     0839:        ss = sstate_state_fromvars(d)
 *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
     0841:}
     0842:
     0843:#
     0844:# Shell function to decompress and prepare a package for installation
File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py',
lineno: 555, function: report_unihash
     0551:
     0552:            if "." in self.method:
     0553:                (module, method) = self.method.rsplit('.', 1)
     0554:                locs['method'] =
getattr(importlib.import_module(module), method)
 *** 0555:                outhash = bb.utils.better_eval('method(path,
sigfile, task, d)', locs)
     0556:            else:
     0557:                outhash = bb.utils.better_eval(self.method +
'(path, sigfile, task, d)', locs)
     0558:
     0559:            try:
File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py',
lineno: 420, function: better_eval
     0416:    if extraglobals:
     0417:        ctx = copy.copy(ctx)
     0418:        for g in extraglobals:
     0419:            ctx[g] = extraglobals[g]
 *** 0420:    return eval(source, ctx, locals)
     0421:
     0422:@contextmanager
     0423:def fileslocked(files):
     0424:    """Context manager for locking and unlocking file locks."""
File: '<string>', lineno: 1, function: <module>
  File "<string>", line 1, in <module>

File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
lineno: 595, function: OEOuthashBasic
     0591:            process(root)
     0592:            for f in files:
     0593:                if f == 'fixmepath':
     0594:                    continue
 *** 0595:                process(os.path.join(root, f))
     0596:    finally:
     0597:        os.chdir(prev_dir)
     0598:
     0599:    return h.hexdigest()
File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
lineno: 551, function: process
     0547:                    add_perm(stat.S_IXOTH, 'x')
     0548:
     0549:                if include_owners:
     0550:                    try:
 *** 0551:                        update_hash(" %10s" %
pwd.getpwuid(s.st_uid).pw_name)
     0552:                        update_hash(" %10s" %
grp.getgrgid(s.st_gid).gr_name)
     0553:                    except KeyError:
     0554:                        bb.warn("KeyError in %s" % path)
     0555:                        raise
Exception: KeyError: 'getpwuid(): uid not found: 1000'

ERROR: Logfile of failure stored in:
/home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454

Is this known issue or something related to my host setup?

Thanks and BR,

marek

-- 
as simple and primitive as possible
-------------------------------------------------
Marek Belisko - OPEN-NANDRA
Freelance Developer

Ruska Nova Ves 219 | Presov, 08005 Slovak Republic
Tel: +421 915 052 184
skype: marekwhite
twitter: #opennandra
web: http://open-nandra.com

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Joshua Watt]

[-- Attachment #1: Type: text/plain, Size: 3939 bytes --]


On 11/16/20 2:38 PM, Marek Belisko wrote:
> Hi,
>
> I'm bumping my project based on zeus to dunfell. I've update all
> layers and in one of my recipes I'm seeing following issue (not see on
> zeus at all):
> WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png
> ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function
> in exec_python_func() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
>       0001:
>   *** 0002:sstate_report_unihash(d)
>       0003:
> File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass',
> lineno: 840, function: sstate_report_unihash
>       0836:    report_unihash = getattr(bb.parse.siggen, 'report_unihash', None)
>       0837:
>       0838:    if report_unihash:
>       0839:        ss = sstate_state_fromvars(d)
>   *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
>       0841:}
>       0842:
>       0843:#
>       0844:# Shell function to decompress and prepare a package for installation
> File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py',
> lineno: 555, function: report_unihash
>       0551:
>       0552:            if "." in self.method:
>       0553:                (module, method) = self.method.rsplit('.', 1)
>       0554:                locs['method'] =
> getattr(importlib.import_module(module), method)
>   *** 0555:                outhash = bb.utils.better_eval('method(path,
> sigfile, task, d)', locs)
>       0556:            else:
>       0557:                outhash = bb.utils.better_eval(self.method +
> '(path, sigfile, task, d)', locs)
>       0558:
>       0559:            try:
> File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py',
> lineno: 420, function: better_eval
>       0416:    if extraglobals:
>       0417:        ctx = copy.copy(ctx)
>       0418:        for g in extraglobals:
>       0419:            ctx[g] = extraglobals[g]
>   *** 0420:    return eval(source, ctx, locals)
>       0421:
>       0422:@contextmanager
>       0423:def fileslocked(files):
>       0424:    """Context manager for locking and unlocking file locks."""
> File: '<string>', lineno: 1, function: <module>
>    File "<string>", line 1, in <module>
>
> File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
> lineno: 595, function: OEOuthashBasic
>       0591:            process(root)
>       0592:            for f in files:
>       0593:                if f == 'fixmepath':
>       0594:                    continue
>   *** 0595:                process(os.path.join(root, f))
>       0596:    finally:
>       0597:        os.chdir(prev_dir)
>       0598:
>       0599:    return h.hexdigest()
> File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
> lineno: 551, function: process
>       0547:                    add_perm(stat.S_IXOTH, 'x')
>       0548:
>       0549:                if include_owners:
>       0550:                    try:
>   *** 0551:                        update_hash(" %10s" %
> pwd.getpwuid(s.st_uid).pw_name)
>       0552:                        update_hash(" %10s" %
> grp.getgrgid(s.st_gid).gr_name)
>       0553:                    except KeyError:
>       0554:                        bb.warn("KeyError in %s" % path)
>       0555:                        raise
> Exception: KeyError: 'getpwuid(): uid not found: 1000'
>
> ERROR: Logfile of failure stored in:
> /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454
>
> Is this known issue or something related to my host setup?

This means the recipe is having host contamination (where the UID of the 
user doing the build is leaking into the file system).

>
> Thanks and BR,
>
> marek
>
>
> 
>

[-- Attachment #2: Type: text/html, Size: 4452 bytes --]

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Marek Belisko]

On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com> wrote:
>
>
> On 11/16/20 2:38 PM, Marek Belisko wrote:
>
> Hi,
>
> I'm bumping my project based on zeus to dunfell. I've update all
> layers and in one of my recipes I'm seeing following issue (not see on
> zeus at all):
> WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png
> ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function
> in exec_python_func() autogenerated:
>
> The stack trace of python calls that resulted in this exception/failure was:
> File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
>      0001:
>  *** 0002:sstate_report_unihash(d)
>      0003:
> File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass',
> lineno: 840, function: sstate_report_unihash
>      0836:    report_unihash = getattr(bb.parse.siggen, 'report_unihash', None)
>      0837:
>      0838:    if report_unihash:
>      0839:        ss = sstate_state_fromvars(d)
>  *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
>      0841:}
>      0842:
>      0843:#
>      0844:# Shell function to decompress and prepare a package for installation
> File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py',
> lineno: 555, function: report_unihash
>      0551:
>      0552:            if "." in self.method:
>      0553:                (module, method) = self.method.rsplit('.', 1)
>      0554:                locs['method'] =
> getattr(importlib.import_module(module), method)
>  *** 0555:                outhash = bb.utils.better_eval('method(path,
> sigfile, task, d)', locs)
>      0556:            else:
>      0557:                outhash = bb.utils.better_eval(self.method +
> '(path, sigfile, task, d)', locs)
>      0558:
>      0559:            try:
> File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py',
> lineno: 420, function: better_eval
>      0416:    if extraglobals:
>      0417:        ctx = copy.copy(ctx)
>      0418:        for g in extraglobals:
>      0419:            ctx[g] = extraglobals[g]
>  *** 0420:    return eval(source, ctx, locals)
>      0421:
>      0422:@contextmanager
>      0423:def fileslocked(files):
>      0424:    """Context manager for locking and unlocking file locks."""
> File: '<string>', lineno: 1, function: <module>
>   File "<string>", line 1, in <module>
>
> File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
> lineno: 595, function: OEOuthashBasic
>      0591:            process(root)
>      0592:            for f in files:
>      0593:                if f == 'fixmepath':
>      0594:                    continue
>  *** 0595:                process(os.path.join(root, f))
>      0596:    finally:
>      0597:        os.chdir(prev_dir)
>      0598:
>      0599:    return h.hexdigest()
> File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
> lineno: 551, function: process
>      0547:                    add_perm(stat.S_IXOTH, 'x')
>      0548:
>      0549:                if include_owners:
>      0550:                    try:
>  *** 0551:                        update_hash(" %10s" %
> pwd.getpwuid(s.st_uid).pw_name)
>      0552:                        update_hash(" %10s" %
> grp.getgrgid(s.st_gid).gr_name)
>      0553:                    except KeyError:
>      0554:                        bb.warn("KeyError in %s" % path)
>      0555:                        raise
> Exception: KeyError: 'getpwuid(): uid not found: 1000'
>
> ERROR: Logfile of failure stored in:
> /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454
>
> Is this known issue or something related to my host setup?
>
> This means the recipe is having host contamination (where the UID of the user doing the build is leaking into the file system).
Some ideas what can be the source of that. This recipe is basically
just copying files from archive to final rootfs without any special
processing or so.
>
> Thanks and BR,
>
> marek
>
>
> 
>

BR,

marek

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 4900 bytes --]

https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442

is an example where it triggers this error, but doesn't trigger the more
common host-user-contaminated QA error (unless you happened to use UID 1001
on host for the user running bitbake).

Similarly when the files are "installed" with e.g. "cp -a" for which layers
usually use "cp -R --no-dereference --preserve=mode,links -v".

On Mon, Nov 16, 2020 at 10:10 PM Marek Belisko <marek.belisko@gmail.com>
wrote:

> On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com> wrote:
> >
> >
> > On 11/16/20 2:38 PM, Marek Belisko wrote:
> >
> > Hi,
> >
> > I'm bumping my project based on zeus to dunfell. I've update all
> > layers and in one of my recipes I'm seeing following issue (not see on
> > zeus at all):
> > WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png
> > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function
> > in exec_python_func() autogenerated:
> >
> > The stack trace of python calls that resulted in this exception/failure
> was:
> > File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
> >      0001:
> >  *** 0002:sstate_report_unihash(d)
> >      0003:
> > File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass',
> > lineno: 840, function: sstate_report_unihash
> >      0836:    report_unihash = getattr(bb.parse.siggen,
> 'report_unihash', None)
> >      0837:
> >      0838:    if report_unihash:
> >      0839:        ss = sstate_state_fromvars(d)
> >  *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
> >      0841:}
> >      0842:
> >      0843:#
> >      0844:# Shell function to decompress and prepare a package for
> installation
> > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py',
> > lineno: 555, function: report_unihash
> >      0551:
> >      0552:            if "." in self.method:
> >      0553:                (module, method) = self.method.rsplit('.', 1)
> >      0554:                locs['method'] =
> > getattr(importlib.import_module(module), method)
> >  *** 0555:                outhash = bb.utils.better_eval('method(path,
> > sigfile, task, d)', locs)
> >      0556:            else:
> >      0557:                outhash = bb.utils.better_eval(self.method +
> > '(path, sigfile, task, d)', locs)
> >      0558:
> >      0559:            try:
> > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py',
> > lineno: 420, function: better_eval
> >      0416:    if extraglobals:
> >      0417:        ctx = copy.copy(ctx)
> >      0418:        for g in extraglobals:
> >      0419:            ctx[g] = extraglobals[g]
> >  *** 0420:    return eval(source, ctx, locals)
> >      0421:
> >      0422:@contextmanager
> >      0423:def fileslocked(files):
> >      0424:    """Context manager for locking and unlocking file locks."""
> > File: '<string>', lineno: 1, function: <module>
> >   File "<string>", line 1, in <module>
> >
> > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
> > lineno: 595, function: OEOuthashBasic
> >      0591:            process(root)
> >      0592:            for f in files:
> >      0593:                if f == 'fixmepath':
> >      0594:                    continue
> >  *** 0595:                process(os.path.join(root, f))
> >      0596:    finally:
> >      0597:        os.chdir(prev_dir)
> >      0598:
> >      0599:    return h.hexdigest()
> > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py',
> > lineno: 551, function: process
> >      0547:                    add_perm(stat.S_IXOTH, 'x')
> >      0548:
> >      0549:                if include_owners:
> >      0550:                    try:
> >  *** 0551:                        update_hash(" %10s" %
> > pwd.getpwuid(s.st_uid).pw_name)
> >      0552:                        update_hash(" %10s" %
> > grp.getgrgid(s.st_gid).gr_name)
> >      0553:                    except KeyError:
> >      0554:                        bb.warn("KeyError in %s" % path)
> >      0555:                        raise
> > Exception: KeyError: 'getpwuid(): uid not found: 1000'
> >
> > ERROR: Logfile of failure stored in:
> >
> /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454
> >
> > Is this known issue or something related to my host setup?
> >
> > This means the recipe is having host contamination (where the UID of the
> user doing the build is leaking into the file system).
> Some ideas what can be the source of that. This recipe is basically
> just copying files from archive to final rootfs without any special
> processing or so.
> >
> > Thanks and BR,
> >
> > marek
> >
> >
> >
> >
>
> BR,
>
> marek
>
> 
>
>

[-- Attachment #2: Type: text/html, Size: 6524 bytes --]

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Nicolas Jeker]

On Mon, 2020-11-16 at 23:27 +0100, Martin Jansa wrote:
> https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> 
> is an example where it triggers this error, but doesn't trigger the
> more common host-user-contaminated QA error (unless you happened to
> use UID 1001 on host for the user running bitbake).
> 
> Similarly when the files are "installed" with e.g. "cp -a" for which
> layers usually use "cp -R --no-dereference --preserve=mode,links -v".

On a related note, the do_install reference in the manual gives some
advice on how to use cp and tar without contaminating the build.

The relevant part:

Safe methods for installing files include the following:
 * The install utility. This utility is the preferred method.
 * The cp command with the "--no-preserve=ownership" option.
 * The tar command with the "--no-same-owner" option. See the
   bin_package.bbclass file in the meta/classes directory of the
   Source Directory for an example.

https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-tasks-install

> On Mon, Nov 16, 2020 at 10:10 PM Marek Belisko <
> marek.belisko@gmail.com> wrote:
> > On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com>
> > wrote:
> > > 
> > > 
> > > On 11/16/20 2:38 PM, Marek Belisko wrote:
> > > 
> > > Hi,
> > > 
> > > I'm bumping my project based on zeus to dunfell. I've update all
> > > layers and in one of my recipes I'm seeing following issue (not
> > see on
> > > zeus at all):
> > > WARNING: cv-my-test-1.0-r0 do_package: KeyError in
> > ./package/srv/10%.png
> > > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python
> > function
> > > in exec_python_func() autogenerated:
> > > 
> > > The stack trace of python calls that resulted in this
> > exception/failure was:
> > > File: 'exec_python_func() autogenerated', lineno: 2, function:
> > <module>
> > >       0001:
> > >   *** 0002:sstate_report_unihash(d)
> > >       0003:
> > > File: '/home/ubuntu/projects/my-test-
> > /poky/meta/classes/sstate.bbclass',
> > > lineno: 840, function: sstate_report_unihash
> > >       0836:    report_unihash = getattr(bb.parse.siggen,
> > 'report_unihash', None)
> > >       0837:
> > >       0838:    if report_unihash:
> > >       0839:        ss = sstate_state_fromvars(d)
> > >   *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
> > >       0841:}
> > >       0842:
> > >       0843:#
> > >       0844:# Shell function to decompress and prepare a package
> > for installation
> > > File: '/home/ubuntu/projects/my-test-
> > /poky/bitbake/lib/bb/siggen.py',
> > > lineno: 555, function: report_unihash
> > >       0551:
> > >       0552:            if "." in self.method:
> > >       0553:                (module, method) =
> > self.method.rsplit('.', 1)
> > >       0554:                locs['method'] =
> > > getattr(importlib.import_module(module), method)
> > >   *** 0555:                outhash =
> > bb.utils.better_eval('method(path,
> > > sigfile, task, d)', locs)
> > >       0556:            else:
> > >       0557:                outhash =
> > bb.utils.better_eval(self.method +
> > > '(path, sigfile, task, d)', locs)
> > >       0558:
> > >       0559:            try:
> > > File: '/home/ubuntu/projects/my-test-
> > /poky/bitbake/lib/bb/utils.py',
> > > lineno: 420, function: better_eval
> > >       0416:    if extraglobals:
> > >       0417:        ctx = copy.copy(ctx)
> > >       0418:        for g in extraglobals:
> > >       0419:            ctx[g] = extraglobals[g]
> > >   *** 0420:    return eval(source, ctx, locals)
> > >       0421:
> > >       0422:@contextmanager
> > >       0423:def fileslocked(files):
> > >       0424:    """Context manager for locking and unlocking file
> > locks."""
> > > File: '<string>', lineno: 1, function: <module>
> > >    File "<string>", line 1, in <module>
> > > 
> > > File: '/home/ubuntu/projects/my-test-
> > /poky/meta/lib/oe/sstatesig.py',
> > > lineno: 595, function: OEOuthashBasic
> > >       0591:            process(root)
> > >       0592:            for f in files:
> > >       0593:                if f == 'fixmepath':
> > >       0594:                    continue
> > >   *** 0595:                process(os.path.join(root, f))
> > >       0596:    finally:
> > >       0597:        os.chdir(prev_dir)
> > >       0598:
> > >       0599:    return h.hexdigest()
> > > File: '/home/ubuntu/projects/my-test-
> > /poky/meta/lib/oe/sstatesig.py',
> > > lineno: 551, function: process
> > >       0547:                    add_perm(stat.S_IXOTH, 'x')
> > >       0548:
> > >       0549:                if include_owners:
> > >       0550:                    try:
> > >   *** 0551:                        update_hash(" %10s" %
> > > pwd.getpwuid(s.st_uid).pw_name)
> > >       0552:                        update_hash(" %10s" %
> > > grp.getgrgid(s.st_gid).gr_name)
> > >       0553:                    except KeyError:
> > >       0554:                        bb.warn("KeyError in %s" %
> > path)
> > >       0555:                        raise
> > > Exception: KeyError: 'getpwuid(): uid not found: 1000'
> > > 
> > > ERROR: Logfile of failure stored in:
> > > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-
> > linux/cv-my-test/1.0-r0/temp/log.do_package.27454
> > > 
> > > Is this known issue or something related to my host setup?
> > > 
> > > This means the recipe is having host contamination (where the UID
> > of the user doing the build is leaking into the file system).
> > Some ideas what can be the source of that. This recipe is basically
> > just copying files from archive to final rootfs without any special
> > processing or so.
> > > 
> > > Thanks and BR,
> > > 
> > > marek
> > > 
> > > 
> > > 
> > > 
> > 
> > BR,
> > 
> > marek
> > 
> > 
> > 
> 

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Marek Belisko]

On Tue, Nov 17, 2020 at 2:19 PM Nicolas Jeker <n.jeker@delisys.ch> wrote:
>
> On Mon, 2020-11-16 at 23:27 +0100, Martin Jansa wrote:
> > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> >
> > is an example where it triggers this error, but doesn't trigger the
> > more common host-user-contaminated QA error (unless you happened to
> > use UID 1001 on host for the user running bitbake).
> >
> > Similarly when the files are "installed" with e.g. "cp -a" for which
> > layers usually use "cp -R --no-dereference --preserve=mode,links -v".
>
> On a related note, the do_install reference in the manual gives some
> advice on how to use cp and tar without contaminating the build.
>
> The relevant part:
Thanks all for valuable inputs. Issue solved +1
>
> Safe methods for installing files include the following:
>  * The install utility. This utility is the preferred method.
>  * The cp command with the "--no-preserve=ownership" option.
>  * The tar command with the "--no-same-owner" option. See the
>    bin_package.bbclass file in the meta/classes directory of the
>    Source Directory for an example.
>
> https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-tasks-install
>
> > On Mon, Nov 16, 2020 at 10:10 PM Marek Belisko <
> > marek.belisko@gmail.com> wrote:
> > > On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com>
> > > wrote:
> > > >
> > > >
> > > > On 11/16/20 2:38 PM, Marek Belisko wrote:
> > > >
> > > > Hi,
> > > >
> > > > I'm bumping my project based on zeus to dunfell. I've update all
> > > > layers and in one of my recipes I'm seeing following issue (not
> > > see on
> > > > zeus at all):
> > > > WARNING: cv-my-test-1.0-r0 do_package: KeyError in
> > > ./package/srv/10%.png
> > > > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python
> > > function
> > > > in exec_python_func() autogenerated:
> > > >
> > > > The stack trace of python calls that resulted in this
> > > exception/failure was:
> > > > File: 'exec_python_func() autogenerated', lineno: 2, function:
> > > <module>
> > > >       0001:
> > > >   *** 0002:sstate_report_unihash(d)
> > > >       0003:
> > > > File: '/home/ubuntu/projects/my-test-
> > > /poky/meta/classes/sstate.bbclass',
> > > > lineno: 840, function: sstate_report_unihash
> > > >       0836:    report_unihash = getattr(bb.parse.siggen,
> > > 'report_unihash', None)
> > > >       0837:
> > > >       0838:    if report_unihash:
> > > >       0839:        ss = sstate_state_fromvars(d)
> > > >   *** 0840:        report_unihash(os.getcwd(), ss['task'], d)
> > > >       0841:}
> > > >       0842:
> > > >       0843:#
> > > >       0844:# Shell function to decompress and prepare a package
> > > for installation
> > > > File: '/home/ubuntu/projects/my-test-
> > > /poky/bitbake/lib/bb/siggen.py',
> > > > lineno: 555, function: report_unihash
> > > >       0551:
> > > >       0552:            if "." in self.method:
> > > >       0553:                (module, method) =
> > > self.method.rsplit('.', 1)
> > > >       0554:                locs['method'] =
> > > > getattr(importlib.import_module(module), method)
> > > >   *** 0555:                outhash =
> > > bb.utils.better_eval('method(path,
> > > > sigfile, task, d)', locs)
> > > >       0556:            else:
> > > >       0557:                outhash =
> > > bb.utils.better_eval(self.method +
> > > > '(path, sigfile, task, d)', locs)
> > > >       0558:
> > > >       0559:            try:
> > > > File: '/home/ubuntu/projects/my-test-
> > > /poky/bitbake/lib/bb/utils.py',
> > > > lineno: 420, function: better_eval
> > > >       0416:    if extraglobals:
> > > >       0417:        ctx = copy.copy(ctx)
> > > >       0418:        for g in extraglobals:
> > > >       0419:            ctx[g] = extraglobals[g]
> > > >   *** 0420:    return eval(source, ctx, locals)
> > > >       0421:
> > > >       0422:@contextmanager
> > > >       0423:def fileslocked(files):
> > > >       0424:    """Context manager for locking and unlocking file
> > > locks."""
> > > > File: '<string>', lineno: 1, function: <module>
> > > >    File "<string>", line 1, in <module>
> > > >
> > > > File: '/home/ubuntu/projects/my-test-
> > > /poky/meta/lib/oe/sstatesig.py',
> > > > lineno: 595, function: OEOuthashBasic
> > > >       0591:            process(root)
> > > >       0592:            for f in files:
> > > >       0593:                if f == 'fixmepath':
> > > >       0594:                    continue
> > > >   *** 0595:                process(os.path.join(root, f))
> > > >       0596:    finally:
> > > >       0597:        os.chdir(prev_dir)
> > > >       0598:
> > > >       0599:    return h.hexdigest()
> > > > File: '/home/ubuntu/projects/my-test-
> > > /poky/meta/lib/oe/sstatesig.py',
> > > > lineno: 551, function: process
> > > >       0547:                    add_perm(stat.S_IXOTH, 'x')
> > > >       0548:
> > > >       0549:                if include_owners:
> > > >       0550:                    try:
> > > >   *** 0551:                        update_hash(" %10s" %
> > > > pwd.getpwuid(s.st_uid).pw_name)
> > > >       0552:                        update_hash(" %10s" %
> > > > grp.getgrgid(s.st_gid).gr_name)
> > > >       0553:                    except KeyError:
> > > >       0554:                        bb.warn("KeyError in %s" %
> > > path)
> > > >       0555:                        raise
> > > > Exception: KeyError: 'getpwuid(): uid not found: 1000'
> > > >
> > > > ERROR: Logfile of failure stored in:
> > > > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-
> > > linux/cv-my-test/1.0-r0/temp/log.do_package.27454
> > > >
> > > > Is this known issue or something related to my host setup?
> > > >
> > > > This means the recipe is having host contamination (where the UID
> > > of the user doing the build is leaking into the file system).
> > > Some ideas what can be the source of that. This recipe is basically
> > > just copying files from archive to final rootfs without any special
> > > processing or so.
> > > >
> > > > Thanks and BR,
> > > >
> > > > marek
> > > >
> > > >
> > > >
> > > >
> > >
> > > BR,
> > >
> > > marek
> > >
> > >
> > >
> >
>
>
>
> 
>

Re: KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Thomas Hill]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

Hi Martin!

On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:

> 
> https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> 
> 
> is an example where it triggers this error, but doesn't trigger the more
> common host-user-contaminated QA error (unless you happened to use UID
> 1001 on host for the user running bitbake).
> 

I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running?

Thanks!
Tom

[-- Attachment #2: Type: text/html, Size: 832 bytes --]

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1027 bytes --]

On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org
<tom.hill=inbox.lv@lists.yoctoproject.org> wrote:

> Hi Martin!
>
> On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:
>
>
> https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
>
> is an example where it triggers this error, but doesn't trigger the more
> common host-user-contaminated QA error (unless you happened to use UID 1001
> on host for the user running bitbake).
>
> I have here a similar problem with one of my own packages. It happens that
> my bitbake user uses UID 1001. Do you have more information why this is a
> problem? Should it be enough to change the UID to 1002 to get everything
> running?
>

No, you should chown the files to be owned by the expected user which
exists in the image (probably root like in my commit). Changing the UID of
the user on host is very bad work around (as it will fail for the next
person building the same image with host user 1001.

[-- Attachment #2: Type: text/html, Size: 1539 bytes --]

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Thomas Hill]

On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote:
> On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote:
> > Hi Martin!

> > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:
> > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442

> > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake).
>  I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running?

> No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001.

Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111 did not solve the issue.
I will open a new thread because I don't see why this fails. I use oe_runmake in my do_install function and got the impression that oe_runmake should take care of this via fakeroot.

Tom

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Richard Purdie]

On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote:
> On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote:
> > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote:
> > > Hi Martin!
> 
> > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:
> > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> 
> > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake).
> >  I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running?
> 
> > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001.
> 
> Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111
> did not solve the issue.
> I will open a new thread because I don't see why this fails. I use oe_runmake
> in my do_install function and got the impression that oe_runmake should take 
> care of this via fakeroot.

When you install files during do_install, you need to be clear about who
you want to own the end result.

If you do something like "touch ${D}/x", the it will be owned by the default
user which in a fakeroot context under pseudo is root.

If however you cp a file to ${D}/x, it would depend what you told cp
to do about ownership. If the original file was owned by user 1001, it
may try and preserve that.

We don't know what your code is doing in do_install but its almost certainly
not setting the file ownership correctly.

Cheers,

Richard

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Thomas Hill]

Hi Richard!

On Fri, 7 May 2021, 15:28, Richard Purdie <richard.purdie@linuxfoundation.org>
> On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote:
> > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote:
> > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote:
> > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:
> > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake).
> > > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running?
> >
> > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001.
> >
> > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111
> > did not solve the issue.
> > I will open a new thread because I don't see why this fails. I use oe_runmake
> > in my do_install function and got the impression that oe_runmake should take 
> > care of this via fakeroot.

> When you install files during do_install, you need to be clear about who
> you want to own the end result.

See my other mail for more details - subject:
"Path ./package/usr/lib/libcryptopp.so.8 is owned by uid 1111, gid 1111, which doesn't match any ..."

> If you do something like "touch ${D}/x", the it will be owned by the default
> user which in a fakeroot context under pseudo is root.
> 
> If however you cp a file to ${D}/x, it would depend what you told cp
> to do about ownership. If the original file was owned by user 1001, it
> may try and preserve that.

I do not touch any files myself. The do_install function uses only
"oe_runmake install-lib". No "touch", no "cp", nothing.
The GNUmakefile uses "mkdir", "cp", "chmod" but I patched it so it
uses "install" instead of "cp" in my recipe.

> We don't know what your code is doing in do_install but its almost certainly
> not setting the file ownership correctly.

My other mail has more details. I did not append the GNUmakefile. It is
quite large. I think it ist easier to get it directly from the original git-repository.
<https://github.com/weidai11/cryptopp/blob/9dcc26c58213abb8351fbb1b2a7a1d2c667366e4/GNUmakefile>

Tom

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 3121 bytes --]

On Mon, May 10, 2021 at 12:08:22PM +0300, Thomas Hill via lists.yoctoproject.org wrote:
> Hi Richard!
> 
> On Fri, 7 May 2021, 15:28, Richard Purdie <richard.purdie@linuxfoundation.org>
> > On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote:
> > > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote:
> > > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote:
> > > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:
> > > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> > > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake).
> > > > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running?
> > >
> > > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001.
> > >
> > > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111
> > > did not solve the issue.
> > > I will open a new thread because I don't see why this fails. I use oe_runmake
> > > in my do_install function and got the impression that oe_runmake should take 
> > > care of this via fakeroot.
> 
> > When you install files during do_install, you need to be clear about who
> > you want to own the end result.
> 
> See my other mail for more details - subject:
> "Path ./package/usr/lib/libcryptopp.so.8 is owned by uid 1111, gid 1111, which doesn't match any ..."
> 
> > If you do something like "touch ${D}/x", the it will be owned by the default
> > user which in a fakeroot context under pseudo is root.
> > 
> > If however you cp a file to ${D}/x, it would depend what you told cp
> > to do about ownership. If the original file was owned by user 1001, it
> > may try and preserve that.
> 
> I do not touch any files myself. The do_install function uses only
> "oe_runmake install-lib". No "touch", no "cp", nothing.
> The GNUmakefile uses "mkdir", "cp", "chmod" but I patched it so it
> uses "install" instead of "cp" in my recipe.
> 
> > We don't know what your code is doing in do_install but its almost certainly
> > not setting the file ownership correctly.
> 
> My other mail has more details. I did not append the GNUmakefile. It is
> quite large. I think it ist easier to get it directly from the original git-repository.
> <https://github.com/weidai11/cryptopp/blob/9dcc26c58213abb8351fbb1b2a7a1d2c667366e4/GNUmakefile>

0002_libcryptopp_8.2.0_use-install-instead-of-cp.patch from your recipe
might be interesting as well to guess what went wrong in your case.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 201 bytes --]

Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase

[Thomas Hill]

[-- Attachment #1: Type: text/plain, Size: 3394 bytes --]

Hi Martin!

On Mon, 10 May 2021, 12:25, Martin Jansa <martin.jansa@gmail.com> wrote:
> On Mon, May 10, 2021 at 12:08:22PM +0300, Thomas Hill via lists.yoctoproject.org wrote:
> > On Fri, 7 May 2021, 15:28, Richard Purdie <richard.purdie@linuxfoundation.org>
> > > On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote:
> > > > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote:
> > > > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote:
> > > > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote:
> > > > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442
> > > > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake).
> > > > > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running?
> > > >
> > > > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001.
> > > >
> > > > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111
> > > > did not solve the issue.
> > > > I will open a new thread because I don't see why this fails. I use oe_runmake
> > > > in my do_install function and got the impression that oe_runmake should take 
> > > > care of this via fakeroot.
> >
> > > When you install files during do_install, you need to be clear about who
> > > you want to own the end result.
> >
> > See my other mail for more details - subject:
> > "Path ./package/usr/lib/libcryptopp.so.8 is owned by uid 1111, gid 1111, which doesn't match any ..."
> >
> > > If you do something like "touch ${D}/x", the it will be owned by the default
> > > user which in a fakeroot context under pseudo is root.
> > >
> > > If however you cp a file to ${D}/x, it would depend what you told cp
> > > to do about ownership. If the original file was owned by user 1001, it
> > > may try and preserve that.
> >
> > I do not touch any files myself. The do_install function uses only
> > "oe_runmake install-lib". No "touch", no "cp", nothing.
> > The GNUmakefile uses "mkdir", "cp", "chmod" but I patched it so it
> > uses "install" instead of "cp" in my recipe.
> >
> > > We don't know what your code is doing in do_install but its almost certainly
> > > not setting the file ownership correctly.
> >
> > My other mail has more details. I did not append the GNUmakefile. It is
> > quite large. I think it ist easier to get it directly from the original git-repository.
> > <https://github.com/weidai11/cryptopp/blob/9dcc26c58213abb8351fbb1b2a7a1d2c667366e4/GNUmakefile>

> 0002_libcryptopp_8.2.0_use-install-instead-of-cp.patch from your recipe
> might be interesting as well to guess what went wrong in your case.

Nothing fancy is going on. I only replace "cp" with "install" - see appended file.

Thanks for your time!

Tom


[-- Attachment #2: 0002_libcryptopp_8.2.0_use-install-instead-of-cp.patch --]
[-- Type: application/octet-stream, Size: 303 bytes --]

diff -uprN a/GNUmakefile b/GNUmakefile
--- a/GNUmakefile	2021-04-29 13:24:26.260327764 +0200
+++ b/GNUmakefile	2021-04-29 13:47:28.769504438 +0200
@@ -19,7 +19,7 @@ AR ?= ar
 ARFLAGS ?= -cr # ar needs the dash on OpenBSD
 RANLIB ?= ranlib
 
-CP ?= cp
+CP ?= install
 MV ?= mv
 RM ?= rm -f
 GREP ?= grep