* KeyError: 'getpwuid(): uid not found: 1000' in do_package phase @ 2020-11-16 20:38 Marek Belisko 2020-11-16 20:52 ` [yocto] " Joshua Watt 0 siblings, 1 reply; 13+ messages in thread From: Marek Belisko @ 2020-11-16 20:38 UTC (permalink / raw) To: yocto Hi, I'm bumping my project based on zeus to dunfell. I've update all layers and in one of my recipes I'm seeing following issue (not see on zeus at all): WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function in exec_python_func() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_python_func() autogenerated', lineno: 2, function: <module> 0001: *** 0002:sstate_report_unihash(d) 0003: File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass', lineno: 840, function: sstate_report_unihash 0836: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) 0837: 0838: if report_unihash: 0839: ss = sstate_state_fromvars(d) *** 0840: report_unihash(os.getcwd(), ss['task'], d) 0841:} 0842: 0843:# 0844:# Shell function to decompress and prepare a package for installation File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py', lineno: 555, function: report_unihash 0551: 0552: if "." in self.method: 0553: (module, method) = self.method.rsplit('.', 1) 0554: locs['method'] = getattr(importlib.import_module(module), method) *** 0555: outhash = bb.utils.better_eval('method(path, sigfile, task, d)', locs) 0556: else: 0557: outhash = bb.utils.better_eval(self.method + '(path, sigfile, task, d)', locs) 0558: 0559: try: File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py', lineno: 420, function: better_eval 0416: if extraglobals: 0417: ctx = copy.copy(ctx) 0418: for g in extraglobals: 0419: ctx[g] = extraglobals[g] *** 0420: return eval(source, ctx, locals) 0421: 0422:@contextmanager 0423:def fileslocked(files): 0424: """Context manager for locking and unlocking file locks.""" File: '<string>', lineno: 1, function: <module> File "<string>", line 1, in <module> File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', lineno: 595, function: OEOuthashBasic 0591: process(root) 0592: for f in files: 0593: if f == 'fixmepath': 0594: continue *** 0595: process(os.path.join(root, f)) 0596: finally: 0597: os.chdir(prev_dir) 0598: 0599: return h.hexdigest() File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', lineno: 551, function: process 0547: add_perm(stat.S_IXOTH, 'x') 0548: 0549: if include_owners: 0550: try: *** 0551: update_hash(" %10s" % pwd.getpwuid(s.st_uid).pw_name) 0552: update_hash(" %10s" % grp.getgrgid(s.st_gid).gr_name) 0553: except KeyError: 0554: bb.warn("KeyError in %s" % path) 0555: raise Exception: KeyError: 'getpwuid(): uid not found: 1000' ERROR: Logfile of failure stored in: /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454 Is this known issue or something related to my host setup? Thanks and BR, marek -- as simple and primitive as possible ------------------------------------------------- Marek Belisko - OPEN-NANDRA Freelance Developer Ruska Nova Ves 219 | Presov, 08005 Slovak Republic Tel: +421 915 052 184 skype: marekwhite twitter: #opennandra web: http://open-nandra.com ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2020-11-16 20:38 KeyError: 'getpwuid(): uid not found: 1000' in do_package phase Marek Belisko @ 2020-11-16 20:52 ` Joshua Watt 2020-11-16 21:10 ` Marek Belisko 0 siblings, 1 reply; 13+ messages in thread From: Joshua Watt @ 2020-11-16 20:52 UTC (permalink / raw) To: Marek Belisko, yocto [-- Attachment #1: Type: text/plain, Size: 3939 bytes --] On 11/16/20 2:38 PM, Marek Belisko wrote: > Hi, > > I'm bumping my project based on zeus to dunfell. I've update all > layers and in one of my recipes I'm seeing following issue (not see on > zeus at all): > WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function > in exec_python_func() autogenerated: > > The stack trace of python calls that resulted in this exception/failure was: > File: 'exec_python_func() autogenerated', lineno: 2, function: <module> > 0001: > *** 0002:sstate_report_unihash(d) > 0003: > File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass', > lineno: 840, function: sstate_report_unihash > 0836: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) > 0837: > 0838: if report_unihash: > 0839: ss = sstate_state_fromvars(d) > *** 0840: report_unihash(os.getcwd(), ss['task'], d) > 0841:} > 0842: > 0843:# > 0844:# Shell function to decompress and prepare a package for installation > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py', > lineno: 555, function: report_unihash > 0551: > 0552: if "." in self.method: > 0553: (module, method) = self.method.rsplit('.', 1) > 0554: locs['method'] = > getattr(importlib.import_module(module), method) > *** 0555: outhash = bb.utils.better_eval('method(path, > sigfile, task, d)', locs) > 0556: else: > 0557: outhash = bb.utils.better_eval(self.method + > '(path, sigfile, task, d)', locs) > 0558: > 0559: try: > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py', > lineno: 420, function: better_eval > 0416: if extraglobals: > 0417: ctx = copy.copy(ctx) > 0418: for g in extraglobals: > 0419: ctx[g] = extraglobals[g] > *** 0420: return eval(source, ctx, locals) > 0421: > 0422:@contextmanager > 0423:def fileslocked(files): > 0424: """Context manager for locking and unlocking file locks.""" > File: '<string>', lineno: 1, function: <module> > File "<string>", line 1, in <module> > > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', > lineno: 595, function: OEOuthashBasic > 0591: process(root) > 0592: for f in files: > 0593: if f == 'fixmepath': > 0594: continue > *** 0595: process(os.path.join(root, f)) > 0596: finally: > 0597: os.chdir(prev_dir) > 0598: > 0599: return h.hexdigest() > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', > lineno: 551, function: process > 0547: add_perm(stat.S_IXOTH, 'x') > 0548: > 0549: if include_owners: > 0550: try: > *** 0551: update_hash(" %10s" % > pwd.getpwuid(s.st_uid).pw_name) > 0552: update_hash(" %10s" % > grp.getgrgid(s.st_gid).gr_name) > 0553: except KeyError: > 0554: bb.warn("KeyError in %s" % path) > 0555: raise > Exception: KeyError: 'getpwuid(): uid not found: 1000' > > ERROR: Logfile of failure stored in: > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454 > > Is this known issue or something related to my host setup? This means the recipe is having host contamination (where the UID of the user doing the build is leaking into the file system). > > Thanks and BR, > > marek > > > > [-- Attachment #2: Type: text/html, Size: 4452 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2020-11-16 20:52 ` [yocto] " Joshua Watt @ 2020-11-16 21:10 ` Marek Belisko 2020-11-16 22:27 ` Martin Jansa 0 siblings, 1 reply; 13+ messages in thread From: Marek Belisko @ 2020-11-16 21:10 UTC (permalink / raw) To: Joshua Watt; +Cc: yocto On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com> wrote: > > > On 11/16/20 2:38 PM, Marek Belisko wrote: > > Hi, > > I'm bumping my project based on zeus to dunfell. I've update all > layers and in one of my recipes I'm seeing following issue (not see on > zeus at all): > WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function > in exec_python_func() autogenerated: > > The stack trace of python calls that resulted in this exception/failure was: > File: 'exec_python_func() autogenerated', lineno: 2, function: <module> > 0001: > *** 0002:sstate_report_unihash(d) > 0003: > File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass', > lineno: 840, function: sstate_report_unihash > 0836: report_unihash = getattr(bb.parse.siggen, 'report_unihash', None) > 0837: > 0838: if report_unihash: > 0839: ss = sstate_state_fromvars(d) > *** 0840: report_unihash(os.getcwd(), ss['task'], d) > 0841:} > 0842: > 0843:# > 0844:# Shell function to decompress and prepare a package for installation > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py', > lineno: 555, function: report_unihash > 0551: > 0552: if "." in self.method: > 0553: (module, method) = self.method.rsplit('.', 1) > 0554: locs['method'] = > getattr(importlib.import_module(module), method) > *** 0555: outhash = bb.utils.better_eval('method(path, > sigfile, task, d)', locs) > 0556: else: > 0557: outhash = bb.utils.better_eval(self.method + > '(path, sigfile, task, d)', locs) > 0558: > 0559: try: > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py', > lineno: 420, function: better_eval > 0416: if extraglobals: > 0417: ctx = copy.copy(ctx) > 0418: for g in extraglobals: > 0419: ctx[g] = extraglobals[g] > *** 0420: return eval(source, ctx, locals) > 0421: > 0422:@contextmanager > 0423:def fileslocked(files): > 0424: """Context manager for locking and unlocking file locks.""" > File: '<string>', lineno: 1, function: <module> > File "<string>", line 1, in <module> > > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', > lineno: 595, function: OEOuthashBasic > 0591: process(root) > 0592: for f in files: > 0593: if f == 'fixmepath': > 0594: continue > *** 0595: process(os.path.join(root, f)) > 0596: finally: > 0597: os.chdir(prev_dir) > 0598: > 0599: return h.hexdigest() > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', > lineno: 551, function: process > 0547: add_perm(stat.S_IXOTH, 'x') > 0548: > 0549: if include_owners: > 0550: try: > *** 0551: update_hash(" %10s" % > pwd.getpwuid(s.st_uid).pw_name) > 0552: update_hash(" %10s" % > grp.getgrgid(s.st_gid).gr_name) > 0553: except KeyError: > 0554: bb.warn("KeyError in %s" % path) > 0555: raise > Exception: KeyError: 'getpwuid(): uid not found: 1000' > > ERROR: Logfile of failure stored in: > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454 > > Is this known issue or something related to my host setup? > > This means the recipe is having host contamination (where the UID of the user doing the build is leaking into the file system). Some ideas what can be the source of that. This recipe is basically just copying files from archive to final rootfs without any special processing or so. > > Thanks and BR, > > marek > > > > BR, marek ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2020-11-16 21:10 ` Marek Belisko @ 2020-11-16 22:27 ` Martin Jansa 2020-11-17 13:18 ` Nicolas Jeker 2021-05-06 8:57 ` Thomas Hill 0 siblings, 2 replies; 13+ messages in thread From: Martin Jansa @ 2020-11-16 22:27 UTC (permalink / raw) To: Marek Belisko; +Cc: Joshua Watt, yocto [-- Attachment #1: Type: text/plain, Size: 4900 bytes --] https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake). Similarly when the files are "installed" with e.g. "cp -a" for which layers usually use "cp -R --no-dereference --preserve=mode,links -v". On Mon, Nov 16, 2020 at 10:10 PM Marek Belisko <marek.belisko@gmail.com> wrote: > On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com> wrote: > > > > > > On 11/16/20 2:38 PM, Marek Belisko wrote: > > > > Hi, > > > > I'm bumping my project based on zeus to dunfell. I've update all > > layers and in one of my recipes I'm seeing following issue (not see on > > zeus at all): > > WARNING: cv-my-test-1.0-r0 do_package: KeyError in ./package/srv/10%.png > > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python function > > in exec_python_func() autogenerated: > > > > The stack trace of python calls that resulted in this exception/failure > was: > > File: 'exec_python_func() autogenerated', lineno: 2, function: <module> > > 0001: > > *** 0002:sstate_report_unihash(d) > > 0003: > > File: '/home/ubuntu/projects/my-test-/poky/meta/classes/sstate.bbclass', > > lineno: 840, function: sstate_report_unihash > > 0836: report_unihash = getattr(bb.parse.siggen, > 'report_unihash', None) > > 0837: > > 0838: if report_unihash: > > 0839: ss = sstate_state_fromvars(d) > > *** 0840: report_unihash(os.getcwd(), ss['task'], d) > > 0841:} > > 0842: > > 0843:# > > 0844:# Shell function to decompress and prepare a package for > installation > > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/siggen.py', > > lineno: 555, function: report_unihash > > 0551: > > 0552: if "." in self.method: > > 0553: (module, method) = self.method.rsplit('.', 1) > > 0554: locs['method'] = > > getattr(importlib.import_module(module), method) > > *** 0555: outhash = bb.utils.better_eval('method(path, > > sigfile, task, d)', locs) > > 0556: else: > > 0557: outhash = bb.utils.better_eval(self.method + > > '(path, sigfile, task, d)', locs) > > 0558: > > 0559: try: > > File: '/home/ubuntu/projects/my-test-/poky/bitbake/lib/bb/utils.py', > > lineno: 420, function: better_eval > > 0416: if extraglobals: > > 0417: ctx = copy.copy(ctx) > > 0418: for g in extraglobals: > > 0419: ctx[g] = extraglobals[g] > > *** 0420: return eval(source, ctx, locals) > > 0421: > > 0422:@contextmanager > > 0423:def fileslocked(files): > > 0424: """Context manager for locking and unlocking file locks.""" > > File: '<string>', lineno: 1, function: <module> > > File "<string>", line 1, in <module> > > > > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', > > lineno: 595, function: OEOuthashBasic > > 0591: process(root) > > 0592: for f in files: > > 0593: if f == 'fixmepath': > > 0594: continue > > *** 0595: process(os.path.join(root, f)) > > 0596: finally: > > 0597: os.chdir(prev_dir) > > 0598: > > 0599: return h.hexdigest() > > File: '/home/ubuntu/projects/my-test-/poky/meta/lib/oe/sstatesig.py', > > lineno: 551, function: process > > 0547: add_perm(stat.S_IXOTH, 'x') > > 0548: > > 0549: if include_owners: > > 0550: try: > > *** 0551: update_hash(" %10s" % > > pwd.getpwuid(s.st_uid).pw_name) > > 0552: update_hash(" %10s" % > > grp.getgrgid(s.st_gid).gr_name) > > 0553: except KeyError: > > 0554: bb.warn("KeyError in %s" % path) > > 0555: raise > > Exception: KeyError: 'getpwuid(): uid not found: 1000' > > > > ERROR: Logfile of failure stored in: > > > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky-linux/cv-my-test/1.0-r0/temp/log.do_package.27454 > > > > Is this known issue or something related to my host setup? > > > > This means the recipe is having host contamination (where the UID of the > user doing the build is leaking into the file system). > Some ideas what can be the source of that. This recipe is basically > just copying files from archive to final rootfs without any special > processing or so. > > > > Thanks and BR, > > > > marek > > > > > > > > > > BR, > > marek > > > > [-- Attachment #2: Type: text/html, Size: 6524 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2020-11-16 22:27 ` Martin Jansa @ 2020-11-17 13:18 ` Nicolas Jeker 2020-11-17 18:53 ` Marek Belisko 2021-05-06 8:57 ` Thomas Hill 1 sibling, 1 reply; 13+ messages in thread From: Nicolas Jeker @ 2020-11-17 13:18 UTC (permalink / raw) To: Martin Jansa; +Cc: yocto On Mon, 2020-11-16 at 23:27 +0100, Martin Jansa wrote: > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > is an example where it triggers this error, but doesn't trigger the > more common host-user-contaminated QA error (unless you happened to > use UID 1001 on host for the user running bitbake). > > Similarly when the files are "installed" with e.g. "cp -a" for which > layers usually use "cp -R --no-dereference --preserve=mode,links -v". On a related note, the do_install reference in the manual gives some advice on how to use cp and tar without contaminating the build. The relevant part: Safe methods for installing files include the following: * The install utility. This utility is the preferred method. * The cp command with the "--no-preserve=ownership" option. * The tar command with the "--no-same-owner" option. See the bin_package.bbclass file in the meta/classes directory of the Source Directory for an example. https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-tasks-install > On Mon, Nov 16, 2020 at 10:10 PM Marek Belisko < > marek.belisko@gmail.com> wrote: > > On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com> > > wrote: > > > > > > > > > On 11/16/20 2:38 PM, Marek Belisko wrote: > > > > > > Hi, > > > > > > I'm bumping my project based on zeus to dunfell. I've update all > > > layers and in one of my recipes I'm seeing following issue (not > > see on > > > zeus at all): > > > WARNING: cv-my-test-1.0-r0 do_package: KeyError in > > ./package/srv/10%.png > > > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python > > function > > > in exec_python_func() autogenerated: > > > > > > The stack trace of python calls that resulted in this > > exception/failure was: > > > File: 'exec_python_func() autogenerated', lineno: 2, function: > > <module> > > > 0001: > > > *** 0002:sstate_report_unihash(d) > > > 0003: > > > File: '/home/ubuntu/projects/my-test- > > /poky/meta/classes/sstate.bbclass', > > > lineno: 840, function: sstate_report_unihash > > > 0836: report_unihash = getattr(bb.parse.siggen, > > 'report_unihash', None) > > > 0837: > > > 0838: if report_unihash: > > > 0839: ss = sstate_state_fromvars(d) > > > *** 0840: report_unihash(os.getcwd(), ss['task'], d) > > > 0841:} > > > 0842: > > > 0843:# > > > 0844:# Shell function to decompress and prepare a package > > for installation > > > File: '/home/ubuntu/projects/my-test- > > /poky/bitbake/lib/bb/siggen.py', > > > lineno: 555, function: report_unihash > > > 0551: > > > 0552: if "." in self.method: > > > 0553: (module, method) = > > self.method.rsplit('.', 1) > > > 0554: locs['method'] = > > > getattr(importlib.import_module(module), method) > > > *** 0555: outhash = > > bb.utils.better_eval('method(path, > > > sigfile, task, d)', locs) > > > 0556: else: > > > 0557: outhash = > > bb.utils.better_eval(self.method + > > > '(path, sigfile, task, d)', locs) > > > 0558: > > > 0559: try: > > > File: '/home/ubuntu/projects/my-test- > > /poky/bitbake/lib/bb/utils.py', > > > lineno: 420, function: better_eval > > > 0416: if extraglobals: > > > 0417: ctx = copy.copy(ctx) > > > 0418: for g in extraglobals: > > > 0419: ctx[g] = extraglobals[g] > > > *** 0420: return eval(source, ctx, locals) > > > 0421: > > > 0422:@contextmanager > > > 0423:def fileslocked(files): > > > 0424: """Context manager for locking and unlocking file > > locks.""" > > > File: '<string>', lineno: 1, function: <module> > > > File "<string>", line 1, in <module> > > > > > > File: '/home/ubuntu/projects/my-test- > > /poky/meta/lib/oe/sstatesig.py', > > > lineno: 595, function: OEOuthashBasic > > > 0591: process(root) > > > 0592: for f in files: > > > 0593: if f == 'fixmepath': > > > 0594: continue > > > *** 0595: process(os.path.join(root, f)) > > > 0596: finally: > > > 0597: os.chdir(prev_dir) > > > 0598: > > > 0599: return h.hexdigest() > > > File: '/home/ubuntu/projects/my-test- > > /poky/meta/lib/oe/sstatesig.py', > > > lineno: 551, function: process > > > 0547: add_perm(stat.S_IXOTH, 'x') > > > 0548: > > > 0549: if include_owners: > > > 0550: try: > > > *** 0551: update_hash(" %10s" % > > > pwd.getpwuid(s.st_uid).pw_name) > > > 0552: update_hash(" %10s" % > > > grp.getgrgid(s.st_gid).gr_name) > > > 0553: except KeyError: > > > 0554: bb.warn("KeyError in %s" % > > path) > > > 0555: raise > > > Exception: KeyError: 'getpwuid(): uid not found: 1000' > > > > > > ERROR: Logfile of failure stored in: > > > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky- > > linux/cv-my-test/1.0-r0/temp/log.do_package.27454 > > > > > > Is this known issue or something related to my host setup? > > > > > > This means the recipe is having host contamination (where the UID > > of the user doing the build is leaking into the file system). > > Some ideas what can be the source of that. This recipe is basically > > just copying files from archive to final rootfs without any special > > processing or so. > > > > > > Thanks and BR, > > > > > > marek > > > > > > > > > > > > > > > > BR, > > > > marek > > > > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2020-11-17 13:18 ` Nicolas Jeker @ 2020-11-17 18:53 ` Marek Belisko 0 siblings, 0 replies; 13+ messages in thread From: Marek Belisko @ 2020-11-17 18:53 UTC (permalink / raw) To: Nicolas Jeker; +Cc: Martin Jansa, yocto On Tue, Nov 17, 2020 at 2:19 PM Nicolas Jeker <n.jeker@delisys.ch> wrote: > > On Mon, 2020-11-16 at 23:27 +0100, Martin Jansa wrote: > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > > is an example where it triggers this error, but doesn't trigger the > > more common host-user-contaminated QA error (unless you happened to > > use UID 1001 on host for the user running bitbake). > > > > Similarly when the files are "installed" with e.g. "cp -a" for which > > layers usually use "cp -R --no-dereference --preserve=mode,links -v". > > On a related note, the do_install reference in the manual gives some > advice on how to use cp and tar without contaminating the build. > > The relevant part: Thanks all for valuable inputs. Issue solved +1 > > Safe methods for installing files include the following: > * The install utility. This utility is the preferred method. > * The cp command with the "--no-preserve=ownership" option. > * The tar command with the "--no-same-owner" option. See the > bin_package.bbclass file in the meta/classes directory of the > Source Directory for an example. > > https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#ref-tasks-install > > > On Mon, Nov 16, 2020 at 10:10 PM Marek Belisko < > > marek.belisko@gmail.com> wrote: > > > On Mon, Nov 16, 2020 at 9:52 PM Joshua Watt <jpewhacker@gmail.com> > > > wrote: > > > > > > > > > > > > On 11/16/20 2:38 PM, Marek Belisko wrote: > > > > > > > > Hi, > > > > > > > > I'm bumping my project based on zeus to dunfell. I've update all > > > > layers and in one of my recipes I'm seeing following issue (not > > > see on > > > > zeus at all): > > > > WARNING: cv-my-test-1.0-r0 do_package: KeyError in > > > ./package/srv/10%.png > > > > ERROR: cv-my-test-1.0-r0 do_package: Error executing a python > > > function > > > > in exec_python_func() autogenerated: > > > > > > > > The stack trace of python calls that resulted in this > > > exception/failure was: > > > > File: 'exec_python_func() autogenerated', lineno: 2, function: > > > <module> > > > > 0001: > > > > *** 0002:sstate_report_unihash(d) > > > > 0003: > > > > File: '/home/ubuntu/projects/my-test- > > > /poky/meta/classes/sstate.bbclass', > > > > lineno: 840, function: sstate_report_unihash > > > > 0836: report_unihash = getattr(bb.parse.siggen, > > > 'report_unihash', None) > > > > 0837: > > > > 0838: if report_unihash: > > > > 0839: ss = sstate_state_fromvars(d) > > > > *** 0840: report_unihash(os.getcwd(), ss['task'], d) > > > > 0841:} > > > > 0842: > > > > 0843:# > > > > 0844:# Shell function to decompress and prepare a package > > > for installation > > > > File: '/home/ubuntu/projects/my-test- > > > /poky/bitbake/lib/bb/siggen.py', > > > > lineno: 555, function: report_unihash > > > > 0551: > > > > 0552: if "." in self.method: > > > > 0553: (module, method) = > > > self.method.rsplit('.', 1) > > > > 0554: locs['method'] = > > > > getattr(importlib.import_module(module), method) > > > > *** 0555: outhash = > > > bb.utils.better_eval('method(path, > > > > sigfile, task, d)', locs) > > > > 0556: else: > > > > 0557: outhash = > > > bb.utils.better_eval(self.method + > > > > '(path, sigfile, task, d)', locs) > > > > 0558: > > > > 0559: try: > > > > File: '/home/ubuntu/projects/my-test- > > > /poky/bitbake/lib/bb/utils.py', > > > > lineno: 420, function: better_eval > > > > 0416: if extraglobals: > > > > 0417: ctx = copy.copy(ctx) > > > > 0418: for g in extraglobals: > > > > 0419: ctx[g] = extraglobals[g] > > > > *** 0420: return eval(source, ctx, locals) > > > > 0421: > > > > 0422:@contextmanager > > > > 0423:def fileslocked(files): > > > > 0424: """Context manager for locking and unlocking file > > > locks.""" > > > > File: '<string>', lineno: 1, function: <module> > > > > File "<string>", line 1, in <module> > > > > > > > > File: '/home/ubuntu/projects/my-test- > > > /poky/meta/lib/oe/sstatesig.py', > > > > lineno: 595, function: OEOuthashBasic > > > > 0591: process(root) > > > > 0592: for f in files: > > > > 0593: if f == 'fixmepath': > > > > 0594: continue > > > > *** 0595: process(os.path.join(root, f)) > > > > 0596: finally: > > > > 0597: os.chdir(prev_dir) > > > > 0598: > > > > 0599: return h.hexdigest() > > > > File: '/home/ubuntu/projects/my-test- > > > /poky/meta/lib/oe/sstatesig.py', > > > > lineno: 551, function: process > > > > 0547: add_perm(stat.S_IXOTH, 'x') > > > > 0548: > > > > 0549: if include_owners: > > > > 0550: try: > > > > *** 0551: update_hash(" %10s" % > > > > pwd.getpwuid(s.st_uid).pw_name) > > > > 0552: update_hash(" %10s" % > > > > grp.getgrgid(s.st_gid).gr_name) > > > > 0553: except KeyError: > > > > 0554: bb.warn("KeyError in %s" % > > > path) > > > > 0555: raise > > > > Exception: KeyError: 'getpwuid(): uid not found: 1000' > > > > > > > > ERROR: Logfile of failure stored in: > > > > /home/ubuntu/projects/my-test-/build/tmp/work/aarch64-poky- > > > linux/cv-my-test/1.0-r0/temp/log.do_package.27454 > > > > > > > > Is this known issue or something related to my host setup? > > > > > > > > This means the recipe is having host contamination (where the UID > > > of the user doing the build is leaking into the file system). > > > Some ideas what can be the source of that. This recipe is basically > > > just copying files from archive to final rootfs without any special > > > processing or so. > > > > > > > > Thanks and BR, > > > > > > > > marek > > > > > > > > > > > > > > > > > > > > > > BR, > > > > > > marek > > > > > > > > > > > > > > > > ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2020-11-16 22:27 ` Martin Jansa 2020-11-17 13:18 ` Nicolas Jeker @ 2021-05-06 8:57 ` Thomas Hill 2021-05-06 10:44 ` [yocto] " Martin Jansa 1 sibling, 1 reply; 13+ messages in thread From: Thomas Hill @ 2021-05-06 8:57 UTC (permalink / raw) To: yocto [-- Attachment #1: Type: text/plain, Size: 634 bytes --] Hi Martin! On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > is an example where it triggers this error, but doesn't trigger the more > common host-user-contaminated QA error (unless you happened to use UID > 1001 on host for the user running bitbake). > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running? Thanks! Tom [-- Attachment #2: Type: text/html, Size: 832 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2021-05-06 8:57 ` Thomas Hill @ 2021-05-06 10:44 ` Martin Jansa 2021-05-07 7:10 ` Thomas Hill 0 siblings, 1 reply; 13+ messages in thread From: Martin Jansa @ 2021-05-06 10:44 UTC (permalink / raw) To: tom.hill; +Cc: Yocto-mailing-list [-- Attachment #1: Type: text/plain, Size: 1027 bytes --] On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote: > Hi Martin! > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > is an example where it triggers this error, but doesn't trigger the more > common host-user-contaminated QA error (unless you happened to use UID 1001 > on host for the user running bitbake). > > I have here a similar problem with one of my own packages. It happens that > my bitbake user uses UID 1001. Do you have more information why this is a > problem? Should it be enough to change the UID to 1002 to get everything > running? > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001. [-- Attachment #2: Type: text/html, Size: 1539 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2021-05-06 10:44 ` [yocto] " Martin Jansa @ 2021-05-07 7:10 ` Thomas Hill 2021-05-07 12:28 ` Richard Purdie 0 siblings, 1 reply; 13+ messages in thread From: Thomas Hill @ 2021-05-07 7:10 UTC (permalink / raw) To: Yocto-mailing-list On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote: > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote: > > Hi Martin! > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake). > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running? > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001. Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111 did not solve the issue. I will open a new thread because I don't see why this fails. I use oe_runmake in my do_install function and got the impression that oe_runmake should take care of this via fakeroot. Tom ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2021-05-07 7:10 ` Thomas Hill @ 2021-05-07 12:28 ` Richard Purdie 2021-05-10 9:08 ` Thomas Hill 0 siblings, 1 reply; 13+ messages in thread From: Richard Purdie @ 2021-05-07 12:28 UTC (permalink / raw) To: tom.hill, Yocto-mailing-list On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote: > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote: > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote: > > > Hi Martin! > > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake). > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running? > > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001. > > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111 > did not solve the issue. > I will open a new thread because I don't see why this fails. I use oe_runmake > in my do_install function and got the impression that oe_runmake should take > care of this via fakeroot. When you install files during do_install, you need to be clear about who you want to own the end result. If you do something like "touch ${D}/x", the it will be owned by the default user which in a fakeroot context under pseudo is root. If however you cp a file to ${D}/x, it would depend what you told cp to do about ownership. If the original file was owned by user 1001, it may try and preserve that. We don't know what your code is doing in do_install but its almost certainly not setting the file ownership correctly. Cheers, Richard ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2021-05-07 12:28 ` Richard Purdie @ 2021-05-10 9:08 ` Thomas Hill 2021-05-10 9:25 ` Martin Jansa 0 siblings, 1 reply; 13+ messages in thread From: Thomas Hill @ 2021-05-10 9:08 UTC (permalink / raw) To: Yocto-mailing-list Hi Richard! On Fri, 7 May 2021, 15:28, Richard Purdie <richard.purdie@linuxfoundation.org> > On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote: > > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote: > > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote: > > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake). > > > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running? > > > > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001. > > > > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111 > > did not solve the issue. > > I will open a new thread because I don't see why this fails. I use oe_runmake > > in my do_install function and got the impression that oe_runmake should take > > care of this via fakeroot. > When you install files during do_install, you need to be clear about who > you want to own the end result. See my other mail for more details - subject: "Path ./package/usr/lib/libcryptopp.so.8 is owned by uid 1111, gid 1111, which doesn't match any ..." > If you do something like "touch ${D}/x", the it will be owned by the default > user which in a fakeroot context under pseudo is root. > > If however you cp a file to ${D}/x, it would depend what you told cp > to do about ownership. If the original file was owned by user 1001, it > may try and preserve that. I do not touch any files myself. The do_install function uses only "oe_runmake install-lib". No "touch", no "cp", nothing. The GNUmakefile uses "mkdir", "cp", "chmod" but I patched it so it uses "install" instead of "cp" in my recipe. > We don't know what your code is doing in do_install but its almost certainly > not setting the file ownership correctly. My other mail has more details. I did not append the GNUmakefile. It is quite large. I think it ist easier to get it directly from the original git-repository. <https://github.com/weidai11/cryptopp/blob/9dcc26c58213abb8351fbb1b2a7a1d2c667366e4/GNUmakefile> Tom ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2021-05-10 9:08 ` Thomas Hill @ 2021-05-10 9:25 ` Martin Jansa 2021-05-11 7:21 ` Thomas Hill 0 siblings, 1 reply; 13+ messages in thread From: Martin Jansa @ 2021-05-10 9:25 UTC (permalink / raw) To: tom.hill; +Cc: Yocto-mailing-list [-- Attachment #1: Type: text/plain, Size: 3121 bytes --] On Mon, May 10, 2021 at 12:08:22PM +0300, Thomas Hill via lists.yoctoproject.org wrote: > Hi Richard! > > On Fri, 7 May 2021, 15:28, Richard Purdie <richard.purdie@linuxfoundation.org> > > On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote: > > > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote: > > > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote: > > > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake). > > > > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running? > > > > > > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001. > > > > > > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111 > > > did not solve the issue. > > > I will open a new thread because I don't see why this fails. I use oe_runmake > > > in my do_install function and got the impression that oe_runmake should take > > > care of this via fakeroot. > > > When you install files during do_install, you need to be clear about who > > you want to own the end result. > > See my other mail for more details - subject: > "Path ./package/usr/lib/libcryptopp.so.8 is owned by uid 1111, gid 1111, which doesn't match any ..." > > > If you do something like "touch ${D}/x", the it will be owned by the default > > user which in a fakeroot context under pseudo is root. > > > > If however you cp a file to ${D}/x, it would depend what you told cp > > to do about ownership. If the original file was owned by user 1001, it > > may try and preserve that. > > I do not touch any files myself. The do_install function uses only > "oe_runmake install-lib". No "touch", no "cp", nothing. > The GNUmakefile uses "mkdir", "cp", "chmod" but I patched it so it > uses "install" instead of "cp" in my recipe. > > > We don't know what your code is doing in do_install but its almost certainly > > not setting the file ownership correctly. > > My other mail has more details. I did not append the GNUmakefile. It is > quite large. I think it ist easier to get it directly from the original git-repository. > <https://github.com/weidai11/cryptopp/blob/9dcc26c58213abb8351fbb1b2a7a1d2c667366e4/GNUmakefile> 0002_libcryptopp_8.2.0_use-install-instead-of-cp.patch from your recipe might be interesting as well to guess what went wrong in your case. [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 201 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [yocto] KeyError: 'getpwuid(): uid not found: 1000' in do_package phase 2021-05-10 9:25 ` Martin Jansa @ 2021-05-11 7:21 ` Thomas Hill 0 siblings, 0 replies; 13+ messages in thread From: Thomas Hill @ 2021-05-11 7:21 UTC (permalink / raw) To: Yocto-mailing-list [-- Attachment #1: Type: text/plain, Size: 3394 bytes --] Hi Martin! On Mon, 10 May 2021, 12:25, Martin Jansa <martin.jansa@gmail.com> wrote: > On Mon, May 10, 2021 at 12:08:22PM +0300, Thomas Hill via lists.yoctoproject.org wrote: > > On Fri, 7 May 2021, 15:28, Richard Purdie <richard.purdie@linuxfoundation.org> > > > On Fri, 2021-05-07 at 10:10 +0300, Thomas Hill via lists.yoctoproject.org wrote: > > > > On Thu, 6 May 2021, 13:44 Martin Jansa <Martin.Jansa@gmail.com> wrote: > > > > > On Thu, May 6, 2021 at 10:57 AM Thomas Hill via lists.yoctoproject.org <tom.hill=inbox.lv@lists.yoctoproject.org> wrote: > > > > > > > On Mon, Nov 16, 2020 at 02:28 PM, Martin Jansa wrote: > > > > > > > https://github.com/webOS-ports/meta-webos-ports/commit/9fd17a67cdbed92df13a14b002a189b4c6c2d442 > > > > > > > is an example where it triggers this error, but doesn't trigger the more common host-user-contaminated QA error (unless you happened to use UID 1001 on host for the user running bitbake). > > > > > > I have here a similar problem with one of my own packages. It happens that my bitbake user uses UID 1001. Do you have more information why this is a problem? Should it be enough to change the UID to 1002 to get everything running? > > > > > > > > > No, you should chown the files to be owned by the expected user which exists in the image (probably root like in my commit). Changing the UID of the user on host is very bad work around (as it will fail for the next person building the same image with host user 1001. > > > > > > > > Ok. Thanks. I can confirm that the change of the bitbake users UID to 1111 > > > > did not solve the issue. > > > > I will open a new thread because I don't see why this fails. I use oe_runmake > > > > in my do_install function and got the impression that oe_runmake should take > > > > care of this via fakeroot. > > > > > When you install files during do_install, you need to be clear about who > > > you want to own the end result. > > > > See my other mail for more details - subject: > > "Path ./package/usr/lib/libcryptopp.so.8 is owned by uid 1111, gid 1111, which doesn't match any ..." > > > > > If you do something like "touch ${D}/x", the it will be owned by the default > > > user which in a fakeroot context under pseudo is root. > > > > > > If however you cp a file to ${D}/x, it would depend what you told cp > > > to do about ownership. If the original file was owned by user 1001, it > > > may try and preserve that. > > > > I do not touch any files myself. The do_install function uses only > > "oe_runmake install-lib". No "touch", no "cp", nothing. > > The GNUmakefile uses "mkdir", "cp", "chmod" but I patched it so it > > uses "install" instead of "cp" in my recipe. > > > > > We don't know what your code is doing in do_install but its almost certainly > > > not setting the file ownership correctly. > > > > My other mail has more details. I did not append the GNUmakefile. It is > > quite large. I think it ist easier to get it directly from the original git-repository. > > <https://github.com/weidai11/cryptopp/blob/9dcc26c58213abb8351fbb1b2a7a1d2c667366e4/GNUmakefile> > 0002_libcryptopp_8.2.0_use-install-instead-of-cp.patch from your recipe > might be interesting as well to guess what went wrong in your case. Nothing fancy is going on. I only replace "cp" with "install" - see appended file. Thanks for your time! Tom [-- Attachment #2: 0002_libcryptopp_8.2.0_use-install-instead-of-cp.patch --] [-- Type: application/octet-stream, Size: 303 bytes --] diff -uprN a/GNUmakefile b/GNUmakefile --- a/GNUmakefile 2021-04-29 13:24:26.260327764 +0200 +++ b/GNUmakefile 2021-04-29 13:47:28.769504438 +0200 @@ -19,7 +19,7 @@ AR ?= ar ARFLAGS ?= -cr # ar needs the dash on OpenBSD RANLIB ?= ranlib -CP ?= cp +CP ?= install MV ?= mv RM ?= rm -f GREP ?= grep ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2021-05-11 7:21 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-11-16 20:38 KeyError: 'getpwuid(): uid not found: 1000' in do_package phase Marek Belisko 2020-11-16 20:52 ` [yocto] " Joshua Watt 2020-11-16 21:10 ` Marek Belisko 2020-11-16 22:27 ` Martin Jansa 2020-11-17 13:18 ` Nicolas Jeker 2020-11-17 18:53 ` Marek Belisko 2021-05-06 8:57 ` Thomas Hill 2021-05-06 10:44 ` [yocto] " Martin Jansa 2021-05-07 7:10 ` Thomas Hill 2021-05-07 12:28 ` Richard Purdie 2021-05-10 9:08 ` Thomas Hill 2021-05-10 9:25 ` Martin Jansa 2021-05-11 7:21 ` Thomas Hill
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.