* Fix some NPM related problems
@ 2022-05-13 13:48 Zoltan Boszormenyi
2022-05-13 13:48 ` [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing Zoltan Boszormenyi
2022-05-13 13:48 ` [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name Zoltan Boszormenyi
0 siblings, 2 replies; 8+ messages in thread
From: Zoltan Boszormenyi @ 2022-05-13 13:48 UTC (permalink / raw)
To: openembedded-core; +Cc: Jean-Marie LEMETAYER
When trying to create recipes for Angular so that "ng build"
can be run, I ran into some problems. Two of them I was able fix.
The first two problems are in the same class and boil down to
the fact that some node module archives are created by an idiotic
tar client (probably on Windows) and contain wrong permissions.
Problem 1) Directories have no "x" permission bit
This cannot be worked around as tar creates the directory as is
but creating the first file in the directory will fail.
tar exits with an error and there is no option for it to
override directory permissions.
license-webpack-plugin 2.3.1, a dependency for Angular 11 is
such a contender.
Problem 2) package.json have no "w" permission bit
This can be worked around with
[PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing
Problem 3) Recipes with "inherit npm" install the /usr/lib/node symlink
I needed to create a complete set of Angular recipes where "ng build"
can run. But when trying to use them in DEPENDS for an Angular based
project, this error kicked in from staging.bbclass:
bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname]))
This is fixed by ignoring the sole /usr/lib/node symlink in staging.bbclass
when checking for identical files with
[PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name
I discovered this issue in Yocto 3.3.
I cc-ed relevant parties who edited either bbclass around where
I just modified them.
Best regards,
Zoltán Böszörményi
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing 2022-05-13 13:48 Fix some NPM related problems Zoltan Boszormenyi @ 2022-05-13 13:48 ` Zoltan Boszormenyi 2022-05-13 13:52 ` [OE-core] " Alexander Kanavin 2022-05-13 13:48 ` [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name Zoltan Boszormenyi 1 sibling, 1 reply; 8+ messages in thread From: Zoltan Boszormenyi @ 2022-05-13 13:48 UTC (permalink / raw) To: openembedded-core Cc: Jean-Marie LEMETAYER, Zoltán Böszörményi From: Zoltán Böszörményi <zboszor@gmail.com> Some node module archives in npmjs.org contain wrong permissions. I found a case with package.json in the archive being r-xr-xr-x for which open(..., "w") fails. Set the manifest file permissions to 0666 just in case. Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com> --- meta/classes/npm.bbclass | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass index ba50fcac20..7f52ec6061 100644 --- a/meta/classes/npm.bbclass +++ b/meta/classes/npm.bbclass @@ -202,6 +202,7 @@ python npm_do_configure() { if has_shrinkwrap_file: _update_manifest("devDependencies") + os.chmod(cached_manifest_file, 0o666) with open(cached_manifest_file, "w") as f: json.dump(cached_manifest, f, indent=2) -- 2.36.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [OE-core] [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing 2022-05-13 13:48 ` [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing Zoltan Boszormenyi @ 2022-05-13 13:52 ` Alexander Kanavin [not found] ` <b47fd784-6d1d-4c89-a61a-d517210ac41a@gmail.com> 0 siblings, 1 reply; 8+ messages in thread From: Alexander Kanavin @ 2022-05-13 13:52 UTC (permalink / raw) To: Böszörményi Zoltán Cc: OE-core, Jean-Marie LEMETAYER, Zoltán Böszörményi Apologies, 666 is not right. You shouldn't grant rights to 'others'. Besides, 6 is 110, which is rw-, no? Alex On Fri, 13 May 2022 at 15:49, Zoltan Boszormenyi via lists.openembedded.org <zboszor=pr.hu@lists.openembedded.org> wrote: > > From: Zoltán Böszörményi <zboszor@gmail.com> > > Some node module archives in npmjs.org contain wrong permissions. > I found a case with package.json in the archive being r-xr-xr-x > for which open(..., "w") fails. Set the manifest file permissions > to 0666 just in case. > > Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com> > --- > meta/classes/npm.bbclass | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass > index ba50fcac20..7f52ec6061 100644 > --- a/meta/classes/npm.bbclass > +++ b/meta/classes/npm.bbclass > @@ -202,6 +202,7 @@ python npm_do_configure() { > if has_shrinkwrap_file: > _update_manifest("devDependencies") > > + os.chmod(cached_manifest_file, 0o666) > with open(cached_manifest_file, "w") as f: > json.dump(cached_manifest, f, indent=2) > > -- > 2.36.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#165565): https://lists.openembedded.org/g/openembedded-core/message/165565 > Mute This Topic: https://lists.openembedded.org/mt/91080992/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > ^ permalink raw reply [flat|nested] 8+ messages in thread
[parent not found: <b47fd784-6d1d-4c89-a61a-d517210ac41a@gmail.com>]
* Re: [OE-core] [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing [not found] ` <b47fd784-6d1d-4c89-a61a-d517210ac41a@gmail.com> @ 2022-05-13 14:00 ` Alexander Kanavin 0 siblings, 0 replies; 8+ messages in thread From: Alexander Kanavin @ 2022-05-13 14:00 UTC (permalink / raw) To: Böszörményi Zoltán Cc: Böszörményi Zoltán, OE-core, Jean-Marie LEMETAYER I think you should just add the needed permissions, not re-set them fully: u+rw or similar. Alex On Fri, 13 May 2022 at 15:58, Böszörményi Zoltán <zboszor@gmail.com> wrote: > > 2022. 05. 13. 15:52 keltezéssel, Alexander Kanavin írta: > > Apologies, 666 is not right. You shouldn't grant rights to 'others'. > > Besides, 6 is 110, which is rw-, no? > > Reading the file is also important so the r and w permissions > should be there. > > But 664 or 644 is definitely better security-wise. > > I will re-send a v2. > > > > > Alex > > > > On Fri, 13 May 2022 at 15:49, Zoltan Boszormenyi via > > lists.openembedded.org <zboszor=pr.hu@lists.openembedded.org> wrote: > >> From: Zoltán Böszörményi <zboszor@gmail.com> > >> > >> Some node module archives in npmjs.org contain wrong permissions. > >> I found a case with package.json in the archive being r-xr-xr-x > >> for which open(..., "w") fails. Set the manifest file permissions > >> to 0666 just in case. > >> > >> Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com> > >> --- > >> meta/classes/npm.bbclass | 1 + > >> 1 file changed, 1 insertion(+) > >> > >> diff --git a/meta/classes/npm.bbclass b/meta/classes/npm.bbclass > >> index ba50fcac20..7f52ec6061 100644 > >> --- a/meta/classes/npm.bbclass > >> +++ b/meta/classes/npm.bbclass > >> @@ -202,6 +202,7 @@ python npm_do_configure() { > >> if has_shrinkwrap_file: > >> _update_manifest("devDependencies") > >> > >> + os.chmod(cached_manifest_file, 0o666) > >> with open(cached_manifest_file, "w") as f: > >> json.dump(cached_manifest, f, indent=2) > >> > >> -- > >> 2.36.1 > >> > >> > >> -=-=-=-=-=-=-=-=-=-=-=- > >> Links: You receive all messages sent to this group. > >> View/Reply Online (#165565): https://lists.openembedded.org/g/openembedded-core/message/165565 > >> Mute This Topic: https://lists.openembedded.org/mt/91080992/1686489 > >> Group Owner: openembedded-core+owner@lists.openembedded.org > >> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > >> -=-=-=-=-=-=-=-=-=-=-=- > >> > ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name 2022-05-13 13:48 Fix some NPM related problems Zoltan Boszormenyi 2022-05-13 13:48 ` [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing Zoltan Boszormenyi @ 2022-05-13 13:48 ` Zoltan Boszormenyi 2022-05-13 13:55 ` [OE-core] " Alexander Kanavin 1 sibling, 1 reply; 8+ messages in thread From: Zoltan Boszormenyi @ 2022-05-13 13:48 UTC (permalink / raw) To: openembedded-core Cc: Jean-Marie LEMETAYER, Zoltán Böszörményi From: Zoltán Böszörményi <zboszor@gmail.com> Recipes using "inherit npm" install the same /usr/lib/node symlink that points to node_modules. When a recipe DEPENDS on multiple npm based recipe, do_prepare_recipe_sysroot() fails. Ignore this symlink in a way that the set of ignored symlinks can be extended later if needed. Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com> --- meta/classes/staging.bbclass | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass index 9fc8f4f283..f7b6056219 100644 --- a/meta/classes/staging.bbclass +++ b/meta/classes/staging.bbclass @@ -528,6 +528,7 @@ python extend_recipe_sysroot() { with open(manifest, "r") as f: manifests[dep] = manifest + ignored_dests = [ ''.join([d.getVar('nonarch_libdir'), '/node' ]) ] for l in f: l = l.strip() if l.endswith("/fixmepath"): @@ -545,7 +546,10 @@ python extend_recipe_sysroot() { hashname = targetdir + dest if not hashname.endswith("/"): if hashname in fileset: - bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname])) + if os.path.islink(hashname) and dest in ignored_dests: + bb.note("The symlink '%s' %s wants to install is already installed by %s, ignoring" % (dest, c, fileset[hashname])) + else: + bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname])) else: fileset[hashname] = c -- 2.36.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [OE-core] [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name 2022-05-13 13:48 ` [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name Zoltan Boszormenyi @ 2022-05-13 13:55 ` Alexander Kanavin 2022-05-13 14:02 ` Zoltan Boszormenyi 0 siblings, 1 reply; 8+ messages in thread From: Alexander Kanavin @ 2022-05-13 13:55 UTC (permalink / raw) To: Böszörményi Zoltán Cc: OE-core, Jean-Marie LEMETAYER, Zoltán Böszörményi Apologies, but special-casing this in staging class is not right either. The problem should be addressed at the making of sysroot-destdir/ step from the npm class. Alex On Fri, 13 May 2022 at 15:49, Zoltan Boszormenyi via lists.openembedded.org <zboszor=pr.hu@lists.openembedded.org> wrote: > > From: Zoltán Böszörményi <zboszor@gmail.com> > > Recipes using "inherit npm" install the same /usr/lib/node symlink > that points to node_modules. When a recipe DEPENDS on multiple > npm based recipe, do_prepare_recipe_sysroot() fails. > > Ignore this symlink in a way that the set of ignored symlinks > can be extended later if needed. > > Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com> > --- > meta/classes/staging.bbclass | 6 +++++- > 1 file changed, 5 insertions(+), 1 deletion(-) > > diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass > index 9fc8f4f283..f7b6056219 100644 > --- a/meta/classes/staging.bbclass > +++ b/meta/classes/staging.bbclass > @@ -528,6 +528,7 @@ python extend_recipe_sysroot() { > > with open(manifest, "r") as f: > manifests[dep] = manifest > + ignored_dests = [ ''.join([d.getVar('nonarch_libdir'), '/node' ]) ] > for l in f: > l = l.strip() > if l.endswith("/fixmepath"): > @@ -545,7 +546,10 @@ python extend_recipe_sysroot() { > hashname = targetdir + dest > if not hashname.endswith("/"): > if hashname in fileset: > - bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname])) > + if os.path.islink(hashname) and dest in ignored_dests: > + bb.note("The symlink '%s' %s wants to install is already installed by %s, ignoring" % (dest, c, fileset[hashname])) > + else: > + bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname])) > else: > fileset[hashname] = c > > -- > 2.36.1 > > > -=-=-=-=-=-=-=-=-=-=-=- > Links: You receive all messages sent to this group. > View/Reply Online (#165566): https://lists.openembedded.org/g/openembedded-core/message/165566 > Mute This Topic: https://lists.openembedded.org/mt/91080993/1686489 > Group Owner: openembedded-core+owner@lists.openembedded.org > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com] > -=-=-=-=-=-=-=-=-=-=-=- > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [OE-core] [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name 2022-05-13 13:55 ` [OE-core] " Alexander Kanavin @ 2022-05-13 14:02 ` Zoltan Boszormenyi 2022-05-13 14:14 ` richard.purdie 0 siblings, 1 reply; 8+ messages in thread From: Zoltan Boszormenyi @ 2022-05-13 14:02 UTC (permalink / raw) To: openembedded-core 2022. 05. 13. 15:55 keltezéssel, Alexander Kanavin írta: > Apologies, but special-casing this in staging class is not right > either. The problem should be addressed at the making of > sysroot-destdir/ step from the npm class. npm.bbclass creates the symlink in sysroot-destdir for recipe A But then it's the job of staging.bbclass to populate a recipe B's recipe-sysroot and recipe-sysroot-native from recipe A. How can this be worked around from npm.bbclass? > > Alex > > > On Fri, 13 May 2022 at 15:49, Zoltan Boszormenyi via > lists.openembedded.org <zboszor=pr.hu@lists.openembedded.org> wrote: >> >> From: Zoltán Böszörményi <zboszor@gmail.com> >> >> Recipes using "inherit npm" install the same /usr/lib/node symlink >> that points to node_modules. When a recipe DEPENDS on multiple >> npm based recipe, do_prepare_recipe_sysroot() fails. >> >> Ignore this symlink in a way that the set of ignored symlinks >> can be extended later if needed. >> >> Signed-off-by: Zoltán Böszörményi <zboszor@gmail.com> >> --- >> meta/classes/staging.bbclass | 6 +++++- >> 1 file changed, 5 insertions(+), 1 deletion(-) >> >> diff --git a/meta/classes/staging.bbclass b/meta/classes/staging.bbclass >> index 9fc8f4f283..f7b6056219 100644 >> --- a/meta/classes/staging.bbclass >> +++ b/meta/classes/staging.bbclass >> @@ -528,6 +528,7 @@ python extend_recipe_sysroot() { >> >> with open(manifest, "r") as f: >> manifests[dep] = manifest >> + ignored_dests = [ ''.join([d.getVar('nonarch_libdir'), '/node' ]) ] >> for l in f: >> l = l.strip() >> if l.endswith("/fixmepath"): >> @@ -545,7 +546,10 @@ python extend_recipe_sysroot() { >> hashname = targetdir + dest >> if not hashname.endswith("/"): >> if hashname in fileset: >> - bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname])) >> + if os.path.islink(hashname) and dest in ignored_dests: >> + bb.note("The symlink '%s' %s wants to install is already installed by %s, ignoring" % (dest, c, fileset[hashname])) >> + else: >> + bb.fatal("The file %s is installed by both %s and %s, aborting" % (dest, c, fileset[hashname])) >> else: >> fileset[hashname] = c >> >> -- >> 2.36.1 >> >> >> >> >> >> >> >> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [OE-core] [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name 2022-05-13 14:02 ` Zoltan Boszormenyi @ 2022-05-13 14:14 ` richard.purdie 0 siblings, 0 replies; 8+ messages in thread From: richard.purdie @ 2022-05-13 14:14 UTC (permalink / raw) To: zboszor, openembedded-core On Fri, 2022-05-13 at 16:02 +0200, Zoltan Boszormenyi via lists.openembedded.org wrote: > 2022. 05. 13. 15:55 keltezéssel, Alexander Kanavin írta: > > Apologies, but special-casing this in staging class is not right > > either. The problem should be addressed at the making of > > sysroot-destdir/ step from the npm class. > > npm.bbclass creates the symlink in sysroot-destdir for recipe A > > But then it's the job of staging.bbclass to populate a recipe B's > recipe-sysroot and recipe-sysroot-native from recipe A. > > How can this be worked around from npm.bbclass? You could move the line from npm.bbclass into the node recipe so it is only created once? Cheers, Richard ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2022-05-13 14:14 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-05-13 13:48 Fix some NPM related problems Zoltan Boszormenyi
2022-05-13 13:48 ` [PATCH 1/2] npm.bbclass: Fix file permissions before opening it for writing Zoltan Boszormenyi
2022-05-13 13:52 ` [OE-core] " Alexander Kanavin
[not found] ` <b47fd784-6d1d-4c89-a61a-d517210ac41a@gmail.com>
2022-05-13 14:00 ` Alexander Kanavin
2022-05-13 13:48 ` [PATCH 2/2] staging.bbclass: Ignore installing multiple symlinks of the same name Zoltan Boszormenyi
2022-05-13 13:55 ` [OE-core] " Alexander Kanavin
2022-05-13 14:02 ` Zoltan Boszormenyi
2022-05-13 14:14 ` richard.purdie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.