* [PATCH] package_ipk.bbclass: print opkg output on error @ 2020-05-19 18:36 Konrad Weihmann 2020-05-19 22:02 ` [OE-core] " Richard Purdie 0 siblings, 1 reply; 10+ messages in thread From: Konrad Weihmann @ 2020-05-19 18:36 UTC (permalink / raw) To: openembedded-core; +Cc: Konrad Weihmann in case the opkg command fails in before no output of the tool itself was printed to assist the user with debugging the issue. Print all output of the tool by using CalledProcessError wrapper around the call Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> --- meta/classes/package_ipk.bbclass | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass index c008559e4a..5e7d552cc5 100644 --- a/meta/classes/package_ipk.bbclass +++ b/meta/classes/package_ipk.bbclass @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): conffiles.close() os.chdir(basedir) - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), - stderr=subprocess.STDOUT, - shell=True) + try: + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), + stderr=subprocess.STDOUT, + shell=True) + except subprocess.CalledProcessError as exp: + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) if d.getVar('IPK_SIGN_PACKAGES') == '1': ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) -- 2.20.1 ^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-19 18:36 [PATCH] package_ipk.bbclass: print opkg output on error Konrad Weihmann @ 2020-05-19 22:02 ` Richard Purdie 2020-05-19 22:13 ` Denys Dmytriyenko ` (2 more replies) 0 siblings, 3 replies; 10+ messages in thread From: Richard Purdie @ 2020-05-19 22:02 UTC (permalink / raw) To: Konrad Weihmann, openembedded-core On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: > in case the opkg command fails in before no output of the tool > itself was printed to assist the user with debugging the issue. > Print all output of the tool by using CalledProcessError wrapper around > the call > > Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> > --- > meta/classes/package_ipk.bbclass | 11 +++++++---- > 1 file changed, 7 insertions(+), 4 deletions(-) > > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass > index c008559e4a..5e7d552cc5 100644 > --- a/meta/classes/package_ipk.bbclass > +++ b/meta/classes/package_ipk.bbclass > @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): > conffiles.close() > > os.chdir(basedir) > - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > - stderr=subprocess.STDOUT, > - shell=True) > + try: > + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > + stderr=subprocess.STDOUT, > + shell=True) > + except subprocess.CalledProcessError as exp: > + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) > > if d.getVar('IPK_SIGN_PACKAGES') == '1': > ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) I get the feeling we have a problem with our subprocess calls and debugging but its hard to know what to do about it. We can patch each call site like this but it starts to make we wonder whether we need a general wrapper, or we're missing something in bitbake or elsewhere to help with things. There is this code: http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 which is meant to help but it clearly isn't helping/catching all cases. I think we are going to need to look at a wider/more general solution or at least have a plan for one. Cheers, Richard ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-19 22:02 ` [OE-core] " Richard Purdie @ 2020-05-19 22:13 ` Denys Dmytriyenko 2020-05-19 22:14 ` Khem Raj 2020-05-20 20:17 ` Konrad Weihmann 2 siblings, 0 replies; 10+ messages in thread From: Denys Dmytriyenko @ 2020-05-19 22:13 UTC (permalink / raw) To: Richard Purdie; +Cc: Konrad Weihmann, openembedded-core On Tue, May 19, 2020 at 11:02:46PM +0100, Richard Purdie wrote: > On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: > > in case the opkg command fails in before no output of the tool > > itself was printed to assist the user with debugging the issue. > > Print all output of the tool by using CalledProcessError wrapper around > > the call > > > > Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> > > --- > > meta/classes/package_ipk.bbclass | 11 +++++++---- > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass > > index c008559e4a..5e7d552cc5 100644 > > --- a/meta/classes/package_ipk.bbclass > > +++ b/meta/classes/package_ipk.bbclass > > @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): > > conffiles.close() > > > > os.chdir(basedir) > > - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > > - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > > - stderr=subprocess.STDOUT, > > - shell=True) > > + try: > > + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > > + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > > + stderr=subprocess.STDOUT, > > + shell=True) > > + except subprocess.CalledProcessError as exp: > > + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) > > > > if d.getVar('IPK_SIGN_PACKAGES') == '1': > > ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) > > I get the feeling we have a problem with our subprocess calls and > debugging but its hard to know what to do about it. We can patch each > call site like this but it starts to make we wonder whether we need a > general wrapper, or we're missing something in bitbake or elsewhere to > help with things. > > There is this code: > > http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 > > which is meant to help but it clearly isn't helping/catching all cases. > > I think we are going to need to look at a wider/more general solution > or at least have a plan for one. I would just like to add my +1 here - debugging opkg errors was quite challenging at times due to inadequate logs. While other areas of the project have seen tremendous improvements in the meaningful logs department, poor old opkg was lacking. So, a more generic approach may be required - I'm all for it. -- Denys > Cheers, > > Richard > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-19 22:02 ` [OE-core] " Richard Purdie 2020-05-19 22:13 ` Denys Dmytriyenko @ 2020-05-19 22:14 ` Khem Raj 2020-05-20 5:07 ` Konrad Weihmann 2020-05-20 20:17 ` Konrad Weihmann 2 siblings, 1 reply; 10+ messages in thread From: Khem Raj @ 2020-05-19 22:14 UTC (permalink / raw) To: Richard Purdie Cc: Konrad Weihmann, Patches and discussions about the oe-core layer On Tue, May 19, 2020 at 3:02 PM Richard Purdie <richard.purdie@linuxfoundation.org> wrote: > > On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: > > in case the opkg command fails in before no output of the tool > > itself was printed to assist the user with debugging the issue. > > Print all output of the tool by using CalledProcessError wrapper around > > the call > > > > Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> > > --- > > meta/classes/package_ipk.bbclass | 11 +++++++---- > > 1 file changed, 7 insertions(+), 4 deletions(-) > > > > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass > > index c008559e4a..5e7d552cc5 100644 > > --- a/meta/classes/package_ipk.bbclass > > +++ b/meta/classes/package_ipk.bbclass > > @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): > > conffiles.close() > > > > os.chdir(basedir) > > - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > > - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > > - stderr=subprocess.STDOUT, > > - shell=True) > > + try: > > + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > > + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > > + stderr=subprocess.STDOUT, > > + shell=True) > > + except subprocess.CalledProcessError as exp: > > + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) > > > > if d.getVar('IPK_SIGN_PACKAGES') == '1': > > ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) > > I get the feeling we have a problem with our subprocess calls and > debugging but its hard to know what to do about it. We can patch each > call site like this but it starts to make we wonder whether we need a > general wrapper, or we're missing something in bitbake or elsewhere to > help with things. > > There is this code: > > http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 > > which is meant to help but it clearly isn't helping/catching all cases. > > I think we are going to need to look at a wider/more general solution > or at least have a plan for one. > are you think of wrapping subprocess.check_output calls with some post processing ? > Cheers, > > Richard > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-19 22:14 ` Khem Raj @ 2020-05-20 5:07 ` Konrad Weihmann 2020-05-20 5:34 ` Khem Raj 0 siblings, 1 reply; 10+ messages in thread From: Konrad Weihmann @ 2020-05-20 5:07 UTC (permalink / raw) To: Khem Raj, Richard Purdie; +Cc: Patches and discussions about the oe-core layer Hi all, I don't like the idea of wrapping subprocess.check_* calls once more. IMHO too many possible workflows are possible so chances are low catching them all appropriately. check_output is a wrapper around subprocess.run, which is a wrapper around subprocess.popen which is wrapper of popen, so an additional wrapper doesn't sound like the best idea so far. I'm okay with generalizing the output of a failed call so maybe just http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n330 could be used more widely. It all comes down to handling subprocess failures and in this particular case it was simply missing. Long story short, I'd rather fix them one by one (with respect to actual code flow) than abstracting things even more. Regards Konrad On 20.05.20 00:14, Khem Raj wrote: > On Tue, May 19, 2020 at 3:02 PM Richard Purdie > <richard.purdie@linuxfoundation.org> wrote: >> >> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: >>> in case the opkg command fails in before no output of the tool >>> itself was printed to assist the user with debugging the issue. >>> Print all output of the tool by using CalledProcessError wrapper around >>> the call >>> >>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> >>> --- >>> meta/classes/package_ipk.bbclass | 11 +++++++---- >>> 1 file changed, 7 insertions(+), 4 deletions(-) >>> >>> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass >>> index c008559e4a..5e7d552cc5 100644 >>> --- a/meta/classes/package_ipk.bbclass >>> +++ b/meta/classes/package_ipk.bbclass >>> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): >>> conffiles.close() >>> >>> os.chdir(basedir) >>> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), >>> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), >>> - stderr=subprocess.STDOUT, >>> - shell=True) >>> + try: >>> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), >>> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), >>> + stderr=subprocess.STDOUT, >>> + shell=True) >>> + except subprocess.CalledProcessError as exp: >>> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) >>> >>> if d.getVar('IPK_SIGN_PACKAGES') == '1': >>> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) >> >> I get the feeling we have a problem with our subprocess calls and >> debugging but its hard to know what to do about it. We can patch each >> call site like this but it starts to make we wonder whether we need a >> general wrapper, or we're missing something in bitbake or elsewhere to >> help with things. >> >> There is this code: >> >> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 >> >> which is meant to help but it clearly isn't helping/catching all cases. >> >> I think we are going to need to look at a wider/more general solution >> or at least have a plan for one. >> > > > are you think of wrapping subprocess.check_output calls with some post > processing ? > >> Cheers, >> >> Richard >> >> ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-20 5:07 ` Konrad Weihmann @ 2020-05-20 5:34 ` Khem Raj 0 siblings, 0 replies; 10+ messages in thread From: Khem Raj @ 2020-05-20 5:34 UTC (permalink / raw) To: Konrad Weihmann Cc: Richard Purdie, Patches and discussions about the oe-core layer On Tue, May 19, 2020 at 10:07 PM Konrad Weihmann <kweihmann@outlook.com> wrote: > > Hi all, > > I don't like the idea of wrapping subprocess.check_* calls once more. > IMHO too many possible workflows are possible so chances are low > catching them all appropriately. > > check_output is a wrapper around subprocess.run, which is a wrapper > around subprocess.popen which is wrapper of popen, so an additional > wrapper doesn't sound like the best idea so far. > right, that's what I was contemplating as well. > I'm okay with generalizing the output of a failed call so maybe just > http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n330 > could be used more widely. > > It all comes down to handling subprocess failures and in this particular > case it was simply missing. > > Long story short, I'd rather fix them one by one (with respect to actual > code flow) than abstracting things even more. > > Regards > Konrad > > On 20.05.20 00:14, Khem Raj wrote: > > On Tue, May 19, 2020 at 3:02 PM Richard Purdie > > <richard.purdie@linuxfoundation.org> wrote: > >> > >> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: > >>> in case the opkg command fails in before no output of the tool > >>> itself was printed to assist the user with debugging the issue. > >>> Print all output of the tool by using CalledProcessError wrapper around > >>> the call > >>> > >>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> > >>> --- > >>> meta/classes/package_ipk.bbclass | 11 +++++++---- > >>> 1 file changed, 7 insertions(+), 4 deletions(-) > >>> > >>> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass > >>> index c008559e4a..5e7d552cc5 100644 > >>> --- a/meta/classes/package_ipk.bbclass > >>> +++ b/meta/classes/package_ipk.bbclass > >>> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): > >>> conffiles.close() > >>> > >>> os.chdir(basedir) > >>> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > >>> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > >>> - stderr=subprocess.STDOUT, > >>> - shell=True) > >>> + try: > >>> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > >>> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > >>> + stderr=subprocess.STDOUT, > >>> + shell=True) > >>> + except subprocess.CalledProcessError as exp: > >>> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) > >>> > >>> if d.getVar('IPK_SIGN_PACKAGES') == '1': > >>> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) > >> > >> I get the feeling we have a problem with our subprocess calls and > >> debugging but its hard to know what to do about it. We can patch each > >> call site like this but it starts to make we wonder whether we need a > >> general wrapper, or we're missing something in bitbake or elsewhere to > >> help with things. > >> > >> There is this code: > >> > >> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 > >> > >> which is meant to help but it clearly isn't helping/catching all cases. > >> > >> I think we are going to need to look at a wider/more general solution > >> or at least have a plan for one. > >> > > > > > > are you think of wrapping subprocess.check_output calls with some post > > processing ? > > > >> Cheers, > >> > >> Richard > >> > >> ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-19 22:02 ` [OE-core] " Richard Purdie 2020-05-19 22:13 ` Denys Dmytriyenko 2020-05-19 22:14 ` Khem Raj @ 2020-05-20 20:17 ` Konrad Weihmann 2020-05-20 21:28 ` Khem Raj 2020-05-21 13:24 ` Richard Purdie 2 siblings, 2 replies; 10+ messages in thread From: Konrad Weihmann @ 2020-05-20 20:17 UTC (permalink / raw) To: Richard Purdie, openembedded-core On a second thought it should be bb.error not bb.fatal - are all okay with fixing just this one for now? On 20.05.20 00:02, Richard Purdie wrote: > On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: >> in case the opkg command fails in before no output of the tool >> itself was printed to assist the user with debugging the issue. >> Print all output of the tool by using CalledProcessError wrapper around >> the call >> >> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> >> --- >> meta/classes/package_ipk.bbclass | 11 +++++++---- >> 1 file changed, 7 insertions(+), 4 deletions(-) >> >> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass >> index c008559e4a..5e7d552cc5 100644 >> --- a/meta/classes/package_ipk.bbclass >> +++ b/meta/classes/package_ipk.bbclass >> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): >> conffiles.close() >> >> os.chdir(basedir) >> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), >> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), >> - stderr=subprocess.STDOUT, >> - shell=True) >> + try: >> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), >> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), >> + stderr=subprocess.STDOUT, >> + shell=True) >> + except subprocess.CalledProcessError as exp: >> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) >> >> if d.getVar('IPK_SIGN_PACKAGES') == '1': >> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) > > I get the feeling we have a problem with our subprocess calls and > debugging but its hard to know what to do about it. We can patch each > call site like this but it starts to make we wonder whether we need a > general wrapper, or we're missing something in bitbake or elsewhere to > help with things. > > There is this code: > > http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 > > which is meant to help but it clearly isn't helping/catching all cases. > > I think we are going to need to look at a wider/more general solution > or at least have a plan for one. > > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-20 20:17 ` Konrad Weihmann @ 2020-05-20 21:28 ` Khem Raj 2020-05-21 13:24 ` Richard Purdie 1 sibling, 0 replies; 10+ messages in thread From: Khem Raj @ 2020-05-20 21:28 UTC (permalink / raw) To: Konrad Weihmann Cc: Richard Purdie, Patches and discussions about the oe-core layer On Wed, May 20, 2020 at 1:17 PM Konrad Weihmann <kweihmann@outlook.com> wrote: > > On a second thought it should be bb.error not bb.fatal - are all okay > with fixing just this one for now? I am yes. > > On 20.05.20 00:02, Richard Purdie wrote: > > On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote: > >> in case the opkg command fails in before no output of the tool > >> itself was printed to assist the user with debugging the issue. > >> Print all output of the tool by using CalledProcessError wrapper around > >> the call > >> > >> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com> > >> --- > >> meta/classes/package_ipk.bbclass | 11 +++++++---- > >> 1 file changed, 7 insertions(+), 4 deletions(-) > >> > >> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass > >> index c008559e4a..5e7d552cc5 100644 > >> --- a/meta/classes/package_ipk.bbclass > >> +++ b/meta/classes/package_ipk.bbclass > >> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d): > >> conffiles.close() > >> > >> os.chdir(basedir) > >> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > >> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > >> - stderr=subprocess.STDOUT, > >> - shell=True) > >> + try: > >> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"), > >> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir), > >> + stderr=subprocess.STDOUT, > >> + shell=True) > >> + except subprocess.CalledProcessError as exp: > >> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8"))) > >> > >> if d.getVar('IPK_SIGN_PACKAGES') == '1': > >> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR')) > > > > I get the feeling we have a problem with our subprocess calls and > > debugging but its hard to know what to do about it. We can patch each > > call site like this but it starts to make we wonder whether we need a > > general wrapper, or we're missing something in bitbake or elsewhere to > > help with things. > > > > There is this code: > > > > http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378 > > > > which is meant to help but it clearly isn't helping/catching all cases. > > > > I think we are going to need to look at a wider/more general solution > > or at least have a plan for one. > > > > Cheers, > > > > Richard > > > ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-20 20:17 ` Konrad Weihmann 2020-05-20 21:28 ` Khem Raj @ 2020-05-21 13:24 ` Richard Purdie 2020-05-21 13:48 ` Konrad Weihmann 1 sibling, 1 reply; 10+ messages in thread From: Richard Purdie @ 2020-05-21 13:24 UTC (permalink / raw) To: Konrad Weihmann, openembedded-core On Wed, 2020-05-20 at 22:17 +0200, Konrad Weihmann wrote: > On a second thought it should be bb.error not bb.fatal - are all > okay with fixing just this one for now? I think the problem here is specific to the way parallelism is done with oe.utils.multiprocess_launch. A quick grep shows a few users: meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelflinks.values(), d) meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelf.keys(), d) meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d)) meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, debugstaticdir, debugstaticlibdir, debugstaticappend, debugsrcdir, d)) meta/classes/package.bbclass: oe.utils.multiprocess_launch(oe.package.runstrip, sfiles, d) meta/classes/package.bbclass: oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d, meta/classes/package.bbclass: processed = oe.utils.multiprocess_launch(oe.package.filedeprunner, pkglist, d) meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(linux_so, linuxlist, d, extraargs=(pkg, pkgver, d)) meta/classes/package_ipk.bbclass: oe.utils.multiprocess_launch(ipk_write_pkg, packages.split(), d, extraargs=(d,)) meta/classes/package_deb.bbclass: oe.utils.multiprocess_launch(deb_write_pkg, packages.split(), d, extraargs=(d,)) and I think any subprocess usage within any of these functions is at risk from this issue. Is there some way we could add a fix to oe.utils.multiprocess_launch() ? Cheers, Richard ^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error 2020-05-21 13:24 ` Richard Purdie @ 2020-05-21 13:48 ` Konrad Weihmann 0 siblings, 0 replies; 10+ messages in thread From: Konrad Weihmann @ 2020-05-21 13:48 UTC (permalink / raw) To: Richard Purdie, openembedded-core Yeah you're right 244cbcce0ecc4691a9ddfb0a44dc487ff7af0670 should already fix that. The last time I encountered this issue was on thud, where this fix never got applied - so all should be fine, patch can be ignored, as all branches starting from warrior are safe, there is really no need for any action here On 21.05.20 15:24, Richard Purdie wrote: > On Wed, 2020-05-20 at 22:17 +0200, Konrad Weihmann wrote: >> On a second thought it should be bb.error not bb.fatal - are all >> okay with fixing just this one for now? > > I think the problem here is specific to the way parallelism is done > with oe.utils.multiprocess_launch. > > A quick grep shows a few users: > > meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelflinks.values(), d) > meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelf.keys(), d) > meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d)) > meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, debugstaticdir, debugstaticlibdir, debugstaticappend, debugsrcdir, d)) > meta/classes/package.bbclass: oe.utils.multiprocess_launch(oe.package.runstrip, sfiles, d) > meta/classes/package.bbclass: oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d, > meta/classes/package.bbclass: processed = oe.utils.multiprocess_launch(oe.package.filedeprunner, pkglist, d) > meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(linux_so, linuxlist, d, extraargs=(pkg, pkgver, d)) > meta/classes/package_ipk.bbclass: oe.utils.multiprocess_launch(ipk_write_pkg, packages.split(), d, extraargs=(d,)) > meta/classes/package_deb.bbclass: oe.utils.multiprocess_launch(deb_write_pkg, packages.split(), d, extraargs=(d,)) > > and I think any subprocess usage within any of these functions is at > risk from this issue. > > Is there some way we could add a fix to oe.utils.multiprocess_launch() > ? > > Cheers, > > Richard > ^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-05-21 13:48 UTC | newest] Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-05-19 18:36 [PATCH] package_ipk.bbclass: print opkg output on error Konrad Weihmann 2020-05-19 22:02 ` [OE-core] " Richard Purdie 2020-05-19 22:13 ` Denys Dmytriyenko 2020-05-19 22:14 ` Khem Raj 2020-05-20 5:07 ` Konrad Weihmann 2020-05-20 5:34 ` Khem Raj 2020-05-20 20:17 ` Konrad Weihmann 2020-05-20 21:28 ` Khem Raj 2020-05-21 13:24 ` Richard Purdie 2020-05-21 13:48 ` Konrad Weihmann
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.