* [PATCH] package_ipk.bbclass: print opkg output on error
@ 2020-05-19 18:36 Konrad Weihmann
2020-05-19 22:02 ` [OE-core] " Richard Purdie
0 siblings, 1 reply; 10+ messages in thread
From: Konrad Weihmann @ 2020-05-19 18:36 UTC (permalink / raw)
To: openembedded-core; +Cc: Konrad Weihmann
in case the opkg command fails in before no output of the tool
itself was printed to assist the user with debugging the issue.
Print all output of the tool by using CalledProcessError wrapper around
the call
Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
---
meta/classes/package_ipk.bbclass | 11 +++++++----
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index c008559e4a..5e7d552cc5 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
conffiles.close()
os.chdir(basedir)
- subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
- d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
- stderr=subprocess.STDOUT,
- shell=True)
+ try:
+ subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
+ d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
+ stderr=subprocess.STDOUT,
+ shell=True)
+ except subprocess.CalledProcessError as exp:
+ bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
if d.getVar('IPK_SIGN_PACKAGES') == '1':
ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
--
2.20.1
^ permalink raw reply related [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-19 18:36 [PATCH] package_ipk.bbclass: print opkg output on error Konrad Weihmann
@ 2020-05-19 22:02 ` Richard Purdie
2020-05-19 22:13 ` Denys Dmytriyenko
` (2 more replies)
0 siblings, 3 replies; 10+ messages in thread
From: Richard Purdie @ 2020-05-19 22:02 UTC (permalink / raw)
To: Konrad Weihmann, openembedded-core
On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
> in case the opkg command fails in before no output of the tool
> itself was printed to assist the user with debugging the issue.
> Print all output of the tool by using CalledProcessError wrapper around
> the call
>
> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> ---
> meta/classes/package_ipk.bbclass | 11 +++++++----
> 1 file changed, 7 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> index c008559e4a..5e7d552cc5 100644
> --- a/meta/classes/package_ipk.bbclass
> +++ b/meta/classes/package_ipk.bbclass
> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
> conffiles.close()
>
> os.chdir(basedir)
> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> - stderr=subprocess.STDOUT,
> - shell=True)
> + try:
> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> + stderr=subprocess.STDOUT,
> + shell=True)
> + except subprocess.CalledProcessError as exp:
> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
>
> if d.getVar('IPK_SIGN_PACKAGES') == '1':
> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
I get the feeling we have a problem with our subprocess calls and
debugging but its hard to know what to do about it. We can patch each
call site like this but it starts to make we wonder whether we need a
general wrapper, or we're missing something in bitbake or elsewhere to
help with things.
There is this code:
http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
which is meant to help but it clearly isn't helping/catching all cases.
I think we are going to need to look at a wider/more general solution
or at least have a plan for one.
Cheers,
Richard
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-19 22:02 ` [OE-core] " Richard Purdie
@ 2020-05-19 22:13 ` Denys Dmytriyenko
2020-05-19 22:14 ` Khem Raj
2020-05-20 20:17 ` Konrad Weihmann
2 siblings, 0 replies; 10+ messages in thread
From: Denys Dmytriyenko @ 2020-05-19 22:13 UTC (permalink / raw)
To: Richard Purdie; +Cc: Konrad Weihmann, openembedded-core
On Tue, May 19, 2020 at 11:02:46PM +0100, Richard Purdie wrote:
> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
> > in case the opkg command fails in before no output of the tool
> > itself was printed to assist the user with debugging the issue.
> > Print all output of the tool by using CalledProcessError wrapper around
> > the call
> >
> > Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> > ---
> > meta/classes/package_ipk.bbclass | 11 +++++++----
> > 1 file changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> > index c008559e4a..5e7d552cc5 100644
> > --- a/meta/classes/package_ipk.bbclass
> > +++ b/meta/classes/package_ipk.bbclass
> > @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
> > conffiles.close()
> >
> > os.chdir(basedir)
> > - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> > - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> > - stderr=subprocess.STDOUT,
> > - shell=True)
> > + try:
> > + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> > + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> > + stderr=subprocess.STDOUT,
> > + shell=True)
> > + except subprocess.CalledProcessError as exp:
> > + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
> >
> > if d.getVar('IPK_SIGN_PACKAGES') == '1':
> > ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
>
> I get the feeling we have a problem with our subprocess calls and
> debugging but its hard to know what to do about it. We can patch each
> call site like this but it starts to make we wonder whether we need a
> general wrapper, or we're missing something in bitbake or elsewhere to
> help with things.
>
> There is this code:
>
> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
>
> which is meant to help but it clearly isn't helping/catching all cases.
>
> I think we are going to need to look at a wider/more general solution
> or at least have a plan for one.
I would just like to add my +1 here - debugging opkg errors was quite
challenging at times due to inadequate logs. While other areas of the
project have seen tremendous improvements in the meaningful logs
department, poor old opkg was lacking. So, a more generic approach may
be required - I'm all for it.
--
Denys
> Cheers,
>
> Richard
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-19 22:02 ` [OE-core] " Richard Purdie
2020-05-19 22:13 ` Denys Dmytriyenko
@ 2020-05-19 22:14 ` Khem Raj
2020-05-20 5:07 ` Konrad Weihmann
2020-05-20 20:17 ` Konrad Weihmann
2 siblings, 1 reply; 10+ messages in thread
From: Khem Raj @ 2020-05-19 22:14 UTC (permalink / raw)
To: Richard Purdie
Cc: Konrad Weihmann, Patches and discussions about the oe-core layer
On Tue, May 19, 2020 at 3:02 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
> > in case the opkg command fails in before no output of the tool
> > itself was printed to assist the user with debugging the issue.
> > Print all output of the tool by using CalledProcessError wrapper around
> > the call
> >
> > Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> > ---
> > meta/classes/package_ipk.bbclass | 11 +++++++----
> > 1 file changed, 7 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> > index c008559e4a..5e7d552cc5 100644
> > --- a/meta/classes/package_ipk.bbclass
> > +++ b/meta/classes/package_ipk.bbclass
> > @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
> > conffiles.close()
> >
> > os.chdir(basedir)
> > - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> > - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> > - stderr=subprocess.STDOUT,
> > - shell=True)
> > + try:
> > + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> > + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> > + stderr=subprocess.STDOUT,
> > + shell=True)
> > + except subprocess.CalledProcessError as exp:
> > + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
> >
> > if d.getVar('IPK_SIGN_PACKAGES') == '1':
> > ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
>
> I get the feeling we have a problem with our subprocess calls and
> debugging but its hard to know what to do about it. We can patch each
> call site like this but it starts to make we wonder whether we need a
> general wrapper, or we're missing something in bitbake or elsewhere to
> help with things.
>
> There is this code:
>
> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
>
> which is meant to help but it clearly isn't helping/catching all cases.
>
> I think we are going to need to look at a wider/more general solution
> or at least have a plan for one.
>
are you think of wrapping subprocess.check_output calls with some post
processing ?
> Cheers,
>
> Richard
>
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-19 22:14 ` Khem Raj
@ 2020-05-20 5:07 ` Konrad Weihmann
2020-05-20 5:34 ` Khem Raj
0 siblings, 1 reply; 10+ messages in thread
From: Konrad Weihmann @ 2020-05-20 5:07 UTC (permalink / raw)
To: Khem Raj, Richard Purdie; +Cc: Patches and discussions about the oe-core layer
Hi all,
I don't like the idea of wrapping subprocess.check_* calls once more.
IMHO too many possible workflows are possible so chances are low
catching them all appropriately.
check_output is a wrapper around subprocess.run, which is a wrapper
around subprocess.popen which is wrapper of popen, so an additional
wrapper doesn't sound like the best idea so far.
I'm okay with generalizing the output of a failed call so maybe just
http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n330
could be used more widely.
It all comes down to handling subprocess failures and in this particular
case it was simply missing.
Long story short, I'd rather fix them one by one (with respect to actual
code flow) than abstracting things even more.
Regards
Konrad
On 20.05.20 00:14, Khem Raj wrote:
> On Tue, May 19, 2020 at 3:02 PM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
>>
>> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
>>> in case the opkg command fails in before no output of the tool
>>> itself was printed to assist the user with debugging the issue.
>>> Print all output of the tool by using CalledProcessError wrapper around
>>> the call
>>>
>>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
>>> ---
>>> meta/classes/package_ipk.bbclass | 11 +++++++----
>>> 1 file changed, 7 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
>>> index c008559e4a..5e7d552cc5 100644
>>> --- a/meta/classes/package_ipk.bbclass
>>> +++ b/meta/classes/package_ipk.bbclass
>>> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
>>> conffiles.close()
>>>
>>> os.chdir(basedir)
>>> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
>>> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
>>> - stderr=subprocess.STDOUT,
>>> - shell=True)
>>> + try:
>>> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
>>> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
>>> + stderr=subprocess.STDOUT,
>>> + shell=True)
>>> + except subprocess.CalledProcessError as exp:
>>> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
>>>
>>> if d.getVar('IPK_SIGN_PACKAGES') == '1':
>>> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
>>
>> I get the feeling we have a problem with our subprocess calls and
>> debugging but its hard to know what to do about it. We can patch each
>> call site like this but it starts to make we wonder whether we need a
>> general wrapper, or we're missing something in bitbake or elsewhere to
>> help with things.
>>
>> There is this code:
>>
>> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
>>
>> which is meant to help but it clearly isn't helping/catching all cases.
>>
>> I think we are going to need to look at a wider/more general solution
>> or at least have a plan for one.
>>
>
>
> are you think of wrapping subprocess.check_output calls with some post
> processing ?
>
>> Cheers,
>>
>> Richard
>>
>>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-20 5:07 ` Konrad Weihmann
@ 2020-05-20 5:34 ` Khem Raj
0 siblings, 0 replies; 10+ messages in thread
From: Khem Raj @ 2020-05-20 5:34 UTC (permalink / raw)
To: Konrad Weihmann
Cc: Richard Purdie, Patches and discussions about the oe-core layer
On Tue, May 19, 2020 at 10:07 PM Konrad Weihmann <kweihmann@outlook.com> wrote:
>
> Hi all,
>
> I don't like the idea of wrapping subprocess.check_* calls once more.
> IMHO too many possible workflows are possible so chances are low
> catching them all appropriately.
>
> check_output is a wrapper around subprocess.run, which is a wrapper
> around subprocess.popen which is wrapper of popen, so an additional
> wrapper doesn't sound like the best idea so far.
>
right, that's what I was contemplating as well.
> I'm okay with generalizing the output of a failed call so maybe just
> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n330
> could be used more widely.
>
> It all comes down to handling subprocess failures and in this particular
> case it was simply missing.
>
> Long story short, I'd rather fix them one by one (with respect to actual
> code flow) than abstracting things even more.
>
> Regards
> Konrad
>
> On 20.05.20 00:14, Khem Raj wrote:
> > On Tue, May 19, 2020 at 3:02 PM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> >>
> >> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
> >>> in case the opkg command fails in before no output of the tool
> >>> itself was printed to assist the user with debugging the issue.
> >>> Print all output of the tool by using CalledProcessError wrapper around
> >>> the call
> >>>
> >>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> >>> ---
> >>> meta/classes/package_ipk.bbclass | 11 +++++++----
> >>> 1 file changed, 7 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> >>> index c008559e4a..5e7d552cc5 100644
> >>> --- a/meta/classes/package_ipk.bbclass
> >>> +++ b/meta/classes/package_ipk.bbclass
> >>> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
> >>> conffiles.close()
> >>>
> >>> os.chdir(basedir)
> >>> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> >>> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> >>> - stderr=subprocess.STDOUT,
> >>> - shell=True)
> >>> + try:
> >>> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> >>> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> >>> + stderr=subprocess.STDOUT,
> >>> + shell=True)
> >>> + except subprocess.CalledProcessError as exp:
> >>> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
> >>>
> >>> if d.getVar('IPK_SIGN_PACKAGES') == '1':
> >>> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> >>
> >> I get the feeling we have a problem with our subprocess calls and
> >> debugging but its hard to know what to do about it. We can patch each
> >> call site like this but it starts to make we wonder whether we need a
> >> general wrapper, or we're missing something in bitbake or elsewhere to
> >> help with things.
> >>
> >> There is this code:
> >>
> >> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
> >>
> >> which is meant to help but it clearly isn't helping/catching all cases.
> >>
> >> I think we are going to need to look at a wider/more general solution
> >> or at least have a plan for one.
> >>
> >
> >
> > are you think of wrapping subprocess.check_output calls with some post
> > processing ?
> >
> >> Cheers,
> >>
> >> Richard
> >>
> >>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-19 22:02 ` [OE-core] " Richard Purdie
2020-05-19 22:13 ` Denys Dmytriyenko
2020-05-19 22:14 ` Khem Raj
@ 2020-05-20 20:17 ` Konrad Weihmann
2020-05-20 21:28 ` Khem Raj
2020-05-21 13:24 ` Richard Purdie
2 siblings, 2 replies; 10+ messages in thread
From: Konrad Weihmann @ 2020-05-20 20:17 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
On a second thought it should be bb.error not bb.fatal - are all okay
with fixing just this one for now?
On 20.05.20 00:02, Richard Purdie wrote:
> On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
>> in case the opkg command fails in before no output of the tool
>> itself was printed to assist the user with debugging the issue.
>> Print all output of the tool by using CalledProcessError wrapper around
>> the call
>>
>> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
>> ---
>> meta/classes/package_ipk.bbclass | 11 +++++++----
>> 1 file changed, 7 insertions(+), 4 deletions(-)
>>
>> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
>> index c008559e4a..5e7d552cc5 100644
>> --- a/meta/classes/package_ipk.bbclass
>> +++ b/meta/classes/package_ipk.bbclass
>> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
>> conffiles.close()
>>
>> os.chdir(basedir)
>> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
>> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
>> - stderr=subprocess.STDOUT,
>> - shell=True)
>> + try:
>> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
>> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
>> + stderr=subprocess.STDOUT,
>> + shell=True)
>> + except subprocess.CalledProcessError as exp:
>> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
>>
>> if d.getVar('IPK_SIGN_PACKAGES') == '1':
>> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
>
> I get the feeling we have a problem with our subprocess calls and
> debugging but its hard to know what to do about it. We can patch each
> call site like this but it starts to make we wonder whether we need a
> general wrapper, or we're missing something in bitbake or elsewhere to
> help with things.
>
> There is this code:
>
> http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
>
> which is meant to help but it clearly isn't helping/catching all cases.
>
> I think we are going to need to look at a wider/more general solution
> or at least have a plan for one.
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-20 20:17 ` Konrad Weihmann
@ 2020-05-20 21:28 ` Khem Raj
2020-05-21 13:24 ` Richard Purdie
1 sibling, 0 replies; 10+ messages in thread
From: Khem Raj @ 2020-05-20 21:28 UTC (permalink / raw)
To: Konrad Weihmann
Cc: Richard Purdie, Patches and discussions about the oe-core layer
On Wed, May 20, 2020 at 1:17 PM Konrad Weihmann <kweihmann@outlook.com> wrote:
>
> On a second thought it should be bb.error not bb.fatal - are all okay
> with fixing just this one for now?
I am yes.
>
> On 20.05.20 00:02, Richard Purdie wrote:
> > On Tue, 2020-05-19 at 20:36 +0200, Konrad Weihmann wrote:
> >> in case the opkg command fails in before no output of the tool
> >> itself was printed to assist the user with debugging the issue.
> >> Print all output of the tool by using CalledProcessError wrapper around
> >> the call
> >>
> >> Signed-off-by: Konrad Weihmann <kweihmann@outlook.com>
> >> ---
> >> meta/classes/package_ipk.bbclass | 11 +++++++----
> >> 1 file changed, 7 insertions(+), 4 deletions(-)
> >>
> >> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> >> index c008559e4a..5e7d552cc5 100644
> >> --- a/meta/classes/package_ipk.bbclass
> >> +++ b/meta/classes/package_ipk.bbclass
> >> @@ -223,10 +223,13 @@ def ipk_write_pkg(pkg, d):
> >> conffiles.close()
> >>
> >> os.chdir(basedir)
> >> - subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> >> - d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> >> - stderr=subprocess.STDOUT,
> >> - shell=True)
> >> + try:
> >> + subprocess.check_output("PATH=\"%s\" %s %s %s" % (localdata.getVar("PATH"),
> >> + d.getVar("OPKGBUILDCMD"), pkg, pkgoutdir),
> >> + stderr=subprocess.STDOUT,
> >> + shell=True)
> >> + except subprocess.CalledProcessError as exp:
> >> + bb.fatal("{} failed with {}: {}".format(exp.cmd, exp.returncode, exp.output.decode("utf-8")))
> >>
> >> if d.getVar('IPK_SIGN_PACKAGES') == '1':
> >> ipkver = "%s-%s" % (d.getVar('PKGV'), d.getVar('PKGR'))
> >
> > I get the feeling we have a problem with our subprocess calls and
> > debugging but its hard to know what to do about it. We can patch each
> > call site like this but it starts to make we wonder whether we need a
> > general wrapper, or we're missing something in bitbake or elsewhere to
> > help with things.
> >
> > There is this code:
> >
> > http://git.yoctoproject.org/cgit.cgi/poky/tree/bitbake/lib/bb/utils.py#n378
> >
> > which is meant to help but it clearly isn't helping/catching all cases.
> >
> > I think we are going to need to look at a wider/more general solution
> > or at least have a plan for one.
> >
> > Cheers,
> >
> > Richard
> >
>
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-20 20:17 ` Konrad Weihmann
2020-05-20 21:28 ` Khem Raj
@ 2020-05-21 13:24 ` Richard Purdie
2020-05-21 13:48 ` Konrad Weihmann
1 sibling, 1 reply; 10+ messages in thread
From: Richard Purdie @ 2020-05-21 13:24 UTC (permalink / raw)
To: Konrad Weihmann, openembedded-core
On Wed, 2020-05-20 at 22:17 +0200, Konrad Weihmann wrote:
> On a second thought it should be bb.error not bb.fatal - are all
> okay with fixing just this one for now?
I think the problem here is specific to the way parallelism is done
with oe.utils.multiprocess_launch.
A quick grep shows a few users:
meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelflinks.values(), d)
meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelf.keys(), d)
meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d))
meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, debugstaticdir, debugstaticlibdir, debugstaticappend, debugsrcdir, d))
meta/classes/package.bbclass: oe.utils.multiprocess_launch(oe.package.runstrip, sfiles, d)
meta/classes/package.bbclass: oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d,
meta/classes/package.bbclass: processed = oe.utils.multiprocess_launch(oe.package.filedeprunner, pkglist, d)
meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(linux_so, linuxlist, d, extraargs=(pkg, pkgver, d))
meta/classes/package_ipk.bbclass: oe.utils.multiprocess_launch(ipk_write_pkg, packages.split(), d, extraargs=(d,))
meta/classes/package_deb.bbclass: oe.utils.multiprocess_launch(deb_write_pkg, packages.split(), d, extraargs=(d,))
and I think any subprocess usage within any of these functions is at
risk from this issue.
Is there some way we could add a fix to oe.utils.multiprocess_launch()
?
Cheers,
Richard
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: [OE-core] [PATCH] package_ipk.bbclass: print opkg output on error
2020-05-21 13:24 ` Richard Purdie
@ 2020-05-21 13:48 ` Konrad Weihmann
0 siblings, 0 replies; 10+ messages in thread
From: Konrad Weihmann @ 2020-05-21 13:48 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
Yeah you're right 244cbcce0ecc4691a9ddfb0a44dc487ff7af0670 should
already fix that. The last time I encountered this issue was on thud,
where this fix never got applied - so all should be fine, patch can be
ignored, as all branches starting from warrior are safe, there is really
no need for any action here
On 21.05.20 15:24, Richard Purdie wrote:
> On Wed, 2020-05-20 at 22:17 +0200, Konrad Weihmann wrote:
>> On a second thought it should be bb.error not bb.fatal - are all
>> okay with fixing just this one for now?
>
> I think the problem here is specific to the way parallelism is done
> with oe.utils.multiprocess_launch.
>
> A quick grep shows a few users:
>
> meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelflinks.values(), d)
> meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(oe.package.is_elf, checkelf.keys(), d)
> meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitdebuginfo, list(elffiles), d, extraargs=(dvar, debugdir, debuglibdir, debugappend, debugsrcdir, d))
> meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(splitstaticdebuginfo, staticlibs, d, extraargs=(dvar, debugstaticdir, debugstaticlibdir, debugstaticappend, debugsrcdir, d))
> meta/classes/package.bbclass: oe.utils.multiprocess_launch(oe.package.runstrip, sfiles, d)
> meta/classes/package.bbclass: oe.utils.multiprocess_launch(inject_minidebuginfo, list(elffiles), d,
> meta/classes/package.bbclass: processed = oe.utils.multiprocess_launch(oe.package.filedeprunner, pkglist, d)
> meta/classes/package.bbclass: results = oe.utils.multiprocess_launch(linux_so, linuxlist, d, extraargs=(pkg, pkgver, d))
> meta/classes/package_ipk.bbclass: oe.utils.multiprocess_launch(ipk_write_pkg, packages.split(), d, extraargs=(d,))
> meta/classes/package_deb.bbclass: oe.utils.multiprocess_launch(deb_write_pkg, packages.split(), d, extraargs=(d,))
>
> and I think any subprocess usage within any of these functions is at
> risk from this issue.
>
> Is there some way we could add a fix to oe.utils.multiprocess_launch()
> ?
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2020-05-21 13:48 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-05-19 18:36 [PATCH] package_ipk.bbclass: print opkg output on error Konrad Weihmann
2020-05-19 22:02 ` [OE-core] " Richard Purdie
2020-05-19 22:13 ` Denys Dmytriyenko
2020-05-19 22:14 ` Khem Raj
2020-05-20 5:07 ` Konrad Weihmann
2020-05-20 5:34 ` Khem Raj
2020-05-20 20:17 ` Konrad Weihmann
2020-05-20 21:28 ` Khem Raj
2020-05-21 13:24 ` Richard Purdie
2020-05-21 13:48 ` Konrad Weihmann
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.