[PATCH] devtool-source.bbclass: Only create each patch branch once

[PATCH] devtool-source.bbclass: Only create each patch branch once

[Olof Johansson]

For conditonally applied patches based on SRC_URI overrides, the
devtool-source class would try to create a new branch for each override
assignment as a postfunc to do_patch, but if the same override was used
multiple times, it would try to create the same branch multiple times,
causing errors like

> Exception: bb.process.ExecutionError: Execution of
\   'git checkout f0f0f0f0f0ff0f0f0f0f0f0f0f0f0f0f0ff0f0f0 -b devtool-override-foo'
\   failed with exit code 128:
> fatal: A branch named 'devtool-override-foo' already exists.

This change makes sure that the devtool-source bbclass will only create
one branch per override.

Change-Id: I8adca0b54179793ca92478ad5b3b1b6e0448e26b
Signed-off-by: Olof Johansson <olofjn@axis.com>
---
 meta/classes/devtool-source.bbclass | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
index 56882a4..fa4e999 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -169,16 +169,15 @@ python devtool_post_patch() {
         except bb.process.ExecutionError:
             pass
 
-    extra_overrides = d.getVar('DEVTOOL_EXTRA_OVERRIDES')
-    if extra_overrides:
-        extra_override_list = extra_overrides.split(':')
+    devtool_overrides = set(d.getVar('DEVTOOL_EXTRA_OVERRIDES').split(':') or [])
+    if devtool_overrides:
         devbranch = d.getVar('DEVTOOL_DEVBRANCH')
         default_overrides = d.getVar('OVERRIDES').split(':')
         no_overrides = []
         # First, we may have some overrides that are referred to in the recipe set in
         # our configuration, so we need to make a branch that excludes those
         for override in default_overrides:
-            if override not in extra_override_list:
+            if override not in devtool_overrides:
                 no_overrides.append(override)
         if default_overrides != no_overrides:
             # Some overrides are active in the current configuration, so
@@ -196,7 +195,7 @@ python devtool_post_patch() {
         else:
             bb.process.run('git checkout %s -b devtool-no-overrides' % devbranch, cwd=srcsubdir)
 
-        for override in extra_override_list:
+        for override in devtool_overrides:
             localdata = bb.data.createCopy(d)
             if override in default_overrides:
                 bb.process.run('git branch devtool-override-%s %s' % (override, devbranch), cwd=srcsubdir)
-- 
2.11.0

Re: [PATCH] devtool-source.bbclass: Only create each patch branch once

[Peter Kjellerstedt]

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org <openembedded-
> core-bounces@lists.openembedded.org> On Behalf Of Olof Johansson
> Sent: den 10 oktober 2018 17:35
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH] devtool-source.bbclass: Only create each
> patch branch once
> 
> For conditonally applied patches based on SRC_URI overrides, the
> devtool-source class would try to create a new branch for each override
> assignment as a postfunc to do_patch, but if the same override was used
> multiple times, it would try to create the same branch multiple times,
> causing errors like
> 
> > Exception: bb.process.ExecutionError: Execution of
> \   'git checkout f0f0f0f0f0ff0f0f0f0f0f0f0f0f0f0f0ff0f0f0 -b devtool-override-foo'
> \   failed with exit code 128:
> > fatal: A branch named 'devtool-override-foo' already exists.
> 
> This change makes sure that the devtool-source bbclass will only create
> one branch per override.
> 
> Change-Id: I8adca0b54179793ca92478ad5b3b1b6e0448e26b

You probably want to remove the Change-Id before applying this...

> Signed-off-by: Olof Johansson <olofjn@axis.com>
> ---
>  meta/classes/devtool-source.bbclass | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
> index 56882a4..fa4e999 100644
> --- a/meta/classes/devtool-source.bbclass
> +++ b/meta/classes/devtool-source.bbclass
> @@ -169,16 +169,15 @@ python devtool_post_patch() {
>          except bb.process.ExecutionError:
>              pass
> 
> -    extra_overrides = d.getVar('DEVTOOL_EXTRA_OVERRIDES')
> -    if extra_overrides:
> -        extra_override_list = extra_overrides.split(':')
> +    devtool_overrides = set(d.getVar('DEVTOOL_EXTRA_OVERRIDES').split(':') or [])
> +    if devtool_overrides:
>          devbranch = d.getVar('DEVTOOL_DEVBRANCH')
>          default_overrides = d.getVar('OVERRIDES').split(':')
>          no_overrides = []
>          # First, we may have some overrides that are referred to in the recipe set in
>          # our configuration, so we need to make a branch that excludes those
>          for override in default_overrides:
> -            if override not in extra_override_list:
> +            if override not in devtool_overrides:
>                  no_overrides.append(override)
>          if default_overrides != no_overrides:
>              # Some overrides are active in the current configuration, so
> @@ -196,7 +195,7 @@ python devtool_post_patch() {
>          else:
>              bb.process.run('git checkout %s -b devtool-no-overrides' % devbranch, cwd=srcsubdir)
> 
> -        for override in extra_override_list:
> +        for override in devtool_overrides:
>              localdata = bb.data.createCopy(d)
>              if override in default_overrides:
>                  bb.process.run('git branch devtool-override-%s %s' % (override, devbranch), cwd=srcsubdir)
> --
> 2.11.0

//Peter

Re: [PATCH] devtool-source.bbclass: Only create each patch branch once

[Olof Johansson]

On 18-10-10 17:48 +0200, Peter Kjellerstedt wrote:
> > Change-Id: I8adca0b54179793ca92478ad5b3b1b6e0448e26b
> 
> You probably want to remove the Change-Id before applying this...

Sorry! :( I'll probably forget to remove that again in the
future, I should try making git format-patch automatically strip
that. Thanks Peter!

-- 
olofjn

Re: [PATCH] devtool-source.bbclass: Only create each patch branch once

[Richard Purdie]

On Wed, 2018-10-10 at 17:35 +0200, Olof Johansson wrote:
> For conditonally applied patches based on SRC_URI overrides, the
> devtool-source class would try to create a new branch for each
> override
> assignment as a postfunc to do_patch, but if the same override was
> used
> multiple times, it would try to create the same branch multiple
> times,
> causing errors like
> 
> > Exception: bb.process.ExecutionError: Execution of
> 
> \   'git checkout f0f0f0f0f0ff0f0f0f0f0f0f0f0f0f0f0ff0f0f0 -b
> devtool-override-foo'
> \   failed with exit code 128:
> > fatal: A branch named 'devtool-override-foo' already exists.
> 
> This change makes sure that the devtool-source bbclass will only
> create
> one branch per override.
> 
> Change-Id: I8adca0b54179793ca92478ad5b3b1b6e0448e26b
> Signed-off-by: Olof Johansson <olofjn@axis.com>
> ---
>  meta/classes/devtool-source.bbclass | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)

This breaks the oe-selftest devtool tests:

https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/46751/raw

oe-selftest -r devtool

should reproduce.

Cheers,

Richard

Re: [PATCH] devtool-source.bbclass: Only create each patch branch once

[Olof Johansson]

On 18-10-11 10:34 +0100, Richard Purdie wrote:
> This breaks the oe-selftest devtool tests:
> 
> https://autobuilder.yoctoproject.org/typhoon/api/v2/logs/46751/raw
> 
> oe-selftest -r devtool
> 
> should reproduce.

Thanks! Was under the impression that the DEVTOOL_EXTRA_OVERRIDES
was initialized to "", but that's not the case. Should it be? A
simple fix could be to just add DEVTOOL_EXTRA_OVERRIDES ??= "" to
the top of the bbclass. Is that the most appropriate fix? Or should
I make sure to handle None values coming back from getVar?


-- 
olofjn

[PATCH] devtool-source.bbclass: Only create each patch branch once

[Olof Johansson]

For conditonally applied patches based on SRC_URI overrides, the
devtool-source class would try to create a new branch for each override
assignment as a postfunc to do_patch, but if the same override was used
multiple times, it would try to create the same branch multiple times,
causing errors like

> Exception: bb.process.ExecutionError: Execution of
\   'git checkout f0f0f0f0f0ff0f0f0f0f0f0f0f0f0f0f0ff0f0f0 -b devtool-override-foo'
\   failed with exit code 128:
> fatal: A branch named 'devtool-override-foo' already exists.

This change makes sure that the devtool-source bbclass will only create
one branch per override.

Signed-off-by: Olof Johansson <olofjn@axis.com>
---
 meta/classes/devtool-source.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
---
Peter Kjellerstedt pointed out some issues with v1; this updated change is
simpler. In addition to what Richard said about it causing failures in
oe-selftest. I had a issues running oe-selftest, mostly because of the large
storage requirements of /tmp and issues with company internal git hooks.  I
think I managed to run all the tests with passing results now, but please be
vigilante of testcase failures; sorry in advance!

diff --git a/meta/classes/devtool-source.bbclass b/meta/classes/devtool-source.bbclass
index 67cd0bafb20..1372e32c9e5 100644
--- a/meta/classes/devtool-source.bbclass
+++ b/meta/classes/devtool-source.bbclass
@@ -183,14 +183,14 @@ python devtool_post_patch() {
 
     extra_overrides = d.getVar('DEVTOOL_EXTRA_OVERRIDES')
     if extra_overrides:
-        extra_override_list = extra_overrides.split(':')
+        extra_overrides = set(extra_overrides.split(':'))
         devbranch = d.getVar('DEVTOOL_DEVBRANCH')
         default_overrides = d.getVar('OVERRIDES').split(':')
         no_overrides = []
         # First, we may have some overrides that are referred to in the recipe set in
         # our configuration, so we need to make a branch that excludes those
         for override in default_overrides:
-            if override not in extra_override_list:
+            if override not in extra_overrides:
                 no_overrides.append(override)
         if default_overrides != no_overrides:
             # Some overrides are active in the current configuration, so
@@ -208,7 +208,7 @@ python devtool_post_patch() {
         else:
             bb.process.run('git checkout %s -b devtool-no-overrides' % devbranch, cwd=srcsubdir)
 
-        for override in extra_override_list:
+        for override in extra_overrides:
             localdata = bb.data.createCopy(d)
             if override in default_overrides:
                 bb.process.run('git branch devtool-override-%s %s' % (override, devbranch), cwd=srcsubdir)
-- 
2.11.0

Re: [PATCH] devtool-source.bbclass: Only create each patch branch once

[Olof Johansson]

On 18-10-19 18:00 +0200, Olof Johansson wrote:
> For conditonally applied patches based on SRC_URI overrides, the
> devtool-source class would try to create a new branch for each override
> assignment as a postfunc to do_patch, but if the same override was used
> multiple times, it would try to create the same branch multiple times,

Oops, this should have been a v2; should i repost? I'll repost
now, but let me know if this is redundant.

-- 
olofjn