[PATCH] Support add-layer to prepend

[PATCH] Support add-layer to prepend

[nvhieudt11]

[-- Attachment #1: Type: text/plain, Size: 2620 bytes --]

As you know, layer order in BBLAYERS can affect in parsing recipes process,
in some cases, users want to add a layer on the top of BBLAYERS variable

So, add "--prepend" option for bitbake-layers to support add-layer to prepend

Signed-off-by: Hieu Van Nguyen <hieu2.nguyen@lge.com>
---
lib/bb/utils.py        | 7 +++++--
lib/bblayers/action.py | 3 ++-
2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lib/bb/utils.py b/lib/bb/utils.py
index 068b631c..086f1cc8 100644
--- a/lib/bb/utils.py
+++ b/lib/bb/utils.py
@@ -1412,7 +1412,7 @@ def edit_metadata_file(meta_file, variables, varfunc):
return updated

-def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
+def edit_bblayers_conf(bblayers_conf, add, remove, prepend=None, edit_cb=None):
"""Edit bblayers.conf, adding and/or removing layers
Parameters:
bblayers_conf: path to bblayers.conf file to edit
@@ -1482,7 +1482,10 @@ def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
for addlayer in addlayers:
if addlayer not in bblayers:
updated = True
-                    bblayers.append(addlayer)
+                    if prepend:
+                        bblayers.insert(0,addlayer)
+                    else:
+                        bblayers.append(addlayer)
del addlayers[:]

if edit_cb:
diff --git a/lib/bblayers/action.py b/lib/bblayers/action.py
index a8f26993..57d7195c 100644
--- a/lib/bblayers/action.py
+++ b/lib/bblayers/action.py
@@ -49,7 +49,7 @@ class ActionPlugin(LayerPlugin):
shutil.copy2(bblayers_conf, backup)

try:
-            notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None)
+            notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None, args.prepend)
self.tinfoil.modified_files()
if not (args.force or notadded):
try:
@@ -267,6 +267,7 @@ build results (as the layer priority order has effectively changed).
def register_commands(self, sp):
parser_add_layer = self.add_command(sp, 'add-layer', self.do_add_layer, parserecipes=False)
parser_add_layer.add_argument('layerdir', nargs='+', help='Layer directory/directories to add')
+        parser_add_layer.add_argument('--prepend', action='store_true', help='Prepend layer directory/directories')

parser_remove_layer = self.add_command(sp, 'remove-layer', self.do_remove_layer, parserecipes=False)
parser_remove_layer.add_argument('layerdir', nargs='+', help='Layer directory/directories to remove (wildcards allowed, enclose in quotes to avoid shell expansion)')
--
2.43.0

[-- Attachment #2: Type: text/html, Size: 4389 bytes --]

Re: [bitbake-devel] [PATCH] Support add-layer to prepend

[Alexandre Belloni]

Hello,

Your patch doesn't apply, it is missing whitespaces at the beginning of
lines. Can you fix and resend?

Thanks!

On 05/02/2024 03:31:13-0800, nvhieudt11@gmail.com wrote:
> As you know, layer order in BBLAYERS can affect in parsing recipes process,
> in some cases, users want to add a layer on the top of BBLAYERS variable
> 
> So, add "--prepend" option for bitbake-layers to support add-layer to prepend
> 
> Signed-off-by: Hieu Van Nguyen <hieu2.nguyen@lge.com>
> ---
> lib/bb/utils.py� � � � | 7 +++++--
> lib/bblayers/action.py | 3 ++-
> 2 files changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/bb/utils.py b/lib/bb/utils.py
> index 068b631c..086f1cc8 100644
> --- a/lib/bb/utils.py
> +++ b/lib/bb/utils.py
> @@ -1412,7 +1412,7 @@ def edit_metadata_file(meta_file, variables, varfunc):
> return updated
> 
> -def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
> +def edit_bblayers_conf(bblayers_conf, add, remove, prepend=None, edit_cb=None):
> """Edit bblayers.conf, adding and/or removing layers
> Parameters:
> bblayers_conf: path to bblayers.conf file to edit
> @@ -1482,7 +1482,10 @@ def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
> for addlayer in addlayers:
> if addlayer not in bblayers:
> updated = True
> -� � � � � � � � � � bblayers.append(addlayer)
> +� � � � � � � � � � if prepend:
> +� � � � � � � � � � � � bblayers.insert(0,addlayer)
> +� � � � � � � � � � else:
> +� � � � � � � � � � � � bblayers.append(addlayer)
> del addlayers[:]
> 
> if edit_cb:
> diff --git a/lib/bblayers/action.py b/lib/bblayers/action.py
> index a8f26993..57d7195c 100644
> --- a/lib/bblayers/action.py
> +++ b/lib/bblayers/action.py
> @@ -49,7 +49,7 @@ class ActionPlugin(LayerPlugin):
> shutil.copy2(bblayers_conf, backup)
> 
> try:
> -� � � � � � notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None)
> +� � � � � � notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None, args.prepend)
> self.tinfoil.modified_files()
> if not (args.force or notadded):
> try:
> @@ -267,6 +267,7 @@ build results (as the layer priority order has effectively changed).
> def register_commands(self, sp):
> parser_add_layer = self.add_command(sp, 'add-layer', self.do_add_layer, parserecipes=False)
> parser_add_layer.add_argument('layerdir', nargs='+', help='Layer directory/directories to add')
> +� � � � parser_add_layer.add_argument('--prepend', action='store_true', help='Prepend layer directory/directories')
> 
> parser_remove_layer = self.add_command(sp, 'remove-layer', self.do_remove_layer, parserecipes=False)
> parser_remove_layer.add_argument('layerdir', nargs='+', help='Layer directory/directories to remove (wildcards allowed, enclose in quotes to avoid shell expansion)')
> --
> 2.43.0

> 
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#15823): https://lists.openembedded.org/g/bitbake-devel/message/15823
> Mute This Topic: https://lists.openembedded.org/mt/104173959/3617179
> Group Owner: bitbake-devel+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
> 


-- 
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [PATCH] Support add-layer to prepend

[nvhieudt11]

[-- Attachment #1.1: Type: text/plain, Size: 364 bytes --]

Hi Alexandre Belloni,

Sorry for the late reply.
I'm not sure why you can't apply my patch. I tried applying it again and there was no problem.
I'd like to attach my patch, could you please check it again (0001-bitbake-Support-add-layer-to-prepend.patch)
https://github.com/hieunv0828/bitbake/commit/fc670b7ba577958eb18b108458f24fd2c260e225

Thanks!
Hieu

[-- Attachment #1.2: Type: text/html, Size: 585 bytes --]

[-- Attachment #2: 0001-bitbake-Support-add-layer-to-prepend.patch --]
[-- Type: application/octet-stream, Size: 2972 bytes --]

From fc670b7ba577958eb18b108458f24fd2c260e225 Mon Sep 17 00:00:00 2001
From: Hieu Van Nguyen <hieu2.nguyen@lge.com>
Date: Mon, 5 Feb 2024 08:01:43 +0000
Subject: [PATCH] bitbake: Support add-layer to prepend

As you know, layer order in BBLAYERS can affect in parsing recipes process,
in some cases, users want to add a layer on the top of BBLAYERS variable

So, add "--prepend" option for bitbake-layers to support add-layer to prepend

Signed-off-by: Hieu Van Nguyen <hieu2.nguyen@lge.com>
---
 lib/bb/utils.py        | 7 +++++--
 lib/bblayers/action.py | 3 ++-
 2 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lib/bb/utils.py b/lib/bb/utils.py
index 068b631c..086f1cc8 100644
--- a/lib/bb/utils.py
+++ b/lib/bb/utils.py
@@ -1412,7 +1412,7 @@ def edit_metadata_file(meta_file, variables, varfunc):
     return updated
 
 
-def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
+def edit_bblayers_conf(bblayers_conf, add, remove, prepend=None, edit_cb=None):
     """Edit bblayers.conf, adding and/or removing layers
     Parameters:
         bblayers_conf: path to bblayers.conf file to edit
@@ -1482,7 +1482,10 @@ def edit_bblayers_conf(bblayers_conf, add, remove, edit_cb=None):
             for addlayer in addlayers:
                 if addlayer not in bblayers:
                     updated = True
-                    bblayers.append(addlayer)
+                    if prepend:
+                        bblayers.insert(0,addlayer)
+                    else:
+                        bblayers.append(addlayer)
             del addlayers[:]
 
         if edit_cb:
diff --git a/lib/bblayers/action.py b/lib/bblayers/action.py
index a8f26993..57d7195c 100644
--- a/lib/bblayers/action.py
+++ b/lib/bblayers/action.py
@@ -49,7 +49,7 @@ class ActionPlugin(LayerPlugin):
         shutil.copy2(bblayers_conf, backup)
 
         try:
-            notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None)
+            notadded, _ = bb.utils.edit_bblayers_conf(bblayers_conf, layerdirs, None, args.prepend)
             self.tinfoil.modified_files()
             if not (args.force or notadded):
                 try:
@@ -267,6 +267,7 @@ build results (as the layer priority order has effectively changed).
     def register_commands(self, sp):
         parser_add_layer = self.add_command(sp, 'add-layer', self.do_add_layer, parserecipes=False)
         parser_add_layer.add_argument('layerdir', nargs='+', help='Layer directory/directories to add')
+        parser_add_layer.add_argument('--prepend', action='store_true', help='Prepend layer directory/directories')
 
         parser_remove_layer = self.add_command(sp, 'remove-layer', self.do_remove_layer, parserecipes=False)
         parser_remove_layer.add_argument('layerdir', nargs='+', help='Layer directory/directories to remove (wildcards allowed, enclose in quotes to avoid shell expansion)')
-- 
2.43.0

Re: [bitbake-devel] [PATCH] Support add-layer to prepend

[Richard Purdie]

On Mon, 2024-02-05 at 03:31 -0800, nvhieudt11@gmail.com wrote:
> As you know, layer order in BBLAYERS can affect in parsing recipes process,
> in some cases, users want to add a layer on the top of BBLAYERS variable
>  
> So, add "--prepend" option for bitbake-layers to support add-layer to prepend
>  
> Signed-off-by: Hieu Van Nguyen <hieu2.nguyen@lge.com>
> ---
>  lib/bb/utils.py        | 7 +++++--
>  lib/bblayers/action.py | 3 ++-
>  2 files changed, 7 insertions(+), 3 deletions(-)


Unfortunately this patch has caused some really obscure build failures e.g.:

https://autobuilder.yoctoproject.org/typhoon/#/builders/80/builds/6487/steps/14/logs/stdio
and
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/6519/steps/14/logs/stdio

To prove it, you can in parallel run:

inotifywait . -m -e create -e access  ./recipes-devtools/makedevs/ ./recipes-devtools/ ./recipes-devtools/makedevs/makedevs/

on the meta directory and then in a different terminal:

oe-selftest -r devtool.DevtoolUpdateTests.test_devtool_update_recipe_local_files -j 2 -K

and what you'll see is:

./recipes-devtools/makedevs/makedevs/ CREATE makedevs.c
./recipes-devtools/makedevs/ CREATE makedevs_1.0.1.bb

amongst the accesses.

The devtool test code (meta/lib/oeqa/selftest/cases/devtool.py)
isolates it's changes to a copy of the meta directory using the
function you're chaging but this change stops that copy from working
correctly and breaks that isolation.

Cheers,

Richard

Re: [PATCH] Support add-layer to prepend

[nvhieudt11]

[-- Attachment #1: Type: text/plain, Size: 271 bytes --]

Hi Richard,

I updated my commit and run oe-selftest as your suggestion, All required tests passed.

Here is my updated commit: https://github.com/hieunv0828/bitbake/commit/d8793aa3641fb0a47e0cd7b4c141a4c14a18006c.patch
Please help me check again.

Thanks,
Hieu

[-- Attachment #2: Type: text/html, Size: 608 bytes --]

RE: [bitbake-devel] [PATCH] Support add-layer to prepend

[Peter Kjellerstedt]

[-- Attachment #1: Type: text/plain, Size: 631 bytes --]

Please change the commit subject prefix from “bitbake:” to “bitbake-layers:”.

//Peter

From: bitbake-devel@lists.openembedded.org <bitbake-devel@lists.openembedded.org> On Behalf Of nvhieudt11@gmail.com
Sent: den 28 mars 2024 07:50
To: bitbake-devel@lists.openembedded.org
Subject: Re: [bitbake-devel] [PATCH] Support add-layer to prepend


Hi Richard,

I updated my commit and run oe-selftest as your suggestion, All required tests passed.

Here is my updated commit: https://github.com/hieunv0828/bitbake/commit/d8793aa3641fb0a47e0cd7b4c141a4c14a18006c.patch
Please help me check again.

Thanks,
Hieu

[-- Attachment #2: Type: text/html, Size: 3248 bytes --]

Re: [PATCH] Support add-layer to prepend

[nvhieudt11]

[-- Attachment #1: Type: text/plain, Size: 176 bytes --]

Hi Peter,

This is my update: https://github.com/hieunv0828/bitbake/commit/832eb2763cca72103dfc92716a46356f2061f7bf.patch
Could you please check it again.

Thanks,
Hieu

[-- Attachment #2: Type: text/html, Size: 342 bytes --]