All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES
@ 2020-07-23  9:51 Khasim Mohammed
  2020-07-24  0:58 ` [OE-core] " Lee Chee Yang
  2020-08-16 17:58 ` Khasim Mohammed
  0 siblings, 2 replies; 4+ messages in thread
From: Khasim Mohammed @ 2020-07-23  9:51 UTC (permalink / raw)
  To: openembedded-core; +Cc: nd, Khasim Syed Mohammed

List of files defined using IMAGE_BOOT_FILES are installed into
the boot partition when preparing an image using the wic tool with
the bootimg-efi source plugin.

The code snippet introduced is taken as is from bootimg-partition.py

Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782
Issue-Id: PLATFORMS-3134
Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com>
---
 scripts/lib/wic/plugins/source/bootimg-efi.py | 60 +++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 2cfdc10ecd..14c1723577 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -13,6 +13,9 @@
 import logging
 import os
 import shutil
+import re
+
+from glob import glob
 
 from wic import WicError
 from wic.engine import get_custom_config
@@ -209,6 +212,57 @@ class BootimgEFIPlugin(SourcePlugin):
         except KeyError:
             raise WicError("bootimg-efi requires a loader, none specified")
 
+        if get_bitbake_var("IMAGE_BOOT_FILES") is None:
+            logger.debug('No boot files defined in IMAGE_BOOT_FILES')
+        else:
+            boot_files = None
+            for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), (None, None)):
+                if fmt:
+                    var = fmt % id
+                else:
+                    var = ""
+
+                boot_files = get_bitbake_var("IMAGE_BOOT_FILES" + var)
+                if boot_files:
+                    break
+
+            logger.debug('Boot files: %s', boot_files)
+
+            # list of tuples (src_name, dst_name)
+            deploy_files = []
+            for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
+                if ';' in src_entry:
+                    dst_entry = tuple(src_entry.split(';'))
+                    if not dst_entry[0] or not dst_entry[1]:
+                        raise WicError('Malformed boot file entry: %s' % src_entry)
+                else:
+                    dst_entry = (src_entry, src_entry)
+
+                logger.debug('Destination entry: %r', dst_entry)
+                deploy_files.append(dst_entry)
+
+            cls.install_task = [];
+            for deploy_entry in deploy_files:
+                src, dst = deploy_entry
+                if '*' in src:
+                    # by default install files under their basename
+                    entry_name_fn = os.path.basename
+                    if dst != src:
+                        # unless a target name was given, then treat name
+                        # as a directory and append a basename
+                        entry_name_fn = lambda name: \
+                                        os.path.join(dst,
+                                                     os.path.basename(name))
+
+                    srcs = glob(os.path.join(kernel_dir, src))
+
+                    logger.debug('Globbed sources: %s', ', '.join(srcs))
+                    for entry in srcs:
+                        src = os.path.relpath(entry, kernel_dir)
+                        entry_dst_name = entry_name_fn(entry)
+                        cls.install_task.append((src, entry_dst_name))
+                else:
+                    cls.install_task.append((src, dst))
 
     @classmethod
     def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
@@ -238,6 +292,12 @@ class BootimgEFIPlugin(SourcePlugin):
             (staging_kernel_dir, kernel, hdddir, kernel)
         exec_cmd(install_cmd)
 
+        if get_bitbake_var("IMAGE_BOOT_FILES"):
+            for src_path, dst_path in cls.install_task:
+                install_cmd = "install -m 0644 -D %s %s" \
+                              % (os.path.join(kernel_dir, src_path),
+                                 os.path.join(hdddir, dst_path))
+                exec_cmd(install_cmd)
 
         try:
             if source_params['loader'] == 'grub-efi':
-- 
2.17.1


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: [OE-core] [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES
  2020-07-23  9:51 [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES Khasim Mohammed
@ 2020-07-24  0:58 ` Lee Chee Yang
       [not found]   ` <DB8PR08MB4969882A75AE20A8E92E93BFFA770@DB8PR08MB4969.eurprd08.prod.outlook.com>
  2020-08-16 17:58 ` Khasim Mohammed
  1 sibling, 1 reply; 4+ messages in thread
From: Lee Chee Yang @ 2020-07-24  0:58 UTC (permalink / raw)
  To: Khasim Mohammed, openembedded-core; +Cc: nd

Hi,
I think currently IMAGE_BOOT_FILES test run on bootimg-partition only. 
Could you add test cases for bootimg-efi too?

-----Original Message-----
From: openembedded-core@lists.openembedded.org <openembedded-core@lists.openembedded.org> On Behalf Of Khasim Mohammed
Sent: Thursday, 23 July, 2020 5:52 PM
To: openembedded-core@lists.openembedded.org
Cc: nd@arm.com; Khasim Syed Mohammed <khasim.mohammed@arm.com>
Subject: [OE-core] [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES

List of files defined using IMAGE_BOOT_FILES are installed into the boot partition when preparing an image using the wic tool with the bootimg-efi source plugin.

The code snippet introduced is taken as is from bootimg-partition.py

Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782
Issue-Id: PLATFORMS-3134
Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com>
---
 scripts/lib/wic/plugins/source/bootimg-efi.py | 60 +++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 2cfdc10ecd..14c1723577 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -13,6 +13,9 @@
 import logging
 import os
 import shutil
+import re
+
+from glob import glob
 
 from wic import WicError
 from wic.engine import get_custom_config @@ -209,6 +212,57 @@ class BootimgEFIPlugin(SourcePlugin):
         except KeyError:
             raise WicError("bootimg-efi requires a loader, none specified")
 
+        if get_bitbake_var("IMAGE_BOOT_FILES") is None:
+            logger.debug('No boot files defined in IMAGE_BOOT_FILES')
+        else:
+            boot_files = None
+            for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), (None, None)):
+                if fmt:
+                    var = fmt % id
+                else:
+                    var = ""
+
+                boot_files = get_bitbake_var("IMAGE_BOOT_FILES" + var)
+                if boot_files:
+                    break
+
+            logger.debug('Boot files: %s', boot_files)
+
+            # list of tuples (src_name, dst_name)
+            deploy_files = []
+            for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
+                if ';' in src_entry:
+                    dst_entry = tuple(src_entry.split(';'))
+                    if not dst_entry[0] or not dst_entry[1]:
+                        raise WicError('Malformed boot file entry: %s' % src_entry)
+                else:
+                    dst_entry = (src_entry, src_entry)
+
+                logger.debug('Destination entry: %r', dst_entry)
+                deploy_files.append(dst_entry)
+
+            cls.install_task = [];
+            for deploy_entry in deploy_files:
+                src, dst = deploy_entry
+                if '*' in src:
+                    # by default install files under their basename
+                    entry_name_fn = os.path.basename
+                    if dst != src:
+                        # unless a target name was given, then treat name
+                        # as a directory and append a basename
+                        entry_name_fn = lambda name: \
+                                        os.path.join(dst,
+                                                     
+ os.path.basename(name))
+
+                    srcs = glob(os.path.join(kernel_dir, src))
+
+                    logger.debug('Globbed sources: %s', ', '.join(srcs))
+                    for entry in srcs:
+                        src = os.path.relpath(entry, kernel_dir)
+                        entry_dst_name = entry_name_fn(entry)
+                        cls.install_task.append((src, entry_dst_name))
+                else:
+                    cls.install_task.append((src, dst))
 
     @classmethod
     def do_prepare_partition(cls, part, source_params, creator, cr_workdir, @@ -238,6 +292,12 @@ class BootimgEFIPlugin(SourcePlugin):
             (staging_kernel_dir, kernel, hdddir, kernel)
         exec_cmd(install_cmd)
 
+        if get_bitbake_var("IMAGE_BOOT_FILES"):
+            for src_path, dst_path in cls.install_task:
+                install_cmd = "install -m 0644 -D %s %s" \
+                              % (os.path.join(kernel_dir, src_path),
+                                 os.path.join(hdddir, dst_path))
+                exec_cmd(install_cmd)
 
         try:
             if source_params['loader'] == 'grub-efi':
--
2.17.1


^ permalink raw reply related	[flat|nested] 4+ messages in thread

* Re: Fw: [OE-core] [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES
       [not found]   ` <DB8PR08MB4969882A75AE20A8E92E93BFFA770@DB8PR08MB4969.eurprd08.prod.outlook.com>
@ 2020-07-24  6:46     ` Khasim Mohammed
  0 siblings, 0 replies; 4+ messages in thread
From: Khasim Mohammed @ 2020-07-24  6:46 UTC (permalink / raw)
  To: openembedded-core

Hi Lee,

On Fri, 2020-07-24 at 12:12 +0530, Khasim Mohammed wrote:
>
> ________________________________________
> From: openembedded-core@lists.openembedded.org <
> openembedded-core@lists.openembedded.org> on behalf of Lee Chee Yang
> via lists.openembedded.org <
> chee.yang.lee=intel.com@lists.openembedded.org>
> Sent: Friday, July 24, 2020 6:28 AM
> To: Khasim Mohammed; openembedded-core@lists.openembedded.org
> Cc: nd
> Subject: Re: [OE-core] [PATCH] wic/bootimg-efi: Add support for
> IMAGE_BOOT_FILES
>
> Hi,
> I think currently IMAGE_BOOT_FILES test run on bootimg-partition
> only.
yes.

> Could you add test cases for bootimg-efi too?
>

I posted a separate patch to test IMAGE_BOOT_FILES for bootimg-efi.

Thanks.

Regards,
Khasim

> -----Original Message-----
> From: openembedded-core@lists.openembedded.org <
> openembedded-core@lists.openembedded.org> On Behalf Of Khasim
> Mohammed
> Sent: Thursday, 23 July, 2020 5:52 PM
> To: openembedded-core@lists.openembedded.org
> Cc: nd@arm.com; Khasim Syed Mohammed <khasim.mohammed@arm.com>
> Subject: [OE-core] [PATCH] wic/bootimg-efi: Add support for
> IMAGE_BOOT_FILES
>
> List of files defined using IMAGE_BOOT_FILES are installed into the
> boot partition when preparing an image using the wic tool with the
> bootimg-efi source plugin.
>
> The code snippet introduced is taken as is from bootimg-partition.py
>
> Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782
> Issue-Id: PLATFORMS-3134
> Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com>
> ---
>  scripts/lib/wic/plugins/source/bootimg-efi.py | 60
> +++++++++++++++++++
>  1 file changed, 60 insertions(+)
>
> diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py
> b/scripts/lib/wic/plugins/source/bootimg-efi.py
> index 2cfdc10ecd..14c1723577 100644
> --- a/scripts/lib/wic/plugins/source/bootimg-efi.py
> +++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
> @@ -13,6 +13,9 @@
>  import logging
>  import os
>  import shutil
> +import re
> +
> +from glob import glob
>
>  from wic import WicError
>  from wic.engine import get_custom_config @@ -209,6 +212,57 @@ class
> BootimgEFIPlugin(SourcePlugin):
>          except KeyError:
>              raise WicError("bootimg-efi requires a loader, none
> specified")
>
> +        if get_bitbake_var("IMAGE_BOOT_FILES") is None:
> +            logger.debug('No boot files defined in
> IMAGE_BOOT_FILES')
> +        else:
> +            boot_files = None
> +            for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s",
> part.label), (None, None)):
> +                if fmt:
> +                    var = fmt % id
> +                else:
> +                    var = ""
> +
> +                boot_files = get_bitbake_var("IMAGE_BOOT_FILES" +
> var)
> +                if boot_files:
> +                    break
> +
> +            logger.debug('Boot files: %s', boot_files)
> +
> +            # list of tuples (src_name, dst_name)
> +            deploy_files = []
> +            for src_entry in re.findall(r'[\w;\-\./\*]+',
> boot_files):
> +                if ';' in src_entry:
> +                    dst_entry = tuple(src_entry.split(';'))
> +                    if not dst_entry[0] or not dst_entry[1]:
> +                        raise WicError('Malformed boot file entry:
> %s' % src_entry)
> +                else:
> +                    dst_entry = (src_entry, src_entry)
> +
> +                logger.debug('Destination entry: %r', dst_entry)
> +                deploy_files.append(dst_entry)
> +
> +            cls.install_task = [];
> +            for deploy_entry in deploy_files:
> +                src, dst = deploy_entry
> +                if '*' in src:
> +                    # by default install files under their basename
> +                    entry_name_fn = os.path.basename
> +                    if dst != src:
> +                        # unless a target name was given, then treat
> name
> +                        # as a directory and append a basename
> +                        entry_name_fn = lambda name: \
> +                                        os.path.join(dst,
> +
> + os.path.basename(name))
> +
> +                    srcs = glob(os.path.join(kernel_dir, src))
> +
> +                    logger.debug('Globbed sources: %s', ',
> '.join(srcs))
> +                    for entry in srcs:
> +                        src = os.path.relpath(entry, kernel_dir)
> +                        entry_dst_name = entry_name_fn(entry)
> +                        cls.install_task.append((src,
> entry_dst_name))
> +                else:
> +                    cls.install_task.append((src, dst))
>
>      @classmethod
>      def do_prepare_partition(cls, part, source_params, creator,
> cr_workdir, @@ -238,6 +292,12 @@ class
> BootimgEFIPlugin(SourcePlugin):
>              (staging_kernel_dir, kernel, hdddir, kernel)
>          exec_cmd(install_cmd)
>
> +        if get_bitbake_var("IMAGE_BOOT_FILES"):
> +            for src_path, dst_path in cls.install_task:
> +                install_cmd = "install -m 0644 -D %s %s" \
> +                              % (os.path.join(kernel_dir, src_path),
> +                                 os.path.join(hdddir, dst_path))
> +                exec_cmd(install_cmd)
>
>          try:
>              if source_params['loader'] == 'grub-efi':
> --
> 2.17.1
>

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES
  2020-07-23  9:51 [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES Khasim Mohammed
  2020-07-24  0:58 ` [OE-core] " Lee Chee Yang
@ 2020-08-16 17:58 ` Khasim Mohammed
  1 sibling, 0 replies; 4+ messages in thread
From: Khasim Mohammed @ 2020-08-16 17:58 UTC (permalink / raw)
  To: openembedded-core; +Cc: nd

Hi,

On Thu, 2020-07-23 at 15:21 +0530, Khasim Syed Mohammed wrote:
> List of files defined using IMAGE_BOOT_FILES are installed into
> the boot partition when preparing an image using the wic tool with
> the bootimg-efi source plugin.
> 
> The code snippet introduced is taken as is from bootimg-partition.py
> 
> Change-Id: I8dbb6b4e7c24870f587a6f31e6ae4a87d7033782
> Issue-Id: PLATFORMS-3134
> Signed-off-by: Khasim Syed Mohammed <khasim.mohammed@arm.com>
> ---
>  scripts/lib/wic/plugins/source/bootimg-efi.py | 60
> +++++++++++++++++++
>  1 file changed, 60 insertions(+)
> 
> diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py
> b/scripts/lib/wic/plugins/source/bootimg-efi.py
> index 2cfdc10ecd..14c1723577 100644
> --- a/scripts/lib/wic/plugins/source/bootimg-efi.py
> +++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
> @@ -13,6 +13,9 @@
>  import logging
>  import os
>  import shutil
> +import re
> +
> +from glob import glob
>  
>  from wic import WicError
>  from wic.engine import get_custom_config
> @@ -209,6 +212,57 @@ class BootimgEFIPlugin(SourcePlugin):
>          except KeyError:
>              raise WicError("bootimg-efi requires a loader, none
> specified")
>  
> +        if get_bitbake_var("IMAGE_BOOT_FILES") is None:
> +            logger.debug('No boot files defined in
> IMAGE_BOOT_FILES')
> +        else:
> +            boot_files = None
> +            for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", 
> part.label), (None, None)):
> +                if fmt:
> +                    var = fmt % id
> +                else:
> +                    var = ""
> +
> +                boot_files = get_bitbake_var("IMAGE_BOOT_FILES" +
> var)
> +                if boot_files:
> +                    break
> +
> +            logger.debug('Boot files: %s', boot_files)
> +
> +            # list of tuples (src_name, dst_name)
> +            deploy_files = []
> +            for src_entry in re.findall(r'[\w;\-\./\*]+',
> boot_files):
> +                if ';' in src_entry:
> +                    dst_entry = tuple(src_entry.split(';'))
> +                    if not dst_entry[0] or not dst_entry[1]:
> +                        raise WicError('Malformed boot file entry:
> %s' % src_entry)
> +                else:
> +                    dst_entry = (src_entry, src_entry)
> +
> +                logger.debug('Destination entry: %r', dst_entry)
> +                deploy_files.append(dst_entry)
> +
> +            cls.install_task = [];
> +            for deploy_entry in deploy_files:
> +                src, dst = deploy_entry
> +                if '*' in src:
> +                    # by default install files under their basename
> +                    entry_name_fn = os.path.basename
> +                    if dst != src:
> +                        # unless a target name was given, then treat
> name
> +                        # as a directory and append a basename
> +                        entry_name_fn = lambda name: \
> +                                        os.path.join(dst,
> +                                                     os.path.basenam
> e(name))
> +
> +                    srcs = glob(os.path.join(kernel_dir, src))
> +
> +                    logger.debug('Globbed sources: %s', ',
> '.join(srcs))
> +                    for entry in srcs:
> +                        src = os.path.relpath(entry, kernel_dir)
> +                        entry_dst_name = entry_name_fn(entry)
> +                        cls.install_task.append((src,
> entry_dst_name))
> +                else:
> +                    cls.install_task.append((src, dst))
>  
>      @classmethod
>      def do_prepare_partition(cls, part, source_params, creator,
> cr_workdir,
> @@ -238,6 +292,12 @@ class BootimgEFIPlugin(SourcePlugin):
>              (staging_kernel_dir, kernel, hdddir, kernel)
>          exec_cmd(install_cmd)
>  
> +        if get_bitbake_var("IMAGE_BOOT_FILES"):
> +            for src_path, dst_path in cls.install_task:
> +                install_cmd = "install -m 0644 -D %s %s" \
> +                              % (os.path.join(kernel_dir, src_path),
> +                                 os.path.join(hdddir, dst_path))
> +                exec_cmd(install_cmd)
>  
>          try:
>              if source_params['loader'] == 'grub-efi':

This patch was recently merged to master 
https://git.openembedded.org/openembedded-core/commit/scripts/lib/wic/plugins/source/bootimg-efi.py?id=a44ab3a4ee5b3c57812909c6194456f299d6ba7f

The IMAGE_BOOT_FILES feature is important for N1SDP ARM Platform and
it's used to copy few DTBs to boot partition. Without this feature the
wic image for N1SDP will support only ACPI boot and it will be broken
for DTB based booting, as this feature is working on master its only
missing on dunfell branch.

I hereby request to consider this patch for dunfell branch as well. It
will help ARM partners to use WIC image for both ACPI and DTBs based
booting and avoid customizing the build for N1SDP.

Let me know if you find any issues in merging this patch to dunfell
branch.

Thanks

Regards,
Khasim


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2020-08-16 17:58 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-07-23  9:51 [PATCH] wic/bootimg-efi: Add support for IMAGE_BOOT_FILES Khasim Mohammed
2020-07-24  0:58 ` [OE-core] " Lee Chee Yang
     [not found]   ` <DB8PR08MB4969882A75AE20A8E92E93BFFA770@DB8PR08MB4969.eurprd08.prod.outlook.com>
2020-07-24  6:46     ` Fw: " Khasim Mohammed
2020-08-16 17:58 ` Khasim Mohammed

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.