* [PATCH 0/4] add wic based image installer
@ 2019-05-08 6:40 chee.yang.lee
2019-05-08 6:40 ` [PATCH 1/4] wic: add new installer-partition plugin chee.yang.lee
` (5 more replies)
0 siblings, 6 replies; 13+ messages in thread
From: chee.yang.lee @ 2019-05-08 6:40 UTC (permalink / raw)
To: openembedded-core
From: Chee Yang Lee <chee.yang.lee@intel.com>
Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image.
2 partition needed for installable image
- first partition build using source plugin bootimg-efi and configure to run install
- second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition)
These patches:
- add new source plugin for second partition
- add new .wks
- add new wic dependency and set default value for required variable
- allow source plugin bootimg-efi to configure to install
To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf.
Chee Yang Lee (4):
wic: add new installer-partition plugin
wic: bootimg-efi: add label source parameter
wic: add new wic-installer.wks for wic based image installer
image_types_wic: add dependency for wic based image installer
meta/classes/image_types_wic.bbclass | 6 +
scripts/lib/wic/canned-wks/wic-installer.wks.in | 7 +
scripts/lib/wic/plugins/source/bootimg-efi.py | 20 ++-
.../lib/wic/plugins/source/installer-partition.py | 192 +++++++++++++++++++++
4 files changed, 220 insertions(+), 5 deletions(-)
create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in
create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py
--
2.7.4
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH 1/4] wic: add new installer-partition plugin 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee @ 2019-05-08 6:40 ` chee.yang.lee 2019-05-08 6:41 ` [PATCH 2/4] wic: bootimg-efi: add label source parameter chee.yang.lee ` (4 subsequent siblings) 5 siblings, 0 replies; 13+ messages in thread From: chee.yang.lee @ 2019-05-08 6:40 UTC (permalink / raw) To: openembedded-core From: Chee Yang Lee <chee.yang.lee@intel.com> This patch implement 'installer-partition.py' source plugin for wic. The plugin create an image with systemd-boot and rootfs. The generated image can be use to create wic based image installer. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> --- .../lib/wic/plugins/source/installer-partition.py | 192 +++++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py diff --git a/scripts/lib/wic/plugins/source/installer-partition.py b/scripts/lib/wic/plugins/source/installer-partition.py new file mode 100644 index 0000000..4d8343f --- /dev/null +++ b/scripts/lib/wic/plugins/source/installer-partition.py @@ -0,0 +1,192 @@ +# Copyright (c) 2019, Intel Corporation. +# All rights reserved. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. +# +# DESCRIPTION +# This implements the 'installer-partition' source plugin class for 'wic' +# This plugin prepare partition with content to be install on target machine. +# +# AUTHORS +# Lee Chee Yang <Chee.Yang.Lee (at] intel.com> +# + +import logging +import os + +from wic import WicError +from wic.engine import get_custom_config +from wic.pluginbase import SourcePlugin +from wic.misc import (exec_cmd, exec_native_cmd, get_bitbake_var) + +logger = logging.getLogger('wic') + +class InstallerImagePlugin(SourcePlugin): + """ + Populate content for wic image based installer + """ + + name = 'installer-partition' + + @classmethod + def do_configure_partition(cls, part, source_params, creator, cr_workdir, + oe_builddir, bootimg_dir, kernel_dir, + native_sysroot): + """ + Called before do_prepare_partition(), creates loader-specific config + """ + if not kernel_dir: + kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE") + if not kernel_dir: + raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") + staging_kernel_dir = kernel_dir + + partition_dir = "%s/%s-%s" % (cr_workdir, part.label, part.lineno) + + install_cmd = "install -d %s/EFI/BOOT" % partition_dir + exec_cmd(install_cmd) + + install_cmd = "install -d %s/loader/entries" % partition_dir + exec_cmd(install_cmd) + + bootloader = creator.ks.bootloader + + loader_conf = "" + loader_conf += "default boot\n" + loader_conf += "timeout %d\n" % bootloader.timeout + + initrd = source_params.get('initrd') + + if initrd: + cp_cmd = "cp %s/%s %s" % (kernel_dir, initrd, partition_dir) + exec_cmd(cp_cmd, True) + else: + logger.debug("Ignoring missing initrd") + + logger.debug("Writing systemd-boot config " + "%s/loader/loader.conf", partition_dir) + cfg = open("%s/loader/loader.conf" % partition_dir, "w") + cfg.write(loader_conf) + cfg.close() + + kernel = get_bitbake_var("KERNEL_IMAGETYPE") + if not kernel: + kernel = "bzImage" + + install_cmd = "install -m 0644 %s/%s %s/%s" % \ + (staging_kernel_dir, kernel, partition_dir, kernel) + exec_cmd(install_cmd) + + configfile = creator.ks.bootloader.configfile + custom_cfg = None + if configfile: + custom_cfg = get_custom_config(configfile) + if custom_cfg: + # Use a custom configuration for systemd-boot + boot_conf = custom_cfg + logger.debug("Using custom configuration file " + "%s for systemd-boots's boot.conf", configfile) + else: + raise WicError("configfile is specified but failed to " + "get it from %s.", configfile) + + if not custom_cfg: + # Create systemd-boot configuration using parameters from wks file + title = source_params.get('title') + + boot_conf = "" + boot_conf += "title %s\n" % (title if title else "boot") + boot_conf += "linux /%s\n" % kernel + boot_conf += "options LABEL=Boot %s\n" % (bootloader.append) + + if initrd: + boot_conf += "initrd /%s\n" % initrd + + for mod in [x for x in os.listdir(kernel_dir) if x.startswith("systemd-")]: + cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, partition_dir, mod[8:]) + exec_cmd(cp_cmd, True) + + logger.debug("Writing systemd-boot config " + "%s/loader/entries/boot.conf", partition_dir) + cfg = open("%s//loader/entries/boot.conf" % partition_dir, "w") + cfg.write(boot_conf) + cfg.close() + + + @classmethod + def do_prepare_partition(cls, part, source_params, creator, cr_workdir, + oe_builddir, bootimg_dir, kernel_dir, + rootfs_dir, native_sysroot): + """ + Called to do the actual content population for a partition, + prepare systemd bootloader and rootfs image + """ + partition_dir = "%s/%s-%s" % (cr_workdir, part.label, part.lineno) + + rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, part.label, part.lineno, part.fstype) + + image_deploy_dir = get_bitbake_var("IMGDEPLOYDIR") + if not image_deploy_dir: + raise WicError("Couldn't find IMGDEPLOYDIR, exiting") + + rootfs_img = "%s/%s.%s" % (image_deploy_dir, + get_bitbake_var("IMAGE_LINK_NAME"), part.fstype ) + if not os.path.isfile(rootfs_img): + raise WicError("Couldn't find %s, exiting" % rootfs_img) + + install_cmd = "install -m 0644 %s %s/rootfs.img" % \ + (rootfs_img, partition_dir) + exec_cmd(install_cmd) + + #look for actual size required for the partition + du_cmd = "du -ks %s" % partition_dir + out = exec_cmd(du_cmd) + part.size = cls.get_partition_size(part, int(out.split()[0])) + logger.debug("Set partition size to %d ", part.size) + + with open(rootfs, 'w') as sparse: + os.ftruncate(sparse.fileno(), part.size * 1024) + + extraopts = part.mkfs_extraopts or "-i 8192" + + label_str = "" + if part.label: + label_str = "-L %s" % part.label + + mkfs_cmd = "mkfs.%s -F %s %s -U %s %s -d %s" % \ + (part.fstype, extraopts, label_str, part.fsuuid, rootfs, partition_dir) + exec_native_cmd(mkfs_cmd, native_sysroot) + + part.source_file = rootfs + + + def get_partition_size(part, actual_partition_size=0): + """ + Calculate the required size of the partition, taking into consideration + --size/--fixed-size/--extra-space flags specified in kickstart file. + Raises an error if the `actual_partition_size` is larger than fixed-size. + """ + if part.fixed_size: + partition_size = part.fixed_size + if actual_partition_size > partition_size: + raise WicError("Actual partition size (%d kB) is larger than " + "allowed size %d kB" % + (actual_partition_size, partition_size)) + else: + extra_space = 0 + if extra_space < part.extra_space: + extra_space = part.extra_space + partition_size = actual_partition_size + extra_space + + return partition_size -- 2.7.4 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 2/4] wic: bootimg-efi: add label source parameter 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee 2019-05-08 6:40 ` [PATCH 1/4] wic: add new installer-partition plugin chee.yang.lee @ 2019-05-08 6:41 ` chee.yang.lee 2019-05-08 6:41 ` [PATCH 3/4] wic: add new wic-installer.wks for wic based image installer chee.yang.lee ` (3 subsequent siblings) 5 siblings, 0 replies; 13+ messages in thread From: chee.yang.lee @ 2019-05-08 6:41 UTC (permalink / raw) To: openembedded-core From: Chee Yang Lee <chee.yang.lee@intel.com> Add new source parameter label to allow boot.conf/grub.cfg label option to set as "install-efi", so far it's hardcoded to "Boot". The label remain to "Boot" for systemd-boot or blank for grub-efi whenever label are not "install-efi". when label set to "install-efi" the image can boot into image installation. Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> --- scripts/lib/wic/plugins/source/bootimg-efi.py | 20 +++++++++++++++----- 1 file changed, 15 insertions(+), 5 deletions(-) diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py index 83a7e18..0a0c5bd 100644 --- a/scripts/lib/wic/plugins/source/bootimg-efi.py +++ b/scripts/lib/wic/plugins/source/bootimg-efi.py @@ -86,9 +86,14 @@ class BootimgEFIPlugin(SourcePlugin): grubefi_conf += "menuentry '%s'{\n" % (title if title else "boot") kernel = "/bzImage" + label = source_params.get('label') - grubefi_conf += "linux %s root=%s rootwait %s\n" \ - % (kernel, creator.rootdev, bootloader.append) + if label == "install-efi": + grubefi_conf += "linux %s LABEL=%s rootwait %s\n" \ + % (kernel, label, bootloader.append) + else: + grubefi_conf += "linux %s root=%s rootwait %s\n" \ + % (kernel, creator.rootdev, bootloader.append) if initrd: grubefi_conf += "initrd /%s\n" % initrd @@ -154,12 +159,17 @@ class BootimgEFIPlugin(SourcePlugin): # Create systemd-boot configuration using parameters from wks file kernel = "/bzImage" title = source_params.get('title') - + label = source_params.get('label') boot_conf = "" boot_conf += "title %s\n" % (title if title else "boot") boot_conf += "linux %s\n" % kernel - boot_conf += "options LABEL=Boot root=%s %s\n" % \ - (creator.rootdev, bootloader.append) + + if label == "install-efi": + boot_conf += "options LABEL=%s %s\n" % \ + (label, bootloader.append) + else: + boot_conf += "options LABEL=Boot root=%s %s\n" % \ + (creator.rootdev, bootloader.append) if initrd: boot_conf += "initrd /%s\n" % initrd -- 2.7.4 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 3/4] wic: add new wic-installer.wks for wic based image installer 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee 2019-05-08 6:40 ` [PATCH 1/4] wic: add new installer-partition plugin chee.yang.lee 2019-05-08 6:41 ` [PATCH 2/4] wic: bootimg-efi: add label source parameter chee.yang.lee @ 2019-05-08 6:41 ` chee.yang.lee 2019-05-08 6:41 ` [PATCH 4/4] image_types_wic: add dependency " chee.yang.lee ` (2 subsequent siblings) 5 siblings, 0 replies; 13+ messages in thread From: chee.yang.lee @ 2019-05-08 6:41 UTC (permalink / raw) To: openembedded-core From: Chee Yang Lee <chee.yang.lee@intel.com> Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> --- scripts/lib/wic/canned-wks/wic-installer.wks.in | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in diff --git a/scripts/lib/wic/canned-wks/wic-installer.wks.in b/scripts/lib/wic/canned-wks/wic-installer.wks.in new file mode 100644 index 0000000..a7b018d --- /dev/null +++ b/scripts/lib/wic/canned-wks/wic-installer.wks.in @@ -0,0 +1,7 @@ +# description: Create wic installable image + +part /boot --source bootimg-efi --sourceparams="loader=systemd-boot,title=install,label=install-efi,initrd=${INITRD_IMAGE_LIVE}-${MACHINE}.${INITRAMFS_FSTYPES}" --ondisk sda --label install --active --align 1024 --use-uuid + +part / --source installer-partition --ondisk sda --fstype=ext4 --label image --use-uuid + +bootloader --ptable gpt --timeout=5 --append="rootwait console=ttyS0,115200 console=tty0" -- 2.7.4 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH 4/4] image_types_wic: add dependency for wic based image installer 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee ` (2 preceding siblings ...) 2019-05-08 6:41 ` [PATCH 3/4] wic: add new wic-installer.wks for wic based image installer chee.yang.lee @ 2019-05-08 6:41 ` chee.yang.lee 2019-05-15 15:33 ` [PATCH 0/4] add " Tom Rini 2019-05-16 1:31 ` Lee, Chee Yang 5 siblings, 0 replies; 13+ messages in thread From: chee.yang.lee @ 2019-05-08 6:41 UTC (permalink / raw) To: openembedded-core From: Chee Yang Lee <chee.yang.lee@intel.com> initramfs and rootfs image are required to build wic based image installer, add both as dependency for do_image_wic. initiate default value for LIVE_ROOTFS_TYPE, INITRD_IMAGE_LIVE and INITRD_LIVE Signed-off-by: Chee Yang Lee <chee.yang.lee@intel.com> --- meta/classes/image_types_wic.bbclass | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/meta/classes/image_types_wic.bbclass b/meta/classes/image_types_wic.bbclass index 5b40a9e..e4f2b8f 100644 --- a/meta/classes/image_types_wic.bbclass +++ b/meta/classes/image_types_wic.bbclass @@ -102,9 +102,15 @@ python () { # a variable and let the metadata deal with the deps. d.setVar('_WKS_TEMPLATE', body) bb.build.addtask('do_write_wks_template', 'do_image_wic', None, d) + d.appendVarFlag('do_image_wic', 'depends', ' %s:do_image_complete' % d.getVar('INITRD_IMAGE_LIVE')) bb.build.addtask('do_image_wic', 'do_image_complete', None, d) } +LIVE_ROOTFS_TYPE ?= "ext4" +INITRD_IMAGE_LIVE ?= "${MLPREFIX}core-image-minimal-initramfs" +INITRD_LIVE ?= "${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE_LIVE}-${MACHINE}.${INITRAMFS_FSTYPES}" +IMAGE_TYPEDEP_wic = "${LIVE_ROOTFS_TYPE}" + # # Write environment variables used by wic # to tmp/sysroots/<machine>/imgdata/<image>.env -- 2.7.4 ^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH 0/4] add wic based image installer 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee ` (3 preceding siblings ...) 2019-05-08 6:41 ` [PATCH 4/4] image_types_wic: add dependency " chee.yang.lee @ 2019-05-15 15:33 ` Tom Rini 2019-05-16 1:31 ` Lee, Chee Yang 5 siblings, 0 replies; 13+ messages in thread From: Tom Rini @ 2019-05-15 15:33 UTC (permalink / raw) To: chee.yang.lee; +Cc: openembedded-core [-- Attachment #1: Type: text/plain, Size: 1246 bytes --] On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com wrote: > From: Chee Yang Lee <chee.yang.lee@intel.com> > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image. > > 2 partition needed for installable image > - first partition build using source plugin bootimg-efi and configure to run install > - second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition) > > These patches: > - add new source plugin for second partition > - add new .wks > - add new wic dependency and set default value for required variable > - allow source plugin bootimg-efi to configure to install > > > To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf. I'm confused as to why we need most of these changes to support the end goal. The enhancement to bootimg-efi makes conceptual sense. I don't see why we need a new plugin to do the rest, however. What's being done here that we don't already do today? I say this having made installer images for EFI machines using wic (in sumo, even). Thanks! -- Tom [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/4] add wic based image installer 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee ` (4 preceding siblings ...) 2019-05-15 15:33 ` [PATCH 0/4] add " Tom Rini @ 2019-05-16 1:31 ` Lee, Chee Yang 5 siblings, 0 replies; 13+ messages in thread From: Lee, Chee Yang @ 2019-05-16 1:31 UTC (permalink / raw) To: Lee, Chee Yang, openembedded-core Any comments on these patches? -----Original Message----- From: openembedded-core-bounces@lists.openembedded.org <openembedded-core-bounces@lists.openembedded.org> On Behalf Of chee.yang.lee@intel.com Sent: Wednesday, May 8, 2019 2:41 PM To: openembedded-core@lists.openembedded.org Subject: [OE-core] [PATCH 0/4] add wic based image installer From: Chee Yang Lee <chee.yang.lee@intel.com> Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image. 2 partition needed for installable image - first partition build using source plugin bootimg-efi and configure to run install - second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition) These patches: - add new source plugin for second partition - add new .wks - add new wic dependency and set default value for required variable - allow source plugin bootimg-efi to configure to install To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf. Chee Yang Lee (4): wic: add new installer-partition plugin wic: bootimg-efi: add label source parameter wic: add new wic-installer.wks for wic based image installer image_types_wic: add dependency for wic based image installer meta/classes/image_types_wic.bbclass | 6 + scripts/lib/wic/canned-wks/wic-installer.wks.in | 7 + scripts/lib/wic/plugins/source/bootimg-efi.py | 20 ++- .../lib/wic/plugins/source/installer-partition.py | 192 +++++++++++++++++++++ 4 files changed, 220 insertions(+), 5 deletions(-) create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py -- 2.7.4 -- _______________________________________________ Openembedded-core mailing list Openembedded-core@lists.openembedded.org http://lists.openembedded.org/mailman/listinfo/openembedded-core ^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH 0/4] add wic based image installer
@ 2019-05-16 2:23 Lee, Chee Yang
2019-05-16 2:37 ` Tom Rini
0 siblings, 1 reply; 13+ messages in thread
From: Lee, Chee Yang @ 2019-05-16 2:23 UTC (permalink / raw)
To: trini; +Cc: Openembedded-core
Hi Tom,
Would you mind to share with me how you make the installer image using wic?
-------- Forwarded Message --------
From: Tom Rini <trini@konsulko.com>
To: chee.yang.lee@intel.com
Cc: openembedded-core@lists.openembedded.org
Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer
Date: Wed, 15 May 2019 11:33:43 -0400
On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com
wrote:
> From: Chee Yang Lee <chee.yang.lee@intel.com>
>
> Existing installable image (hddimg/ISO) has limit of 4GB size, so to
> build larger installable image, we can build it in wic based image.
>
> 2 partition needed for installable image
> - first partition build using source plugin bootimg-efi and
> configure to run install
> - second partition build with root.img, systemd-boot and kernel to
> be install on target machine (this partition build using new source
> plugin installer-partition)
>
> These patches:
> - add new source plugin for second partition
> - add new .wks
> - add new wic dependency and set default value for required variable
> - allow source plugin bootimg-efi to configure to install
>
>
> To build the image, set WKS_FILE="wic-installer.wks.in" in
> local.conf.
I'm confused as to why we need most of these changes to support the end
goal. The enhancement to bootimg-efi makes conceptual sense. I don't
see why we need a new plugin to do the rest, however. What's being
done
here that we don't already do today? I say this having made installer
images for EFI machines using wic (in sumo, even). Thanks!
--
Tom
--
_______________________________________________
Openembedded-core mailing list
Openembedded-core@lists.openembedded.org
http://lists.openembedded.org/mailman/listinfo/openembedded-core
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH 0/4] add wic based image installer 2019-05-16 2:23 Lee, Chee Yang @ 2019-05-16 2:37 ` Tom Rini 2019-05-16 11:48 ` Lee, Chee Yang 0 siblings, 1 reply; 13+ messages in thread From: Tom Rini @ 2019-05-16 2:37 UTC (permalink / raw) To: Lee, Chee Yang; +Cc: Openembedded-core [-- Attachment #1: Type: text/plain, Size: 2351 bytes --] On Thu, May 16, 2019 at 02:23:05AM +0000, Lee, Chee Yang wrote: > Hi Tom, > > Would you mind to share with me how you make the installer image using wic? I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate the partition as needed. I can't paste the exact image as it was done for a customer. > > -------- Forwarded Message -------- > From: Tom Rini <trini@konsulko.com> > To: chee.yang.lee@intel.com > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer > Date: Wed, 15 May 2019 11:33:43 -0400 > > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com > wrote: > > > From: Chee Yang Lee <chee.yang.lee@intel.com> > > > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > > build larger installable image, we can build it in wic based image. > > > > 2 partition needed for installable image > > - first partition build using source plugin bootimg-efi and > > configure to run install > > - second partition build with root.img, systemd-boot and kernel to > > be install on target machine (this partition build using new source > > plugin installer-partition) > > > > These patches: > > - add new source plugin for second partition > > - add new .wks > > - add new wic dependency and set default value for required variable > > - allow source plugin bootimg-efi to configure to install > > > > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > > local.conf. > > I'm confused as to why we need most of these changes to support the end > goal. The enhancement to bootimg-efi makes conceptual sense. I don't > see why we need a new plugin to do the rest, however. What's being > done > here that we don't already do today? I say this having made installer > images for EFI machines using wic (in sumo, even). Thanks! > > -- > Tom > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > -- Tom [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/4] add wic based image installer 2019-05-16 2:37 ` Tom Rini @ 2019-05-16 11:48 ` Lee, Chee Yang 2019-05-16 12:19 ` Tom Rini 0 siblings, 1 reply; 13+ messages in thread From: Lee, Chee Yang @ 2019-05-16 11:48 UTC (permalink / raw) To: trini; +Cc: Openembedded-core I couldn’t figure out how to make it with sdimage-bootpart.wks, can you tell me what did you set for IMAGE_BOOT_FILES ? To further explain the idea, I'm trying to use the new source plugin and .wks.in to build a wic image that "mimic" a hddimg, where I can dd the wic image to an USB drive and use that usb drive to install image to my target machine (like hddimg did ). first partition allow to boot into initramfs and it will execute the install script. The script will then install system-boot configuration, efi, kernel, rootfs.img and initrd from second partition to target machine. My sample image here: https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing On 5/16/19, 10:38 AM, "Tom Rini" <trini@konsulko.com> wrote: On Thu, May 16, 2019 at 02:23:05AM +0000, Lee, Chee Yang wrote: > Hi Tom, > > Would you mind to share with me how you make the installer image using wic? I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate the partition as needed. I can't paste the exact image as it was done for a customer. > > -------- Forwarded Message -------- > From: Tom Rini <trini@konsulko.com> > To: chee.yang.lee@intel.com > Cc: openembedded-core@lists.openembedded.org > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer > Date: Wed, 15 May 2019 11:33:43 -0400 > > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com > wrote: > > > From: Chee Yang Lee <chee.yang.lee@intel.com> > > > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > > build larger installable image, we can build it in wic based image. > > > > 2 partition needed for installable image > > - first partition build using source plugin bootimg-efi and > > configure to run install > > - second partition build with root.img, systemd-boot and kernel to > > be install on target machine (this partition build using new source > > plugin installer-partition) > > > > These patches: > > - add new source plugin for second partition > > - add new .wks > > - add new wic dependency and set default value for required variable > > - allow source plugin bootimg-efi to configure to install > > > > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > > local.conf. > > I'm confused as to why we need most of these changes to support the end > goal. The enhancement to bootimg-efi makes conceptual sense. I don't > see why we need a new plugin to do the rest, however. What's being > done > here that we don't already do today? I say this having made installer > images for EFI machines using wic (in sumo, even). Thanks! > > -- > Tom > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > -- Tom ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/4] add wic based image installer 2019-05-16 11:48 ` Lee, Chee Yang @ 2019-05-16 12:19 ` Tom Rini 2019-05-21 7:46 ` Lee, Chee Yang 0 siblings, 1 reply; 13+ messages in thread From: Tom Rini @ 2019-05-16 12:19 UTC (permalink / raw) To: Lee, Chee Yang; +Cc: Openembedded-core [-- Attachment #1: Type: text/plain, Size: 4602 bytes --] On Thu, May 16, 2019 at 11:48:02AM +0000, Lee, Chee Yang wrote: > I couldn’t figure out how to make it with sdimage-bootpart.wks, can > you tell me what did you set for IMAGE_BOOT_FILES ? > > To further explain the idea, I'm trying to use the new source plugin > and .wks.in to build a wic image that "mimic" a hddimg, where I can dd > the wic image to an USB drive and use that usb drive to install image > to my target machine (like hddimg did ). > first partition allow to boot into initramfs and it will execute the > install script. The script will then install system-boot > configuration, efi, kernel, rootfs.img and initrd from second > partition to target machine. > > My sample image here: > https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing OK, so some cleaned up snippets from the installer image recipe: # List of external things we must have had complete. do_rootfs[depends] += "core-image-minimal-initramfs:do_image_complete" do_rootfs[depends] += "full-image-to-install:do_image_complete" do_rootfs[depends] += "grub-efi:do_deploy" do_rootfs[depends] += "virtual/kernel:do_deploy" WKS_FILE = "sdimage-bootpart.wks" IMAGE_BOOT_FILES_append = "\ ${KERNEL_IMAGETYPE};EFI/BOOT/${KERNEL_IMAGETYPE} \ core-image-minimal-initramfs-${MACHINE}.cpio.gz;EFI/BOOT/initrd \ full-image-to-install-${MACHINE}.wic.xz;rootfs.img \ grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi \ grub-installer.cfg;EFI/BOOT/grub.cfg \ " And note that I reworked the installer for our case to write then update the real image's wic image rather than copy contents and then modify files due to other constraints (this project is also why I was pushing various changes to meta-secure-core). > > > On 5/16/19, 10:38 AM, "Tom Rini" <trini@konsulko.com> wrote: > > On Thu, May 16, 2019 at 02:23:05AM +0000, Lee, Chee Yang wrote: > > Hi Tom, > > > > Would you mind to share with me how you make the installer image using wic? > > I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate > the partition as needed. I can't paste the exact image as it was done > for a customer. > > > > > -------- Forwarded Message -------- > > From: Tom Rini <trini@konsulko.com> > > To: chee.yang.lee@intel.com > > Cc: openembedded-core@lists.openembedded.org > > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer > > Date: Wed, 15 May 2019 11:33:43 -0400 > > > > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com > > wrote: > > > > > From: Chee Yang Lee <chee.yang.lee@intel.com> > > > > > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > > > build larger installable image, we can build it in wic based image. > > > > > > 2 partition needed for installable image > > > - first partition build using source plugin bootimg-efi and > > > configure to run install > > > - second partition build with root.img, systemd-boot and kernel to > > > be install on target machine (this partition build using new source > > > plugin installer-partition) > > > > > > These patches: > > > - add new source plugin for second partition > > > - add new .wks > > > - add new wic dependency and set default value for required variable > > > - allow source plugin bootimg-efi to configure to install > > > > > > > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > > > local.conf. > > > > I'm confused as to why we need most of these changes to support the end > > goal. The enhancement to bootimg-efi makes conceptual sense. I don't > > see why we need a new plugin to do the rest, however. What's being > > done > > here that we don't already do today? I say this having made installer > > images for EFI machines using wic (in sumo, even). Thanks! > > > > -- > > Tom > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > > > > > > -- > Tom > > -- Tom [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/4] add wic based image installer 2019-05-16 12:19 ` Tom Rini @ 2019-05-21 7:46 ` Lee, Chee Yang 2019-05-21 14:13 ` Tom Rini 0 siblings, 1 reply; 13+ messages in thread From: Lee, Chee Yang @ 2019-05-21 7:46 UTC (permalink / raw) To: Openembedded-core, trini Tom, Thanks for the sharing I try out this method and it works however I notice that sdimage-bootpart.wks format bootimg-partition with vfat still limit the rootfs.img size to 4GB. Anyway, with the similar trick and using a new .wks with combination of 2 partition below could build the installer image for image greater than 4GB. - bootimg-efi with "wic: bootimg-efi: add label source parameter" patch - bootimg-partition ( format in ext4 ) so I think it would be great to keep the enhancement for bootimg-efi while ignore the other 3 patches. On 5/16/19, 8:20 PM, "Tom Rini" <trini@konsulko.com> wrote: On Thu, May 16, 2019 at 11:48:02AM +0000, Lee, Chee Yang wrote: > I couldn’t figure out how to make it with sdimage-bootpart.wks, can > you tell me what did you set for IMAGE_BOOT_FILES ? > > To further explain the idea, I'm trying to use the new source plugin > and .wks.in to build a wic image that "mimic" a hddimg, where I can dd > the wic image to an USB drive and use that usb drive to install image > to my target machine (like hddimg did ). > first partition allow to boot into initramfs and it will execute the > install script. The script will then install system-boot > configuration, efi, kernel, rootfs.img and initrd from second > partition to target machine. > > My sample image here: > https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing OK, so some cleaned up snippets from the installer image recipe: # List of external things we must have had complete. do_rootfs[depends] += "core-image-minimal-initramfs:do_image_complete" do_rootfs[depends] += "full-image-to-install:do_image_complete" do_rootfs[depends] += "grub-efi:do_deploy" do_rootfs[depends] += "virtual/kernel:do_deploy" WKS_FILE = "sdimage-bootpart.wks" IMAGE_BOOT_FILES_append = "\ ${KERNEL_IMAGETYPE};EFI/BOOT/${KERNEL_IMAGETYPE} \ core-image-minimal-initramfs-${MACHINE}.cpio.gz;EFI/BOOT/initrd \ full-image-to-install-${MACHINE}.wic.xz;rootfs.img \ grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi \ grub-installer.cfg;EFI/BOOT/grub.cfg \ " And note that I reworked the installer for our case to write then update the real image's wic image rather than copy contents and then modify files due to other constraints (this project is also why I was pushing various changes to meta-secure-core). > > > On 5/16/19, 10:38 AM, "Tom Rini" <trini@konsulko.com> wrote: > > On Thu, May 16, 2019 at 02:23:05AM +0000, Lee, Chee Yang wrote: > > Hi Tom, > > > > Would you mind to share with me how you make the installer image using wic? > > I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate > the partition as needed. I can't paste the exact image as it was done > for a customer. > > > > > -------- Forwarded Message -------- > > From: Tom Rini <trini@konsulko.com> > > To: chee.yang.lee@intel.com > > Cc: openembedded-core@lists.openembedded.org > > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer > > Date: Wed, 15 May 2019 11:33:43 -0400 > > > > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com > > wrote: > > > > > From: Chee Yang Lee <chee.yang.lee@intel.com> > > > > > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > > > build larger installable image, we can build it in wic based image. > > > > > > 2 partition needed for installable image > > > - first partition build using source plugin bootimg-efi and > > > configure to run install > > > - second partition build with root.img, systemd-boot and kernel to > > > be install on target machine (this partition build using new source > > > plugin installer-partition) > > > > > > These patches: > > > - add new source plugin for second partition > > > - add new .wks > > > - add new wic dependency and set default value for required variable > > > - allow source plugin bootimg-efi to configure to install > > > > > > > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > > > local.conf. > > > > I'm confused as to why we need most of these changes to support the end > > goal. The enhancement to bootimg-efi makes conceptual sense. I don't > > see why we need a new plugin to do the rest, however. What's being > > done > > here that we don't already do today? I say this having made installer > > images for EFI machines using wic (in sumo, even). Thanks! > > > > -- > > Tom > > -- > > _______________________________________________ > > Openembedded-core mailing list > > Openembedded-core@lists.openembedded.org > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > > > > > > -- > Tom > > -- Tom ^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH 0/4] add wic based image installer 2019-05-21 7:46 ` Lee, Chee Yang @ 2019-05-21 14:13 ` Tom Rini 0 siblings, 0 replies; 13+ messages in thread From: Tom Rini @ 2019-05-21 14:13 UTC (permalink / raw) To: Lee, Chee Yang; +Cc: Openembedded-core [-- Attachment #1: Type: text/plain, Size: 6583 bytes --] Thanks! FWIW, I really think an installer improvement would be go to from "copy image contents to install media, update slightly" to "write compressed image to disk (use bmaptool to handle decomp), update a few things like UUIDs". This would buy us a lot of space as a .xz compressed image is going to tend to be a lot smaller than 4GB, even with a ton of things in it. On Tue, May 21, 2019 at 07:46:52AM +0000, Lee, Chee Yang wrote: > Tom, Thanks for the sharing > I try out this method and it works > however I notice that sdimage-bootpart.wks format bootimg-partition with vfat still limit the rootfs.img size to 4GB. > Anyway, with the similar trick and using a new .wks with combination of 2 partition below could build the installer image for image greater than 4GB. > - bootimg-efi with "wic: bootimg-efi: add label source parameter" patch > - bootimg-partition ( format in ext4 ) > so I think it would be great to keep the enhancement for bootimg-efi while ignore the other 3 patches. > > > On 5/16/19, 8:20 PM, "Tom Rini" <trini@konsulko.com> wrote: > > On Thu, May 16, 2019 at 11:48:02AM +0000, Lee, Chee Yang wrote: > > > I couldn’t figure out how to make it with sdimage-bootpart.wks, can > > you tell me what did you set for IMAGE_BOOT_FILES ? > > > > To further explain the idea, I'm trying to use the new source plugin > > and .wks.in to build a wic image that "mimic" a hddimg, where I can dd > > the wic image to an USB drive and use that usb drive to install image > > to my target machine (like hddimg did ). > > first partition allow to boot into initramfs and it will execute the > > install script. The script will then install system-boot > > configuration, efi, kernel, rootfs.img and initrd from second > > partition to target machine. > > > > My sample image here: > > https://drive.google.com/file/d/1c1FCJ2ghWNzsXK1zbOAVz1bz6G3H5zx5/view?usp=sharing > > OK, so some cleaned up snippets from the installer image recipe: > # List of external things we must have had complete. > do_rootfs[depends] += "core-image-minimal-initramfs:do_image_complete" > do_rootfs[depends] += "full-image-to-install:do_image_complete" > do_rootfs[depends] += "grub-efi:do_deploy" > do_rootfs[depends] += "virtual/kernel:do_deploy" > > WKS_FILE = "sdimage-bootpart.wks" > > IMAGE_BOOT_FILES_append = "\ > ${KERNEL_IMAGETYPE};EFI/BOOT/${KERNEL_IMAGETYPE} \ > core-image-minimal-initramfs-${MACHINE}.cpio.gz;EFI/BOOT/initrd \ > full-image-to-install-${MACHINE}.wic.xz;rootfs.img \ > grub-efi-bootx64.efi;EFI/BOOT/bootx64.efi \ > grub-installer.cfg;EFI/BOOT/grub.cfg \ > " > > And note that I reworked the installer for our case to write then update > the real image's wic image rather than copy contents and then modify > files due to other constraints (this project is also why I was pushing > various changes to meta-secure-core). > > > > > > > On 5/16/19, 10:38 AM, "Tom Rini" <trini@konsulko.com> wrote: > > > > On Thu, May 16, 2019 at 02:23:05AM +0000, Lee, Chee Yang wrote: > > > Hi Tom, > > > > > > Would you mind to share with me how you make the installer image using wic? > > > > I ended up using sdimage-bootpart.wks and IMAGE_BOOT_FILES to populate > > the partition as needed. I can't paste the exact image as it was done > > for a customer. > > > > > > > > -------- Forwarded Message -------- > > > From: Tom Rini <trini@konsulko.com> > > > To: chee.yang.lee@intel.com > > > Cc: openembedded-core@lists.openembedded.org > > > Subject: Re: [OE-core] [PATCH 0/4] add wic based image installer > > > Date: Wed, 15 May 2019 11:33:43 -0400 > > > > > > On Wed, May 08, 2019 at 02:40:58PM +0800, chee.yang.lee@intel.com > > > wrote: > > > > > > > From: Chee Yang Lee <chee.yang.lee@intel.com> > > > > > > > > Existing installable image (hddimg/ISO) has limit of 4GB size, so to > > > > build larger installable image, we can build it in wic based image. > > > > > > > > 2 partition needed for installable image > > > > - first partition build using source plugin bootimg-efi and > > > > configure to run install > > > > - second partition build with root.img, systemd-boot and kernel to > > > > be install on target machine (this partition build using new source > > > > plugin installer-partition) > > > > > > > > These patches: > > > > - add new source plugin for second partition > > > > - add new .wks > > > > - add new wic dependency and set default value for required variable > > > > - allow source plugin bootimg-efi to configure to install > > > > > > > > > > > > To build the image, set WKS_FILE="wic-installer.wks.in" in > > > > local.conf. > > > > > > I'm confused as to why we need most of these changes to support the end > > > goal. The enhancement to bootimg-efi makes conceptual sense. I don't > > > see why we need a new plugin to do the rest, however. What's being > > > done > > > here that we don't already do today? I say this having made installer > > > images for EFI machines using wic (in sumo, even). Thanks! > > > > > > -- > > > Tom > > > -- > > > _______________________________________________ > > > Openembedded-core mailing list > > > Openembedded-core@lists.openembedded.org > > > http://lists.openembedded.org/mailman/listinfo/openembedded-core > > > > > > > > > > > > > -- > > Tom > > > > > > -- > Tom > > > -- > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- Tom [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2019-05-21 14:13 UTC | newest] Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-05-08 6:40 [PATCH 0/4] add wic based image installer chee.yang.lee 2019-05-08 6:40 ` [PATCH 1/4] wic: add new installer-partition plugin chee.yang.lee 2019-05-08 6:41 ` [PATCH 2/4] wic: bootimg-efi: add label source parameter chee.yang.lee 2019-05-08 6:41 ` [PATCH 3/4] wic: add new wic-installer.wks for wic based image installer chee.yang.lee 2019-05-08 6:41 ` [PATCH 4/4] image_types_wic: add dependency " chee.yang.lee 2019-05-15 15:33 ` [PATCH 0/4] add " Tom Rini 2019-05-16 1:31 ` Lee, Chee Yang 2019-05-16 2:23 Lee, Chee Yang 2019-05-16 2:37 ` Tom Rini 2019-05-16 11:48 ` Lee, Chee Yang 2019-05-16 12:19 ` Tom Rini 2019-05-21 7:46 ` Lee, Chee Yang 2019-05-21 14:13 ` Tom Rini
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.