[PATCH v3 00/11] Fix for #10835 - WIC should not rely on hddimg creation for finding all needed artifacts

[PATCH v3 00/11] Fix for #10835 - WIC should not rely on hddimg creation for finding all needed artifacts

[Ed Bartosh]

Hi,

This patchset usage of hddimg from wic codebase due to planned deprecation of hddimg.

While working on this wic-related code in meta/classes/ has been moved to
image-wic.bbclass to make it more maintainable.

Changes in v2: rebased on top of ross/mut
Changes in v3: removed patches that add tasks, fixed test cases,
               rebased on top of patches that make use of artifacts from DEPLOY_DIR_IMAGE,
               rebased on to pf ross/mut

The following changes since commit 2624674adefd86d15bc0bec2bb2a3876f785b947:

  selftest: check results in test_image_vars_dir* (2017-01-27 17:27:30 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/wic/wip
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wip

Ed Bartosh (10):
  image-wic: move wic code to image-wic.bbclass
  wic: use INITRD_LIVE in isoimage-isohybrid
  wic: isoimage-isohybrid: stop using HDDDIR
  image-wic: remove HDDDIR from WICVARS
  isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCH
  selftest: stop using hddimg in the wic test suite
  selftest: wic: fix test_iso_image test case
  grub-efi.bbclass: use 'grub-efi-' prefix
  wic-tools: add dependency to systemd-boot
  isoimage-isohybrid: renamed variable hdd_dir

Tom Zanussi (1):
  wic: Look for image artifacts in a common location

 meta/classes/grub-efi.bbclass                      |  10 +-
 meta/classes/image-wic.bbclass                     | 120 +++++++++++++++++++++
 meta/classes/image.bbclass                         |  25 +----
 meta/classes/image_types.bbclass                   |  95 ----------------
 meta/lib/oeqa/selftest/wic.py                      |   6 +-
 meta/recipes-core/meta/wic-tools.bb                |   4 +-
 scripts/lib/wic/plugins/source/bootimg-efi.py      |  39 +++++--
 .../lib/wic/plugins/source/isoimage-isohybrid.py   |  36 ++++---
 8 files changed, 182 insertions(+), 153 deletions(-)
 create mode 100644 meta/classes/image-wic.bbclass

-- 
2.1.4

[PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

There is a lot of wic code in image.bbclass and image_types.bbclass
Having all code separated in one file should make it more readable
and easier to maintain.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/image-wic.bbclass   | 120 +++++++++++++++++++++++++++++++++++++++
 meta/classes/image.bbclass       |  25 +-------
 meta/classes/image_types.bbclass |  95 -------------------------------
 3 files changed, 122 insertions(+), 118 deletions(-)
 create mode 100644 meta/classes/image-wic.bbclass

diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.bbclass
new file mode 100644
index 0000000..2acfd65
--- /dev/null
+++ b/meta/classes/image-wic.bbclass
@@ -0,0 +1,120 @@
+# The WICVARS variable is used to define list of bitbake variables used in wic code
+# variables from this list is written to <image>.env file
+WICVARS ?= "\
+           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES \
+           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
+           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
+
+WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
+WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
+WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
+WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
+
+def wks_search(files, search_path):
+    for f in files:
+        if os.path.isabs(f):
+            if os.path.exists(f):
+                return f
+        else:
+            searched = bb.utils.which(search_path, f)
+            if searched:
+                return searched
+
+WIC_CREATE_EXTRA_ARGS ?= ""
+
+IMAGE_CMD_wic () {
+	out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
+	wks="${WKS_FULL_PATH}"
+	if [ -z "$wks" ]; then
+		bbfatal "No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
+	fi
+
+	BUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}
+	mv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"
+	rm -rf "$out/"
+}
+IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
+
+# Rebuild when the wks file or vars in WICVARS change
+USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
+WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
+do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
+do_image_wic[depends] += "wic-tools:do_build"
+
+python () {
+    if d.getVar('USING_WIC') and 'do_bootimg' in d:
+        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
+}
+
+python do_write_wks_template () {
+    """Write out expanded template contents to WKS_FULL_PATH."""
+    import re
+
+    template_body = d.getVar('_WKS_TEMPLATE')
+
+    # Remove any remnant variable references left behind by the expansion
+    # due to undefined variables
+    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
+    while True:
+        new_body = re.sub(expand_var_regexp, '', template_body)
+        if new_body == template_body:
+            break
+        else:
+            template_body = new_body
+
+    wks_file = d.getVar('WKS_FULL_PATH')
+    with open(wks_file, 'w') as f:
+        f.write(template_body)
+}
+
+python () {
+    if d.getVar('USING_WIC'):
+        wks_file_u = d.getVar('WKS_FULL_PATH', False)
+        wks_file = d.expand(wks_file_u)
+        base, ext = os.path.splitext(wks_file)
+        if ext == '.in' and os.path.exists(wks_file):
+            wks_out_file = os.path.join(d.getVar('WORKDIR'), os.path.basename(base))
+            d.setVar('WKS_FULL_PATH', wks_out_file)
+            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
+            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
+
+            # We need to re-parse each time the file changes, and bitbake
+            # needs to be told about that explicitly.
+            bb.parse.mark_dependency(d, wks_file)
+
+            try:
+                with open(wks_file, 'r') as f:
+                    body = f.read()
+            except (IOError, OSError) as exc:
+                pass
+            else:
+                # Previously, I used expandWithRefs to get the dependency list
+                # and add it to WICVARS, but there's no point re-parsing the
+                # file in process_wks_template as well, so just put it in
+                # a variable and let the metadata deal with the deps.
+                d.setVar('_WKS_TEMPLATE', body)
+                bb.build.addtask('do_write_wks_template', 'do_image_wic', None, d)
+}
+
+#
+# Write environment variables used by wic
+# to tmp/sysroots/<machine>/imgdata/<image>.env
+#
+python do_rootfs_wicenv () {
+    wicvars = d.getVar('WICVARS')
+    if not wicvars:
+        return
+
+    stdir = d.getVar('STAGING_DIR')
+    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
+    bb.utils.mkdirhier(outdir)
+    basename = d.getVar('IMAGE_BASENAME')
+    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
+        for var in wicvars.split():
+            value = d.getVar(var)
+            if value:
+                envf.write('%s="%s"\n' % (var, value.strip()))
+}
+addtask do_rootfs_wicenv after do_image before do_image_wic
+do_rootfs_wicenv[vardeps] += "${WICVARS}"
+do_rootfs_wicenv[prefuncs] = 'set_image_size'
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 13fd996..1a549e9 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -192,6 +192,8 @@ python () {
 IMAGE_CLASSES += "image_types"
 inherit ${IMAGE_CLASSES}
 
+inherit image-wic
+
 IMAGE_POSTPROCESS_COMMAND ?= ""
 
 # some default locales
@@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
 }
 addtask do_image_qa after do_image_complete before do_build
 
-#
-# Write environment variables used by wic
-# to tmp/sysroots/<machine>/imgdata/<image>.env
-#
-python do_rootfs_wicenv () {
-    wicvars = d.getVar('WICVARS')
-    if not wicvars:
-        return
-
-    stdir = d.getVar('STAGING_DIR')
-    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
-    bb.utils.mkdirhier(outdir)
-    basename = d.getVar('IMAGE_BASENAME')
-    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
-        for var in wicvars.split():
-            value = d.getVar(var)
-            if value:
-                envf.write('%s="%s"\n' % (var, value.strip()))
-}
-addtask do_rootfs_wicenv after do_image before do_image_wic
-do_rootfs_wicenv[vardeps] += "${WICVARS}"
-do_rootfs_wicenv[prefuncs] = 'set_image_size'
-
 def setup_debugfs_variables(d):
     d.appendVar('IMAGE_ROOTFS', '-dbg')
     d.appendVar('IMAGE_LINK_NAME', '-dbg')
diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
index ad95dc6..e2ecf93 100644
--- a/meta/classes/image_types.bbclass
+++ b/meta/classes/image_types.bbclass
@@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
 
 IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs ${MKUBIFS_ARGS}"
 
-WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
-WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
-WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in '${BBPATH}:${COREBASE}'.split(':'))}"
-WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(), '${WKS_SEARCH_PATH}') or ''}"
-
-def wks_search(files, search_path):
-    for f in files:
-        if os.path.isabs(f):
-            if os.path.exists(f):
-                return f
-        else:
-            searched = bb.utils.which(search_path, f)
-            if searched:
-                return searched
-
-WIC_CREATE_EXTRA_ARGS ?= ""
-
-IMAGE_CMD_wic () {
-	out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
-	wks="${WKS_FULL_PATH}"
-	if [ -z "$wks" ]; then
-		bbfatal "No kickstart files from WKS_FILES were found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
-	fi
-
-	BUILDDIR="${TOPDIR}" wic create "$wks" --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/" ${WIC_CREATE_EXTRA_ARGS}
-	mv "$out/$(basename "${wks%.wks}")"*.direct "$out${IMAGE_NAME_SUFFIX}.wic"
-	rm -rf "$out/"
-}
-IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
-
-# Rebuild when the wks file or vars in WICVARS change
-USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
-WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
-do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
-do_image_wic[depends] += "wic-tools:do_build"
-
-python () {
-    if d.getVar('USING_WIC') and 'do_bootimg' in d:
-        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
-}
-
-python do_write_wks_template () {
-    """Write out expanded template contents to WKS_FULL_PATH."""
-    import re
-
-    template_body = d.getVar('_WKS_TEMPLATE')
-
-    # Remove any remnant variable references left behind by the expansion
-    # due to undefined variables
-    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
-    while True:
-        new_body = re.sub(expand_var_regexp, '', template_body)
-        if new_body == template_body:
-            break
-        else:
-            template_body = new_body
-
-    wks_file = d.getVar('WKS_FULL_PATH')
-    with open(wks_file, 'w') as f:
-        f.write(template_body)
-}
-
-python () {
-    if d.getVar('USING_WIC'):
-        wks_file_u = d.getVar('WKS_FULL_PATH', False)
-        wks_file = d.expand(wks_file_u)
-        base, ext = os.path.splitext(wks_file)
-        if ext == '.in' and os.path.exists(wks_file):
-            wks_out_file = os.path.join(d.getVar('WORKDIR'), os.path.basename(base))
-            d.setVar('WKS_FULL_PATH', wks_out_file)
-            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
-            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
-
-            # We need to re-parse each time the file changes, and bitbake
-            # needs to be told about that explicitly.
-            bb.parse.mark_dependency(d, wks_file)
-
-            try:
-                with open(wks_file, 'r') as f:
-                    body = f.read()
-            except (IOError, OSError) as exc:
-                pass
-            else:
-                # Previously, I used expandWithRefs to get the dependency list
-                # and add it to WICVARS, but there's no point re-parsing the
-                # file in process_wks_template as well, so just put it in
-                # a variable and let the metadata deal with the deps.
-                d.setVar('_WKS_TEMPLATE', body)
-                bb.build.addtask('do_write_wks_template', 'do_image_wic', None, d)
-}
-
 EXTRA_IMAGECMD = ""
 
 inherit siteinfo
@@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
 # The IMAGE_TYPES_MASKED variable will be used to mask out from the IMAGE_FSTYPES,
 # images that will not be built at do_rootfs time: vmdk, vdi, qcow2, hdddirect, hddimg, iso, etc.
 IMAGE_TYPES_MASKED ?= ""
-
-# The WICVARS variable is used to define list of bitbake variables used in wic code
-# variables from this list is written to <image>.env file
-WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
-- 
2.1.4

[PATCH v3 02/11] wic: use INITRD_LIVE in isoimage-isohybrid

[Ed Bartosh]

INITRD variable is not set if hddimg is disabled.
isoimage-isohybrid can't get correct name for initrd if INITRD
variable is not set.

Added INITRD_LIVE to WICVARS and used it in isoimage-isohybrid
code to get initrd artifact name. Used INITRD if INITRD_LIVE is not set.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/image-wic.bbclass                       | 4 ++--
 meta/lib/oeqa/selftest/wic.py                        | 2 +-
 scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 2 +-
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.bbclass
index 2acfd65..ab2e541 100644
--- a/meta/classes/image-wic.bbclass
+++ b/meta/classes/image-wic.bbclass
@@ -2,8 +2,8 @@
 # variables from this list is written to <image>.env file
 WICVARS ?= "\
            BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES \
-           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
-           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
+           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR MACHINE_ARCH \
+           RECIPE_SYSROOT_NATIVE ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
 
 WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 53631fa..8eb77ae 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -373,7 +373,7 @@ class Wic(oeSelfTest):
         wicvars = set(get_bb_var('WICVARS', image).split())
         # filter out optional variables
         wicvars = wicvars.difference(('HDDDIR', 'IMAGE_BOOT_FILES',
-                                      'INITRD', 'ISODIR'))
+                                      'INITRD', 'INITRD_LIVE', 'ISODIR'))
         with open(path) as envfile:
             content = dict(line.split("=", 1) for line in envfile)
             # test if variables used by wic present in the .env file
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index a8a5dc0..fd25190 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -144,7 +144,7 @@ class IsoImagePlugin(SourcePlugin):
         Create path for initramfs image
         """
 
-        initrd = get_bitbake_var("INITRD")
+        initrd = get_bitbake_var("INITRD_LIVE") or get_bitbake_var("INITRD")
         if not initrd:
             initrd_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
             if not initrd_dir:
-- 
2.1.4

[PATCH v3 03/11] wic: isoimage-isohybrid: stop using HDDDIR

[Ed Bartosh]

Stop using HDDDIR in isoimage-isohybrid wic plugin.
This variable is set by hddimg code, which is going to be
removed soon.

All required artifacts should be available from ISODIR.
wic-image.bbclass has been modified to build iso artifacts,
so it should be safe to remove usage of HDDDIR.

[YOCTO #10835]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 10 ++--------
 1 file changed, 2 insertions(+), 8 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index fd25190..a637ce5 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -249,12 +249,8 @@ class IsoImagePlugin(SourcePlugin):
         part.rootfs_dir = rootfs_dir
 
         # Prepare rootfs.img
-        hdd_dir = get_bitbake_var("HDDDIR")
         img_iso_dir = get_bitbake_var("ISODIR")
-
-        rootfs_img = "%s/rootfs.img" % hdd_dir
-        if not os.path.isfile(rootfs_img):
-            rootfs_img = "%s/rootfs.img" % img_iso_dir
+        rootfs_img = "%s/rootfs.img" % img_iso_dir
         if not os.path.isfile(rootfs_img):
             # check if rootfs.img is in deploydir
             deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
@@ -283,9 +279,7 @@ class IsoImagePlugin(SourcePlugin):
             os.remove(part.source_file)
 
         # Prepare initial ramdisk
-        initrd = "%s/initrd" % hdd_dir
-        if not os.path.isfile(initrd):
-            initrd = "%s/initrd" % img_iso_dir
+        initrd = "%s/initrd" % img_iso_dir
         if not os.path.isfile(initrd):
             initrd = cls._build_initramfs_path(rootfs_dir, cr_workdir)
 
-- 
2.1.4

[PATCH v3 04/11] image-wic: remove HDDDIR from WICVARS

[Ed Bartosh]

Removed HDDDIR as it's not used by wic anymore.
Stopped usage of HDDDIR in wic test suite.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/image-wic.bbclass | 2 +-
 meta/lib/oeqa/selftest/wic.py  | 4 ++--
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.bbclass
index ab2e541..bd79073 100644
--- a/meta/classes/image-wic.bbclass
+++ b/meta/classes/image-wic.bbclass
@@ -1,7 +1,7 @@
 # The WICVARS variable is used to define list of bitbake variables used in wic code
 # variables from this list is written to <image>.env file
 WICVARS ?= "\
-           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES \
+           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD IMAGE_BASENAME IMAGE_BOOT_FILES \
            IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR MACHINE_ARCH \
            RECIPE_SYSROOT_NATIVE ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
 
diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index 8eb77ae..ebbb280 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -372,8 +372,8 @@ class Wic(oeSelfTest):
 
         wicvars = set(get_bb_var('WICVARS', image).split())
         # filter out optional variables
-        wicvars = wicvars.difference(('HDDDIR', 'IMAGE_BOOT_FILES',
-                                      'INITRD', 'INITRD_LIVE', 'ISODIR'))
+        wicvars = wicvars.difference(('IMAGE_BOOT_FILES', 'INITRD',
+                                      'INITRD_LIVE', 'ISODIR'))
         with open(path) as envfile:
             content = dict(line.split("=", 1) for line in envfile)
             # test if variables used by wic present in the .env file
-- 
2.1.4

[PATCH v3 05/11] wic: Look for image artifacts in a common location

[Ed Bartosh]

From: Tom Zanussi <tom.zanussi@linux.intel.com>

Rather than have each image type look for artifacts in image-specific
locations, move towards having them look for artifacts in a common
location, in this case DEPLOY_DIR_IMAGE

Use the existing deploy.bbclass to have the bootloaders put their binaries
in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image

Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com>
Signed-off-by: Tom Zanussi <tom.zanussi@linux.intel.com>
Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/lib/oeqa/selftest/wic.py                      |  4 +--
 scripts/lib/wic/plugins/source/bootimg-efi.py      | 39 ++++++++++++++++------
 .../lib/wic/plugins/source/isoimage-isohybrid.py   | 18 +++++++---
 3 files changed, 45 insertions(+), 16 deletions(-)

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index ebbb280..da68e25 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -372,8 +372,8 @@ class Wic(oeSelfTest):
 
         wicvars = set(get_bb_var('WICVARS', image).split())
         # filter out optional variables
-        wicvars = wicvars.difference(('IMAGE_BOOT_FILES', 'INITRD',
-                                      'INITRD_LIVE', 'ISODIR'))
+        wicvars = wicvars.difference(('DEPLOY_DIR_IMAGE', 'IMAGE_BOOT_FILES',
+                                      'INITRD', 'INITRD_LIVE', 'ISODIR'))
         with open(path) as envfile:
             content = dict(line.split("=", 1) for line in envfile)
             # test if variables used by wic present in the .env file
diff --git a/scripts/lib/wic/plugins/source/bootimg-efi.py b/scripts/lib/wic/plugins/source/bootimg-efi.py
index 305e910..74a1557 100644
--- a/scripts/lib/wic/plugins/source/bootimg-efi.py
+++ b/scripts/lib/wic/plugins/source/bootimg-efi.py
@@ -42,7 +42,7 @@ class BootimgEFIPlugin(SourcePlugin):
     name = 'bootimg-efi'
 
     @classmethod
-    def do_configure_grubefi(cls, hdddir, creator, cr_workdir):
+    def do_configure_grubefi(cls, hdddir, creator, cr_workdir, source_params):
         """
         Create loader-specific (grub-efi) config
         """
@@ -82,7 +82,7 @@ class BootimgEFIPlugin(SourcePlugin):
         cfg.close()
 
     @classmethod
-    def do_configure_systemdboot(cls, hdddir, creator, cr_workdir):
+    def do_configure_systemdboot(cls, hdddir, creator, cr_workdir, source_params):
         """
         Create loader-specific systemd-boot/gummiboot config
         """
@@ -98,6 +98,19 @@ class BootimgEFIPlugin(SourcePlugin):
         loader_conf += "default boot\n"
         loader_conf += "timeout %d\n" % bootloader.timeout
 
+        initrd = source_params.get('initrd')
+
+        if initrd:
+            # obviously we need to have a common common deploy var
+            bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+            if not bootimg_dir:
+                msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
+
+            cp_cmd = "cp %s/%s %s" % (bootimg_dir, initrd, hdddir)
+            exec_cmd(cp_cmd, True)
+        else:
+            msger.debug("Ignoring missing initrd")
+
         msger.debug("Writing systemd-boot config %s/hdd/boot/loader/loader.conf" \
                         % cr_workdir)
         cfg = open("%s/hdd/boot/loader/loader.conf" % cr_workdir, "w")
@@ -127,6 +140,9 @@ class BootimgEFIPlugin(SourcePlugin):
             boot_conf += "options LABEL=Boot root=%s %s\n" % \
                              (creator.rootdev, bootloader.append)
 
+            if initrd:
+                boot_conf += "initrd /%s\n" % initrd
+
         msger.debug("Writing systemd-boot config %s/hdd/boot/loader/entries/boot.conf" \
                         % cr_workdir)
         cfg = open("%s/hdd/boot/loader/entries/boot.conf" % cr_workdir, "w")
@@ -148,9 +164,9 @@ class BootimgEFIPlugin(SourcePlugin):
 
         try:
             if source_params['loader'] == 'grub-efi':
-                cls.do_configure_grubefi(hdddir, creator, cr_workdir)
+                cls.do_configure_grubefi(hdddir, creator, cr_workdir, source_params)
             elif source_params['loader'] == 'systemd-boot':
-                cls.do_configure_systemdboot(hdddir, creator, cr_workdir)
+                cls.do_configure_systemdboot(hdddir, creator, cr_workdir, source_params)
             else:
                 msger.error("unrecognized bootimg-efi loader: %s" % source_params['loader'])
         except KeyError:
@@ -167,9 +183,9 @@ class BootimgEFIPlugin(SourcePlugin):
         In this case, prepare content for an EFI (grub) boot partition.
         """
         if not bootimg_dir:
-            bootimg_dir = get_bitbake_var("HDDDIR")
+            bootimg_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
             if not bootimg_dir:
-                msger.error("Couldn't find HDDDIR, exiting\n")
+                 msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
             # just so the result notes display it
             creator.set_bootimg_dir(bootimg_dir)
 
@@ -181,17 +197,20 @@ class BootimgEFIPlugin(SourcePlugin):
             (staging_kernel_dir, hdddir)
         exec_cmd(install_cmd)
 
+
         try:
             if source_params['loader'] == 'grub-efi':
                 shutil.copyfile("%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir,
                                 "%s/grub.cfg" % cr_workdir)
-                cp_cmd = "cp %s/EFI/BOOT/* %s/EFI/BOOT" % (bootimg_dir, hdddir)
-                exec_cmd(cp_cmd, True)
+                for mod in [x for x in os.listdir(bootimg_dir) if x.startswith("grub-efi-")]:
+                    cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (bootimg_dir, mod, hdddir, mod[9:])
+                    exec_cmd(cp_cmd, True)
                 shutil.move("%s/grub.cfg" % cr_workdir,
                             "%s/hdd/boot/EFI/BOOT/grub.cfg" % cr_workdir)
             elif source_params['loader'] == 'systemd-boot':
-                cp_cmd = "cp %s/EFI/BOOT/* %s/EFI/BOOT" % (bootimg_dir, hdddir)
-                exec_cmd(cp_cmd, True)
+                for mod in [x for x in os.listdir(bootimg_dir) if x.startswith("systemd-")]:
+                    cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (bootimg_dir, mod, hdddir, mod[8:])
+                    exec_cmd(cp_cmd, True)
             else:
                 msger.error("unrecognized bootimg-efi loader: %s" % source_params['loader'])
         except KeyError:
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index a637ce5..b54a229 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -249,6 +249,7 @@ class IsoImagePlugin(SourcePlugin):
         part.rootfs_dir = rootfs_dir
 
         # Prepare rootfs.img
+        hdd_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
         img_iso_dir = get_bitbake_var("ISODIR")
         rootfs_img = "%s/rootfs.img" % img_iso_dir
         if not os.path.isfile(rootfs_img):
@@ -278,10 +279,19 @@ class IsoImagePlugin(SourcePlugin):
         if os.path.isfile(part.source_file):
             os.remove(part.source_file)
 
-        # Prepare initial ramdisk
-        initrd = "%s/initrd" % img_iso_dir
-        if not os.path.isfile(initrd):
-            initrd = cls._build_initramfs_path(rootfs_dir, cr_workdir)
+        # Support using a different initrd other than default
+        if source_params.get('initrd'):
+            initrd = source_params['initrd']
+            if not hdd_dir:
+                msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
+            cp_cmd = "cp %s/%s %s" % (hdd_dir, initrd, cr_workdir)
+        else:
+            # Prepare initial ramdisk
+            initrd = "%s/initrd" % hdd_dir
+            if not os.path.isfile(initrd):
+                initrd = "%s/initrd" % img_iso_dir
+            if not os.path.isfile(initrd):
+                initrd = cls._build_initramfs_path(rootfs_dir, cr_workdir)
 
         install_cmd = "install -m 0644 %s %s/initrd" \
             % (initrd, isodir)
-- 
2.1.4

[PATCH v3 06/11] isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCH

[Ed Bartosh]

isoimage-sihybrid plugin uses MACHINE_ARCH to get the name of initrd image.
It doesn't work for all machines, for example for quemux86-64 machine
MACHINE_ARCH is quemux86_64 and initrd name is
core-image-minimal-initramfs-qemux86-64.cpio.gz

Used TRANSLATED_TARGET_ARCH variable to get the initrd image name.
Replaced MACHINE_ARCH->TRANSLATED_TARGET_ARCH in WICVARS variable to
make it available from <image>.env file.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/image-wic.bbclass                       | 4 ++--
 scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 8 ++++----
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.bbclass
index bd79073..3e98959 100644
--- a/meta/classes/image-wic.bbclass
+++ b/meta/classes/image-wic.bbclass
@@ -2,8 +2,8 @@
 # variables from this list is written to <image>.env file
 WICVARS ?= "\
            BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD IMAGE_BASENAME IMAGE_BOOT_FILES \
-           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR MACHINE_ARCH \
-           RECIPE_SYSROOT_NATIVE ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
+           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD INITRD_LIVE ISODIR RECIPE_SYSROOT_NATIVE \
+           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS TRANSLATED_TARGET_ARCH"
 
 WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
 WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index b54a229..e4637a3 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -158,11 +158,11 @@ class IsoImagePlugin(SourcePlugin):
             if not image_type:
                 msger.error("Couldn't find INITRAMFS_FSTYPES, exiting.\n")
 
-            machine_arch = get_bitbake_var("MACHINE_ARCH")
-            if not machine_arch:
-                msger.error("Couldn't find MACHINE_ARCH, exiting.\n")
+            target_arch = get_bitbake_var("TRANSLATED_TARGET_ARCH")
+            if not target_arch:
+                msger.error("Couldn't find TRANSLATED_TARGET_ARCH, exiting.\n")
 
-            initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, image_name, machine_arch, image_type))[0]
+            initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, image_name, target_arch, image_type))[0]
 
         if not os.path.exists(initrd):
             # Create initrd from rootfs directory
-- 
2.1.4

[PATCH v3 07/11] selftest: stop using hddimg in the wic test suite

[Ed Bartosh]

Removed hddimg from IMAGE_FEATURES as wic code doesn't
use hddimg anymore.

[YOCTO #10835]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/lib/oeqa/selftest/wic.py | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index da68e25..dce83ba 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -42,8 +42,7 @@ class Wic(oeSelfTest):
 
     def setUpLocal(self):
         """This code is executed before each test method."""
-        self.write_config('IMAGE_FSTYPES += " hddimg"\n'
-                          'MACHINE_FEATURES_append = " efi"\n')
+        self.write_config('MACHINE_FEATURES_append = " efi"\n')
 
         # Do this here instead of in setUpClass as the base setUp does some
         # clean up which can result in the native tools built earlier in
-- 
2.1.4

[PATCH v3 08/11] selftest: wic: fix test_iso_image test case

[Ed Bartosh]

Added "iso" to IMAGE_FSTYPES to build iso artifacts required
to fix test of isoimage-isohybrid wic plugin.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/lib/oeqa/selftest/wic.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/wic.py b/meta/lib/oeqa/selftest/wic.py
index dce83ba..6a6b54c 100644
--- a/meta/lib/oeqa/selftest/wic.py
+++ b/meta/lib/oeqa/selftest/wic.py
@@ -42,7 +42,8 @@ class Wic(oeSelfTest):
 
     def setUpLocal(self):
         """This code is executed before each test method."""
-        self.write_config('MACHINE_FEATURES_append = " efi"\n')
+        self.write_config('IMAGE_FSTYPES = "iso"\n'
+                          'MACHINE_FEATURES_append = " efi"\n')
 
         # Do this here instead of in setUpClass as the base setUp does some
         # clean up which can result in the native tools built earlier in
-- 
2.1.4

[PATCH v3 09/11] grub-efi.bbclass: use 'grub-efi-' prefix

[Ed Bartosh]

grub-efi recipe added 'grub-efi-' prefix to the file name of
efi binary. Changed grub-efi.bbclass accordingly.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/classes/grub-efi.bbclass | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/meta/classes/grub-efi.bbclass b/meta/classes/grub-efi.bbclass
index 3dc9146..8adda09 100644
--- a/meta/classes/grub-efi.bbclass
+++ b/meta/classes/grub-efi.bbclass
@@ -40,13 +40,15 @@ efi_populate() {
 
 	install -d ${DEST}${EFIDIR}
 
-	GRUB_IMAGE="bootia32.efi"
+	GRUB_IMAGE="grub-efi-bootia32.efi"
+	DEST_IMAGE="bootia32.efi"
 	if [ "${TARGET_ARCH}" = "x86_64" ]; then
-		GRUB_IMAGE="bootx64.efi"
+		GRUB_IMAGE="grub-efi-bootx64.efi"
+		DEST_IMAGE="bootx64.efi"
 	fi
-	install -m 0644 ${DEPLOY_DIR_IMAGE}/${GRUB_IMAGE} ${DEST}${EFIDIR}
+	install -m 0644 ${DEPLOY_DIR_IMAGE}/${GRUB_IMAGE} ${DEST}${EFIDIR}/${DEST_IMAGE}
 	EFIPATH=$(echo "${EFIDIR}" | sed 's/\//\\/g')
-	printf 'fs0:%s\%s\n' "$EFIPATH" "$GRUB_IMAGE" >${DEST}/startup.nsh
+	printf 'fs0:%s\%s\n' "$EFIPATH" "$DEST_IMAGE" >${DEST}/startup.nsh
 
 	install -m 0644 ${GRUB_CFG} ${DEST}${EFIDIR}/grub.cfg
 }
-- 
2.1.4

[PATCH v3 10/11] wic-tools: add dependency to systemd-boot

[Ed Bartosh]

Added systemd-boot to the list of dependencies of wic-tools
as wic bootimg-efi plugin depends on it.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 meta/recipes-core/meta/wic-tools.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-core/meta/wic-tools.bb b/meta/recipes-core/meta/wic-tools.bb
index aa09c82..1d9ea9c 100644
--- a/meta/recipes-core/meta/wic-tools.bb
+++ b/meta/recipes-core/meta/wic-tools.bb
@@ -3,8 +3,8 @@ SUMMARY = "A meta recipe to build native tools used by wic."
 LICENSE = "MIT"
 
 DEPENDS = "parted-native syslinux-native gptfdisk-native dosfstools-native mtools-native bmap-tools-native grub-efi-native cdrtools-native"
-DEPENDS_append_x86 = " syslinux grub-efi"
-DEPENDS_append_x86-64 = " syslinux grub-efi"
+DEPENDS_append_x86 = " syslinux grub-efi systemd-boot"
+DEPENDS_append_x86-64 = " syslinux grub-efi systemd-boot"
 
 INHIBIT_DEFAULT_DEPS = "1"
 inherit nopackages
-- 
2.1.4

[PATCH v3 11/11] isoimage-isohybrid: renamed variable hdd_dir

[Ed Bartosh]

Renamed variable hdd_dir to deploy_dir as this variable is
assigned to the value of DEPLOY_DIR_IMAGE.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
index e4637a3..bceaa84 100644
--- a/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
+++ b/scripts/lib/wic/plugins/source/isoimage-isohybrid.py
@@ -249,7 +249,7 @@ class IsoImagePlugin(SourcePlugin):
         part.rootfs_dir = rootfs_dir
 
         # Prepare rootfs.img
-        hdd_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+        deploy_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
         img_iso_dir = get_bitbake_var("ISODIR")
         rootfs_img = "%s/rootfs.img" % img_iso_dir
         if not os.path.isfile(rootfs_img):
@@ -282,12 +282,12 @@ class IsoImagePlugin(SourcePlugin):
         # Support using a different initrd other than default
         if source_params.get('initrd'):
             initrd = source_params['initrd']
-            if not hdd_dir:
+            if not deploy_dir:
                 msger.error("Couldn't find DEPLOY_DIR_IMAGE, exiting\n")
-            cp_cmd = "cp %s/%s %s" % (hdd_dir, initrd, cr_workdir)
+            cp_cmd = "cp %s/%s %s" % (deploy_dir, initrd, cr_workdir)
         else:
             # Prepare initial ramdisk
-            initrd = "%s/initrd" % hdd_dir
+            initrd = "%s/initrd" % deploy_dir
             if not os.path.isfile(initrd):
                 initrd = "%s/initrd" % img_iso_dir
             if not os.path.isfile(initrd):
-- 
2.1.4

✗ patchtest: failure for Fix for #10835 - WIC should not rely on hddimg creation for finding all needed artifacts (rev3)

[Patchwork]

== Series Details ==

Series: Fix for #10835 - WIC should not rely on hddimg creation for finding all needed artifacts (rev3)
Revision: 3
URL   : https://patchwork.openembedded.org/series/4802/
State : failure

== Summary ==


Thank you for submitting this patch series to OpenEmbedded Core. This is
an automated response. Several tests have been executed on the proposed
series by patchtest resulting in the following failures:



* Issue             Series does not apply on top of target branch [test_series_merge_on_head] 
  Suggested fix    Rebase your series on top of targeted branch
  Targeted branch  master (currently at 3a617a4c66)



If you believe any of these test results are incorrect, please reply to the
mailing list (openembedded-core@lists.openembedded.org) raising your concerns.
Otherwise we would appreciate you correcting the issues and submitting a new
version of the patchset if applicable. Please ensure you add/increment the
version number when sending the new version (i.e. [PATCH] -> [PATCH v2] ->
[PATCH v3] -> ...).

---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest
Test suite:     http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ola x Nilsson]

Please name the class image_wic.bbclass instead.

As dashes are not allowed in function names, you cannot use 
the image_wic_do_write_wks_template pattern of function 
names if you use a class name with a dash in it.

-- 
Ola x Nilsson

> -----Original Message-----
> From: openembedded-core-bounces@lists.openembedded.org
> [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf
> Of Ed Bartosh
> Sent: den 27 januari 2017 21:20
> To: openembedded-core@lists.openembedded.org
> Subject: [OE-core] [PATCH v3 01/11] image-wic: move wic code to image-
> wic.bbclass
> 
> There is a lot of wic code in image.bbclass and image_types.bbclass Having all
> code separated in one file should make it more readable and easier to
> maintain.
> 
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>  meta/classes/image-wic.bbclass   | 120
> +++++++++++++++++++++++++++++++++++++++
>  meta/classes/image.bbclass       |  25 +-------
>  meta/classes/image_types.bbclass |  95 -------------------------------
>  3 files changed, 122 insertions(+), 118 deletions(-)  create mode 100644
> meta/classes/image-wic.bbclass
> 
> diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-
> wic.bbclass new file mode 100644 index 0000000..2acfd65
> --- /dev/null
> +++ b/meta/classes/image-wic.bbclass
> @@ -0,0 +1,120 @@
> +# The WICVARS variable is used to define list of bitbake variables used
> +in wic code # variables from this list is written to <image>.env file
> +WICVARS ?= "\
> +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES \
> +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> TARGET_SYS"
> +
> +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in
> '${BBPATH}:${COREBASE}'.split(':'))}"
> +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> '${WKS_SEARCH_PATH}') or ''}"
> +
> +def wks_search(files, search_path):
> +    for f in files:
> +        if os.path.isabs(f):
> +            if os.path.exists(f):
> +                return f
> +        else:
> +            searched = bb.utils.which(search_path, f)
> +            if searched:
> +                return searched
> +
> +WIC_CREATE_EXTRA_ARGS ?= ""
> +
> +IMAGE_CMD_wic () {
> +	out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> +	wks="${WKS_FULL_PATH}"
> +	if [ -z "$wks" ]; then
> +		bbfatal "No kickstart files from WKS_FILES were
> found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> +	fi
> +
> +	BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> "$out/" ${WIC_CREATE_EXTRA_ARGS}
> +	mv "$out/$(basename "${wks%.wks}")"*.direct
> "$out${IMAGE_NAME_SUFFIX}.wic"
> +	rm -rf "$out/"
> +}
> +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> +
> +# Rebuild when the wks file or vars in WICVARS change USING_WIC =
> +"${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for
> c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> +do_image_wic[depends] += "wic-tools:do_build"
> +
> +python () {
> +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d) }
> +
> +python do_write_wks_template () {
> +    """Write out expanded template contents to WKS_FULL_PATH."""
> +    import re
> +
> +    template_body = d.getVar('_WKS_TEMPLATE')
> +
> +    # Remove any remnant variable references left behind by the expansion
> +    # due to undefined variables
> +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> +    while True:
> +        new_body = re.sub(expand_var_regexp, '', template_body)
> +        if new_body == template_body:
> +            break
> +        else:
> +            template_body = new_body
> +
> +    wks_file = d.getVar('WKS_FULL_PATH')
> +    with open(wks_file, 'w') as f:
> +        f.write(template_body)
> +}
> +
> +python () {
> +    if d.getVar('USING_WIC'):
> +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> +        wks_file = d.expand(wks_file_u)
> +        base, ext = os.path.splitext(wks_file)
> +        if ext == '.in' and os.path.exists(wks_file):
> +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> os.path.basename(base))
> +            d.setVar('WKS_FULL_PATH', wks_out_file)
> +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> +            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> +
> +            # We need to re-parse each time the file changes, and bitbake
> +            # needs to be told about that explicitly.
> +            bb.parse.mark_dependency(d, wks_file)
> +
> +            try:
> +                with open(wks_file, 'r') as f:
> +                    body = f.read()
> +            except (IOError, OSError) as exc:
> +                pass
> +            else:
> +                # Previously, I used expandWithRefs to get the dependency list
> +                # and add it to WICVARS, but there's no point re-parsing the
> +                # file in process_wks_template as well, so just put it in
> +                # a variable and let the metadata deal with the deps.
> +                d.setVar('_WKS_TEMPLATE', body)
> +                bb.build.addtask('do_write_wks_template',
> +'do_image_wic', None, d) }
> +
> +#
> +# Write environment variables used by wic # to
> +tmp/sysroots/<machine>/imgdata/<image>.env
> +#
> +python do_rootfs_wicenv () {
> +    wicvars = d.getVar('WICVARS')
> +    if not wicvars:
> +        return
> +
> +    stdir = d.getVar('STAGING_DIR')
> +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> +    bb.utils.mkdirhier(outdir)
> +    basename = d.getVar('IMAGE_BASENAME')
> +    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> +        for var in wicvars.split():
> +            value = d.getVar(var)
> +            if value:
> +                envf.write('%s="%s"\n' % (var, value.strip())) }
> +addtask do_rootfs_wicenv after do_image before do_image_wic
> +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index
> 13fd996..1a549e9 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -192,6 +192,8 @@ python () {
>  IMAGE_CLASSES += "image_types"
>  inherit ${IMAGE_CLASSES}
> 
> +inherit image-wic
> +
>  IMAGE_POSTPROCESS_COMMAND ?= ""
> 
>  # some default locales
> @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {  }  addtask
> do_image_qa after do_image_complete before do_build
> 
> -#
> -# Write environment variables used by wic -# to
> tmp/sysroots/<machine>/imgdata/<image>.env
> -#
> -python do_rootfs_wicenv () {
> -    wicvars = d.getVar('WICVARS')
> -    if not wicvars:
> -        return
> -
> -    stdir = d.getVar('STAGING_DIR')
> -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> -    bb.utils.mkdirhier(outdir)
> -    basename = d.getVar('IMAGE_BASENAME')
> -    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> -        for var in wicvars.split():
> -            value = d.getVar(var)
> -            if value:
> -                envf.write('%s="%s"\n' % (var, value.strip()))
> -}
> -addtask do_rootfs_wicenv after do_image before do_image_wic -
> do_rootfs_wicenv[vardeps] += "${WICVARS}"
> -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> -
>  def setup_debugfs_variables(d):
>      d.appendVar('IMAGE_ROOTFS', '-dbg')
>      d.appendVar('IMAGE_LINK_NAME', '-dbg') diff --git
> a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> index ad95dc6..e2ecf93 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> 
>  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> ${MKUBIFS_ARGS}"
> 
> -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in
> '${BBPATH}:${COREBASE}'.split(':'))}"
> -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> '${WKS_SEARCH_PATH}') or ''}"
> -
> -def wks_search(files, search_path):
> -    for f in files:
> -        if os.path.isabs(f):
> -            if os.path.exists(f):
> -                return f
> -        else:
> -            searched = bb.utils.which(search_path, f)
> -            if searched:
> -                return searched
> -
> -WIC_CREATE_EXTRA_ARGS ?= ""
> -
> -IMAGE_CMD_wic () {
> -	out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> -	wks="${WKS_FULL_PATH}"
> -	if [ -z "$wks" ]; then
> -		bbfatal "No kickstart files from WKS_FILES were
> found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> -	fi
> -
> -	BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> "$out/" ${WIC_CREATE_EXTRA_ARGS}
> -	mv "$out/$(basename "${wks%.wks}")"*.direct
> "$out${IMAGE_NAME_SUFFIX}.wic"
> -	rm -rf "$out/"
> -}
> -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> -
> -# Rebuild when the wks file or vars in WICVARS change -USING_WIC =
> "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c
> in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> -do_image_wic[depends] += "wic-tools:do_build"
> -
> -python () {
> -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> -}
> -
> -python do_write_wks_template () {
> -    """Write out expanded template contents to WKS_FULL_PATH."""
> -    import re
> -
> -    template_body = d.getVar('_WKS_TEMPLATE')
> -
> -    # Remove any remnant variable references left behind by the expansion
> -    # due to undefined variables
> -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> -    while True:
> -        new_body = re.sub(expand_var_regexp, '', template_body)
> -        if new_body == template_body:
> -            break
> -        else:
> -            template_body = new_body
> -
> -    wks_file = d.getVar('WKS_FULL_PATH')
> -    with open(wks_file, 'w') as f:
> -        f.write(template_body)
> -}
> -
> -python () {
> -    if d.getVar('USING_WIC'):
> -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> -        wks_file = d.expand(wks_file_u)
> -        base, ext = os.path.splitext(wks_file)
> -        if ext == '.in' and os.path.exists(wks_file):
> -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> os.path.basename(base))
> -            d.setVar('WKS_FULL_PATH', wks_out_file)
> -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> -            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> -
> -            # We need to re-parse each time the file changes, and bitbake
> -            # needs to be told about that explicitly.
> -            bb.parse.mark_dependency(d, wks_file)
> -
> -            try:
> -                with open(wks_file, 'r') as f:
> -                    body = f.read()
> -            except (IOError, OSError) as exc:
> -                pass
> -            else:
> -                # Previously, I used expandWithRefs to get the dependency list
> -                # and add it to WICVARS, but there's no point re-parsing the
> -                # file in process_wks_template as well, so just put it in
> -                # a variable and let the metadata deal with the deps.
> -                d.setVar('_WKS_TEMPLATE', body)
> -                bb.build.addtask('do_write_wks_template', 'do_image_wic', None,
> d)
> -}
> -
>  EXTRA_IMAGECMD = ""
> 
>  inherit siteinfo
> @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
>  # The IMAGE_TYPES_MASKED variable will be used to mask out from the
> IMAGE_FSTYPES,  # images that will not be built at do_rootfs time: vmdk, vdi,
> qcow2, hdddirect, hddimg, iso, etc.
>  IMAGE_TYPES_MASKED ?= ""
> -
> -# The WICVARS variable is used to define list of bitbake variables used in wic
> code -# variables from this list is written to <image>.env file -WICVARS ?=
> "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME
> IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> RECIPE_SYSROOT_NATIVE ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> STAGING_LIBDIR TARGET_SYS"
> --
> 2.1.4
> 
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

On Mon, Jan 30, 2017 at 10:25:12AM +0000, Ola x Nilsson wrote:
> Please name the class image_wic.bbclass instead.
> 
> As dashes are not allowed in function names, you cannot use 
> the image_wic_do_write_wks_template pattern of function 
> names if you use a class name with a dash in it.
> 

Thanks for pointing out to it. Will fix it in the next versison of this
patchset.

> -- 
> Ola x Nilsson
> 
> > -----Original Message-----
> > From: openembedded-core-bounces@lists.openembedded.org
> > [mailto:openembedded-core-bounces@lists.openembedded.org] On Behalf
> > Of Ed Bartosh
> > Sent: den 27 januari 2017 21:20
> > To: openembedded-core@lists.openembedded.org
> > Subject: [OE-core] [PATCH v3 01/11] image-wic: move wic code to image-
> > wic.bbclass
> > 
> > There is a lot of wic code in image.bbclass and image_types.bbclass Having all
> > code separated in one file should make it more readable and easier to
> > maintain.
> > 
> > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > ---
> >  meta/classes/image-wic.bbclass   | 120
> > +++++++++++++++++++++++++++++++++++++++
> >  meta/classes/image.bbclass       |  25 +-------
> >  meta/classes/image_types.bbclass |  95 -------------------------------
> >  3 files changed, 122 insertions(+), 118 deletions(-)  create mode 100644
> > meta/classes/image-wic.bbclass
> > 
> > diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-
> > wic.bbclass new file mode 100644 index 0000000..2acfd65
> > --- /dev/null
> > +++ b/meta/classes/image-wic.bbclass
> > @@ -0,0 +1,120 @@
> > +# The WICVARS variable is used to define list of bitbake variables used
> > +in wic code # variables from this list is written to <image>.env file
> > +WICVARS ?= "\
> > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES \
> > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> > ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > TARGET_SYS"
> > +
> > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in
> > '${BBPATH}:${COREBASE}'.split(':'))}"
> > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > '${WKS_SEARCH_PATH}') or ''}"
> > +
> > +def wks_search(files, search_path):
> > +    for f in files:
> > +        if os.path.isabs(f):
> > +            if os.path.exists(f):
> > +                return f
> > +        else:
> > +            searched = bb.utils.which(search_path, f)
> > +            if searched:
> > +                return searched
> > +
> > +WIC_CREATE_EXTRA_ARGS ?= ""
> > +
> > +IMAGE_CMD_wic () {
> > +	out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > +	wks="${WKS_FULL_PATH}"
> > +	if [ -z "$wks" ]; then
> > +		bbfatal "No kickstart files from WKS_FILES were
> > found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > +	fi
> > +
> > +	BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > "$out/" ${WIC_CREATE_EXTRA_ARGS}
> > +	mv "$out/$(basename "${wks%.wks}")"*.direct
> > "$out${IMAGE_NAME_SUFFIX}.wic"
> > +	rm -rf "$out/"
> > +}
> > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > +
> > +# Rebuild when the wks file or vars in WICVARS change USING_WIC =
> > +"${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for
> > c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > +do_image_wic[depends] += "wic-tools:do_build"
> > +
> > +python () {
> > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d) }
> > +
> > +python do_write_wks_template () {
> > +    """Write out expanded template contents to WKS_FULL_PATH."""
> > +    import re
> > +
> > +    template_body = d.getVar('_WKS_TEMPLATE')
> > +
> > +    # Remove any remnant variable references left behind by the expansion
> > +    # due to undefined variables
> > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > +    while True:
> > +        new_body = re.sub(expand_var_regexp, '', template_body)
> > +        if new_body == template_body:
> > +            break
> > +        else:
> > +            template_body = new_body
> > +
> > +    wks_file = d.getVar('WKS_FULL_PATH')
> > +    with open(wks_file, 'w') as f:
> > +        f.write(template_body)
> > +}
> > +
> > +python () {
> > +    if d.getVar('USING_WIC'):
> > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > +        wks_file = d.expand(wks_file_u)
> > +        base, ext = os.path.splitext(wks_file)
> > +        if ext == '.in' and os.path.exists(wks_file):
> > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > os.path.basename(base))
> > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > +            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > +
> > +            # We need to re-parse each time the file changes, and bitbake
> > +            # needs to be told about that explicitly.
> > +            bb.parse.mark_dependency(d, wks_file)
> > +
> > +            try:
> > +                with open(wks_file, 'r') as f:
> > +                    body = f.read()
> > +            except (IOError, OSError) as exc:
> > +                pass
> > +            else:
> > +                # Previously, I used expandWithRefs to get the dependency list
> > +                # and add it to WICVARS, but there's no point re-parsing the
> > +                # file in process_wks_template as well, so just put it in
> > +                # a variable and let the metadata deal with the deps.
> > +                d.setVar('_WKS_TEMPLATE', body)
> > +                bb.build.addtask('do_write_wks_template',
> > +'do_image_wic', None, d) }
> > +
> > +#
> > +# Write environment variables used by wic # to
> > +tmp/sysroots/<machine>/imgdata/<image>.env
> > +#
> > +python do_rootfs_wicenv () {
> > +    wicvars = d.getVar('WICVARS')
> > +    if not wicvars:
> > +        return
> > +
> > +    stdir = d.getVar('STAGING_DIR')
> > +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > +    bb.utils.mkdirhier(outdir)
> > +    basename = d.getVar('IMAGE_BASENAME')
> > +    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > +        for var in wicvars.split():
> > +            value = d.getVar(var)
> > +            if value:
> > +                envf.write('%s="%s"\n' % (var, value.strip())) }
> > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index
> > 13fd996..1a549e9 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -192,6 +192,8 @@ python () {
> >  IMAGE_CLASSES += "image_types"
> >  inherit ${IMAGE_CLASSES}
> > 
> > +inherit image-wic
> > +
> >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > 
> >  # some default locales
> > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {  }  addtask
> > do_image_qa after do_image_complete before do_build
> > 
> > -#
> > -# Write environment variables used by wic -# to
> > tmp/sysroots/<machine>/imgdata/<image>.env
> > -#
> > -python do_rootfs_wicenv () {
> > -    wicvars = d.getVar('WICVARS')
> > -    if not wicvars:
> > -        return
> > -
> > -    stdir = d.getVar('STAGING_DIR')
> > -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > -    bb.utils.mkdirhier(outdir)
> > -    basename = d.getVar('IMAGE_BASENAME')
> > -    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > -        for var in wicvars.split():
> > -            value = d.getVar(var)
> > -            if value:
> > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > -}
> > -addtask do_rootfs_wicenv after do_image before do_image_wic -
> > do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > -
> >  def setup_debugfs_variables(d):
> >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> >      d.appendVar('IMAGE_LINK_NAME', '-dbg') diff --git
> > a/meta/classes/image_types.bbclass b/meta/classes/image_types.bbclass
> > index ad95dc6..e2ecf93 100644
> > --- a/meta/classes/image_types.bbclass
> > +++ b/meta/classes/image_types.bbclass
> > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > 
> >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > ${MKUBIFS_ARGS}"
> > 
> > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l for l in
> > '${BBPATH}:${COREBASE}'.split(':'))}"
> > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > '${WKS_SEARCH_PATH}') or ''}"
> > -
> > -def wks_search(files, search_path):
> > -    for f in files:
> > -        if os.path.isabs(f):
> > -            if os.path.exists(f):
> > -                return f
> > -        else:
> > -            searched = bb.utils.which(search_path, f)
> > -            if searched:
> > -                return searched
> > -
> > -WIC_CREATE_EXTRA_ARGS ?= ""
> > -
> > -IMAGE_CMD_wic () {
> > -	out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > -	wks="${WKS_FULL_PATH}"
> > -	if [ -z "$wks" ]; then
> > -		bbfatal "No kickstart files from WKS_FILES were
> > found: ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > -	fi
> > -
> > -	BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > "$out/" ${WIC_CREATE_EXTRA_ARGS}
> > -	mv "$out/$(basename "${wks%.wks}")"*.direct
> > "$out${IMAGE_NAME_SUFFIX}.wic"
> > -	rm -rf "$out/"
> > -}
> > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > -
> > -# Rebuild when the wks file or vars in WICVARS change -USING_WIC =
> > "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + ' '.join('wic.%s' % c for c
> > in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > -do_image_wic[depends] += "wic-tools:do_build"
> > -
> > -python () {
> > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > -}
> > -
> > -python do_write_wks_template () {
> > -    """Write out expanded template contents to WKS_FULL_PATH."""
> > -    import re
> > -
> > -    template_body = d.getVar('_WKS_TEMPLATE')
> > -
> > -    # Remove any remnant variable references left behind by the expansion
> > -    # due to undefined variables
> > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > -    while True:
> > -        new_body = re.sub(expand_var_regexp, '', template_body)
> > -        if new_body == template_body:
> > -            break
> > -        else:
> > -            template_body = new_body
> > -
> > -    wks_file = d.getVar('WKS_FULL_PATH')
> > -    with open(wks_file, 'w') as f:
> > -        f.write(template_body)
> > -}
> > -
> > -python () {
> > -    if d.getVar('USING_WIC'):
> > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > -        wks_file = d.expand(wks_file_u)
> > -        base, ext = os.path.splitext(wks_file)
> > -        if ext == '.in' and os.path.exists(wks_file):
> > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > os.path.basename(base))
> > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > -            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > -
> > -            # We need to re-parse each time the file changes, and bitbake
> > -            # needs to be told about that explicitly.
> > -            bb.parse.mark_dependency(d, wks_file)
> > -
> > -            try:
> > -                with open(wks_file, 'r') as f:
> > -                    body = f.read()
> > -            except (IOError, OSError) as exc:
> > -                pass
> > -            else:
> > -                # Previously, I used expandWithRefs to get the dependency list
> > -                # and add it to WICVARS, but there's no point re-parsing the
> > -                # file in process_wks_template as well, so just put it in
> > -                # a variable and let the metadata deal with the deps.
> > -                d.setVar('_WKS_TEMPLATE', body)
> > -                bb.build.addtask('do_write_wks_template', 'do_image_wic', None,
> > d)
> > -}
> > -
> >  EXTRA_IMAGECMD = ""
> > 
> >  inherit siteinfo
> > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> >  # The IMAGE_TYPES_MASKED variable will be used to mask out from the
> > IMAGE_FSTYPES,  # images that will not be built at do_rootfs time: vmdk, vdi,
> > qcow2, hdddirect, hddimg, iso, etc.
> >  IMAGE_TYPES_MASKED ?= ""
> > -
> > -# The WICVARS variable is used to define list of bitbake variables used in wic
> > code -# variables from this list is written to <image>.env file -WICVARS ?=
> > "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME
> > IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> > RECIPE_SYSROOT_NATIVE ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> > STAGING_LIBDIR TARGET_SYS"
> > --
> > 2.1.4
> > 
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core

-- 
--
Regards,
Ed

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> Why didn't you make this image_types_wic.bbclass and use IMAGE_CLASSES to
> load it?

Because of the following:
- IMAGE_CLASSES[doc] = "A list of classes that all images should
  inherit." I'm not sure all images should include wic class. I'll probably
  make this inheritance conditional.
- so far IMAGE_CLASSES is used for qemuboot, image_types,
  image_types_uboot and testimage,
  so the usage is more or less follows the description. wic is out of
  that usage scenario, I believe.
- 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
  "image_types"\n inherit ${IMAGE_CLASSES}


> 
> On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <ed.bartosh@linux.intel.com>
> wrote:
> 
> > There is a lot of wic code in image.bbclass and image_types.bbclass
> > Having all code separated in one file should make it more readable
> > and easier to maintain.
> >
> > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > ---
> >  meta/classes/image-wic.bbclass   | 120 ++++++++++++++++++++++++++++++
> > +++++++++
> >  meta/classes/image.bbclass       |  25 +-------
> >  meta/classes/image_types.bbclass |  95 -------------------------------
> >  3 files changed, 122 insertions(+), 118 deletions(-)
> >  create mode 100644 meta/classes/image-wic.bbclass
> >
> > diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.
> > bbclass
> > new file mode 100644
> > index 0000000..2acfd65
> > --- /dev/null
> > +++ b/meta/classes/image-wic.bbclass
> > @@ -0,0 +1,120 @@
> > +# The WICVARS variable is used to define list of bitbake variables used
> > in wic code
> > +# variables from this list is written to <image>.env file
> > +WICVARS ?= "\
> > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR
> > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > TARGET_SYS"
> > +
> > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l
> > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > '${WKS_SEARCH_PATH}') or ''}"
> > +
> > +def wks_search(files, search_path):
> > +    for f in files:
> > +        if os.path.isabs(f):
> > +            if os.path.exists(f):
> > +                return f
> > +        else:
> > +            searched = bb.utils.which(search_path, f)
> > +            if searched:
> > +                return searched
> > +
> > +WIC_CREATE_EXTRA_ARGS ?= ""
> > +
> > +IMAGE_CMD_wic () {
> > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > +       wks="${WKS_FULL_PATH}"
> > +       if [ -z "$wks" ]; then
> > +               bbfatal "No kickstart files from WKS_FILES were found:
> > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > +       fi
> > +
> > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > ${WIC_CREATE_EXTRA_ARGS}
> > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > "$out${IMAGE_NAME_SUFFIX}.wic"
> > +       rm -rf "$out/"
> > +}
> > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > +
> > +# Rebuild when the wks file or vars in WICVARS change
> > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > +do_image_wic[depends] += "wic-tools:do_build"
> > +
> > +python () {
> > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > +}
> > +
> > +python do_write_wks_template () {
> > +    """Write out expanded template contents to WKS_FULL_PATH."""
> > +    import re
> > +
> > +    template_body = d.getVar('_WKS_TEMPLATE')
> > +
> > +    # Remove any remnant variable references left behind by the expansion
> > +    # due to undefined variables
> > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > +    while True:
> > +        new_body = re.sub(expand_var_regexp, '', template_body)
> > +        if new_body == template_body:
> > +            break
> > +        else:
> > +            template_body = new_body
> > +
> > +    wks_file = d.getVar('WKS_FULL_PATH')
> > +    with open(wks_file, 'w') as f:
> > +        f.write(template_body)
> > +}
> > +
> > +python () {
> > +    if d.getVar('USING_WIC'):
> > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > +        wks_file = d.expand(wks_file_u)
> > +        base, ext = os.path.splitext(wks_file)
> > +        if ext == '.in' and os.path.exists(wks_file):
> > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > os.path.basename(base))
> > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > +            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > +
> > +            # We need to re-parse each time the file changes, and bitbake
> > +            # needs to be told about that explicitly.
> > +            bb.parse.mark_dependency(d, wks_file)
> > +
> > +            try:
> > +                with open(wks_file, 'r') as f:
> > +                    body = f.read()
> > +            except (IOError, OSError) as exc:
> > +                pass
> > +            else:
> > +                # Previously, I used expandWithRefs to get the dependency
> > list
> > +                # and add it to WICVARS, but there's no point re-parsing
> > the
> > +                # file in process_wks_template as well, so just put it in
> > +                # a variable and let the metadata deal with the deps.
> > +                d.setVar('_WKS_TEMPLATE', body)
> > +                bb.build.addtask('do_write_wks_template',
> > 'do_image_wic', None, d)
> > +}
> > +
> > +#
> > +# Write environment variables used by wic
> > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > +#
> > +python do_rootfs_wicenv () {
> > +    wicvars = d.getVar('WICVARS')
> > +    if not wicvars:
> > +        return
> > +
> > +    stdir = d.getVar('STAGING_DIR')
> > +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > +    bb.utils.mkdirhier(outdir)
> > +    basename = d.getVar('IMAGE_BASENAME')
> > +    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > +        for var in wicvars.split():
> > +            value = d.getVar(var)
> > +            if value:
> > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > +}
> > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > index 13fd996..1a549e9 100644
> > --- a/meta/classes/image.bbclass
> > +++ b/meta/classes/image.bbclass
> > @@ -192,6 +192,8 @@ python () {
> >  IMAGE_CLASSES += "image_types"
> >  inherit ${IMAGE_CLASSES}
> >
> > +inherit image-wic
> > +
> >  IMAGE_POSTPROCESS_COMMAND ?= ""
> >
> >  # some default locales
> > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> >  }
> >  addtask do_image_qa after do_image_complete before do_build
> >
> > -#
> > -# Write environment variables used by wic
> > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > -#
> > -python do_rootfs_wicenv () {
> > -    wicvars = d.getVar('WICVARS')
> > -    if not wicvars:
> > -        return
> > -
> > -    stdir = d.getVar('STAGING_DIR')
> > -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > -    bb.utils.mkdirhier(outdir)
> > -    basename = d.getVar('IMAGE_BASENAME')
> > -    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > -        for var in wicvars.split():
> > -            value = d.getVar(var)
> > -            if value:
> > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > -}
> > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > -
> >  def setup_debugfs_variables(d):
> >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.
> > bbclass
> > index ad95dc6..e2ecf93 100644
> > --- a/meta/classes/image_types.bbclass
> > +++ b/meta/classes/image_types.bbclass
> > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> >
> >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs ${MKUBIFS_ARGS}"
> >
> > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l
> > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > '${WKS_SEARCH_PATH}') or ''}"
> > -
> > -def wks_search(files, search_path):
> > -    for f in files:
> > -        if os.path.isabs(f):
> > -            if os.path.exists(f):
> > -                return f
> > -        else:
> > -            searched = bb.utils.which(search_path, f)
> > -            if searched:
> > -                return searched
> > -
> > -WIC_CREATE_EXTRA_ARGS ?= ""
> > -
> > -IMAGE_CMD_wic () {
> > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > -       wks="${WKS_FULL_PATH}"
> > -       if [ -z "$wks" ]; then
> > -               bbfatal "No kickstart files from WKS_FILES were found:
> > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > -       fi
> > -
> > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > ${WIC_CREATE_EXTRA_ARGS}
> > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > "$out${IMAGE_NAME_SUFFIX}.wic"
> > -       rm -rf "$out/"
> > -}
> > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > -
> > -# Rebuild when the wks file or vars in WICVARS change
> > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > -do_image_wic[depends] += "wic-tools:do_build"
> > -
> > -python () {
> > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > -}
> > -
> > -python do_write_wks_template () {
> > -    """Write out expanded template contents to WKS_FULL_PATH."""
> > -    import re
> > -
> > -    template_body = d.getVar('_WKS_TEMPLATE')
> > -
> > -    # Remove any remnant variable references left behind by the expansion
> > -    # due to undefined variables
> > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > -    while True:
> > -        new_body = re.sub(expand_var_regexp, '', template_body)
> > -        if new_body == template_body:
> > -            break
> > -        else:
> > -            template_body = new_body
> > -
> > -    wks_file = d.getVar('WKS_FULL_PATH')
> > -    with open(wks_file, 'w') as f:
> > -        f.write(template_body)
> > -}
> > -
> > -python () {
> > -    if d.getVar('USING_WIC'):
> > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > -        wks_file = d.expand(wks_file_u)
> > -        base, ext = os.path.splitext(wks_file)
> > -        if ext == '.in' and os.path.exists(wks_file):
> > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > os.path.basename(base))
> > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > -            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > -
> > -            # We need to re-parse each time the file changes, and bitbake
> > -            # needs to be told about that explicitly.
> > -            bb.parse.mark_dependency(d, wks_file)
> > -
> > -            try:
> > -                with open(wks_file, 'r') as f:
> > -                    body = f.read()
> > -            except (IOError, OSError) as exc:
> > -                pass
> > -            else:
> > -                # Previously, I used expandWithRefs to get the dependency
> > list
> > -                # and add it to WICVARS, but there's no point re-parsing
> > the
> > -                # file in process_wks_template as well, so just put it in
> > -                # a variable and let the metadata deal with the deps.
> > -                d.setVar('_WKS_TEMPLATE', body)
> > -                bb.build.addtask('do_write_wks_template',
> > 'do_image_wic', None, d)
> > -}
> > -
> >  EXTRA_IMAGECMD = ""
> >
> >  inherit siteinfo
> > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> >  # The IMAGE_TYPES_MASKED variable will be used to mask out from the
> > IMAGE_FSTYPES,
> >  # images that will not be built at do_rootfs time: vmdk, vdi, qcow2,
> > hdddirect, hddimg, iso, etc.
> >  IMAGE_TYPES_MASKED ?= ""
> > -
> > -# The WICVARS variable is used to define list of bitbake variables used
> > in wic code
> > -# variables from this list is written to <image>.env file
> > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE
> > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
> > --
> > 2.1.4
> >
> > --
> > _______________________________________________
> > Openembedded-core mailing list
> > Openembedded-core@lists.openembedded.org
> > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> >

-- 
--
Regards,
Ed

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Rick Altherr]

[-- Attachment #1: Type: text/plain, Size: 13318 bytes --]

Why didn't you make this image_types_wic.bbclass and use IMAGE_CLASSES to
load it?

On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> There is a lot of wic code in image.bbclass and image_types.bbclass
> Having all code separated in one file should make it more readable
> and easier to maintain.
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>  meta/classes/image-wic.bbclass   | 120 ++++++++++++++++++++++++++++++
> +++++++++
>  meta/classes/image.bbclass       |  25 +-------
>  meta/classes/image_types.bbclass |  95 -------------------------------
>  3 files changed, 122 insertions(+), 118 deletions(-)
>  create mode 100644 meta/classes/image-wic.bbclass
>
> diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.
> bbclass
> new file mode 100644
> index 0000000..2acfd65
> --- /dev/null
> +++ b/meta/classes/image-wic.bbclass
> @@ -0,0 +1,120 @@
> +# The WICVARS variable is used to define list of bitbake variables used
> in wic code
> +# variables from this list is written to <image>.env file
> +WICVARS ?= "\
> +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> IMAGE_BASENAME IMAGE_BOOT_FILES \
> +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR
> MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> TARGET_SYS"
> +
> +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l
> for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> '${WKS_SEARCH_PATH}') or ''}"
> +
> +def wks_search(files, search_path):
> +    for f in files:
> +        if os.path.isabs(f):
> +            if os.path.exists(f):
> +                return f
> +        else:
> +            searched = bb.utils.which(search_path, f)
> +            if searched:
> +                return searched
> +
> +WIC_CREATE_EXTRA_ARGS ?= ""
> +
> +IMAGE_CMD_wic () {
> +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> +       wks="${WKS_FULL_PATH}"
> +       if [ -z "$wks" ]; then
> +               bbfatal "No kickstart files from WKS_FILES were found:
> ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> +       fi
> +
> +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> ${WIC_CREATE_EXTRA_ARGS}
> +       mv "$out/$(basename "${wks%.wks}")"*.direct
> "$out${IMAGE_NAME_SUFFIX}.wic"
> +       rm -rf "$out/"
> +}
> +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> +
> +# Rebuild when the wks file or vars in WICVARS change
> +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> +do_image_wic[depends] += "wic-tools:do_build"
> +
> +python () {
> +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> +}
> +
> +python do_write_wks_template () {
> +    """Write out expanded template contents to WKS_FULL_PATH."""
> +    import re
> +
> +    template_body = d.getVar('_WKS_TEMPLATE')
> +
> +    # Remove any remnant variable references left behind by the expansion
> +    # due to undefined variables
> +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> +    while True:
> +        new_body = re.sub(expand_var_regexp, '', template_body)
> +        if new_body == template_body:
> +            break
> +        else:
> +            template_body = new_body
> +
> +    wks_file = d.getVar('WKS_FULL_PATH')
> +    with open(wks_file, 'w') as f:
> +        f.write(template_body)
> +}
> +
> +python () {
> +    if d.getVar('USING_WIC'):
> +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> +        wks_file = d.expand(wks_file_u)
> +        base, ext = os.path.splitext(wks_file)
> +        if ext == '.in' and os.path.exists(wks_file):
> +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> os.path.basename(base))
> +            d.setVar('WKS_FULL_PATH', wks_out_file)
> +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> +            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> +
> +            # We need to re-parse each time the file changes, and bitbake
> +            # needs to be told about that explicitly.
> +            bb.parse.mark_dependency(d, wks_file)
> +
> +            try:
> +                with open(wks_file, 'r') as f:
> +                    body = f.read()
> +            except (IOError, OSError) as exc:
> +                pass
> +            else:
> +                # Previously, I used expandWithRefs to get the dependency
> list
> +                # and add it to WICVARS, but there's no point re-parsing
> the
> +                # file in process_wks_template as well, so just put it in
> +                # a variable and let the metadata deal with the deps.
> +                d.setVar('_WKS_TEMPLATE', body)
> +                bb.build.addtask('do_write_wks_template',
> 'do_image_wic', None, d)
> +}
> +
> +#
> +# Write environment variables used by wic
> +# to tmp/sysroots/<machine>/imgdata/<image>.env
> +#
> +python do_rootfs_wicenv () {
> +    wicvars = d.getVar('WICVARS')
> +    if not wicvars:
> +        return
> +
> +    stdir = d.getVar('STAGING_DIR')
> +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> +    bb.utils.mkdirhier(outdir)
> +    basename = d.getVar('IMAGE_BASENAME')
> +    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> +        for var in wicvars.split():
> +            value = d.getVar(var)
> +            if value:
> +                envf.write('%s="%s"\n' % (var, value.strip()))
> +}
> +addtask do_rootfs_wicenv after do_image before do_image_wic
> +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 13fd996..1a549e9 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -192,6 +192,8 @@ python () {
>  IMAGE_CLASSES += "image_types"
>  inherit ${IMAGE_CLASSES}
>
> +inherit image-wic
> +
>  IMAGE_POSTPROCESS_COMMAND ?= ""
>
>  # some default locales
> @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
>  }
>  addtask do_image_qa after do_image_complete before do_build
>
> -#
> -# Write environment variables used by wic
> -# to tmp/sysroots/<machine>/imgdata/<image>.env
> -#
> -python do_rootfs_wicenv () {
> -    wicvars = d.getVar('WICVARS')
> -    if not wicvars:
> -        return
> -
> -    stdir = d.getVar('STAGING_DIR')
> -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> -    bb.utils.mkdirhier(outdir)
> -    basename = d.getVar('IMAGE_BASENAME')
> -    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> -        for var in wicvars.split():
> -            value = d.getVar(var)
> -            if value:
> -                envf.write('%s="%s"\n' % (var, value.strip()))
> -}
> -addtask do_rootfs_wicenv after do_image before do_image_wic
> -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> -
>  def setup_debugfs_variables(d):
>      d.appendVar('IMAGE_ROOTFS', '-dbg')
>      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> diff --git a/meta/classes/image_types.bbclass b/meta/classes/image_types.
> bbclass
> index ad95dc6..e2ecf93 100644
> --- a/meta/classes/image_types.bbclass
> +++ b/meta/classes/image_types.bbclass
> @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
>
>  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs ${MKUBIFS_ARGS}"
>
> -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks' % l
> for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> '${WKS_SEARCH_PATH}') or ''}"
> -
> -def wks_search(files, search_path):
> -    for f in files:
> -        if os.path.isabs(f):
> -            if os.path.exists(f):
> -                return f
> -        else:
> -            searched = bb.utils.which(search_path, f)
> -            if searched:
> -                return searched
> -
> -WIC_CREATE_EXTRA_ARGS ?= ""
> -
> -IMAGE_CMD_wic () {
> -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> -       wks="${WKS_FULL_PATH}"
> -       if [ -z "$wks" ]; then
> -               bbfatal "No kickstart files from WKS_FILES were found:
> ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> -       fi
> -
> -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> ${WIC_CREATE_EXTRA_ARGS}
> -       mv "$out/$(basename "${wks%.wks}")"*.direct
> "$out${IMAGE_NAME_SUFFIX}.wic"
> -       rm -rf "$out/"
> -}
> -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> -
> -# Rebuild when the wks file or vars in WICVARS change
> -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '', d)}"
> -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> -do_image_wic[depends] += "wic-tools:do_build"
> -
> -python () {
> -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> -}
> -
> -python do_write_wks_template () {
> -    """Write out expanded template contents to WKS_FULL_PATH."""
> -    import re
> -
> -    template_body = d.getVar('_WKS_TEMPLATE')
> -
> -    # Remove any remnant variable references left behind by the expansion
> -    # due to undefined variables
> -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> -    while True:
> -        new_body = re.sub(expand_var_regexp, '', template_body)
> -        if new_body == template_body:
> -            break
> -        else:
> -            template_body = new_body
> -
> -    wks_file = d.getVar('WKS_FULL_PATH')
> -    with open(wks_file, 'w') as f:
> -        f.write(template_body)
> -}
> -
> -python () {
> -    if d.getVar('USING_WIC'):
> -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> -        wks_file = d.expand(wks_file_u)
> -        base, ext = os.path.splitext(wks_file)
> -        if ext == '.in' and os.path.exists(wks_file):
> -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> os.path.basename(base))
> -            d.setVar('WKS_FULL_PATH', wks_out_file)
> -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> -            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> -
> -            # We need to re-parse each time the file changes, and bitbake
> -            # needs to be told about that explicitly.
> -            bb.parse.mark_dependency(d, wks_file)
> -
> -            try:
> -                with open(wks_file, 'r') as f:
> -                    body = f.read()
> -            except (IOError, OSError) as exc:
> -                pass
> -            else:
> -                # Previously, I used expandWithRefs to get the dependency
> list
> -                # and add it to WICVARS, but there's no point re-parsing
> the
> -                # file in process_wks_template as well, so just put it in
> -                # a variable and let the metadata deal with the deps.
> -                d.setVar('_WKS_TEMPLATE', body)
> -                bb.build.addtask('do_write_wks_template',
> 'do_image_wic', None, d)
> -}
> -
>  EXTRA_IMAGECMD = ""
>
>  inherit siteinfo
> @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
>  # The IMAGE_TYPES_MASKED variable will be used to mask out from the
> IMAGE_FSTYPES,
>  # images that will not be built at do_rootfs time: vmdk, vdi, qcow2,
> hdddirect, hddimg, iso, etc.
>  IMAGE_TYPES_MASKED ?= ""
> -
> -# The WICVARS variable is used to define list of bitbake variables used
> in wic code
> -# variables from this list is written to <image>.env file
> -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE
> ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
> --
> 2.1.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>

[-- Attachment #2: Type: text/html, Size: 16868 bytes --]

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> Hmm.  In local.conf.sample.extended, I find:
> 
> # Additional image generation features
> #
> # The following is a list of classes to import to use in the generation of
> images
> # currently an example class is image_types_uboot
> # IMAGE_CLASSES = " image_types_uboot"
> 
> Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.  I'd
> expect a machine config to add wic to IMAGE_CLASSES if it needs wic output.
> 

So far all machine configs add wic to IMAGE_TYPES and it works just
fine. Why to change?

> On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> wrote:
> 
> > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > Why didn't you make this image_types_wic.bbclass and use IMAGE_CLASSES to
> > > load it?
> >
> > Because of the following:
> > - IMAGE_CLASSES[doc] = "A list of classes that all images should
> >   inherit." I'm not sure all images should include wic class. I'll probably
> >   make this inheritance conditional.
> > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> >   image_types_uboot and testimage,
> >   so the usage is more or less follows the description. wic is out of
> >   that usage scenario, I believe.
> > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> >   "image_types"\n inherit ${IMAGE_CLASSES}
> >
> >
> > >
> > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <ed.bartosh@linux.intel.com
> > >
> > > wrote:
> > >
> > > > There is a lot of wic code in image.bbclass and image_types.bbclass
> > > > Having all code separated in one file should make it more readable
> > > > and easier to maintain.
> > > >
> > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > ---
> > > >  meta/classes/image-wic.bbclass   | 120 ++++++++++++++++++++++++++++++
> > > > +++++++++
> > > >  meta/classes/image.bbclass       |  25 +-------
> > > >  meta/classes/image_types.bbclass |  95 ------------------------------
> > -
> > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > >
> > > > diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.
> > > > bbclass
> > > > new file mode 100644
> > > > index 0000000..2acfd65
> > > > --- /dev/null
> > > > +++ b/meta/classes/image-wic.bbclass
> > > > @@ -0,0 +1,120 @@
> > > > +# The WICVARS variable is used to define list of bitbake variables
> > used
> > > > in wic code
> > > > +# variables from this list is written to <image>.env file
> > > > +WICVARS ?= "\
> > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> > ISODIR
> > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > > TARGET_SYS"
> > > > +
> > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks'
> > % l
> > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > +
> > > > +def wks_search(files, search_path):
> > > > +    for f in files:
> > > > +        if os.path.isabs(f):
> > > > +            if os.path.exists(f):
> > > > +                return f
> > > > +        else:
> > > > +            searched = bb.utils.which(search_path, f)
> > > > +            if searched:
> > > > +                return searched
> > > > +
> > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > +
> > > > +IMAGE_CMD_wic () {
> > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > +       wks="${WKS_FULL_PATH}"
> > > > +       if [ -z "$wks" ]; then
> > > > +               bbfatal "No kickstart files from WKS_FILES were found:
> > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > +       fi
> > > > +
> > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > +       rm -rf "$out/"
> > > > +}
> > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > +
> > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '',
> > d)}"
> > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > +
> > > > +python () {
> > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > +}
> > > > +
> > > > +python do_write_wks_template () {
> > > > +    """Write out expanded template contents to WKS_FULL_PATH."""
> > > > +    import re
> > > > +
> > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > +
> > > > +    # Remove any remnant variable references left behind by the
> > expansion
> > > > +    # due to undefined variables
> > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > +    while True:
> > > > +        new_body = re.sub(expand_var_regexp, '', template_body)
> > > > +        if new_body == template_body:
> > > > +            break
> > > > +        else:
> > > > +            template_body = new_body
> > > > +
> > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > +    with open(wks_file, 'w') as f:
> > > > +        f.write(template_body)
> > > > +}
> > > > +
> > > > +python () {
> > > > +    if d.getVar('USING_WIC'):
> > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > +        wks_file = d.expand(wks_file_u)
> > > > +        base, ext = os.path.splitext(wks_file)
> > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > os.path.basename(base))
> > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > +            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > > > +
> > > > +            # We need to re-parse each time the file changes, and
> > bitbake
> > > > +            # needs to be told about that explicitly.
> > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > +
> > > > +            try:
> > > > +                with open(wks_file, 'r') as f:
> > > > +                    body = f.read()
> > > > +            except (IOError, OSError) as exc:
> > > > +                pass
> > > > +            else:
> > > > +                # Previously, I used expandWithRefs to get the
> > dependency
> > > > list
> > > > +                # and add it to WICVARS, but there's no point
> > re-parsing
> > > > the
> > > > +                # file in process_wks_template as well, so just put
> > it in
> > > > +                # a variable and let the metadata deal with the deps.
> > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > +                bb.build.addtask('do_write_wks_template',
> > > > 'do_image_wic', None, d)
> > > > +}
> > > > +
> > > > +#
> > > > +# Write environment variables used by wic
> > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > +#
> > > > +python do_rootfs_wicenv () {
> > > > +    wicvars = d.getVar('WICVARS')
> > > > +    if not wicvars:
> > > > +        return
> > > > +
> > > > +    stdir = d.getVar('STAGING_DIR')
> > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > > +    bb.utils.mkdirhier(outdir)
> > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > +    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > > > +        for var in wicvars.split():
> > > > +            value = d.getVar(var)
> > > > +            if value:
> > > > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > +}
> > > > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > > > index 13fd996..1a549e9 100644
> > > > --- a/meta/classes/image.bbclass
> > > > +++ b/meta/classes/image.bbclass
> > > > @@ -192,6 +192,8 @@ python () {
> > > >  IMAGE_CLASSES += "image_types"
> > > >  inherit ${IMAGE_CLASSES}
> > > >
> > > > +inherit image-wic
> > > > +
> > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > >
> > > >  # some default locales
> > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > >  }
> > > >  addtask do_image_qa after do_image_complete before do_build
> > > >
> > > > -#
> > > > -# Write environment variables used by wic
> > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > -#
> > > > -python do_rootfs_wicenv () {
> > > > -    wicvars = d.getVar('WICVARS')
> > > > -    if not wicvars:
> > > > -        return
> > > > -
> > > > -    stdir = d.getVar('STAGING_DIR')
> > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > > -    bb.utils.mkdirhier(outdir)
> > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > -    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > > > -        for var in wicvars.split():
> > > > -            value = d.getVar(var)
> > > > -            if value:
> > > > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > -}
> > > > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > -
> > > >  def setup_debugfs_variables(d):
> > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > diff --git a/meta/classes/image_types.bbclass
> > b/meta/classes/image_types.
> > > > bbclass
> > > > index ad95dc6..e2ecf93 100644
> > > > --- a/meta/classes/image_types.bbclass
> > > > +++ b/meta/classes/image_types.bbclass
> > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > >
> > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > ${MKUBIFS_ARGS}"
> > > >
> > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks'
> > % l
> > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > -
> > > > -def wks_search(files, search_path):
> > > > -    for f in files:
> > > > -        if os.path.isabs(f):
> > > > -            if os.path.exists(f):
> > > > -                return f
> > > > -        else:
> > > > -            searched = bb.utils.which(search_path, f)
> > > > -            if searched:
> > > > -                return searched
> > > > -
> > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > -
> > > > -IMAGE_CMD_wic () {
> > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > -       wks="${WKS_FULL_PATH}"
> > > > -       if [ -z "$wks" ]; then
> > > > -               bbfatal "No kickstart files from WKS_FILES were found:
> > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > -       fi
> > > > -
> > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > -       rm -rf "$out/"
> > > > -}
> > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > -
> > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '',
> > d)}"
> > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > -
> > > > -python () {
> > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > -}
> > > > -
> > > > -python do_write_wks_template () {
> > > > -    """Write out expanded template contents to WKS_FULL_PATH."""
> > > > -    import re
> > > > -
> > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > -
> > > > -    # Remove any remnant variable references left behind by the
> > expansion
> > > > -    # due to undefined variables
> > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > -    while True:
> > > > -        new_body = re.sub(expand_var_regexp, '', template_body)
> > > > -        if new_body == template_body:
> > > > -            break
> > > > -        else:
> > > > -            template_body = new_body
> > > > -
> > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > -    with open(wks_file, 'w') as f:
> > > > -        f.write(template_body)
> > > > -}
> > > > -
> > > > -python () {
> > > > -    if d.getVar('USING_WIC'):
> > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > -        wks_file = d.expand(wks_file_u)
> > > > -        base, ext = os.path.splitext(wks_file)
> > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > os.path.basename(base))
> > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > -            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > > > -
> > > > -            # We need to re-parse each time the file changes, and
> > bitbake
> > > > -            # needs to be told about that explicitly.
> > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > -
> > > > -            try:
> > > > -                with open(wks_file, 'r') as f:
> > > > -                    body = f.read()
> > > > -            except (IOError, OSError) as exc:
> > > > -                pass
> > > > -            else:
> > > > -                # Previously, I used expandWithRefs to get the
> > dependency
> > > > list
> > > > -                # and add it to WICVARS, but there's no point
> > re-parsing
> > > > the
> > > > -                # file in process_wks_template as well, so just put
> > it in
> > > > -                # a variable and let the metadata deal with the deps.
> > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > -                bb.build.addtask('do_write_wks_template',
> > > > 'do_image_wic', None, d)
> > > > -}
> > > > -
> > > >  EXTRA_IMAGECMD = ""
> > > >
> > > >  inherit siteinfo
> > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out from the
> > > > IMAGE_FSTYPES,
> > > >  # images that will not be built at do_rootfs time: vmdk, vdi, qcow2,
> > > > hdddirect, hddimg, iso, etc.
> > > >  IMAGE_TYPES_MASKED ?= ""
> > > > -
> > > > -# The WICVARS variable is used to define list of bitbake variables
> > used
> > > > in wic code
> > > > -# variables from this list is written to <image>.env file
> > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE
> > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
> > > > --
> > > > 2.1.4
> > > >
> > > > --
> > > > _______________________________________________
> > > > Openembedded-core mailing list
> > > > Openembedded-core@lists.openembedded.org
> > > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > > >
> >
> > --
> > --
> > Regards,
> > Ed
> >

-- 
--
Regards,
Ed

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Rick Altherr]

[-- Attachment #1: Type: text/plain, Size: 15913 bytes --]

Hmm.  In local.conf.sample.extended, I find:

# Additional image generation features
#
# The following is a list of classes to import to use in the generation of
images
# currently an example class is image_types_uboot
# IMAGE_CLASSES = " image_types_uboot"

Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.  I'd
expect a machine config to add wic to IMAGE_CLASSES if it needs wic output.

On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > Why didn't you make this image_types_wic.bbclass and use IMAGE_CLASSES to
> > load it?
>
> Because of the following:
> - IMAGE_CLASSES[doc] = "A list of classes that all images should
>   inherit." I'm not sure all images should include wic class. I'll probably
>   make this inheritance conditional.
> - so far IMAGE_CLASSES is used for qemuboot, image_types,
>   image_types_uboot and testimage,
>   so the usage is more or less follows the description. wic is out of
>   that usage scenario, I believe.
> - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
>   "image_types"\n inherit ${IMAGE_CLASSES}
>
>
> >
> > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <ed.bartosh@linux.intel.com
> >
> > wrote:
> >
> > > There is a lot of wic code in image.bbclass and image_types.bbclass
> > > Having all code separated in one file should make it more readable
> > > and easier to maintain.
> > >
> > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > ---
> > >  meta/classes/image-wic.bbclass   | 120 ++++++++++++++++++++++++++++++
> > > +++++++++
> > >  meta/classes/image.bbclass       |  25 +-------
> > >  meta/classes/image_types.bbclass |  95 ------------------------------
> -
> > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > >  create mode 100644 meta/classes/image-wic.bbclass
> > >
> > > diff --git a/meta/classes/image-wic.bbclass b/meta/classes/image-wic.
> > > bbclass
> > > new file mode 100644
> > > index 0000000..2acfd65
> > > --- /dev/null
> > > +++ b/meta/classes/image-wic.bbclass
> > > @@ -0,0 +1,120 @@
> > > +# The WICVARS variable is used to define list of bitbake variables
> used
> > > in wic code
> > > +# variables from this list is written to <image>.env file
> > > +WICVARS ?= "\
> > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> ISODIR
> > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > TARGET_SYS"
> > > +
> > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks'
> % l
> > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > '${WKS_SEARCH_PATH}') or ''}"
> > > +
> > > +def wks_search(files, search_path):
> > > +    for f in files:
> > > +        if os.path.isabs(f):
> > > +            if os.path.exists(f):
> > > +                return f
> > > +        else:
> > > +            searched = bb.utils.which(search_path, f)
> > > +            if searched:
> > > +                return searched
> > > +
> > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > +
> > > +IMAGE_CMD_wic () {
> > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > +       wks="${WKS_FULL_PATH}"
> > > +       if [ -z "$wks" ]; then
> > > +               bbfatal "No kickstart files from WKS_FILES were found:
> > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > +       fi
> > > +
> > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > > ${WIC_CREATE_EXTRA_ARGS}
> > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > +       rm -rf "$out/"
> > > +}
> > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > +
> > > +# Rebuild when the wks file or vars in WICVARS change
> > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '',
> d)}"
> > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > +do_image_wic[depends] += "wic-tools:do_build"
> > > +
> > > +python () {
> > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > +}
> > > +
> > > +python do_write_wks_template () {
> > > +    """Write out expanded template contents to WKS_FULL_PATH."""
> > > +    import re
> > > +
> > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > +
> > > +    # Remove any remnant variable references left behind by the
> expansion
> > > +    # due to undefined variables
> > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > +    while True:
> > > +        new_body = re.sub(expand_var_regexp, '', template_body)
> > > +        if new_body == template_body:
> > > +            break
> > > +        else:
> > > +            template_body = new_body
> > > +
> > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > +    with open(wks_file, 'w') as f:
> > > +        f.write(template_body)
> > > +}
> > > +
> > > +python () {
> > > +    if d.getVar('USING_WIC'):
> > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > +        wks_file = d.expand(wks_file_u)
> > > +        base, ext = os.path.splitext(wks_file)
> > > +        if ext == '.in' and os.path.exists(wks_file):
> > > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > os.path.basename(base))
> > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > +            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > > +
> > > +            # We need to re-parse each time the file changes, and
> bitbake
> > > +            # needs to be told about that explicitly.
> > > +            bb.parse.mark_dependency(d, wks_file)
> > > +
> > > +            try:
> > > +                with open(wks_file, 'r') as f:
> > > +                    body = f.read()
> > > +            except (IOError, OSError) as exc:
> > > +                pass
> > > +            else:
> > > +                # Previously, I used expandWithRefs to get the
> dependency
> > > list
> > > +                # and add it to WICVARS, but there's no point
> re-parsing
> > > the
> > > +                # file in process_wks_template as well, so just put
> it in
> > > +                # a variable and let the metadata deal with the deps.
> > > +                d.setVar('_WKS_TEMPLATE', body)
> > > +                bb.build.addtask('do_write_wks_template',
> > > 'do_image_wic', None, d)
> > > +}
> > > +
> > > +#
> > > +# Write environment variables used by wic
> > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > +#
> > > +python do_rootfs_wicenv () {
> > > +    wicvars = d.getVar('WICVARS')
> > > +    if not wicvars:
> > > +        return
> > > +
> > > +    stdir = d.getVar('STAGING_DIR')
> > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > +    bb.utils.mkdirhier(outdir)
> > > +    basename = d.getVar('IMAGE_BASENAME')
> > > +    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > > +        for var in wicvars.split():
> > > +            value = d.getVar(var)
> > > +            if value:
> > > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > > +}
> > > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> > > index 13fd996..1a549e9 100644
> > > --- a/meta/classes/image.bbclass
> > > +++ b/meta/classes/image.bbclass
> > > @@ -192,6 +192,8 @@ python () {
> > >  IMAGE_CLASSES += "image_types"
> > >  inherit ${IMAGE_CLASSES}
> > >
> > > +inherit image-wic
> > > +
> > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > >
> > >  # some default locales
> > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > >  }
> > >  addtask do_image_qa after do_image_complete before do_build
> > >
> > > -#
> > > -# Write environment variables used by wic
> > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > -#
> > > -python do_rootfs_wicenv () {
> > > -    wicvars = d.getVar('WICVARS')
> > > -    if not wicvars:
> > > -        return
> > > -
> > > -    stdir = d.getVar('STAGING_DIR')
> > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > -    bb.utils.mkdirhier(outdir)
> > > -    basename = d.getVar('IMAGE_BASENAME')
> > > -    with open(os.path.join(outdir, basename) + '.env', 'w') as envf:
> > > -        for var in wicvars.split():
> > > -            value = d.getVar(var)
> > > -            if value:
> > > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > > -}
> > > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > -
> > >  def setup_debugfs_variables(d):
> > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > diff --git a/meta/classes/image_types.bbclass
> b/meta/classes/image_types.
> > > bbclass
> > > index ad95dc6..e2ecf93 100644
> > > --- a/meta/classes/image_types.bbclass
> > > +++ b/meta/classes/image_types.bbclass
> > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > >
> > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> ${MKUBIFS_ARGS}"
> > >
> > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/canned-wks'
> % l
> > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > '${WKS_SEARCH_PATH}') or ''}"
> > > -
> > > -def wks_search(files, search_path):
> > > -    for f in files:
> > > -        if os.path.isabs(f):
> > > -            if os.path.exists(f):
> > > -                return f
> > > -        else:
> > > -            searched = bb.utils.which(search_path, f)
> > > -            if searched:
> > > -                return searched
> > > -
> > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > -
> > > -IMAGE_CMD_wic () {
> > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > -       wks="${WKS_FULL_PATH}"
> > > -       if [ -z "$wks" ]; then
> > > -               bbfatal "No kickstart files from WKS_FILES were found:
> > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > -       fi
> > > -
> > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o "$out/"
> > > ${WIC_CREATE_EXTRA_ARGS}
> > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > -       rm -rf "$out/"
> > > -}
> > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > -
> > > -# Rebuild when the wks file or vars in WICVARS change
> > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1', '',
> d)}"
> > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > -do_image_wic[depends] += "wic-tools:do_build"
> > > -
> > > -python () {
> > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > -}
> > > -
> > > -python do_write_wks_template () {
> > > -    """Write out expanded template contents to WKS_FULL_PATH."""
> > > -    import re
> > > -
> > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > -
> > > -    # Remove any remnant variable references left behind by the
> expansion
> > > -    # due to undefined variables
> > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > -    while True:
> > > -        new_body = re.sub(expand_var_regexp, '', template_body)
> > > -        if new_body == template_body:
> > > -            break
> > > -        else:
> > > -            template_body = new_body
> > > -
> > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > -    with open(wks_file, 'w') as f:
> > > -        f.write(template_body)
> > > -}
> > > -
> > > -python () {
> > > -    if d.getVar('USING_WIC'):
> > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > -        wks_file = d.expand(wks_file_u)
> > > -        base, ext = os.path.splitext(wks_file)
> > > -        if ext == '.in' and os.path.exists(wks_file):
> > > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > os.path.basename(base))
> > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > -            d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True')
> > > -
> > > -            # We need to re-parse each time the file changes, and
> bitbake
> > > -            # needs to be told about that explicitly.
> > > -            bb.parse.mark_dependency(d, wks_file)
> > > -
> > > -            try:
> > > -                with open(wks_file, 'r') as f:
> > > -                    body = f.read()
> > > -            except (IOError, OSError) as exc:
> > > -                pass
> > > -            else:
> > > -                # Previously, I used expandWithRefs to get the
> dependency
> > > list
> > > -                # and add it to WICVARS, but there's no point
> re-parsing
> > > the
> > > -                # file in process_wks_template as well, so just put
> it in
> > > -                # a variable and let the metadata deal with the deps.
> > > -                d.setVar('_WKS_TEMPLATE', body)
> > > -                bb.build.addtask('do_write_wks_template',
> > > 'do_image_wic', None, d)
> > > -}
> > > -
> > >  EXTRA_IMAGECMD = ""
> > >
> > >  inherit siteinfo
> > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > >  # The IMAGE_TYPES_MASKED variable will be used to mask out from the
> > > IMAGE_FSTYPES,
> > >  # images that will not be built at do_rootfs time: vmdk, vdi, qcow2,
> > > hdddirect, hddimg, iso, etc.
> > >  IMAGE_TYPES_MASKED ?= ""
> > > -
> > > -# The WICVARS variable is used to define list of bitbake variables
> used
> > > in wic code
> > > -# variables from this list is written to <image>.env file
> > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD HDDDIR
> > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE
> > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
> > > --
> > > 2.1.4
> > >
> > > --
> > > _______________________________________________
> > > Openembedded-core mailing list
> > > Openembedded-core@lists.openembedded.org
> > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > >
>
> --
> --
> Regards,
> Ed
>

[-- Attachment #2: Type: text/html, Size: 21967 bytes --]

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

On Mon, Jan 30, 2017 at 10:25:59AM -0800, Rick Altherr wrote:
> I'm not clear on which path is the preferred one.  There are lots of bits
> and pieces in image_types.bbclass that implement various image types.
>  uboot got added as a separate class at some point.  The comments in
> local.conf.sample.extended imply IMAGE_CLASSES should be used to load
> additional image_types_* classes to add support for additional image
> types.  Having the wic image type implemented in a separate
> image-wic.bbclass that is directly inherited by image.bbclass adds a 3rd
> approach.  Which one do we want contributors to use in the future?
> 

I didn't want to create even more confusion. What I wanted is stated in the
commit message - to put existing wic code into a file for better
maintenance. If this is more confusing than having wic code in different
places of image.class and image_types.class then we can just drop this
patch. However, I personally find it more maintainable this way.

Suggesting people to change machine configs just because wic code is
moved to separate file doesn't look good to me either.

> On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> wrote:
> 
> > On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > > Hmm.  In local.conf.sample.extended, I find:
> > >
> > > # Additional image generation features
> > > #
> > > # The following is a list of classes to import to use in the generation
> > of
> > > images
> > > # currently an example class is image_types_uboot
> > > # IMAGE_CLASSES = " image_types_uboot"
> > >
> > > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.  I'd
> > > expect a machine config to add wic to IMAGE_CLASSES if it needs wic
> > output.
> > >
> >
> > So far all machine configs add wic to IMAGE_TYPES and it works just
> > fine. Why to change?
> >
> > > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> > > wrote:
> > >
> > > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > > > Why didn't you make this image_types_wic.bbclass and use
> > IMAGE_CLASSES to
> > > > > load it?
> > > >
> > > > Because of the following:
> > > > - IMAGE_CLASSES[doc] = "A list of classes that all images should
> > > >   inherit." I'm not sure all images should include wic class. I'll
> > probably
> > > >   make this inheritance conditional.
> > > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > > >   image_types_uboot and testimage,
> > > >   so the usage is more or less follows the description. wic is out of
> > > >   that usage scenario, I believe.
> > > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> > > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > > >
> > > >
> > > > >
> > > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> > ed.bartosh@linux.intel.com
> > > > >
> > > > > wrote:
> > > > >
> > > > > > There is a lot of wic code in image.bbclass and image_types.bbclass
> > > > > > Having all code separated in one file should make it more readable
> > > > > > and easier to maintain.
> > > > > >
> > > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > > ---
> > > > > >  meta/classes/image-wic.bbclass   | 120
> > ++++++++++++++++++++++++++++++
> > > > > > +++++++++
> > > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > > >  meta/classes/image_types.bbclass |  95
> > ------------------------------
> > > > -
> > > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > > >
> > > > > > diff --git a/meta/classes/image-wic.bbclass
> > b/meta/classes/image-wic.
> > > > > > bbclass
> > > > > > new file mode 100644
> > > > > > index 0000000..2acfd65
> > > > > > --- /dev/null
> > > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > > @@ -0,0 +1,120 @@
> > > > > > +# The WICVARS variable is used to define list of bitbake variables
> > > > used
> > > > > > in wic code
> > > > > > +# variables from this list is written to <image>.env file
> > > > > > +WICVARS ?= "\
> > > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > HDDDIR
> > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> > > > ISODIR
> > > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > > > > TARGET_SYS"
> > > > > > +
> > > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > canned-wks'
> > > > % l
> > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > +
> > > > > > +def wks_search(files, search_path):
> > > > > > +    for f in files:
> > > > > > +        if os.path.isabs(f):
> > > > > > +            if os.path.exists(f):
> > > > > > +                return f
> > > > > > +        else:
> > > > > > +            searched = bb.utils.which(search_path, f)
> > > > > > +            if searched:
> > > > > > +                return searched
> > > > > > +
> > > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > +
> > > > > > +IMAGE_CMD_wic () {
> > > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > +       wks="${WKS_FULL_PATH}"
> > > > > > +       if [ -z "$wks" ]; then
> > > > > > +               bbfatal "No kickstart files from WKS_FILES were
> > found:
> > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > > +       fi
> > > > > > +
> > > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > "$out/"
> > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > +       rm -rf "$out/"
> > > > > > +}
> > > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > +
> > > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1',
> > '',
> > > > d)}"
> > > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > > +
> > > > > > +python () {
> > > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > > +}
> > > > > > +
> > > > > > +python do_write_wks_template () {
> > > > > > +    """Write out expanded template contents to WKS_FULL_PATH."""
> > > > > > +    import re
> > > > > > +
> > > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > +
> > > > > > +    # Remove any remnant variable references left behind by the
> > > > expansion
> > > > > > +    # due to undefined variables
> > > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > +    while True:
> > > > > > +        new_body = re.sub(expand_var_regexp, '', template_body)
> > > > > > +        if new_body == template_body:
> > > > > > +            break
> > > > > > +        else:
> > > > > > +            template_body = new_body
> > > > > > +
> > > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > +    with open(wks_file, 'w') as f:
> > > > > > +        f.write(template_body)
> > > > > > +}
> > > > > > +
> > > > > > +python () {
> > > > > > +    if d.getVar('USING_WIC'):
> > > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > +        wks_file = d.expand(wks_file_u)
> > > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > > os.path.basename(base))
> > > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> > '${WKS_TEMPLATE_PATH}:True')
> > > > > > +
> > > > > > +            # We need to re-parse each time the file changes, and
> > > > bitbake
> > > > > > +            # needs to be told about that explicitly.
> > > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > > +
> > > > > > +            try:
> > > > > > +                with open(wks_file, 'r') as f:
> > > > > > +                    body = f.read()
> > > > > > +            except (IOError, OSError) as exc:
> > > > > > +                pass
> > > > > > +            else:
> > > > > > +                # Previously, I used expandWithRefs to get the
> > > > dependency
> > > > > > list
> > > > > > +                # and add it to WICVARS, but there's no point
> > > > re-parsing
> > > > > > the
> > > > > > +                # file in process_wks_template as well, so just
> > put
> > > > it in
> > > > > > +                # a variable and let the metadata deal with the
> > deps.
> > > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > > +                bb.build.addtask('do_write_wks_template',
> > > > > > 'do_image_wic', None, d)
> > > > > > +}
> > > > > > +
> > > > > > +#
> > > > > > +# Write environment variables used by wic
> > > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > +#
> > > > > > +python do_rootfs_wicenv () {
> > > > > > +    wicvars = d.getVar('WICVARS')
> > > > > > +    if not wicvars:
> > > > > > +        return
> > > > > > +
> > > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > > > > +    bb.utils.mkdirhier(outdir)
> > > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > > +    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > envf:
> > > > > > +        for var in wicvars.split():
> > > > > > +            value = d.getVar(var)
> > > > > > +            if value:
> > > > > > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > > +}
> > > > > > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > diff --git a/meta/classes/image.bbclass
> > b/meta/classes/image.bbclass
> > > > > > index 13fd996..1a549e9 100644
> > > > > > --- a/meta/classes/image.bbclass
> > > > > > +++ b/meta/classes/image.bbclass
> > > > > > @@ -192,6 +192,8 @@ python () {
> > > > > >  IMAGE_CLASSES += "image_types"
> > > > > >  inherit ${IMAGE_CLASSES}
> > > > > >
> > > > > > +inherit image-wic
> > > > > > +
> > > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > > >
> > > > > >  # some default locales
> > > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > > >  }
> > > > > >  addtask do_image_qa after do_image_complete before do_build
> > > > > >
> > > > > > -#
> > > > > > -# Write environment variables used by wic
> > > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > -#
> > > > > > -python do_rootfs_wicenv () {
> > > > > > -    wicvars = d.getVar('WICVARS')
> > > > > > -    if not wicvars:
> > > > > > -        return
> > > > > > -
> > > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > > > > -    bb.utils.mkdirhier(outdir)
> > > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > > -    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > envf:
> > > > > > -        for var in wicvars.split():
> > > > > > -            value = d.getVar(var)
> > > > > > -            if value:
> > > > > > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > > -}
> > > > > > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > -
> > > > > >  def setup_debugfs_variables(d):
> > > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > > diff --git a/meta/classes/image_types.bbclass
> > > > b/meta/classes/image_types.
> > > > > > bbclass
> > > > > > index ad95dc6..e2ecf93 100644
> > > > > > --- a/meta/classes/image_types.bbclass
> > > > > > +++ b/meta/classes/image_types.bbclass
> > > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > > >
> > > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > > > ${MKUBIFS_ARGS}"
> > > > > >
> > > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > canned-wks'
> > > > % l
> > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > -
> > > > > > -def wks_search(files, search_path):
> > > > > > -    for f in files:
> > > > > > -        if os.path.isabs(f):
> > > > > > -            if os.path.exists(f):
> > > > > > -                return f
> > > > > > -        else:
> > > > > > -            searched = bb.utils.which(search_path, f)
> > > > > > -            if searched:
> > > > > > -                return searched
> > > > > > -
> > > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > -
> > > > > > -IMAGE_CMD_wic () {
> > > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > -       wks="${WKS_FULL_PATH}"
> > > > > > -       if [ -z "$wks" ]; then
> > > > > > -               bbfatal "No kickstart files from WKS_FILES were
> > found:
> > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > > -       fi
> > > > > > -
> > > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > "$out/"
> > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > -       rm -rf "$out/"
> > > > > > -}
> > > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > -
> > > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1',
> > '',
> > > > d)}"
> > > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > > -
> > > > > > -python () {
> > > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > > -}
> > > > > > -
> > > > > > -python do_write_wks_template () {
> > > > > > -    """Write out expanded template contents to WKS_FULL_PATH."""
> > > > > > -    import re
> > > > > > -
> > > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > -
> > > > > > -    # Remove any remnant variable references left behind by the
> > > > expansion
> > > > > > -    # due to undefined variables
> > > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > -    while True:
> > > > > > -        new_body = re.sub(expand_var_regexp, '', template_body)
> > > > > > -        if new_body == template_body:
> > > > > > -            break
> > > > > > -        else:
> > > > > > -            template_body = new_body
> > > > > > -
> > > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > -    with open(wks_file, 'w') as f:
> > > > > > -        f.write(template_body)
> > > > > > -}
> > > > > > -
> > > > > > -python () {
> > > > > > -    if d.getVar('USING_WIC'):
> > > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > -        wks_file = d.expand(wks_file_u)
> > > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > > os.path.basename(base))
> > > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> > '${WKS_TEMPLATE_PATH}:True')
> > > > > > -
> > > > > > -            # We need to re-parse each time the file changes, and
> > > > bitbake
> > > > > > -            # needs to be told about that explicitly.
> > > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > > -
> > > > > > -            try:
> > > > > > -                with open(wks_file, 'r') as f:
> > > > > > -                    body = f.read()
> > > > > > -            except (IOError, OSError) as exc:
> > > > > > -                pass
> > > > > > -            else:
> > > > > > -                # Previously, I used expandWithRefs to get the
> > > > dependency
> > > > > > list
> > > > > > -                # and add it to WICVARS, but there's no point
> > > > re-parsing
> > > > > > the
> > > > > > -                # file in process_wks_template as well, so just
> > put
> > > > it in
> > > > > > -                # a variable and let the metadata deal with the
> > deps.
> > > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > > -                bb.build.addtask('do_write_wks_template',
> > > > > > 'do_image_wic', None, d)
> > > > > > -}
> > > > > > -
> > > > > >  EXTRA_IMAGECMD = ""
> > > > > >
> > > > > >  inherit siteinfo
> > > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out from
> > the
> > > > > > IMAGE_FSTYPES,
> > > > > >  # images that will not be built at do_rootfs time: vmdk, vdi,
> > qcow2,
> > > > > > hdddirect, hddimg, iso, etc.
> > > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > > -
> > > > > > -# The WICVARS variable is used to define list of bitbake variables
> > > > used
> > > > > > in wic code
> > > > > > -# variables from this list is written to <image>.env file
> > > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > HDDDIR
> > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE
> > > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
> > > > > > --
> > > > > > 2.1.4
> > > > > >
> > > > > > --
> > > > > > _______________________________________________
> > > > > > Openembedded-core mailing list
> > > > > > Openembedded-core@lists.openembedded.org
> > > > > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > > > > >
> > > >
> > > > --
> > > > --
> > > > Regards,
> > > > Ed
> > > >
> >
> > --
> > --
> > Regards,
> > Ed
> >

-- 
--
Regards,
Ed

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Rick Altherr]

[-- Attachment #1: Type: text/plain, Size: 18455 bytes --]

I'm not clear on which path is the preferred one.  There are lots of bits
and pieces in image_types.bbclass that implement various image types.
 uboot got added as a separate class at some point.  The comments in
local.conf.sample.extended imply IMAGE_CLASSES should be used to load
additional image_types_* classes to add support for additional image
types.  Having the wic image type implemented in a separate
image-wic.bbclass that is directly inherited by image.bbclass adds a 3rd
approach.  Which one do we want contributors to use in the future?

On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > Hmm.  In local.conf.sample.extended, I find:
> >
> > # Additional image generation features
> > #
> > # The following is a list of classes to import to use in the generation
> of
> > images
> > # currently an example class is image_types_uboot
> > # IMAGE_CLASSES = " image_types_uboot"
> >
> > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.  I'd
> > expect a machine config to add wic to IMAGE_CLASSES if it needs wic
> output.
> >
>
> So far all machine configs add wic to IMAGE_TYPES and it works just
> fine. Why to change?
>
> > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> > wrote:
> >
> > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > > Why didn't you make this image_types_wic.bbclass and use
> IMAGE_CLASSES to
> > > > load it?
> > >
> > > Because of the following:
> > > - IMAGE_CLASSES[doc] = "A list of classes that all images should
> > >   inherit." I'm not sure all images should include wic class. I'll
> probably
> > >   make this inheritance conditional.
> > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > >   image_types_uboot and testimage,
> > >   so the usage is more or less follows the description. wic is out of
> > >   that usage scenario, I believe.
> > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > >
> > >
> > > >
> > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> ed.bartosh@linux.intel.com
> > > >
> > > > wrote:
> > > >
> > > > > There is a lot of wic code in image.bbclass and image_types.bbclass
> > > > > Having all code separated in one file should make it more readable
> > > > > and easier to maintain.
> > > > >
> > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > ---
> > > > >  meta/classes/image-wic.bbclass   | 120
> ++++++++++++++++++++++++++++++
> > > > > +++++++++
> > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > >  meta/classes/image_types.bbclass |  95
> ------------------------------
> > > -
> > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > >
> > > > > diff --git a/meta/classes/image-wic.bbclass
> b/meta/classes/image-wic.
> > > > > bbclass
> > > > > new file mode 100644
> > > > > index 0000000..2acfd65
> > > > > --- /dev/null
> > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > @@ -0,0 +1,120 @@
> > > > > +# The WICVARS variable is used to define list of bitbake variables
> > > used
> > > > > in wic code
> > > > > +# variables from this list is written to <image>.env file
> > > > > +WICVARS ?= "\
> > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> HDDDIR
> > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD
> > > ISODIR
> > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > > > TARGET_SYS"
> > > > > +
> > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> canned-wks'
> > > % l
> > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > +
> > > > > +def wks_search(files, search_path):
> > > > > +    for f in files:
> > > > > +        if os.path.isabs(f):
> > > > > +            if os.path.exists(f):
> > > > > +                return f
> > > > > +        else:
> > > > > +            searched = bb.utils.which(search_path, f)
> > > > > +            if searched:
> > > > > +                return searched
> > > > > +
> > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > +
> > > > > +IMAGE_CMD_wic () {
> > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > +       wks="${WKS_FULL_PATH}"
> > > > > +       if [ -z "$wks" ]; then
> > > > > +               bbfatal "No kickstart files from WKS_FILES were
> found:
> > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > +       fi
> > > > > +
> > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> "$out/"
> > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > +       rm -rf "$out/"
> > > > > +}
> > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > +
> > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1',
> '',
> > > d)}"
> > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > +
> > > > > +python () {
> > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > +}
> > > > > +
> > > > > +python do_write_wks_template () {
> > > > > +    """Write out expanded template contents to WKS_FULL_PATH."""
> > > > > +    import re
> > > > > +
> > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > +
> > > > > +    # Remove any remnant variable references left behind by the
> > > expansion
> > > > > +    # due to undefined variables
> > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > +    while True:
> > > > > +        new_body = re.sub(expand_var_regexp, '', template_body)
> > > > > +        if new_body == template_body:
> > > > > +            break
> > > > > +        else:
> > > > > +            template_body = new_body
> > > > > +
> > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > +    with open(wks_file, 'w') as f:
> > > > > +        f.write(template_body)
> > > > > +}
> > > > > +
> > > > > +python () {
> > > > > +    if d.getVar('USING_WIC'):
> > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > +        wks_file = d.expand(wks_file_u)
> > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > os.path.basename(base))
> > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> '${WKS_TEMPLATE_PATH}:True')
> > > > > +
> > > > > +            # We need to re-parse each time the file changes, and
> > > bitbake
> > > > > +            # needs to be told about that explicitly.
> > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > +
> > > > > +            try:
> > > > > +                with open(wks_file, 'r') as f:
> > > > > +                    body = f.read()
> > > > > +            except (IOError, OSError) as exc:
> > > > > +                pass
> > > > > +            else:
> > > > > +                # Previously, I used expandWithRefs to get the
> > > dependency
> > > > > list
> > > > > +                # and add it to WICVARS, but there's no point
> > > re-parsing
> > > > > the
> > > > > +                # file in process_wks_template as well, so just
> put
> > > it in
> > > > > +                # a variable and let the metadata deal with the
> deps.
> > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > +                bb.build.addtask('do_write_wks_template',
> > > > > 'do_image_wic', None, d)
> > > > > +}
> > > > > +
> > > > > +#
> > > > > +# Write environment variables used by wic
> > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > +#
> > > > > +python do_rootfs_wicenv () {
> > > > > +    wicvars = d.getVar('WICVARS')
> > > > > +    if not wicvars:
> > > > > +        return
> > > > > +
> > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > > > +    bb.utils.mkdirhier(outdir)
> > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > +    with open(os.path.join(outdir, basename) + '.env', 'w') as
> envf:
> > > > > +        for var in wicvars.split():
> > > > > +            value = d.getVar(var)
> > > > > +            if value:
> > > > > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > +}
> > > > > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > diff --git a/meta/classes/image.bbclass
> b/meta/classes/image.bbclass
> > > > > index 13fd996..1a549e9 100644
> > > > > --- a/meta/classes/image.bbclass
> > > > > +++ b/meta/classes/image.bbclass
> > > > > @@ -192,6 +192,8 @@ python () {
> > > > >  IMAGE_CLASSES += "image_types"
> > > > >  inherit ${IMAGE_CLASSES}
> > > > >
> > > > > +inherit image-wic
> > > > > +
> > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > >
> > > > >  # some default locales
> > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > >  }
> > > > >  addtask do_image_qa after do_image_complete before do_build
> > > > >
> > > > > -#
> > > > > -# Write environment variables used by wic
> > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > -#
> > > > > -python do_rootfs_wicenv () {
> > > > > -    wicvars = d.getVar('WICVARS')
> > > > > -    if not wicvars:
> > > > > -        return
> > > > > -
> > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'), 'imgdata')
> > > > > -    bb.utils.mkdirhier(outdir)
> > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > -    with open(os.path.join(outdir, basename) + '.env', 'w') as
> envf:
> > > > > -        for var in wicvars.split():
> > > > > -            value = d.getVar(var)
> > > > > -            if value:
> > > > > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > -}
> > > > > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > -
> > > > >  def setup_debugfs_variables(d):
> > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > diff --git a/meta/classes/image_types.bbclass
> > > b/meta/classes/image_types.
> > > > > bbclass
> > > > > index ad95dc6..e2ecf93 100644
> > > > > --- a/meta/classes/image_types.bbclass
> > > > > +++ b/meta/classes/image_types.bbclass
> > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > >
> > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > > ${MKUBIFS_ARGS}"
> > > > >
> > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for p in
> > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> canned-wks'
> > > % l
> > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > -
> > > > > -def wks_search(files, search_path):
> > > > > -    for f in files:
> > > > > -        if os.path.isabs(f):
> > > > > -            if os.path.exists(f):
> > > > > -                return f
> > > > > -        else:
> > > > > -            searched = bb.utils.which(search_path, f)
> > > > > -            if searched:
> > > > > -                return searched
> > > > > -
> > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > -
> > > > > -IMAGE_CMD_wic () {
> > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > -       wks="${WKS_FULL_PATH}"
> > > > > -       if [ -z "$wks" ]; then
> > > > > -               bbfatal "No kickstart files from WKS_FILES were
> found:
> > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > -       fi
> > > > > -
> > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> "$out/"
> > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > -       rm -rf "$out/"
> > > > > -}
> > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > -
> > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic ' + '
> > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()), '1',
> '',
> > > d)}"
> > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > -
> > > > > -python () {
> > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > -}
> > > > > -
> > > > > -python do_write_wks_template () {
> > > > > -    """Write out expanded template contents to WKS_FULL_PATH."""
> > > > > -    import re
> > > > > -
> > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > -
> > > > > -    # Remove any remnant variable references left behind by the
> > > expansion
> > > > > -    # due to undefined variables
> > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > -    while True:
> > > > > -        new_body = re.sub(expand_var_regexp, '', template_body)
> > > > > -        if new_body == template_body:
> > > > > -            break
> > > > > -        else:
> > > > > -            template_body = new_body
> > > > > -
> > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > -    with open(wks_file, 'w') as f:
> > > > > -        f.write(template_body)
> > > > > -}
> > > > > -
> > > > > -python () {
> > > > > -    if d.getVar('USING_WIC'):
> > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > -        wks_file = d.expand(wks_file_u)
> > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > os.path.basename(base))
> > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> '${WKS_TEMPLATE_PATH}:True')
> > > > > -
> > > > > -            # We need to re-parse each time the file changes, and
> > > bitbake
> > > > > -            # needs to be told about that explicitly.
> > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > -
> > > > > -            try:
> > > > > -                with open(wks_file, 'r') as f:
> > > > > -                    body = f.read()
> > > > > -            except (IOError, OSError) as exc:
> > > > > -                pass
> > > > > -            else:
> > > > > -                # Previously, I used expandWithRefs to get the
> > > dependency
> > > > > list
> > > > > -                # and add it to WICVARS, but there's no point
> > > re-parsing
> > > > > the
> > > > > -                # file in process_wks_template as well, so just
> put
> > > it in
> > > > > -                # a variable and let the metadata deal with the
> deps.
> > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > -                bb.build.addtask('do_write_wks_template',
> > > > > 'do_image_wic', None, d)
> > > > > -}
> > > > > -
> > > > >  EXTRA_IMAGECMD = ""
> > > > >
> > > > >  inherit siteinfo
> > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out from
> the
> > > > > IMAGE_FSTYPES,
> > > > >  # images that will not be built at do_rootfs time: vmdk, vdi,
> qcow2,
> > > > > hdddirect, hddimg, iso, etc.
> > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > -
> > > > > -# The WICVARS variable is used to define list of bitbake variables
> > > used
> > > > > in wic code
> > > > > -# variables from this list is written to <image>.env file
> > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> HDDDIR
> > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH RECIPE_SYSROOT_NATIVE
> > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR TARGET_SYS"
> > > > > --
> > > > > 2.1.4
> > > > >
> > > > > --
> > > > > _______________________________________________
> > > > > Openembedded-core mailing list
> > > > > Openembedded-core@lists.openembedded.org
> > > > > http://lists.openembedded.org/mailman/listinfo/openembedded-core
> > > > >
> > >
> > > --
> > > --
> > > Regards,
> > > Ed
> > >
>
> --
> --
> Regards,
> Ed
>

[-- Attachment #2: Type: text/html, Size: 26912 bytes --]

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Rick Altherr]

[-- Attachment #1: Type: text/plain, Size: 21179 bytes --]

Agreed.  What if it was image_types_wic.bbclass and you did something
similar to build_uboot() in image.bbclass?

On Mon, Jan 30, 2017 at 10:15 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> On Mon, Jan 30, 2017 at 10:25:59AM -0800, Rick Altherr wrote:
> > I'm not clear on which path is the preferred one.  There are lots of bits
> > and pieces in image_types.bbclass that implement various image types.
> >  uboot got added as a separate class at some point.  The comments in
> > local.conf.sample.extended imply IMAGE_CLASSES should be used to load
> > additional image_types_* classes to add support for additional image
> > types.  Having the wic image type implemented in a separate
> > image-wic.bbclass that is directly inherited by image.bbclass adds a 3rd
> > approach.  Which one do we want contributors to use in the future?
> >
>
> I didn't want to create even more confusion. What I wanted is stated in the
> commit message - to put existing wic code into a file for better
> maintenance. If this is more confusing than having wic code in different
> places of image.class and image_types.class then we can just drop this
> patch. However, I personally find it more maintainable this way.
>
> Suggesting people to change machine configs just because wic code is
> moved to separate file doesn't look good to me either.
>
> > On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> > wrote:
> >
> > > On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > > > Hmm.  In local.conf.sample.extended, I find:
> > > >
> > > > # Additional image generation features
> > > > #
> > > > # The following is a list of classes to import to use in the
> generation
> > > of
> > > > images
> > > > # currently an example class is image_types_uboot
> > > > # IMAGE_CLASSES = " image_types_uboot"
> > > >
> > > > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.
> I'd
> > > > expect a machine config to add wic to IMAGE_CLASSES if it needs wic
> > > output.
> > > >
> > >
> > > So far all machine configs add wic to IMAGE_TYPES and it works just
> > > fine. Why to change?
> > >
> > > > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <
> ed.bartosh@linux.intel.com>
> > > > wrote:
> > > >
> > > > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > > > > Why didn't you make this image_types_wic.bbclass and use
> > > IMAGE_CLASSES to
> > > > > > load it?
> > > > >
> > > > > Because of the following:
> > > > > - IMAGE_CLASSES[doc] = "A list of classes that all images should
> > > > >   inherit." I'm not sure all images should include wic class. I'll
> > > probably
> > > > >   make this inheritance conditional.
> > > > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > > > >   image_types_uboot and testimage,
> > > > >   so the usage is more or less follows the description. wic is out
> of
> > > > >   that usage scenario, I believe.
> > > > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> > > > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > > > >
> > > > >
> > > > > >
> > > > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> > > ed.bartosh@linux.intel.com
> > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > There is a lot of wic code in image.bbclass and
> image_types.bbclass
> > > > > > > Having all code separated in one file should make it more
> readable
> > > > > > > and easier to maintain.
> > > > > > >
> > > > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > > > ---
> > > > > > >  meta/classes/image-wic.bbclass   | 120
> > > ++++++++++++++++++++++++++++++
> > > > > > > +++++++++
> > > > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > > > >  meta/classes/image_types.bbclass |  95
> > > ------------------------------
> > > > > -
> > > > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > > > >
> > > > > > > diff --git a/meta/classes/image-wic.bbclass
> > > b/meta/classes/image-wic.
> > > > > > > bbclass
> > > > > > > new file mode 100644
> > > > > > > index 0000000..2acfd65
> > > > > > > --- /dev/null
> > > > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > > > @@ -0,0 +1,120 @@
> > > > > > > +# The WICVARS variable is used to define list of bitbake
> variables
> > > > > used
> > > > > > > in wic code
> > > > > > > +# variables from this list is written to <image>.env file
> > > > > > > +WICVARS ?= "\
> > > > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > > HDDDIR
> > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES
> INITRD
> > > > > ISODIR
> > > > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> STAGING_LIBDIR
> > > > > > > TARGET_SYS"
> > > > > > > +
> > > > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for
> p in
> > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > canned-wks'
> > > > > % l
> > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > +
> > > > > > > +def wks_search(files, search_path):
> > > > > > > +    for f in files:
> > > > > > > +        if os.path.isabs(f):
> > > > > > > +            if os.path.exists(f):
> > > > > > > +                return f
> > > > > > > +        else:
> > > > > > > +            searched = bb.utils.which(search_path, f)
> > > > > > > +            if searched:
> > > > > > > +                return searched
> > > > > > > +
> > > > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > +
> > > > > > > +IMAGE_CMD_wic () {
> > > > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > +       wks="${WKS_FULL_PATH}"
> > > > > > > +       if [ -z "$wks" ]; then
> > > > > > > +               bbfatal "No kickstart files from WKS_FILES were
> > > found:
> > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > > > +       fi
> > > > > > > +
> > > > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > > "$out/"
> > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > +       rm -rf "$out/"
> > > > > > > +}
> > > > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > +
> > > > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic
> ' + '
> > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> '1',
> > > '',
> > > > > d)}"
> > > > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > +
> > > > > > > +python () {
> > > > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > > > +}
> > > > > > > +
> > > > > > > +python do_write_wks_template () {
> > > > > > > +    """Write out expanded template contents to
> WKS_FULL_PATH."""
> > > > > > > +    import re
> > > > > > > +
> > > > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > +
> > > > > > > +    # Remove any remnant variable references left behind by
> the
> > > > > expansion
> > > > > > > +    # due to undefined variables
> > > > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > +    while True:
> > > > > > > +        new_body = re.sub(expand_var_regexp, '',
> template_body)
> > > > > > > +        if new_body == template_body:
> > > > > > > +            break
> > > > > > > +        else:
> > > > > > > +            template_body = new_body
> > > > > > > +
> > > > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > +    with open(wks_file, 'w') as f:
> > > > > > > +        f.write(template_body)
> > > > > > > +}
> > > > > > > +
> > > > > > > +python () {
> > > > > > > +    if d.getVar('USING_WIC'):
> > > > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > +        wks_file = d.expand(wks_file_u)
> > > > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > > > os.path.basename(base))
> > > > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > +
> > > > > > > +            # We need to re-parse each time the file changes,
> and
> > > > > bitbake
> > > > > > > +            # needs to be told about that explicitly.
> > > > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > > > +
> > > > > > > +            try:
> > > > > > > +                with open(wks_file, 'r') as f:
> > > > > > > +                    body = f.read()
> > > > > > > +            except (IOError, OSError) as exc:
> > > > > > > +                pass
> > > > > > > +            else:
> > > > > > > +                # Previously, I used expandWithRefs to get the
> > > > > dependency
> > > > > > > list
> > > > > > > +                # and add it to WICVARS, but there's no point
> > > > > re-parsing
> > > > > > > the
> > > > > > > +                # file in process_wks_template as well, so
> just
> > > put
> > > > > it in
> > > > > > > +                # a variable and let the metadata deal with
> the
> > > deps.
> > > > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > +                bb.build.addtask('do_write_wks_template',
> > > > > > > 'do_image_wic', None, d)
> > > > > > > +}
> > > > > > > +
> > > > > > > +#
> > > > > > > +# Write environment variables used by wic
> > > > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > +#
> > > > > > > +python do_rootfs_wicenv () {
> > > > > > > +    wicvars = d.getVar('WICVARS')
> > > > > > > +    if not wicvars:
> > > > > > > +        return
> > > > > > > +
> > > > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> 'imgdata')
> > > > > > > +    bb.utils.mkdirhier(outdir)
> > > > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > +    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > > envf:
> > > > > > > +        for var in wicvars.split():
> > > > > > > +            value = d.getVar(var)
> > > > > > > +            if value:
> > > > > > > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > > > +}
> > > > > > > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > diff --git a/meta/classes/image.bbclass
> > > b/meta/classes/image.bbclass
> > > > > > > index 13fd996..1a549e9 100644
> > > > > > > --- a/meta/classes/image.bbclass
> > > > > > > +++ b/meta/classes/image.bbclass
> > > > > > > @@ -192,6 +192,8 @@ python () {
> > > > > > >  IMAGE_CLASSES += "image_types"
> > > > > > >  inherit ${IMAGE_CLASSES}
> > > > > > >
> > > > > > > +inherit image-wic
> > > > > > > +
> > > > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > > > >
> > > > > > >  # some default locales
> > > > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > > > >  }
> > > > > > >  addtask do_image_qa after do_image_complete before do_build
> > > > > > >
> > > > > > > -#
> > > > > > > -# Write environment variables used by wic
> > > > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > -#
> > > > > > > -python do_rootfs_wicenv () {
> > > > > > > -    wicvars = d.getVar('WICVARS')
> > > > > > > -    if not wicvars:
> > > > > > > -        return
> > > > > > > -
> > > > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> 'imgdata')
> > > > > > > -    bb.utils.mkdirhier(outdir)
> > > > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > -    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > > envf:
> > > > > > > -        for var in wicvars.split():
> > > > > > > -            value = d.getVar(var)
> > > > > > > -            if value:
> > > > > > > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > > > -}
> > > > > > > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > -
> > > > > > >  def setup_debugfs_variables(d):
> > > > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > > > diff --git a/meta/classes/image_types.bbclass
> > > > > b/meta/classes/image_types.
> > > > > > > bbclass
> > > > > > > index ad95dc6..e2ecf93 100644
> > > > > > > --- a/meta/classes/image_types.bbclass
> > > > > > > +++ b/meta/classes/image_types.bbclass
> > > > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > > > >
> > > > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > > > > ${MKUBIFS_ARGS}"
> > > > > > >
> > > > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for
> p in
> > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > canned-wks'
> > > > > % l
> > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > -
> > > > > > > -def wks_search(files, search_path):
> > > > > > > -    for f in files:
> > > > > > > -        if os.path.isabs(f):
> > > > > > > -            if os.path.exists(f):
> > > > > > > -                return f
> > > > > > > -        else:
> > > > > > > -            searched = bb.utils.which(search_path, f)
> > > > > > > -            if searched:
> > > > > > > -                return searched
> > > > > > > -
> > > > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > -
> > > > > > > -IMAGE_CMD_wic () {
> > > > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > -       wks="${WKS_FULL_PATH}"
> > > > > > > -       if [ -z "$wks" ]; then
> > > > > > > -               bbfatal "No kickstart files from WKS_FILES were
> > > found:
> > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > > > -       fi
> > > > > > > -
> > > > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > > "$out/"
> > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > -       rm -rf "$out/"
> > > > > > > -}
> > > > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > -
> > > > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic
> ' + '
> > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> '1',
> > > '',
> > > > > d)}"
> > > > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > -
> > > > > > > -python () {
> > > > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > > > -}
> > > > > > > -
> > > > > > > -python do_write_wks_template () {
> > > > > > > -    """Write out expanded template contents to
> WKS_FULL_PATH."""
> > > > > > > -    import re
> > > > > > > -
> > > > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > -
> > > > > > > -    # Remove any remnant variable references left behind by
> the
> > > > > expansion
> > > > > > > -    # due to undefined variables
> > > > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > -    while True:
> > > > > > > -        new_body = re.sub(expand_var_regexp, '',
> template_body)
> > > > > > > -        if new_body == template_body:
> > > > > > > -            break
> > > > > > > -        else:
> > > > > > > -            template_body = new_body
> > > > > > > -
> > > > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > -    with open(wks_file, 'w') as f:
> > > > > > > -        f.write(template_body)
> > > > > > > -}
> > > > > > > -
> > > > > > > -python () {
> > > > > > > -    if d.getVar('USING_WIC'):
> > > > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > -        wks_file = d.expand(wks_file_u)
> > > > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > > > os.path.basename(base))
> > > > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > -
> > > > > > > -            # We need to re-parse each time the file changes,
> and
> > > > > bitbake
> > > > > > > -            # needs to be told about that explicitly.
> > > > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > > > -
> > > > > > > -            try:
> > > > > > > -                with open(wks_file, 'r') as f:
> > > > > > > -                    body = f.read()
> > > > > > > -            except (IOError, OSError) as exc:
> > > > > > > -                pass
> > > > > > > -            else:
> > > > > > > -                # Previously, I used expandWithRefs to get the
> > > > > dependency
> > > > > > > list
> > > > > > > -                # and add it to WICVARS, but there's no point
> > > > > re-parsing
> > > > > > > the
> > > > > > > -                # file in process_wks_template as well, so
> just
> > > put
> > > > > it in
> > > > > > > -                # a variable and let the metadata deal with
> the
> > > deps.
> > > > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > -                bb.build.addtask('do_write_wks_template',
> > > > > > > 'do_image_wic', None, d)
> > > > > > > -}
> > > > > > > -
> > > > > > >  EXTRA_IMAGECMD = ""
> > > > > > >
> > > > > > >  inherit siteinfo
> > > > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out
> from
> > > the
> > > > > > > IMAGE_FSTYPES,
> > > > > > >  # images that will not be built at do_rootfs time: vmdk, vdi,
> > > qcow2,
> > > > > > > hdddirect, hddimg, iso, etc.
> > > > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > > > -
> > > > > > > -# The WICVARS variable is used to define list of bitbake
> variables
> > > > > used
> > > > > > > in wic code
> > > > > > > -# variables from this list is written to <image>.env file
> > > > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > > HDDDIR
> > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > > > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> RECIPE_SYSROOT_NATIVE
> > > > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> TARGET_SYS"
> > > > > > > --
> > > > > > > 2.1.4
> > > > > > >
> > > > > > > --
> > > > > > > _______________________________________________
> > > > > > > Openembedded-core mailing list
> > > > > > > Openembedded-core@lists.openembedded.org
> > > > > > > http://lists.openembedded.org/mailman/listinfo/openembedded-
> core
> > > > > > >
> > > > >
> > > > > --
> > > > > --
> > > > > Regards,
> > > > > Ed
> > > > >
> > >
> > > --
> > > --
> > > Regards,
> > > Ed
> > >
>
> --
> --
> Regards,
> Ed
>

[-- Attachment #2: Type: text/html, Size: 32487 bytes --]

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

On Mon, Jan 30, 2017 at 10:41:27AM -0800, Rick Altherr wrote:
> Agreed.  What if it was image_types_wic.bbclass and you did something
> similar to build_uboot() in image.bbclass?
>

I can do this for now:
IMAGE_TYPE_wic = "image_type_wic"
inherit ${IMAGE_TYPE_wic}

which is the same as 'inherit image_type_wic', just takes 2 lines, but
looks more consistent.

does this look ok?

> On Mon, Jan 30, 2017 at 10:15 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> wrote:
> 
> > On Mon, Jan 30, 2017 at 10:25:59AM -0800, Rick Altherr wrote:
> > > I'm not clear on which path is the preferred one.  There are lots of bits
> > > and pieces in image_types.bbclass that implement various image types.
> > >  uboot got added as a separate class at some point.  The comments in
> > > local.conf.sample.extended imply IMAGE_CLASSES should be used to load
> > > additional image_types_* classes to add support for additional image
> > > types.  Having the wic image type implemented in a separate
> > > image-wic.bbclass that is directly inherited by image.bbclass adds a 3rd
> > > approach.  Which one do we want contributors to use in the future?
> > >
> >
> > I didn't want to create even more confusion. What I wanted is stated in the
> > commit message - to put existing wic code into a file for better
> > maintenance. If this is more confusing than having wic code in different
> > places of image.class and image_types.class then we can just drop this
> > patch. However, I personally find it more maintainable this way.
> >
> > Suggesting people to change machine configs just because wic code is
> > moved to separate file doesn't look good to me either.
> >
> > > On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> > > wrote:
> > >
> > > > On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > > > > Hmm.  In local.conf.sample.extended, I find:
> > > > >
> > > > > # Additional image generation features
> > > > > #
> > > > > # The following is a list of classes to import to use in the
> > generation
> > > > of
> > > > > images
> > > > > # currently an example class is image_types_uboot
> > > > > # IMAGE_CLASSES = " image_types_uboot"
> > > > >
> > > > > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.
> > I'd
> > > > > expect a machine config to add wic to IMAGE_CLASSES if it needs wic
> > > > output.
> > > > >
> > > >
> > > > So far all machine configs add wic to IMAGE_TYPES and it works just
> > > > fine. Why to change?
> > > >
> > > > > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <
> > ed.bartosh@linux.intel.com>
> > > > > wrote:
> > > > >
> > > > > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > > > > > Why didn't you make this image_types_wic.bbclass and use
> > > > IMAGE_CLASSES to
> > > > > > > load it?
> > > > > >
> > > > > > Because of the following:
> > > > > > - IMAGE_CLASSES[doc] = "A list of classes that all images should
> > > > > >   inherit." I'm not sure all images should include wic class. I'll
> > > > probably
> > > > > >   make this inheritance conditional.
> > > > > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > > > > >   image_types_uboot and testimage,
> > > > > >   so the usage is more or less follows the description. wic is out
> > of
> > > > > >   that usage scenario, I believe.
> > > > > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> > > > > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> > > > ed.bartosh@linux.intel.com
> > > > > > >
> > > > > > > wrote:
> > > > > > >
> > > > > > > > There is a lot of wic code in image.bbclass and
> > image_types.bbclass
> > > > > > > > Having all code separated in one file should make it more
> > readable
> > > > > > > > and easier to maintain.
> > > > > > > >
> > > > > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > > > > ---
> > > > > > > >  meta/classes/image-wic.bbclass   | 120
> > > > ++++++++++++++++++++++++++++++
> > > > > > > > +++++++++
> > > > > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > > > > >  meta/classes/image_types.bbclass |  95
> > > > ------------------------------
> > > > > > -
> > > > > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > > > > >
> > > > > > > > diff --git a/meta/classes/image-wic.bbclass
> > > > b/meta/classes/image-wic.
> > > > > > > > bbclass
> > > > > > > > new file mode 100644
> > > > > > > > index 0000000..2acfd65
> > > > > > > > --- /dev/null
> > > > > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > > > > @@ -0,0 +1,120 @@
> > > > > > > > +# The WICVARS variable is used to define list of bitbake
> > variables
> > > > > > used
> > > > > > > > in wic code
> > > > > > > > +# variables from this list is written to <image>.env file
> > > > > > > > +WICVARS ?= "\
> > > > > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > > > HDDDIR
> > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES
> > INITRD
> > > > > > ISODIR
> > > > > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> > STAGING_LIBDIR
> > > > > > > > TARGET_SYS"
> > > > > > > > +
> > > > > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for
> > p in
> > > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > > canned-wks'
> > > > > > % l
> > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > +
> > > > > > > > +def wks_search(files, search_path):
> > > > > > > > +    for f in files:
> > > > > > > > +        if os.path.isabs(f):
> > > > > > > > +            if os.path.exists(f):
> > > > > > > > +                return f
> > > > > > > > +        else:
> > > > > > > > +            searched = bb.utils.which(search_path, f)
> > > > > > > > +            if searched:
> > > > > > > > +                return searched
> > > > > > > > +
> > > > > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > +
> > > > > > > > +IMAGE_CMD_wic () {
> > > > > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > +       wks="${WKS_FULL_PATH}"
> > > > > > > > +       if [ -z "$wks" ]; then
> > > > > > > > +               bbfatal "No kickstart files from WKS_FILES were
> > > > found:
> > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > > > > +       fi
> > > > > > > > +
> > > > > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > > > "$out/"
> > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > +       rm -rf "$out/"
> > > > > > > > +}
> > > > > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > > +
> > > > > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic
> > ' + '
> > > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> > '1',
> > > > '',
> > > > > > d)}"
> > > > > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > +
> > > > > > > > +python () {
> > > > > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > +        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +python do_write_wks_template () {
> > > > > > > > +    """Write out expanded template contents to
> > WKS_FULL_PATH."""
> > > > > > > > +    import re
> > > > > > > > +
> > > > > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > +
> > > > > > > > +    # Remove any remnant variable references left behind by
> > the
> > > > > > expansion
> > > > > > > > +    # due to undefined variables
> > > > > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > > +    while True:
> > > > > > > > +        new_body = re.sub(expand_var_regexp, '',
> > template_body)
> > > > > > > > +        if new_body == template_body:
> > > > > > > > +            break
> > > > > > > > +        else:
> > > > > > > > +            template_body = new_body
> > > > > > > > +
> > > > > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > +    with open(wks_file, 'w') as f:
> > > > > > > > +        f.write(template_body)
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +python () {
> > > > > > > > +    if d.getVar('USING_WIC'):
> > > > > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > +        wks_file = d.expand(wks_file_u)
> > > > > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > +            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > > > > os.path.basename(base))
> > > > > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > +
> > > > > > > > +            # We need to re-parse each time the file changes,
> > and
> > > > > > bitbake
> > > > > > > > +            # needs to be told about that explicitly.
> > > > > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > +
> > > > > > > > +            try:
> > > > > > > > +                with open(wks_file, 'r') as f:
> > > > > > > > +                    body = f.read()
> > > > > > > > +            except (IOError, OSError) as exc:
> > > > > > > > +                pass
> > > > > > > > +            else:
> > > > > > > > +                # Previously, I used expandWithRefs to get the
> > > > > > dependency
> > > > > > > > list
> > > > > > > > +                # and add it to WICVARS, but there's no point
> > > > > > re-parsing
> > > > > > > > the
> > > > > > > > +                # file in process_wks_template as well, so
> > just
> > > > put
> > > > > > it in
> > > > > > > > +                # a variable and let the metadata deal with
> > the
> > > > deps.
> > > > > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > +                bb.build.addtask('do_write_wks_template',
> > > > > > > > 'do_image_wic', None, d)
> > > > > > > > +}
> > > > > > > > +
> > > > > > > > +#
> > > > > > > > +# Write environment variables used by wic
> > > > > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > +#
> > > > > > > > +python do_rootfs_wicenv () {
> > > > > > > > +    wicvars = d.getVar('WICVARS')
> > > > > > > > +    if not wicvars:
> > > > > > > > +        return
> > > > > > > > +
> > > > > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > 'imgdata')
> > > > > > > > +    bb.utils.mkdirhier(outdir)
> > > > > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > +    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > > > envf:
> > > > > > > > +        for var in wicvars.split():
> > > > > > > > +            value = d.getVar(var)
> > > > > > > > +            if value:
> > > > > > > > +                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > > > > +}
> > > > > > > > +addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > diff --git a/meta/classes/image.bbclass
> > > > b/meta/classes/image.bbclass
> > > > > > > > index 13fd996..1a549e9 100644
> > > > > > > > --- a/meta/classes/image.bbclass
> > > > > > > > +++ b/meta/classes/image.bbclass
> > > > > > > > @@ -192,6 +192,8 @@ python () {
> > > > > > > >  IMAGE_CLASSES += "image_types"
> > > > > > > >  inherit ${IMAGE_CLASSES}
> > > > > > > >
> > > > > > > > +inherit image-wic
> > > > > > > > +
> > > > > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > > > > >
> > > > > > > >  # some default locales
> > > > > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > > > > >  }
> > > > > > > >  addtask do_image_qa after do_image_complete before do_build
> > > > > > > >
> > > > > > > > -#
> > > > > > > > -# Write environment variables used by wic
> > > > > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > -#
> > > > > > > > -python do_rootfs_wicenv () {
> > > > > > > > -    wicvars = d.getVar('WICVARS')
> > > > > > > > -    if not wicvars:
> > > > > > > > -        return
> > > > > > > > -
> > > > > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > 'imgdata')
> > > > > > > > -    bb.utils.mkdirhier(outdir)
> > > > > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > -    with open(os.path.join(outdir, basename) + '.env', 'w') as
> > > > envf:
> > > > > > > > -        for var in wicvars.split():
> > > > > > > > -            value = d.getVar(var)
> > > > > > > > -            if value:
> > > > > > > > -                envf.write('%s="%s"\n' % (var, value.strip()))
> > > > > > > > -}
> > > > > > > > -addtask do_rootfs_wicenv after do_image before do_image_wic
> > > > > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > -
> > > > > > > >  def setup_debugfs_variables(d):
> > > > > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > > > > diff --git a/meta/classes/image_types.bbclass
> > > > > > b/meta/classes/image_types.
> > > > > > > > bbclass
> > > > > > > > index ad95dc6..e2ecf93 100644
> > > > > > > > --- a/meta/classes/image_types.bbclass
> > > > > > > > +++ b/meta/classes/image_types.bbclass
> > > > > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > > > > >
> > > > > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > > > > > ${MKUBIFS_ARGS}"
> > > > > > > >
> > > > > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p for
> > p in
> > > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > > canned-wks'
> > > > > > % l
> > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > -
> > > > > > > > -def wks_search(files, search_path):
> > > > > > > > -    for f in files:
> > > > > > > > -        if os.path.isabs(f):
> > > > > > > > -            if os.path.exists(f):
> > > > > > > > -                return f
> > > > > > > > -        else:
> > > > > > > > -            searched = bb.utils.which(search_path, f)
> > > > > > > > -            if searched:
> > > > > > > > -                return searched
> > > > > > > > -
> > > > > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > -
> > > > > > > > -IMAGE_CMD_wic () {
> > > > > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > -       wks="${WKS_FULL_PATH}"
> > > > > > > > -       if [ -z "$wks" ]; then
> > > > > > > > -               bbfatal "No kickstart files from WKS_FILES were
> > > > found:
> > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES appropriately."
> > > > > > > > -       fi
> > > > > > > > -
> > > > > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e "${IMAGE_BASENAME}" -o
> > > > "$out/"
> > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > -       rm -rf "$out/"
> > > > > > > > -}
> > > > > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > > -
> > > > > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES', 'wic
> > ' + '
> > > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> > '1',
> > > > '',
> > > > > > d)}"
> > > > > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else ''}"
> > > > > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > -
> > > > > > > > -python () {
> > > > > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > -        bb.build.addtask('do_image_wic', '', 'do_bootimg', d)
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -python do_write_wks_template () {
> > > > > > > > -    """Write out expanded template contents to
> > WKS_FULL_PATH."""
> > > > > > > > -    import re
> > > > > > > > -
> > > > > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > -
> > > > > > > > -    # Remove any remnant variable references left behind by
> > the
> > > > > > expansion
> > > > > > > > -    # due to undefined variables
> > > > > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > > -    while True:
> > > > > > > > -        new_body = re.sub(expand_var_regexp, '',
> > template_body)
> > > > > > > > -        if new_body == template_body:
> > > > > > > > -            break
> > > > > > > > -        else:
> > > > > > > > -            template_body = new_body
> > > > > > > > -
> > > > > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > -    with open(wks_file, 'w') as f:
> > > > > > > > -        f.write(template_body)
> > > > > > > > -}
> > > > > > > > -
> > > > > > > > -python () {
> > > > > > > > -    if d.getVar('USING_WIC'):
> > > > > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > -        wks_file = d.expand(wks_file_u)
> > > > > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > -            wks_out_file = os.path.join(d.getVar('WORKDIR'),
> > > > > > > > os.path.basename(base))
> > > > > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > -
> > > > > > > > -            # We need to re-parse each time the file changes,
> > and
> > > > > > bitbake
> > > > > > > > -            # needs to be told about that explicitly.
> > > > > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > -
> > > > > > > > -            try:
> > > > > > > > -                with open(wks_file, 'r') as f:
> > > > > > > > -                    body = f.read()
> > > > > > > > -            except (IOError, OSError) as exc:
> > > > > > > > -                pass
> > > > > > > > -            else:
> > > > > > > > -                # Previously, I used expandWithRefs to get the
> > > > > > dependency
> > > > > > > > list
> > > > > > > > -                # and add it to WICVARS, but there's no point
> > > > > > re-parsing
> > > > > > > > the
> > > > > > > > -                # file in process_wks_template as well, so
> > just
> > > > put
> > > > > > it in
> > > > > > > > -                # a variable and let the metadata deal with
> > the
> > > > deps.
> > > > > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > -                bb.build.addtask('do_write_wks_template',
> > > > > > > > 'do_image_wic', None, d)
> > > > > > > > -}
> > > > > > > > -
> > > > > > > >  EXTRA_IMAGECMD = ""
> > > > > > > >
> > > > > > > >  inherit siteinfo
> > > > > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > > > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out
> > from
> > > > the
> > > > > > > > IMAGE_FSTYPES,
> > > > > > > >  # images that will not be built at do_rootfs time: vmdk, vdi,
> > > > qcow2,
> > > > > > > > hdddirect, hddimg, iso, etc.
> > > > > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > > > > -
> > > > > > > > -# The WICVARS variable is used to define list of bitbake
> > variables
> > > > > > used
> > > > > > > > in wic code
> > > > > > > > -# variables from this list is written to <image>.env file
> > > > > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE FAKEROOTCMD
> > > > HDDDIR
> > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS
> > > > > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> > RECIPE_SYSROOT_NATIVE
> > > > > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > TARGET_SYS"
> > > > > > > > --
> > > > > > > > 2.1.4
> > > > > > > >
> > > > > > > > --
> > > > > > > > _______________________________________________
> > > > > > > > Openembedded-core mailing list
> > > > > > > > Openembedded-core@lists.openembedded.org
> > > > > > > > http://lists.openembedded.org/mailman/listinfo/openembedded-
> > core
> > > > > > > >
> > > > > >
> > > > > > --
> > > > > > --
> > > > > > Regards,
> > > > > > Ed
> > > > > >
> > > >
> > > > --
> > > > --
> > > > Regards,
> > > > Ed
> > > >
> >
> > --
> > --
> > Regards,
> > Ed
> >

-- 
--
Regards,
Ed

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Rick Altherr]

[-- Attachment #1: Type: text/plain, Size: 23790 bytes --]

LGTM

On Mon, Jan 30, 2017 at 10:42 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> On Mon, Jan 30, 2017 at 10:41:27AM -0800, Rick Altherr wrote:
> > Agreed.  What if it was image_types_wic.bbclass and you did something
> > similar to build_uboot() in image.bbclass?
> >
>
> I can do this for now:
> IMAGE_TYPE_wic = "image_type_wic"
> inherit ${IMAGE_TYPE_wic}
>
> which is the same as 'inherit image_type_wic', just takes 2 lines, but
> looks more consistent.
>
> does this look ok?
>
> > On Mon, Jan 30, 2017 at 10:15 AM, Ed Bartosh <ed.bartosh@linux.intel.com
> >
> > wrote:
> >
> > > On Mon, Jan 30, 2017 at 10:25:59AM -0800, Rick Altherr wrote:
> > > > I'm not clear on which path is the preferred one.  There are lots of
> bits
> > > > and pieces in image_types.bbclass that implement various image types.
> > > >  uboot got added as a separate class at some point.  The comments in
> > > > local.conf.sample.extended imply IMAGE_CLASSES should be used to load
> > > > additional image_types_* classes to add support for additional image
> > > > types.  Having the wic image type implemented in a separate
> > > > image-wic.bbclass that is directly inherited by image.bbclass adds a
> 3rd
> > > > approach.  Which one do we want contributors to use in the future?
> > > >
> > >
> > > I didn't want to create even more confusion. What I wanted is stated
> in the
> > > commit message - to put existing wic code into a file for better
> > > maintenance. If this is more confusing than having wic code in
> different
> > > places of image.class and image_types.class then we can just drop this
> > > patch. However, I personally find it more maintainable this way.
> > >
> > > Suggesting people to change machine configs just because wic code is
> > > moved to separate file doesn't look good to me either.
> > >
> > > > On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <
> ed.bartosh@linux.intel.com>
> > > > wrote:
> > > >
> > > > > On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > > > > > Hmm.  In local.conf.sample.extended, I find:
> > > > > >
> > > > > > # Additional image generation features
> > > > > > #
> > > > > > # The following is a list of classes to import to use in the
> > > generation
> > > > > of
> > > > > > images
> > > > > > # currently an example class is image_types_uboot
> > > > > > # IMAGE_CLASSES = " image_types_uboot"
> > > > > >
> > > > > > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.
> > > I'd
> > > > > > expect a machine config to add wic to IMAGE_CLASSES if it needs
> wic
> > > > > output.
> > > > > >
> > > > >
> > > > > So far all machine configs add wic to IMAGE_TYPES and it works just
> > > > > fine. Why to change?
> > > > >
> > > > > > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <
> > > ed.bartosh@linux.intel.com>
> > > > > > wrote:
> > > > > >
> > > > > > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > > > > > > Why didn't you make this image_types_wic.bbclass and use
> > > > > IMAGE_CLASSES to
> > > > > > > > load it?
> > > > > > >
> > > > > > > Because of the following:
> > > > > > > - IMAGE_CLASSES[doc] = "A list of classes that all images
> should
> > > > > > >   inherit." I'm not sure all images should include wic class.
> I'll
> > > > > probably
> > > > > > >   make this inheritance conditional.
> > > > > > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > > > > > >   image_types_uboot and testimage,
> > > > > > >   so the usage is more or less follows the description. wic is
> out
> > > of
> > > > > > >   that usage scenario, I believe.
> > > > > > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> > > > > > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > > > > > >
> > > > > > >
> > > > > > > >
> > > > > > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> > > > > ed.bartosh@linux.intel.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > There is a lot of wic code in image.bbclass and
> > > image_types.bbclass
> > > > > > > > > Having all code separated in one file should make it more
> > > readable
> > > > > > > > > and easier to maintain.
> > > > > > > > >
> > > > > > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > > > > > ---
> > > > > > > > >  meta/classes/image-wic.bbclass   | 120
> > > > > ++++++++++++++++++++++++++++++
> > > > > > > > > +++++++++
> > > > > > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > > > > > >  meta/classes/image_types.bbclass |  95
> > > > > ------------------------------
> > > > > > > -
> > > > > > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > > > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > > > > > >
> > > > > > > > > diff --git a/meta/classes/image-wic.bbclass
> > > > > b/meta/classes/image-wic.
> > > > > > > > > bbclass
> > > > > > > > > new file mode 100644
> > > > > > > > > index 0000000..2acfd65
> > > > > > > > > --- /dev/null
> > > > > > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > > > > > @@ -0,0 +1,120 @@
> > > > > > > > > +# The WICVARS variable is used to define list of bitbake
> > > variables
> > > > > > > used
> > > > > > > > > in wic code
> > > > > > > > > +# variables from this list is written to <image>.env file
> > > > > > > > > +WICVARS ?= "\
> > > > > > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE
> FAKEROOTCMD
> > > > > HDDDIR
> > > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES
> > > INITRD
> > > > > > > ISODIR
> > > > > > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> > > STAGING_LIBDIR
> > > > > > > > > TARGET_SYS"
> > > > > > > > > +
> > > > > > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p
> for
> > > p in
> > > > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > > > canned-wks'
> > > > > > > % l
> > > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > > +
> > > > > > > > > +def wks_search(files, search_path):
> > > > > > > > > +    for f in files:
> > > > > > > > > +        if os.path.isabs(f):
> > > > > > > > > +            if os.path.exists(f):
> > > > > > > > > +                return f
> > > > > > > > > +        else:
> > > > > > > > > +            searched = bb.utils.which(search_path, f)
> > > > > > > > > +            if searched:
> > > > > > > > > +                return searched
> > > > > > > > > +
> > > > > > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > > +
> > > > > > > > > +IMAGE_CMD_wic () {
> > > > > > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > > +       wks="${WKS_FULL_PATH}"
> > > > > > > > > +       if [ -z "$wks" ]; then
> > > > > > > > > +               bbfatal "No kickstart files from WKS_FILES
> were
> > > > > found:
> > > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> appropriately."
> > > > > > > > > +       fi
> > > > > > > > > +
> > > > > > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e
> "${IMAGE_BASENAME}" -o
> > > > > "$out/"
> > > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > > +       rm -rf "$out/"
> > > > > > > > > +}
> > > > > > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > > > +
> > > > > > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES',
> 'wic
> > > ' + '
> > > > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> > > '1',
> > > > > '',
> > > > > > > d)}"
> > > > > > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else
> ''}"
> > > > > > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > > +
> > > > > > > > > +python () {
> > > > > > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > > +        bb.build.addtask('do_image_wic', '',
> 'do_bootimg', d)
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +python do_write_wks_template () {
> > > > > > > > > +    """Write out expanded template contents to
> > > WKS_FULL_PATH."""
> > > > > > > > > +    import re
> > > > > > > > > +
> > > > > > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > > +
> > > > > > > > > +    # Remove any remnant variable references left behind
> by
> > > the
> > > > > > > expansion
> > > > > > > > > +    # due to undefined variables
> > > > > > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > > > +    while True:
> > > > > > > > > +        new_body = re.sub(expand_var_regexp, '',
> > > template_body)
> > > > > > > > > +        if new_body == template_body:
> > > > > > > > > +            break
> > > > > > > > > +        else:
> > > > > > > > > +            template_body = new_body
> > > > > > > > > +
> > > > > > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > > +    with open(wks_file, 'w') as f:
> > > > > > > > > +        f.write(template_body)
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +python () {
> > > > > > > > > +    if d.getVar('USING_WIC'):
> > > > > > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > > +        wks_file = d.expand(wks_file_u)
> > > > > > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > > +            wks_out_file = os.path.join(d.getVar('
> WORKDIR'),
> > > > > > > > > os.path.basename(base))
> > > > > > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> > > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > > +
> > > > > > > > > +            # We need to re-parse each time the file
> changes,
> > > and
> > > > > > > bitbake
> > > > > > > > > +            # needs to be told about that explicitly.
> > > > > > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > > +
> > > > > > > > > +            try:
> > > > > > > > > +                with open(wks_file, 'r') as f:
> > > > > > > > > +                    body = f.read()
> > > > > > > > > +            except (IOError, OSError) as exc:
> > > > > > > > > +                pass
> > > > > > > > > +            else:
> > > > > > > > > +                # Previously, I used expandWithRefs to
> get the
> > > > > > > dependency
> > > > > > > > > list
> > > > > > > > > +                # and add it to WICVARS, but there's no
> point
> > > > > > > re-parsing
> > > > > > > > > the
> > > > > > > > > +                # file in process_wks_template as well, so
> > > just
> > > > > put
> > > > > > > it in
> > > > > > > > > +                # a variable and let the metadata deal
> with
> > > the
> > > > > deps.
> > > > > > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > > +                bb.build.addtask('do_write_wks_template',
> > > > > > > > > 'do_image_wic', None, d)
> > > > > > > > > +}
> > > > > > > > > +
> > > > > > > > > +#
> > > > > > > > > +# Write environment variables used by wic
> > > > > > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > > +#
> > > > > > > > > +python do_rootfs_wicenv () {
> > > > > > > > > +    wicvars = d.getVar('WICVARS')
> > > > > > > > > +    if not wicvars:
> > > > > > > > > +        return
> > > > > > > > > +
> > > > > > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > > 'imgdata')
> > > > > > > > > +    bb.utils.mkdirhier(outdir)
> > > > > > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > > +    with open(os.path.join(outdir, basename) + '.env',
> 'w') as
> > > > > envf:
> > > > > > > > > +        for var in wicvars.split():
> > > > > > > > > +            value = d.getVar(var)
> > > > > > > > > +            if value:
> > > > > > > > > +                envf.write('%s="%s"\n' % (var,
> value.strip()))
> > > > > > > > > +}
> > > > > > > > > +addtask do_rootfs_wicenv after do_image before
> do_image_wic
> > > > > > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > > diff --git a/meta/classes/image.bbclass
> > > > > b/meta/classes/image.bbclass
> > > > > > > > > index 13fd996..1a549e9 100644
> > > > > > > > > --- a/meta/classes/image.bbclass
> > > > > > > > > +++ b/meta/classes/image.bbclass
> > > > > > > > > @@ -192,6 +192,8 @@ python () {
> > > > > > > > >  IMAGE_CLASSES += "image_types"
> > > > > > > > >  inherit ${IMAGE_CLASSES}
> > > > > > > > >
> > > > > > > > > +inherit image-wic
> > > > > > > > > +
> > > > > > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > > > > > >
> > > > > > > > >  # some default locales
> > > > > > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > > > > > >  }
> > > > > > > > >  addtask do_image_qa after do_image_complete before
> do_build
> > > > > > > > >
> > > > > > > > > -#
> > > > > > > > > -# Write environment variables used by wic
> > > > > > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > > -#
> > > > > > > > > -python do_rootfs_wicenv () {
> > > > > > > > > -    wicvars = d.getVar('WICVARS')
> > > > > > > > > -    if not wicvars:
> > > > > > > > > -        return
> > > > > > > > > -
> > > > > > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > > 'imgdata')
> > > > > > > > > -    bb.utils.mkdirhier(outdir)
> > > > > > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > > -    with open(os.path.join(outdir, basename) + '.env',
> 'w') as
> > > > > envf:
> > > > > > > > > -        for var in wicvars.split():
> > > > > > > > > -            value = d.getVar(var)
> > > > > > > > > -            if value:
> > > > > > > > > -                envf.write('%s="%s"\n' % (var,
> value.strip()))
> > > > > > > > > -}
> > > > > > > > > -addtask do_rootfs_wicenv after do_image before
> do_image_wic
> > > > > > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > > -
> > > > > > > > >  def setup_debugfs_variables(d):
> > > > > > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > > > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > > > > > diff --git a/meta/classes/image_types.bbclass
> > > > > > > b/meta/classes/image_types.
> > > > > > > > > bbclass
> > > > > > > > > index ad95dc6..e2ecf93 100644
> > > > > > > > > --- a/meta/classes/image_types.bbclass
> > > > > > > > > +++ b/meta/classes/image_types.bbclass
> > > > > > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > > > > > >
> > > > > > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > > > > > > ${MKUBIFS_ARGS}"
> > > > > > > > >
> > > > > > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p
> for
> > > p in
> > > > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > > > canned-wks'
> > > > > > > % l
> > > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > > -
> > > > > > > > > -def wks_search(files, search_path):
> > > > > > > > > -    for f in files:
> > > > > > > > > -        if os.path.isabs(f):
> > > > > > > > > -            if os.path.exists(f):
> > > > > > > > > -                return f
> > > > > > > > > -        else:
> > > > > > > > > -            searched = bb.utils.which(search_path, f)
> > > > > > > > > -            if searched:
> > > > > > > > > -                return searched
> > > > > > > > > -
> > > > > > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > > -
> > > > > > > > > -IMAGE_CMD_wic () {
> > > > > > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > > -       wks="${WKS_FULL_PATH}"
> > > > > > > > > -       if [ -z "$wks" ]; then
> > > > > > > > > -               bbfatal "No kickstart files from WKS_FILES
> were
> > > > > found:
> > > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> appropriately."
> > > > > > > > > -       fi
> > > > > > > > > -
> > > > > > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e
> "${IMAGE_BASENAME}" -o
> > > > > "$out/"
> > > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > > -       rm -rf "$out/"
> > > > > > > > > -}
> > > > > > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > > > -
> > > > > > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES',
> 'wic
> > > ' + '
> > > > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> > > '1',
> > > > > '',
> > > > > > > d)}"
> > > > > > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else
> ''}"
> > > > > > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > > -
> > > > > > > > > -python () {
> > > > > > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > > -        bb.build.addtask('do_image_wic', '',
> 'do_bootimg', d)
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -python do_write_wks_template () {
> > > > > > > > > -    """Write out expanded template contents to
> > > WKS_FULL_PATH."""
> > > > > > > > > -    import re
> > > > > > > > > -
> > > > > > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > > -
> > > > > > > > > -    # Remove any remnant variable references left behind
> by
> > > the
> > > > > > > expansion
> > > > > > > > > -    # due to undefined variables
> > > > > > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > > > -    while True:
> > > > > > > > > -        new_body = re.sub(expand_var_regexp, '',
> > > template_body)
> > > > > > > > > -        if new_body == template_body:
> > > > > > > > > -            break
> > > > > > > > > -        else:
> > > > > > > > > -            template_body = new_body
> > > > > > > > > -
> > > > > > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > > -    with open(wks_file, 'w') as f:
> > > > > > > > > -        f.write(template_body)
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > > -python () {
> > > > > > > > > -    if d.getVar('USING_WIC'):
> > > > > > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > > -        wks_file = d.expand(wks_file_u)
> > > > > > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > > -            wks_out_file = os.path.join(d.getVar('
> WORKDIR'),
> > > > > > > > > os.path.basename(base))
> > > > > > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> > > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > > -
> > > > > > > > > -            # We need to re-parse each time the file
> changes,
> > > and
> > > > > > > bitbake
> > > > > > > > > -            # needs to be told about that explicitly.
> > > > > > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > > -
> > > > > > > > > -            try:
> > > > > > > > > -                with open(wks_file, 'r') as f:
> > > > > > > > > -                    body = f.read()
> > > > > > > > > -            except (IOError, OSError) as exc:
> > > > > > > > > -                pass
> > > > > > > > > -            else:
> > > > > > > > > -                # Previously, I used expandWithRefs to
> get the
> > > > > > > dependency
> > > > > > > > > list
> > > > > > > > > -                # and add it to WICVARS, but there's no
> point
> > > > > > > re-parsing
> > > > > > > > > the
> > > > > > > > > -                # file in process_wks_template as well, so
> > > just
> > > > > put
> > > > > > > it in
> > > > > > > > > -                # a variable and let the metadata deal
> with
> > > the
> > > > > deps.
> > > > > > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > > -                bb.build.addtask('do_write_wks_template',
> > > > > > > > > 'do_image_wic', None, d)
> > > > > > > > > -}
> > > > > > > > > -
> > > > > > > > >  EXTRA_IMAGECMD = ""
> > > > > > > > >
> > > > > > > > >  inherit siteinfo
> > > > > > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > > > > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out
> > > from
> > > > > the
> > > > > > > > > IMAGE_FSTYPES,
> > > > > > > > >  # images that will not be built at do_rootfs time: vmdk,
> vdi,
> > > > > qcow2,
> > > > > > > > > hdddirect, hddimg, iso, etc.
> > > > > > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > > > > > -
> > > > > > > > > -# The WICVARS variable is used to define list of bitbake
> > > variables
> > > > > > > used
> > > > > > > > > in wic code
> > > > > > > > > -# variables from this list is written to <image>.env file
> > > > > > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE
> FAKEROOTCMD
> > > > > HDDDIR
> > > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME
> IMAGE_ROOTFS
> > > > > > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> > > RECIPE_SYSROOT_NATIVE
> > > > > > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > TARGET_SYS"
> > > > > > > > > --
> > > > > > > > > 2.1.4
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > _______________________________________________
> > > > > > > > > Openembedded-core mailing list
> > > > > > > > > Openembedded-core@lists.openembedded.org
> > > > > > > > > http://lists.openembedded.org/
> mailman/listinfo/openembedded-
> > > core
> > > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > --
> > > > > > > Regards,
> > > > > > > Ed
> > > > > > >
> > > > >
> > > > > --
> > > > > --
> > > > > Regards,
> > > > > Ed
> > > > >
> > >
> > > --
> > > --
> > > Regards,
> > > Ed
> > >
>
> --
> --
> Regards,
> Ed
>

[-- Attachment #2: Type: text/html, Size: 38278 bytes --]

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Ed Bartosh]

BTW, to be consistent with this approach we also need to rename
image-live and image-vm, right?  Actually, image-live name was the
reason for me to name image-wic.

On Mon, Jan 30, 2017 at 11:07:22AM -0800, Rick Altherr wrote:
> LGTM
> 
> On Mon, Jan 30, 2017 at 10:42 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
> wrote:
> 
> > On Mon, Jan 30, 2017 at 10:41:27AM -0800, Rick Altherr wrote:
> > > Agreed.  What if it was image_types_wic.bbclass and you did something
> > > similar to build_uboot() in image.bbclass?
> > >
> >
> > I can do this for now:
> > IMAGE_TYPE_wic = "image_type_wic"
> > inherit ${IMAGE_TYPE_wic}
> >
> > which is the same as 'inherit image_type_wic', just takes 2 lines, but
> > looks more consistent.
> >
> > does this look ok?
> >
> > > On Mon, Jan 30, 2017 at 10:15 AM, Ed Bartosh <ed.bartosh@linux.intel.com
> > >
> > > wrote:
> > >
> > > > On Mon, Jan 30, 2017 at 10:25:59AM -0800, Rick Altherr wrote:
> > > > > I'm not clear on which path is the preferred one.  There are lots of
> > bits
> > > > > and pieces in image_types.bbclass that implement various image types.
> > > > >  uboot got added as a separate class at some point.  The comments in
> > > > > local.conf.sample.extended imply IMAGE_CLASSES should be used to load
> > > > > additional image_types_* classes to add support for additional image
> > > > > types.  Having the wic image type implemented in a separate
> > > > > image-wic.bbclass that is directly inherited by image.bbclass adds a
> > 3rd
> > > > > approach.  Which one do we want contributors to use in the future?
> > > > >
> > > >
> > > > I didn't want to create even more confusion. What I wanted is stated
> > in the
> > > > commit message - to put existing wic code into a file for better
> > > > maintenance. If this is more confusing than having wic code in
> > different
> > > > places of image.class and image_types.class then we can just drop this
> > > > patch. However, I personally find it more maintainable this way.
> > > >
> > > > Suggesting people to change machine configs just because wic code is
> > > > moved to separate file doesn't look good to me either.
> > > >
> > > > > On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <
> > ed.bartosh@linux.intel.com>
> > > > > wrote:
> > > > >
> > > > > > On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > > > > > > Hmm.  In local.conf.sample.extended, I find:
> > > > > > >
> > > > > > > # Additional image generation features
> > > > > > > #
> > > > > > > # The following is a list of classes to import to use in the
> > > > generation
> > > > > > of
> > > > > > > images
> > > > > > > # currently an example class is image_types_uboot
> > > > > > > # IMAGE_CLASSES = " image_types_uboot"
> > > > > > >
> > > > > > > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by default.
> > > > I'd
> > > > > > > expect a machine config to add wic to IMAGE_CLASSES if it needs
> > wic
> > > > > > output.
> > > > > > >
> > > > > >
> > > > > > So far all machine configs add wic to IMAGE_TYPES and it works just
> > > > > > fine. Why to change?
> > > > > >
> > > > > > > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <
> > > > ed.bartosh@linux.intel.com>
> > > > > > > wrote:
> > > > > > >
> > > > > > > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr wrote:
> > > > > > > > > Why didn't you make this image_types_wic.bbclass and use
> > > > > > IMAGE_CLASSES to
> > > > > > > > > load it?
> > > > > > > >
> > > > > > > > Because of the following:
> > > > > > > > - IMAGE_CLASSES[doc] = "A list of classes that all images
> > should
> > > > > > > >   inherit." I'm not sure all images should include wic class.
> > I'll
> > > > > > probably
> > > > > > > >   make this inheritance conditional.
> > > > > > > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > > > > > > >   image_types_uboot and testimage,
> > > > > > > >   so the usage is more or less follows the description. wic is
> > out
> > > > of
> > > > > > > >   that usage scenario, I believe.
> > > > > > > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES +=
> > > > > > > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > > > > > > >
> > > > > > > >
> > > > > > > > >
> > > > > > > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> > > > > > ed.bartosh@linux.intel.com
> > > > > > > > >
> > > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > > There is a lot of wic code in image.bbclass and
> > > > image_types.bbclass
> > > > > > > > > > Having all code separated in one file should make it more
> > > > readable
> > > > > > > > > > and easier to maintain.
> > > > > > > > > >
> > > > > > > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > > > > > > ---
> > > > > > > > > >  meta/classes/image-wic.bbclass   | 120
> > > > > > ++++++++++++++++++++++++++++++
> > > > > > > > > > +++++++++
> > > > > > > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > > > > > > >  meta/classes/image_types.bbclass |  95
> > > > > > ------------------------------
> > > > > > > > -
> > > > > > > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > > > > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > > > > > > >
> > > > > > > > > > diff --git a/meta/classes/image-wic.bbclass
> > > > > > b/meta/classes/image-wic.
> > > > > > > > > > bbclass
> > > > > > > > > > new file mode 100644
> > > > > > > > > > index 0000000..2acfd65
> > > > > > > > > > --- /dev/null
> > > > > > > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > > > > > > @@ -0,0 +1,120 @@
> > > > > > > > > > +# The WICVARS variable is used to define list of bitbake
> > > > variables
> > > > > > > > used
> > > > > > > > > > in wic code
> > > > > > > > > > +# variables from this list is written to <image>.env file
> > > > > > > > > > +WICVARS ?= "\
> > > > > > > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE
> > FAKEROOTCMD
> > > > > > HDDDIR
> > > > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > > > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES
> > > > INITRD
> > > > > > > > ISODIR
> > > > > > > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > > > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> > > > STAGING_LIBDIR
> > > > > > > > > > TARGET_SYS"
> > > > > > > > > > +
> > > > > > > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p
> > for
> > > > p in
> > > > > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > > > > canned-wks'
> > > > > > > > % l
> > > > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > > > +
> > > > > > > > > > +def wks_search(files, search_path):
> > > > > > > > > > +    for f in files:
> > > > > > > > > > +        if os.path.isabs(f):
> > > > > > > > > > +            if os.path.exists(f):
> > > > > > > > > > +                return f
> > > > > > > > > > +        else:
> > > > > > > > > > +            searched = bb.utils.which(search_path, f)
> > > > > > > > > > +            if searched:
> > > > > > > > > > +                return searched
> > > > > > > > > > +
> > > > > > > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > > > +
> > > > > > > > > > +IMAGE_CMD_wic () {
> > > > > > > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > > > +       wks="${WKS_FULL_PATH}"
> > > > > > > > > > +       if [ -z "$wks" ]; then
> > > > > > > > > > +               bbfatal "No kickstart files from WKS_FILES
> > were
> > > > > > found:
> > > > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> > appropriately."
> > > > > > > > > > +       fi
> > > > > > > > > > +
> > > > > > > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e
> > "${IMAGE_BASENAME}" -o
> > > > > > "$out/"
> > > > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > > > +       rm -rf "$out/"
> > > > > > > > > > +}
> > > > > > > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > > > > +
> > > > > > > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > > > +USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES',
> > 'wic
> > > > ' + '
> > > > > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> > > > '1',
> > > > > > '',
> > > > > > > > d)}"
> > > > > > > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else
> > ''}"
> > > > > > > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > > > +
> > > > > > > > > > +python () {
> > > > > > > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > > > +        bb.build.addtask('do_image_wic', '',
> > 'do_bootimg', d)
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +python do_write_wks_template () {
> > > > > > > > > > +    """Write out expanded template contents to
> > > > WKS_FULL_PATH."""
> > > > > > > > > > +    import re
> > > > > > > > > > +
> > > > > > > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > > > +
> > > > > > > > > > +    # Remove any remnant variable references left behind
> > by
> > > > the
> > > > > > > > expansion
> > > > > > > > > > +    # due to undefined variables
> > > > > > > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > > > > +    while True:
> > > > > > > > > > +        new_body = re.sub(expand_var_regexp, '',
> > > > template_body)
> > > > > > > > > > +        if new_body == template_body:
> > > > > > > > > > +            break
> > > > > > > > > > +        else:
> > > > > > > > > > +            template_body = new_body
> > > > > > > > > > +
> > > > > > > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > > > +    with open(wks_file, 'w') as f:
> > > > > > > > > > +        f.write(template_body)
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +python () {
> > > > > > > > > > +    if d.getVar('USING_WIC'):
> > > > > > > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > > > +        wks_file = d.expand(wks_file_u)
> > > > > > > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > > > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > > > +            wks_out_file = os.path.join(d.getVar('
> > WORKDIR'),
> > > > > > > > > > os.path.basename(base))
> > > > > > > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> > > > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > > > +
> > > > > > > > > > +            # We need to re-parse each time the file
> > changes,
> > > > and
> > > > > > > > bitbake
> > > > > > > > > > +            # needs to be told about that explicitly.
> > > > > > > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > > > +
> > > > > > > > > > +            try:
> > > > > > > > > > +                with open(wks_file, 'r') as f:
> > > > > > > > > > +                    body = f.read()
> > > > > > > > > > +            except (IOError, OSError) as exc:
> > > > > > > > > > +                pass
> > > > > > > > > > +            else:
> > > > > > > > > > +                # Previously, I used expandWithRefs to
> > get the
> > > > > > > > dependency
> > > > > > > > > > list
> > > > > > > > > > +                # and add it to WICVARS, but there's no
> > point
> > > > > > > > re-parsing
> > > > > > > > > > the
> > > > > > > > > > +                # file in process_wks_template as well, so
> > > > just
> > > > > > put
> > > > > > > > it in
> > > > > > > > > > +                # a variable and let the metadata deal
> > with
> > > > the
> > > > > > deps.
> > > > > > > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > > > +                bb.build.addtask('do_write_wks_template',
> > > > > > > > > > 'do_image_wic', None, d)
> > > > > > > > > > +}
> > > > > > > > > > +
> > > > > > > > > > +#
> > > > > > > > > > +# Write environment variables used by wic
> > > > > > > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > > > +#
> > > > > > > > > > +python do_rootfs_wicenv () {
> > > > > > > > > > +    wicvars = d.getVar('WICVARS')
> > > > > > > > > > +    if not wicvars:
> > > > > > > > > > +        return
> > > > > > > > > > +
> > > > > > > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > > > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > > > 'imgdata')
> > > > > > > > > > +    bb.utils.mkdirhier(outdir)
> > > > > > > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > > > +    with open(os.path.join(outdir, basename) + '.env',
> > 'w') as
> > > > > > envf:
> > > > > > > > > > +        for var in wicvars.split():
> > > > > > > > > > +            value = d.getVar(var)
> > > > > > > > > > +            if value:
> > > > > > > > > > +                envf.write('%s="%s"\n' % (var,
> > value.strip()))
> > > > > > > > > > +}
> > > > > > > > > > +addtask do_rootfs_wicenv after do_image before
> > do_image_wic
> > > > > > > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > > > diff --git a/meta/classes/image.bbclass
> > > > > > b/meta/classes/image.bbclass
> > > > > > > > > > index 13fd996..1a549e9 100644
> > > > > > > > > > --- a/meta/classes/image.bbclass
> > > > > > > > > > +++ b/meta/classes/image.bbclass
> > > > > > > > > > @@ -192,6 +192,8 @@ python () {
> > > > > > > > > >  IMAGE_CLASSES += "image_types"
> > > > > > > > > >  inherit ${IMAGE_CLASSES}
> > > > > > > > > >
> > > > > > > > > > +inherit image-wic
> > > > > > > > > > +
> > > > > > > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > > > > > > >
> > > > > > > > > >  # some default locales
> > > > > > > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > > > > > > >  }
> > > > > > > > > >  addtask do_image_qa after do_image_complete before
> > do_build
> > > > > > > > > >
> > > > > > > > > > -#
> > > > > > > > > > -# Write environment variables used by wic
> > > > > > > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > > > -#
> > > > > > > > > > -python do_rootfs_wicenv () {
> > > > > > > > > > -    wicvars = d.getVar('WICVARS')
> > > > > > > > > > -    if not wicvars:
> > > > > > > > > > -        return
> > > > > > > > > > -
> > > > > > > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > > > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > > > 'imgdata')
> > > > > > > > > > -    bb.utils.mkdirhier(outdir)
> > > > > > > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > > > -    with open(os.path.join(outdir, basename) + '.env',
> > 'w') as
> > > > > > envf:
> > > > > > > > > > -        for var in wicvars.split():
> > > > > > > > > > -            value = d.getVar(var)
> > > > > > > > > > -            if value:
> > > > > > > > > > -                envf.write('%s="%s"\n' % (var,
> > value.strip()))
> > > > > > > > > > -}
> > > > > > > > > > -addtask do_rootfs_wicenv after do_image before
> > do_image_wic
> > > > > > > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > > > -
> > > > > > > > > >  def setup_debugfs_variables(d):
> > > > > > > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > > > > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > > > > > > diff --git a/meta/classes/image_types.bbclass
> > > > > > > > b/meta/classes/image_types.
> > > > > > > > > > bbclass
> > > > > > > > > > index ad95dc6..e2ecf93 100644
> > > > > > > > > > --- a/meta/classes/image_types.bbclass
> > > > > > > > > > +++ b/meta/classes/image_types.bbclass
> > > > > > > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > > > > > > >
> > > > > > > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > > > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.ubifs
> > > > > > > > ${MKUBIFS_ARGS}"
> > > > > > > > > >
> > > > > > > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic' % p
> > for
> > > > p in
> > > > > > > > > > '${BBPATH}'.split(':'))}:${@':'.join('%s/scripts/lib/wic/
> > > > > > canned-wks'
> > > > > > > > % l
> > > > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.split(),
> > > > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > > > -
> > > > > > > > > > -def wks_search(files, search_path):
> > > > > > > > > > -    for f in files:
> > > > > > > > > > -        if os.path.isabs(f):
> > > > > > > > > > -            if os.path.exists(f):
> > > > > > > > > > -                return f
> > > > > > > > > > -        else:
> > > > > > > > > > -            searched = bb.utils.which(search_path, f)
> > > > > > > > > > -            if searched:
> > > > > > > > > > -                return searched
> > > > > > > > > > -
> > > > > > > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > > > -
> > > > > > > > > > -IMAGE_CMD_wic () {
> > > > > > > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > > > -       wks="${WKS_FULL_PATH}"
> > > > > > > > > > -       if [ -z "$wks" ]; then
> > > > > > > > > > -               bbfatal "No kickstart files from WKS_FILES
> > were
> > > > > > found:
> > > > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> > appropriately."
> > > > > > > > > > -       fi
> > > > > > > > > > -
> > > > > > > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e
> > "${IMAGE_BASENAME}" -o
> > > > > > "$out/"
> > > > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > > > -       rm -rf "$out/"
> > > > > > > > > > -}
> > > > > > > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH WKS_FILES"
> > > > > > > > > > -
> > > > > > > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > > > -USING_WIC = "${@bb.utils.contains_any('IMAGE_FSTYPES',
> > 'wic
> > > > ' + '
> > > > > > > > > > '.join('wic.%s' % c for c in '${CONVERSIONTYPES}'.split()),
> > > > '1',
> > > > > > '',
> > > > > > > > d)}"
> > > > > > > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}' else
> > ''}"
> > > > > > > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > > > -
> > > > > > > > > > -python () {
> > > > > > > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > > > -        bb.build.addtask('do_image_wic', '',
> > 'do_bootimg', d)
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -python do_write_wks_template () {
> > > > > > > > > > -    """Write out expanded template contents to
> > > > WKS_FULL_PATH."""
> > > > > > > > > > -    import re
> > > > > > > > > > -
> > > > > > > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > > > -
> > > > > > > > > > -    # Remove any remnant variable references left behind
> > by
> > > > the
> > > > > > > > expansion
> > > > > > > > > > -    # due to undefined variables
> > > > > > > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}")
> > > > > > > > > > -    while True:
> > > > > > > > > > -        new_body = re.sub(expand_var_regexp, '',
> > > > template_body)
> > > > > > > > > > -        if new_body == template_body:
> > > > > > > > > > -            break
> > > > > > > > > > -        else:
> > > > > > > > > > -            template_body = new_body
> > > > > > > > > > -
> > > > > > > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > > > -    with open(wks_file, 'w') as f:
> > > > > > > > > > -        f.write(template_body)
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > > -python () {
> > > > > > > > > > -    if d.getVar('USING_WIC'):
> > > > > > > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > > > -        wks_file = d.expand(wks_file_u)
> > > > > > > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > > > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > > > -            wks_out_file = os.path.join(d.getVar('
> > WORKDIR'),
> > > > > > > > > > os.path.basename(base))
> > > > > > > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> > > > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > > > -
> > > > > > > > > > -            # We need to re-parse each time the file
> > changes,
> > > > and
> > > > > > > > bitbake
> > > > > > > > > > -            # needs to be told about that explicitly.
> > > > > > > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > > > -
> > > > > > > > > > -            try:
> > > > > > > > > > -                with open(wks_file, 'r') as f:
> > > > > > > > > > -                    body = f.read()
> > > > > > > > > > -            except (IOError, OSError) as exc:
> > > > > > > > > > -                pass
> > > > > > > > > > -            else:
> > > > > > > > > > -                # Previously, I used expandWithRefs to
> > get the
> > > > > > > > dependency
> > > > > > > > > > list
> > > > > > > > > > -                # and add it to WICVARS, but there's no
> > point
> > > > > > > > re-parsing
> > > > > > > > > > the
> > > > > > > > > > -                # file in process_wks_template as well, so
> > > > just
> > > > > > put
> > > > > > > > it in
> > > > > > > > > > -                # a variable and let the metadata deal
> > with
> > > > the
> > > > > > deps.
> > > > > > > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > > > -                bb.build.addtask('do_write_wks_template',
> > > > > > > > > > 'do_image_wic', None, d)
> > > > > > > > > > -}
> > > > > > > > > > -
> > > > > > > > > >  EXTRA_IMAGECMD = ""
> > > > > > > > > >
> > > > > > > > > >  inherit siteinfo
> > > > > > > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > > > > > > >  # The IMAGE_TYPES_MASKED variable will be used to mask out
> > > > from
> > > > > > the
> > > > > > > > > > IMAGE_FSTYPES,
> > > > > > > > > >  # images that will not be built at do_rootfs time: vmdk,
> > vdi,
> > > > > > qcow2,
> > > > > > > > > > hdddirect, hddimg, iso, etc.
> > > > > > > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > > > > > > -
> > > > > > > > > > -# The WICVARS variable is used to define list of bitbake
> > > > variables
> > > > > > > > used
> > > > > > > > > > in wic code
> > > > > > > > > > -# variables from this list is written to <image>.env file
> > > > > > > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE
> > FAKEROOTCMD
> > > > > > HDDDIR
> > > > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME
> > IMAGE_ROOTFS
> > > > > > > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> > > > RECIPE_SYSROOT_NATIVE
> > > > > > > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > > TARGET_SYS"
> > > > > > > > > > --
> > > > > > > > > > 2.1.4
> > > > > > > > > >
> > > > > > > > > > --
> > > > > > > > > > _______________________________________________
> > > > > > > > > > Openembedded-core mailing list
> > > > > > > > > > Openembedded-core@lists.openembedded.org
> > > > > > > > > > http://lists.openembedded.org/
> > mailman/listinfo/openembedded-
> > > > core
> > > > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > --
> > > > > > > > Regards,
> > > > > > > > Ed
> > > > > > > >
> > > > > >
> > > > > > --
> > > > > > --
> > > > > > Regards,
> > > > > > Ed
> > > > > >
> > > >
> > > > --
> > > > --
> > > > Regards,
> > > > Ed
> > > >
> >
> > --
> > --
> > Regards,
> > Ed
> >

-- 
--
Regards,
Ed

Re: [PATCH v3 01/11] image-wic: move wic code to image-wic.bbclass

[Rick Altherr]

[-- Attachment #1: Type: text/plain, Size: 26884 bytes --]

I guess so.  Or rename image_types_uboot to image-uboot.  Personally, I'm
going to assume image-live has to do with adding packages to generate a
live image whereas image_types_live would be for turning an image into a
bootable ISO.

On Wed, Feb 1, 2017 at 7:00 AM, Ed Bartosh <ed.bartosh@linux.intel.com>
wrote:

> BTW, to be consistent with this approach we also need to rename
> image-live and image-vm, right?  Actually, image-live name was the
> reason for me to name image-wic.
>
> On Mon, Jan 30, 2017 at 11:07:22AM -0800, Rick Altherr wrote:
> > LGTM
> >
> > On Mon, Jan 30, 2017 at 10:42 AM, Ed Bartosh <ed.bartosh@linux.intel.com
> >
> > wrote:
> >
> > > On Mon, Jan 30, 2017 at 10:41:27AM -0800, Rick Altherr wrote:
> > > > Agreed.  What if it was image_types_wic.bbclass and you did something
> > > > similar to build_uboot() in image.bbclass?
> > > >
> > >
> > > I can do this for now:
> > > IMAGE_TYPE_wic = "image_type_wic"
> > > inherit ${IMAGE_TYPE_wic}
> > >
> > > which is the same as 'inherit image_type_wic', just takes 2 lines, but
> > > looks more consistent.
> > >
> > > does this look ok?
> > >
> > > > On Mon, Jan 30, 2017 at 10:15 AM, Ed Bartosh <
> ed.bartosh@linux.intel.com
> > > >
> > > > wrote:
> > > >
> > > > > On Mon, Jan 30, 2017 at 10:25:59AM -0800, Rick Altherr wrote:
> > > > > > I'm not clear on which path is the preferred one.  There are
> lots of
> > > bits
> > > > > > and pieces in image_types.bbclass that implement various image
> types.
> > > > > >  uboot got added as a separate class at some point.  The
> comments in
> > > > > > local.conf.sample.extended imply IMAGE_CLASSES should be used to
> load
> > > > > > additional image_types_* classes to add support for additional
> image
> > > > > > types.  Having the wic image type implemented in a separate
> > > > > > image-wic.bbclass that is directly inherited by image.bbclass
> adds a
> > > 3rd
> > > > > > approach.  Which one do we want contributors to use in the
> future?
> > > > > >
> > > > >
> > > > > I didn't want to create even more confusion. What I wanted is
> stated
> > > in the
> > > > > commit message - to put existing wic code into a file for better
> > > > > maintenance. If this is more confusing than having wic code in
> > > different
> > > > > places of image.class and image_types.class then we can just drop
> this
> > > > > patch. However, I personally find it more maintainable this way.
> > > > >
> > > > > Suggesting people to change machine configs just because wic code
> is
> > > > > moved to separate file doesn't look good to me either.
> > > > >
> > > > > > On Mon, Jan 30, 2017 at 9:45 AM, Ed Bartosh <
> > > ed.bartosh@linux.intel.com>
> > > > > > wrote:
> > > > > >
> > > > > > > On Mon, Jan 30, 2017 at 09:47:42AM -0800, Rick Altherr wrote:
> > > > > > > > Hmm.  In local.conf.sample.extended, I find:
> > > > > > > >
> > > > > > > > # Additional image generation features
> > > > > > > > #
> > > > > > > > # The following is a list of classes to import to use in the
> > > > > generation
> > > > > > > of
> > > > > > > > images
> > > > > > > > # currently an example class is image_types_uboot
> > > > > > > > # IMAGE_CLASSES = " image_types_uboot"
> > > > > > > >
> > > > > > > > Indeed, image_types_uboot isn't part of IMAGE_CLASSES by
> default.
> > > > > I'd
> > > > > > > > expect a machine config to add wic to IMAGE_CLASSES if it
> needs
> > > wic
> > > > > > > output.
> > > > > > > >
> > > > > > >
> > > > > > > So far all machine configs add wic to IMAGE_TYPES and it works
> just
> > > > > > > fine. Why to change?
> > > > > > >
> > > > > > > > On Mon, Jan 30, 2017 at 9:18 AM, Ed Bartosh <
> > > > > ed.bartosh@linux.intel.com>
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > On Mon, Jan 30, 2017 at 09:27:54AM -0800, Rick Altherr
> wrote:
> > > > > > > > > > Why didn't you make this image_types_wic.bbclass and use
> > > > > > > IMAGE_CLASSES to
> > > > > > > > > > load it?
> > > > > > > > >
> > > > > > > > > Because of the following:
> > > > > > > > > - IMAGE_CLASSES[doc] = "A list of classes that all images
> > > should
> > > > > > > > >   inherit." I'm not sure all images should include wic
> class.
> > > I'll
> > > > > > > probably
> > > > > > > > >   make this inheritance conditional.
> > > > > > > > > - so far IMAGE_CLASSES is used for qemuboot, image_types,
> > > > > > > > >   image_types_uboot and testimage,
> > > > > > > > >   so the usage is more or less follows the description.
> wic is
> > > out
> > > > > of
> > > > > > > > >   that usage scenario, I believe.
> > > > > > > > > - 'inherit image_wic' is more explicit than IMAGE_CLASSES
> +=
> > > > > > > > >   "image_types"\n inherit ${IMAGE_CLASSES}
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > On Fri, Jan 27, 2017 at 12:19 PM, Ed Bartosh <
> > > > > > > ed.bartosh@linux.intel.com
> > > > > > > > > >
> > > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > There is a lot of wic code in image.bbclass and
> > > > > image_types.bbclass
> > > > > > > > > > > Having all code separated in one file should make it
> more
> > > > > readable
> > > > > > > > > > > and easier to maintain.
> > > > > > > > > > >
> > > > > > > > > > > Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> > > > > > > > > > > ---
> > > > > > > > > > >  meta/classes/image-wic.bbclass   | 120
> > > > > > > ++++++++++++++++++++++++++++++
> > > > > > > > > > > +++++++++
> > > > > > > > > > >  meta/classes/image.bbclass       |  25 +-------
> > > > > > > > > > >  meta/classes/image_types.bbclass |  95
> > > > > > > ------------------------------
> > > > > > > > > -
> > > > > > > > > > >  3 files changed, 122 insertions(+), 118 deletions(-)
> > > > > > > > > > >  create mode 100644 meta/classes/image-wic.bbclass
> > > > > > > > > > >
> > > > > > > > > > > diff --git a/meta/classes/image-wic.bbclass
> > > > > > > b/meta/classes/image-wic.
> > > > > > > > > > > bbclass
> > > > > > > > > > > new file mode 100644
> > > > > > > > > > > index 0000000..2acfd65
> > > > > > > > > > > --- /dev/null
> > > > > > > > > > > +++ b/meta/classes/image-wic.bbclass
> > > > > > > > > > > @@ -0,0 +1,120 @@
> > > > > > > > > > > +# The WICVARS variable is used to define list of
> bitbake
> > > > > variables
> > > > > > > > > used
> > > > > > > > > > > in wic code
> > > > > > > > > > > +# variables from this list is written to <image>.env
> file
> > > > > > > > > > > +WICVARS ?= "\
> > > > > > > > > > > +           BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE
> > > FAKEROOTCMD
> > > > > > > HDDDIR
> > > > > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES \
> > > > > > > > > > > +           IMAGE_LINK_NAME IMAGE_ROOTFS
> INITRAMFS_FSTYPES
> > > > > INITRD
> > > > > > > > > ISODIR
> > > > > > > > > > > MACHINE_ARCH RECIPE_SYSROOT_NATIVE \
> > > > > > > > > > > +           ROOTFS_SIZE STAGING_DATADIR STAGING_DIR
> > > > > STAGING_LIBDIR
> > > > > > > > > > > TARGET_SYS"
> > > > > > > > > > > +
> > > > > > > > > > > +WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > > > > +WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > > > > +WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic'
> % p
> > > for
> > > > > p in
> > > > > > > > > > > '${BBPATH}'.split(':'))}:${@':
> '.join('%s/scripts/lib/wic/
> > > > > > > canned-wks'
> > > > > > > > > % l
> > > > > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > > > > +WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.
> split(),
> > > > > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > > > > +
> > > > > > > > > > > +def wks_search(files, search_path):
> > > > > > > > > > > +    for f in files:
> > > > > > > > > > > +        if os.path.isabs(f):
> > > > > > > > > > > +            if os.path.exists(f):
> > > > > > > > > > > +                return f
> > > > > > > > > > > +        else:
> > > > > > > > > > > +            searched = bb.utils.which(search_path, f)
> > > > > > > > > > > +            if searched:
> > > > > > > > > > > +                return searched
> > > > > > > > > > > +
> > > > > > > > > > > +WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > > > > +
> > > > > > > > > > > +IMAGE_CMD_wic () {
> > > > > > > > > > > +       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > > > > +       wks="${WKS_FULL_PATH}"
> > > > > > > > > > > +       if [ -z "$wks" ]; then
> > > > > > > > > > > +               bbfatal "No kickstart files from
> WKS_FILES
> > > were
> > > > > > > found:
> > > > > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> > > appropriately."
> > > > > > > > > > > +       fi
> > > > > > > > > > > +
> > > > > > > > > > > +       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e
> > > "${IMAGE_BASENAME}" -o
> > > > > > > "$out/"
> > > > > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > > > > +       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > > > > +       rm -rf "$out/"
> > > > > > > > > > > +}
> > > > > > > > > > > +IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH
> WKS_FILES"
> > > > > > > > > > > +
> > > > > > > > > > > +# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > > > > +USING_WIC = "${@bb.utils.contains_any('
> IMAGE_FSTYPES',
> > > 'wic
> > > > > ' + '
> > > > > > > > > > > '.join('wic.%s' % c for c in
> '${CONVERSIONTYPES}'.split()),
> > > > > '1',
> > > > > > > '',
> > > > > > > > > d)}"
> > > > > > > > > > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}'
> else
> > > ''}"
> > > > > > > > > > > +do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > > > > +do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > > > > +
> > > > > > > > > > > +python () {
> > > > > > > > > > > +    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > > > > +        bb.build.addtask('do_image_wic', '',
> > > 'do_bootimg', d)
> > > > > > > > > > > +}
> > > > > > > > > > > +
> > > > > > > > > > > +python do_write_wks_template () {
> > > > > > > > > > > +    """Write out expanded template contents to
> > > > > WKS_FULL_PATH."""
> > > > > > > > > > > +    import re
> > > > > > > > > > > +
> > > > > > > > > > > +    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > > > > +
> > > > > > > > > > > +    # Remove any remnant variable references left
> behind
> > > by
> > > > > the
> > > > > > > > > expansion
> > > > > > > > > > > +    # due to undefined variables
> > > > > > > > > > > +    expand_var_regexp = re.compile(r"\${[^{}@\n\t
> :]+}")
> > > > > > > > > > > +    while True:
> > > > > > > > > > > +        new_body = re.sub(expand_var_regexp, '',
> > > > > template_body)
> > > > > > > > > > > +        if new_body == template_body:
> > > > > > > > > > > +            break
> > > > > > > > > > > +        else:
> > > > > > > > > > > +            template_body = new_body
> > > > > > > > > > > +
> > > > > > > > > > > +    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > > > > +    with open(wks_file, 'w') as f:
> > > > > > > > > > > +        f.write(template_body)
> > > > > > > > > > > +}
> > > > > > > > > > > +
> > > > > > > > > > > +python () {
> > > > > > > > > > > +    if d.getVar('USING_WIC'):
> > > > > > > > > > > +        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > > > > +        wks_file = d.expand(wks_file_u)
> > > > > > > > > > > +        base, ext = os.path.splitext(wks_file)
> > > > > > > > > > > +        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > > > > +            wks_out_file = os.path.join(d.getVar('
> > > WORKDIR'),
> > > > > > > > > > > os.path.basename(base))
> > > > > > > > > > > +            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > > > > +            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > > > > +            d.setVar('WKS_FILE_CHECKSUM',
> > > > > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > > > > +
> > > > > > > > > > > +            # We need to re-parse each time the file
> > > changes,
> > > > > and
> > > > > > > > > bitbake
> > > > > > > > > > > +            # needs to be told about that explicitly.
> > > > > > > > > > > +            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > > > > +
> > > > > > > > > > > +            try:
> > > > > > > > > > > +                with open(wks_file, 'r') as f:
> > > > > > > > > > > +                    body = f.read()
> > > > > > > > > > > +            except (IOError, OSError) as exc:
> > > > > > > > > > > +                pass
> > > > > > > > > > > +            else:
> > > > > > > > > > > +                # Previously, I used expandWithRefs to
> > > get the
> > > > > > > > > dependency
> > > > > > > > > > > list
> > > > > > > > > > > +                # and add it to WICVARS, but there's
> no
> > > point
> > > > > > > > > re-parsing
> > > > > > > > > > > the
> > > > > > > > > > > +                # file in process_wks_template as
> well, so
> > > > > just
> > > > > > > put
> > > > > > > > > it in
> > > > > > > > > > > +                # a variable and let the metadata deal
> > > with
> > > > > the
> > > > > > > deps.
> > > > > > > > > > > +                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > > > > +                bb.build.addtask('do_write_
> wks_template',
> > > > > > > > > > > 'do_image_wic', None, d)
> > > > > > > > > > > +}
> > > > > > > > > > > +
> > > > > > > > > > > +#
> > > > > > > > > > > +# Write environment variables used by wic
> > > > > > > > > > > +# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > > > > +#
> > > > > > > > > > > +python do_rootfs_wicenv () {
> > > > > > > > > > > +    wicvars = d.getVar('WICVARS')
> > > > > > > > > > > +    if not wicvars:
> > > > > > > > > > > +        return
> > > > > > > > > > > +
> > > > > > > > > > > +    stdir = d.getVar('STAGING_DIR')
> > > > > > > > > > > +    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > > > > 'imgdata')
> > > > > > > > > > > +    bb.utils.mkdirhier(outdir)
> > > > > > > > > > > +    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > > > > +    with open(os.path.join(outdir, basename) + '.env',
> > > 'w') as
> > > > > > > envf:
> > > > > > > > > > > +        for var in wicvars.split():
> > > > > > > > > > > +            value = d.getVar(var)
> > > > > > > > > > > +            if value:
> > > > > > > > > > > +                envf.write('%s="%s"\n' % (var,
> > > value.strip()))
> > > > > > > > > > > +}
> > > > > > > > > > > +addtask do_rootfs_wicenv after do_image before
> > > do_image_wic
> > > > > > > > > > > +do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > > > > +do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > > > > diff --git a/meta/classes/image.bbclass
> > > > > > > b/meta/classes/image.bbclass
> > > > > > > > > > > index 13fd996..1a549e9 100644
> > > > > > > > > > > --- a/meta/classes/image.bbclass
> > > > > > > > > > > +++ b/meta/classes/image.bbclass
> > > > > > > > > > > @@ -192,6 +192,8 @@ python () {
> > > > > > > > > > >  IMAGE_CLASSES += "image_types"
> > > > > > > > > > >  inherit ${IMAGE_CLASSES}
> > > > > > > > > > >
> > > > > > > > > > > +inherit image-wic
> > > > > > > > > > > +
> > > > > > > > > > >  IMAGE_POSTPROCESS_COMMAND ?= ""
> > > > > > > > > > >
> > > > > > > > > > >  # some default locales
> > > > > > > > > > > @@ -332,29 +334,6 @@ fakeroot python do_image_qa () {
> > > > > > > > > > >  }
> > > > > > > > > > >  addtask do_image_qa after do_image_complete before
> > > do_build
> > > > > > > > > > >
> > > > > > > > > > > -#
> > > > > > > > > > > -# Write environment variables used by wic
> > > > > > > > > > > -# to tmp/sysroots/<machine>/imgdata/<image>.env
> > > > > > > > > > > -#
> > > > > > > > > > > -python do_rootfs_wicenv () {
> > > > > > > > > > > -    wicvars = d.getVar('WICVARS')
> > > > > > > > > > > -    if not wicvars:
> > > > > > > > > > > -        return
> > > > > > > > > > > -
> > > > > > > > > > > -    stdir = d.getVar('STAGING_DIR')
> > > > > > > > > > > -    outdir = os.path.join(stdir, d.getVar('MACHINE'),
> > > > > 'imgdata')
> > > > > > > > > > > -    bb.utils.mkdirhier(outdir)
> > > > > > > > > > > -    basename = d.getVar('IMAGE_BASENAME')
> > > > > > > > > > > -    with open(os.path.join(outdir, basename) + '.env',
> > > 'w') as
> > > > > > > envf:
> > > > > > > > > > > -        for var in wicvars.split():
> > > > > > > > > > > -            value = d.getVar(var)
> > > > > > > > > > > -            if value:
> > > > > > > > > > > -                envf.write('%s="%s"\n' % (var,
> > > value.strip()))
> > > > > > > > > > > -}
> > > > > > > > > > > -addtask do_rootfs_wicenv after do_image before
> > > do_image_wic
> > > > > > > > > > > -do_rootfs_wicenv[vardeps] += "${WICVARS}"
> > > > > > > > > > > -do_rootfs_wicenv[prefuncs] = 'set_image_size'
> > > > > > > > > > > -
> > > > > > > > > > >  def setup_debugfs_variables(d):
> > > > > > > > > > >      d.appendVar('IMAGE_ROOTFS', '-dbg')
> > > > > > > > > > >      d.appendVar('IMAGE_LINK_NAME', '-dbg')
> > > > > > > > > > > diff --git a/meta/classes/image_types.bbclass
> > > > > > > > > b/meta/classes/image_types.
> > > > > > > > > > > bbclass
> > > > > > > > > > > index ad95dc6..e2ecf93 100644
> > > > > > > > > > > --- a/meta/classes/image_types.bbclass
> > > > > > > > > > > +++ b/meta/classes/image_types.bbclass
> > > > > > > > > > > @@ -192,97 +192,6 @@ IMAGE_CMD_ubi () {
> > > > > > > > > > >
> > > > > > > > > > >  IMAGE_CMD_ubifs = "mkfs.ubifs -r ${IMAGE_ROOTFS} -o
> > > > > > > > > > > ${IMGDEPLOYDIR}/${IMAGE_NAME}$
> {IMAGE_NAME_SUFFIX}.ubifs
> > > > > > > > > ${MKUBIFS_ARGS}"
> > > > > > > > > > >
> > > > > > > > > > > -WKS_FILE ??= "${IMAGE_BASENAME}.${MACHINE}.wks"
> > > > > > > > > > > -WKS_FILES ?= "${WKS_FILE} ${IMAGE_BASENAME}.wks"
> > > > > > > > > > > -WKS_SEARCH_PATH ?= "${THISDIR}:${@':'.join('%s/wic'
> % p
> > > for
> > > > > p in
> > > > > > > > > > > '${BBPATH}'.split(':'))}:${@':
> '.join('%s/scripts/lib/wic/
> > > > > > > canned-wks'
> > > > > > > > > % l
> > > > > > > > > > > for l in '${BBPATH}:${COREBASE}'.split(':'))}"
> > > > > > > > > > > -WKS_FULL_PATH = "${@wks_search('${WKS_FILES}'.
> split(),
> > > > > > > > > > > '${WKS_SEARCH_PATH}') or ''}"
> > > > > > > > > > > -
> > > > > > > > > > > -def wks_search(files, search_path):
> > > > > > > > > > > -    for f in files:
> > > > > > > > > > > -        if os.path.isabs(f):
> > > > > > > > > > > -            if os.path.exists(f):
> > > > > > > > > > > -                return f
> > > > > > > > > > > -        else:
> > > > > > > > > > > -            searched = bb.utils.which(search_path, f)
> > > > > > > > > > > -            if searched:
> > > > > > > > > > > -                return searched
> > > > > > > > > > > -
> > > > > > > > > > > -WIC_CREATE_EXTRA_ARGS ?= ""
> > > > > > > > > > > -
> > > > > > > > > > > -IMAGE_CMD_wic () {
> > > > > > > > > > > -       out="${IMGDEPLOYDIR}/${IMAGE_NAME}"
> > > > > > > > > > > -       wks="${WKS_FULL_PATH}"
> > > > > > > > > > > -       if [ -z "$wks" ]; then
> > > > > > > > > > > -               bbfatal "No kickstart files from
> WKS_FILES
> > > were
> > > > > > > found:
> > > > > > > > > > > ${WKS_FILES}. Please set WKS_FILE or WKS_FILES
> > > appropriately."
> > > > > > > > > > > -       fi
> > > > > > > > > > > -
> > > > > > > > > > > -       BUILDDIR="${TOPDIR}" wic create "$wks" --vars
> > > > > > > > > > > "${STAGING_DIR}/${MACHINE}/imgdata/" -e
> > > "${IMAGE_BASENAME}" -o
> > > > > > > "$out/"
> > > > > > > > > > > ${WIC_CREATE_EXTRA_ARGS}
> > > > > > > > > > > -       mv "$out/$(basename "${wks%.wks}")"*.direct
> > > > > > > > > > > "$out${IMAGE_NAME_SUFFIX}.wic"
> > > > > > > > > > > -       rm -rf "$out/"
> > > > > > > > > > > -}
> > > > > > > > > > > -IMAGE_CMD_wic[vardepsexclude] = "WKS_FULL_PATH
> WKS_FILES"
> > > > > > > > > > > -
> > > > > > > > > > > -# Rebuild when the wks file or vars in WICVARS change
> > > > > > > > > > > -USING_WIC = "${@bb.utils.contains_any('
> IMAGE_FSTYPES',
> > > 'wic
> > > > > ' + '
> > > > > > > > > > > '.join('wic.%s' % c for c in
> '${CONVERSIONTYPES}'.split()),
> > > > > '1',
> > > > > > > '',
> > > > > > > > > d)}"
> > > > > > > > > > > -WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' %
> > > > > > > > > > > os.path.exists('${WKS_FULL_PATH}') if '${USING_WIC}'
> else
> > > ''}"
> > > > > > > > > > > -do_image_wic[file-checksums] += "${WKS_FILE_CHECKSUM}"
> > > > > > > > > > > -do_image_wic[depends] += "wic-tools:do_build"
> > > > > > > > > > > -
> > > > > > > > > > > -python () {
> > > > > > > > > > > -    if d.getVar('USING_WIC') and 'do_bootimg' in d:
> > > > > > > > > > > -        bb.build.addtask('do_image_wic', '',
> > > 'do_bootimg', d)
> > > > > > > > > > > -}
> > > > > > > > > > > -
> > > > > > > > > > > -python do_write_wks_template () {
> > > > > > > > > > > -    """Write out expanded template contents to
> > > > > WKS_FULL_PATH."""
> > > > > > > > > > > -    import re
> > > > > > > > > > > -
> > > > > > > > > > > -    template_body = d.getVar('_WKS_TEMPLATE')
> > > > > > > > > > > -
> > > > > > > > > > > -    # Remove any remnant variable references left
> behind
> > > by
> > > > > the
> > > > > > > > > expansion
> > > > > > > > > > > -    # due to undefined variables
> > > > > > > > > > > -    expand_var_regexp = re.compile(r"\${[^{}@\n\t
> :]+}")
> > > > > > > > > > > -    while True:
> > > > > > > > > > > -        new_body = re.sub(expand_var_regexp, '',
> > > > > template_body)
> > > > > > > > > > > -        if new_body == template_body:
> > > > > > > > > > > -            break
> > > > > > > > > > > -        else:
> > > > > > > > > > > -            template_body = new_body
> > > > > > > > > > > -
> > > > > > > > > > > -    wks_file = d.getVar('WKS_FULL_PATH')
> > > > > > > > > > > -    with open(wks_file, 'w') as f:
> > > > > > > > > > > -        f.write(template_body)
> > > > > > > > > > > -}
> > > > > > > > > > > -
> > > > > > > > > > > -python () {
> > > > > > > > > > > -    if d.getVar('USING_WIC'):
> > > > > > > > > > > -        wks_file_u = d.getVar('WKS_FULL_PATH', False)
> > > > > > > > > > > -        wks_file = d.expand(wks_file_u)
> > > > > > > > > > > -        base, ext = os.path.splitext(wks_file)
> > > > > > > > > > > -        if ext == '.in' and os.path.exists(wks_file):
> > > > > > > > > > > -            wks_out_file = os.path.join(d.getVar('
> > > WORKDIR'),
> > > > > > > > > > > os.path.basename(base))
> > > > > > > > > > > -            d.setVar('WKS_FULL_PATH', wks_out_file)
> > > > > > > > > > > -            d.setVar('WKS_TEMPLATE_PATH', wks_file_u)
> > > > > > > > > > > -            d.setVar('WKS_FILE_CHECKSUM',
> > > > > > > '${WKS_TEMPLATE_PATH}:True')
> > > > > > > > > > > -
> > > > > > > > > > > -            # We need to re-parse each time the file
> > > changes,
> > > > > and
> > > > > > > > > bitbake
> > > > > > > > > > > -            # needs to be told about that explicitly.
> > > > > > > > > > > -            bb.parse.mark_dependency(d, wks_file)
> > > > > > > > > > > -
> > > > > > > > > > > -            try:
> > > > > > > > > > > -                with open(wks_file, 'r') as f:
> > > > > > > > > > > -                    body = f.read()
> > > > > > > > > > > -            except (IOError, OSError) as exc:
> > > > > > > > > > > -                pass
> > > > > > > > > > > -            else:
> > > > > > > > > > > -                # Previously, I used expandWithRefs to
> > > get the
> > > > > > > > > dependency
> > > > > > > > > > > list
> > > > > > > > > > > -                # and add it to WICVARS, but there's
> no
> > > point
> > > > > > > > > re-parsing
> > > > > > > > > > > the
> > > > > > > > > > > -                # file in process_wks_template as
> well, so
> > > > > just
> > > > > > > put
> > > > > > > > > it in
> > > > > > > > > > > -                # a variable and let the metadata deal
> > > with
> > > > > the
> > > > > > > deps.
> > > > > > > > > > > -                d.setVar('_WKS_TEMPLATE', body)
> > > > > > > > > > > -                bb.build.addtask('do_write_
> wks_template',
> > > > > > > > > > > 'do_image_wic', None, d)
> > > > > > > > > > > -}
> > > > > > > > > > > -
> > > > > > > > > > >  EXTRA_IMAGECMD = ""
> > > > > > > > > > >
> > > > > > > > > > >  inherit siteinfo
> > > > > > > > > > > @@ -380,7 +289,3 @@ IMAGE_EXTENSION_live = "hddimg iso"
> > > > > > > > > > >  # The IMAGE_TYPES_MASKED variable will be used to
> mask out
> > > > > from
> > > > > > > the
> > > > > > > > > > > IMAGE_FSTYPES,
> > > > > > > > > > >  # images that will not be built at do_rootfs time:
> vmdk,
> > > vdi,
> > > > > > > qcow2,
> > > > > > > > > > > hdddirect, hddimg, iso, etc.
> > > > > > > > > > >  IMAGE_TYPES_MASKED ?= ""
> > > > > > > > > > > -
> > > > > > > > > > > -# The WICVARS variable is used to define list of
> bitbake
> > > > > variables
> > > > > > > > > used
> > > > > > > > > > > in wic code
> > > > > > > > > > > -# variables from this list is written to <image>.env
> file
> > > > > > > > > > > -WICVARS ?= "BBLAYERS IMGDEPLOYDIR DEPLOY_DIR_IMAGE
> > > FAKEROOTCMD
> > > > > > > HDDDIR
> > > > > > > > > > > IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME
> > > IMAGE_ROOTFS
> > > > > > > > > > > INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH
> > > > > RECIPE_SYSROOT_NATIVE
> > > > > > > > > > > ROOTFS_SIZE STAGING_DATADIR STAGING_DIR STAGING_LIBDIR
> > > > > TARGET_SYS"
> > > > > > > > > > > --
> > > > > > > > > > > 2.1.4
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > > _______________________________________________
> > > > > > > > > > > Openembedded-core mailing list
> > > > > > > > > > > Openembedded-core@lists.openembedded.org
> > > > > > > > > > > http://lists.openembedded.org/
> > > mailman/listinfo/openembedded-
> > > > > core
> > > > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > --
> > > > > > > > > Regards,
> > > > > > > > > Ed
> > > > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > --
> > > > > > > Regards,
> > > > > > > Ed
> > > > > > >
> > > > >
> > > > > --
> > > > > --
> > > > > Regards,
> > > > > Ed
> > > > >
> > >
> > > --
> > > --
> > > Regards,
> > > Ed
> > >
>
> --
> --
> Regards,
> Ed
>

[-- Attachment #2: Type: text/html, Size: 44935 bytes --]