[PATCH 0/6][PULL] Hob: Bug fixes

[PATCH 0/6][PULL] Hob: Bug fixes

[Dongxiao Xu]

Hi Richard,

This pull request contains several Hob bug fixes, please help to review and pull.

Thanks,
Dongxiao

The following changes since commit d49db15badb77855cef855ee73430fcbc16b6916:

  Hob: a minor fix on pmake (2012-04-05 18:47:37 +0100)

are available in the git repository at:
  git://git.pokylinux.org/poky-contrib dxu4/hob-bugfix
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/hob-bugfix

Dongxiao Xu (6):
  Hob: Change base image to "Create your own image" if customized
  data_smart: Improve the calculation of config hash
  Hob: Fix the pattern patch for image name
  Hob: Fix tooltips in image configuration page
  Hob: Fix tooltips in Settings dialog
  Hob: Add an extra 50M space if zypper is selected

 lib/bb/data_smart.py                       |   10 +++++--
 lib/bb/ui/crumbs/builder.py                |    3 ++
 lib/bb/ui/crumbs/hig.py                    |   38 ++++++++++++++-------------
 lib/bb/ui/crumbs/imageconfigurationpage.py |   19 ++++++++------
 lib/bb/ui/crumbs/imagedetailspage.py       |    3 +-
 lib/bb/ui/crumbs/packageselectionpage.py   |    8 ++++++
 lib/bb/ui/crumbs/recipeselectionpage.py    |    7 ++++-
 7 files changed, 56 insertions(+), 32 deletions(-)

-- 
1.7.4.1

[PATCH 1/6] Hob: Change base image to "Create your own image" if customized

[Dongxiao Xu]

Once user did customization to his base image, we change the base image
to be "Create your own image" to avoid some issues caused by the
relationship between base image and its default recipes and packages.

This fixes [YOCTO #2211]

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 lib/bb/ui/crumbs/builder.py                |    3 +++
 lib/bb/ui/crumbs/imageconfigurationpage.py |    2 ++
 lib/bb/ui/crumbs/packageselectionpage.py   |    6 ++++++
 lib/bb/ui/crumbs/recipeselectionpage.py    |    5 +++++
 4 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/lib/bb/ui/crumbs/builder.py b/lib/bb/ui/crumbs/builder.py
index a3410d4..bc7e610 100755
--- a/lib/bb/ui/crumbs/builder.py
+++ b/lib/bb/ui/crumbs/builder.py
@@ -272,6 +272,9 @@ class Builder(gtk.Window):
         self.recipe_model = recipe_model
         self.package_model = package_model
 
+        # Indicate whether user has customized the image
+        self.customized = False
+
         # create visual elements
         self.create_visual_elements()
 
diff --git a/lib/bb/ui/crumbs/imageconfigurationpage.py b/lib/bb/ui/crumbs/imageconfigurationpage.py
index d8cb752..5c2539d 100644
--- a/lib/bb/ui/crumbs/imageconfigurationpage.py
+++ b/lib/bb/ui/crumbs/imageconfigurationpage.py
@@ -291,6 +291,8 @@ class ImageConfigurationPage (HobPage):
         if not selected_image:
             return
 
+        self.builder.customized = False
+
         selected_recipes = []
 
         image_path = self.builder.recipe_model.pn_path[selected_image]
diff --git a/lib/bb/ui/crumbs/packageselectionpage.py b/lib/bb/ui/crumbs/packageselectionpage.py
index 7926636..dd0df25 100755
--- a/lib/bb/ui/crumbs/packageselectionpage.py
+++ b/lib/bb/ui/crumbs/packageselectionpage.py
@@ -90,6 +90,7 @@ class PackageSelectionPage (HobPage):
         super(PackageSelectionPage, self).__init__(builder, "Packages")
 
         # set invisiable members
+        self.recipe_model = self.builder.recipe_model
         self.package_model = self.builder.package_model
 
         # create visual elements
@@ -183,6 +184,11 @@ class PackageSelectionPage (HobPage):
             self.package_model.exclude_item(item_path=path)
 
         self.refresh_selection()
+        if not self.builder.customized:
+            self.builder.customized = True
+            self.builder.configuration.selected_image = self.recipe_model.__dummy_image__
+            self.builder.rcppkglist_populated()
+
         self.builder.window_sensitive(True)
 
     def table_toggled_cb(self, table, cell, view_path, toggled_columnid, view_tree):
diff --git a/lib/bb/ui/crumbs/recipeselectionpage.py b/lib/bb/ui/crumbs/recipeselectionpage.py
index e4616a8..2d438d5 100755
--- a/lib/bb/ui/crumbs/recipeselectionpage.py
+++ b/lib/bb/ui/crumbs/recipeselectionpage.py
@@ -187,6 +187,11 @@ class RecipeSelectionPage (HobPage):
             self.recipe_model.exclude_item(item_path=path)
 
         self.refresh_selection()
+        if not self.builder.customized:
+            self.builder.customized = True
+            self.builder.configuration.selected_image = self.recipe_model.__dummy_image__
+            self.builder.rcppkglist_populated()
+
         self.builder.window_sensitive(True)
 
     def table_toggled_cb(self, table, cell, view_path, toggled_columnid, view_tree):
-- 
1.7.4.1

[PATCH 2/6] data_smart: Improve the calculation of config hash

[Dongxiao Xu]

The order of keys are not sensitive for config hash, so we need to
identify its order while calculating the md5 value.

While for certain values, order is also not sensitive (for example,
BBINCLUDED), we also need to identify its order while calculating md5
value.

This could fix the problem that Martin Jansa reported in the mailing
list:

http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 lib/bb/data_smart.py |   10 +++++++---
 1 files changed, 7 insertions(+), 3 deletions(-)

diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
index 2c200db..cc61a03 100644
--- a/lib/bb/data_smart.py
+++ b/lib/bb/data_smart.py
@@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
         self.delVar(var)
 
     def get_hash(self):
-        data = ""
+        data = {}
         config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
+        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
         keys = set(key for key in iter(self) if not key.startswith("__"))
         for key in keys:
             if key in config_whitelist:
                 continue
             value = self.getVar(key, False) or ""
-            data = data + key + ': ' + str(value) + '\n'
+            if key in config_sort:
+                value = " ".join(sorted(value.split()))
+            data.update({key:value})
 
-        return hashlib.md5(data).hexdigest()
+        data_str = str([(k, data[k]) for k in sorted(data.keys())])
+        return hashlib.md5(data_str).hexdigest()
-- 
1.7.4.1

[PATCH 3/6] Hob: Fix the pattern patch for image name

[Dongxiao Xu]

Sometimes, users may open an image that is not built by Hob, therefore
its image name is not started with "hob-image-". This commit sets a
looser rule for runnable image matching.

This fixes [YOCTO #2240]

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 lib/bb/ui/crumbs/imagedetailspage.py |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/lib/bb/ui/crumbs/imagedetailspage.py b/lib/bb/ui/crumbs/imagedetailspage.py
index b685d11..277f9d4 100755
--- a/lib/bb/ui/crumbs/imagedetailspage.py
+++ b/lib/bb/ui/crumbs/imagedetailspage.py
@@ -272,8 +272,7 @@ class ImageDetailsPage (HobPage):
     def test_mach_runnable(self, image_name):
         mach_runnable = False
         for t in self.builder.parameters.runnable_machine_patterns:
-            mach_string = image_name.strip(self.builder.hob_image + '-')
-            if mach_string.startswith(t):
+            if t in image_name:
                 mach_runnable = True
                 break
         return mach_runnable
-- 
1.7.4.1

[PATCH 4/6] Hob: Fix tooltips in image configuration page

[Dongxiao Xu]

This fixes [YOCTO #2228]

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 lib/bb/ui/crumbs/imageconfigurationpage.py |   17 +++++++++--------
 1 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/lib/bb/ui/crumbs/imageconfigurationpage.py b/lib/bb/ui/crumbs/imageconfigurationpage.py
index 5c2539d..9013e1e 100644
--- a/lib/bb/ui/crumbs/imageconfigurationpage.py
+++ b/lib/bb/ui/crumbs/imageconfigurationpage.py
@@ -49,19 +49,19 @@ class ImageConfigurationPage (HobPage):
             "Templates",
             hic.ICON_TEMPLATES_DISPLAY_FILE,
             hic.ICON_TEMPLATES_HOVER_FILE,
-            "Load a hob building template saved before",
+            "Load a previously saved template",
             self.template_button_clicked_cb)
         my_images_button = self.append_toolbar_button(self.toolbar,
-            "My images",
+            "Images",
             hic.ICON_IMAGES_DISPLAY_FILE,
             hic.ICON_IMAGES_HOVER_FILE,
-            "Open images built out previously for running or deployment",
+            "Open previously built images",
             self.my_images_button_clicked_cb)
         settings_button = self.append_toolbar_button(self.toolbar,
             "Settings",
             hic.ICON_SETTINGS_DISPLAY_FILE,
             hic.ICON_SETTINGS_HOVER_FILE,
-            "Other advanced settings for build",
+            "View additional build settings",
             self.settings_button_clicked_cb)
 
         self.config_top_button = self.add_onto_top_bar(self.toolbar)
@@ -138,7 +138,7 @@ class ImageConfigurationPage (HobPage):
 
         self.machine_title_desc = gtk.Label()
         self.machine_title_desc.set_alignment(0.0, 0.5)
-        mark = ("<span %s>This is the profile of the target machine for which you"
+        mark = ("<span %s>Your selection is the profile of the target machine for which you"
         " are building the image.\n</span>") % (self.span_tag('medium'))
         self.machine_title_desc.set_markup(mark)
 
@@ -155,8 +155,8 @@ class ImageConfigurationPage (HobPage):
         markup = "Layers are a powerful mechanism to extend the Yocto Project "
         markup += "with your own functionality.\n"
         markup += "For more on layers, check the <a href=\""
-        markup += "http://www.yoctoproject.org/docs/current/poky-ref-manual/"
-        markup += "poky-ref-manual.html#usingpoky-changes-layers\">reference manual</a>."
+        markup += "http://www.yoctoproject.org/docs/current/dev-manual/"
+        markup += "dev-manual.html#understanding-and-using-layers\">reference manual</a>."
         self.layer_info_icon = HobInfoButton(markup, self.get_parent())
 
         self.progress_box = gtk.HBox(False, 6)
@@ -235,7 +235,7 @@ class ImageConfigurationPage (HobPage):
         # create button "Build image"
         just_bake_button = HobButton("Build image")
         just_bake_button.set_size_request(205, 49)
-        just_bake_button.set_tooltip_text("Build image to get your target image")
+        just_bake_button.set_tooltip_text("Build target image")
         just_bake_button.connect("clicked", self.just_bake_button_clicked_cb)
         button_box.pack_end(just_bake_button, expand=False, fill=False)
 
@@ -245,6 +245,7 @@ class ImageConfigurationPage (HobPage):
         # create button "Build Packages"
         build_packages_button = HobAltButton("Build packages")
         build_packages_button.connect("clicked", self.build_packages_button_clicked_cb)
+        build_packages_button.set_tooltip_text("Build recipes into packages")
         button_box.pack_end(build_packages_button, expand=False, fill=False)
 
         return button_box
-- 
1.7.4.1

[PATCH 5/6] Hob: Fix tooltips in Settings dialog

[Dongxiao Xu]

This fixes [YOCTO #2229]

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 lib/bb/ui/crumbs/hig.py                 |   38 ++++++++++++++++--------------
 lib/bb/ui/crumbs/recipeselectionpage.py |    2 +-
 2 files changed, 21 insertions(+), 19 deletions(-)

diff --git a/lib/bb/ui/crumbs/hig.py b/lib/bb/ui/crumbs/hig.py
index 9adb281..016fb59 100644
--- a/lib/bb/ui/crumbs/hig.py
+++ b/lib/bb/ui/crumbs/hig.py
@@ -182,7 +182,7 @@ class AdvancedSettingDialog (CrumbsDialog):
                 check_hbox.pack_start(check_button, expand=False, fill=False)
         check_hbox.show_all()
 
-    def gen_pkgfmt_widget(self, curr_package_format, all_package_format, tooltip=""):
+    def gen_pkgfmt_widget(self, curr_package_format, all_package_format, tooltip_combo="", tooltip_extra=""):
         pkgfmt_hbox = gtk.HBox(False, 24)
 
         rootfs_vbox = gtk.VBox(False, 6)
@@ -195,8 +195,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         if curr_package_format:
             rootfs_format = curr_package_format.split()[0]
 
-        tooltip = "Package format that is used to generate rootfs"
-        rootfs_format_widget, rootfs_combo = self.gen_combo_widget(rootfs_format, all_package_format, tooltip)
+        rootfs_format_widget, rootfs_combo = self.gen_combo_widget(rootfs_format, all_package_format, tooltip_combo)
         rootfs_vbox.pack_start(rootfs_format_widget, expand=False, fill=False)
 
         extra_vbox = gtk.VBox(False, 6)
@@ -214,8 +213,7 @@ class AdvancedSettingDialog (CrumbsDialog):
                 check_button.set_active(is_active)
                 check_hbox.pack_start(check_button, expand=False, fill=False)
 
-        tooltip = "Extra package formats to build"
-        info = HobInfoButton(tooltip, self)
+        info = HobInfoButton(tooltip_extra, self)
         check_hbox.pack_end(info, expand=False, fill=False)
 
         rootfs_combo.connect("changed", self.rootfs_combo_changed_cb, all_package_format, check_hbox)
@@ -383,7 +381,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         table = gtk.Table(rows + 1, 10, True)
         advanced_vbox.pack_start(table, expand=False, fill=False)
 
-        tooltip = "Select image file system types that will be used."
+        tooltip = "Image file system types you want."
         info = HobInfoButton(tooltip, self)
         label = self.gen_label_widget("<span weight=\"bold\">Select image types:</span>")
         table.attach(label, 0, 9, 0, 1)
@@ -411,16 +409,16 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">Packaging format:</span>")
-        tooltip = "Select package formats that will be used. "
-        tooltip += "The first format will be used for final image"
-        pkgfmt_widget, self.rootfs_combo, self.check_hbox = self.gen_pkgfmt_widget(self.configuration.curr_package_format, self.all_package_formats, tooltip)
+        tooltip_combo = "Selects the package format used to generate rootfs."
+        tooltip_extra = "Selects extra package formats to build"
+        pkgfmt_widget, self.rootfs_combo, self.check_hbox = self.gen_pkgfmt_widget(self.configuration.curr_package_format, self.all_package_formats, tooltip_combo, tooltip_extra)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(pkgfmt_widget, expand=False, fill=False)
 
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">Image rootfs size: (MB)</span>")
-        tooltip = "Sets the size of your target image.\nThis is the basic size of your target image, unless your selected package size exceeds this value, or you set value to \"Image Extra Size\"."
+        tooltip = "Sets the basic size of your target image.\nThis is the basic size of your target image unless your selected package size exceeds this value or you select \'Image Extra Size\'."
         rootfs_size_widget, self.rootfs_size_spinner = self.gen_spinner_widget(int(self.configuration.image_rootfs_size*1.0/1024), 0, 1024, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(rootfs_size_widget, expand=False, fill=False)
@@ -428,7 +426,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">Image extra size: (MB)</span>")
-        tooltip = "Sets the extra free space of your target image.\nDefaultly, system will reserve 30% of your image size as your free space. If your image contains zypper, it will bring in 50MB more space. The maximum free space is 1024MB."
+        tooltip = "Sets the extra free space of your target image.\nBy default, the system reserves 30% of your image size as free space. If your image contains zypper, it brings in 50MB more space. The maximum free space is 1024MB."
         extra_size_widget, self.extra_size_spinner = self.gen_spinner_widget(int(self.configuration.image_extra_size*1.0/1024), 0, 1024, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(extra_size_widget, expand=False, fill=False)
@@ -448,7 +446,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         self.toolchain_checkbox.set_active(self.configuration.toolchain_build)
         sub_hbox.pack_start(self.toolchain_checkbox, expand=False, fill=False)
 
-        tooltip = "This is the Host platform you would like to run the toolchain"
+        tooltip = "Selects the Host platform for which you want to run the toolchain"
         sdk_machine_widget, self.sdk_machine_combo = self.gen_combo_widget(self.configuration.curr_sdk_machine, self.all_sdk_machines, tooltip)
         sub_hbox.pack_start(sdk_machine_widget, expand=False, fill=False)
 
@@ -461,7 +459,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">Select distro:</span>")
-        tooltip = "This is the Yocto distribution you would like to use"
+        tooltip = "Selects the Yocto Project distribution you want"
         distro_widget, self.distro_combo = self.gen_combo_widget(self.configuration.curr_distro, self.all_distros, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(distro_widget, expand=False, fill=False)
@@ -469,7 +467,9 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">BB number threads:</span>")
-        tooltip = "Sets the number of threads that bitbake tasks can run simultaneously"
+        tooltip = "Sets the number of threads that BitBake tasks can simultaneously run. See the <a href=\""
+        tooltip += "http://www.yoctoproject.org/docs/current/poky-ref-manual/"
+        tooltip += "poky-ref-manual.html#var-BB_NUMBER_THREADS\">Poky reference manual</a> for information"
         bbthread_widget, self.bb_spinner = self.gen_spinner_widget(self.configuration.bbthread, 1, self.max_threads, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(bbthread_widget, expand=False, fill=False)
@@ -477,14 +477,16 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">Parallel make:</span>")
-        tooltip = "Sets the make parallism, as known as 'make -j'"
+        tooltip = "Sets the maximum number of threads the host can use during the build. See the <a href=\""
+        tooltip += "http://www.yoctoproject.org/docs/current/poky-ref-manual/"
+        tooltip += "poky-ref-manual.html#var-PARALLEL_MAKE\">Poky reference manual</a> for information"
         pmake_widget, self.pmake_spinner = self.gen_spinner_widget(self.configuration.pmake, 1, self.max_threads, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(pmake_widget, expand=False, fill=False)
 
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
-        label = self.gen_label_widget("<span weight=\"bold\">Set download directory:</span>")
+        label = self.gen_label_widget("<span weight=\"bold\">Select download directory:</span>")
         tooltip = "Select a folder that caches the upstream project source code"
         dldir_widget, self.dldir_text = self.gen_entry_widget(self.configuration.dldir, self, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
@@ -501,7 +503,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=False, fill=False)
         label = self.gen_label_widget("<span weight=\"bold\">Select SSTATE mirror:</span>")
-        tooltip = "Select the prebuilt mirror that will fasten your build speed"
+        tooltip = "Select the pre-built mirror that will speed your build"
         sstatemirror_widget, self.sstatemirror_text = self.gen_entry_widget(self.configuration.sstatemirror, self, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(sstatemirror_widget, expand=False, fill=False)
@@ -577,7 +579,7 @@ class AdvancedSettingDialog (CrumbsDialog):
         sub_vbox = gtk.VBox(False, 6)
         advanced_vbox.pack_start(sub_vbox, expand=True, fill=True)
         label = self.gen_label_widget("<span weight=\"bold\">Add your own variables:</span>")
-        tooltip = "This is the key/value pair for your extra settings"
+        tooltip = "These are key/value pairs for your extra settings. Click \'Add\' and then directly edit the key and the value"
         setting_widget, self.setting_store = self.gen_editable_settings(self.configuration.extra_setting, tooltip)
         sub_vbox.pack_start(label, expand=False, fill=False)
         sub_vbox.pack_start(setting_widget, expand=True, fill=True)
diff --git a/lib/bb/ui/crumbs/recipeselectionpage.py b/lib/bb/ui/crumbs/recipeselectionpage.py
index 2d438d5..44c94bf 100755
--- a/lib/bb/ui/crumbs/recipeselectionpage.py
+++ b/lib/bb/ui/crumbs/recipeselectionpage.py
@@ -153,7 +153,7 @@ class RecipeSelectionPage (HobPage):
 
         self.build_packages_button = HobButton('Build packages')
         self.build_packages_button.set_size_request(205, 49)
-        self.build_packages_button.set_tooltip_text("Build packages for customization")
+        self.build_packages_button.set_tooltip_text("Build selected recipes into packages")
         self.build_packages_button.set_flags(gtk.CAN_DEFAULT)
         self.build_packages_button.grab_default()
         self.build_packages_button.connect("clicked", self.build_packages_clicked_cb)
-- 
1.7.4.1

[PATCH 6/6] Hob: Add an extra 50M space if zypper is selected

[Dongxiao Xu]

If zypper is selected, RPM packaging will add extra 50M free space to
the final image. We need to reflect it in package selection page.

Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
---
 lib/bb/ui/crumbs/packageselectionpage.py |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/lib/bb/ui/crumbs/packageselectionpage.py b/lib/bb/ui/crumbs/packageselectionpage.py
index dd0df25..91d3e57 100755
--- a/lib/bb/ui/crumbs/packageselectionpage.py
+++ b/lib/bb/ui/crumbs/packageselectionpage.py
@@ -171,6 +171,8 @@ class PackageSelectionPage (HobPage):
         image_extra_size = self.builder.configuration.image_extra_size * 1024 # image_extra_size is KB
         base_size = image_overhead_factor * selected_packages_size
         image_total_size = max(base_size, image_rootfs_size) + image_extra_size
+        if "zypper" in self.builder.configuration.selected_packages:
+            image_total_size += (51200 * 1024)
         image_total_size_str = HobPage._size_to_string(image_total_size)
 
         self.label.set_text("Packages included: %s\nSelected packages size: %s\nTotal image size: %s" %
-- 
1.7.4.1

Re: [PATCH 0/6][PULL] Hob: Bug fixes

[Richard Purdie]

On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> Hi Richard,
> 
> This pull request contains several Hob bug fixes, please help to review and pull.
> 
> Thanks,
> Dongxiao
> 
> The following changes since commit d49db15badb77855cef855ee73430fcbc16b6916:
> 
>   Hob: a minor fix on pmake (2012-04-05 18:47:37 +0100)
> 
> are available in the git repository at:
>   git://git.pokylinux.org/poky-contrib dxu4/hob-bugfix
>   http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/hob-bugfix
> 
> Dongxiao Xu (6):
>   Hob: Change base image to "Create your own image" if customized
>   data_smart: Improve the calculation of config hash
>   Hob: Fix the pattern patch for image name
>   Hob: Fix tooltips in image configuration page
>   Hob: Fix tooltips in Settings dialog
>   Hob: Add an extra 50M space if zypper is selected

I've merged these apart from the data_smart fix which I'll send a reply
about.

Cheers,

Richard

Re: [PATCH 2/6] data_smart: Improve the calculation of config hash

[Richard Purdie]

On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> The order of keys are not sensitive for config hash, so we need to
> identify its order while calculating the md5 value.
> 
> While for certain values, order is also not sensitive (for example,
> BBINCLUDED), we also need to identify its order while calculating md5
> value.
> 
> This could fix the problem that Martin Jansa reported in the mailing
> list:
> 
> http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html
> 
> Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> ---
>  lib/bb/data_smart.py |   10 +++++++---
>  1 files changed, 7 insertions(+), 3 deletions(-)
> 
> diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> index 2c200db..cc61a03 100644
> --- a/lib/bb/data_smart.py
> +++ b/lib/bb/data_smart.py
> @@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
>          self.delVar(var)
>  
>      def get_hash(self):
> -        data = ""
> +        data = {}
>          config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
> +        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
>          keys = set(key for key in iter(self) if not key.startswith("__"))
>          for key in keys:
>              if key in config_whitelist:
>                  continue
>              value = self.getVar(key, False) or ""
> -            data = data + key + ': ' + str(value) + '\n'
> +            if key in config_sort:
> +                value = " ".join(sorted(value.split()))
> +            data.update({key:value})
>  
> -        return hashlib.md5(data).hexdigest()
> +        data_str = str([(k, data[k]) for k in sorted(data.keys())])
> +        return hashlib.md5(data_str).hexdigest()


This and the corresponding change in bitbake.conf look rather worrying
to me. The order in BBINCLUDED is significant and if it changes we
should be reparsing.

Looking at the code, get_file_depends(d) is probably buggy due to the
use of set() which could reorder the data. We need to keep the data in
order there and this issue should then be resolved.

Lets fix the real bug.

Cheers,

Richard

Re: [PATCH 2/6] data_smart: Improve the calculation of config hash

[Xu, Dongxiao]

On Wed, 2012-04-11 at 00:00 +0100, Richard Purdie wrote:
> On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> > The order of keys are not sensitive for config hash, so we need to
> > identify its order while calculating the md5 value.
> > 
> > While for certain values, order is also not sensitive (for example,
> > BBINCLUDED), we also need to identify its order while calculating md5
> > value.
> > 
> > This could fix the problem that Martin Jansa reported in the mailing
> > list:
> > 
> > http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html
> > 
> > Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> > ---
> >  lib/bb/data_smart.py |   10 +++++++---
> >  1 files changed, 7 insertions(+), 3 deletions(-)
> > 
> > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> > index 2c200db..cc61a03 100644
> > --- a/lib/bb/data_smart.py
> > +++ b/lib/bb/data_smart.py
> > @@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
> >          self.delVar(var)
> >  
> >      def get_hash(self):
> > -        data = ""
> > +        data = {}
> >          config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
> > +        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
> >          keys = set(key for key in iter(self) if not key.startswith("__"))
> >          for key in keys:
> >              if key in config_whitelist:
> >                  continue
> >              value = self.getVar(key, False) or ""
> > -            data = data + key + ': ' + str(value) + '\n'
> > +            if key in config_sort:
> > +                value = " ".join(sorted(value.split()))
> > +            data.update({key:value})
> >  
> > -        return hashlib.md5(data).hexdigest()
> > +        data_str = str([(k, data[k]) for k in sorted(data.keys())])
> > +        return hashlib.md5(data_str).hexdigest()
> 
> 
> This and the corresponding change in bitbake.conf look rather worrying
> to me. The order in BBINCLUDED is significant and if it changes we
> should be reparsing.

Hi Richard,

Why do you say the order in BBINCLUDED is significant?

I saw the original code ignores the order when handling __depends and
__base_depends.

For example:

def mark_dependency(d, f):
    if f.startswith('./'):
        f = "%s/%s" % (os.getcwd(), f[2:]) 
    deps = d.getVar('__depends') or set() 
    deps.update([(f, cached_mtime(f))])
    d.setVar('__depends', deps)    

I think the get_file_depends(d) just follows the original logic.

Or do you mean the mark_dependency(d, f) is also buggy?

Thanks,
Dongxiao

> 
> Looking at the code, get_file_depends(d) is probably buggy due to the
> use of set() which could reorder the data. We need to keep the data in
> order there and this issue should then be resolved.
> 
> Lets fix the real bug.
> 
> Cheers,
> 
> Richard
> 
> 
> 

Re: [PATCH 2/6] data_smart: Improve the calculation of config hash

[Richard Purdie]

On Mon, 2012-04-16 at 15:15 +0800, Xu, Dongxiao wrote:
> On Wed, 2012-04-11 at 00:00 +0100, Richard Purdie wrote:
> > On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> > > The order of keys are not sensitive for config hash, so we need to
> > > identify its order while calculating the md5 value.
> > > 
> > > While for certain values, order is also not sensitive (for example,
> > > BBINCLUDED), we also need to identify its order while calculating md5
> > > value.
> > > 
> > > This could fix the problem that Martin Jansa reported in the mailing
> > > list:
> > > 
> > > http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html
> > > 
> > > Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> > > ---
> > >  lib/bb/data_smart.py |   10 +++++++---
> > >  1 files changed, 7 insertions(+), 3 deletions(-)
> > > 
> > > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> > > index 2c200db..cc61a03 100644
> > > --- a/lib/bb/data_smart.py
> > > +++ b/lib/bb/data_smart.py
> > > @@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
> > >          self.delVar(var)
> > >  
> > >      def get_hash(self):
> > > -        data = ""
> > > +        data = {}
> > >          config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
> > > +        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
> > >          keys = set(key for key in iter(self) if not key.startswith("__"))
> > >          for key in keys:
> > >              if key in config_whitelist:
> > >                  continue
> > >              value = self.getVar(key, False) or ""
> > > -            data = data + key + ': ' + str(value) + '\n'
> > > +            if key in config_sort:
> > > +                value = " ".join(sorted(value.split()))
> > > +            data.update({key:value})
> > >  
> > > -        return hashlib.md5(data).hexdigest()
> > > +        data_str = str([(k, data[k]) for k in sorted(data.keys())])
> > > +        return hashlib.md5(data_str).hexdigest()
> > 
> > 
> > This and the corresponding change in bitbake.conf look rather worrying
> > to me. The order in BBINCLUDED is significant and if it changes we
> > should be reparsing.
> 
> Hi Richard,
> 
> Why do you say the order in BBINCLUDED is significant?
> 
> I saw the original code ignores the order when handling __depends and
> __base_depends.
> 
> For example:
> 
> def mark_dependency(d, f):
>     if f.startswith('./'):
>         f = "%s/%s" % (os.getcwd(), f[2:]) 
>     deps = d.getVar('__depends') or set() 
>     deps.update([(f, cached_mtime(f))])
>     d.setVar('__depends', deps)    
> 
> I think the get_file_depends(d) just follows the original logic.
> 
> Or do you mean the mark_dependency(d, f) is also buggy?

In that case I think the original code is also buggy.

Cheers,

Richard

Re: [PATCH 2/6] data_smart: Improve the calculation of config hash

[Xu, Dongxiao]

On Mon, 2012-04-16 at 09:28 +0100, Richard Purdie wrote:
> On Mon, 2012-04-16 at 15:15 +0800, Xu, Dongxiao wrote:
> > On Wed, 2012-04-11 at 00:00 +0100, Richard Purdie wrote:
> > > On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> > > > The order of keys are not sensitive for config hash, so we need to
> > > > identify its order while calculating the md5 value.
> > > > 
> > > > While for certain values, order is also not sensitive (for example,
> > > > BBINCLUDED), we also need to identify its order while calculating md5
> > > > value.
> > > > 
> > > > This could fix the problem that Martin Jansa reported in the mailing
> > > > list:
> > > > 
> > > > http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html
> > > > 
> > > > Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> > > > ---
> > > >  lib/bb/data_smart.py |   10 +++++++---
> > > >  1 files changed, 7 insertions(+), 3 deletions(-)
> > > > 
> > > > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> > > > index 2c200db..cc61a03 100644
> > > > --- a/lib/bb/data_smart.py
> > > > +++ b/lib/bb/data_smart.py
> > > > @@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
> > > >          self.delVar(var)
> > > >  
> > > >      def get_hash(self):
> > > > -        data = ""
> > > > +        data = {}
> > > >          config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
> > > > +        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
> > > >          keys = set(key for key in iter(self) if not key.startswith("__"))
> > > >          for key in keys:
> > > >              if key in config_whitelist:
> > > >                  continue
> > > >              value = self.getVar(key, False) or ""
> > > > -            data = data + key + ': ' + str(value) + '\n'
> > > > +            if key in config_sort:
> > > > +                value = " ".join(sorted(value.split()))
> > > > +            data.update({key:value})
> > > >  
> > > > -        return hashlib.md5(data).hexdigest()
> > > > +        data_str = str([(k, data[k]) for k in sorted(data.keys())])
> > > > +        return hashlib.md5(data_str).hexdigest()
> > > 
> > > 
> > > This and the corresponding change in bitbake.conf look rather worrying
> > > to me. The order in BBINCLUDED is significant and if it changes we
> > > should be reparsing.
> > 
> > Hi Richard,
> > 
> > Why do you say the order in BBINCLUDED is significant?
> > 
> > I saw the original code ignores the order when handling __depends and
> > __base_depends.
> > 
> > For example:
> > 
> > def mark_dependency(d, f):
> >     if f.startswith('./'):
> >         f = "%s/%s" % (os.getcwd(), f[2:]) 
> >     deps = d.getVar('__depends') or set() 
> >     deps.update([(f, cached_mtime(f))])
> >     d.setVar('__depends', deps)    
> > 
> > I think the get_file_depends(d) just follows the original logic.
> > 
> > Or do you mean the mark_dependency(d, f) is also buggy?
> 
> In that case I think the original code is also buggy.

Could you explain more why __depends should be ordered? I saw the
variable's definition is set() type and there is no code depends on the
order of "__depends".

Thanks,
Dongxiao

> 
> Cheers,
> 
> Richard
> 

Re: [PATCH 2/6] data_smart: Improve the calculation of config hash

[Richard Purdie]

On Mon, 2012-04-16 at 16:41 +0800, Xu, Dongxiao wrote:
> On Mon, 2012-04-16 at 09:28 +0100, Richard Purdie wrote:
> > On Mon, 2012-04-16 at 15:15 +0800, Xu, Dongxiao wrote:
> > > On Wed, 2012-04-11 at 00:00 +0100, Richard Purdie wrote:
> > > > On Mon, 2012-04-09 at 16:41 +0800, Dongxiao Xu wrote:
> > > > > The order of keys are not sensitive for config hash, so we need to
> > > > > identify its order while calculating the md5 value.
> > > > > 
> > > > > While for certain values, order is also not sensitive (for example,
> > > > > BBINCLUDED), we also need to identify its order while calculating md5
> > > > > value.
> > > > > 
> > > > > This could fix the problem that Martin Jansa reported in the mailing
> > > > > list:
> > > > > 
> > > > > http://lists.linuxtogo.org/pipermail/bitbake-devel/2012-March/002122.html
> > > > > 
> > > > > Signed-off-by: Dongxiao Xu <dongxiao.xu@intel.com>
> > > > > ---
> > > > >  lib/bb/data_smart.py |   10 +++++++---
> > > > >  1 files changed, 7 insertions(+), 3 deletions(-)
> > > > > 
> > > > > diff --git a/lib/bb/data_smart.py b/lib/bb/data_smart.py
> > > > > index 2c200db..cc61a03 100644
> > > > > --- a/lib/bb/data_smart.py
> > > > > +++ b/lib/bb/data_smart.py
> > > > > @@ -462,13 +462,17 @@ class DataSmart(MutableMapping):
> > > > >          self.delVar(var)
> > > > >  
> > > > >      def get_hash(self):
> > > > > -        data = ""
> > > > > +        data = {}
> > > > >          config_whitelist = set((self.getVar("BB_HASHCONFIG_WHITELIST", True) or "").split())
> > > > > +        config_sort = set((self.getVar("BB_HASHCONFIG_SORT", True) or "").split())
> > > > >          keys = set(key for key in iter(self) if not key.startswith("__"))
> > > > >          for key in keys:
> > > > >              if key in config_whitelist:
> > > > >                  continue
> > > > >              value = self.getVar(key, False) or ""
> > > > > -            data = data + key + ': ' + str(value) + '\n'
> > > > > +            if key in config_sort:
> > > > > +                value = " ".join(sorted(value.split()))
> > > > > +            data.update({key:value})
> > > > >  
> > > > > -        return hashlib.md5(data).hexdigest()
> > > > > +        data_str = str([(k, data[k]) for k in sorted(data.keys())])
> > > > > +        return hashlib.md5(data_str).hexdigest()
> > > > 
> > > > 
> > > > This and the corresponding change in bitbake.conf look rather worrying
> > > > to me. The order in BBINCLUDED is significant and if it changes we
> > > > should be reparsing.
> > > 
> > > Hi Richard,
> > > 
> > > Why do you say the order in BBINCLUDED is significant?
> > > 
> > > I saw the original code ignores the order when handling __depends and
> > > __base_depends.
> > > 
> > > For example:
> > > 
> > > def mark_dependency(d, f):
> > >     if f.startswith('./'):
> > >         f = "%s/%s" % (os.getcwd(), f[2:]) 
> > >     deps = d.getVar('__depends') or set() 
> > >     deps.update([(f, cached_mtime(f))])
> > >     d.setVar('__depends', deps)    
> > > 
> > > I think the get_file_depends(d) just follows the original logic.
> > > 
> > > Or do you mean the mark_dependency(d, f) is also buggy?
> > 
> > In that case I think the original code is also buggy.
> 
> Could you explain more why __depends should be ordered? I saw the
> variable's definition is set() type and there is no code depends on the
> order of "__depends".

Well, your new code does!

Bitbake does care deeply about the order things are included in so I
believe this variable should reflect that.

Cheers,

Richard