[PATCH] bitbake: cooker: Rename __depends in all multiconfigs

[PATCH] bitbake: cooker: Rename __depends in all multiconfigs

[Joshua Watt]

From: Joshua Watt <Joshua.Watt@garmin.com>

The renaming of the __depends variable to __base_depends and file
watches needs to occurs for all multiconfigs, not just the base config.
Failing to do this for all multiconfigs will result in a huge increase
in the size of the parsing cache (about 5x for a single mulitconfig)
because all multiconfig caches will still depend on the base config
files. This will also seen a similar jump in the amount of time required
to load the parsing cache from memory, both because the cache is larger
and because of explosion of additional existence checks that must be
done for the base files.

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 bitbake/lib/bb/cooker.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index adc41014e64..78658a1683e 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -391,8 +391,9 @@ class BBCooker:
         if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
             self.disableDataTracking()
 
-        self.data.renameVar("__depends", "__base_depends")
-        self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher)
+        for mc in self.databuilder.mcdata.values():
+            mc.renameVar("__depends", "__base_depends")
+            self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher)
 
         self.baseconfig_valid = True
         self.parsecache_valid = False
-- 
2.21.0

[PATCH v2] bitbake: cooker: Rename __depends in all multiconfigs

[Joshua Watt]

The renaming of the __depends variable to __base_depends and file
watches needs to occurs for all multiconfigs, not just the base config.
Failing to do this for all multiconfigs will result in a huge increase
in the size of the parsing cache (about 5x for a single mulitconfig)
because all multiconfig caches will still depend on the base config
files. This will also seen a similar jump in the amount of time required
to load the parsing cache from memory, both because the cache is larger
and because of explosion of additional existence checks that must be
done for the base files.

[YOCTO #13359]

Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
---
 bitbake/lib/bb/cooker.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
index adc41014e6..78658a1683 100644
--- a/bitbake/lib/bb/cooker.py
+++ b/bitbake/lib/bb/cooker.py
@@ -391,8 +391,9 @@ class BBCooker:
         if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
             self.disableDataTracking()
 
-        self.data.renameVar("__depends", "__base_depends")
-        self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher)
+        for mc in self.databuilder.mcdata.values():
+            mc.renameVar("__depends", "__base_depends")
+            self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher)
 
         self.baseconfig_valid = True
         self.parsecache_valid = False
-- 
2.21.0

Re: [PATCH v2] bitbake: cooker: Rename __depends in all multiconfigs

[Joshua Watt]

Ping

On 5/22/19 8:35 AM, Joshua Watt wrote:
> The renaming of the __depends variable to __base_depends and file
> watches needs to occurs for all multiconfigs, not just the base config.
> Failing to do this for all multiconfigs will result in a huge increase
> in the size of the parsing cache (about 5x for a single mulitconfig)
> because all multiconfig caches will still depend on the base config
> files. This will also seen a similar jump in the amount of time required
> to load the parsing cache from memory, both because the cache is larger
> and because of explosion of additional existence checks that must be
> done for the base files.
>
> [YOCTO #13359]
>
> Signed-off-by: Joshua Watt <JPEWhacker@gmail.com>
> ---
>   bitbake/lib/bb/cooker.py | 5 +++--
>   1 file changed, 3 insertions(+), 2 deletions(-)
>
> diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py
> index adc41014e6..78658a1683 100644
> --- a/bitbake/lib/bb/cooker.py
> +++ b/bitbake/lib/bb/cooker.py
> @@ -391,8 +391,9 @@ class BBCooker:
>           if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
>               self.disableDataTracking()
>   
> -        self.data.renameVar("__depends", "__base_depends")
> -        self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher)
> +        for mc in self.databuilder.mcdata.values():
> +            mc.renameVar("__depends", "__base_depends")
> +            self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher)
>   
>           self.baseconfig_valid = True
>           self.parsecache_valid = False