[1.44 0/3] Merge request

[1.44 0/3] Merge request

[akuster]

Please consider these changes for 1.44.

The following changes since commit 6836184ef5220488a1127413c7d2e523fc37e2e9:

  lib/bb/utils.py: Preserve ownership of symlink (2020-03-06 17:25:10 +0000)

are available in the Git repository at:

  git://git.openembedded.org/bitbake-contrib stable/1.44-next
  http://cgit.openembedded.org/bitbake-contrib/log/?h=stable/1.44-next

Peter Kjellerstedt (3):
  knotty: Make the bb.command.CommandExit event terminate bitbake
  fetch2: Make fetcher_compare_revisions() work
  fetch2: Allow ${AUTOREV} to be used when BB_SRCREV_POLICY is "cache"

 lib/bb/fetch2/__init__.py | 51 +++++++++++++++++++++------------------
 lib/bb/ui/knotty.py       |  1 +
 2 files changed, 29 insertions(+), 23 deletions(-)

-- 
2.17.1

[1.44 1/3] knotty: Make the bb.command.CommandExit event terminate bitbake

[akuster]

From: Peter Kjellerstedt <pkj@axis.com>

This matches the other bb.command.Command* events and without it,
running `bitbake --revisions-changed` will hang indefinitely if there
are changed revisions.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 40520d229c8ea51ee9784184ab5d13a82dd1eb61)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 lib/bb/ui/knotty.py | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lib/bb/ui/knotty.py b/lib/bb/ui/knotty.py
index c6abb2a1..5402f7c9 100644
--- a/lib/bb/ui/knotty.py
+++ b/lib/bb/ui/knotty.py
@@ -591,6 +591,7 @@ def main(server, eventHandler, params, tf = TerminalFilter):
             if isinstance(event, bb.command.CommandExit):
                 if not return_value:
                     return_value = event.exitcode
+                main.shutdown = 2
                 continue
             if isinstance(event, (bb.command.CommandCompleted, bb.cooker.CookerExit)):
                 main.shutdown = 2
-- 
2.17.1

[1.44 2/3] fetch2: Make fetcher_compare_revisions() work

[akuster]

From: Peter Kjellerstedt <pkj@axis.com>

This seems to have been broken for a very long time. Now it also works
regardless of BB_SRCREV_POLICY.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ffd663a8e07e2e39e8ca2d2493f4f98037c5f9e4)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 lib/bb/fetch2/__init__.py | 33 ++++++++++++++-------------------
 1 file changed, 14 insertions(+), 19 deletions(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 4a781bde..14005492 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -484,17 +484,22 @@ def fetcher_init(d):
     Called to initialize the fetchers once the configuration data is known.
     Calls before this must not hit the cache.
     """
+
+    revs = bb.persist_data.persist('BB_URI_HEADREVS', d)
+    try:
+        # fetcher_init is called multiple times, so make sure we only save the
+        # revs the first time it is called.
+        if not bb.fetch2.saved_headrevs:
+            bb.fetch2.saved_headrevs = dict(revs)
+    except:
+        pass
+
     # When to drop SCM head revisions controlled by user policy
     srcrev_policy = d.getVar('BB_SRCREV_POLICY') or "clear"
     if srcrev_policy == "cache":
         logger.debug(1, "Keeping SRCREV cache due to cache policy of: %s", srcrev_policy)
     elif srcrev_policy == "clear":
         logger.debug(1, "Clearing SRCREV cache due to cache policy of: %s", srcrev_policy)
-        revs = bb.persist_data.persist('BB_URI_HEADREVS', d)
-        try:
-            bb.fetch2.saved_headrevs = revs.items()
-        except:
-            pass
         revs.clear()
     else:
         raise FetchError("Invalid SRCREV cache policy of: %s" % srcrev_policy)
@@ -513,22 +518,12 @@ def fetcher_parse_done():
 
 def fetcher_compare_revisions(d):
     """
-    Compare the revisions in the persistant cache with current values and
-    return true/false on whether they've changed.
+    Compare the revisions in the persistent cache with the saved values from
+    when bitbake was started and return true if they have changed.
     """
 
-    data = bb.persist_data.persist('BB_URI_HEADREVS', d).items()
-    data2 = bb.fetch2.saved_headrevs
-
-    changed = False
-    for key in data:
-        if key not in data2 or data2[key] != data[key]:
-            logger.debug(1, "%s changed", key)
-            changed = True
-            return True
-        else:
-            logger.debug(2, "%s did not change", key)
-    return False
+    headrevs = dict(bb.persist_data.persist('BB_URI_HEADREVS', d))
+    return headrevs != bb.fetch2.saved_headrevs
 
 def mirror_from_string(data):
     mirrors = (data or "").replace('\\n',' ').split()
-- 
2.17.1

[1.44 3/3] fetch2: Allow ${AUTOREV} to be used when BB_SRCREV_POLICY is "cache"

[akuster]

From: Peter Kjellerstedt <pkj@axis.com>

Mark any keys used to cache the srcrevs for a recipe as "dontcache" if
BB_DONT_CACHE is set for the recipe. Remove any such keys upon the
next bitbake run even if BB_SRCREV_POLICY is set to "cache". This will
make sure the srcrev is updated as expected if ${AUTOREV} is used.

Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit ba093a38539960e645e994a66ed7872a604c00a9)
Signed-off-by: Armin Kuster <akuster808@gmail.com>
---
 lib/bb/fetch2/__init__.py | 18 ++++++++++++++----
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/lib/bb/fetch2/__init__.py b/lib/bb/fetch2/__init__.py
index 14005492..397367f3 100644
--- a/lib/bb/fetch2/__init__.py
+++ b/lib/bb/fetch2/__init__.py
@@ -498,6 +498,11 @@ def fetcher_init(d):
     srcrev_policy = d.getVar('BB_SRCREV_POLICY') or "clear"
     if srcrev_policy == "cache":
         logger.debug(1, "Keeping SRCREV cache due to cache policy of: %s", srcrev_policy)
+        # We still need to remove keys that are marked with "dontcache".
+        for key in list(revs.keys()):
+            if key.startswith("dontcache-"):
+                logger.debug(1, "Removing SRCREV key: %s" % key)
+                revs.pop(key)
     elif srcrev_policy == "clear":
         logger.debug(1, "Clearing SRCREV cache due to cache policy of: %s", srcrev_policy)
         revs.clear()
@@ -717,9 +722,8 @@ def subprocess_setup():
     signal.signal(signal.SIGPIPE, signal.SIG_DFL)
 
 def get_autorev(d):
-    #  only not cache src rev in autorev case
-    if d.getVar('BB_SRCREV_POLICY') != "cache":
-        d.setVar('BB_DONT_CACHE', '1')
+    # Do not cache the srcrev in the autorev case
+    d.setVar('BB_DONT_CACHE', '1')
     return "AUTOINC"
 
 def get_srcrev(d, method_name='sortable_revision'):
@@ -1562,7 +1566,13 @@ class FetchMethod(object):
         return True, str(latest_rev)
 
     def generate_revision_key(self, ud, d, name):
-        return self._revision_key(ud, d, name)
+        key = self._revision_key(ud, d, name)
+        if d.getVar('BB_DONT_CACHE'):
+            # Mark the key so it can be removed on the next bitbake run even if
+            # BB_SRCREV_POLICY is "cache".
+            return "dontcache-%s" % key
+        else:
+            return key
 
     def latest_versionstring(self, ud, d):
         """
-- 
2.17.1

Re: [bitbake-devel] [1.44 3/3] fetch2: Allow ${AUTOREV} to be used when BB_SRCREV_POLICY is "cache"

[Richard Purdie]

On Mon, 2020-04-27 at 19:17 -0700, akuster wrote:
> From: Peter Kjellerstedt <pkj@axis.com>
> 
> Mark any keys used to cache the srcrevs for a recipe as "dontcache" if
> BB_DONT_CACHE is set for the recipe. Remove any such keys upon the
> next bitbake run even if BB_SRCREV_POLICY is set to "cache". This will
> make sure the srcrev is updated as expected if ${AUTOREV} is used.
> 
> Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> (cherry picked from commit ba093a38539960e645e994a66ed7872a604c00a9)
> Signed-off-by: Armin Kuster <akuster808@gmail.com>
> ---
>  lib/bb/fetch2/__init__.py | 18 ++++++++++++++----
>  1 file changed, 14 insertions(+), 4 deletions(-)

This was later reverted, so no:

http://git.yoctoproject.org/cgit.cgi/poky/commit/bitbake?id=12fc1c5637bca79a7a5735d9a8e819aefd0dd19b

Cheers,

Richard