[PATCH 1/3] scripts/machine-summary: improve layer_path

[PATCH 1/3] scripts/machine-summary: improve layer_path

[Ross Burton]

As this can get called lots but the data doesn't change, cache
the responses.  Also return a pathlib.Path, as this is 2022.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 scripts/machine-summary.py | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/scripts/machine-summary.py b/scripts/machine-summary.py
index 4888a007..c07c2d15 100755
--- a/scripts/machine-summary.py
+++ b/scripts/machine-summary.py
@@ -30,16 +30,23 @@ def safe_patches(patches):
             return False
     return True
 
-def layer_path(layername, d):
+def layer_path(layername: str, d) -> pathlib.Path:
     """
     Return the path to the specified layer, or None if the layer isn't present.
     """
-    import re
+    if not hasattr(layer_path, "cache"):
+        # Don't use functools.lru_cache as we don't want d changing to invalidate the cache
+        layer_path.cache = {}
+
+    if layername in layer_path.cache:
+        return layer_path.cache[layername]
+
     bbpath = d.getVar("BBPATH").split(":")
     pattern = d.getVar('BBFILE_PATTERN_' + layername)
     for path in reversed(sorted(bbpath)):
         if re.match(pattern, path + "/"):
-            return path
+            layer_path.cache[layername] = pathlib.Path(path)
+            return layer_path.cache[layername]
     return None
 
 def extract_patch_info(src_uri, d):
-- 
2.25.1

[PATCH 2/3] scripts/machine-summary: link patches to cgit

[Ross Burton]

Add links from the patch names to the patches in cgit for ease of
inspection.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 scripts/machine-summary.py        | 20 +++++++++++++++++---
 scripts/report-details.html.jinja |  7 +++++--
 2 files changed, 22 insertions(+), 5 deletions(-)

diff --git a/scripts/machine-summary.py b/scripts/machine-summary.py
index c07c2d15..47d8ff34 100755
--- a/scripts/machine-summary.py
+++ b/scripts/machine-summary.py
@@ -49,6 +49,19 @@ def layer_path(layername: str, d) -> pathlib.Path:
             return layer_path.cache[layername]
     return None
 
+def get_url_for_patch(layer: str, localpath: pathlib.Path, d) -> str:
+    relative = localpath.relative_to(layer_path(layer, d))
+
+    # TODO: use layerindexlib
+    # TODO: assumes default branch
+    if layer == "core":
+        return f"https://git.openembedded.org/openembedded-core/tree/meta/{relative}"
+    elif layer in ("meta-arm", "meta-arm-bsp", "arm-toolchain", "meta-atp", "meta-gem5"):
+        return f"https://git.yoctoproject.org/meta-arm/tree/{layer}/{relative}"
+    else:
+        print(f"WARNING: Don't know web URL for layer {layer}", file=sys.stderr)
+        return None
+
 def extract_patch_info(src_uri, d):
     """
     Parse the specified patch entry from a SRC_URI and return (base name, layer name, status) tuple
@@ -56,9 +69,10 @@ def extract_patch_info(src_uri, d):
     import bb.fetch, bb.utils
 
     info = {}
-    localpath = bb.fetch.decodeurl(src_uri)[2]
-    info["name"] = os.path.basename(localpath)
-    info["layer"] = bb.utils.get_file_layer(localpath, d)
+    localpath = pathlib.Path(bb.fetch.decodeurl(src_uri)[2])
+    info["name"] = localpath.name
+    info["layer"] = bb.utils.get_file_layer(str(localpath), d)
+    info["url"] = get_url_for_patch(info["layer"], localpath, d)
 
     status = "Unknown"
     with open(localpath, errors="ignore") as f:
diff --git a/scripts/report-details.html.jinja b/scripts/report-details.html.jinja
index 19ede510..a656c261 100644
--- a/scripts/report-details.html.jinja
+++ b/scripts/report-details.html.jinja
@@ -47,8 +47,11 @@
     <tbody>
       {% for pinfo in data.patches %}
       <tr>
-        <!-- TODO links to cgit -->
-        <td>{{ pinfo.name }}</td>
+        <td>
+          {% if pinfo.url %}<a href="{{pinfo.url}}">{% endif %}
+          {{ pinfo.name }}
+          {% if pinfo.url %}</a>{% endif %}
+        </td>
         <td>{{ pinfo.layer }}</td>
         <!-- TODO: tooltip with full status? -->
         <td class="has-text-centered">{{ make_patch_tag(pinfo.status)}}</td>
-- 
2.25.1

[PATCH 3/3] CI: enable memory-resident bitbake

[Ross Burton]

Enable memory-resident bitbake by setting the server timeout to 60s.

For most builds this doesn't have an impact as bitbake is ran once, but
this reduces the time for the pending-upgrades report from 10 minutes to
3 minutes, as it doesn't need to constantly restart bitbake.

Signed-off-by: Ross Burton <ross.burton@arm.com>
---
 ci/base.yml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/ci/base.yml b/ci/base.yml
index 0af0b62d..795bd11f 100644
--- a/ci/base.yml
+++ b/ci/base.yml
@@ -26,6 +26,7 @@ env:
 
 local_conf_header:
   base: |
+    BB_SERVER_TIMEOUT = "60"
     CONF_VERSION = "2"
     PACKAGE_CLASSES = "package_ipk"
     LICENSE_FLAGS_WHITELIST += "armcompiler Arm-FVP-EULA"
-- 
2.25.1

Re: [PATCH 1/3] scripts/machine-summary: improve layer_path

[Jon Mason]

On Wed, 26 Jan 2022 19:12:51 +0000, Ross Burton wrote:
> As this can get called lots but the data doesn't change, cache
> the responses.  Also return a pathlib.Path, as this is 2022.

Applied, thanks!

[1/3] scripts/machine-summary: improve layer_path
      commit: b1e4cff75df54ab6d49c3b1a67e4fef806de571d
[2/3] scripts/machine-summary: link patches to cgit
      commit: 57f03491be7838279999a537384e0c9987e052d7
[3/3] CI: enable memory-resident bitbake
      commit: 828ad702acf6ea00907a9bdcd6537b32cc0ff78b

Best regards,
-- 
Jon Mason <jon.mason@arm.com>