openembedded-core.lists.openembedded.org archive mirror
 help / color / mirror / Atom feed
From: Richard Purdie <richard.purdie@linuxfoundation.org>
To: openembedded-core@lists.openembedded.org
Subject: [PATCH 3/7] reproducible: Move class function code into library
Date: Thu, 14 Oct 2021 13:10:21 +0100	[thread overview]
Message-ID: <20211014121025.2913401-3-richard.purdie@linuxfoundation.org> (raw)
In-Reply-To: <20211014121025.2913401-1-richard.purdie@linuxfoundation.org>

To try and avoid parse/memory overhead of functions within bitbake,
move the bulk of the reproducibility functions to the function library.

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
 meta/classes/base.bbclass               |  2 +-
 meta/classes/reproducible_build.bbclass | 37 ++-----------------------
 meta/lib/oe/reproducible.py             | 33 ++++++++++++++++++++++
 3 files changed, 36 insertions(+), 36 deletions(-)

diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 59fd46e5d43..bca3944ae70 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -12,7 +12,7 @@ inherit logging
 
 OE_EXTRA_IMPORTS ?= ""
 
-OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa ${OE_EXTRA_IMPORTS}"
+OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa oe.reproducible ${OE_EXTRA_IMPORTS}"
 OE_IMPORTS[type] = "list"
 
 PACKAGECONFIG_CONFARGS ??= ""
diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index 3f661794c61..0f45b782e5d 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -91,19 +91,8 @@ addtask do_deploy_source_date_epoch_setscene
 addtask do_deploy_source_date_epoch before do_configure after do_patch
 
 python create_source_date_epoch_stamp() {
-    import oe.reproducible
-
-    epochfile = d.getVar('SDE_FILE')
-    tmp_file = "%s.new" % epochfile
-
     source_date_epoch = oe.reproducible.get_source_date_epoch(d, d.getVar('S'))
-
-    bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
-    bb.utils.mkdirhier(d.getVar('SDE_DIR'))
-    with open(tmp_file, 'w') as f:
-        f.write(str(source_date_epoch))
-
-    os.rename(tmp_file, epochfile)
+    oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), d)
 }
 
 EPOCHTASK = "do_deploy_source_date_epoch"
@@ -112,29 +101,7 @@ EPOCHTASK = "do_deploy_source_date_epoch"
 do_unpack[postfuncs] += "create_source_date_epoch_stamp"
 
 def get_source_date_epoch_value(d):
-    epochfile = d.getVar('SDE_FILE')
-    cached, efile = d.getVar('__CACHED_SOURCE_DATE_EPOCH') or (None, None)
-    if cached and efile == epochfile:
-        return cached
-
-    if cached and epochfile != efile:
-        bb.debug(1, "Epoch file changed from %s to %s" % (efile, epochfile))
-
-    source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
-    try:
-        with open(epochfile, 'r') as f:
-            s = f.read()
-            try:
-                source_date_epoch = int(s)
-            except ValueError:
-                bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK" % s)
-                source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
-        bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
-    except FileNotFoundError:
-        bb.debug(1, "Cannot find %s. SOURCE_DATE_EPOCH will default to %d" % (epochfile, source_date_epoch))
-
-    d.setVar('__CACHED_SOURCE_DATE_EPOCH', (str(source_date_epoch), epochfile))
-    return str(source_date_epoch)
+    return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
 
 export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
 BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
diff --git a/meta/lib/oe/reproducible.py b/meta/lib/oe/reproducible.py
index 204b9bd734a..a5000574cfa 100644
--- a/meta/lib/oe/reproducible.py
+++ b/meta/lib/oe/reproducible.py
@@ -106,3 +106,36 @@ def get_source_date_epoch(d, sourcedir):
         fixed_source_date_epoch(d)       # Last resort
     )
 
+def epochfile_read(epochfile, d):
+    cached, efile = d.getVar('__CACHED_SOURCE_DATE_EPOCH') or (None, None)
+    if cached and efile == epochfile:
+        return cached
+
+    if cached and epochfile != efile:
+        bb.debug(1, "Epoch file changed from %s to %s" % (efile, epochfile))
+
+    source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
+    try:
+        with open(epochfile, 'r') as f:
+            s = f.read()
+            try:
+                source_date_epoch = int(s)
+            except ValueError:
+                bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK" % s)
+                source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
+        bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
+    except FileNotFoundError:
+        bb.debug(1, "Cannot find %s. SOURCE_DATE_EPOCH will default to %d" % (epochfile, source_date_epoch))
+
+    d.setVar('__CACHED_SOURCE_DATE_EPOCH', (str(source_date_epoch), epochfile))
+    return str(source_date_epoch)
+
+def epochfile_write(source_date_epoch, epochfile, d):
+
+    bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
+    bb.utils.mkdirhier(os.path.dirname(epochfile))
+
+    tmp_file = "%s.new" % epochfile
+    with open(tmp_file, 'w') as f:
+        f.write(str(source_date_epoch))
+    os.rename(tmp_file, epochfile)
-- 
2.32.0



  parent reply	other threads:[~2021-10-14 12:10 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
2021-10-14 12:10 ` [PATCH 2/7] reproducible_build: Drop obsolete sstate workaround Richard Purdie
2021-10-14 12:10 ` Richard Purdie [this message]
2021-10-14 12:10 ` [PATCH 4/7] reproducible: Move variable definitions to bitbake.conf Richard Purdie
2021-10-14 12:10 ` [PATCH 5/7] reproducible: Merge code into base.bbclass Richard Purdie
2021-10-14 12:10 ` [PATCH 6/7] python: Update now reproducibile builds are the default Richard Purdie
2021-10-14 12:10 ` [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable Richard Purdie
2021-10-14 12:28   ` [OE-core] " Bruce Ashfield
2021-10-14 12:31     ` Richard Purdie
2021-10-14 12:38       ` Bruce Ashfield
2021-10-14 13:45         ` Alexandre Belloni
2021-10-14 13:46           ` Bruce Ashfield
2021-10-14 15:26             ` Richard Purdie
2021-10-14 16:36 ` [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Khem Raj
2021-10-14 21:48   ` Richard Purdie
2021-10-14 23:22     ` Khem Raj

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20211014121025.2913401-3-richard.purdie@linuxfoundation.org \
    --to=richard.purdie@linuxfoundation.org \
    --cc=openembedded-core@lists.openembedded.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).