* [PATCH 2/7] reproducible_build: Drop obsolete sstate workaround
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
@ 2021-10-14 12:10 ` Richard Purdie
2021-10-14 12:10 ` [PATCH 3/7] reproducible: Move class function code into library Richard Purdie
` (5 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:10 UTC (permalink / raw)
To: openembedded-core
sstate has been sufficiently invalidated since this change was made, drop
the workaround now.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/reproducible_build.bbclass | 4 ----
1 file changed, 4 deletions(-)
diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index fd90d771d2b..3f661794c61 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -126,10 +126,6 @@ def get_source_date_epoch_value(d):
s = f.read()
try:
source_date_epoch = int(s)
- # workaround for old sstate with SDE_FILE content being 0 - use SOURCE_DATE_EPOCH_FALLBACK
- if source_date_epoch == 0 :
- source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
- bb.warn("SOURCE_DATE_EPOCH value from sstate '%s' is deprecated/invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK '%s'" % (s, source_date_epoch))
except ValueError:
bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK" % s)
source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
--
2.32.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 3/7] reproducible: Move class function code into library
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
2021-10-14 12:10 ` [PATCH 2/7] reproducible_build: Drop obsolete sstate workaround Richard Purdie
@ 2021-10-14 12:10 ` Richard Purdie
2021-10-14 12:10 ` [PATCH 4/7] reproducible: Move variable definitions to bitbake.conf Richard Purdie
` (4 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:10 UTC (permalink / raw)
To: openembedded-core
To try and avoid parse/memory overhead of functions within bitbake,
move the bulk of the reproducibility functions to the function library.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/base.bbclass | 2 +-
meta/classes/reproducible_build.bbclass | 37 ++-----------------------
meta/lib/oe/reproducible.py | 33 ++++++++++++++++++++++
3 files changed, 36 insertions(+), 36 deletions(-)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 59fd46e5d43..bca3944ae70 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -12,7 +12,7 @@ inherit logging
OE_EXTRA_IMPORTS ?= ""
-OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa ${OE_EXTRA_IMPORTS}"
+OE_IMPORTS += "os sys time oe.path oe.utils oe.types oe.package oe.packagegroup oe.sstatesig oe.lsb oe.cachedpath oe.license oe.qa oe.reproducible ${OE_EXTRA_IMPORTS}"
OE_IMPORTS[type] = "list"
PACKAGECONFIG_CONFARGS ??= ""
diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index 3f661794c61..0f45b782e5d 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -91,19 +91,8 @@ addtask do_deploy_source_date_epoch_setscene
addtask do_deploy_source_date_epoch before do_configure after do_patch
python create_source_date_epoch_stamp() {
- import oe.reproducible
-
- epochfile = d.getVar('SDE_FILE')
- tmp_file = "%s.new" % epochfile
-
source_date_epoch = oe.reproducible.get_source_date_epoch(d, d.getVar('S'))
-
- bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
- bb.utils.mkdirhier(d.getVar('SDE_DIR'))
- with open(tmp_file, 'w') as f:
- f.write(str(source_date_epoch))
-
- os.rename(tmp_file, epochfile)
+ oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), d)
}
EPOCHTASK = "do_deploy_source_date_epoch"
@@ -112,29 +101,7 @@ EPOCHTASK = "do_deploy_source_date_epoch"
do_unpack[postfuncs] += "create_source_date_epoch_stamp"
def get_source_date_epoch_value(d):
- epochfile = d.getVar('SDE_FILE')
- cached, efile = d.getVar('__CACHED_SOURCE_DATE_EPOCH') or (None, None)
- if cached and efile == epochfile:
- return cached
-
- if cached and epochfile != efile:
- bb.debug(1, "Epoch file changed from %s to %s" % (efile, epochfile))
-
- source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
- try:
- with open(epochfile, 'r') as f:
- s = f.read()
- try:
- source_date_epoch = int(s)
- except ValueError:
- bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK" % s)
- source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
- bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
- except FileNotFoundError:
- bb.debug(1, "Cannot find %s. SOURCE_DATE_EPOCH will default to %d" % (epochfile, source_date_epoch))
-
- d.setVar('__CACHED_SOURCE_DATE_EPOCH', (str(source_date_epoch), epochfile))
- return str(source_date_epoch)
+ return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
diff --git a/meta/lib/oe/reproducible.py b/meta/lib/oe/reproducible.py
index 204b9bd734a..a5000574cfa 100644
--- a/meta/lib/oe/reproducible.py
+++ b/meta/lib/oe/reproducible.py
@@ -106,3 +106,36 @@ def get_source_date_epoch(d, sourcedir):
fixed_source_date_epoch(d) # Last resort
)
+def epochfile_read(epochfile, d):
+ cached, efile = d.getVar('__CACHED_SOURCE_DATE_EPOCH') or (None, None)
+ if cached and efile == epochfile:
+ return cached
+
+ if cached and epochfile != efile:
+ bb.debug(1, "Epoch file changed from %s to %s" % (efile, epochfile))
+
+ source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
+ try:
+ with open(epochfile, 'r') as f:
+ s = f.read()
+ try:
+ source_date_epoch = int(s)
+ except ValueError:
+ bb.warn("SOURCE_DATE_EPOCH value '%s' is invalid. Reverting to SOURCE_DATE_EPOCH_FALLBACK" % s)
+ source_date_epoch = int(d.getVar('SOURCE_DATE_EPOCH_FALLBACK'))
+ bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
+ except FileNotFoundError:
+ bb.debug(1, "Cannot find %s. SOURCE_DATE_EPOCH will default to %d" % (epochfile, source_date_epoch))
+
+ d.setVar('__CACHED_SOURCE_DATE_EPOCH', (str(source_date_epoch), epochfile))
+ return str(source_date_epoch)
+
+def epochfile_write(source_date_epoch, epochfile, d):
+
+ bb.debug(1, "SOURCE_DATE_EPOCH: %d" % source_date_epoch)
+ bb.utils.mkdirhier(os.path.dirname(epochfile))
+
+ tmp_file = "%s.new" % epochfile
+ with open(tmp_file, 'w') as f:
+ f.write(str(source_date_epoch))
+ os.rename(tmp_file, epochfile)
--
2.32.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 4/7] reproducible: Move variable definitions to bitbake.conf
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
2021-10-14 12:10 ` [PATCH 2/7] reproducible_build: Drop obsolete sstate workaround Richard Purdie
2021-10-14 12:10 ` [PATCH 3/7] reproducible: Move class function code into library Richard Purdie
@ 2021-10-14 12:10 ` Richard Purdie
2021-10-14 12:10 ` [PATCH 5/7] reproducible: Merge code into base.bbclass Richard Purdie
` (3 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:10 UTC (permalink / raw)
To: openembedded-core
The reproducibility code was originally developed as separate
standalone class but development is no longer experimental and
the code is widely tested and used by default for poky. Reproducible
builds are the direction we need to take as a project.
Transition the core variable definitions to bitbake.conf as part of
a move to make these part of the default workflow.
This also helps reduce test matrix complexity as there is now one
code path.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/reproducible_build.bbclass | 11 -----------
.../classes/reproducible_build_simple.bbclass | 9 ---------
meta/conf/bitbake.conf | 19 ++++++++++++++++++-
3 files changed, 18 insertions(+), 21 deletions(-)
delete mode 100644 meta/classes/reproducible_build_simple.bbclass
diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
index 0f45b782e5d..f38be1a7654 100644
--- a/meta/classes/reproducible_build.bbclass
+++ b/meta/classes/reproducible_build.bbclass
@@ -51,15 +51,6 @@
#
# Once the value is determined, it is stored in the recipe's SDE_FILE.
-BUILD_REPRODUCIBLE_BINARIES ??= '1'
-inherit reproducible_build_simple
-
-SDE_DIR = "${WORKDIR}/source-date-epoch"
-SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
-SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
-
-# A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
-export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
SSTATETASKS += "do_deploy_source_date_epoch"
@@ -103,5 +94,3 @@ do_unpack[postfuncs] += "create_source_date_epoch_stamp"
def get_source_date_epoch_value(d):
return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
-export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
-BB_HASHBASE_WHITELIST += "SOURCE_DATE_EPOCH"
diff --git a/meta/classes/reproducible_build_simple.bbclass b/meta/classes/reproducible_build_simple.bbclass
deleted file mode 100644
index 393372993d0..00000000000
--- a/meta/classes/reproducible_build_simple.bbclass
+++ /dev/null
@@ -1,9 +0,0 @@
-# Setup default environment for reproducible builds.
-
-BUILD_REPRODUCIBLE_BINARIES = "1"
-
-export PYTHONHASHSEED = "0"
-export PERL_HASH_SEED = "0"
-export SOURCE_DATE_EPOCH ??= "1520598896"
-
-REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index cd828251b9b..1db9b3fc051 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -628,6 +628,22 @@ SELECTED_OPTIMIZATION[vardeps] += "FULL_OPTIMIZATION DEBUG_OPTIMIZATION DEBUG_BU
BUILD_OPTIMIZATION = "${@oe.utils.vartrue('DEBUG_BUILD', '-Og -g -feliminate-unused-debug-types', '-O2', d)} -pipe"
BUILD_OPTIMIZATION[vardeps] += "DEBUG_BUILD"
+##################################################################
+# Reproducibility
+##################################################################
+
+SDE_DIR = "${WORKDIR}/source-date-epoch"
+SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
+SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
+
+BUILD_REPRODUCIBLE_BINARIES = "1"
+export PYTHONHASHSEED = "0"
+export PERL_HASH_SEED = "0"
+export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
+# A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
+export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
+REPRODUCIBLE_TIMESTAMP_ROOTFS ??= "1520598896"
+
##################################################################
# Settings used by bitbake-layers.
##################################################################
@@ -908,7 +924,8 @@ BB_HASHEXCLUDE_COMMON ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DI
SSTATE_HASHEQUIV_METHOD SSTATE_HASHEQUIV_REPORT_TASKDATA \
SSTATE_HASHEQUIV_OWNER CCACHE_TOP_DIR BB_HASHSERVE GIT_CEILING_DIRECTORIES \
OMP_NUM_THREADS BB_CURRENTTASK"
-BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS BUILDHISTORY_DIR SSTATE_DIR "
+BB_HASHBASE_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} PSEUDO_IGNORE_PATHS BUILDHISTORY_DIR \
+ SSTATE_DIR SOURCE_DATE_EPOCH"
BB_HASHCONFIG_WHITELIST ?= "${BB_HASHEXCLUDE_COMMON} DATE TIME SSH_AGENT_PID \
SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_EXTRAWHITE DISABLE_SANITY_CHECKS \
PARALLEL_MAKE BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \
--
2.32.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 5/7] reproducible: Merge code into base.bbclass
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
` (2 preceding siblings ...)
2021-10-14 12:10 ` [PATCH 4/7] reproducible: Move variable definitions to bitbake.conf Richard Purdie
@ 2021-10-14 12:10 ` Richard Purdie
2021-10-14 12:10 ` [PATCH 6/7] python: Update now reproducibile builds are the default Richard Purdie
` (2 subsequent siblings)
6 siblings, 0 replies; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:10 UTC (permalink / raw)
To: openembedded-core
Reproducibility is here to stay and needs to be part of our default workflow.
Move the remaining code to base.bbclass so it is always a first class citizen
and it is clear people need to be mindful of it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/base.bbclass | 38 ++++++++
meta/classes/externalsrc.bbclass | 4 +-
meta/classes/image-artifact-names.bbclass | 2 +-
meta/classes/package_deb.bbclass | 3 +-
meta/classes/package_ipk.bbclass | 3 +-
meta/classes/package_rpm.bbclass | 3 +-
meta/classes/reproducible_build.bbclass | 96 --------------------
meta/lib/oe/reproducible.py | 51 +++++++++++
meta/lib/oeqa/selftest/cases/reproducible.py | 1 -
9 files changed, 95 insertions(+), 106 deletions(-)
delete mode 100644 meta/classes/reproducible_build.bbclass
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index bca3944ae70..a65fcc6c1db 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -180,6 +180,44 @@ python base_do_unpack() {
bb.fatal("Bitbake Fetcher Error: " + repr(e))
}
+SSTATETASKS += "do_deploy_source_date_epoch"
+
+do_deploy_source_date_epoch () {
+ mkdir -p ${SDE_DEPLOYDIR}
+ if [ -e ${SDE_FILE} ]; then
+ echo "Deploying SDE from ${SDE_FILE} -> ${SDE_DEPLOYDIR}."
+ cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt
+ else
+ echo "${SDE_FILE} not found!"
+ fi
+}
+
+python do_deploy_source_date_epoch_setscene () {
+ sstate_setscene(d)
+ bb.utils.mkdirhier(d.getVar('SDE_DIR'))
+ sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), '__source_date_epoch.txt')
+ if os.path.exists(sde_file):
+ target = d.getVar('SDE_FILE')
+ bb.debug(1, "Moving setscene SDE file %s -> %s" % (sde_file, target))
+ bb.utils.rename(sde_file, target)
+ else:
+ bb.debug(1, "%s not found!" % sde_file)
+}
+
+do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
+do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
+addtask do_deploy_source_date_epoch_setscene
+addtask do_deploy_source_date_epoch before do_configure after do_patch
+
+python create_source_date_epoch_stamp() {
+ source_date_epoch = oe.reproducible.get_source_date_epoch(d, d.getVar('S'))
+ oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), d)
+}
+do_unpack[postfuncs] += "create_source_date_epoch_stamp"
+
+def get_source_date_epoch_value(d):
+ return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
+
def get_layers_branch_rev(d):
layers = (d.getVar("BBLAYERS") or "").split()
layers_branch_rev = ["%-20s = \"%s:%s\"" % (os.path.basename(i), \
diff --git a/meta/classes/externalsrc.bbclass b/meta/classes/externalsrc.bbclass
index ad93b2d2abf..abfe24bacec 100644
--- a/meta/classes/externalsrc.bbclass
+++ b/meta/classes/externalsrc.bbclass
@@ -109,8 +109,8 @@ python () {
if local_srcuri and task in fetch_tasks:
continue
bb.build.deltask(task, d)
- if bb.data.inherits_class('reproducible_build', d) and task == 'do_unpack':
- # The reproducible_build's create_source_date_epoch_stamp function must
+ if task == 'do_unpack':
+ # The reproducible build create_source_date_epoch_stamp function must
# be run after the source is available and before the
# do_deploy_source_date_epoch task. In the normal case, it's attached
# to do_unpack as a postfuncs, but since we removed do_unpack (above)
diff --git a/meta/classes/image-artifact-names.bbclass b/meta/classes/image-artifact-names.bbclass
index 67f04e81658..f5769e520f5 100644
--- a/meta/classes/image-artifact-names.bbclass
+++ b/meta/classes/image-artifact-names.bbclass
@@ -15,7 +15,7 @@ IMAGE_LINK_NAME ?= "${IMAGE_BASENAME}-${MACHINE}"
IMAGE_NAME_SUFFIX ??= ".rootfs"
python () {
- if bb.data.inherits_class('reproducible_build', d) and bb.data.inherits_class('deploy', d) and d.getVar("IMAGE_VERSION_SUFFIX") == "-${DATETIME}":
+ if bb.data.inherits_class('deploy', d) and d.getVar("IMAGE_VERSION_SUFFIX") == "-${DATETIME}":
import datetime
d.setVar("IMAGE_VERSION_SUFFIX", "-" + datetime.datetime.fromtimestamp(int(d.getVar("SOURCE_DATE_EPOCH")), datetime.timezone.utc).strftime('%Y%m%d%H%M%S'))
d.setVarFlag("IMAGE_VERSION_SUFFIX", "vardepvalue", "")
diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 65dbe6c3923..1ae6393d37d 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -315,8 +315,7 @@ python do_package_write_deb () {
do_package_write_deb[dirs] = "${PKGWRITEDIRDEB}"
do_package_write_deb[cleandirs] = "${PKGWRITEDIRDEB}"
do_package_write_deb[depends] += "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'), 'do_populate_sysroot')}"
-EPOCHTASK ??= ""
-addtask package_write_deb after do_packagedata do_package ${EPOCHTASK} before do_build
+addtask package_write_deb after do_packagedata do_package do_deploy_source_date_epoch before do_build
PACKAGEINDEXDEPS += "dpkg-native:do_populate_sysroot"
PACKAGEINDEXDEPS += "apt-native:do_populate_sysroot"
diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 776fe8ed23b..902b7f94c84 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -274,8 +274,7 @@ python do_package_write_ipk () {
do_package_write_ipk[dirs] = "${PKGWRITEDIRIPK}"
do_package_write_ipk[cleandirs] = "${PKGWRITEDIRIPK}"
do_package_write_ipk[depends] += "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'), 'do_populate_sysroot')}"
-EPOCHTASK ??= ""
-addtask package_write_ipk after do_packagedata do_package ${EPOCHTASK} before do_build
+addtask package_write_ipk after do_packagedata do_package do_deploy_source_date_epoch before do_build
PACKAGEINDEXDEPS += "opkg-utils-native:do_populate_sysroot"
PACKAGEINDEXDEPS += "opkg-native:do_populate_sysroot"
diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index e738806fd72..b0754421a33 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -748,8 +748,7 @@ python do_package_write_rpm () {
do_package_write_rpm[dirs] = "${PKGWRITEDIRRPM}"
do_package_write_rpm[cleandirs] = "${PKGWRITEDIRRPM}"
do_package_write_rpm[depends] += "${@oe.utils.build_depends_string(d.getVar('PACKAGE_WRITE_DEPS'), 'do_populate_sysroot')}"
-EPOCHTASK ??= ""
-addtask package_write_rpm after do_packagedata do_package ${EPOCHTASK} before do_build
+addtask package_write_rpm after do_packagedata do_package do_deploy_source_date_epoch before do_build
PACKAGEINDEXDEPS += "rpm-native:do_populate_sysroot"
PACKAGEINDEXDEPS += "createrepo-c-native:do_populate_sysroot"
diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
deleted file mode 100644
index f38be1a7654..00000000000
--- a/meta/classes/reproducible_build.bbclass
+++ /dev/null
@@ -1,96 +0,0 @@
-# reproducible_build.bbclass
-#
-# Sets the default SOURCE_DATE_EPOCH in each component's build environment.
-# The format is number of seconds since the system epoch.
-#
-# Upstream components (generally) respect this environment variable,
-# using it in place of the "current" date and time.
-# See https://reproducible-builds.org/specs/source-date-epoch/
-#
-# The default value of SOURCE_DATE_EPOCH comes from the function
-# get_source_date_epoch_value which reads from the SDE_FILE, or if the file
-# is not available (or set to 0) will use the fallback of
-# SOURCE_DATE_EPOCH_FALLBACK.
-#
-# The SDE_FILE is normally constructed from the function
-# create_source_date_epoch_stamp which is typically added as a postfuncs to
-# the do_unpack task. If a recipe does NOT have do_unpack, it should be added
-# to a task that runs after the source is available and before the
-# do_deploy_source_date_epoch task is executed.
-#
-# If a recipe wishes to override the default behavior it should set it's own
-# SOURCE_DATE_EPOCH or override the do_deploy_source_date_epoch_stamp task
-# with recipe-specific functionality to write the appropriate
-# SOURCE_DATE_EPOCH into the SDE_FILE.
-#
-# SOURCE_DATE_EPOCH is intended to be a reproducible value. This value should
-# be reproducible for anyone who builds the same revision from the same
-# sources.
-#
-# There are 4 ways the create_source_date_epoch_stamp function determines what
-# becomes SOURCE_DATE_EPOCH:
-#
-# 1. Use the value from __source_date_epoch.txt file if this file exists.
-# This file was most likely created in the previous build by one of the
-# following methods 2,3,4.
-# Alternatively, it can be provided by a recipe via SRC_URI.
-#
-# If the file does not exist:
-#
-# 2. If there is a git checkout, use the last git commit timestamp.
-# Git does not preserve file timestamps on checkout.
-#
-# 3. Use the mtime of "known" files such as NEWS, CHANGLELOG, ...
-# This works for well-kept repositories distributed via tarball.
-#
-# 4. Use the modification time of the youngest file in the source tree, if
-# there is one.
-# This will be the newest file from the distribution tarball, if any.
-#
-# 5. Fall back to a fixed timestamp (SOURCE_DATE_EPOCH_FALLBACK).
-#
-# Once the value is determined, it is stored in the recipe's SDE_FILE.
-
-
-SSTATETASKS += "do_deploy_source_date_epoch"
-
-do_deploy_source_date_epoch () {
- mkdir -p ${SDE_DEPLOYDIR}
- if [ -e ${SDE_FILE} ]; then
- echo "Deploying SDE from ${SDE_FILE} -> ${SDE_DEPLOYDIR}."
- cp -p ${SDE_FILE} ${SDE_DEPLOYDIR}/__source_date_epoch.txt
- else
- echo "${SDE_FILE} not found!"
- fi
-}
-
-python do_deploy_source_date_epoch_setscene () {
- sstate_setscene(d)
- bb.utils.mkdirhier(d.getVar('SDE_DIR'))
- sde_file = os.path.join(d.getVar('SDE_DEPLOYDIR'), '__source_date_epoch.txt')
- if os.path.exists(sde_file):
- target = d.getVar('SDE_FILE')
- bb.debug(1, "Moving setscene SDE file %s -> %s" % (sde_file, target))
- bb.utils.rename(sde_file, target)
- else:
- bb.debug(1, "%s not found!" % sde_file)
-}
-
-do_deploy_source_date_epoch[dirs] = "${SDE_DEPLOYDIR}"
-do_deploy_source_date_epoch[sstate-plaindirs] = "${SDE_DEPLOYDIR}"
-addtask do_deploy_source_date_epoch_setscene
-addtask do_deploy_source_date_epoch before do_configure after do_patch
-
-python create_source_date_epoch_stamp() {
- source_date_epoch = oe.reproducible.get_source_date_epoch(d, d.getVar('S'))
- oe.reproducible.epochfile_write(source_date_epoch, d.getVar('SDE_FILE'), d)
-}
-
-EPOCHTASK = "do_deploy_source_date_epoch"
-
-# Generate the stamp after do_unpack runs
-do_unpack[postfuncs] += "create_source_date_epoch_stamp"
-
-def get_source_date_epoch_value(d):
- return oe.reproducible.epochfile_read(d.getVar('SDE_FILE'), d)
-
diff --git a/meta/lib/oe/reproducible.py b/meta/lib/oe/reproducible.py
index a5000574cfa..4fb99d963ce 100644
--- a/meta/lib/oe/reproducible.py
+++ b/meta/lib/oe/reproducible.py
@@ -5,6 +5,57 @@ import os
import subprocess
import bb
+# For reproducible builds, this code sets the default SOURCE_DATE_EPOCH in each
+# component's build environment. The format is number of seconds since the
+# system epoch.
+#
+# Upstream components (generally) respect this environment variable,
+# using it in place of the "current" date and time.
+# See https://reproducible-builds.org/specs/source-date-epoch/
+#
+# The default value of SOURCE_DATE_EPOCH comes from the function
+# get_source_date_epoch_value which reads from the SDE_FILE, or if the file
+# is not available will use the fallback of SOURCE_DATE_EPOCH_FALLBACK.
+#
+# The SDE_FILE is normally constructed from the function
+# create_source_date_epoch_stamp which is typically added as a postfuncs to
+# the do_unpack task. If a recipe does NOT have do_unpack, it should be added
+# to a task that runs after the source is available and before the
+# do_deploy_source_date_epoch task is executed.
+#
+# If a recipe wishes to override the default behavior it should set it's own
+# SOURCE_DATE_EPOCH or override the do_deploy_source_date_epoch_stamp task
+# with recipe-specific functionality to write the appropriate
+# SOURCE_DATE_EPOCH into the SDE_FILE.
+#
+# SOURCE_DATE_EPOCH is intended to be a reproducible value. This value should
+# be reproducible for anyone who builds the same revision from the same
+# sources.
+#
+# There are 4 ways the create_source_date_epoch_stamp function determines what
+# becomes SOURCE_DATE_EPOCH:
+#
+# 1. Use the value from __source_date_epoch.txt file if this file exists.
+# This file was most likely created in the previous build by one of the
+# following methods 2,3,4.
+# Alternatively, it can be provided by a recipe via SRC_URI.
+#
+# If the file does not exist:
+#
+# 2. If there is a git checkout, use the last git commit timestamp.
+# Git does not preserve file timestamps on checkout.
+#
+# 3. Use the mtime of "known" files such as NEWS, CHANGLELOG, ...
+# This works for well-kept repositories distributed via tarball.
+#
+# 4. Use the modification time of the youngest file in the source tree, if
+# there is one.
+# This will be the newest file from the distribution tarball, if any.
+#
+# 5. Fall back to a fixed timestamp (SOURCE_DATE_EPOCH_FALLBACK).
+#
+# Once the value is determined, it is stored in the recipe's SDE_FILE.
+
def get_source_date_epoch_from_known_files(d, sourcedir):
source_date_epoch = None
newest_file = None
diff --git a/meta/lib/oeqa/selftest/cases/reproducible.py b/meta/lib/oeqa/selftest/cases/reproducible.py
index e4582cb82a6..2e983d2f175 100644
--- a/meta/lib/oeqa/selftest/cases/reproducible.py
+++ b/meta/lib/oeqa/selftest/cases/reproducible.py
@@ -219,7 +219,6 @@ class ReproducibleTests(OESelftestTestCase):
bb.utils.remove(tmpdir, recurse=True)
config = textwrap.dedent('''\
- INHERIT += "reproducible_build"
PACKAGE_CLASSES = "{package_classes}"
INHIBIT_PACKAGE_STRIP = "1"
TMPDIR = "{tmpdir}"
--
2.32.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 6/7] python: Update now reproducibile builds are the default
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
` (3 preceding siblings ...)
2021-10-14 12:10 ` [PATCH 5/7] reproducible: Merge code into base.bbclass Richard Purdie
@ 2021-10-14 12:10 ` Richard Purdie
2021-10-14 12:10 ` [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable Richard Purdie
2021-10-14 16:36 ` [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Khem Raj
6 siblings, 0 replies; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:10 UTC (permalink / raw)
To: openembedded-core
Now reproducible builds are the default move the qemu-usermode check to anon
python code so that invalid configurations are detected.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/recipes-devtools/python/python3_3.10.0.bb | 14 +++++---------
1 file changed, 5 insertions(+), 9 deletions(-)
diff --git a/meta/recipes-devtools/python/python3_3.10.0.bb b/meta/recipes-devtools/python/python3_3.10.0.bb
index 2e850710b68..e3300b6495b 100644
--- a/meta/recipes-devtools/python/python3_3.10.0.bb
+++ b/meta/recipes-devtools/python/python3_3.10.0.bb
@@ -95,15 +95,8 @@ CACHED_CONFIGUREVARS = " \
ac_cv_working_tzset=yes \
"
-def possibly_include_pgo(d):
- # PGO currently causes builds to not be reproducible, so disable it for
- # now. See YOCTO #13407
- if bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', True, False, d) and d.getVar('BUILD_REPRODUCIBLE_BINARIES') != '1':
- return 'pgo'
-
- return ''
-
-PACKAGECONFIG:class-target ??= "readline ${@possibly_include_pgo(d)} gdbm ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
+# PGO currently causes builds to not be reproducible so disable by default, see YOCTO #13407
+PACKAGECONFIG:class-target ??= "readline gdbm ${@bb.utils.filter('DISTRO_FEATURES', 'lto', d)}"
PACKAGECONFIG:class-native ??= "readline gdbm"
PACKAGECONFIG:class-nativesdk ??= "readline gdbm"
PACKAGECONFIG[readline] = ",,readline"
@@ -320,6 +313,9 @@ python(){
d.setVar('PACKAGES', ' '.join(packages))
d.setVar('ALLOW_EMPTY:${PN}-modules', '1')
d.setVar('ALLOW_EMPTY:${PN}-pkgutil', '1')
+
+ if "pgo" in d.getVar("PACKAGECONFIG").split() and not bb.utils.contains('MACHINE_FEATURES', 'qemu-usermode', True, False, d):
+ bb.fatal("pgo cannot be enabled as there is no qemu-usermode support for this architecture/machine")
}
# Files needed to create a new manifest
--
2.32.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
` (4 preceding siblings ...)
2021-10-14 12:10 ` [PATCH 6/7] python: Update now reproducibile builds are the default Richard Purdie
@ 2021-10-14 12:10 ` Richard Purdie
2021-10-14 12:28 ` [OE-core] " Bruce Ashfield
2021-10-14 16:36 ` [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Khem Raj
6 siblings, 1 reply; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:10 UTC (permalink / raw)
To: openembedded-core
We want things to be reproduicble and the variable doesn't really change
much any more. Drop the remaining uses and make those code paths always
active.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
---
meta/classes/image-prelink.bbclass | 13 ++-----
meta/classes/image.bbclass | 14 +++----
meta/classes/kernel.bbclass | 54 +++++++++++++--------------
meta/conf/bitbake.conf | 1 -
meta/lib/oe/sstatesig.py | 2 +-
meta/recipes-core/busybox/busybox.inc | 10 ++---
6 files changed, 41 insertions(+), 53 deletions(-)
diff --git a/meta/classes/image-prelink.bbclass b/meta/classes/image-prelink.bbclass
index 0da094a5518..8158eeaf4ca 100644
--- a/meta/classes/image-prelink.bbclass
+++ b/meta/classes/image-prelink.bbclass
@@ -46,17 +46,12 @@ prelink_image () {
dynamic_loader=${@get_linuxloader(d)}
# prelink!
- if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
- bbnote " prelink: BUILD_REPRODUCIBLE_BINARIES..."
- if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
- export PRELINK_TIMESTAMP=`git log -1 --pretty=%ct `
- else
- export PRELINK_TIMESTAMP=$REPRODUCIBLE_TIMESTAMP_ROOTFS
- fi
- ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -am -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
+ if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
+ export PRELINK_TIMESTAMP=`git log -1 --pretty=%ct `
else
- ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
+ export PRELINK_TIMESTAMP=$REPRODUCIBLE_TIMESTAMP_ROOTFS
fi
+ ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -am -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
# Remove the prelink.conf if we had to add it.
if [ "$dummy_prelink_conf" = "true" ]; then
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index c2f32320278..2fa69a40d10 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -651,17 +651,15 @@ ROOTFS_PREPROCESS_COMMAND += "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge'
POPULATE_SDK_PRE_TARGET_COMMAND += "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', 'create_merged_usr_symlinks_sdk; ', '',d)}"
reproducible_final_image_task () {
- if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
+ if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
+ REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1 --pretty=%ct 2>/dev/null` || true
if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
- REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1 --pretty=%ct 2>/dev/null` || true
- if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
- REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(d.getVar("BBPATH"), "conf/bitbake.conf")}`
- fi
+ REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(d.getVar("BBPATH"), "conf/bitbake.conf")}`
fi
- # Set mtime of all files to a reproducible value
- bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
- find ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
fi
+ # Set mtime of all files to a reproducible value
+ bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
+ find ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
}
systemd_preset_all () {
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 4acec1877e0..3dd9b4446db 100644
--- a/meta/classes/kernel.bbclass
+++ b/meta/classes/kernel.bbclass
@@ -324,21 +324,20 @@ addtask bundle_initramfs after do_install before do_deploy
kernel_do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
- if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
- # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
- # be set....
- if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
- # The source directory is not necessarily a git repository, so we
- # specify the git-dir to ensure that git does not query a
- # repository in any parent directory.
- SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
- fi
-
- ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
- export KBUILD_BUILD_TIMESTAMP="$ts"
- export KCONFIG_NOTIMESTAMP=1
- bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
+ # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
+ # be set....
+ if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
+ # The source directory is not necessarily a git repository, so we
+ # specify the git-dir to ensure that git does not query a
+ # repository in any parent directory.
+ SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
fi
+
+ ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
+ export KBUILD_BUILD_TIMESTAMP="$ts"
+ export KCONFIG_NOTIMESTAMP=1
+ bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
+
# The $use_alternate_initrd is only set from
# do_bundle_initramfs() This variable is specifically for the
# case where we are making a second pass at the kernel
@@ -364,21 +363,20 @@ kernel_do_compile() {
do_compile_kernelmodules() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
- if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
- # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
- # be set....
- if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
- # The source directory is not necessarily a git repository, so we
- # specify the git-dir to ensure that git does not query a
- # repository in any parent directory.
- SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
- fi
-
- ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
- export KBUILD_BUILD_TIMESTAMP="$ts"
- export KCONFIG_NOTIMESTAMP=1
- bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
+ # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
+ # be set....
+ if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
+ # The source directory is not necessarily a git repository, so we
+ # specify the git-dir to ensure that git does not query a
+ # repository in any parent directory.
+ SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
fi
+
+ ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
+ export KBUILD_BUILD_TIMESTAMP="$ts"
+ export KCONFIG_NOTIMESTAMP=1
+ bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
+
if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1db9b3fc051..790f2f7a8c4 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -636,7 +636,6 @@ SDE_DIR = "${WORKDIR}/source-date-epoch"
SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
-BUILD_REPRODUCIBLE_BINARIES = "1"
export PYTHONHASHSEED = "0"
export PERL_HASH_SEED = "0"
export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
index c2e3e2f4f58..038404e377a 100644
--- a/meta/lib/oe/sstatesig.py
+++ b/meta/lib/oe/sstatesig.py
@@ -489,7 +489,7 @@ def OEOuthashBasic(path, sigfile, task, d):
include_timestamps = False
include_root = True
if task == "package":
- include_timestamps = d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1'
+ include_timestamps = True
include_root = False
extra_content = d.getVar('HASHEQUIV_HASH_VERSION')
diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
index 808c3dc7009..622325aabb3 100644
--- a/meta/recipes-core/busybox/busybox.inc
+++ b/meta/recipes-core/busybox/busybox.inc
@@ -110,9 +110,8 @@ python () {
}
do_prepare_config () {
- if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
- export KCONFIG_NOTIMESTAMP=1
- fi
+ export KCONFIG_NOTIMESTAMP=1
+
sed -e '/CONFIG_STATIC/d' \
< ${WORKDIR}/defconfig > ${S}/.config
echo "# CONFIG_STATIC is not set" >> .config
@@ -143,9 +142,8 @@ do_configure () {
do_compile() {
unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
- if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
- export KCONFIG_NOTIMESTAMP=1
- fi
+ export KCONFIG_NOTIMESTAMP=1
+
if [ "${BUSYBOX_SPLIT_SUID}" = "1" -a x`grep "CONFIG_FEATURE_INDIVIDUAL=y" .config` = x ]; then
# split the .config into two parts, and make two busybox binaries
if [ -e .config.orig ]; then
--
2.32.0
^ permalink raw reply related [flat|nested] 16+ messages in thread* Re: [OE-core] [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 12:10 ` [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable Richard Purdie
@ 2021-10-14 12:28 ` Bruce Ashfield
2021-10-14 12:31 ` Richard Purdie
0 siblings, 1 reply; 16+ messages in thread
From: Bruce Ashfield @ 2021-10-14 12:28 UTC (permalink / raw)
To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer
On Thu, Oct 14, 2021 at 8:10 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> We want things to be reproduicble and the variable doesn't really change
> much any more. Drop the remaining uses and make those code paths always
> active.
It wasn't clear to me from reading the patch. What is the way that
someone would now get the current timestamp into a kernel build, if
that's the behaviour that they want ?
Bruce
>
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> meta/classes/image-prelink.bbclass | 13 ++-----
> meta/classes/image.bbclass | 14 +++----
> meta/classes/kernel.bbclass | 54 +++++++++++++--------------
> meta/conf/bitbake.conf | 1 -
> meta/lib/oe/sstatesig.py | 2 +-
> meta/recipes-core/busybox/busybox.inc | 10 ++---
> 6 files changed, 41 insertions(+), 53 deletions(-)
>
> diff --git a/meta/classes/image-prelink.bbclass b/meta/classes/image-prelink.bbclass
> index 0da094a5518..8158eeaf4ca 100644
> --- a/meta/classes/image-prelink.bbclass
> +++ b/meta/classes/image-prelink.bbclass
> @@ -46,17 +46,12 @@ prelink_image () {
> dynamic_loader=${@get_linuxloader(d)}
>
> # prelink!
> - if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> - bbnote " prelink: BUILD_REPRODUCIBLE_BINARIES..."
> - if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> - export PRELINK_TIMESTAMP=`git log -1 --pretty=%ct `
> - else
> - export PRELINK_TIMESTAMP=$REPRODUCIBLE_TIMESTAMP_ROOTFS
> - fi
> - ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -am -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
> + if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> + export PRELINK_TIMESTAMP=`git log -1 --pretty=%ct `
> else
> - ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -amR -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
> + export PRELINK_TIMESTAMP=$REPRODUCIBLE_TIMESTAMP_ROOTFS
> fi
> + ${STAGING_SBINDIR_NATIVE}/prelink --root ${IMAGE_ROOTFS} -am -N -c ${sysconfdir}/prelink.conf --dynamic-linker $dynamic_loader
>
> # Remove the prelink.conf if we had to add it.
> if [ "$dummy_prelink_conf" = "true" ]; then
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index c2f32320278..2fa69a40d10 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -651,17 +651,15 @@ ROOTFS_PREPROCESS_COMMAND += "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge'
> POPULATE_SDK_PRE_TARGET_COMMAND += "${@bb.utils.contains('DISTRO_FEATURES', 'usrmerge', 'create_merged_usr_symlinks_sdk; ', '',d)}"
>
> reproducible_final_image_task () {
> - if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> + if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> + REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1 --pretty=%ct 2>/dev/null` || true
> if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> - REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1 --pretty=%ct 2>/dev/null` || true
> - if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then
> - REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(d.getVar("BBPATH"), "conf/bitbake.conf")}`
> - fi
> + REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(d.getVar("BBPATH"), "conf/bitbake.conf")}`
> fi
> - # Set mtime of all files to a reproducible value
> - bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
> - find ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
> fi
> + # Set mtime of all files to a reproducible value
> + bbnote "reproducible_final_image_task: mtime set to $REPRODUCIBLE_TIMESTAMP_ROOTFS"
> + find ${IMAGE_ROOTFS} -print0 | xargs -0 touch -h --date=@$REPRODUCIBLE_TIMESTAMP_ROOTFS
> }
>
> systemd_preset_all () {
> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
> index 4acec1877e0..3dd9b4446db 100644
> --- a/meta/classes/kernel.bbclass
> +++ b/meta/classes/kernel.bbclass
> @@ -324,21 +324,20 @@ addtask bundle_initramfs after do_install before do_deploy
>
> kernel_do_compile() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> - if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> - # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
> - # be set....
> - if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
> - # The source directory is not necessarily a git repository, so we
> - # specify the git-dir to ensure that git does not query a
> - # repository in any parent directory.
> - SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
> - fi
> -
> - ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
> - export KBUILD_BUILD_TIMESTAMP="$ts"
> - export KCONFIG_NOTIMESTAMP=1
> - bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
> + # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
> + # be set....
> + if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
> + # The source directory is not necessarily a git repository, so we
> + # specify the git-dir to ensure that git does not query a
> + # repository in any parent directory.
> + SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
> fi
> +
> + ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
> + export KBUILD_BUILD_TIMESTAMP="$ts"
> + export KCONFIG_NOTIMESTAMP=1
> + bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
> +
> # The $use_alternate_initrd is only set from
> # do_bundle_initramfs() This variable is specifically for the
> # case where we are making a second pass at the kernel
> @@ -364,21 +363,20 @@ kernel_do_compile() {
>
> do_compile_kernelmodules() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
> - if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> - # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
> - # be set....
> - if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
> - # The source directory is not necessarily a git repository, so we
> - # specify the git-dir to ensure that git does not query a
> - # repository in any parent directory.
> - SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
> - fi
> -
> - ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
> - export KBUILD_BUILD_TIMESTAMP="$ts"
> - export KCONFIG_NOTIMESTAMP=1
> - bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
> + # kernel sources do not use do_unpack, so SOURCE_DATE_EPOCH may not
> + # be set....
> + if [ "${SOURCE_DATE_EPOCH}" = "" -o "${SOURCE_DATE_EPOCH}" = "0" ]; then
> + # The source directory is not necessarily a git repository, so we
> + # specify the git-dir to ensure that git does not query a
> + # repository in any parent directory.
> + SOURCE_DATE_EPOCH=`git --git-dir="${S}/.git" log -1 --pretty=%ct 2>/dev/null || echo "${REPRODUCIBLE_TIMESTAMP_ROOTFS}"`
> fi
> +
> + ts=`LC_ALL=C date -d @$SOURCE_DATE_EPOCH`
> + export KBUILD_BUILD_TIMESTAMP="$ts"
> + export KCONFIG_NOTIMESTAMP=1
> + bbnote "KBUILD_BUILD_TIMESTAMP: $ts"
> +
> if (grep -q -i -e '^CONFIG_MODULES=y$' ${B}/.config); then
> oe_runmake -C ${B} ${PARALLEL_MAKE} modules CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 1db9b3fc051..790f2f7a8c4 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -636,7 +636,6 @@ SDE_DIR = "${WORKDIR}/source-date-epoch"
> SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
> SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
>
> -BUILD_REPRODUCIBLE_BINARIES = "1"
> export PYTHONHASHSEED = "0"
> export PERL_HASH_SEED = "0"
> export SOURCE_DATE_EPOCH ?= "${@get_source_date_epoch_value(d)}"
> diff --git a/meta/lib/oe/sstatesig.py b/meta/lib/oe/sstatesig.py
> index c2e3e2f4f58..038404e377a 100644
> --- a/meta/lib/oe/sstatesig.py
> +++ b/meta/lib/oe/sstatesig.py
> @@ -489,7 +489,7 @@ def OEOuthashBasic(path, sigfile, task, d):
> include_timestamps = False
> include_root = True
> if task == "package":
> - include_timestamps = d.getVar('BUILD_REPRODUCIBLE_BINARIES') == '1'
> + include_timestamps = True
> include_root = False
> extra_content = d.getVar('HASHEQUIV_HASH_VERSION')
>
> diff --git a/meta/recipes-core/busybox/busybox.inc b/meta/recipes-core/busybox/busybox.inc
> index 808c3dc7009..622325aabb3 100644
> --- a/meta/recipes-core/busybox/busybox.inc
> +++ b/meta/recipes-core/busybox/busybox.inc
> @@ -110,9 +110,8 @@ python () {
> }
>
> do_prepare_config () {
> - if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> - export KCONFIG_NOTIMESTAMP=1
> - fi
> + export KCONFIG_NOTIMESTAMP=1
> +
> sed -e '/CONFIG_STATIC/d' \
> < ${WORKDIR}/defconfig > ${S}/.config
> echo "# CONFIG_STATIC is not set" >> .config
> @@ -143,9 +142,8 @@ do_configure () {
>
> do_compile() {
> unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS
> - if [ "${BUILD_REPRODUCIBLE_BINARIES}" = "1" ]; then
> - export KCONFIG_NOTIMESTAMP=1
> - fi
> + export KCONFIG_NOTIMESTAMP=1
> +
> if [ "${BUSYBOX_SPLIT_SUID}" = "1" -a x`grep "CONFIG_FEATURE_INDIVIDUAL=y" .config` = x ]; then
> # split the .config into two parts, and make two busybox binaries
> if [ -e .config.orig ]; then
> --
> 2.32.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#156943): https://lists.openembedded.org/g/openembedded-core/message/156943
> Mute This Topic: https://lists.openembedded.org/mt/86310659/1050810
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [bruce.ashfield@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [OE-core] [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 12:28 ` [OE-core] " Bruce Ashfield
@ 2021-10-14 12:31 ` Richard Purdie
2021-10-14 12:38 ` Bruce Ashfield
0 siblings, 1 reply; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 12:31 UTC (permalink / raw)
To: Bruce Ashfield; +Cc: Patches and discussions about the oe-core layer
On Thu, 2021-10-14 at 08:28 -0400, Bruce Ashfield wrote:
> On Thu, Oct 14, 2021 at 8:10 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > We want things to be reproduicble and the variable doesn't really change
> > much any more. Drop the remaining uses and make those code paths always
> > active.
>
> It wasn't clear to me from reading the patch. What is the way that
> someone would now get the current timestamp into a kernel build, if
> that's the behaviour that they want ?
With this change they probably don't get the option. If we want that to be
configurable, we should probably move the control to a different variable which
is focused specifically on the kernel. The hardest bit is probably picking a
name!
You don't really want to turn off all reproducibility everywhere for just the
kernel timestamping change.
Cheers,
Richard
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OE-core] [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 12:31 ` Richard Purdie
@ 2021-10-14 12:38 ` Bruce Ashfield
2021-10-14 13:45 ` Alexandre Belloni
0 siblings, 1 reply; 16+ messages in thread
From: Bruce Ashfield @ 2021-10-14 12:38 UTC (permalink / raw)
To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer
On Thu, Oct 14, 2021 at 8:32 AM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2021-10-14 at 08:28 -0400, Bruce Ashfield wrote:
> > On Thu, Oct 14, 2021 at 8:10 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > >
> > > We want things to be reproduicble and the variable doesn't really change
> > > much any more. Drop the remaining uses and make those code paths always
> > > active.
> >
> > It wasn't clear to me from reading the patch. What is the way that
> > someone would now get the current timestamp into a kernel build, if
> > that's the behaviour that they want ?
>
> With this change they probably don't get the option. If we want that to be
> configurable, we should probably move the control to a different variable which
> is focused specifically on the kernel. The hardest bit is probably picking a
> name!
This is probably the most surprising feature of reproducibility for
kernel developers, and a question that I've gotten multiple times
("what happened to my timestamp ?" followed by "how do I turn this
off?")
Most kernel developers already don't really like the "yocto workflow",
and this adds another element in that category. It is common in a
debug scenario to check the timestamp of the running kernel to make
sure that you've actually booted the one you just built.
>
> You don't really want to turn off all reproducibility everywhere for just the
> kernel timestamping change.
Agreed.
And agreed on picking the name, could it be something around
KERNEL_DEBUG ? And we could even put a bbnote (that they won't see,
but that's the best I can think of) that indicates that
reproducibility is on and the timestamp is set to <x> .. and the
opposite when reproducibility is off ?
Bruce
>
> Cheers,
>
> Richard
>
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [OE-core] [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 12:38 ` Bruce Ashfield
@ 2021-10-14 13:45 ` Alexandre Belloni
2021-10-14 13:46 ` Bruce Ashfield
0 siblings, 1 reply; 16+ messages in thread
From: Alexandre Belloni @ 2021-10-14 13:45 UTC (permalink / raw)
To: Bruce Ashfield
Cc: Richard Purdie, Patches and discussions about the oe-core layer
On 14/10/2021 08:38:46-0400, Bruce Ashfield wrote:
> On Thu, Oct 14, 2021 at 8:32 AM Richard Purdie
> <richard.purdie@linuxfoundation.org> wrote:
> >
> > On Thu, 2021-10-14 at 08:28 -0400, Bruce Ashfield wrote:
> > > On Thu, Oct 14, 2021 at 8:10 AM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > >
> > > > We want things to be reproduicble and the variable doesn't really change
> > > > much any more. Drop the remaining uses and make those code paths always
> > > > active.
> > >
> > > It wasn't clear to me from reading the patch. What is the way that
> > > someone would now get the current timestamp into a kernel build, if
> > > that's the behaviour that they want ?
> >
> > With this change they probably don't get the option. If we want that to be
> > configurable, we should probably move the control to a different variable which
> > is focused specifically on the kernel. The hardest bit is probably picking a
> > name!
>
> This is probably the most surprising feature of reproducibility for
> kernel developers, and a question that I've gotten multiple times
> ("what happened to my timestamp ?" followed by "how do I turn this
> off?")
>
> Most kernel developers already don't really like the "yocto workflow",
> and this adds another element in that category. It is common in a
> debug scenario to check the timestamp of the running kernel to make
> sure that you've actually booted the one you just built.
>
But do kernel developers actually do their development using YP?
I definitively not. I do all my BSP work outside of any build system and
then I finally integrate everything once this is working/ready.
It did indeed surprise me the first time I saw that but this doesn't
have any actual effect on my workflow. Also, the issue was already
existing if what you were trying to build was already in the sstate
cache (e.G from a nightly build), it would contain the time of the
actual build.
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [OE-core] [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 13:45 ` Alexandre Belloni
@ 2021-10-14 13:46 ` Bruce Ashfield
2021-10-14 15:26 ` Richard Purdie
0 siblings, 1 reply; 16+ messages in thread
From: Bruce Ashfield @ 2021-10-14 13:46 UTC (permalink / raw)
To: Alexandre Belloni
Cc: Richard Purdie, Patches and discussions about the oe-core layer
On Thu, Oct 14, 2021 at 9:45 AM Alexandre Belloni
<alexandre.belloni@bootlin.com> wrote:
>
> On 14/10/2021 08:38:46-0400, Bruce Ashfield wrote:
> > On Thu, Oct 14, 2021 at 8:32 AM Richard Purdie
> > <richard.purdie@linuxfoundation.org> wrote:
> > >
> > > On Thu, 2021-10-14 at 08:28 -0400, Bruce Ashfield wrote:
> > > > On Thu, Oct 14, 2021 at 8:10 AM Richard Purdie
> > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > >
> > > > > We want things to be reproduicble and the variable doesn't really change
> > > > > much any more. Drop the remaining uses and make those code paths always
> > > > > active.
> > > >
> > > > It wasn't clear to me from reading the patch. What is the way that
> > > > someone would now get the current timestamp into a kernel build, if
> > > > that's the behaviour that they want ?
> > >
> > > With this change they probably don't get the option. If we want that to be
> > > configurable, we should probably move the control to a different variable which
> > > is focused specifically on the kernel. The hardest bit is probably picking a
> > > name!
> >
> > This is probably the most surprising feature of reproducibility for
> > kernel developers, and a question that I've gotten multiple times
> > ("what happened to my timestamp ?" followed by "how do I turn this
> > off?")
> >
> > Most kernel developers already don't really like the "yocto workflow",
> > and this adds another element in that category. It is common in a
> > debug scenario to check the timestamp of the running kernel to make
> > sure that you've actually booted the one you just built.
> >
>
> But do kernel developers actually do their development using YP?
> I definitively not. I do all my BSP work outside of any build system and
> then I finally integrate everything once this is working/ready.
Many do, yup!
Bruce
>
> It did indeed surprise me the first time I saw that but this doesn't
> have any actual effect on my workflow. Also, the issue was already
> existing if what you were trying to build was already in the sstate
> cache (e.G from a nightly build), it would contain the time of the
> actual build.
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
--
- Thou shalt not follow the NULL pointer, for chaos and madness await
thee at its end
- "Use the force Harry" - Gandalf, Star Trek II
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [OE-core] [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable
2021-10-14 13:46 ` Bruce Ashfield
@ 2021-10-14 15:26 ` Richard Purdie
0 siblings, 0 replies; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 15:26 UTC (permalink / raw)
To: Bruce Ashfield, Alexandre Belloni
Cc: Patches and discussions about the oe-core layer
On Thu, 2021-10-14 at 09:46 -0400, Bruce Ashfield wrote:
> On Thu, Oct 14, 2021 at 9:45 AM Alexandre Belloni
> <alexandre.belloni@bootlin.com> wrote:
> >
> > On 14/10/2021 08:38:46-0400, Bruce Ashfield wrote:
> > > On Thu, Oct 14, 2021 at 8:32 AM Richard Purdie
> > > <richard.purdie@linuxfoundation.org> wrote:
> > > >
> > > > On Thu, 2021-10-14 at 08:28 -0400, Bruce Ashfield wrote:
> > > > > On Thu, Oct 14, 2021 at 8:10 AM Richard Purdie
> > > > > <richard.purdie@linuxfoundation.org> wrote:
> > > > > >
> > > > > > We want things to be reproduicble and the variable doesn't really change
> > > > > > much any more. Drop the remaining uses and make those code paths always
> > > > > > active.
> > > > >
> > > > > It wasn't clear to me from reading the patch. What is the way that
> > > > > someone would now get the current timestamp into a kernel build, if
> > > > > that's the behaviour that they want ?
> > > >
> > > > With this change they probably don't get the option. If we want that to be
> > > > configurable, we should probably move the control to a different variable which
> > > > is focused specifically on the kernel. The hardest bit is probably picking a
> > > > name!
> > >
> > > This is probably the most surprising feature of reproducibility for
> > > kernel developers, and a question that I've gotten multiple times
> > > ("what happened to my timestamp ?" followed by "how do I turn this
> > > off?")
> > >
> > > Most kernel developers already don't really like the "yocto workflow",
> > > and this adds another element in that category. It is common in a
> > > debug scenario to check the timestamp of the running kernel to make
> > > sure that you've actually booted the one you just built.
> > >
> >
> > But do kernel developers actually do their development using YP?
> > I definitively not. I do all my BSP work outside of any build system and
> > then I finally integrate everything once this is working/ready.
>
> Many do, yup!
I've sent out a new version of this patch with an additional one which gives
people a setting to control the timestamp for the kernel.
Cheers,
Richard
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning
2021-10-14 12:10 [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Richard Purdie
` (5 preceding siblings ...)
2021-10-14 12:10 ` [PATCH 7/7] reproducible: Drop BUILD_REPRODUCIBLE_BINARIES variable Richard Purdie
@ 2021-10-14 16:36 ` Khem Raj
2021-10-14 21:48 ` Richard Purdie
6 siblings, 1 reply; 16+ messages in thread
From: Khem Raj @ 2021-10-14 16:36 UTC (permalink / raw)
To: Richard Purdie, openembedded-core
On 10/14/21 5:10 AM, Richard Purdie wrote:
> We now pass compiler options that mean the compiler will adjust these values to
> SOURCE_DATE_EPOCH, the compiler warnings are now unneeded.
>
Is there a chance of it being overwritten by component's build system
in that case this warning may still be handy
> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
> ---
> meta/classes/reproducible_build.bbclass | 3 ---
> 1 file changed, 3 deletions(-)
>
> diff --git a/meta/classes/reproducible_build.bbclass b/meta/classes/reproducible_build.bbclass
> index 7571c116c82..fd90d771d2b 100644
> --- a/meta/classes/reproducible_build.bbclass
> +++ b/meta/classes/reproducible_build.bbclass
> @@ -58,9 +58,6 @@ SDE_DIR = "${WORKDIR}/source-date-epoch"
> SDE_FILE = "${SDE_DIR}/__source_date_epoch.txt"
> SDE_DEPLOYDIR = "${WORKDIR}/deploy-source-date-epoch"
>
> -# Enable compiler warning when the __TIME__, __DATE__ and __TIMESTAMP__ macros are used.
> -TARGET_CC_ARCH:append:class-target = " -Wdate-time"
> -
> # A SOURCE_DATE_EPOCH of '0' might be misinterpreted as no SDE
> export SOURCE_DATE_EPOCH_FALLBACK ??= "1302044400"
>
>
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#156939): https://lists.openembedded.org/g/openembedded-core/message/156939
> Mute This Topic: https://lists.openembedded.org/mt/86310655/1997914
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [raj.khem@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 16+ messages in thread* Re: [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning
2021-10-14 16:36 ` [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning Khem Raj
@ 2021-10-14 21:48 ` Richard Purdie
2021-10-14 23:22 ` Khem Raj
0 siblings, 1 reply; 16+ messages in thread
From: Richard Purdie @ 2021-10-14 21:48 UTC (permalink / raw)
To: Khem Raj, openembedded-core
On Thu, 2021-10-14 at 09:36 -0700, Khem Raj wrote:
>
> On 10/14/21 5:10 AM, Richard Purdie wrote:
> > We now pass compiler options that mean the compiler will adjust these values to
> > SOURCE_DATE_EPOCH, the compiler warnings are now unneeded.
> >
>
> Is there a chance of it being overwritten by component's build system
> in that case this warning may still be handy
The warnings are basically lost as noise in the log.do_compile files so I'm not
sure it buys us anything...
Cheers,
Richard
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [OE-core] [PATCH 1/7] reproducible_build: Drop now unneeded compiler warning
2021-10-14 21:48 ` Richard Purdie
@ 2021-10-14 23:22 ` Khem Raj
0 siblings, 0 replies; 16+ messages in thread
From: Khem Raj @ 2021-10-14 23:22 UTC (permalink / raw)
To: Richard Purdie; +Cc: Patches and discussions about the oe-core layer
On Thu, Oct 14, 2021 at 2:48 PM Richard Purdie
<richard.purdie@linuxfoundation.org> wrote:
>
> On Thu, 2021-10-14 at 09:36 -0700, Khem Raj wrote:
> >
> > On 10/14/21 5:10 AM, Richard Purdie wrote:
> > > We now pass compiler options that mean the compiler will adjust these values to
> > > SOURCE_DATE_EPOCH, the compiler warnings are now unneeded.
> > >
> >
> > Is there a chance of it being overwritten by component's build system
> > in that case this warning may still be handy
>
> The warnings are basically lost as noise in the log.do_compile files so I'm not
> sure it buys us anything...
From system build POV perhaps right. This does seem helpful for folks
who are developing components using the project
and might be transitioning to use reproducible builds. Of course, they
can add the option to their project if that is important to them.
thirdly SOURCE_DATE_EPOCH is gcc specific and keeping it around helps
non-gcc ccmpilers too. But, I guess folks can always customize
so its fine if deleting it is solving some other problem.
>
> Cheers,
>
> Richard
>
^ permalink raw reply [flat|nested] 16+ messages in thread