* [PATCH 1/4] erofs-utils: fix build error without lz4 library
@ 2020-10-30 12:30 Gao Xiang
2020-10-30 12:30 ` [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH Gao Xiang
` (3 more replies)
0 siblings, 4 replies; 8+ messages in thread
From: Gao Xiang @ 2020-10-30 12:30 UTC (permalink / raw)
To: linux-erofs
This fixes a recent build error if lz4 library doesn't install,
/bin/sh ../libtool --tag=CC --mode=link gcc -Wall -Werror -I../include -g -O2 -o mkfs.erofs mkfs_erofs-main.o -luuid ../lib/liberofs.la -llz4
libtool: link: gcc -Wall -Werror -I../include -g -O2 -o mkfs.erofs mkfs_erofs-main.o -luuid ../lib/.libs/liberofs.a -llz4
/usr/bin/ld: cannot find -llz4
Fixes: c497d89e5eac ("erofs-utils: enhance static linking for lz4 1.8.x")
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
configure.ac | 2 ++
mkfs/Makefile.am | 2 +-
2 files changed, 3 insertions(+), 1 deletion(-)
diff --git a/configure.ac b/configure.ac
index 0f40a84..bff1e29 100644
--- a/configure.ac
+++ b/configure.ac
@@ -240,7 +240,9 @@ if test "x${have_lz4}" = "xyes"; then
else
test -z "${with_lz4_libdir}" || LZ4_LIBS="-R${with_lz4_libdir} $LZ4_LIBS"
fi
+ liblz4_LIBS="${LZ4_LIBS}"
fi
+AC_SUBST([liblz4_LIBS])
AC_CONFIG_FILES([Makefile
man/Makefile
diff --git a/mkfs/Makefile.am b/mkfs/Makefile.am
index ecc468c..8b8e051 100644
--- a/mkfs/Makefile.am
+++ b/mkfs/Makefile.am
@@ -6,5 +6,5 @@ bin_PROGRAMS = mkfs.erofs
AM_CPPFLAGS = ${libuuid_CFLAGS} ${libselinux_CFLAGS}
mkfs_erofs_SOURCES = main.c
mkfs_erofs_CFLAGS = -Wall -Werror -I$(top_srcdir)/include
-mkfs_erofs_LDADD = ${libuuid_LIBS} $(top_builddir)/lib/liberofs.la ${libselinux_LIBS} ${LZ4_LIBS}
+mkfs_erofs_LDADD = ${libuuid_LIBS} $(top_builddir)/lib/liberofs.la ${libselinux_LIBS} ${liblz4_LIBS}
--
2.18.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH
2020-10-30 12:30 [PATCH 1/4] erofs-utils: fix build error without lz4 library Gao Xiang
@ 2020-10-30 12:30 ` Gao Xiang
2020-11-01 14:38 ` Li GuiFu via Linux-erofs
2020-10-30 12:30 ` [PATCH 3/4] mkfs: introduce erofs_mkfs_default_options() Gao Xiang
` (2 subsequent siblings)
3 siblings, 1 reply; 8+ messages in thread
From: Gao Xiang @ 2020-10-30 12:30 UTC (permalink / raw)
To: linux-erofs
Currently, we use -T to set a given UNIX timestamp for all
files, yet reproducible builds [1] requires what is called
"timestamp clamping", IOW, a timestamp must be used no later
than the value of this variable.
Let's support $SOURCE_DATE_EPOCH as well.
[1] https://reproducible-builds.org/specs/source-date-epoch/
Suggested-by: nl6720 <nl6720@gmail.com>
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
include/erofs/config.h | 7 +++++++
lib/inode.c | 15 +++++++++++++--
mkfs/main.c | 34 ++++++++++++++++++++++++++++++++++
3 files changed, 54 insertions(+), 2 deletions(-)
diff --git a/include/erofs/config.h b/include/erofs/config.h
index e425ce2..02ddf59 100644
--- a/include/erofs/config.h
+++ b/include/erofs/config.h
@@ -29,11 +29,18 @@ enum {
FORCE_INODE_EXTENDED,
};
+enum {
+ TIMESTAMP_NONE,
+ TIMESTAMP_FIXED,
+ TIMESTAMP_CLAMPING,
+};
+
struct erofs_configure {
const char *c_version;
int c_dbg_lvl;
bool c_dry_run;
bool c_legacy_compress;
+ char c_timeinherit;
#ifdef HAVE_LIBSELINUX
struct selabel_handle *sehnd;
diff --git a/lib/inode.c b/lib/inode.c
index 5695bbc..fee5c96 100644
--- a/lib/inode.c
+++ b/lib/inode.c
@@ -729,8 +729,19 @@ int erofs_fill_inode(struct erofs_inode *inode,
inode->i_mode = st->st_mode;
inode->i_uid = st->st_uid;
inode->i_gid = st->st_gid;
- inode->i_ctime = sbi.build_time;
- inode->i_ctime_nsec = sbi.build_time_nsec;
+ inode->i_ctime = st->st_ctime;
+ inode->i_ctime_nsec = st->st_ctim.tv_nsec;
+
+ switch (cfg.c_timeinherit) {
+ case TIMESTAMP_CLAMPING:
+ if (st->st_ctime < sbi.build_time)
+ break;
+ case TIMESTAMP_FIXED:
+ inode->i_ctime = sbi.build_time;
+ inode->i_ctime_nsec = sbi.build_time_nsec;
+ default:
+ break;
+ }
inode->i_nlink = 1; /* fix up later if needed */
switch (inode->i_mode & S_IFMT) {
diff --git a/mkfs/main.c b/mkfs/main.c
index 6dda9e3..5c41fc0 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -198,6 +198,7 @@ static int mkfs_parse_options_cfg(int argc, char *argv[])
erofs_err("invalid UNIX timestamp %s", optarg);
return -EINVAL;
}
+ cfg.c_timeinherit = TIMESTAMP_FIXED;
break;
case 2:
opt = erofs_parse_exclude_path(optarg, false);
@@ -381,6 +382,33 @@ static void erofs_mkfs_generate_uuid(void)
erofs_info("filesystem UUID: %s", uuid_str);
}
+/* https://reproducible-builds.org/specs/source-date-epoch/ for more details */
+int parse_source_date_epoch(void)
+{
+ char *source_date_epoch;
+ unsigned long long epoch = -1ULL;
+ char *endptr;
+
+ source_date_epoch = getenv("SOURCE_DATE_EPOCH");
+ if (!source_date_epoch)
+ return 0;
+
+ epoch = strtoull(source_date_epoch, &endptr, 10);
+ if (epoch == -1ULL || *endptr != '\0') {
+ erofs_err("Environment variable $SOURCE_DATE_EPOCH %s is invalid",
+ source_date_epoch);
+ return -EINVAL;
+ }
+
+ if (cfg.c_force_inodeversion != FORCE_INODE_EXTENDED)
+ erofs_info("SOURCE_DATE_EPOCH is set, forcely generate extended inodes instead");
+
+ cfg.c_force_inodeversion = FORCE_INODE_EXTENDED;
+ cfg.c_unix_timestamp = epoch;
+ cfg.c_timeinherit = TIMESTAMP_CLAMPING;
+ return 0;
+}
+
int main(int argc, char **argv)
{
int err = 0;
@@ -405,6 +433,12 @@ int main(int argc, char **argv)
return 1;
}
+ err = parse_source_date_epoch();
+ if (err) {
+ usage();
+ return 1;
+ }
+
err = lstat64(cfg.c_src_path, &st);
if (err)
return 1;
--
2.18.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 3/4] mkfs: introduce erofs_mkfs_default_options()
2020-10-30 12:30 [PATCH 1/4] erofs-utils: fix build error without lz4 library Gao Xiang
2020-10-30 12:30 ` [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH Gao Xiang
@ 2020-10-30 12:30 ` Gao Xiang
2020-11-01 14:55 ` Li GuiFu via Linux-erofs
2020-10-30 12:30 ` [PATCH 4/4] mkfs: add option to use a pre-defined UUID Gao Xiang
2020-11-01 14:35 ` [PATCH 1/4] erofs-utils: fix build error without lz4 library Li GuiFu via Linux-erofs
3 siblings, 1 reply; 8+ messages in thread
From: Gao Xiang @ 2020-10-30 12:30 UTC (permalink / raw)
To: linux-erofs
Gather all default settings, and generate UUID before
parse_options_cfg(), therefore the UUID can be overridden
later by command line for reproducible images.
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
mkfs/main.c | 21 ++++++++++++---------
1 file changed, 12 insertions(+), 9 deletions(-)
diff --git a/mkfs/main.c b/mkfs/main.c
index 5c41fc0..0e17314 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -368,18 +368,18 @@ static int erofs_mkfs_superblock_csum_set(void)
return 0;
}
-static void erofs_mkfs_generate_uuid(void)
+static void erofs_mkfs_default_options(void)
{
- char uuid_str[37] = "not available";
+ cfg.c_legacy_compress = false;
+ sbi.feature_incompat = EROFS_FEATURE_INCOMPAT_LZ4_0PADDING;
+ sbi.feature_compat = EROFS_FEATURE_COMPAT_SB_CHKSUM;
+ /* generate a default uuid first */
#ifdef HAVE_LIBUUID
do {
uuid_generate(sbi.uuid);
} while (uuid_is_null(sbi.uuid));
-
- uuid_unparse_lower(sbi.uuid, uuid_str);
#endif
- erofs_info("filesystem UUID: %s", uuid_str);
}
/* https://reproducible-builds.org/specs/source-date-epoch/ for more details */
@@ -418,13 +418,12 @@ int main(int argc, char **argv)
struct stat64 st;
erofs_blk_t nblocks;
struct timeval t;
+ char uuid_str[37] = "not available";
erofs_init_configure();
fprintf(stderr, "%s %s\n", basename(argv[0]), cfg.c_version);
- cfg.c_legacy_compress = false;
- sbi.feature_incompat = EROFS_FEATURE_INCOMPAT_LZ4_0PADDING;
- sbi.feature_compat = EROFS_FEATURE_COMPAT_SB_CHKSUM;
+ erofs_mkfs_default_options();
err = mkfs_parse_options_cfg(argc, argv);
if (err) {
@@ -495,7 +494,11 @@ int main(int argc, char **argv)
goto exit;
}
- erofs_mkfs_generate_uuid();
+#ifdef HAVE_LIBUUID
+ uuid_unparse_lower(sbi.uuid, uuid_str);
+#endif
+ erofs_info("filesystem UUID: %s", uuid_str);
+
erofs_inode_manager_init();
err = erofs_build_shared_xattrs_from_path(cfg.c_src_path);
--
2.18.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH 4/4] mkfs: add option to use a pre-defined UUID
2020-10-30 12:30 [PATCH 1/4] erofs-utils: fix build error without lz4 library Gao Xiang
2020-10-30 12:30 ` [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH Gao Xiang
2020-10-30 12:30 ` [PATCH 3/4] mkfs: introduce erofs_mkfs_default_options() Gao Xiang
@ 2020-10-30 12:30 ` Gao Xiang
2020-11-01 14:55 ` Li GuiFu via Linux-erofs
2020-11-01 14:35 ` [PATCH 1/4] erofs-utils: fix build error without lz4 library Li GuiFu via Linux-erofs
3 siblings, 1 reply; 8+ messages in thread
From: Gao Xiang @ 2020-10-30 12:30 UTC (permalink / raw)
To: linux-erofs
Usage: mkfs.erofs -U<uuid> <imagefile> <srcdir>
The filesystem UUID can now be optionally specified during filesystem
creation. The default behavior is still to generate a random UUID.
This is useful for reproducible builds.
Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
---
man/mkfs.erofs.1 | 6 ++++++
mkfs/main.c | 13 ++++++++++++-
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/man/mkfs.erofs.1 b/man/mkfs.erofs.1
index 891c5a8..dcaf9d7 100644
--- a/man/mkfs.erofs.1
+++ b/man/mkfs.erofs.1
@@ -52,6 +52,12 @@ Forcely generate extended inodes (64-byte inodes) to output.
Set all files to the given UNIX timestamp. Reproducible builds requires setting
all to a specific one.
.TP
+.BI "\-U " UUID
+Set the universally unique identifier (UUID) of the filesystem to
+.IR UUID .
+The format of the UUID is a series of hex digits separated by hyphens,
+like this: "c1b9d5a2-f162-11cf-9ece-0020afc76f16".
+.TP
.BI "\-\-exclude-path=" path
Ignore file that matches the exact literal path.
You may give multiple `--exclude-path' options.
diff --git a/mkfs/main.c b/mkfs/main.c
index 0e17314..c63b274 100644
--- a/mkfs/main.c
+++ b/mkfs/main.c
@@ -66,6 +66,9 @@ static void usage(void)
" -x# set xattr tolerance to # (< 0, disable xattrs; default 2)\n"
" -EX[,...] X=extended options\n"
" -T# set a fixed UNIX timestamp # to all files\n"
+#ifdef HAVE_LIBUUID
+ " -UX use a given filesystem UUID\n"
+#endif
" --exclude-path=X avoid including file X (X = exact literal path)\n"
" --exclude-regex=X avoid including files that match X (X = regular expression)\n"
#ifdef HAVE_LIBSELINUX
@@ -149,7 +152,7 @@ static int mkfs_parse_options_cfg(int argc, char *argv[])
char *endptr;
int opt, i;
- while((opt = getopt_long(argc, argv, "d:x:z:E:T:",
+ while((opt = getopt_long(argc, argv, "d:x:z:E:T:U:",
long_options, NULL)) != -1) {
switch (opt) {
case 'z':
@@ -200,6 +203,14 @@ static int mkfs_parse_options_cfg(int argc, char *argv[])
}
cfg.c_timeinherit = TIMESTAMP_FIXED;
break;
+#ifdef HAVE_LIBUUID
+ case 'U':
+ if (uuid_parse(optarg, sbi.uuid)) {
+ erofs_err("invalid UUID %s", optarg);
+ return -EINVAL;
+ }
+ break;
+#endif
case 2:
opt = erofs_parse_exclude_path(optarg, false);
if (opt) {
--
2.18.1
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH 1/4] erofs-utils: fix build error without lz4 library
2020-10-30 12:30 [PATCH 1/4] erofs-utils: fix build error without lz4 library Gao Xiang
` (2 preceding siblings ...)
2020-10-30 12:30 ` [PATCH 4/4] mkfs: add option to use a pre-defined UUID Gao Xiang
@ 2020-11-01 14:35 ` Li GuiFu via Linux-erofs
3 siblings, 0 replies; 8+ messages in thread
From: Li GuiFu via Linux-erofs @ 2020-11-01 14:35 UTC (permalink / raw)
To: Gao Xiang, linux-erofs
On 2020/10/30 20:30, Gao Xiang wrote:
> This fixes a recent build error if lz4 library doesn't install,
>
> /bin/sh ../libtool --tag=CC --mode=link gcc -Wall -Werror -I../include -g -O2 -o mkfs.erofs mkfs_erofs-main.o -luuid ../lib/liberofs.la -llz4
> libtool: link: gcc -Wall -Werror -I../include -g -O2 -o mkfs.erofs mkfs_erofs-main.o -luuid ../lib/.libs/liberofs.a -llz4
> /usr/bin/ld: cannot find -llz4
>
> Fixes: c497d89e5eac ("erofs-utils: enhance static linking for lz4 1.8.x")
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
It looks good
Reviewed-by: Li Guifu <bluce.lee@aliyun.com>
Thanks,
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH
2020-10-30 12:30 ` [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH Gao Xiang
@ 2020-11-01 14:38 ` Li GuiFu via Linux-erofs
0 siblings, 0 replies; 8+ messages in thread
From: Li GuiFu via Linux-erofs @ 2020-11-01 14:38 UTC (permalink / raw)
To: Gao Xiang, linux-erofs
On 2020/10/30 20:30, Gao Xiang wrote:
> Currently, we use -T to set a given UNIX timestamp for all
> files, yet reproducible builds [1] requires what is called
> "timestamp clamping", IOW, a timestamp must be used no later
> than the value of this variable.
>
> Let's support $SOURCE_DATE_EPOCH as well.
>
> [1] https://reproducible-builds.org/specs/source-date-epoch/
> Suggested-by: nl6720 <nl6720@gmail.com>
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
It looks good
Reviewed-by: Li Guifu <bluce.lee@aliyun.com>
Thanks,
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 3/4] mkfs: introduce erofs_mkfs_default_options()
2020-10-30 12:30 ` [PATCH 3/4] mkfs: introduce erofs_mkfs_default_options() Gao Xiang
@ 2020-11-01 14:55 ` Li GuiFu via Linux-erofs
0 siblings, 0 replies; 8+ messages in thread
From: Li GuiFu via Linux-erofs @ 2020-11-01 14:55 UTC (permalink / raw)
To: Gao Xiang, linux-erofs
On 2020/10/30 20:30, Gao Xiang wrote:
> Gather all default settings, and generate UUID before
> parse_options_cfg(), therefore the UUID can be overridden
> later by command line for reproducible images.
>
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
> ---
It looks good
Reviewed-by: Li Guifu <bluce.lee@aliyun.com>
Thanks,
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH 4/4] mkfs: add option to use a pre-defined UUID
2020-10-30 12:30 ` [PATCH 4/4] mkfs: add option to use a pre-defined UUID Gao Xiang
@ 2020-11-01 14:55 ` Li GuiFu via Linux-erofs
0 siblings, 0 replies; 8+ messages in thread
From: Li GuiFu via Linux-erofs @ 2020-11-01 14:55 UTC (permalink / raw)
To: Gao Xiang, linux-erofs
On 2020/10/30 20:30, Gao Xiang wrote:
> Usage: mkfs.erofs -U<uuid> <imagefile> <srcdir>
>
> The filesystem UUID can now be optionally specified during filesystem
> creation. The default behavior is still to generate a random UUID.
>
> This is useful for reproducible builds.
>
> Signed-off-by: Gao Xiang <hsiangkao@redhat.com>
It looks good
Reviewed-by: Li Guifu <bluce.lee@aliyun.com>
Thanks,
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2020-11-01 14:56 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-30 12:30 [PATCH 1/4] erofs-utils: fix build error without lz4 library Gao Xiang
2020-10-30 12:30 ` [PATCH 2/4] erofs-utils: support $SOURCE_DATE_EPOCH Gao Xiang
2020-11-01 14:38 ` Li GuiFu via Linux-erofs
2020-10-30 12:30 ` [PATCH 3/4] mkfs: introduce erofs_mkfs_default_options() Gao Xiang
2020-11-01 14:55 ` Li GuiFu via Linux-erofs
2020-10-30 12:30 ` [PATCH 4/4] mkfs: add option to use a pre-defined UUID Gao Xiang
2020-11-01 14:55 ` Li GuiFu via Linux-erofs
2020-11-01 14:35 ` [PATCH 1/4] erofs-utils: fix build error without lz4 library Li GuiFu via Linux-erofs
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.