[PATCH RESEND 1/3] initramfs: move unnecessary memcmp from hot path

[PATCH RESEND 1/3] initramfs: move unnecessary memcmp from hot path

[David Disseldorp]

do_header() is called for each cpio entry and first checks for "newc"
magic before parsing further. The magic check includes a special case
error message if POSIX.1 ASCII (cpio -H odc) magic is detected. This
special case POSIX.1 check needn't be done in the hot path, so move it
under the non-newc-magic error path.

Signed-off-by: David Disseldorp <ddiss@suse.de>
---
 init/initramfs.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/init/initramfs.c b/init/initramfs.c
index af27abc59643..f01590cefa2d 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -256,12 +256,11 @@ static int __init do_collect(void)
 
 static int __init do_header(void)
 {
-	if (memcmp(collected, "070707", 6)==0) {
-		error("incorrect cpio method used: use -H newc option");
-		return 1;
-	}
 	if (memcmp(collected, "070701", 6)) {
-		error("no cpio magic");
+		if (memcmp(collected, "070707", 6) == 0)
+			error("incorrect cpio method used: use -H newc option");
+		else
+			error("no cpio magic");
 		return 1;
 	}
 	parse_header(collected);
-- 
2.26.2

[PATCH RESEND 2/3] initramfs: print helpful cpio error on "crc" magic

[David Disseldorp]

Contrary to the buffer-format.rst documentation, initramfs cpio
extraction does not support "crc" archives, which carry "070702"
header magic. Make it a little clearer that "newc" (magic="070701") is
the only supported cpio format, by extending the POSIX.1 ASCII
(magic="070707") specific error message to also cover "crc" magic.

Signed-off-by: David Disseldorp <ddiss@suse.de>
---
 init/initramfs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/init/initramfs.c b/init/initramfs.c
index f01590cefa2d..19b1c70446fc 100644
--- a/init/initramfs.c
+++ b/init/initramfs.c
@@ -257,7 +257,7 @@ static int __init do_collect(void)
 static int __init do_header(void)
 {
 	if (memcmp(collected, "070701", 6)) {
-		if (memcmp(collected, "070707", 6) == 0)
+		if (memcmp(collected, "0707", 4) == 0)
 			error("incorrect cpio method used: use -H newc option");
 		else
 			error("no cpio magic");
-- 
2.26.2

[PATCH v2 3/3] docs: remove mention of "crc" cpio format support

[David Disseldorp]

init/initramfs.c only supports extraction of cpio archives carrying the
"newc" header magic ("070701"). Remove statements indicating support for
the "crc" cpio format.

Signed-off-by: David Disseldorp <ddiss@suse.de>
---
 .../early-userspace/buffer-format.rst         | 24 +++++++------------
 1 file changed, 8 insertions(+), 16 deletions(-)

Version 2:
- reword initramfs padding description, as suggested by Matthew Wilcox

diff --git a/Documentation/driver-api/early-userspace/buffer-format.rst b/Documentation/driver-api/early-userspace/buffer-format.rst
index 7f74e301fdf3..0df76bca444c 100644
--- a/Documentation/driver-api/early-userspace/buffer-format.rst
+++ b/Documentation/driver-api/early-userspace/buffer-format.rst
@@ -14,10 +14,10 @@ is different.  The initramfs buffer contains an archive which is
 expanded into a ramfs filesystem; this document details the format of
 the initramfs buffer format.
 
-The initramfs buffer format is based around the "newc" or "crc" CPIO
-formats, and can be created with the cpio(1) utility.  The cpio
-archive can be compressed using gzip(1).  One valid version of an
-initramfs buffer is thus a single .cpio.gz file.
+The initramfs buffer format is based around the "newc" CPIO format, and
+can be created with the cpio(1) utility.  The cpio archive can be
+compressed using gzip(1).  One valid version of an initramfs buffer is
+thus a single .cpio.gz file.
 
 The full format of the initramfs buffer is defined by the following
 grammar, where::
@@ -40,9 +40,8 @@ grammar, where::
 
 
 In human terms, the initramfs buffer contains a collection of
-compressed and/or uncompressed cpio archives (in the "newc" or "crc"
-formats); arbitrary amounts zero bytes (for padding) can be added
-between members.
+compressed and/or uncompressed cpio archives (in the "newc" format),
+with arbitrary amount of zero-byte padding between members.
 
 The cpio "TRAILER!!!" entry (cpio end-of-archive) is optional, but is
 not ignored; see "handling of hard links" below.
@@ -55,7 +54,7 @@ by the ASCII string "000012ac"):
 ============= ================== ==============================================
 Field name    Field size	 Meaning
 ============= ================== ==============================================
-c_magic	      6 bytes		 The string "070701" or "070702"
+c_magic	      6 bytes		 The string "070701"
 c_ino	      8 bytes		 File inode number
 c_mode	      8 bytes		 File mode and permissions
 c_uid	      8 bytes		 File uid
@@ -68,8 +67,7 @@ c_min	      8 bytes		 Minor part of file device number
 c_rmaj	      8 bytes		 Major part of device node reference
 c_rmin	      8 bytes		 Minor part of device node reference
 c_namesize    8 bytes		 Length of filename, including final \0
-c_chksum      8 bytes		 Checksum of data field if c_magic is 070702;
-				 otherwise zero
+c_chksum      8 bytes		 Ignored; reserved for unsupported "crc" format
 ============= ================== ==============================================
 
 The c_mode field matches the contents of st_mode returned by stat(2)
@@ -78,12 +76,6 @@ on Linux, and encodes the file type and file permissions.
 The c_filesize should be zero for any file which is not a regular file
 or symlink.
 
-The c_chksum field contains a simple 32-bit unsigned sum of all the
-bytes in the data field.  cpio(1) refers to this as "crc", which is
-clearly incorrect (a cyclic redundancy check is a different and
-significantly stronger integrity check), however, this is the
-algorithm used.
-
 If the filename is "TRAILER!!!" this is actually an end-of-archive
 marker; the c_filesize for an end-of-archive marker must be zero.
 
-- 
2.26.2

Re: [PATCH RESEND 1/3] initramfs: move unnecessary memcmp from hot path

[David Disseldorp]

Ping, any feedback on this change?

I think it's a no brainer, but for kicks I ran a few unrealistic micro
benchmarks on my laptop. Extraction time for a cpio image with 1M+
directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged
across 20 runs of:
  qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
        -kernel ~/linux/arch/x86/boot/bzImage \
        -initrd ./initrds/gen_cpio.out \
        -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
        | awk '/Trying to unpack rootfs/ {start_ts = $2};
               /Freeing initrd memory/ {end_ts = $2}
               END {printf "%f\n", end_ts - start_ts}'

Cheers, David

On Wed, 21 Jul 2021 13:51:51 +0200, David Disseldorp wrote:

> do_header() is called for each cpio entry and first checks for "newc"
> magic before parsing further. The magic check includes a special case
> error message if POSIX.1 ASCII (cpio -H odc) magic is detected. This
> special case POSIX.1 check needn't be done in the hot path, so move it
> under the non-newc-magic error path.
> 
> Signed-off-by: David Disseldorp <ddiss@suse.de>
> ---
>  init/initramfs.c | 9 ++++-----
>  1 file changed, 4 insertions(+), 5 deletions(-)
> 
> diff --git a/init/initramfs.c b/init/initramfs.c
> index af27abc59643..f01590cefa2d 100644
> --- a/init/initramfs.c
> +++ b/init/initramfs.c
> @@ -256,12 +256,11 @@ static int __init do_collect(void)
>  
>  static int __init do_header(void)
>  {
> -	if (memcmp(collected, "070707", 6)==0) {
> -		error("incorrect cpio method used: use -H newc option");
> -		return 1;
> -	}
>  	if (memcmp(collected, "070701", 6)) {
> -		error("no cpio magic");
> +		if (memcmp(collected, "070707", 6) == 0)
> +			error("incorrect cpio method used: use -H newc option");
> +		else
> +			error("no cpio magic");
>  		return 1;
>  	}
>  	parse_header(collected);

Re: [PATCH RESEND 1/3] initramfs: move unnecessary memcmp from hot path

[Al Viro]

On Wed, Aug 04, 2021 at 11:31:29AM +0200, David Disseldorp wrote:
> Ping, any feedback on this change?
> 
> I think it's a no brainer, but for kicks I ran a few unrealistic micro
> benchmarks on my laptop. Extraction time for a cpio image with 1M+
> directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged
> across 20 runs of:
>   qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
>         -kernel ~/linux/arch/x86/boot/bzImage \
>         -initrd ./initrds/gen_cpio.out \
>         -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
>         | awk '/Trying to unpack rootfs/ {start_ts = $2};
>                /Freeing initrd memory/ {end_ts = $2}
>                END {printf "%f\n", end_ts - start_ts}'

What was the dispersion for those runs?

Re: [PATCH RESEND 1/3] initramfs: move unnecessary memcmp from hot path

[David Disseldorp]

On Wed, 4 Aug 2021 12:57:16 +0000, Al Viro wrote:

> On Wed, Aug 04, 2021 at 11:31:29AM +0200, David Disseldorp wrote:
> > Ping, any feedback on this change?
> > 
> > I think it's a no brainer, but for kicks I ran a few unrealistic micro
> > benchmarks on my laptop. Extraction time for a cpio image with 1M+
> > directories improved by 5ms (pre: 14.614s, post: 14.609s), when averaged
> > across 20 runs of:
> >   qemu-system-x86_64 -machine accel=kvm -smp cpus=1 -m 10240 \
> >         -kernel ~/linux/arch/x86/boot/bzImage \
> >         -initrd ./initrds/gen_cpio.out \
> >         -append "initramfs_async=0 console=ttyS0 panic=0" -nographic \
> >         | awk '/Trying to unpack rootfs/ {start_ts = $2};
> >                /Freeing initrd memory/ {end_ts = $2}
> >                END {printf "%f\n", end_ts - start_ts}'  
> 
> What was the dispersion for those runs?

Too high for the 5ms to be considered statistically significant. Std
deviations were pre: 171ms, post: 214ms... <sigh> I'll redo this on a
proper test rig.

Cheers, David