[PATCH 0/2] Add uncompressed Linux banner to s390 bzImage

[PATCH 0/2] Add uncompressed Linux banner to s390 bzImage

[Petr Tesarik]

These patches make it easy to determine the kernel version from a
compressed binary by scanning for the Linux banner string in the
uncompressed portion of bzImage.

Petr Tesarik (2):
  init: Separate banner from init_uts_ns
  s390: add Linux banner to the compressed image

 arch/s390/boot/compressed/Makefile |  1 +
 init/Makefile                      |  2 +-
 init/banner.c                      | 21 +++++++++++++++++++++
 init/version.c                     | 10 ----------
 4 files changed, 23 insertions(+), 11 deletions(-)
 create mode 100644 init/banner.c

-- 
2.16.4

[PATCH 1/2] init: Separate banner from init_uts_ns

[Petr Tesarik]

The Linux banner is self-contained, so it could be theoretically
added to separately linked binaries (e.g. the kernel decompressor).
Unfortunately, it lives in the same object file as init_uts_ns,
which needs additional symbols from other object files.

Let's divorce the two.

Signed-off-by: Petr Tesarik <ptesarik@suse.com>
---
 init/Makefile  |  2 +-
 init/banner.c  | 21 +++++++++++++++++++++
 init/version.c | 10 ----------
 3 files changed, 22 insertions(+), 11 deletions(-)
 create mode 100644 init/banner.c

diff --git a/init/Makefile b/init/Makefile
index a3e5ce2bcf08..5009ea808ce4 100644
--- a/init/Makefile
+++ b/init/Makefile
@@ -5,7 +5,7 @@
 
 ccflags-y := -fno-function-sections -fno-data-sections
 
-obj-y                          := main.o version.o mounts.o
+obj-y                          := main.o version.o banner.o mounts.o
 ifneq ($(CONFIG_BLK_DEV_INITRD),y)
 obj-y                          += noinitramfs.o
 else
diff --git a/init/banner.c b/init/banner.c
new file mode 100644
index 000000000000..653ee081d474
--- /dev/null
+++ b/init/banner.c
@@ -0,0 +1,21 @@
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ *  linux/init/banner.c
+ *
+ *  Copyright (C) 1992  Theodore Ts'o
+ *
+ *  May be freely distributed as part of Linux.
+ */
+
+#include <generated/compile.h>
+#include <generated/utsrelease.h>
+
+/* FIXED STRINGS! Don't touch! */
+const char linux_banner[] =
+	"Linux version " UTS_RELEASE " (" LINUX_COMPILE_BY "@"
+	LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION "\n";
+
+const char linux_proc_banner[] =
+	"%s version %s"
+	" (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ")"
+	" (" LINUX_COMPILER ") %s\n";
diff --git a/init/version.c b/init/version.c
index cba341161b58..989cf9f9fe2c 100644
--- a/init/version.c
+++ b/init/version.c
@@ -42,14 +42,4 @@ struct uts_namespace init_uts_ns = {
 };
 EXPORT_SYMBOL_GPL(init_uts_ns);
 
-/* FIXED STRINGS! Don't touch! */
-const char linux_banner[] =
-	"Linux version " UTS_RELEASE " (" LINUX_COMPILE_BY "@"
-	LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION "\n";
-
-const char linux_proc_banner[] =
-	"%s version %s"
-	" (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ")"
-	" (" LINUX_COMPILER ") %s\n";
-
 BUILD_SALT;
-- 
2.16.4

[PATCH 2/2] s390: add Linux banner to the compressed image

[Petr Tesarik]

Various tools determine the kernel version from a given binary by
scanning for the Linux banner string. This does not work if the
banner string is compressed, but we can link it once more into the
uncompressed portion of bzImage.

Signed-off-by: Petr Tesarik <ptesarik@suse.com>
---
 arch/s390/boot/compressed/Makefile | 1 +
 1 file changed, 1 insertion(+)

diff --git a/arch/s390/boot/compressed/Makefile b/arch/s390/boot/compressed/Makefile
index fa529c5b4486..9bc4685477c5 100644
--- a/arch/s390/boot/compressed/Makefile
+++ b/arch/s390/boot/compressed/Makefile
@@ -11,6 +11,7 @@ UBSAN_SANITIZE := n
 KASAN_SANITIZE := n
 
 obj-y	:= $(if $(CONFIG_KERNEL_UNCOMPRESSED),,decompressor.o) piggy.o info.o
+obj-y   += ../../../../init/banner.o
 targets	:= vmlinux.lds vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2
 targets += vmlinux.bin.xz vmlinux.bin.lzma vmlinux.bin.lzo vmlinux.bin.lz4
 targets += info.bin $(obj-y)
-- 
2.16.4

Re: [PATCH 2/2] s390: add Linux banner to the compressed image

[Vasily Gorbik]

On Fri, Jul 12, 2019 at 07:21:01PM +0200, Petr Tesarik wrote:
> Various tools determine the kernel version from a given binary by
> scanning for the Linux banner string. This does not work if the
> banner string is compressed, but we can link it once more into the
> uncompressed portion of bzImage.
> 
> Signed-off-by: Petr Tesarik <ptesarik@suse.com>
> ---
>  arch/s390/boot/compressed/Makefile | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/arch/s390/boot/compressed/Makefile b/arch/s390/boot/compressed/Makefile
> index fa529c5b4486..9bc4685477c5 100644
> --- a/arch/s390/boot/compressed/Makefile
> +++ b/arch/s390/boot/compressed/Makefile
> @@ -11,6 +11,7 @@ UBSAN_SANITIZE := n
>  KASAN_SANITIZE := n
>  
>  obj-y	:= $(if $(CONFIG_KERNEL_UNCOMPRESSED),,decompressor.o) piggy.o info.o
> +obj-y   += ../../../../init/banner.o

We don't reuse objects from another build stage, we rebuild them with
distinct decompressor's build flags.
$ git grep "ctype.[oc]" -- arch/s390/boot
arch/s390/boot/Makefile:obj-y   += ctype.o text_dma.o
arch/s390/boot/ctype.c:#include "../../../lib/ctype.c"

Besides that, there is a special CONFIG_KERNEL_UNCOMPRESSED mode, with
which "strings vmlinuz | grep 'Linux version'" I assume you are using
would still yield result. Adding the second version of banner would
produce duplicated result in this case.

But even before discussing solutions I would like to understand the
problem first. Which specific tools are you referring to? What are they
good for? And how do they get the kernel version from other architectures
compressed images?

Re: [PATCH 2/2] s390: add Linux banner to the compressed image

[Petr Tesarik]

On Sun, 14 Jul 2019 16:35:33 +0200
Vasily Gorbik <gor@linux.ibm.com> wrote:

> On Fri, Jul 12, 2019 at 07:21:01PM +0200, Petr Tesarik wrote:
> > Various tools determine the kernel version from a given binary by
> > scanning for the Linux banner string. This does not work if the
> > banner string is compressed, but we can link it once more into the
> > uncompressed portion of bzImage.
> > 
> > Signed-off-by: Petr Tesarik <ptesarik@suse.com>
> > ---
> >  arch/s390/boot/compressed/Makefile | 1 +
> >  1 file changed, 1 insertion(+)
> > 
> > diff --git a/arch/s390/boot/compressed/Makefile b/arch/s390/boot/compressed/Makefile
> > index fa529c5b4486..9bc4685477c5 100644
> > --- a/arch/s390/boot/compressed/Makefile
> > +++ b/arch/s390/boot/compressed/Makefile
> > @@ -11,6 +11,7 @@ UBSAN_SANITIZE := n
> >  KASAN_SANITIZE := n
> >  
> >  obj-y	:= $(if $(CONFIG_KERNEL_UNCOMPRESSED),,decompressor.o) piggy.o info.o
> > +obj-y   += ../../../../init/banner.o  
> 
> We don't reuse objects from another build stage, we rebuild them with
> distinct decompressor's build flags.
> $ git grep "ctype.[oc]" -- arch/s390/boot
> arch/s390/boot/Makefile:obj-y   += ctype.o text_dma.o
> arch/s390/boot/ctype.c:#include "../../../lib/ctype.c"

Those flags do not make a difference for a simple object file like the
Linux banner, but I get your point, and I cannot see an issues with
rebuilding the banner for the decompressor.

> Besides that, there is a special CONFIG_KERNEL_UNCOMPRESSED mode, with
> which "strings vmlinuz | grep 'Linux version'" I assume you are using
> would still yield result. Adding the second version of banner would
> produce duplicated result in this case.

Sure, and AFAICT that's not a problem. But the point here is that the
production kernel should be compressed for all those well-known
reasons, but such image is then not recognized.

> But even before discussing solutions I would like to understand the
> problem first. Which specific tools are you referring to? What are they
> good for? And how do they get the kernel version from other architectures
> compressed images?

The tool I'm aware of is called get_kernel_version. It's built as part
of openSUSE aaa_base and is used at install time. I'm not quite sure
how it is used, but I have added Raymund Will to Cc; he can provide
more information. There's also an open bug for it:

  https://bugzilla.opensuse.org/show_bug.cgi?id=1139939

As for other architectures, I only know about x86. The x86 compressed
binary contains a header with various pointers, among them a pointer to
the kernel version string, so it must be added to the decompressor (cf.
arch/x86/boot/version.c).

If you prefer to have a similar header for other architectures, then
that would be fine with me, but it's a bit more involved, because it
would set up a new ABI...

HTH,
Petr T

Re: [PATCH 2/2] s390: add Linux banner to the compressed image

[Vasily Gorbik]

On Sun, Jul 14, 2019 at 03:52:52PM +0000, Petr Tesarik wrote:
> On Sun, 14 Jul 2019 16:35:33 +0200
> Vasily Gorbik <gor@linux.ibm.com> wrote:
> 
> > On Fri, Jul 12, 2019 at 07:21:01PM +0200, Petr Tesarik wrote:
> > > Various tools determine the kernel version from a given binary by
> > > scanning for the Linux banner string. This does not work if the
> > > banner string is compressed, but we can link it once more into the
> > > uncompressed portion of bzImage.
> 
> > But even before discussing solutions I would like to understand the
> > problem first. Which specific tools are you referring to? What are they
> > good for? And how do they get the kernel version from other architectures
> > compressed images?
> 
> The tool I'm aware of is called get_kernel_version. It's built as part
> of openSUSE aaa_base and is used at install time. I'm not quite sure
> how it is used, but I have added Raymund Will to Cc; he can provide
> more information. There's also an open bug for it:
> 
>   https://bugzilla.opensuse.org/show_bug.cgi?id=1139939

Oh, I see, found it, thanks. Very interesting tool.
https://github.com/openSUSE/aaa_base/blob/master/get_kernel_version.c

And the only usage of this tool I found is to get the kernel version of
/boot/image (on s390) to run depmod during
yast-installation/src/clients/network_finish.rb

I also see that queries to rpm are already done from
yast-yast2/library/system/src/modules/Kernel.rb
Wouldn't it be more reliable (and portable) to just get the kernel
version from rpm metadata? Without using unreliable tools? Or find some
other solution, since this is the only use case for the tool?
$ rpm -qf --qf '%{VERSION}-%{RELEASE}.%{ARCH}\n' /boot/vmlinuz-5.1.17-300.fc30.x86_64
5.1.17-300.fc30.x86_64
[it looks like openSUSE kernel rpms don't have metadata to reconstruct
full kernel version currently, but that could be improved?]

Anyhow, I'm not opposed to an idea to make it possible to detect the
kernel version from bzImage. But it should be reliable. So, see the
follow on patch I'm sending.

[PATCH] s390: enable detection of kernel version from bzImage

[Vasily Gorbik]

Extend "parmarea" to include an offset of the version string, which is
stored as 8-byte big endian value.

To retrieve version string from bzImage reliably, one should check the
presence of "S390EP" ascii string at 0x10008 (available since v3.2),
then read the version string offset from 0x10428 (which has been 0
since v3.2 up to now). The string is null terminated.

Could be retrieved with the following "file" command magic (requires
file v5.34):
8 string \x02\x00\x00\x18\x60\x00\x00\x50\x02\x00\x00\x68\x60\x00\x00\x50\x40\x40\x40\x40\x40\x40\x40\x40 Linux S390
>0x10008       string          S390EP
>>0x10428      bequad          >0
>>>(0x10428.Q) string          >\0             \b, version %s

Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
---
 arch/s390/boot/Makefile       | 2 +-
 arch/s390/boot/head.S         | 1 +
 arch/s390/boot/version.c      | 6 ++++++
 arch/s390/include/asm/setup.h | 4 +++-
 4 files changed, 11 insertions(+), 2 deletions(-)
 create mode 100644 arch/s390/boot/version.c

diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
index 7cba96e7587b..4cf0bddb7d92 100644
--- a/arch/s390/boot/Makefile
+++ b/arch/s390/boot/Makefile
@@ -36,7 +36,7 @@ CFLAGS_sclp_early_core.o += -I$(srctree)/drivers/s390/char
 
 obj-y	:= head.o als.o startup.o mem_detect.o ipl_parm.o ipl_report.o
 obj-y	+= string.o ebcdic.o sclp_early_core.o mem.o ipl_vmparm.o cmdline.o
-obj-y	+= ctype.o text_dma.o
+obj-y	+= version.o ctype.o text_dma.o
 obj-$(CONFIG_PROTECTED_VIRTUALIZATION_GUEST)	+= uv.o
 obj-$(CONFIG_RELOCATABLE)	+= machine_kexec_reloc.o
 obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
diff --git a/arch/s390/boot/head.S b/arch/s390/boot/head.S
index 028aab03a9e7..2087bed6e60f 100644
--- a/arch/s390/boot/head.S
+++ b/arch/s390/boot/head.S
@@ -361,6 +361,7 @@ ENTRY(startup_kdump)
 	.quad	0			# INITRD_SIZE
 	.quad	0			# OLDMEM_BASE
 	.quad	0			# OLDMEM_SIZE
+	.quad	kernel_version		# points to kernel version string
 
 	.org	COMMAND_LINE
 	.byte	"root=/dev/ram0 ro"
diff --git a/arch/s390/boot/version.c b/arch/s390/boot/version.c
new file mode 100644
index 000000000000..ea5e49651931
--- /dev/null
+++ b/arch/s390/boot/version.c
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <generated/utsrelease.h>
+#include <generated/compile.h>
+
+const char kernel_version[] = UTS_RELEASE
+	" (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ") " UTS_VERSION;
diff --git a/arch/s390/include/asm/setup.h b/arch/s390/include/asm/setup.h
index 925889d360c1..e5d28a475f76 100644
--- a/arch/s390/include/asm/setup.h
+++ b/arch/s390/include/asm/setup.h
@@ -54,6 +54,7 @@
 #define INITRD_SIZE_OFFSET	0x10410
 #define OLDMEM_BASE_OFFSET	0x10418
 #define OLDMEM_SIZE_OFFSET	0x10420
+#define KERNEL_VERSION_OFFSET	0x10428
 #define COMMAND_LINE_OFFSET	0x10480
 
 #ifndef __ASSEMBLY__
@@ -74,7 +75,8 @@ struct parmarea {
 	unsigned long initrd_size;			/* 0x10410 */
 	unsigned long oldmem_base;			/* 0x10418 */
 	unsigned long oldmem_size;			/* 0x10420 */
-	char pad1[0x10480 - 0x10428];			/* 0x10428 - 0x10480 */
+	unsigned long kernel_version;			/* 0x10428 */
+	char pad1[0x10480 - 0x10430];			/* 0x10430 - 0x10480 */
 	char command_line[ARCH_COMMAND_LINE_SIZE];	/* 0x10480 */
 };
 
-- 
2.21.0

Re: [PATCH] s390: enable detection of kernel version from bzImage

[Petr Tesarik]

On Tue, 16 Jul 2019 00:12:19 +0200
Vasily Gorbik <gor@linux.ibm.com> wrote:

> Extend "parmarea" to include an offset of the version string, which is
> stored as 8-byte big endian value.
> 
> To retrieve version string from bzImage reliably, one should check the
> presence of "S390EP" ascii string at 0x10008 (available since v3.2),
> then read the version string offset from 0x10428 (which has been 0
> since v3.2 up to now). The string is null terminated.
> 
> Could be retrieved with the following "file" command magic (requires
> file v5.34):
> 8 string \x02\x00\x00\x18\x60\x00\x00\x50\x02\x00\x00\x68\x60\x00\x00\x50\x40\x40\x40\x40\x40\x40\x40\x40 Linux S390
> >0x10008       string          S390EP  
> >>0x10428      bequad          >0  
> >>>(0x10428.Q) string          >\0             \b, version %s  
> 
> Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>

This looks great! Much cleaner than the original approach.

Thank you,
Petr T

> ---
>  arch/s390/boot/Makefile       | 2 +-
>  arch/s390/boot/head.S         | 1 +
>  arch/s390/boot/version.c      | 6 ++++++
>  arch/s390/include/asm/setup.h | 4 +++-
>  4 files changed, 11 insertions(+), 2 deletions(-)
>  create mode 100644 arch/s390/boot/version.c
> 
> diff --git a/arch/s390/boot/Makefile b/arch/s390/boot/Makefile
> index 7cba96e7587b..4cf0bddb7d92 100644
> --- a/arch/s390/boot/Makefile
> +++ b/arch/s390/boot/Makefile
> @@ -36,7 +36,7 @@ CFLAGS_sclp_early_core.o += -I$(srctree)/drivers/s390/char
>  
>  obj-y	:= head.o als.o startup.o mem_detect.o ipl_parm.o ipl_report.o
>  obj-y	+= string.o ebcdic.o sclp_early_core.o mem.o ipl_vmparm.o cmdline.o
> -obj-y	+= ctype.o text_dma.o
> +obj-y	+= version.o ctype.o text_dma.o
>  obj-$(CONFIG_PROTECTED_VIRTUALIZATION_GUEST)	+= uv.o
>  obj-$(CONFIG_RELOCATABLE)	+= machine_kexec_reloc.o
>  obj-$(CONFIG_RANDOMIZE_BASE)	+= kaslr.o
> diff --git a/arch/s390/boot/head.S b/arch/s390/boot/head.S
> index 028aab03a9e7..2087bed6e60f 100644
> --- a/arch/s390/boot/head.S
> +++ b/arch/s390/boot/head.S
> @@ -361,6 +361,7 @@ ENTRY(startup_kdump)
>  	.quad	0			# INITRD_SIZE
>  	.quad	0			# OLDMEM_BASE
>  	.quad	0			# OLDMEM_SIZE
> +	.quad	kernel_version		# points to kernel version string
>  
>  	.org	COMMAND_LINE
>  	.byte	"root=/dev/ram0 ro"
> diff --git a/arch/s390/boot/version.c b/arch/s390/boot/version.c
> new file mode 100644
> index 000000000000..ea5e49651931
> --- /dev/null
> +++ b/arch/s390/boot/version.c
> @@ -0,0 +1,6 @@
> +// SPDX-License-Identifier: GPL-2.0
> +#include <generated/utsrelease.h>
> +#include <generated/compile.h>
> +
> +const char kernel_version[] = UTS_RELEASE
> +	" (" LINUX_COMPILE_BY "@" LINUX_COMPILE_HOST ") " UTS_VERSION;
> diff --git a/arch/s390/include/asm/setup.h b/arch/s390/include/asm/setup.h
> index 925889d360c1..e5d28a475f76 100644
> --- a/arch/s390/include/asm/setup.h
> +++ b/arch/s390/include/asm/setup.h
> @@ -54,6 +54,7 @@
>  #define INITRD_SIZE_OFFSET	0x10410
>  #define OLDMEM_BASE_OFFSET	0x10418
>  #define OLDMEM_SIZE_OFFSET	0x10420
> +#define KERNEL_VERSION_OFFSET	0x10428
>  #define COMMAND_LINE_OFFSET	0x10480
>  
>  #ifndef __ASSEMBLY__
> @@ -74,7 +75,8 @@ struct parmarea {
>  	unsigned long initrd_size;			/* 0x10410 */
>  	unsigned long oldmem_base;			/* 0x10418 */
>  	unsigned long oldmem_size;			/* 0x10420 */
> -	char pad1[0x10480 - 0x10428];			/* 0x10428 - 0x10480 */
> +	unsigned long kernel_version;			/* 0x10428 */
> +	char pad1[0x10480 - 0x10430];			/* 0x10430 - 0x10480 */
>  	char command_line[ARCH_COMMAND_LINE_SIZE];	/* 0x10480 */
>  };
>  

Re: [PATCH] s390: enable detection of kernel version from bzImage

[Vasily Gorbik]

On Tue, Jul 16, 2019 at 10:30:14AM +0000, Petr Tesarik wrote:
> On Tue, 16 Jul 2019 00:12:19 +0200
> Vasily Gorbik <gor@linux.ibm.com> wrote:
> 
> > Extend "parmarea" to include an offset of the version string, which is
> > stored as 8-byte big endian value.
> > 
> > To retrieve version string from bzImage reliably, one should check the
> > presence of "S390EP" ascii string at 0x10008 (available since v3.2),
> > then read the version string offset from 0x10428 (which has been 0
> > since v3.2 up to now). The string is null terminated.
> > 
> > Could be retrieved with the following "file" command magic (requires
> > file v5.34):
> > 8 string \x02\x00\x00\x18\x60\x00\x00\x50\x02\x00\x00\x68\x60\x00\x00\x50\x40\x40\x40\x40\x40\x40\x40\x40 Linux S390
> > >0x10008       string          S390EP  
> > >>0x10428      bequad          >0  
> > >>>(0x10428.Q) string          >\0             \b, version %s  
> > 
> > Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>
> 
> This looks great! Much cleaner than the original approach.
> 
> Thank you,
> Petr T

Then I'll add
Reported-by: Petr Tesarik <ptesarik@suse.com>
Suggested-by: Petr Tesarik <ptesarik@suse.com>
if you don't mind and try to queue that for 5.3.

Re: [PATCH] s390: enable detection of kernel version from bzImage

[Petr Tesarik]

[-- Attachment #1: Type: text/plain, Size: 1475 bytes --]

On Tue, 16 Jul 2019 15:11:38 +0200
Vasily Gorbik <gor@linux.ibm.com> wrote:

> On Tue, Jul 16, 2019 at 10:30:14AM +0000, Petr Tesarik wrote:
> > On Tue, 16 Jul 2019 00:12:19 +0200
> > Vasily Gorbik <gor@linux.ibm.com> wrote:
> >   
> > > Extend "parmarea" to include an offset of the version string, which is
> > > stored as 8-byte big endian value.
> > > 
> > > To retrieve version string from bzImage reliably, one should check the
> > > presence of "S390EP" ascii string at 0x10008 (available since v3.2),
> > > then read the version string offset from 0x10428 (which has been 0
> > > since v3.2 up to now). The string is null terminated.
> > > 
> > > Could be retrieved with the following "file" command magic (requires
> > > file v5.34):
> > > 8 string \x02\x00\x00\x18\x60\x00\x00\x50\x02\x00\x00\x68\x60\x00\x00\x50\x40\x40\x40\x40\x40\x40\x40\x40 Linux S390  
> > > >0x10008       string          S390EP    
> > > >>0x10428      bequad          >0    
> > > >>>(0x10428.Q) string          >\0             \b, version %s    
> > > 
> > > Signed-off-by: Vasily Gorbik <gor@linux.ibm.com>  
> > 
> > This looks great! Much cleaner than the original approach.
> > 
> > Thank you,
> > Petr T  
> 
> Then I'll add
> Reported-by: Petr Tesarik <ptesarik@suse.com>
> Suggested-by: Petr Tesarik <ptesarik@suse.com>
> if you don't mind and try to queue that for 5.3.

Oh, sure, please add these lines and go ahead.

Thank you again,
Petr T

[-- Attachment #2: Digitální podpis OpenPGP --]
[-- Type: application/pgp-signature, Size: 488 bytes --]