'make O=' indigestion with module signing

'make O=' indigestion with module signing

[Valdis Klētnieks]

[-- Attachment #1: Type: text/plain, Size: 1771 bytes --]

So, I tried doing a 'make O=... allmodconfig', with a setup where the uid of
the build process had write permission to the O= directory, but intentionally
did *not* have write permission to the source tree (so they couldn't mess up
the tree - I got tired of having to repeatedly do 'make mrproper' because of
pilot error)

allmodconfig gave me a .config that had:

CONFIG_MODULE_SIG_FORMAT=y
CONFIG_MODULE_SIG=y
CONFIG_MODULE_SIG_FORCE=y
CONFIG_MODULE_SIG_ALL=y
CONFIG_MODULE_SIG_SHA1=y
# CONFIG_MODULE_SIG_SHA224 is not set
# CONFIG_MODULE_SIG_SHA256 is not set
# CONFIG_MODULE_SIG_SHA384 is not set
# CONFIG_MODULE_SIG_SHA512 is not set
CONFIG_MODULE_SIG_HASH="sha1"
CONFIG_IMA_APPRAISE_REQUIRE_MODULE_SIGS=y
CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"

What i *expected* was that multiple builds with different O= would each
generate themselves a unique signing key and put it in their own O= directory
and stay out of each other's way.

What actually happened:

  EXTRACT_CERTS   /usr/src/linux-next/"certs/signing_key.pem"
At main.c:142:
- SSL error:0200100D:system library:fopen:Permission denied: ../crypto/bio/bss_file.c:69
- SSL error:2006D002:BIO routines:BIO_new_file:system lib: ../crypto/bio/bss_file.c:78
extract-cert: /usr/src/linux-next/certs/signing_key.pem: Permission denied
make[2]: *** [/usr/src/linux-next/certs/Makefile:106: certs/signing_key.x509] Error 1
make[1]: *** [/usr/src/linux-next/Makefile:1847: certs] Error 2
make[1]: Leaving directory '/usr/src/linux-next/out/arm64'
make: *** [Makefile:215: __sub-make] Error 2

It tried to put the key into the source tree rather than the build tree.

Before I try to code up a fix for this, is this intentionally designed
behavior, or have I just managed to trip over a rarely-tested corner case?

[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]

Re: 'make O=' indigestion with module signing

[David Howells]

Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

> What i *expected* was that multiple builds with different O= would each
> generate themselves a unique signing key and put it in their own O= directory
> and stay out of each other's way.

Hmmm...  Works for me.  I use separate build dirs all the time.

What version of the kernel are you using and what's the build command line -
in particular the full O= option?

David

Re: 'make O=' indigestion with module signing

[Valdis Klētnieks]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=us-ascii, Size: 1602 bytes --]

On Thu, 11 Mar 2021 09:34:01 +0000, David Howells said:
> Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
>
> > What i *expected* was that multiple builds with different O= would each
> > generate themselves a unique signing key and put it in their own O= directory
> > and stay out of each other's way.
>
> Hmmm...  Works for me.  I use separate build dirs all the time.
>
> What version of the kernel are you using and what's the build command line -
> in particular the full O= option?

This is linux-next as of yesterday. For testing, I've been using:

LANG=C make O=/tmp/arm64 V=1 ARCH=arm64 ASFLAGS='-mcpu=all' CROSS_COMPILE=/opt/aarch64/bin/aarch64-linux-gnu- certs/

and it insists on trying to make the certs in /usr/src/linux-next rather than /tmp/arm64:

make -f /usr/src/linux-next/scripts/Makefile.build obj=certs \
single-build= \
need-builtin=1 need-modorder=1
  scripts/extract-cert /usr/src/linux-next/"certs/signing_key.pem" certs/signing_key.x509
At main.c:142:
- SSL error:0200100D:system library:fopen:Permission denied: ../crypto/bio/bss_file.c:69
- SSL error:2006D002:BIO routines:BIO_new_file:system lib: ../crypto/bio/bss_file.c:78
extract-cert: /usr/src/linux-next/certs/signing_key.pem: Permission denied
make[2]: *** [/usr/src/linux-next/certs/Makefile:106: certs/signing_key.x509] Error 1
make[1]: *** [/usr/src/linux-next/Makefile:1847: certs] Error 2
make[1]: Leaving directory '/tmp/arm64'
make: *** [Makefile:215: __sub-make] Error 2


Is it possible that it works for you because although you have
separate build dirs, it's still able to write to the source tree?


[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]

Re: 'make O=' indigestion with module signing

[David Howells]

I wonder...  Can you grab branch keys-cve-2020-26541-branch from:

	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/

and try that?  If that breaks, can you try dropping the top four commits?

David

Re: 'make O=' indigestion with module signing

[Valdis Klētnieks]

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

On Thu, 11 Mar 2021 10:49:14 +0000, David Howells said:
> I wonder...  Can you grab branch keys-cve-2020-26541-branch from:
>
> 	https://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs.git/
>
> and try that?  If that breaks, can you try dropping the top four commits?

[/usr/src/linux-next] git checkout keys-cve-2020-26541-branch
Checking out files: 100% (13062/13062), done.
Previous HEAD position was b01d57bfdc41 Add linux-next specific files for 20210310
Branch 'keys-cve-2020-26541-branch' set up to track remote branch 'keys-cve-2020-26541-branch' from 'linux-fs'.
Switched to a new branch 'keys-cve-2020-26541-branch'

That still didn't work, and dropping off the 4 commits from Eric Snowberg
didn't change things.

I checked out next-20210310, did a 'make mrproper', and tested as the
owner of the source tree rather than as a different user...

LANG=C make O=/tmp/test-as-owner  V=1 ARCH=arm64 ASFLAGS='-mcpu=all' CROSS_COMPILE=/opt/aarch64/bin/aarch64-linux-gnu- certs/

make -f /usr/src/linux-next/scripts/Makefile.build obj=certs \
single-build= \
need-builtin=1 need-modorder=1
  scripts/extract-cert /usr/src/linux-next/"certs/signing_key.pem" certs/signing_key.x509
Extracted cert: /CN=Build time autogenerated kernel key
  /opt/aarch64/bin/aarch64-linux-gnu-gcc (...)  -o certs/system_keyring.o /usr/src/linux-next/certs/system_keyring.c

And the files ended up where they belonged:

 ls -l /tmp/test-as-owner/certs/
total 72
-rw-r--r-- 1 source source  1288 Mar 11 06:33 blacklist_nohashes.o
-rw-r--r-- 1 source source 18496 Mar 11 06:33 blacklist.o
-rw-r--r-- 1 source source   542 Mar 11 06:33 built-in.a
-rw-r--r-- 1 source source  5856 Mar 11 06:33 common.o
-rw-r--r-- 1 source source     0 Mar 11 06:33 modules.order
-rw-r--r-- 1 source source  1184 Mar 11 06:33 revocation_certificates.o
-rw-r--r-- 1 source source  1357 Mar 11 06:33 signing_key.x509
-rw-r--r-- 1 source source  6888 Mar 11 06:33 system_certificates.o
-rw-r--r-- 1 source source 17504 Mar 11 06:33 system_keyring.o
-rw-r--r-- 1 source source     0 Mar 11 06:33 x509_certificate_list
-rw-r--r-- 1 source source     0 Mar 11 06:33 x509_revocation_list

So there's something weird going on with scripts/extract-cert when running
as a userid other than the owner of the source tree..  I wonder if it's actually
an OpenSSL issue...

I'll look at it some more later today...

[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]

Re: 'make O=' indigestion with module signing

[David Howells]

Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

> So there's something weird going on with scripts/extract-cert when running
> as a userid other than the owner of the source tree..  I wonder if it's
> actually an OpenSSL issue...

I cloned next-20210311 as one user then built it as another user using:

	LANG=C nice -19 make O=/data/git/next-20210311-build -j8 allmodconfig
	LANG=C nice -19 make O=/data/git/next-20210311-build -j8

It built with no problems.  The building user definitely can't create/modify
files in the source directory.

Interestingly, the following line in the output from mine:

	EXTRACT_CERTS   certs/signing_key.pem

doesn't show the full path as it does in yours:

	EXTRACT_CERTS   /usr/src/linux-next/"certs/signing_key.pem"

but I don't know why.  There are some odd quotes in your line also which may
be related to the problem.  The relevant config line looks the same:

	CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"

I'll have to try with the aarch64 build, see if it's something in that that's
the problem.

David

Re: 'make O=' indigestion with module signing

[David Howells]

David Howells <dhowells@redhat.com> wrote:

> I'll have to try with the aarch64 build, see if it's something in that that's
> the problem.

That works too... kind of.  Building in the certs/ dir is fine - and all the
cert generation and extraction is done in the expected place, but the link
fails with errors like:

aarch64-linux-gnu-ld: arch/arm64/kernel/paravirt.o: relocation R_AARCH64_ABS32 against `__crc_pv_ops' can not be used when making a shared object
arch/arm64/kernel/paravirt.o:(__patchable_function_entries+0x0): dangerous relocation: unsupported relocation
arch/arm64/kernel/paravirt.o:(__patchable_function_entries+0x8): dangerous relocation: unsupported relocation
arch/arm64/kernel/paravirt.o:(__patchable_function_entries+0x10): dangerous relocation: unsupported relocation
arch/arm64/kernel/paravirt.o:(__patchable_function_entries+0x18): dangerous relocation: unsupported relocation
...

David

Re: 'make O=' indigestion with module signing

[Valdis Klētnieks]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

On Thu, 11 Mar 2021 12:04:19 +0000, David Howells said:

> 	EXTRACT_CERTS   /usr/src/linux-next/"certs/signing_key.pem"
>
> but I don't know why.  There are some odd quotes in your line also which may
> be related to the problem.  The relevant config line looks the same:
>
> 	CONFIG_MODULE_SIG_KEY="certs/signing_key.pem"

Aha.  I figured it out.

If you have a *totally* clean source tree, 'make -O' works for all users.
If you have in the past done a build in the tree, and then done a 'make mrproper'
to clean it out so 'make -O' doesn't complain, it fails because it
finds an *old* certs/signing_key.pem in /usr/src/linux-next and tries to
put the new generated files in the same directory.

So the root cause was: 'make mrproper doesn't clean certs/' out enough,
and this chunk of certs/Makefile

# If CONFIG_MODULE_SIG_KEY isn't a PKCS#11 URI, depend on it
ifeq ($(patsubst pkcs11:%,%,$(firstword $(MODULE_SIG_KEY_FILENAME))),$(firstword $(MODULE_SIG_KEY_FILENAME)))
X509_DEP := $(MODULE_SIG_KEY_SRCPREFIX)$(MODULE_SIG_KEY_FILENAME)
endif

MODULE_SIG_KEY_SRCPREFIX was where my /usr/src/linux-next was coming from...

I admit not being sure how (or if) this should be fixed




[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]

Re: 'make O=' indigestion with module signing

[David Howells]

Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

> So the root cause was: 'make mrproper doesn't clean certs/' out enough,
> and this chunk of certs/Makefile
> ...
> I admit not being sure how (or if) this should be fixed

It's tricky because CONFIG_MODULE_SIG_KEY may not point to a file, let alone a
file that was autogenerated - it can be given a PKCS#11 URI, for instance.  I
had to put in the autogeneration based on a magic config string value to stop
randconfig blowing up - but it only does the autogeneration if you don't put
in your own file there before building.

Possibly I can add something like:

	clean-files := signing_key.pem x509.genkey

inside the

	ifeq ($(CONFIG_MODULE_SIG_KEY),"certs/signing_key.pem")
	...
	endif

section.

David

Re: 'make O=' indigestion with module signing

[Valdis Klētnieks]

[-- Attachment #1: Type: text/plain, Size: 441 bytes --]

On Fri, 12 Mar 2021 09:01:36 +0000, David Howells said:

> Possibly I can add something like:
>
> 	clean-files := signing_key.pem x509.genkey
>
> inside the
>
> 	ifeq ($(CONFIG_MODULE_SIG_KEY),"certs/signing_key.pem")
> 	...
> 	endif

Would that remove them on a 'make clean', or only a 'make mrproper'?
The latter sounds like the correct solution to me, as the signing key should
have (roughly) the same lifetime rules as the .config file.

[-- Attachment #2: Type: application/pgp-signature, Size: 832 bytes --]

[PATCH] certs: Clean up signing_key.pem and x509.genkey on make mrproper

[David Howells]

Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:

> > Possibly I can add something like:
> >
> > 	clean-files := signing_key.pem x509.genkey
> >
> > inside the
> >
> > 	ifeq ($(CONFIG_MODULE_SIG_KEY),"certs/signing_key.pem")
> > 	...
> > 	endif
> 
> Would that remove them on a 'make clean', or only a 'make mrproper'?
> The latter sounds like the correct solution to me, as the signing key should
> have (roughly) the same lifetime rules as the .config file.

It would appear that it works on neither.  Neither of them seem to have any
CONFIG_xxx symbols set.

How about the attached patch?

David
---
commit 95897dc8dc13ad13c637a477a1ead9b63ff1fafa
Author: David Howells <dhowells@redhat.com>
Date:   Fri Mar 12 10:48:25 2021 +0000

    certs: Clean up signing_key.pem and x509.genkey on make mrproper
    
    Autogenerated signing_key.pem and x509.genkey files aren't removed from the
    build certs/ directory when "make mrproper" is run.  This is somewhat
    deliberate since the "file" is specified by the CONFIG_MODULE_SIG_KEY
    string option and may not be in the build tree - and may not even be a
    filename, but rather a PKCS#7 URI (also the config variables doesn't seem
    to be set when cleaning).
    
    Fix this by unconditionally listing signing_key.pem and x509.genkey for
    removal from the build certs/ directory - which will just do nothing if
    they're not there, and shouldn't remove signing keys that are configured to
    be elsewhere.
    
    Note that this will permanently erase the autogenerated private key, so
    anyone that is relying on it still being around after doing make mrproper
    will no longer find it.
    
    Fixes: cfc411e7fff3 ("Move certificate handling to its own directory")
    Reported-by: Valdis Klētnieks <valdis.kletnieks@vt.edu>
    Signed-off-by: David Howells <dhowells@redhat.com>
    Link: https://lore.kernel.org/r/134696.1615510534@turing-police/ [1]

diff --git a/certs/Makefile b/certs/Makefile
index f4c25b67aad9..2ae1dd518bc7 100644
--- a/certs/Makefile
+++ b/certs/Makefile
@@ -104,3 +104,5 @@ targets += signing_key.x509
 $(obj)/signing_key.x509: scripts/extract-cert $(X509_DEP) FORCE
 	$(call if_changed,extract_certs,$(MODULE_SIG_KEY_SRCPREFIX)$(CONFIG_MODULE_SIG_KEY))
 endif # CONFIG_MODULE_SIG
+
+clean-files += signing_key.pem x509.genkey

Re: 'make O=' indigestion with module signing

[Ingo Molnar]

* David Howells <dhowells@redhat.com> wrote:

> Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
> 
> > So the root cause was: 'make mrproper doesn't clean certs/' out enough,
> > and this chunk of certs/Makefile
> > ...
> > I admit not being sure how (or if) this should be fixed
> 
> It's tricky because CONFIG_MODULE_SIG_KEY may not point to a file, let alone a
> file that was autogenerated - it can be given a PKCS#11 URI, for instance.  I
> had to put in the autogeneration based on a magic config string value to stop
> randconfig blowing up - but it only does the autogeneration if you don't put
> in your own file there before building.
> 
> Possibly I can add something like:
> 
> 	clean-files := signing_key.pem x509.genkey
> 
> inside the
> 
> 	ifeq ($(CONFIG_MODULE_SIG_KEY),"certs/signing_key.pem")
> 	...
> 	endif
> 
> section.

Note that this bug is now upstream in Linus's tree and has broken
the deb-pkg distro kernel build method.

Would be nice to make 'make mrproper' work again.

Thanks,

	Ingo

Re: 'make O=' indigestion with module signing

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> 
> * David Howells <dhowells@redhat.com> wrote:
> 
> > Valdis Klētnieks <valdis.kletnieks@vt.edu> wrote:
> > 
> > > So the root cause was: 'make mrproper doesn't clean certs/' out enough,
> > > and this chunk of certs/Makefile
> > > ...
> > > I admit not being sure how (or if) this should be fixed
> > 
> > It's tricky because CONFIG_MODULE_SIG_KEY may not point to a file, let alone a
> > file that was autogenerated - it can be given a PKCS#11 URI, for instance.  I
> > had to put in the autogeneration based on a magic config string value to stop
> > randconfig blowing up - but it only does the autogeneration if you don't put
> > in your own file there before building.
> > 
> > Possibly I can add something like:
> > 
> > 	clean-files := signing_key.pem x509.genkey
> > 
> > inside the
> > 
> > 	ifeq ($(CONFIG_MODULE_SIG_KEY),"certs/signing_key.pem")
> > 	...
> > 	endif
> > 
> > section.
> 
> Note that this bug is now upstream in Linus's tree and has broken
> the deb-pkg distro kernel build method.
> 
> Would be nice to make 'make mrproper' work again.

Correction - there appears to be something else going on, but the
error messages are similar:

  At main.c:291:
  - SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
  - SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
  sign-file: debian/linux-image/lib/modules/5.12.0-custom/kernel/arch/x86/crypto/aegis128-aesni.ko: No such file or directory
  At main.c:291:

The error messages look pretty obscure to me. :-/

Thanks,

	Ingo

Re: 'make O=' indigestion with module signing

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> Correction - there appears to be something else going on, but the
> error messages are similar:
> 
>   At main.c:291:
>   - SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
>   - SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
>   sign-file: debian/linux-image/lib/modules/5.12.0-custom/kernel/arch/x86/crypto/aegis128-aesni.ko: No such file or directory
>   At main.c:291:
> 
> The error messages look pretty obscure to me. :-/

I didn't find any stray build files left in the tree, so 'make mrproper'
is innocent I believe.

I ended up with the config tweak below to get the kernel package to build,
which is not an ideal solution. :-/

Let me know if you'd like me to send you the .config and/or any diagnostic
messages or other details.

Thanks,

	Ingo

diff --git a/.config.kepler.ubuntu b/.config.kepler.ubuntu
index 01347a220e54..846a956fcdbd 100644
--- a/.config.kepler.ubuntu
+++ b/.config.kepler.ubuntu
@@ -880,9 +880,8 @@ CONFIG_MODULE_UNLOAD=y
 # CONFIG_MODULE_FORCE_UNLOAD is not set
 # CONFIG_MODVERSIONS is not set
 CONFIG_MODULE_SRCVERSION_ALL=y
-CONFIG_MODULE_SIG=y
-# CONFIG_MODULE_SIG_FORCE is not set
-CONFIG_MODULE_SIG_ALL=y
+# CONFIG_MODULE_SIG is not set
+# CONFIG_MODULE_SIG_ALL is not set
 # CONFIG_MODULE_SIG_SHA1 is not set
 # CONFIG_MODULE_SIG_SHA224 is not set
 # CONFIG_MODULE_SIG_SHA256 is not set
@@ -10177,11 +10176,7 @@ CONFIG_SECURITY_APPARMOR_HASH_DEFAULT=y
 # CONFIG_SECURITY_LOADPIN is not set
 CONFIG_SECURITY_YAMA=y
 CONFIG_SECURITY_SAFESETID=y
-CONFIG_SECURITY_LOCKDOWN_LSM=y
-CONFIG_SECURITY_LOCKDOWN_LSM_EARLY=y
-CONFIG_LOCK_DOWN_KERNEL_FORCE_NONE=y
-# CONFIG_LOCK_DOWN_KERNEL_FORCE_INTEGRITY is not set
-# CONFIG_LOCK_DOWN_KERNEL_FORCE_CONFIDENTIALITY is not set
+# CONFIG_SECURITY_LOCKDOWN_LSM is not set
 # CONFIG_SECURITY_LANDLOCK is not set
 CONFIG_INTEGRITY=y
 CONFIG_INTEGRITY_SIGNATURE=y

Re: 'make O=' indigestion with module signing

[Ingo Molnar]

* Ingo Molnar <mingo@kernel.org> wrote:

> 
> * Ingo Molnar <mingo@kernel.org> wrote:
> 
> > Correction - there appears to be something else going on, but the
> > error messages are similar:
> > 
> >   At main.c:291:
> >   - SSL error:02001002:system library:fopen:No such file or directory: ../crypto/bio/bss_file.c:69
> >   - SSL error:2006D080:BIO routines:BIO_new_file:no such file: ../crypto/bio/bss_file.c:76
> >   sign-file: debian/linux-image/lib/modules/5.12.0-custom/kernel/arch/x86/crypto/aegis128-aesni.ko: No such file or directory
> >   At main.c:291:
> > 
> > The error messages look pretty obscure to me. :-/
> 
> I didn't find any stray build files left in the tree, so 'make mrproper'
> is innocent I believe.
> 
> I ended up with the config tweak below to get the kernel package to build,
> which is not an ideal solution. :-/
> 
> Let me know if you'd like me to send you the .config and/or any diagnostic
> messages or other details.

Update: I now have a config that builds & boots fine, and I've done a 
.config bisection to figure out which config option combination breaks the 
build, which turned out to be the new MODULE_COMPRESS options:

--- .config.good.002	2021-05-10 14:34:48.932206255 +0200
+++ .config.bad.002	2021-05-10 14:40:48.587450742 +0200
@@ -899,8 +899,8 @@ CONFIG_MODULE_SIG_ALL=y
 # CONFIG_MODULE_SIG_SHA384 is not set
 CONFIG_MODULE_SIG_SHA512=y
 CONFIG_MODULE_SIG_HASH="sha512"
-CONFIG_MODULE_COMPRESS_NONE=y
-# CONFIG_MODULE_COMPRESS_GZIP is not set
+# CONFIG_MODULE_COMPRESS_NONE is not set
+CONFIG_MODULE_COMPRESS_GZIP=y
 # CONFIG_MODULE_COMPRESS_XZ is not set
 # CONFIG_MODULE_COMPRESS_ZSTD is not set
 # CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS is not set

Setting it to gzip triggered this build bug.

Thanks,

	Ingo