Still some race in X509 certificates handling

Still some race in X509 certificates handling

[Linus Torvalds]

So this is *really* hard to trigger, but it happened for the second
time (in a month?) today when I tried to boot into a newly built
kernel.

What  happens is that somehow the build gets confused about the
certificates, and my modules refuse to load with something along the
line of "Request for unknown module key 'Magrathea: Glacier signing
key'" (which is obviously the built-in generated key).

When it happens, I can do a rebuild, and the build will say

   X.509 certificate list changed

which is kind of odd, since the list should *always* be just that
single key for me (ie "./signing_key.509").

But even after the rebuild the end result *still* did not work. I
needed to do a "git clean" and rebuild to make it all come out right.
So presumably the modules themselves still had the old stale data, or
something like that. Despite the whole rebuild, and re-installing the
modules and kernel.

As mentioned, this is very rare. I build and boot quite a lot of
kernels particularly during the merge window, and it definitely
doesn't happen every time.

So it really smells like some odd race - I build my kernels with "make
-j16", so there's certainly room for a race or two with any rule that
doesn't have proper dependencies. I don't see it, though.

Anybody see anything fishy in this area?

(Side note: the HHGTTG references are cute, but I suspect we should
rename the key so that it just says something boring like "build-time
autogenerated kernel key" instead. Just so that the error messages are
a bit more readable to people who aren't kernel engineers)

                         Linus

Re: Still some race in X509 certificates handling

[David Howells]

Linus Torvalds <torvalds@linux-foundation.org> wrote:

> When it happens, I can do a rebuild, and the build will say
> 
>    X.509 certificate list changed
> 
> which is kind of odd, since the list should *always* be just that
> single key for me (ie "./signing_key.509").

Did you by any chance set aside a build tree that went wrong?  If so, could
you have a look to see what's in:

	<builddir>/kernel/.x509.list
	<builddir>/kernel/x509_certificate_list (note this is binary)
	<builddir>/x509.genkey

and make sure that:

	<builddir>/signing_key.priv
	<builddir>/signing_key.x509

both exist.  I wonder if the problem might perhaps be due to one of
signing_key.priv or signing_key.x509 getting removed somehow - but not both.
Make seems a bit weird on targets that produce two files, one of which isn't
depended on (it might remove it under some circumstances, I think).


Btw, do you use O=<builddir> when you're building?  That causes a certain
amount of pain to get right because:

 (1) the auto-generated keys have to be placed into the build dir, not the
     source dir;

 (2) we still need to scrape extra X.509 certs from the source dir; and

 (3) we don't want to see the autogenerated X.509 certificate twice if the
     build dir is the same as the source dir.

Actually, we could simplify the makefile a bit and waive (3) if we weeded out
duplicate X.509 certs by X.509 parameter value rather than by filename before
adding them into the kernel.

> (Side note: the HHGTTG references are cute, but I suspect we should
> rename the key so that it just says something boring like "build-time
> autogenerated kernel key" instead. Just so that the error messages are
> a bit more readable to people who aren't kernel engineers)

Awww...

My main point was to try and encourage distributions to supply an x509.genkey
with fields filled in with appropriate info.  I guess that's probably achieved
by now, so I could make it something else.  It has to be specified by an
X.400/X.500 DN, though, so maybe:

	@echo >>x509.genkey "O = Your company name"
	@echo >>x509.genkey "CN = Build time autogenerated kernel key"
	@echo >>x509.genkey "emailAddress = you@your.company"

I would really like to leave O, CN and emailAddress in here because these are
the fields that x509_fabricate_name() uses in the kernel.

David

Re: Still some race in X509 certificates handling

[Linus Torvalds]

On Fri, Feb 13, 2015 at 4:15 AM, David Howells <dhowells@redhat.com> wrote:
>
> Did you by any chance set aside a build tree that went wrong?

No, I didn't, I just wanted to get my testing done so that I could push out.

> Btw, do you use O=<builddir> when you're building?

Nope. Building in the normal source directory. I've never seen the
point of "O=" outside of "one source tree, many objects trees", and
even that is rather questionable in the age of sane and efficient
source control.

> I would really like to leave O, CN and emailAddress in here because these are
> the fields that x509_fabricate_name() uses in the kernel.

I don't mind that part. It's just that "unknown module key
'Magrathea'" that I don't think is exactly obvious unless you've seen
it before..

Just replacing "Magrathea Glacier signing key" with "Kernel build-time
random signing key" would make the error messages when the key is
wrong a bit more natural, wouldn't you say?

Perhaps a bit square and boring, but ..

                           Linus

Re: Still some race in X509 certificates handling

[Michal Marek]

On 2015-02-13 13:15, David Howells wrote:
> Linus Torvalds <torvalds@linux-foundation.org> wrote:
> 
>> When it happens, I can do a rebuild, and the build will say
>>
>>    X.509 certificate list changed
>>
>> which is kind of odd, since the list should *always* be just that
>> single key for me (ie "./signing_key.509").
> 
> Did you by any chance set aside a build tree that went wrong?  If so, could
> you have a look to see what's in:
> 
> 	<builddir>/kernel/.x509.list
> 	<builddir>/kernel/x509_certificate_list (note this is binary)
> 	<builddir>/x509.genkey
> 
> and make sure that:
> 
> 	<builddir>/signing_key.priv
> 	<builddir>/signing_key.x509
> 
> both exist.  I wonder if the problem might perhaps be due to one of
> signing_key.priv or signing_key.x509 getting removed somehow - but not both.

It could also be due to the usage of realpath when building the
certificate list:

X509_CERTIFICATES-$(CONFIG_MODULE_SIG) += $(objtree)/signing_key.x509
X509_CERTIFICATES-raw := $(sort $(foreach CERT,$(X509_CERTIFICATES-y), \
				$(or $(realpath $(CERT)),$(CERT))))

If signing_key.x509 hasn't been generated yet, it will be stored as as
./signing_key.x509. In a later make invocation, realpath will resolve it
into /home/.../signing_key.x509. But then, it should fail every time,
because of the := assignment.

Michal

Re: Still some race in X509 certificates handling

[Michal Marek]

On 2015-02-18 15:04, Michal Marek wrote:
> On 2015-02-13 13:15, David Howells wrote:
>> Linus Torvalds <torvalds@linux-foundation.org> wrote:
>>
>>> When it happens, I can do a rebuild, and the build will say
>>>
>>>    X.509 certificate list changed
>>>
>>> which is kind of odd, since the list should *always* be just that
>>> single key for me (ie "./signing_key.509").
>>
>> Did you by any chance set aside a build tree that went wrong?  If so, could
>> you have a look to see what's in:
>>
>> 	<builddir>/kernel/.x509.list
>> 	<builddir>/kernel/x509_certificate_list (note this is binary)
>> 	<builddir>/x509.genkey
>>
>> and make sure that:
>>
>> 	<builddir>/signing_key.priv
>> 	<builddir>/signing_key.x509
>>
>> both exist.  I wonder if the problem might perhaps be due to one of
>> signing_key.priv or signing_key.x509 getting removed somehow - but not both.
> 
> It could also be due to the usage of realpath when building the

Actually, I sent a patch for the "X.509 certificate list changed"
message last October: https://lkml.org/lkml/2014/10/8/306, please apply
it. However, this is just a spurious rebuild. The cause for the
signature verification error must be something else.

Michal

Re: Still some race in X509 certificates handling

[Linus Torvalds]

On Wed, Feb 18, 2015 at 6:36 AM, Michal Marek <mmarek@suse.cz> wrote:
>
> Actually, I sent a patch for the "X.509 certificate list changed"
> message last October: https://lkml.org/lkml/2014/10/8/306, please apply
> it. However, this is just a spurious rebuild. The cause for the
> signature verification error must be something else.

So I saw the patch originally and it seemed a reasonable cleanup, but
you're the kbuild maintainer, so I always assumed it would come
through a standard kbuild pull.

So color me a bit confused about the "I wrote a patch last October,
but never bothered to add it to my own trees, so please apply it"
thing.

                        Linus

Re: Still some race in X509 certificates handling

[Michal Marek]

Dne 18.2.2015 v 19:49 Linus Torvalds napsal(a):
> On Wed, Feb 18, 2015 at 6:36 AM, Michal Marek <mmarek@suse.cz> wrote:
>>
>> Actually, I sent a patch for the "X.509 certificate list changed"
>> message last October: https://lkml.org/lkml/2014/10/8/306, please apply
>> it. However, this is just a spurious rebuild. The cause for the
>> signature verification error must be something else.
> 
> So I saw the patch originally and it seemed a reasonable cleanup, but
> you're the kbuild maintainer, so I always assumed it would come
> through a standard kbuild pull.

I assumed it would get through David's tree like other modsign patches,
and I forgot about it later. David, is it OK to merge the patch through
the kbuild tree?

Thanks,
Michal