Re: CONFIG_UBSAN_TRAP #UD error message on x86 [was: Re: [CRASH][BISECTED] 6.4.1 crash in boot]

From: Kees Cook <keescook@chromium.org>
To: Jann Horn <jannh@google.com>
Cc: Kees Cook <kees@kernel.org>,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
	Linux LLVM <llvm@lists.linux.dev>,
	the arch/x86 maintainers <x86@kernel.org>
Subject: Re: CONFIG_UBSAN_TRAP #UD error message on x86 [was: Re: [CRASH][BISECTED] 6.4.1 crash in boot]
Date: Wed, 5 Jul 2023 14:08:09 -0700	[thread overview]
Message-ID: <202307051405.E746464@keescook> (raw)
In-Reply-To: <CAG48ez0kmSqS_WJ1=neKpxKmMMkPYgNHfBD6ZSif_EWnso84wQ@mail.gmail.com>

On Wed, Jul 05, 2023 at 05:16:36PM +0200, Jann Horn wrote:
> On Wed, Jul 5, 2023 at 4:10 AM Kees Cook <kees@kernel.org> wrote:
> > On July 4, 2023 4:15:20 PM PDT, Mirsad Todorovac <mirsad.todorovac@alu.unizg.hr> wrote:
> > >On 7/4/23 23:36, Kees Cook wrote:
> > >> On July 4, 2023 10:20:11 AM PDT, Mirsad Todorovac <mirsad.todorovac@alu.unizg.hr> wrote:
> > >>> all Wayland and X11.org GUI applications fail to start, with errors like this one:
> > >>>
> > >>> Jul  4 19:09:07 defiant kernel: [   40.529719] invalid opcode: 0000 [#1] PREEMPT SMP NOPTI
> > >>
> > >> Hmm, is CONFIG_UBSAN_TRAP set?
> > >
> > >marvin@defiant:~/linux/kernel/linux_torvalds$ grep CONFIG_UBSAN_TRAP .config
> > >CONFIG_UBSAN_TRAP=y
> >
> > Ah-ha! Turn that off please. With it off you will get much more useful reports from USBAN.
> 
> It might be useful if the x86 code under handle_invalid_op() at least
> printed a warning about this when the kernel crashes with #UD on a
> system with CONFIG_UBSAN_TRAP=y? It seems pretty unintuitive and
> unhelpful that the kernel just crashes itself with a #UD and no
> further information in this configuration.
> 
> Even just a "WARNING: CONFIG_UBSAN_TRAP active, #UD might be caused by
> that" on every #UD that does not come from a known BUG() location or
> such might be better than nothing...

I've considered it, but usually CONFIG_UBSAN_TRAP isn't accidentally
set. Also, the crash info is something we can get help from on the
compiler side, to mark up where the traps are, similar to what we do
with KCFI, but it hasn't happened yet for x86. For example, arm64
already encodes the details in the trap instruction itself:
https://git.kernel.org/linus/25b84002afb9dc9a91a7ea67166879c13ad82422

> And maybe the Kconfig help text could be clearer on this, too.
> Currently it does say that this turns warnings into "full exceptions
> that abort the running kernel code" but it does not say that the
> exception reporting will become pretty unhelpful, so it's probably not
> really what you'd want for debugging.

Yeah, that's a reasonable change to make. Can you send a patch for this?
I can carry it.

Thanks!

-- 
Kees Cook