Re: [tip:x86/seves] BUILD SUCCESS WITH WARNING e6eb15c9ba3165698488ae5c34920eea20eaa38e

From: Nick Desaulniers <ndesaulniers@google.com>
To: Josh Poimboeuf <jpoimboe@redhat.com>,
	Borislav Petkov <bp@alien8.de>, Marco Elver <elver@google.com>
Cc: Rong Chen <rong.a.chen@intel.com>,
	kernel test robot <lkp@intel.com>,
	"Li, Philip" <philip.li@intel.com>, x86-ml <x86@kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	clang-built-linux <clang-built-linux@googlegroups.com>,
	Arnd Bergmann <arnd@arndb.de>
Subject: Re: [tip:x86/seves] BUILD SUCCESS WITH WARNING e6eb15c9ba3165698488ae5c34920eea20eaa38e
Date: Tue, 15 Sep 2020 14:13:01 -0700	[thread overview]
Message-ID: <CAKwvOdmNzapwEo26m0soAR2OLFxOoERPKbKgvQ0bfoqjNkEVCg@mail.gmail.com> (raw)
In-Reply-To: <CAKwvOdnc8au10g8q8miab89j3tT8UhwnZOMAJdRgkXVrnkhwqQ@mail.gmail.com>

On Tue, Sep 15, 2020 at 1:12 PM Nick Desaulniers
<ndesaulniers@google.com> wrote:
>
> On Tue, Sep 15, 2020 at 10:02 AM Josh Poimboeuf <jpoimboe@redhat.com> wrote:
> >
> > On Tue, Sep 15, 2020 at 06:05:54PM +0200, Borislav Petkov wrote:
> > > On Tue, Sep 15, 2020 at 10:18:16PM +0800, Rong Chen wrote:
> > > > On Tue, Sep 15, 2020 at 03:55:19PM +0200, Borislav Petkov wrote:
> > > > > On Tue, Sep 15, 2020 at 09:42:56PM +0800, kernel test robot wrote:
> > > > > > tree/branch: https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git  x86/seves
> > > > > > branch HEAD: e6eb15c9ba3165698488ae5c34920eea20eaa38e  KVM: nSVM: Avoid freeing uninitialized pointers in svm_set_nested_state()
> > > > > >
> > > > > > Warning in current branch:
> > > > > >
> > > > > > arch/x86/kernel/sev-es.o: warning: objtool: ist_exc_vmm_communication()+0x12: unreachable instruction
> > > > > >
> > > > > > Warning ids grouped by kconfigs:
> > > > > >
> > > > > > clang_recent_errors
> > > > > > `-- x86_64-randconfig-r001-20200913
> > > > > >     `-- arch-x86-kernel-sev-es.o:warning:objtool:ist_exc_vmm_communication():unreachable-instruction
> > >
> > > Ok, I can't trigger this with gcc-9, gcc-10 and clang-10.
> > >
> > > .config below says clang-12 so it could be some experimental
> > > bug^Wfeature :)
> > >
> > > peterz says this has been fixed by:
> > >
> > > 318af7b80b6a ("Revert "kbuild: use -flive-patching when CONFIG_LIVEPATCH is enabled"")
> > >
> > > but that was reportedly for gcc.
> > >
> > > Lemme Cc the people of interest and leave the rest in here for
> > > reference. Note that CONFIG_LIVEPATCH is enabled in that .config.
> >
> > If somebody can share the .o file, I can take a look.
>
> Taking a look at the latest report of unreachable instruction from
> objtool from 0day bot:
> https://groups.google.com/g/clang-built-linux/c/IbF96FeMT8g
>
> $ cd /tmp
> $ git clone --depth=1
> https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git --branch
> x86/seves
> $ cd tip
> $ gunzip <attached config.gz>
> $ cp config .config
> $ make LLVM=1 -j71 2> log.txt
> $ grep warning: log.txt | cut -d ' ' -f 2- | sort | uniq -c
>    3737 warning: objtool: asan.module_ctor()+0xc: call without frame
> pointer save/setup
>    3737 warning: objtool: asan.module_dtor()+0xc: call without frame
> pointer save/setup
>       1 warning: objtool: early_alloc()+0x6b: unreachable instruction
>       1 warning: objtool: ist_exc_vmm_communication()+0x12:
> unreachable instruction
>       1 warning: objtool: kmem_cache_create_usercopy()+0x286:
> unreachable instruction
>       1 warning: objtool: oops_end()+0xc1: unreachable instruction
>       1 warning: objtool: __register_nosave_region()+0x87: unreachable
> instruction
> $ ag ist_exc_vmm_communication
> arch/x86/kernel/sev-es.c
> 1351:           ist_exc_vmm_communication(regs, error_code);
> $ gzip -9 arch/x86/kernel/sev-es.o
> <attached>
>
> If I remove CONFIG_KASAN=y, I actually get way more unreachable
> instruction failures!  Let me play with
> tools/testing/ktest/config-bisect.pl and see if I can narrow down the
> set of configs from the randconfig.  Also, it's possible that there's
> more than one bug here.

tools/testing/ktest/config-bisect.pl narrowed this down to:

%%%%%%%% FAILED TO FIND SINGLE BAD CONFIG %%%%%%%%
Hmm, can't make any more changes without making good == bad?
Difference between good (+) and bad (-)
 UBSAN_TRAP n -> y
+UBSAN_ALIGNMENT=n
See good and bad configs for details:
good: /tmp/x86/tip/good.config.tmp
bad:  /tmp/x86/tip/bad.config.tmp
%%%%%%%% FAILED TO FIND SINGLE BAD CONFIG %%%%%%%%

(Though confirming this, I only need defconfig+CONFIG_UBSAN to repro)

For folks that know how to run tools/testing/ktest/config-bisect.pl,
skip this next section, more so notes to future self and others on our
mailing list to learn how to run this. Arnd Bergmann taught me about
this.
$ cp config_from_0day bad.config
$ make LLVM=1 -j71 defconfig
$ cp .config good.config
$ ./tools/testing/ktest/config-bisect.pl good.config bad.config
1: $ make LLVM=1 -j71 clean bzImage 2> log.txt
2: $ grep warning: log2.txt | cut -d ' ' -f 2- | sort | uniq -c | grep
"unreachable instruction"
if !$ == 0:
tools/testing/ktest/config-bisect.pl good.config bad.config good
else:
tools/testing/ktest/config-bisect.pl good.config bad.config bad
if not finished:
  goto 1

Triple checking what I wrote above; it looks like the randconfig had
BOTH CONFIG_KASAN=y and CONFIG_UBSAN=y enabled.  Aren't the sanitizers
supposed to be mutually exclusive?  If so, we should ensure that via
kconfig these can't be selected together via randconfig.

Another peculiarity I noticed was that it seems that enabling
CONFIG_UBSAN_TRAP=y doesn't allow me to select
CONFIG_UBSAN_ALIGNMENT=y.  Is that intentional? (Looks like it
lib/Kconfig.ubsan:74).

There was actually another thread about CONFIG_UBSAN_TRAP and Clang
today, related to arm64 virtualization extensions.
https://lore.kernel.org/linux-arm-kernel/CANpmjNPpq7LfTHYesz2wTVw6Pqv0FQ2gc-vmSB6Mdov+XWPZiw@mail.gmail.com/T/#u
-- 
Thanks,
~Nick Desaulniers