objtool "no non-local symbols" error with tip of tree LLVM

From: Nathan Chancellor <nathan@kernel.org>
To: Josh Poimboeuf <jpoimboe@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>
Cc: Nick Desaulniers <ndesaulniers@google.com>,
	llvm@lists.linux.dev, linux-kernel@vger.kernel.org,
	kasan-dev@googlegroups.com
Subject: objtool "no non-local symbols" error with tip of tree LLVM
Date: Mon, 16 May 2022 13:47:15 -0700	[thread overview]
Message-ID: <YoK4U9RgQ9N+HhXJ@dev-arch.thelio-3990X> (raw)

Hi Josh and Peter,

After a recent change in LLVM [1], I see warnings (errors?) from objtool
when building x86_64 allmodconfig on 5.15 and 5.17:

  $ make -skj"$(nproc)" KCONFIG_ALLCONFIG=<(echo CONFIG_WERROR) LLVM=1 allmodconfig all
  ...
  mm/highmem.o: warning: objtool: no non-local symbols !?
  mm/highmem.o: warning: objtool: gelf_update_symshndx: invalid section index
  make[2]: *** [scripts/Makefile.build:288: mm/highmem.o] Error 255
  ...
  security/tomoyo/load_policy.o: warning: objtool: no non-local symbols !?
  security/tomoyo/load_policy.o: warning: objtool: gelf_update_symshndx: invalid section index
  make[3]: *** [scripts/Makefile.build:288: security/tomoyo/load_policy.o] Error 255
  ...

I don't see the same errors on x86_64 allmodconfig on mainline so I
bisected the 5.17 branch and came upon commit 4abff6d48dbc ("objtool:
Fix code relocs vs weak symbols"). I wanted to see what 5.17 might be
missing and came to commit ed53a0d97192 ("x86/alternative: Use
.ibt_endbr_seal to seal indirect calls") in mainline, which I think just
hides the issue for allmodconfig. I can reproduce this problem with a
more selective set of config values on mainline:

  $ make -skj"$(nproc)" LLVM=1 defconfig

  $ scripts/config -e KASAN -e SECURITY_TOMOYO -e SECURITY_TOMOYO_OMIT_USERSPACE_LOADER

  $ make -skj"$(nproc)" LLVM=1 olddefconfig security/tomoyo/load_policy.o
  security/tomoyo/load_policy.o: warning: objtool: no non-local symbols !?
  security/tomoyo/load_policy.o: warning: objtool: gelf_update_symshndx: invalid section index
  make[3]: *** [scripts/Makefile.build:288: security/tomoyo/load_policy.o] Error 255
  ...

Looking at the object file, the '.text.asan.module_ctor' section has
disappeared.

Before:

  $ llvm-nm -S security/tomoyo/load_policy.o
  0000000000000000 0000000000000001 t asan.module_ctor

  $ llvm-readelf -s security/tomoyo/load_policy.o

  Symbol table '.symtab' contains 4 entries:
     Num:    Value          Size Type    Bind   Vis       Ndx Name
       0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   UND
       1: 0000000000000000     0 FILE    LOCAL  DEFAULT   ABS load_policy.c
       2: 0000000000000000     0 SECTION LOCAL  DEFAULT     3 .text.asan.module_ctor
       3: 0000000000000000     1 FUNC    LOCAL  DEFAULT     3 asan.module_ctor

After:

  $ llvm-nm -S security/tomoyo/load_policy.o
  0000000000000000 0000000000000001 t asan.module_ctor

  $ llvm-readelf -s security/tomoyo/load_policy.o

  Symbol table '.symtab' contains 3 entries:
     Num:    Value          Size Type    Bind   Vis       Ndx Name
       0: 0000000000000000     0 NOTYPE  LOCAL  DEFAULT   UND
       1: 0000000000000000     0 FILE    LOCAL  DEFAULT   ABS load_policy.c
       2: 0000000000000000     1 FUNC    LOCAL  DEFAULT     3 asan.module_ctor

As far as I understand it, the kernel uses constructors for at least
KASAN and KCSAN, hence why that change impacts the kernel. Beyond that,
I am not really sure whether the LLVM change is problematic or objtool
just is not accounting for something that it should. I am happy to
provide any additional information that might help understand what is
going wrong here.

[1]: https://github.com/llvm/llvm-project/commit/badd088c57d7d18acd337b7868fe8c7974c88c5b

Cheers,
Nathan