From mboxrd@z Thu Jan 1 00:00:00 1970 From: Sami Tolvanen Subject: Re: [PATCH 00/22] add support for Clang LTO Date: Tue, 7 Jul 2020 08:51:07 -0700 Message-ID: <20200707155107.GA3357035@google.com> References: <20200624203200.78870-1-samitolvanen@google.com> <20200629232059.GA3787278@google.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <20200629232059.GA3787278@google.com> Sender: linux-kernel-owner@vger.kernel.org To: Masahiro Yamada , Jiong Wang , Jakub Kicinski Cc: Will Deacon , Greg Kroah-Hartman , "Paul E. McKenney" , Kees Cook , Nick Desaulniers , clang-built-linux , Kernel Hardening , linux-arch , linux-arm-kernel , Linux Kbuild mailing list , Linux Kernel Mailing List , linux-pci@vger.kernel.org, X86 ML List-Id: linux-arch.vger.kernel.org On Mon, Jun 29, 2020 at 04:20:59PM -0700, Sami Tolvanen wrote: > Hi Masahiro, > > On Mon, Jun 29, 2020 at 01:56:19AM +0900, Masahiro Yamada wrote: > > I also got an error for > > ARCH=arm64 allyesconfig + CONFIG_LTO_CLANG=y > > > > > > > > $ make ARCH=arm64 LLVM=1 LLVM_IAS=1 > > CROSS_COMPILE=~/tools/aarch64-linaro-7.5/bin/aarch64-linux-gnu- > > -j24 > > > > ... > > > > GEN .version > > CHK include/generated/compile.h > > UPD include/generated/compile.h > > CC init/version.o > > AR init/built-in.a > > GEN .tmp_initcalls.lds > > GEN .tmp_symversions.lds > > LTO vmlinux.o > > MODPOST vmlinux.symvers > > MODINFO modules.builtin.modinfo > > GEN modules.builtin > > LD .tmp_vmlinux.kallsyms1 > > ld.lld: error: undefined symbol: __compiletime_assert_905 > > >>> referenced by irqbypass.c > > >>> vmlinux.o:(jeq_imm) > > make: *** [Makefile:1161: vmlinux] Error 1 > > I can reproduce this with ToT LLVM and it's BUILD_BUG_ON_MSG(..., "value > too large for the field") in drivers/net/ethernet/netronome/nfp/bpf/jit.c. > Specifically, the FIELD_FIT / __BF_FIELD_CHECK macro in ur_load_imm_any. > > This compiles just fine with an earlier LLVM revision, so it could be a > relatively recent regression. I'll take a look. Thanks for catching this! After spending some time debugging this with Nick, it looks like the error is caused by a recent optimization change in LLVM, which together with the inlining of ur_load_imm_any into jeq_imm, changes a runtime check in FIELD_FIT that would always fail, to a compile-time check that breaks the build. In jeq_imm, we have: /* struct bpf_insn: _s32 imm */ u64 imm = insn->imm; /* sign extend */ ... if (imm >> 32) { /* non-zero only if insn->imm is negative */ /* inlined from ur_load_imm_any */ u32 __imm = imm >> 32; /* therefore, always 0xffffffff */ /* * __imm has a value known at compile-time, which means * __builtin_constant_p(__imm) is true and we end up with * essentially this in __BF_FIELD_CHECK: */ if (__builtin_constant_p(__imm) && __imm <= 255) __compiletime_assert_N(); The compile-time check comes from the following BUILD_BUG_ON_MSG: #define __BF_FIELD_CHECK(_mask, _reg, _val, _pfx) \ ... BUILD_BUG_ON_MSG(__builtin_constant_p(_val) ? \ ~((_mask) >> __bf_shf(_mask)) & (_val) : 0, \ _pfx "value too large for the field"); \ While we could stop the compiler from performing this optimization by telling it to never inline ur_load_imm_any, we feel like a better fix might be to replace FIELD_FIT(UR_REG_IMM_MAX, imm) with a simple imm <= UR_REG_IMM_MAX check that won't trip a compile-time assertion even when the condition is known to fail. Jiong, Jakub, do you see any issues here? Sami From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36952 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728425AbgGGPvQ (ORCPT ); Tue, 7 Jul 2020 11:51:16 -0400 Received: from mail-pl1-x643.google.com (mail-pl1-x643.google.com [IPv6:2607:f8b0:4864:20::643]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id EC9EFC08C5E2 for ; Tue, 7 Jul 2020 08:51:15 -0700 (PDT) Received: by mail-pl1-x643.google.com with SMTP id x8so16013382plm.10 for ; Tue, 07 Jul 2020 08:51:15 -0700 (PDT) Date: Tue, 7 Jul 2020 08:51:07 -0700 From: Sami Tolvanen Subject: Re: [PATCH 00/22] add support for Clang LTO Message-ID: <20200707155107.GA3357035@google.com> References: <20200624203200.78870-1-samitolvanen@google.com> <20200629232059.GA3787278@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20200629232059.GA3787278@google.com> Sender: linux-arch-owner@vger.kernel.org List-ID: To: Masahiro Yamada , Jiong Wang , Jakub Kicinski Cc: Will Deacon , Greg Kroah-Hartman , "Paul E. McKenney" , Kees Cook , Nick Desaulniers , clang-built-linux , Kernel Hardening , linux-arch , linux-arm-kernel , Linux Kbuild mailing list , Linux Kernel Mailing List , linux-pci@vger.kernel.org, X86 ML Message-ID: <20200707155107.H6IUsOXGlD7W3By-qMEfuubAN7xF0Onm_5uHgcmt2xo@z> On Mon, Jun 29, 2020 at 04:20:59PM -0700, Sami Tolvanen wrote: > Hi Masahiro, > > On Mon, Jun 29, 2020 at 01:56:19AM +0900, Masahiro Yamada wrote: > > I also got an error for > > ARCH=arm64 allyesconfig + CONFIG_LTO_CLANG=y > > > > > > > > $ make ARCH=arm64 LLVM=1 LLVM_IAS=1 > > CROSS_COMPILE=~/tools/aarch64-linaro-7.5/bin/aarch64-linux-gnu- > > -j24 > > > > ... > > > > GEN .version > > CHK include/generated/compile.h > > UPD include/generated/compile.h > > CC init/version.o > > AR init/built-in.a > > GEN .tmp_initcalls.lds > > GEN .tmp_symversions.lds > > LTO vmlinux.o > > MODPOST vmlinux.symvers > > MODINFO modules.builtin.modinfo > > GEN modules.builtin > > LD .tmp_vmlinux.kallsyms1 > > ld.lld: error: undefined symbol: __compiletime_assert_905 > > >>> referenced by irqbypass.c > > >>> vmlinux.o:(jeq_imm) > > make: *** [Makefile:1161: vmlinux] Error 1 > > I can reproduce this with ToT LLVM and it's BUILD_BUG_ON_MSG(..., "value > too large for the field") in drivers/net/ethernet/netronome/nfp/bpf/jit.c. > Specifically, the FIELD_FIT / __BF_FIELD_CHECK macro in ur_load_imm_any. > > This compiles just fine with an earlier LLVM revision, so it could be a > relatively recent regression. I'll take a look. Thanks for catching this! After spending some time debugging this with Nick, it looks like the error is caused by a recent optimization change in LLVM, which together with the inlining of ur_load_imm_any into jeq_imm, changes a runtime check in FIELD_FIT that would always fail, to a compile-time check that breaks the build. In jeq_imm, we have: /* struct bpf_insn: _s32 imm */ u64 imm = insn->imm; /* sign extend */ ... if (imm >> 32) { /* non-zero only if insn->imm is negative */ /* inlined from ur_load_imm_any */ u32 __imm = imm >> 32; /* therefore, always 0xffffffff */ /* * __imm has a value known at compile-time, which means * __builtin_constant_p(__imm) is true and we end up with * essentially this in __BF_FIELD_CHECK: */ if (__builtin_constant_p(__imm) && __imm <= 255) __compiletime_assert_N(); The compile-time check comes from the following BUILD_BUG_ON_MSG: #define __BF_FIELD_CHECK(_mask, _reg, _val, _pfx) \ ... BUILD_BUG_ON_MSG(__builtin_constant_p(_val) ? \ ~((_mask) >> __bf_shf(_mask)) & (_val) : 0, \ _pfx "value too large for the field"); \ While we could stop the compiler from performing this optimization by telling it to never inline ur_load_imm_any, we feel like a better fix might be to replace FIELD_FIT(UR_REG_IMM_MAX, imm) with a simple imm <= UR_REG_IMM_MAX check that won't trip a compile-time assertion even when the condition is known to fail. Jiong, Jakub, do you see any issues here? Sami