From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.1 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 70F5DC4741F for ; Wed, 30 Sep 2020 21:58:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 28ECB2064E for ; Wed, 30 Sep 2020 21:58:14 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LHz9Zcam" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730918AbgI3V6N (ORCPT ); Wed, 30 Sep 2020 17:58:13 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46074 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728721AbgI3V6M (ORCPT ); Wed, 30 Sep 2020 17:58:12 -0400 Received: from mail-pg1-x542.google.com (mail-pg1-x542.google.com [IPv6:2607:f8b0:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id ADE87C0613D0 for ; Wed, 30 Sep 2020 14:58:12 -0700 (PDT) Received: by mail-pg1-x542.google.com with SMTP id 197so2063364pge.8 for ; Wed, 30 Sep 2020 14:58:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=t4yHwQDoeC7Qpj3w+ohiUiQzIr7R18uaPZSHZDC/wBA=; b=LHz9Zcam7CeKHZ2FruDPRDuY0e07YHSqVvGalfh32T3Fuffnt2bHgr5rwgpKJmywr5 vYYvj2pj3Prw08WL6qW4WlpRdEOLUCrT+9hncLjFVN1Ft8Md5FT+cXeVzwCbCclkl92o hiVEXzmM9WAhHeXPsScWmhH08ZPlqMeItatEG49WX1OQPxwmMQyfc2iYMB/Z7txm5yp9 RRa0vfTlhSB0oEdLv191LDpEXlrqdDLR97VB59MmnkaiwZ4LOuqrpJLUtPF1GuZMwqmk sGSI3yP3GaxL9/tqCvhaOlXgx1uB7P44MkrgKvdzV+0gcVnwItyMO1mDDu9EjVXS5h+m ERsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=t4yHwQDoeC7Qpj3w+ohiUiQzIr7R18uaPZSHZDC/wBA=; b=HWkdvHYrT01WkFxYf7cTTo01XrR/J8eWmGEN0CgckLj37lLXxI5vhKtNVPoeq+3EjC 1n1Wv0vzBoPq7ygZJO6iDPr18DrR+SkhGaEPbXqrhUTiQjKfKU4TEDuhfjrOZS3uMFo2 Ox/YXQ1kTw14Na5u1lgczRGsg1n+GwVAa3/ECFFkeC+VVx22hxsDn2qilPP1qjbLBhsA 0p8sQElb0YndECWLBYCWb54rQJZ3G0OspfuYFVtZHVyPXY3/B28pfanP1te/F8dMxEJr 2qj5pJpFKolcPFYWQJtj4Wzggj1b4Kb6Zvt5ESe3CYtI56bUqkINb+5EMhZ/0o8OZgtb 4Fwg== X-Gm-Message-State: AOAM531ckS0eQvKqahbBKwDNYU57/Az9TrO6n3G4aVT7GTMp1Uqdl9m4 Gm40fcWw2fQbJv79tIFSO963P933L7ZVNivt0uxBmw== X-Google-Smtp-Source: ABdhPJwzX9Viqx7zQJehaAJaWpHju7adBUzL4TnFBbDcXOlCvSY7YI0nAchHwIA2JZ5jrnbbiiAg20J+xNYGNfRzTkU= X-Received: by 2002:a17:902:c40d:b029:d2:93e8:1f4b with SMTP id k13-20020a170902c40db02900d293e81f4bmr4327278plk.29.1601503091998; Wed, 30 Sep 2020 14:58:11 -0700 (PDT) MIME-Version: 1.0 References: <20200929214631.3516445-1-samitolvanen@google.com> In-Reply-To: <20200929214631.3516445-1-samitolvanen@google.com> From: Nick Desaulniers Date: Wed, 30 Sep 2020 14:58:00 -0700 Message-ID: Subject: Re: [PATCH v4 00/29] Add support for Clang LTO To: Sami Tolvanen Cc: Masahiro Yamada , Will Deacon , Steven Rostedt , Peter Zijlstra , Greg Kroah-Hartman , "Paul E. McKenney" , Kees Cook , clang-built-linux , Kernel Hardening , linux-arch , Linux ARM , Linux Kbuild mailing list , LKML , linux-pci@vger.kernel.org, "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Sep 29, 2020 at 2:46 PM Sami Tolvanen wrote: > > This patch series adds support for building x86_64 and arm64 kernels > with Clang's Link Time Optimization (LTO). > > In addition to performance, the primary motivation for LTO is > to allow Clang's Control-Flow Integrity (CFI) to be used in the > kernel. Google has shipped millions of Pixel devices running three > major kernel versions with LTO+CFI since 2018. > > Most of the patches are build system changes for handling LLVM > bitcode, which Clang produces with LTO instead of ELF object files, > postponing ELF processing until a later stage, and ensuring initcall > ordering. Sami, thanks for continuing to drive the series. I encourage you to keep resending with fixes accumulated or dropped on a weekly cadence. The series worked well for me on arm64, but for x86_64 on mainline I saw a stream of new objtool warnings: testing your LTO series; x86_64 defconfig + CONFIG_THINLTO: ``` LTO vmlinux.o OBJTOOL vmlinux.o vmlinux.o: warning: objtool: wakeup_long64()+0x61: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: .text+0x308a: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: .text+0x30c5: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: copy_user_enhanced_fast_string() falls through to next function copy_user_generic_unrolled() vmlinux.o: warning: objtool: __memcpy_mcsafe() falls through to next function mcsafe_handle_tail() vmlinux.o: warning: objtool: memset() falls through to next function memset_erms() vmlinux.o: warning: objtool: __memcpy() falls through to next function memcpy_erms() vmlinux.o: warning: objtool: __x86_indirect_thunk_rax() falls through to next function __x86_retpoline_rax() vmlinux.o: warning: objtool: __x86_indirect_thunk_rbx() falls through to next function __x86_retpoline_rbx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rcx() falls through to next function __x86_retpoline_rcx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rdx() falls through to next function __x86_retpoline_rdx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rsi() falls through to next function __x86_retpoline_rsi() vmlinux.o: warning: objtool: __x86_indirect_thunk_rdi() falls through to next function __x86_retpoline_rdi() vmlinux.o: warning: objtool: __x86_indirect_thunk_rbp() falls through to next function __x86_retpoline_rbp() vmlinux.o: warning: objtool: __x86_indirect_thunk_r8() falls through to next function __x86_retpoline_r8() vmlinux.o: warning: objtool: __x86_indirect_thunk_r9() falls through to next function __x86_retpoline_r9() vmlinux.o: warning: objtool: __x86_indirect_thunk_r10() falls through to next function __x86_retpoline_r10() vmlinux.o: warning: objtool: __x86_indirect_thunk_r11() falls through to next function __x86_retpoline_r11() vmlinux.o: warning: objtool: __x86_indirect_thunk_r12() falls through to next function __x86_retpoline_r12() vmlinux.o: warning: objtool: __x86_indirect_thunk_r13() falls through to next function __x86_retpoline_r13() vmlinux.o: warning: objtool: __x86_indirect_thunk_r14() falls through to next function __x86_retpoline_r14() vmlinux.o: warning: objtool: __x86_indirect_thunk_r15() falls through to next function __x86_retpoline_r15() ``` I think those should be resolved before I provide any kind of tested by tag. My other piece of feedback was that I like the default ThinLTO, but I think the help text in the Kconfig which is visible during menuconfig could be improved by informing the user the tradeoffs. For example, if CONFIG_THINLTO is disabled, it should be noted that full LTO will be used instead. Also, that full LTO may produce slightly better optimized binaries than ThinLTO, at the cost of not utilizing multiple cores when linking and thus significantly slower to link. Maybe explaining that setting it to "n" implies a full LTO build, which will be much slower to link but possibly slightly faster would be good? It's not visible unless LTO_CLANG and ARCH_SUPPORTS_THINLTO is enabled, so I don't think you need to explain that THINLTO without those is *not* full LTO. I'll leave the precise wording to you. WDYT? Also, when I look at your treewide DISABLE_LTO patch, I think "does that need to be a part of this series, or is it a cleanup that can stand on its own?" I think it may be the latter? Maybe it would help shed one more patch than to have to carry it to just send it? Or did I miss something as to why it should remain a part of this series? -- Thanks, ~Nick Desaulniers From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.5 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A193EC4741F for ; Wed, 30 Sep 2020 21:59:55 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5728A2064E for ; Wed, 30 Sep 2020 21:59:55 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="r8UFbxpn"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="LHz9Zcam" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5728A2064E Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:To:Subject:Message-ID:Date:From:In-Reply-To: References:MIME-Version:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pbMFf8AcQqDZmYvo6yayJVlT8/uJr3/mJ9zMH5Gfmhw=; b=r8UFbxpnPS6IPSABdCZDyTKgX OXZuZxGF6IsFeZCYNoYfDabKapL77rr78+L8I08a/7yHTakBK3E85EMPDIgfbakf3ZKhc83auroRl zzdUNZAHm4mgERUv8/KNWCFLjtlEM/6P8isj5RVbsgAqHyqxg2yFFdkJEy5FBtvne7poUVRkfbPTy q7eo9zdL7sR0BsTluJtyahmExT+68AwDbQaVaSFmF0IlCWaSur5rcLt2+GBFf3Xk1ZhlCYR8QU4Kh AYrzpsqtD9roKiIXOJ46w0HDt7+1pvbeHRUaWbO9THSuCEtJ0ClMNbRA17tr7OlteMid99dXU13fM tZa2To9sA==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kNk76-0003Nf-WD; Wed, 30 Sep 2020 21:58:17 +0000 Received: from mail-pg1-x544.google.com ([2607:f8b0:4864:20::544]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kNk74-0003M0-8j for linux-arm-kernel@lists.infradead.org; Wed, 30 Sep 2020 21:58:15 +0000 Received: by mail-pg1-x544.google.com with SMTP id o25so2094431pgm.0 for ; Wed, 30 Sep 2020 14:58:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=t4yHwQDoeC7Qpj3w+ohiUiQzIr7R18uaPZSHZDC/wBA=; b=LHz9Zcam7CeKHZ2FruDPRDuY0e07YHSqVvGalfh32T3Fuffnt2bHgr5rwgpKJmywr5 vYYvj2pj3Prw08WL6qW4WlpRdEOLUCrT+9hncLjFVN1Ft8Md5FT+cXeVzwCbCclkl92o hiVEXzmM9WAhHeXPsScWmhH08ZPlqMeItatEG49WX1OQPxwmMQyfc2iYMB/Z7txm5yp9 RRa0vfTlhSB0oEdLv191LDpEXlrqdDLR97VB59MmnkaiwZ4LOuqrpJLUtPF1GuZMwqmk sGSI3yP3GaxL9/tqCvhaOlXgx1uB7P44MkrgKvdzV+0gcVnwItyMO1mDDu9EjVXS5h+m ERsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=t4yHwQDoeC7Qpj3w+ohiUiQzIr7R18uaPZSHZDC/wBA=; b=tGNPDAP3uV9xiUyvuOHu5LPnTD5bmIiG+yn5V6k5LC6aKkDM1yDMsCOMDZsTTMU3w4 peT7m9O5xbyftHzG4/jmorGQnl+PNzSNFtd7mhnP60cGOf+o3uwM9P0bZF+K9w93bnZA xxqMcMT2ZGOdpwjIbEEN7WjXQh7TimNtu+PR9q/QVyoL76OZiW6hUY5CD4XkvaPT4PWe H/zXfkcotEGWb27P2x1ooG7rD+Io+CkY2szaz+KPrvoGTWxLCbNkWrYZR2onv7zfb5LY +D7GU3gv/Dtag2TFygHkNvNVi7VVUYAhD0awMoI+vPJwehFTpnAw63JYBG9cM6bd0efT QTZw== X-Gm-Message-State: AOAM530TwyKTnoc6/84lFkK7Zy3ahRvdd+ZxWDZF1FqmZHhZJuQQPOZB 2wUlUjFi35d8cueZOEzgjaO7eENSTMeireF1nrrEMQ== X-Google-Smtp-Source: ABdhPJwzX9Viqx7zQJehaAJaWpHju7adBUzL4TnFBbDcXOlCvSY7YI0nAchHwIA2JZ5jrnbbiiAg20J+xNYGNfRzTkU= X-Received: by 2002:a17:902:c40d:b029:d2:93e8:1f4b with SMTP id k13-20020a170902c40db02900d293e81f4bmr4327278plk.29.1601503091998; Wed, 30 Sep 2020 14:58:11 -0700 (PDT) MIME-Version: 1.0 References: <20200929214631.3516445-1-samitolvanen@google.com> In-Reply-To: <20200929214631.3516445-1-samitolvanen@google.com> From: Nick Desaulniers Date: Wed, 30 Sep 2020 14:58:00 -0700 Message-ID: Subject: Re: [PATCH v4 00/29] Add support for Clang LTO To: Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200930_175814_354990_D984B556 X-CRM114-Status: GOOD ( 24.67 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch , "maintainer:X86 ARCHITECTURE \(32-BIT AND 64-BIT\)" , Kees Cook , "Paul E. McKenney" , Kernel Hardening , Peter Zijlstra , Greg Kroah-Hartman , Masahiro Yamada , Linux Kbuild mailing list , LKML , Steven Rostedt , clang-built-linux , linux-pci@vger.kernel.org, Will Deacon , Linux ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, Sep 29, 2020 at 2:46 PM Sami Tolvanen wrote: > > This patch series adds support for building x86_64 and arm64 kernels > with Clang's Link Time Optimization (LTO). > > In addition to performance, the primary motivation for LTO is > to allow Clang's Control-Flow Integrity (CFI) to be used in the > kernel. Google has shipped millions of Pixel devices running three > major kernel versions with LTO+CFI since 2018. > > Most of the patches are build system changes for handling LLVM > bitcode, which Clang produces with LTO instead of ELF object files, > postponing ELF processing until a later stage, and ensuring initcall > ordering. Sami, thanks for continuing to drive the series. I encourage you to keep resending with fixes accumulated or dropped on a weekly cadence. The series worked well for me on arm64, but for x86_64 on mainline I saw a stream of new objtool warnings: testing your LTO series; x86_64 defconfig + CONFIG_THINLTO: ``` LTO vmlinux.o OBJTOOL vmlinux.o vmlinux.o: warning: objtool: wakeup_long64()+0x61: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: .text+0x308a: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: .text+0x30c5: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: copy_user_enhanced_fast_string() falls through to next function copy_user_generic_unrolled() vmlinux.o: warning: objtool: __memcpy_mcsafe() falls through to next function mcsafe_handle_tail() vmlinux.o: warning: objtool: memset() falls through to next function memset_erms() vmlinux.o: warning: objtool: __memcpy() falls through to next function memcpy_erms() vmlinux.o: warning: objtool: __x86_indirect_thunk_rax() falls through to next function __x86_retpoline_rax() vmlinux.o: warning: objtool: __x86_indirect_thunk_rbx() falls through to next function __x86_retpoline_rbx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rcx() falls through to next function __x86_retpoline_rcx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rdx() falls through to next function __x86_retpoline_rdx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rsi() falls through to next function __x86_retpoline_rsi() vmlinux.o: warning: objtool: __x86_indirect_thunk_rdi() falls through to next function __x86_retpoline_rdi() vmlinux.o: warning: objtool: __x86_indirect_thunk_rbp() falls through to next function __x86_retpoline_rbp() vmlinux.o: warning: objtool: __x86_indirect_thunk_r8() falls through to next function __x86_retpoline_r8() vmlinux.o: warning: objtool: __x86_indirect_thunk_r9() falls through to next function __x86_retpoline_r9() vmlinux.o: warning: objtool: __x86_indirect_thunk_r10() falls through to next function __x86_retpoline_r10() vmlinux.o: warning: objtool: __x86_indirect_thunk_r11() falls through to next function __x86_retpoline_r11() vmlinux.o: warning: objtool: __x86_indirect_thunk_r12() falls through to next function __x86_retpoline_r12() vmlinux.o: warning: objtool: __x86_indirect_thunk_r13() falls through to next function __x86_retpoline_r13() vmlinux.o: warning: objtool: __x86_indirect_thunk_r14() falls through to next function __x86_retpoline_r14() vmlinux.o: warning: objtool: __x86_indirect_thunk_r15() falls through to next function __x86_retpoline_r15() ``` I think those should be resolved before I provide any kind of tested by tag. My other piece of feedback was that I like the default ThinLTO, but I think the help text in the Kconfig which is visible during menuconfig could be improved by informing the user the tradeoffs. For example, if CONFIG_THINLTO is disabled, it should be noted that full LTO will be used instead. Also, that full LTO may produce slightly better optimized binaries than ThinLTO, at the cost of not utilizing multiple cores when linking and thus significantly slower to link. Maybe explaining that setting it to "n" implies a full LTO build, which will be much slower to link but possibly slightly faster would be good? It's not visible unless LTO_CLANG and ARCH_SUPPORTS_THINLTO is enabled, so I don't think you need to explain that THINLTO without those is *not* full LTO. I'll leave the precise wording to you. WDYT? Also, when I look at your treewide DISABLE_LTO patch, I think "does that need to be a part of this series, or is it a cleanup that can stand on its own?" I think it may be the latter? Maybe it would help shed one more patch than to have to carry it to just send it? Or did I miss something as to why it should remain a part of this series? -- Thanks, ~Nick Desaulniers _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.1 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EC70DC4741F for ; Wed, 30 Sep 2020 21:58:32 +0000 (UTC) Received: from mother.openwall.net (mother.openwall.net [195.42.179.200]) by mail.kernel.org (Postfix) with SMTP id F2C9620658 for ; Wed, 30 Sep 2020 21:58:31 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="LHz9Zcam" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F2C9620658 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kernel-hardening-return-20073-kernel-hardening=archiver.kernel.org@lists.openwall.com Received: (qmail 22453 invoked by uid 550); 30 Sep 2020 21:58:25 -0000 Mailing-List: contact kernel-hardening-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Received: (qmail 22421 invoked from network); 30 Sep 2020 21:58:24 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=t4yHwQDoeC7Qpj3w+ohiUiQzIr7R18uaPZSHZDC/wBA=; b=LHz9Zcam7CeKHZ2FruDPRDuY0e07YHSqVvGalfh32T3Fuffnt2bHgr5rwgpKJmywr5 vYYvj2pj3Prw08WL6qW4WlpRdEOLUCrT+9hncLjFVN1Ft8Md5FT+cXeVzwCbCclkl92o hiVEXzmM9WAhHeXPsScWmhH08ZPlqMeItatEG49WX1OQPxwmMQyfc2iYMB/Z7txm5yp9 RRa0vfTlhSB0oEdLv191LDpEXlrqdDLR97VB59MmnkaiwZ4LOuqrpJLUtPF1GuZMwqmk sGSI3yP3GaxL9/tqCvhaOlXgx1uB7P44MkrgKvdzV+0gcVnwItyMO1mDDu9EjVXS5h+m ERsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=t4yHwQDoeC7Qpj3w+ohiUiQzIr7R18uaPZSHZDC/wBA=; b=M9WM3xXbifes8FX+IkW5dPpl1dlTgXi/MTykAviI2zH06M4lBfJ4TOKaPfbBfM5Qbk tj5HKouPqLfLwxEobnuokNASViDyaIG3C0SZxMq1esVbIlGdAbcfTq7ghijhZWZtvMd5 fs+HzvB1YEouqsO2zVs7V2iUKKEB6rqvq6mjpfUqP2GWIDUsosU5FZse+v/yVCmZ1JoU mMHvwOfHLELAtPp/FF3vbQA1APbDDgS0hrmf1XKSt6R2Mb9HQ9pgKzqGGhL82d1lwbLG MPHe3w3xkWF9KZWxqJhgHMLXbG3uueEWElhNsn/xyhqywRlJr4MSiS6QmXIPnZI2HfyZ lp0Q== X-Gm-Message-State: AOAM531n59UWfVbBRShetxTHxLdFIq2o2NH0wTkRwJzOKe667WHRMSm9 wJBDanto0iHlHJj6mFIGiTrPdpSA+nbNkn1OFdMY+g== X-Google-Smtp-Source: ABdhPJwzX9Viqx7zQJehaAJaWpHju7adBUzL4TnFBbDcXOlCvSY7YI0nAchHwIA2JZ5jrnbbiiAg20J+xNYGNfRzTkU= X-Received: by 2002:a17:902:c40d:b029:d2:93e8:1f4b with SMTP id k13-20020a170902c40db02900d293e81f4bmr4327278plk.29.1601503091998; Wed, 30 Sep 2020 14:58:11 -0700 (PDT) MIME-Version: 1.0 References: <20200929214631.3516445-1-samitolvanen@google.com> In-Reply-To: <20200929214631.3516445-1-samitolvanen@google.com> From: Nick Desaulniers Date: Wed, 30 Sep 2020 14:58:00 -0700 Message-ID: Subject: Re: [PATCH v4 00/29] Add support for Clang LTO To: Sami Tolvanen Cc: Masahiro Yamada , Will Deacon , Steven Rostedt , Peter Zijlstra , Greg Kroah-Hartman , "Paul E. McKenney" , Kees Cook , clang-built-linux , Kernel Hardening , linux-arch , Linux ARM , Linux Kbuild mailing list , LKML , linux-pci@vger.kernel.org, "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Content-Type: text/plain; charset="UTF-8" On Tue, Sep 29, 2020 at 2:46 PM Sami Tolvanen wrote: > > This patch series adds support for building x86_64 and arm64 kernels > with Clang's Link Time Optimization (LTO). > > In addition to performance, the primary motivation for LTO is > to allow Clang's Control-Flow Integrity (CFI) to be used in the > kernel. Google has shipped millions of Pixel devices running three > major kernel versions with LTO+CFI since 2018. > > Most of the patches are build system changes for handling LLVM > bitcode, which Clang produces with LTO instead of ELF object files, > postponing ELF processing until a later stage, and ensuring initcall > ordering. Sami, thanks for continuing to drive the series. I encourage you to keep resending with fixes accumulated or dropped on a weekly cadence. The series worked well for me on arm64, but for x86_64 on mainline I saw a stream of new objtool warnings: testing your LTO series; x86_64 defconfig + CONFIG_THINLTO: ``` LTO vmlinux.o OBJTOOL vmlinux.o vmlinux.o: warning: objtool: wakeup_long64()+0x61: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: .text+0x308a: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: .text+0x30c5: indirect jump found in RETPOLINE build vmlinux.o: warning: objtool: copy_user_enhanced_fast_string() falls through to next function copy_user_generic_unrolled() vmlinux.o: warning: objtool: __memcpy_mcsafe() falls through to next function mcsafe_handle_tail() vmlinux.o: warning: objtool: memset() falls through to next function memset_erms() vmlinux.o: warning: objtool: __memcpy() falls through to next function memcpy_erms() vmlinux.o: warning: objtool: __x86_indirect_thunk_rax() falls through to next function __x86_retpoline_rax() vmlinux.o: warning: objtool: __x86_indirect_thunk_rbx() falls through to next function __x86_retpoline_rbx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rcx() falls through to next function __x86_retpoline_rcx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rdx() falls through to next function __x86_retpoline_rdx() vmlinux.o: warning: objtool: __x86_indirect_thunk_rsi() falls through to next function __x86_retpoline_rsi() vmlinux.o: warning: objtool: __x86_indirect_thunk_rdi() falls through to next function __x86_retpoline_rdi() vmlinux.o: warning: objtool: __x86_indirect_thunk_rbp() falls through to next function __x86_retpoline_rbp() vmlinux.o: warning: objtool: __x86_indirect_thunk_r8() falls through to next function __x86_retpoline_r8() vmlinux.o: warning: objtool: __x86_indirect_thunk_r9() falls through to next function __x86_retpoline_r9() vmlinux.o: warning: objtool: __x86_indirect_thunk_r10() falls through to next function __x86_retpoline_r10() vmlinux.o: warning: objtool: __x86_indirect_thunk_r11() falls through to next function __x86_retpoline_r11() vmlinux.o: warning: objtool: __x86_indirect_thunk_r12() falls through to next function __x86_retpoline_r12() vmlinux.o: warning: objtool: __x86_indirect_thunk_r13() falls through to next function __x86_retpoline_r13() vmlinux.o: warning: objtool: __x86_indirect_thunk_r14() falls through to next function __x86_retpoline_r14() vmlinux.o: warning: objtool: __x86_indirect_thunk_r15() falls through to next function __x86_retpoline_r15() ``` I think those should be resolved before I provide any kind of tested by tag. My other piece of feedback was that I like the default ThinLTO, but I think the help text in the Kconfig which is visible during menuconfig could be improved by informing the user the tradeoffs. For example, if CONFIG_THINLTO is disabled, it should be noted that full LTO will be used instead. Also, that full LTO may produce slightly better optimized binaries than ThinLTO, at the cost of not utilizing multiple cores when linking and thus significantly slower to link. Maybe explaining that setting it to "n" implies a full LTO build, which will be much slower to link but possibly slightly faster would be good? It's not visible unless LTO_CLANG and ARCH_SUPPORTS_THINLTO is enabled, so I don't think you need to explain that THINLTO without those is *not* full LTO. I'll leave the precise wording to you. WDYT? Also, when I look at your treewide DISABLE_LTO patch, I think "does that need to be a part of this series, or is it a cleanup that can stand on its own?" I think it may be the latter? Maybe it would help shed one more patch than to have to carry it to just send it? Or did I miss something as to why it should remain a part of this series? -- Thanks, ~Nick Desaulniers