From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-12.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDC36C56201 for ; Sat, 21 Nov 2020 03:14:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BCC2724137 for ; Sat, 21 Nov 2020 03:14:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gsNR/Cpm" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727621AbgKUDON (ORCPT ); Fri, 20 Nov 2020 22:14:13 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38112 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727358AbgKUDOM (ORCPT ); Fri, 20 Nov 2020 22:14:12 -0500 Received: from mail-qt1-x841.google.com (mail-qt1-x841.google.com [IPv6:2607:f8b0:4864:20::841]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 7A4CCC0613CF; Fri, 20 Nov 2020 19:14:12 -0800 (PST) Received: by mail-qt1-x841.google.com with SMTP id e10so1289579qte.4; Fri, 20 Nov 2020 19:14:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=MnceUBKO5CKDULZmr15ARJRXunPRBIoWz0MuHJiULrg=; b=gsNR/CpmvNQ3U+ijqCwO8esji7bm1QnpPb+gjcQIHmOVilGJtCH7wljw5OuoR2+ByD Eu3ArHbg0KOX4QqUwemDaTkU/zaCmVz9NXecmCRSJQykGNIqev281MSCPFW6iyFfCalh OXU5NgKpKUUxeSGcDHdTDiB+8xUj7RQbdjIsa52m2B9JAVMTC4wE43lynK1gJnIiNcfD 9l3VbICmSeaGwxXVG1ypIdI92hsCdUXBe06U+zSDwzqwaBADLFUpJVMg61p6t8YeCIac RAunoKa4yJRKwIKPoTystOV6bK3iAFZIuzfr8A4Wxoe8TDYQp/m8PUNb++rax2JElIaq lP0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=MnceUBKO5CKDULZmr15ARJRXunPRBIoWz0MuHJiULrg=; b=EgbDUOfMxzfU4Cj+0mbak3nJaclCFNOiwP1R5ABmxMSKucFyaViGAZsqXmZj5pgiJL R0l0ePWME+9XeTbwBclsEevSzCB3FuN8okiwSI5xt/htcgjjEfzOFKn7lhXK9ZOjOmde qCPy4+lwCSzfJvK+/rOHIy9t7Ic5QQCd2Do29dH9co8QEYAK9N1WbusYoj1R6JizEM6t P1+XRpLwhSftomTz5OioeRzqDSwyEXXY+rlg/micHbnciLX3L69l1chOP28von0uyKAy ncwp1gwjghzYth7T0IOq+af9GG3Y/UIFZey65Oun6wL2AiRNksR+LT/9TJWKRRdrA4YE /cfw== X-Gm-Message-State: AOAM532N2ds0qWRc/xPpf9pcEIxG35BVEqWzsmLBLROllNi7gE25wdm6 RvnbzrdEwpHO55J7RdNIbOU= X-Google-Smtp-Source: ABdhPJwFM1Zex8JyH4dWcYr9Ean4nPf9JBfv8fPcfexg+IsjIR+dtwfqcokR9OMKLw5/d/3ypGQmrg== X-Received: by 2002:a05:622a:1cd:: with SMTP id t13mr19064421qtw.39.1605928451613; Fri, 20 Nov 2020 19:14:11 -0800 (PST) Received: from ubuntu-m3-large-x86 ([2604:1380:45f1:1d00::1]) by smtp.gmail.com with ESMTPSA id o9sm3406683qte.35.2020.11.20.19.14.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2020 19:14:10 -0800 (PST) Date: Fri, 20 Nov 2020 20:14:09 -0700 From: Nathan Chancellor To: Ard Biesheuvel Cc: Nick Desaulniers , Sami Tolvanen , Masahiro Yamada , Steven Rostedt , Will Deacon , Josh Poimboeuf , Peter Zijlstra , Greg Kroah-Hartman , "Paul E. McKenney" , Kees Cook , clang-built-linux , Kernel Hardening , linux-arch , Linux ARM , Linux Kbuild mailing list , LKML , PCI Subject: Re: [PATCH v7 00/17] Add support for Clang LTO Message-ID: <20201121031409.GA2282710@ubuntu-m3-large-x86> References: <20201118220731.925424-1-samitolvanen@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Nov 20, 2020 at 11:29:51AM +0100, Ard Biesheuvel wrote: > On Thu, 19 Nov 2020 at 00:42, Nick Desaulniers wrote: > > > > On Wed, Nov 18, 2020 at 2:07 PM Sami Tolvanen wrote: > > > > > > This patch series adds support for building the kernel with Clang's > > > Link Time Optimization (LTO). In addition to performance, the primary > > > motivation for LTO is to allow Clang's Control-Flow Integrity (CFI) to > > > be used in the kernel. Google has shipped millions of Pixel devices > > > running three major kernel versions with LTO+CFI since 2018. > > > > > > Most of the patches are build system changes for handling LLVM bitcode, > > > which Clang produces with LTO instead of ELF object files, postponing > > > ELF processing until a later stage, and ensuring initcall ordering. > > > > > > Note that v7 brings back arm64 support as Will has now staged the > > > prerequisite memory ordering patches [1], and drops x86_64 while we work > > > on fixing the remaining objtool warnings [2]. > > > > > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/lto > > > [2] https://lore.kernel.org/lkml/20201114004911.aip52eimk6c2uxd4@treble/ > > > > > > You can also pull this series from > > > > > > https://github.com/samitolvanen/linux.git lto-v7 > > > > Thanks for continuing to drive this series Sami. For the series, > > > > Tested-by: Nick Desaulniers > > > > I did virtualized boot tests with the series applied to aarch64 > > defconfig without CONFIG_LTO, with CONFIG_LTO_CLANG, and a third time > > with CONFIG_THINLTO. If you make changes to the series in follow ups, > > please drop my tested by tag from the modified patches and I'll help > > re-test. Some minor feedback on the Kconfig change, but I'll post it > > off of that patch. > > > > When you say 'virtualized" do you mean QEMU on x86? Or actual > virtualization on an AArch64 KVM host? > > The distinction is important here, given the potential impact of LTO > on things that QEMU simply does not model when it runs in TCG mode on > a foreign host architecture. I have booted this series on my Raspberry Pi 4 (ARCH=arm64 defconfig). $ uname -r 5.10.0-rc4-00108-g830200082c74 $ zgrep LTO /proc/config.gz CONFIG_LTO=y CONFIG_ARCH_SUPPORTS_LTO_CLANG=y CONFIG_ARCH_SUPPORTS_THINLTO=y CONFIG_THINLTO=y # CONFIG_LTO_NONE is not set CONFIG_LTO_CLANG=y # CONFIG_HID_WALTOP is not set and I have taken that same kernel and booted it under QEMU with '-enable-kvm' without any visible issues. I have tested four combinations: clang 12 @ f9f0a4046e11c2b4c130640f343e3b2b5db08c1: * CONFIG_THINLTO=y * CONFIG_THINLTO=n clang 11.0.0 * CONFIG_THINLTO=y * CONFIG_THINLTO=n Tested-by: Nathan Chancellor Cheers, Nathan From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-10.7 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A2380C2D0E4 for ; Sat, 21 Nov 2020 03:14:41 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4B9EF20727 for ; Sat, 21 Nov 2020 03:14:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="Wks6aroZ"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="gsNR/Cpm" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4B9EF20727 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=CNi80i0Gjcmd0UESFxdrZo1y66nXbrrmdq2oRpL08bo=; b=Wks6aroZ4huVZjTYBxwvng10k SvxC+e+XvHGZPXgd7Yv+XporzMkDL8DKuNh/bPKdXFobk3O23IJZmQP16qP6vMA+oEMNmfYbQd/n9 Mr+XtpyrapzoEdzse17UsfYCGbipj0p1BwHkGsIhCrg7B+p8QXdSLCYPaQG6XSDOXSzJYqSSCtTkz xDxvuRgLQWhMsYI5oTrn0WRIbMh+Wxqc9sIL3p0Vaz1jsD3XdA7FWBwlr00B/jfqvqPfji19MmgMr NOC53pSDXIvZcCOySALWtYh5fYmDtYkIJnvuYxwSxfn4sIARbz7nCwQuFOORSCtqhKGQEFEJfoErn 1DTGX0J0w==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kgJLs-00011D-9S; Sat, 21 Nov 2020 03:14:16 +0000 Received: from mail-qt1-x844.google.com ([2607:f8b0:4864:20::844]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kgJLp-000100-AJ for linux-arm-kernel@lists.infradead.org; Sat, 21 Nov 2020 03:14:14 +0000 Received: by mail-qt1-x844.google.com with SMTP id g17so8764649qts.5 for ; Fri, 20 Nov 2020 19:14:13 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=MnceUBKO5CKDULZmr15ARJRXunPRBIoWz0MuHJiULrg=; b=gsNR/CpmvNQ3U+ijqCwO8esji7bm1QnpPb+gjcQIHmOVilGJtCH7wljw5OuoR2+ByD Eu3ArHbg0KOX4QqUwemDaTkU/zaCmVz9NXecmCRSJQykGNIqev281MSCPFW6iyFfCalh OXU5NgKpKUUxeSGcDHdTDiB+8xUj7RQbdjIsa52m2B9JAVMTC4wE43lynK1gJnIiNcfD 9l3VbICmSeaGwxXVG1ypIdI92hsCdUXBe06U+zSDwzqwaBADLFUpJVMg61p6t8YeCIac RAunoKa4yJRKwIKPoTystOV6bK3iAFZIuzfr8A4Wxoe8TDYQp/m8PUNb++rax2JElIaq lP0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=MnceUBKO5CKDULZmr15ARJRXunPRBIoWz0MuHJiULrg=; b=Hebv7B73TsJqp6Ul6OB/wtCEKDWQPFDbNBi/vWEBpyi/UKMrxXlD12oHA5Ly7ZrnoL uvWQqXrh8rE1HlQkuUPGlxIZmXY25j5qMPX/SRRlyWh4L0SDgdJIrXWJio/D4AmJBe8B uqVgMXGrUX3/7RrEpNmHOwcg+Z69X/aLAeoLnlVK3+xZ5h5qGpJ+oaPc8Wcrh2bQFueA kAvZ0NQmWDzXYfgqdSGnVH0oIHEF0/GYhvRB6c3qm8Oz7mE9SdKD+dp6OsM/g1v+vTDW AZH+XL63n7rprRyOK+pNN3Ujgi6TyNIrhgBtBGMkadgi8iosC5k8STxmaZ1I5ihyvvXE V46w== X-Gm-Message-State: AOAM530PYw75h5SO8/4Z8MIgVAQYe9TOqrsCjLKiI8NZOmeAMC0bYZJO mrIdFpSbe/lh5CIGyhqONqwYCWc9sHQ= X-Google-Smtp-Source: ABdhPJwFM1Zex8JyH4dWcYr9Ean4nPf9JBfv8fPcfexg+IsjIR+dtwfqcokR9OMKLw5/d/3ypGQmrg== X-Received: by 2002:a05:622a:1cd:: with SMTP id t13mr19064421qtw.39.1605928451613; Fri, 20 Nov 2020 19:14:11 -0800 (PST) Received: from ubuntu-m3-large-x86 ([2604:1380:45f1:1d00::1]) by smtp.gmail.com with ESMTPSA id o9sm3406683qte.35.2020.11.20.19.14.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Nov 2020 19:14:10 -0800 (PST) Date: Fri, 20 Nov 2020 20:14:09 -0700 From: Nathan Chancellor To: Ard Biesheuvel Subject: Re: [PATCH v7 00/17] Add support for Clang LTO Message-ID: <20201121031409.GA2282710@ubuntu-m3-large-x86> References: <20201118220731.925424-1-samitolvanen@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20201120_221413_454276_0596158A X-CRM114-Status: GOOD ( 33.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch , Kees Cook , "Paul E. McKenney" , Kernel Hardening , Peter Zijlstra , Greg Kroah-Hartman , Masahiro Yamada , Linux Kbuild mailing list , Nick Desaulniers , LKML , Steven Rostedt , clang-built-linux , PCI , Sami Tolvanen , Josh Poimboeuf , Will Deacon , Linux ARM Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, Nov 20, 2020 at 11:29:51AM +0100, Ard Biesheuvel wrote: > On Thu, 19 Nov 2020 at 00:42, Nick Desaulniers wrote: > > > > On Wed, Nov 18, 2020 at 2:07 PM Sami Tolvanen wrote: > > > > > > This patch series adds support for building the kernel with Clang's > > > Link Time Optimization (LTO). In addition to performance, the primary > > > motivation for LTO is to allow Clang's Control-Flow Integrity (CFI) to > > > be used in the kernel. Google has shipped millions of Pixel devices > > > running three major kernel versions with LTO+CFI since 2018. > > > > > > Most of the patches are build system changes for handling LLVM bitcode, > > > which Clang produces with LTO instead of ELF object files, postponing > > > ELF processing until a later stage, and ensuring initcall ordering. > > > > > > Note that v7 brings back arm64 support as Will has now staged the > > > prerequisite memory ordering patches [1], and drops x86_64 while we work > > > on fixing the remaining objtool warnings [2]. > > > > > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git/log/?h=for-next/lto > > > [2] https://lore.kernel.org/lkml/20201114004911.aip52eimk6c2uxd4@treble/ > > > > > > You can also pull this series from > > > > > > https://github.com/samitolvanen/linux.git lto-v7 > > > > Thanks for continuing to drive this series Sami. For the series, > > > > Tested-by: Nick Desaulniers > > > > I did virtualized boot tests with the series applied to aarch64 > > defconfig without CONFIG_LTO, with CONFIG_LTO_CLANG, and a third time > > with CONFIG_THINLTO. If you make changes to the series in follow ups, > > please drop my tested by tag from the modified patches and I'll help > > re-test. Some minor feedback on the Kconfig change, but I'll post it > > off of that patch. > > > > When you say 'virtualized" do you mean QEMU on x86? Or actual > virtualization on an AArch64 KVM host? > > The distinction is important here, given the potential impact of LTO > on things that QEMU simply does not model when it runs in TCG mode on > a foreign host architecture. I have booted this series on my Raspberry Pi 4 (ARCH=arm64 defconfig). $ uname -r 5.10.0-rc4-00108-g830200082c74 $ zgrep LTO /proc/config.gz CONFIG_LTO=y CONFIG_ARCH_SUPPORTS_LTO_CLANG=y CONFIG_ARCH_SUPPORTS_THINLTO=y CONFIG_THINLTO=y # CONFIG_LTO_NONE is not set CONFIG_LTO_CLANG=y # CONFIG_HID_WALTOP is not set and I have taken that same kernel and booted it under QEMU with '-enable-kvm' without any visible issues. I have tested four combinations: clang 12 @ f9f0a4046e11c2b4c130640f343e3b2b5db08c1: * CONFIG_THINLTO=y * CONFIG_THINLTO=n clang 11.0.0 * CONFIG_THINLTO=y * CONFIG_THINLTO=n Tested-by: Nathan Chancellor Cheers, Nathan _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel