From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,FSL_HELO_FAKE, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=no autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7D2CAC43461 for ; Thu, 10 Sep 2020 15:18:42 +0000 (UTC) Received: from merlin.infradead.org (merlin.infradead.org [205.233.59.134]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0F482206CA for ; Thu, 10 Sep 2020 15:18:41 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (2048-bit key) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="EPKOveBP"; dkim=fail reason="signature verification failed" (2048-bit key) header.d=google.com header.i=@google.com header.b="DChiCwze" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0F482206CA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=merlin.20170209; h=Sender:Content-Transfer-Encoding: Content-Type:Cc:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References:Message-ID: Subject:To:From:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ObcoyzTOfv7imqIK/HZSy8rgAe8bMemtBKojMvEkp1s=; b=EPKOveBP0AAI1L9hC0lLTAArM 7ow+i7JnPiJ01RsXMKlcqKzfTTPXOqUBIynM+iLYohXmScXxTjb7t2od/14o1MD1xEfRB8HQ24njp OLkyUcHtFh6PwTVJbBHqWnRPBKHJnAiE1dI5usqKt1RHKm0GUQClqPbIBbJRBUdmNEqXRMENzQHRy gXKOCM6s/5SGGWQYM2f5rsxnNE1JXf+Ctc8YPJCX6Ae8cyvlJCPd4VzYvzwCKOoZ1EEcvy/moQIls 3Uu3Pp8chHI5GJlIBTfCcv8wtH+/bGx5Ei5zWstdZMMgk+WonScR9epNlkd9CtTm2bAElcbYbjiK8 9Yv7jblgg==; Received: from localhost ([::1] helo=merlin.infradead.org) by merlin.infradead.org with esmtp (Exim 4.92.3 #3 (Red Hat Linux)) id 1kGOKE-0001PK-GH; Thu, 10 Sep 2020 15:17:26 +0000 Received: from mail-pl1-x642.google.com ([2607:f8b0:4864:20::642]) by merlin.infradead.org with esmtps (Exim 4.92.3 #3 (Red Hat Linux)) id 1kGOKC-0001OO-HO for linux-arm-kernel@lists.infradead.org; Thu, 10 Sep 2020 15:17:25 +0000 Received: by mail-pl1-x642.google.com with SMTP id x18so1070554pll.6 for ; Thu, 10 Sep 2020 08:17:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=tPQ89cuIIRpR65Gxj5ecjevE7KdkPR+rkFKg2EWJ1Wo=; b=DChiCwzeZGIWq9/ed/gKC6HSdKRqhEs7Cl7tIqKCGERK9KOYJoH5HR7PeRQFbOp0ZQ khZeS1vnyB3+LRhvPvqPzg1++dxbQ65OlIG3+zElXCaofAPycZykfEboSqoECnwgYkc8 MjBJnNXFCtUW2Ff6DrHl+sxDflI9XbnpK+uT5XCk1aFiT5HbdZScO79bQK/bPprVs9ke AKI0Gc5Gmjyb5KEUqFWbhTCjq+hID3NQ5SFYjD5nGdby6rdJDucAdfiBRYrvqN5RUvSp a/e3IDc7KLOgC5+PLCtG+ltNKm9AU2kQQzP0O8rhwbvaICVCr6rc7DiwJMJDGG5odV8E t8+w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=tPQ89cuIIRpR65Gxj5ecjevE7KdkPR+rkFKg2EWJ1Wo=; b=A6q+NdlynD0fY19gKCE6xzOZouMHAIlKJXxciJDxBa0JF1Sb13TqqZMj2whfNykLby 35kqJvcQJyO4nzW8SCrHzIOZkaPE53FE3ZN2EVeYJG5Foa93/AWF2KC7wQ5929xppvz5 344V36zgPFXCwwnUmNn7Lm1ufhWtIPNvLtVfNTgOiLPGyWEwGan7PVPqx1K6EaCWk/AC ToGuWk4WePzJ7Do0000ibfoWDOc8qG6k9HQVzkVRk2ilUGmAiQ8E9OL9HCWMZMX6dzkF XUMJbNww7AP9D37qnh06xcwjckvJE/GZKV2+JMYtFrE/X4QrnimtxmzSElLDjn58f+qU bu+Q== X-Gm-Message-State: AOAM531Vobg0rARCAbXRWEywvYiW4rUD/h8/PDFPLcmOlx7oQ0Cy73cc Q8T0E3JXnrgzby4j3IKx7cBnFw== X-Google-Smtp-Source: ABdhPJzN/GMDnhYVZz/m1t04YSzXdezukOM0X0fLu2KC5nqmbsead/T/1/lyPGlr1z5YJ1TUoUk/HA== X-Received: by 2002:a17:90b:1915:: with SMTP id mp21mr407790pjb.116.1599751042106; Thu, 10 Sep 2020 08:17:22 -0700 (PDT) Received: from google.com ([2620:15c:201:2:f693:9fff:fef4:1b6d]) by smtp.gmail.com with ESMTPSA id z23sm5241512pgv.57.2020.09.10.08.17.20 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 10 Sep 2020 08:17:20 -0700 (PDT) Date: Thu, 10 Sep 2020 08:17:15 -0700 From: Sami Tolvanen To: Masahiro Yamada Subject: Re: [PATCH v2 00/28] Add support for Clang LTO Message-ID: <20200910151715.GB2041735@google.com> References: <20200624203200.78870-1-samitolvanen@google.com> <20200903203053.3411268-1-samitolvanen@google.com> <20200908234643.GF1060586@google.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20200910_111724_613397_D51A20C7 X-CRM114-Status: GOOD ( 46.46 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-arch , X86 ML , Kees Cook , "Paul E. McKenney" , Kernel Hardening , Peter Zijlstra , Greg Kroah-Hartman , Linux Kbuild mailing list , Nick Desaulniers , Linux Kernel Mailing List , Steven Rostedt , clang-built-linux , linux-pci@vger.kernel.org, Will Deacon , linux-arm-kernel Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, Sep 10, 2020 at 10:18:05AM +0900, Masahiro Yamada wrote: > On Wed, Sep 9, 2020 at 8:46 AM Sami Tolvanen wrote: > > > > On Sun, Sep 06, 2020 at 09:24:38AM +0900, Masahiro Yamada wrote: > > > On Fri, Sep 4, 2020 at 5:30 AM Sami Tolvanen wrote: > > > > > > > > This patch series adds support for building x86_64 and arm64 kernels > > > > with Clang's Link Time Optimization (LTO). > > > > > > > > In addition to performance, the primary motivation for LTO is > > > > to allow Clang's Control-Flow Integrity (CFI) to be used in the > > > > kernel. Google has shipped millions of Pixel devices running three > > > > major kernel versions with LTO+CFI since 2018. > > > > > > > > Most of the patches are build system changes for handling LLVM > > > > bitcode, which Clang produces with LTO instead of ELF object files, > > > > postponing ELF processing until a later stage, and ensuring initcall > > > > ordering. > > > > > > > > Note that patches 1-4 are not directly related to LTO, but are > > > > needed to compile LTO kernels with ToT Clang, so I'm including them > > > > in the series for your convenience: > > > > > > > > - Patches 1-3 are required for building the kernel with ToT Clang, > > > > and IAS, and patch 4 is needed to build allmodconfig with LTO. > > > > > > > > - Patches 3-4 are already in linux-next, but not yet in 5.9-rc. > > > > > > > > > > > > > I still do not understand how this patch set works. > > > (only me?) > > > > > > Please let me ask fundamental questions. > > > > > > > > > > > > I applied this series on top of Linus' tree, > > > and compiled for ARCH=arm64. > > > > > > I compared the kernel size with/without LTO. > > > > > > > > > > > > [1] No LTO (arm64 defconfig, CONFIG_LTO_NONE) > > > > > > $ llvm-size vmlinux > > > text data bss dec hex filename > > > 15848692 10099449 493060 26441201 19375f1 vmlinux > > > > > > > > > > > > [2] Clang LTO (arm64 defconfig + CONFIG_LTO_CLANG) > > > > > > $ llvm-size vmlinux > > > text data bss dec hex filename > > > 15906864 10197445 490804 26595113 195cf29 vmlinux > > > > > > > > > I compared the size of raw binary, arch/arm64/boot/Image. > > > Its size increased too. > > > > > > > > > > > > So, in my experiment, enabling CONFIG_LTO_CLANG > > > increases the kernel size. > > > Is this correct? > > > > Yes. LTO does produce larger binaries, mostly due to function > > inlining between translation units, I believe. The compiler people > > can probably give you a more detailed answer here. Without -mllvm > > -import-instr-limit, the binaries would be even larger. > > > > > One more thing, could you teach me > > > how Clang LTO optimizes the code against > > > relocatable objects? > > > > > > > > > > > > When I learned Clang LTO first, I read this document: > > > https://llvm.org/docs/LinkTimeOptimization.html > > > > > > It is easy to confirm the final executable > > > does not contain foo2, foo3... > > > > > > > > > > > > In contrast to userspace programs, > > > kernel modules are basically relocatable objects. > > > > > > Does Clang drop unused symbols from relocatable objects? > > > If so, how? > > > > I don't think the compiler can legally drop global symbols from > > relocatable objects, but it can rename and possibly even drop static > > functions. > > > Compilers can drop static functions without LTO. > Rather, it is a compiler warning > (-Wunused-function), so the code should be cleaned up. > > > > > This is why we need global wrappers for initcalls, for > > example, to have stable symbol names. > > > > Sami > > > > At first, I thought the motivation of LTO > was to remove unused global symbols, and > to perform further optimization. > > > It is true for userspace programs. > In fact, the example of > https://llvm.org/docs/LinkTimeOptimization.html > produces a smaller binary. > > > In contrast, this patch set produces a bigger kernel > because LTO cannot remove any unused symbol. > > So, I do not understand what the benefit is. > > > Is inlining beneficial? > I am not sure. > > > Documentation/process/coding-style.rst > "15) The inline disease" > mentions that inlining is not always > a good thing. > > > As a whole, I still do not understand > the motivation of this patch set. Clang produces faster code with LTO even if unused functions are not removed, and I'm not sure how many unused globals there really are in the kernel that aren't exported for modules. However, as I mentioned in the cover letter, we also need LTO for Control-Flow Integrity (CFI), which we have used in Pixel kernels for a couple of years now, and plan to use in more Android devices in future: https://clang.llvm.org/docs/ControlFlowIntegrity.html Sami _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel