From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E92ECC433DB for ; Thu, 4 Mar 2021 22:17:11 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id B9F1264FF1 for ; Thu, 4 Mar 2021 22:17:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229861AbhCDWRK (ORCPT ); Thu, 4 Mar 2021 17:17:10 -0500 Received: from mail.kernel.org ([198.145.29.99]:53966 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229458AbhCDWRJ (ORCPT ); Thu, 4 Mar 2021 17:17:09 -0500 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5D58664FE4; Thu, 4 Mar 2021 22:17:09 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lHwHL-00HLP8-B7; Thu, 04 Mar 2021 22:17:07 +0000 Date: Thu, 04 Mar 2021 22:17:06 +0000 Message-ID: <878s72sgwt.wl-maz@kernel.org> From: Marc Zyngier To: Sami Tolvanen Cc: Fangrui Song , Nick Desaulniers , James Morse , Nathan Chancellor , Kees Cook , Julien Thierry , Suzuki K Poulose , Catalin Marinas , Will Deacon , linux-arm-kernel , kvmarm@lists.cs.columbia.edu, LKML Subject: Re: [PATCH] KVM: arm64: Disable LTO in hyp In-Reply-To: References: <20210304184544.2014171-1-samitolvanen@google.com> <87k0qmzq5u.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: samitolvanen@google.com, maskray@google.com, ndesaulniers@google.com, james.morse@arm.com, nathan@kernel.org, keescook@chromium.org, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 04 Mar 2021 21:25:41 +0000, Sami Tolvanen wrote: > > On Thu, Mar 4, 2021 at 11:15 AM Marc Zyngier wrote: > > > > On Thu, 04 Mar 2021 18:45:44 +0000, > > Sami Tolvanen wrote: > > > > > > allmodconfig + CONFIG_LTO_CLANG_THIN=y fails to build due to following > > > linker errors: > > > > > > ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21CC): > > > > I assume this message is only an oddity, right? Because > > __guest_enter() is as far as you can imagine from irqbypass.c... > > I'm not sure what's up with the filename in the error message. Fangrui > or Nick probably have a better idea. > > > > relocation R_AARCH64_CONDBR19 out of range: 2031220 is not in > > > [-1048576, 1048575]; references hyp_panic > > > >>> defined in vmlinux.o > > > > > > ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21E0): > > > relocation R_AARCH64_ADR_PREL_LO21 out of range: 2031200 is not in > > > [-1048576, 1048575]; references hyp_panic > > > >>> defined in vmlinux.o > > > > > > As LTO is not really necessary for the hypervisor code, disable it for > > > the hyp directory to fix the build. > > > > Can you shed some light on what the problem is exactly? > > I assume hyp_panic() ends up being placed too far from __guest_enter() > when the kernel is large enough. Possibly something to do with LLVM > always splitting functions into separate sections with LTO. I'm not > sure why the linker cannot shuffle things around to make everyone > happy in this case, but I confirmed that this patch also fixes the > build issue for me: > > diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c > index af8e940d0f03..128197b7c794 100644 > --- a/arch/arm64/kvm/hyp/vhe/switch.c > +++ b/arch/arm64/kvm/hyp/vhe/switch.c > @@ -214,7 +214,7 @@ static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) > } > NOKPROBE_SYMBOL(__hyp_call_panic); > > -void __noreturn hyp_panic(void) > +void __noreturn hyp_panic(void) __section(".text") > { > u64 spsr = read_sysreg_el2(SYS_SPSR); > u64 elr = read_sysreg_el2(SYS_ELR); > We're getting into black-magic territory here. Why wouldn't hyp_panic be in the .text section already? > > > > > > Link: https://github.com/ClangBuiltLinux/linux/issues/1317 > > > Reported-by: Nathan Chancellor > > > Tested-by: Nathan Chancellor > > > Signed-off-by: Sami Tolvanen > > > --- > > > arch/arm64/kvm/hyp/Makefile | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile > > > index 687598e41b21..e8116016e6a8 100644 > > > --- a/arch/arm64/kvm/hyp/Makefile > > > +++ b/arch/arm64/kvm/hyp/Makefile > > > @@ -11,3 +11,6 @@ subdir-ccflags-y := -I$(incdir) \ > > > $(DISABLE_STACKLEAK_PLUGIN) > > > > > > obj-$(CONFIG_KVM) += vhe/ nvhe/ pgtable.o > > > + > > > +# Disable LTO for the files in this directory > > > +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) > > > > > > base-commit: f69d02e37a85645aa90d18cacfff36dba370f797 > > > > Can this be reduced to the nvhe part of the tree? The rest of the > > hypervisor should support being built with LTO, I'd expect. Or am I > > missing something more significant? > > No, this error appears to be about hyp_panic() in the vhe code. While > I'm not sure how beneficial LTO is in hypervisor code, there shouldn't > be any other reason we can't use it there. The VHE part of the hypervisor is the kernel itself, and not any different from the rest of the code. It should be able to benefit from LTO. On the contrary, the nVHE part is what could do without LTO, given that it is essentially a separate object that happens to be bundled with the kernel. Thanks, M. -- Without deviation from the norm, progress is not possible. From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9D35C433DB for ; Thu, 4 Mar 2021 22:17:15 +0000 (UTC) Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by mail.kernel.org (Postfix) with ESMTP id 6C52A64FF0 for ; Thu, 4 Mar 2021 22:17:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 6C52A64FF0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=kvmarm-bounces@lists.cs.columbia.edu Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 0C4A94B555; Thu, 4 Mar 2021 17:17:15 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id WGJXLNagFnj2; Thu, 4 Mar 2021 17:17:13 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id CB39D4B5E9; Thu, 4 Mar 2021 17:17:13 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id C1C434B555 for ; Thu, 4 Mar 2021 17:17:12 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 39R8RFl+XCJA for ; Thu, 4 Mar 2021 17:17:11 -0500 (EST) Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 5CC124B2DC for ; Thu, 4 Mar 2021 17:17:11 -0500 (EST) Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5D58664FE4; Thu, 4 Mar 2021 22:17:09 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lHwHL-00HLP8-B7; Thu, 04 Mar 2021 22:17:07 +0000 Date: Thu, 04 Mar 2021 22:17:06 +0000 Message-ID: <878s72sgwt.wl-maz@kernel.org> From: Marc Zyngier To: Sami Tolvanen Subject: Re: [PATCH] KVM: arm64: Disable LTO in hyp In-Reply-To: References: <20210304184544.2014171-1-samitolvanen@google.com> <87k0qmzq5u.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: samitolvanen@google.com, maskray@google.com, ndesaulniers@google.com, james.morse@arm.com, nathan@kernel.org, keescook@chromium.org, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: Kees Cook , Fangrui Song , Catalin Marinas , Nick Desaulniers , LKML , Nathan Chancellor , linux-arm-kernel , Will Deacon , kvmarm@lists.cs.columbia.edu X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Thu, 04 Mar 2021 21:25:41 +0000, Sami Tolvanen wrote: > > On Thu, Mar 4, 2021 at 11:15 AM Marc Zyngier wrote: > > > > On Thu, 04 Mar 2021 18:45:44 +0000, > > Sami Tolvanen wrote: > > > > > > allmodconfig + CONFIG_LTO_CLANG_THIN=y fails to build due to following > > > linker errors: > > > > > > ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21CC): > > > > I assume this message is only an oddity, right? Because > > __guest_enter() is as far as you can imagine from irqbypass.c... > > I'm not sure what's up with the filename in the error message. Fangrui > or Nick probably have a better idea. > > > > relocation R_AARCH64_CONDBR19 out of range: 2031220 is not in > > > [-1048576, 1048575]; references hyp_panic > > > >>> defined in vmlinux.o > > > > > > ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21E0): > > > relocation R_AARCH64_ADR_PREL_LO21 out of range: 2031200 is not in > > > [-1048576, 1048575]; references hyp_panic > > > >>> defined in vmlinux.o > > > > > > As LTO is not really necessary for the hypervisor code, disable it for > > > the hyp directory to fix the build. > > > > Can you shed some light on what the problem is exactly? > > I assume hyp_panic() ends up being placed too far from __guest_enter() > when the kernel is large enough. Possibly something to do with LLVM > always splitting functions into separate sections with LTO. I'm not > sure why the linker cannot shuffle things around to make everyone > happy in this case, but I confirmed that this patch also fixes the > build issue for me: > > diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c > index af8e940d0f03..128197b7c794 100644 > --- a/arch/arm64/kvm/hyp/vhe/switch.c > +++ b/arch/arm64/kvm/hyp/vhe/switch.c > @@ -214,7 +214,7 @@ static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) > } > NOKPROBE_SYMBOL(__hyp_call_panic); > > -void __noreturn hyp_panic(void) > +void __noreturn hyp_panic(void) __section(".text") > { > u64 spsr = read_sysreg_el2(SYS_SPSR); > u64 elr = read_sysreg_el2(SYS_ELR); > We're getting into black-magic territory here. Why wouldn't hyp_panic be in the .text section already? > > > > > > Link: https://github.com/ClangBuiltLinux/linux/issues/1317 > > > Reported-by: Nathan Chancellor > > > Tested-by: Nathan Chancellor > > > Signed-off-by: Sami Tolvanen > > > --- > > > arch/arm64/kvm/hyp/Makefile | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile > > > index 687598e41b21..e8116016e6a8 100644 > > > --- a/arch/arm64/kvm/hyp/Makefile > > > +++ b/arch/arm64/kvm/hyp/Makefile > > > @@ -11,3 +11,6 @@ subdir-ccflags-y := -I$(incdir) \ > > > $(DISABLE_STACKLEAK_PLUGIN) > > > > > > obj-$(CONFIG_KVM) += vhe/ nvhe/ pgtable.o > > > + > > > +# Disable LTO for the files in this directory > > > +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) > > > > > > base-commit: f69d02e37a85645aa90d18cacfff36dba370f797 > > > > Can this be reduced to the nvhe part of the tree? The rest of the > > hypervisor should support being built with LTO, I'd expect. Or am I > > missing something more significant? > > No, this error appears to be about hyp_panic() in the vhe code. While > I'm not sure how beneficial LTO is in hypervisor code, there shouldn't > be any other reason we can't use it there. The VHE part of the hypervisor is the kernel itself, and not any different from the rest of the code. It should be able to benefit from LTO. On the contrary, the nVHE part is what could do without LTO, given that it is essentially a separate object that happens to be bundled with the kernel. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI, MENTIONS_GIT_HOSTING,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 08AC8C433DB for ; Thu, 4 Mar 2021 22:19:14 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 91ABC64FF1 for ; Thu, 4 Mar 2021 22:19:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 91ABC64FF1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To:Subject:Cc:To: From:Message-ID:Date:Reply-To:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=/f/2v3dU0sS5nLAu0VtUEyy1MmL9m2kWnL1qPrjfNoQ=; b=BeeOnC7NSQE1uBqGdCNcUafpk 53a8gloHhxc/p1axC0YQGTc8ZefmbcEc0EIf7KUgnZjN4RRFD5+W82uKp8OwRMjUSo8NXm5vNvm5P 9BnAishPxd+bVUl0SEVgFl5PBCTFs10JAa6xNH5QcwZeiBQQ7P7E9FelRC3K1YlqgL6xZTeH8boR2 6Y/F07aHRg2qCIahPllEffniaptCFDZ528/uj536UDcEhPb1frO4FrCwnqxLeNNXfZn+kKnQnm8Cw TuseotjPJiWDmqzXYACe8HjBVXtNHwjQwxUo9uQsgzosA8FD3FLy//bLXsCSHgsNimwyqedJ6/JqC yOi7wJ16Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lHwHU-00APrB-FX; Thu, 04 Mar 2021 22:17:16 +0000 Received: from mail.kernel.org ([198.145.29.99]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lHwHP-00APqF-4B for linux-arm-kernel@lists.infradead.org; Thu, 04 Mar 2021 22:17:13 +0000 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 5D58664FE4; Thu, 4 Mar 2021 22:17:09 +0000 (UTC) Received: from 78.163-31-62.static.virginmediabusiness.co.uk ([62.31.163.78] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94) (envelope-from ) id 1lHwHL-00HLP8-B7; Thu, 04 Mar 2021 22:17:07 +0000 Date: Thu, 04 Mar 2021 22:17:06 +0000 Message-ID: <878s72sgwt.wl-maz@kernel.org> From: Marc Zyngier To: Sami Tolvanen Cc: Fangrui Song , Nick Desaulniers , James Morse , Nathan Chancellor , Kees Cook , Julien Thierry , Suzuki K Poulose , Catalin Marinas , Will Deacon , linux-arm-kernel , kvmarm@lists.cs.columbia.edu, LKML Subject: Re: [PATCH] KVM: arm64: Disable LTO in hyp In-Reply-To: References: <20210304184544.2014171-1-samitolvanen@google.com> <87k0qmzq5u.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 62.31.163.78 X-SA-Exim-Rcpt-To: samitolvanen@google.com, maskray@google.com, ndesaulniers@google.com, james.morse@arm.com, nathan@kernel.org, keescook@chromium.org, julien.thierry.kdev@gmail.com, suzuki.poulose@arm.com, catalin.marinas@arm.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, 04 Mar 2021 21:25:41 +0000, Sami Tolvanen wrote: > > On Thu, Mar 4, 2021 at 11:15 AM Marc Zyngier wrote: > > > > On Thu, 04 Mar 2021 18:45:44 +0000, > > Sami Tolvanen wrote: > > > > > > allmodconfig + CONFIG_LTO_CLANG_THIN=y fails to build due to following > > > linker errors: > > > > > > ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21CC): > > > > I assume this message is only an oddity, right? Because > > __guest_enter() is as far as you can imagine from irqbypass.c... > > I'm not sure what's up with the filename in the error message. Fangrui > or Nick probably have a better idea. > > > > relocation R_AARCH64_CONDBR19 out of range: 2031220 is not in > > > [-1048576, 1048575]; references hyp_panic > > > >>> defined in vmlinux.o > > > > > > ld.lld: error: irqbypass.c:(function __guest_enter: .text+0x21E0): > > > relocation R_AARCH64_ADR_PREL_LO21 out of range: 2031200 is not in > > > [-1048576, 1048575]; references hyp_panic > > > >>> defined in vmlinux.o > > > > > > As LTO is not really necessary for the hypervisor code, disable it for > > > the hyp directory to fix the build. > > > > Can you shed some light on what the problem is exactly? > > I assume hyp_panic() ends up being placed too far from __guest_enter() > when the kernel is large enough. Possibly something to do with LLVM > always splitting functions into separate sections with LTO. I'm not > sure why the linker cannot shuffle things around to make everyone > happy in this case, but I confirmed that this patch also fixes the > build issue for me: > > diff --git a/arch/arm64/kvm/hyp/vhe/switch.c b/arch/arm64/kvm/hyp/vhe/switch.c > index af8e940d0f03..128197b7c794 100644 > --- a/arch/arm64/kvm/hyp/vhe/switch.c > +++ b/arch/arm64/kvm/hyp/vhe/switch.c > @@ -214,7 +214,7 @@ static void __hyp_call_panic(u64 spsr, u64 elr, u64 par) > } > NOKPROBE_SYMBOL(__hyp_call_panic); > > -void __noreturn hyp_panic(void) > +void __noreturn hyp_panic(void) __section(".text") > { > u64 spsr = read_sysreg_el2(SYS_SPSR); > u64 elr = read_sysreg_el2(SYS_ELR); > We're getting into black-magic territory here. Why wouldn't hyp_panic be in the .text section already? > > > > > > Link: https://github.com/ClangBuiltLinux/linux/issues/1317 > > > Reported-by: Nathan Chancellor > > > Tested-by: Nathan Chancellor > > > Signed-off-by: Sami Tolvanen > > > --- > > > arch/arm64/kvm/hyp/Makefile | 3 +++ > > > 1 file changed, 3 insertions(+) > > > > > > diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile > > > index 687598e41b21..e8116016e6a8 100644 > > > --- a/arch/arm64/kvm/hyp/Makefile > > > +++ b/arch/arm64/kvm/hyp/Makefile > > > @@ -11,3 +11,6 @@ subdir-ccflags-y := -I$(incdir) \ > > > $(DISABLE_STACKLEAK_PLUGIN) > > > > > > obj-$(CONFIG_KVM) += vhe/ nvhe/ pgtable.o > > > + > > > +# Disable LTO for the files in this directory > > > +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_LTO), $(KBUILD_CFLAGS)) > > > > > > base-commit: f69d02e37a85645aa90d18cacfff36dba370f797 > > > > Can this be reduced to the nvhe part of the tree? The rest of the > > hypervisor should support being built with LTO, I'd expect. Or am I > > missing something more significant? > > No, this error appears to be about hyp_panic() in the vhe code. While > I'm not sure how beneficial LTO is in hypervisor code, there shouldn't > be any other reason we can't use it there. The VHE part of the hypervisor is the kernel itself, and not any different from the rest of the code. It should be able to benefit from LTO. On the contrary, the nVHE part is what could do without LTO, given that it is essentially a separate object that happens to be bundled with the kernel. Thanks, M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel