Re: [RFC PATCH v2] x86/sev: enforce RIP-relative accesses in early SEV/SME code

From: "Hou Wenlong" <houwenlong.hwl@antgroup.com>
To: Kevin Loughlin <kevinloughlin@google.com>
Cc: Borislav Petkov <bp@alien8.de>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	x86@kernel.org, "H. Peter Anvin" <hpa@zytor.com>,
	Andy Lutomirski <luto@kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Nathan Chancellor <nathan@kernel.org>,
	Nick Desaulniers <ndesaulniers@google.com>,
	Bill Wendling <morbo@google.com>,
	Justin Stitt <justinstitt@google.com>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	Michael Kelley <mikelley@microsoft.com>,
	Pankaj Gupta <pankaj.gupta@amd.com>,
	Stephen Rothwell <sfr@canb.auug.org.au>,
	Arnd Bergmann <arnd@arndb.de>,
	Steve Rutherford <srutherford@google.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Vegard Nossum <vegard.nossum@oracle.com>,
	Josh Poimboeuf <jpoimboe@kernel.org>,
	Yuntao Wang <ytcoode@gmail.com>,
	Wang Jinchao <wangjinchao@xfusion.com>,
	David Woodhouse <dwmw@amazon.co.uk>,
	Brian Gerst <brgerst@gmail.com>, Hugh Dickins <hughd@google.com>,
	Ard Biesheuvel <ardb@kernel.org>, Joerg Roedel <jroedel@suse.de>,
	Randy Dunlap <rdunlap@infradead.org>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Dionna Glaze <dionnaglaze@google.com>,
	Brijesh Singh <brijesh.singh@amd.com>,
	Michael Roth <michael.roth@amd.com>,
	"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
	linux-kernel@vger.kernel.org, llvm@lists.linux.dev,
	linux-coco@lists.linux.dev, Ashish Kalra <ashish.kalra@amd.com>,
	Andi Kleen <ak@linux.intel.com>,
	Adam Dunlap <acdunlap@google.com>,
	Peter Gonda <pgonda@google.com>, Jacob Xu <jacobhxu@google.com>,
	Sidharth Telang <sidtelang@google.com>
Subject: Re: [RFC PATCH v2] x86/sev: enforce RIP-relative accesses in early SEV/SME code
Date: Wed, 17 Jan 2024 10:47:02 +0800	[thread overview]
Message-ID: <20240117024702.GA112065@k08j02272.eu95sqa> (raw)
In-Reply-To: <CAGdbjm+fMBnkKDVFpf-fAEbYxTUWR0od2JzTTbEMoPVS3J_Faw@mail.gmail.com>

On Wed, Jan 17, 2024 at 08:07:49AM +0800, Kevin Loughlin wrote:
> On Mon, Jan 15, 2024 at 12:47 PM Borislav Petkov <bp@alien8.de> wrote:
> >
> > On Thu, Jan 11, 2024 at 10:36:50PM +0000, Kevin Loughlin wrote:
> > > SEV/SME code can execute prior to page table fixups for kernel
> > > relocation. However, as with global variables accessed in
> > > __startup_64(), the compiler is not required to generate RIP-relative
> > > accesses for SEV/SME global variables, causing certain flavors of SEV
> > > hosts and guests built with clang to crash during boot.
> >
> > So, about that. If I understand my gcc toolchain folks correctly:
> >
> > mcmodel=kernel - everything fits into the high 31 bit of the address
> > space
> >
> > -fPIE/PIC - position independent
> >
> > And supplied both don't make a whole lotta of sense: if you're building
> > position-independent, then mcmodel=kernel would be overridden by the
> > first.
> >
> > I have no clue why clang enabled it...
> >
> > So, *actually* the proper fix here should be not to add this "fixed_up"
> > gunk everywhere but remove mcmodel=kernel from the build and simply do
> > -fPIE/PIC.
> 
> I believe that the key distinction is that using mcmodel=kernel (upper
> 2 GB of address space) or the similar mcmodel=small (lower 2 GB) means
> the compiler *can* use RIP-relative addressing for globals (because
> everything is within +/- 2GB of RIP) but is not *required* to do so.
> In contrast, fPIE/fPIC *requires* relative addressing but does not
> necessarily require a specific 2 GB placement range. Altogether, I do
> think there are use cases for both options individually. I can't think
> of a reason why gcc wouldn't be able to support mcmodel=kernel in
> conjunction with fPIE off the top of my head, but I admittedly haven't
> looked into it; I simply observed that the combination is not
> currently supported.
> 
> RE: compiling the whole x86-64 kernel with fPIE/fPIC, I believe the
> required changes would be very extensive (see "[PATCH RFC 00/43]
> x86/pie: Make kernel image's virtual address flexible" at
> https://lore.kernel.org/lkml/cover.1682673542.git.houwenlong.hwl@antgroup.com/).

FYI I have sent a small patchset[0] that attempts to build the head code
(head64.c and mem_encrypt_identity.c) as PIE. However, there are still a
few functions that are called by the head code but are not in these two
files.

[0] https://lore.kernel.org/lkml/cover.1689130310.git.houwenlong.hwl@antgroup.com