Re: [PATCH v9 05/43] x86/compressed/64: Detect/setup SEV/SME features earlier in boot

From: Michael Roth <michael.roth@amd.com>
To: Borislav Petkov <bp@alien8.de>
Cc: Brijesh Singh <brijesh.singh@amd.com>, <x86@kernel.org>,
	<linux-kernel@vger.kernel.org>, <kvm@vger.kernel.org>,
	<linux-efi@vger.kernel.org>,
	<platform-driver-x86@vger.kernel.org>,
	<linux-coco@lists.linux.dev>, <linux-mm@kvack.org>,
	Thomas Gleixner <tglx@linutronix.de>,
	Ingo Molnar <mingo@redhat.com>, Joerg Roedel <jroedel@suse.de>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	"H. Peter Anvin" <hpa@zytor.com>,
	Ard Biesheuvel <ardb@kernel.org>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Sean Christopherson <seanjc@google.com>,
	"Vitaly Kuznetsov" <vkuznets@redhat.com>,
	Jim Mattson <jmattson@google.com>,
	"Andy Lutomirski" <luto@kernel.org>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Sergio Lopez <slp@redhat.com>, Peter Gonda <pgonda@google.com>,
	"Peter Zijlstra" <peterz@infradead.org>,
	Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com>,
	David Rientjes <rientjes@google.com>,
	Dov Murik <dovmurik@linux.ibm.com>,
	Tobin Feldman-Fitzthum <tobin@ibm.com>,
	Vlastimil Babka <vbabka@suse.cz>,
	"Kirill A . Shutemov" <kirill@shutemov.name>,
	Andi Kleen <ak@linux.intel.com>,
	"Dr . David Alan Gilbert" <dgilbert@redhat.com>,
	<brijesh.ksingh@gmail.com>, <tony.luck@intel.com>,
	<marcorr@google.com>,
	<sathyanarayanan.kuppuswamy@linux.intel.com>
Subject: Re: [PATCH v9 05/43] x86/compressed/64: Detect/setup SEV/SME features earlier in boot
Date: Tue, 1 Feb 2022 18:52:12 -0600	[thread overview]
Message-ID: <20220202005212.a3fnn6i76ko6u6t5@amd.com> (raw)
In-Reply-To: <YfmmBykN2s0HsiAJ@zn.tnic>

On Tue, Feb 01, 2022 at 10:28:39PM +0100, Borislav Petkov wrote:
> On Tue, Feb 01, 2022 at 02:35:07PM -0600, Michael Roth wrote:
> > Unfortunately rdmsr()/wrmsr()/__rdmsr()/__wrmsr() etc. definitions are all
> > already getting pulled in via:
> > 
> >   misc.h:
> >     #include linux/elf.h
> >       #include linux/thread_info.h
> >         #include linux/cpufeature.h
> >           #include linux/processor.h
> >             #include linux/msr.h
> > 
> > Those definitions aren't usable in boot/compressed because of __ex_table
> > and possibly some other dependency hellishness.
> 
> And they should not be. Mixing kernel proper and decompressor code needs
> to stop and untangling that is a multi-year effort, unfortunately. ;-\
> 
> > Would read_msr()/write_msr() be reasonable alternative names for these new
> > helpers, or something else that better distinguishes them from the
> > kernel proper definitions?
> 
> Nah, just call them rdmsr/wrmsr(). There is already {read,write}_msr()
> tracepoint symbols in kernel proper and there's no point in keeping them
> apart using different names - that ship has long sailed.

Since the kernel proper rdmsr()/wrmsr() definitions are getting pulled in via
misc.h, I have to use a different name to avoid compiler errors. For now I've
gone with rd_msr()/wr_msr(), but no problem changing those if needed.

> > Should we introduce something like this as well for cpucheck.c? Or
> > re-write cpucheck.c to make use of the u64 versions? Or just set the
> > cpucheck.c rework aside for now? (but still introduce the above helpers
> > as boot/msr.h in preparation)?
> 
> How about you model it after
> 
> static int msr_read(u32 msr, struct msr *m)
> 
> from arch/x86/lib/msr.c which takes struct msr from which you can return
> either u32s or a u64?
> 
> The stuff you share between the decompressor and kernel proper you put
> in a arch/x86/include/asm/shared/ folder, for an example, see what we do
> there in the TDX patchset:
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fr%2F20220124150215.36893-11-kirill.shutemov%40linux.intel.com&amp;data=04%7C01%7Cmichael.roth%40amd.com%7C1662b963f1c54f3663df08d9e5c9d6bd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637793477399827883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=rtdo9ci3XjOHn2HphvNE7ciFR6tKh1pVclFuNhXkNGs%3D&amp;reserved=0
> 
> I.e., you move struct msr in such a shared header and then you include
> it everywhere needed.
> 
> The arch/x86/boot/ msr helpers are then plain and simple, without
> tracepoints and exception fixups and you define them in ...boot/msr.c or
> so.
> 
> If the patch gets too big, make sure to split it in a couple so that it
> is clear what happens at each step.
> 
> How does that sound?

Thanks for the suggestions, this works out nicely, but as far as defining
them in boot/msr.c, it looks like the Makefile in boot/compressed doesn't
currently have any instances of linking to objects in boot/, and the status
quo seems to be #include'ing the whole C file in cases where boot/ code is
needed in boot/compressed.

Since the rd_msr/wr_msr are oneliners, it seemed like it might be a
little cleaner to just define them in boot/msr.h as static inline and
include them directly as part of the header.

Here's what it looks like on top of this tree, and roughly how I plan to
split the patches for v10:

- define the rd_msr/wr_msr helpers
  https://github.com/mdroth/linux/commit/982c6c5741478c8f634db8ac0ba36575b5eff946

- use the helpers in boot/compressed/sev.c and boot/cpucheck.c
  https://github.com/mdroth/linux/commit/a16e11f727c01fc478d3b741e1bdd2fd44975d7c

For v10 though I'll likely just drop rd_sev_status_msr() completely and use
rd_msr() directly.

Let me know if I should make any changes and I'll make sure to get those in for
the next spin.

> 
> Thx.
> 
> -- 
> Regards/Gruss,
>     Boris.
> 
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpeople.kernel.org%2Ftglx%2Fnotes-about-netiquette&amp;data=04%7C01%7Cmichael.roth%40amd.com%7C1662b963f1c54f3663df08d9e5c9d6bd%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637793477399827883%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=njDai06MS2mcyMLZ5YvLMcgXSXwfoO01U2c0D%2BE3HG4%3D&amp;reserved=0