Re: [patch] x86/realmode: Make stack lock work in trampoline_compat()

From: Andrew Cooper <andrew.cooper3@citrix.com>
To: Yunhong Jiang <yunhong.jiang@linux.intel.com>,
	Thomas Gleixner <tglx@linutronix.de>
Cc: "Kirill A. Shutemov" <kirill@shutemov.name>,
	LKML <linux-kernel@vger.kernel.org>,
	x86@kernel.org, David Woodhouse <dwmw2@infradead.org>,
	Brian Gerst <brgerst@gmail.com>,
	Arjan van de Veen <arjan@linux.intel.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	Paul McKenney <paulmck@kernel.org>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	Sean Christopherson <seanjc@google.com>,
	Oleksandr Natalenko <oleksandr@natalenko.name>,
	Paul Menzel <pmenzel@molgen.mpg.de>,
	"Guilherme G. Piccoli" <gpiccoli@igalia.com>,
	Piotr Gorski <lucjan.lucjanov@gmail.com>,
	Usama Arif <usama.arif@bytedance.com>,
	Juergen Gross <jgross@suse.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	xen-devel@lists.xenproject.org,
	Russell King <linux@armlinux.org.uk>,
	Arnd Bergmann <arnd@arndb.de>,
	linux-arm-kernel@lists.infradead.org,
	Catalin Marinas <catalin.marinas@arm.com>,
	Will Deacon <will@kernel.org>, Guo Ren <guoren@kernel.org>,
	linux-csky@vger.kernel.org,
	Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
	linux-mips@vger.kernel.org,
	"James E.J. Bottomley" <James.Bottomley@HansenPartnership.com>,
	Helge Deller <deller@gmx.de>,
	linux-parisc@vger.kernel.org,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	linux-riscv@lists.infradead.org,
	Mark Rutland <mark.rutland@arm.com>,
	Sabin Rapan <sabrapan@amazon.com>,
	"Michael Kelley (LINUX)" <mikelley@microsoft.com>,
	Dave Hansen <dave.hansen@linux.intel.com>
Subject: Re: [patch] x86/realmode: Make stack lock work in trampoline_compat()
Date: Fri, 9 Jun 2023 00:57:46 +0100	[thread overview]
Message-ID: <a56a06c2-73fd-5295-3f6c-922ccb078488@citrix.com> (raw)
In-Reply-To: <20230608233402.GA3430@yjiang5-mobl.amr.corp.intel.com>

On 09/06/2023 12:34 am, Yunhong Jiang wrote:
> On Tue, May 30, 2023 at 12:46:22PM +0200, Thomas Gleixner wrote:
>> The stack locking and stack assignment macro LOAD_REALMODE_ESP fails to
>> work when invoked from the 64bit trampoline entry point:
>>
>> trampoline_start64
>>   trampoline_compat
>>     LOAD_REALMODE_ESP <- lock
> One possibly dumb question and hope get some hints.

There's a phrase.  "The only dumb question is the one not asked".

If you have this question, there's an excellent chance that someone else
reading this thread has the same question.

>  The LOAD_REALMODE_ESP is
> defined under .code16 directive and will be used by 32-bit mode caller also. Is
> it ok because the instructions there will be same for both 16-bit and 32-bit? I
> checked
> https://ftp.gnu.org/old-gnu/Manuals/gas-2.9.1/html_chapter/as_16.html#SEC205 and
> don't find much information there.

The position of the LOAD_REALMODE_ESP .macro itself doesn't matter. 
It's just some text which gets pasted elsewhere.  Imagine it just the
same as running the C preprocessor on a file before compiling it.

As you note, some expansions of the macro are in .code16, and some are
not.  This does result in different bytes being emitted.  The default
operands size flips between .code16 and .code32, so there will be some
0x66 prefixes in one mode, and not in others.

The important point is the l suffix on btsl, which forces it to be long
(32bit) irrespective of the default operand size.

So yes, it will work, but that's because gas is handling the differing
encodings automatically based on the default operand size where the
LOAD_REALMODE_ESP gets expanded.

Hope this helps,

~Andrew