Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

From: Andy Lutomirski <luto@amacapital.net>
To: Thomas Gleixner <tglx@linutronix.de>
Cc: Andy Lutomirski <luto@kernel.org>,
	Lars Wendler <wendler.lars@web.de>,
	LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
	Borislav Betkov <bp@alien8.de>,
	Dave Hansen <dave.hansen@linux.intel.com>,
	Peter Zijlstra <peterz@infradead.org>,
	Greg KH <gregkh@linuxfoundation.org>,
	Laura Abbott <labbott@redhat.com>,
	Boris Ostrovsky <boris.ostrovsky@oracle.com>,
	Juergen Gross <jgross@suse.com>
Subject: Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries
Date: Wed, 3 Jan 2018 15:43:08 -0800	[thread overview]
Message-ID: <3C2A7852-06E2-4C95-AD46-AE965B412818@amacapital.net> (raw)
In-Reply-To: <alpine.DEB.2.20.1801032322160.1957@nanos>

> On Jan 3, 2018, at 2:22 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> 
>> On Wed, 3 Jan 2018, Andy Lutomirski wrote:
>> 
>>> On Wed, Jan 3, 2018 at 10:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>>>> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
>>>> 
>>>>> On Wed, 3 Jan 2018, Lars Wendler wrote:
>>>>> Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
>>>>> schrieb Thomas Gleixner <tglx@linutronix.de>:
>>>>>> Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
>>>>>> whether that's a backport issue or a general one?
>>>>> 
>>>>> Same problem with 4.15-rc6. So I suppose that means it's a general
>>>>> issue.
>>>> 
>>>> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
>>>> you please try the patch below?
>>> 
>>> Ok. Found the real issue. This is a problem on AMD boxen.
>>> 
>>> Fix below.
>>> 
>>> Can Xen folks please have a look at that as well?
>>> 
>>> Thanks,
>>> 
>>>        tglx
>>> 
>>> 8<-------------------
>>> 
>>> arch/x86/entry/entry_64_compat.S |   13 ++++++-------
>>> 1 file changed, 6 insertions(+), 7 deletions(-)
>>> 
>>> --- a/arch/x86/entry/entry_64_compat.S
>>> +++ b/arch/x86/entry/entry_64_compat.S
>>> @@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
>>>        /* Interrupts are off on entry. */
>>>        swapgs
>>> 
>>> -       /* Stash user ESP and switch to the kernel stack. */
>>> +       /* Stash user ESP */
>>>        movl    %esp, %r8d
>>> +
>>> +       /* Use %rsp as scratch reg. User ESP is stashed in r8 */
>>> +       SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
>>> +
>>> +       /* Switch to the kernel stack */
>>>        movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp
>>> 
>>>        /* Construct struct pt_regs on stack */
>>> @@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
>>>        pushq   $0                      /* pt_regs->r15 = 0 */
>>> 
>>>        /*
>>> -        * We just saved %rdi so it is safe to clobber.  It is not
>>> -        * preserved during the C calls inside TRACE_IRQS_OFF anyway.
>>> -        */
>>> -       SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
>>> -
>>> -       /*
>>>         * User mode is traced as though IRQs are on, and SYSENTER
>>>         * turned them off.
>>>         */
>> 
>> What's the issue that this is fixing?
> 
>>>        movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp
> 
> before switching CR3 is obviously broken ...
> 
> 

Duh.

This is what happens when we have five hundred versions of the patches and we change how it all works half way through.  And the 0day bot doesn't test the AMD path.

> 