Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Thomas Gleixner]

Lars,

On Wed, 3 Jan 2018, Lars Wendler wrote:

Thanks for the report.

> I already had a conversation with Thomas Gleixner about this issue and
> he asked me to post it here.
> 
> After upgrading my system to vanilla 4.14.11 kernel with
> CONFIG_PAGE_TABLE_ISOLATION=y I could no longer build any 32bit
> binaries because gcc (7.2.0) segfaults.

That's a 64bit binary, right?

> This only happens with CONFIG_PAGE_TABLE_ISOLATION=y
> I can get rid of this issue by setting CONFIG_PAGE_TABLE_ISOLATION=n
> _or_ adding pti=off to the kernel command line (the latter was
> suggested by Thomas).
> 
> Please find attached the kernel's config together with the dmesg output
> of the kernel that produces segfaults and the one with pti=off. I also
> attached a config.log file from a build where gcc segfaulted.
> 
> If you need more information, please don't hesitate to ask.

will do.

Thanks,

	tglx

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Thomas Gleixner]

On Wed, 3 Jan 2018, Thomas Gleixner wrote:
> On Wed, 3 Jan 2018, Lars Wendler wrote:
> Thanks for the report.
> 
> > I already had a conversation with Thomas Gleixner about this issue and
> > he asked me to post it here.
> > 
> > After upgrading my system to vanilla 4.14.11 kernel with
> > CONFIG_PAGE_TABLE_ISOLATION=y I could no longer build any 32bit
> > binaries because gcc (7.2.0) segfaults.
> 
> That's a 64bit binary, right?
> 
> > This only happens with CONFIG_PAGE_TABLE_ISOLATION=y
> > I can get rid of this issue by setting CONFIG_PAGE_TABLE_ISOLATION=n
> > _or_ adding pti=off to the kernel command line (the latter was
> > suggested by Thomas).
> > 
> > Please find attached the kernel's config together with the dmesg output
> > of the kernel that produces segfaults and the one with pti=off. I also
> > attached a config.log file from a build where gcc segfaulted.
> > 
> > If you need more information, please don't hesitate to ask.
> 
> will do.

Here you go.

Is this anything special you are trying to build or is a simple

   gcc -m32 -o hello hello.c

Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
whether that's a backport issue or a general one?

Thanks,

	tglx

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 2055 bytes --]

Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
schrieb Thomas Gleixner <tglx@linutronix.de>:

> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
> > On Wed, 3 Jan 2018, Lars Wendler wrote:
> > Thanks for the report.
> >   
> > > I already had a conversation with Thomas Gleixner about this
> > > issue and he asked me to post it here.
> > > 
> > > After upgrading my system to vanilla 4.14.11 kernel with
> > > CONFIG_PAGE_TABLE_ISOLATION=y I could no longer build any 32bit
> > > binaries because gcc (7.2.0) segfaults.  
> > 
> > That's a 64bit binary, right?

Yes it is.

> > > This only happens with CONFIG_PAGE_TABLE_ISOLATION=y
> > > I can get rid of this issue by setting
> > > CONFIG_PAGE_TABLE_ISOLATION=n _or_ adding pti=off to the kernel
> > > command line (the latter was suggested by Thomas).
> > > 
> > > Please find attached the kernel's config together with the dmesg
> > > output of the kernel that produces segfaults and the one with
> > > pti=off. I also attached a config.log file from a build where gcc
> > > segfaulted.
> > > 
> > > If you need more information, please don't hesitate to ask.  
> > 
> > will do.  
> 
> Here you go.
> 
> Is this anything special you are trying to build or is a simple
> 
>    gcc -m32 -o hello hello.c

A simple hello world compiles as 32bit binary but running it causes a
segfault. Compiling the same hello world as 64bit does not cause any
segfaults. Switching off pti resolves the issue and I can run the 32bit
hello world.

I first found this issue while trying to emerge gdbm-1.14 as 32bit.
Running configure failed and the config.log file said that gcc produced
a segfault while compiling a test binary called "conftest" (see the
gdbm-1.14-abi_x86_32.x86_config.log attachment in my initial report).

> Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
> whether that's a backport issue or a general one?

Same problem with 4.15-rc6. So I suppose that means it's a general
issue.


> Thanks,
> 
> 	tglx

Kind regards

Lars

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Thomas Gleixner]

On Wed, 3 Jan 2018, Lars Wendler wrote:
> Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
> schrieb Thomas Gleixner <tglx@linutronix.de>:
> > Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
> > whether that's a backport issue or a general one?
> 
> Same problem with 4.15-rc6. So I suppose that means it's a general
> issue.

Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
you please try the patch below?

Thanks,

	tglx

8<---------------

commit 52994c256df36fda9a715697431cba9daecb6b11
Author: Thomas Gleixner <tglx@linutronix.de>
Date:   Wed Jan 3 15:57:59 2018 +0100

    x86/pti: Make sure the user/kernel PTEs match
    
    Meelis reported that his K8 Athlon64 emits MCE warnings when PTI is
    enabled:
    
    [Hardware Error]: Error Addr: 0x0000ffff81e000e0
    [Hardware Error]: MC1 Error: L1 TLB multimatch.
    [Hardware Error]: cache level: L1, tx: INSN
    
    The address is in the entry area, which is mapped into kernel _AND_ user
    space. That's special because we switch CR3 while we are executing
    there.
    
    User mapping:
    0xffffffff81e00000-0xffffffff82000000           2M     ro         PSE     GLB x  pmd
    
    Kernel mapping:
    0xffffffff81000000-0xffffffff82000000          16M     ro         PSE         x  pmd
    
    So the K8 is complaining that the TLB entries differ. They differ in the
    GLB bit.
    
    Drop the GLB bit when installing the user shared mapping.
    
    Fixes: 6dc72c3cbca0 ("x86/mm/pti: Share entry text PMD")
    Reported-by: Meelis Roos <mroos@linux.ee>
    Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
    Tested-by: Meelis Roos <mroos@linux.ee>
    Cc: Borislav Petkov <bp@alien8.de>
    Cc: Tom Lendacky <thomas.lendacky@amd.com>
    Cc: stable@vger.kernel.org
    Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1801031407180.1957@nanos

diff --git a/arch/x86/mm/pti.c b/arch/x86/mm/pti.c
index bce8aea65606..2da28ba97508 100644
--- a/arch/x86/mm/pti.c
+++ b/arch/x86/mm/pti.c
@@ -367,7 +367,8 @@ static void __init pti_setup_espfix64(void)
 static void __init pti_clone_entry_text(void)
 {
 	pti_clone_pmds((unsigned long) __entry_text_start,
-			(unsigned long) __irqentry_text_end, _PAGE_RW);
+			(unsigned long) __irqentry_text_end,
+		       _PAGE_RW | _PAGE_GLOBAL);
 }
 
 /*

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Borislav Petkov]

On Wed, Jan 03, 2018 at 02:30:36PM +0100, Lars Wendler wrote:
> Same problem with 4.15-rc6. So I suppose that means it's a general
> issue.

So gentoo recently broke the kernel, see

3ce120b16cc5 ("kbuild: add '-fno-stack-check' to kernel build options")

and you say you hit the same problem with -rc6 and that commit is in
there so your kernel is probably built with -fno-stack-check.

Just a stab in the dark though: if you build your hello world program
with -fno-stack-check, does it segfault still?

Thx.

-- 
Regards/Gruss,
    Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Dave Hansen]

On 01/03/2018 07:21 AM, Thomas Gleixner wrote:
> On Wed, 3 Jan 2018, Lars Wendler wrote:
>> Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
>> schrieb Thomas Gleixner <tglx@linutronix.de>:
>>> Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
>>> whether that's a backport issue or a general one?
>> Same problem with 4.15-rc6. So I suppose that means it's a general
>> issue.
> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
> you please try the patch below?

Hi Lars,

I see the double-faults in your dmesg.  Those are usually indicative of
a failed user->kernel CR3 switch on entering the kernel.

Could you run a copy of the x86 selftests?  From the kernel tree I
usually do:

	cd tools/testing/selftests/x86/
	make
	for i in *_64 *_32; do ./$i || break; done

See if any of those fail.

I wonder if we screwed up something in the SYSCALL code in
entry_64_compat.S.  It's probably been much more lightly tested than all
the other entry code.

I'll see if I can reproduce this.

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Thomas Gleixner]

On Wed, 3 Jan 2018, Thomas Gleixner wrote:

> On Wed, 3 Jan 2018, Lars Wendler wrote:
> > Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
> > schrieb Thomas Gleixner <tglx@linutronix.de>:
> > > Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
> > > whether that's a backport issue or a general one?
> > 
> > Same problem with 4.15-rc6. So I suppose that means it's a general
> > issue.
> 
> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
> you please try the patch below?

Ok. Found the real issue. This is a problem on AMD boxen.

Fix below.

Can Xen folks please have a look at that as well?

Thanks,

	tglx

8<-------------------

arch/x86/entry/entry_64_compat.S |   13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

--- a/arch/x86/entry/entry_64_compat.S
+++ b/arch/x86/entry/entry_64_compat.S
@@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
 	/* Interrupts are off on entry. */
 	swapgs
 
-	/* Stash user ESP and switch to the kernel stack. */
+	/* Stash user ESP */
 	movl	%esp, %r8d
+
+	/* Use %rsp as scratch reg. User ESP is stashed in r8 */
+	SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
+	
+	/* Switch to the kernel stack */
 	movq	PER_CPU_VAR(cpu_current_top_of_stack), %rsp
 
 	/* Construct struct pt_regs on stack */
@@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
 	pushq   $0			/* pt_regs->r15 = 0 */
 
 	/*
-	 * We just saved %rdi so it is safe to clobber.  It is not
-	 * preserved during the C calls inside TRACE_IRQS_OFF anyway.
-	 */
-	SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
-
-	/*
 	 * User mode is traced as though IRQs are on, and SYSENTER
 	 * turned them off.
 	 */

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Laura Abbott]

On 01/03/2018 10:52 AM, Thomas Gleixner wrote:
> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
> 
>> On Wed, 3 Jan 2018, Lars Wendler wrote:
>>> Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
>>> schrieb Thomas Gleixner <tglx@linutronix.de>:
>>>> Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
>>>> whether that's a backport issue or a general one?
>>>
>>> Same problem with 4.15-rc6. So I suppose that means it's a general
>>> issue.
>>
>> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
>> you please try the patch below?
> 
> Ok. Found the real issue. This is a problem on AMD boxen.
> 

Fedora reporter says it fixes it.

> Fix below.
> 
> Can Xen folks please have a look at that as well?
> 
> Thanks,
> 
> 	tglx
> 
> 8<-------------------
> 
> arch/x86/entry/entry_64_compat.S |   13 ++++++-------
>   1 file changed, 6 insertions(+), 7 deletions(-)
> 
> --- a/arch/x86/entry/entry_64_compat.S
> +++ b/arch/x86/entry/entry_64_compat.S
> @@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
>   	/* Interrupts are off on entry. */
>   	swapgs
>   
> -	/* Stash user ESP and switch to the kernel stack. */
> +	/* Stash user ESP */
>   	movl	%esp, %r8d
> +
> +	/* Use %rsp as scratch reg. User ESP is stashed in r8 */
> +	SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
> +	
> +	/* Switch to the kernel stack */
>   	movq	PER_CPU_VAR(cpu_current_top_of_stack), %rsp
>   
>   	/* Construct struct pt_regs on stack */
> @@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
>   	pushq   $0			/* pt_regs->r15 = 0 */
>   
>   	/*
> -	 * We just saved %rdi so it is safe to clobber.  It is not
> -	 * preserved during the C calls inside TRACE_IRQS_OFF anyway.
> -	 */
> -	SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
> -
> -	/*
>   	 * User mode is traced as though IRQs are on, and SYSENTER
>   	 * turned them off.
>   	 */
> 

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Andy Lutomirski]

On Wed, Jan 3, 2018 at 10:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
>
>> On Wed, 3 Jan 2018, Lars Wendler wrote:
>> > Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
>> > schrieb Thomas Gleixner <tglx@linutronix.de>:
>> > > Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
>> > > whether that's a backport issue or a general one?
>> >
>> > Same problem with 4.15-rc6. So I suppose that means it's a general
>> > issue.
>>
>> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
>> you please try the patch below?
>
> Ok. Found the real issue. This is a problem on AMD boxen.
>
> Fix below.
>
> Can Xen folks please have a look at that as well?
>
> Thanks,
>
>         tglx
>
> 8<-------------------
>
> arch/x86/entry/entry_64_compat.S |   13 ++++++-------
>  1 file changed, 6 insertions(+), 7 deletions(-)
>
> --- a/arch/x86/entry/entry_64_compat.S
> +++ b/arch/x86/entry/entry_64_compat.S
> @@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
>         /* Interrupts are off on entry. */
>         swapgs
>
> -       /* Stash user ESP and switch to the kernel stack. */
> +       /* Stash user ESP */
>         movl    %esp, %r8d
> +
> +       /* Use %rsp as scratch reg. User ESP is stashed in r8 */
> +       SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
> +
> +       /* Switch to the kernel stack */
>         movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp
>
>         /* Construct struct pt_regs on stack */
> @@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
>         pushq   $0                      /* pt_regs->r15 = 0 */
>
>         /*
> -        * We just saved %rdi so it is safe to clobber.  It is not
> -        * preserved during the C calls inside TRACE_IRQS_OFF anyway.
> -        */
> -       SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
> -
> -       /*
>          * User mode is traced as though IRQs are on, and SYSENTER
>          * turned them off.
>          */

What's the issue that this is fixing?

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Thomas Gleixner]

On Wed, 3 Jan 2018, Andy Lutomirski wrote:

> On Wed, Jan 3, 2018 at 10:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > On Wed, 3 Jan 2018, Thomas Gleixner wrote:
> >
> >> On Wed, 3 Jan 2018, Lars Wendler wrote:
> >> > Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
> >> > schrieb Thomas Gleixner <tglx@linutronix.de>:
> >> > > Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
> >> > > whether that's a backport issue or a general one?
> >> >
> >> > Same problem with 4.15-rc6. So I suppose that means it's a general
> >> > issue.
> >>
> >> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
> >> you please try the patch below?
> >
> > Ok. Found the real issue. This is a problem on AMD boxen.
> >
> > Fix below.
> >
> > Can Xen folks please have a look at that as well?
> >
> > Thanks,
> >
> >         tglx
> >
> > 8<-------------------
> >
> > arch/x86/entry/entry_64_compat.S |   13 ++++++-------
> >  1 file changed, 6 insertions(+), 7 deletions(-)
> >
> > --- a/arch/x86/entry/entry_64_compat.S
> > +++ b/arch/x86/entry/entry_64_compat.S
> > @@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
> >         /* Interrupts are off on entry. */
> >         swapgs
> >
> > -       /* Stash user ESP and switch to the kernel stack. */
> > +       /* Stash user ESP */
> >         movl    %esp, %r8d
> > +
> > +       /* Use %rsp as scratch reg. User ESP is stashed in r8 */
> > +       SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
> > +
> > +       /* Switch to the kernel stack */
> >         movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp
> >
> >         /* Construct struct pt_regs on stack */
> > @@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
> >         pushq   $0                      /* pt_regs->r15 = 0 */
> >
> >         /*
> > -        * We just saved %rdi so it is safe to clobber.  It is not
> > -        * preserved during the C calls inside TRACE_IRQS_OFF anyway.
> > -        */
> > -       SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
> > -
> > -       /*
> >          * User mode is traced as though IRQs are on, and SYSENTER
> >          * turned them off.
> >          */
> 
> What's the issue that this is fixing?

> >         movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp

before switching CR3 is obviously broken ...

[tip:x86/pti] x86/pti: Switch to kernel CR3 at early in entry_SYSCALL_compat()

[tip-bot for Thomas Gleixner]

Commit-ID:  d7732ba55c4b6a2da339bb12589c515830cfac2c
Gitweb:     https://git.kernel.org/tip/d7732ba55c4b6a2da339bb12589c515830cfac2c
Author:     Thomas Gleixner <tglx@linutronix.de>
AuthorDate: Wed, 3 Jan 2018 19:52:04 +0100
Committer:  Thomas Gleixner <tglx@linutronix.de>
CommitDate: Wed, 3 Jan 2018 23:19:32 +0100

x86/pti: Switch to kernel CR3 at early in entry_SYSCALL_compat()

The preparation for PTI which added CR3 switching to the entry code
misplaced the CR3 switch in entry_SYSCALL_compat().

With PTI enabled the entry code tries to access a per cpu variable after
switching to kernel GS. This fails because that variable is not mapped to
user space. This results in a double fault and in the worst case a kernel
crash.

Move the switch ahead of the access and clobber RSP which has been saved
already.

Fixes: 8a09317b895f ("x86/mm/pti: Prepare the x86/entry assembly code for entry/exit CR3 switching")
Reported-by: Lars Wendler <wendler.lars@web.de>
Reported-by: Laura Abbott <labbott@redhat.com>
Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Borislav Betkov <bp@alien8.de>
Cc: Andy Lutomirski <luto@kernel.org>, 
Cc: Dave Hansen <dave.hansen@linux.intel.com>, 
Cc: Peter Zijlstra <peterz@infradead.org>, 
Cc: Greg KH <gregkh@linuxfoundation.org>, , 
Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com>, 
Cc: Juergen Gross <jgross@suse.com>
Cc: stable@vger.kernel.org
Link: https://lkml.kernel.org/r/alpine.DEB.2.20.1801031949200.1957@nanos

---
 arch/x86/entry/entry_64_compat.S | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/arch/x86/entry/entry_64_compat.S b/arch/x86/entry/entry_64_compat.S
index 40f1700..98d5358 100644
--- a/arch/x86/entry/entry_64_compat.S
+++ b/arch/x86/entry/entry_64_compat.S
@@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
 	/* Interrupts are off on entry. */
 	swapgs
 
-	/* Stash user ESP and switch to the kernel stack. */
+	/* Stash user ESP */
 	movl	%esp, %r8d
+
+	/* Use %rsp as scratch reg. User ESP is stashed in r8 */
+	SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
+
+	/* Switch to the kernel stack */
 	movq	PER_CPU_VAR(cpu_current_top_of_stack), %rsp
 
 	/* Construct struct pt_regs on stack */
@@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwframe)
 	pushq   $0			/* pt_regs->r15 = 0 */
 
 	/*
-	 * We just saved %rdi so it is safe to clobber.  It is not
-	 * preserved during the C calls inside TRACE_IRQS_OFF anyway.
-	 */
-	SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
-
-	/*
 	 * User mode is traced as though IRQs are on, and SYSENTER
 	 * turned them off.
 	 */

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Dave Hansen]

On 01/03/2018 02:14 PM, Andy Lutomirski wrote:
> +       /* Use %rsp as scratch reg. User ESP is stashed in r8 */
> +       SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
> +
> +       /* Switch to the kernel stack */
>         movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp

The stack is unreadable at this point without the CR3 switch.

> What's the issue that this is fixing?

Users doing 32-bit SYSCALLs on the CPUs that support them double fault
since they end up with an %rsp that they can't access.

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 759 bytes --]

Am Wed, 3 Jan 2018 16:40:08 +0100
schrieb Borislav Petkov <bp@alien8.de>:

> On Wed, Jan 03, 2018 at 02:30:36PM +0100, Lars Wendler wrote:
> > Same problem with 4.15-rc6. So I suppose that means it's a general
> > issue.  
> 
> So gentoo recently broke the kernel, see
> 
> 3ce120b16cc5 ("kbuild: add '-fno-stack-check' to kernel build
> options")
> 
> and you say you hit the same problem with -rc6 and that commit is in
> there so your kernel is probably built with -fno-stack-check.
> 
> Just a stab in the dark though: if you build your hello world program
> with -fno-stack-check, does it segfault still?
> 
> Thx.
> 

The issue you mention here only affects Gentoo hardened installations
which I never used.

Kind regards
Lars

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Andy Lutomirski]

> On Jan 3, 2018, at 2:22 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> 
>> On Wed, 3 Jan 2018, Andy Lutomirski wrote:
>> 
>>> On Wed, Jan 3, 2018 at 10:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>>>> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
>>>> 
>>>>> On Wed, 3 Jan 2018, Lars Wendler wrote:
>>>>> Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
>>>>> schrieb Thomas Gleixner <tglx@linutronix.de>:
>>>>>> Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
>>>>>> whether that's a backport issue or a general one?
>>>>> 
>>>>> Same problem with 4.15-rc6. So I suppose that means it's a general
>>>>> issue.
>>>> 
>>>> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
>>>> you please try the patch below?
>>> 
>>> Ok. Found the real issue. This is a problem on AMD boxen.
>>> 
>>> Fix below.
>>> 
>>> Can Xen folks please have a look at that as well?
>>> 
>>> Thanks,
>>> 
>>>        tglx
>>> 
>>> 8<-------------------
>>> 
>>> arch/x86/entry/entry_64_compat.S |   13 ++++++-------
>>> 1 file changed, 6 insertions(+), 7 deletions(-)
>>> 
>>> --- a/arch/x86/entry/entry_64_compat.S
>>> +++ b/arch/x86/entry/entry_64_compat.S
>>> @@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
>>>        /* Interrupts are off on entry. */
>>>        swapgs
>>> 
>>> -       /* Stash user ESP and switch to the kernel stack. */
>>> +       /* Stash user ESP */
>>>        movl    %esp, %r8d
>>> +
>>> +       /* Use %rsp as scratch reg. User ESP is stashed in r8 */
>>> +       SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
>>> +
>>> +       /* Switch to the kernel stack */
>>>        movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp
>>> 
>>>        /* Construct struct pt_regs on stack */
>>> @@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
>>>        pushq   $0                      /* pt_regs->r15 = 0 */
>>> 
>>>        /*
>>> -        * We just saved %rdi so it is safe to clobber.  It is not
>>> -        * preserved during the C calls inside TRACE_IRQS_OFF anyway.
>>> -        */
>>> -       SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
>>> -
>>> -       /*
>>>         * User mode is traced as though IRQs are on, and SYSENTER
>>>         * turned them off.
>>>         */
>> 
>> What's the issue that this is fixing?
> 
>>>        movq    PER_CPU_VAR(cpu_current_top_of_stack), %rsp
> 
> before switching CR3 is obviously broken ...
> 
> 

Duh.

This is what happens when we have five hundred versions of the patches and we change how it all works half way through.  And the 0day bot doesn't test the AMD path.

> 

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Lars Wendler]

[-- Attachment #1: Type: text/plain, Size: 885 bytes --]

Am Wed, 3 Jan 2018 19:52:04 +0100 (CET)
schrieb Thomas Gleixner <tglx@linutronix.de>:

> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
> 
> > On Wed, 3 Jan 2018, Lars Wendler wrote:  
> > > Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
> > > schrieb Thomas Gleixner <tglx@linutronix.de>:  
> > > > Also can you please try Linus v4.15-rc6 with PTI enabled so we
> > > > can see whether that's a backport issue or a general one?  
> > > 
> > > Same problem with 4.15-rc6. So I suppose that means it's a general
> > > issue.  
> > 
> > Just a shot in the dark as I just decoded another issue on a AMD
> > CPU. Can you please try the patch below?  
> 
> Ok. Found the real issue. This is a problem on AMD boxen.
> 
> Fix below.
> 
> Can Xen folks please have a look at that as well?
> 
> Thanks,
> 
> 	tglx

That indeed fixes the issue. Thank you!

Kind regards
Lars

[-- Attachment #2: Digitale Signatur von OpenPGP --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: CONFIG_PAGE_TABLE_ISOLATION=y on x86_64 causes gcc to segfault when building x86_32 binaries

[Boris Ostrovsky]

On 01/03/2018 01:52 PM, Thomas Gleixner wrote:
> On Wed, 3 Jan 2018, Thomas Gleixner wrote:
>
>> On Wed, 3 Jan 2018, Lars Wendler wrote:
>>> Am Wed, 3 Jan 2018 13:05:38 +0100 (CET)
>>> schrieb Thomas Gleixner <tglx@linutronix.de>:
>>>> Also can you please try Linus v4.15-rc6 with PTI enabled so we can see
>>>> whether that's a backport issue or a general one?
>>> Same problem with 4.15-rc6. So I suppose that means it's a general
>>> issue.
>> Just a shot in the dark as I just decoded another issue on a AMD CPU. Can
>> you please try the patch below?
> Ok. Found the real issue. This is a problem on AMD boxen.
>
> Fix below.
>
> Can Xen folks please have a look at that as well?

(Apologies for the delay)

This is not an issue for PV guests.


-boris

>
> Thanks,
>
> 	tglx
>
> 8<-------------------
>
> arch/x86/entry/entry_64_compat.S |   13 ++++++-------
>  1 file changed, 6 insertions(+), 7 deletions(-)
>
> --- a/arch/x86/entry/entry_64_compat.S
> +++ b/arch/x86/entry/entry_64_compat.S
> @@ -190,8 +190,13 @@ ENTRY(entry_SYSCALL_compat)
>  	/* Interrupts are off on entry. */
>  	swapgs
>  
> -	/* Stash user ESP and switch to the kernel stack. */
> +	/* Stash user ESP */
>  	movl	%esp, %r8d
> +
> +	/* Use %rsp as scratch reg. User ESP is stashed in r8 */
> +	SWITCH_TO_KERNEL_CR3 scratch_reg=%rsp
> +	
> +	/* Switch to the kernel stack */
>  	movq	PER_CPU_VAR(cpu_current_top_of_stack), %rsp
>  
>  	/* Construct struct pt_regs on stack */
> @@ -220,12 +225,6 @@ GLOBAL(entry_SYSCALL_compat_after_hwfram
>  	pushq   $0			/* pt_regs->r15 = 0 */
>  
>  	/*
> -	 * We just saved %rdi so it is safe to clobber.  It is not
> -	 * preserved during the C calls inside TRACE_IRQS_OFF anyway.
> -	 */
> -	SWITCH_TO_KERNEL_CR3 scratch_reg=%rdi
> -
> -	/*
>  	 * User mode is traced as though IRQs are on, and SYSENTER
>  	 * turned them off.
>  	 */