From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751634AbcFXUx3 (ORCPT ); Fri, 24 Jun 2016 16:53:29 -0400 Received: from mail-vk0-f42.google.com ([209.85.213.42]:36466 "EHLO mail-vk0-f42.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751286AbcFXUx2 (ORCPT ); Fri, 24 Jun 2016 16:53:28 -0400 MIME-Version: 1.0 In-Reply-To: <20160624205116.4hbnurrnan4afq2t@treble> References: <20160623185340.GO30154@twins.programming.kicks-ass.net> <20160624202530.unmidps4kpebo2na@treble> <20160624205116.4hbnurrnan4afq2t@treble> From: Andy Lutomirski Date: Fri, 24 Jun 2016 13:53:07 -0700 Message-ID: Subject: Re: [PATCH v3 00/13] Virtually mapped stacks with guard pages (x86, core) To: Josh Poimboeuf Cc: Linus Torvalds , Brian Gerst , Peter Zijlstra , Oleg Nesterov , Andy Lutomirski , "the arch/x86 maintainers" , Linux Kernel Mailing List , "linux-arch@vger.kernel.org" , Borislav Petkov , Nadav Amit , Kees Cook , "kernel-hardening@lists.openwall.com" , Jann Horn , Heiko Carstens Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Jun 24, 2016 at 1:51 PM, Josh Poimboeuf wrote: > On Fri, Jun 24, 2016 at 03:25:30PM -0500, Josh Poimboeuf wrote: >> On Fri, Jun 24, 2016 at 11:11:47AM -0700, Linus Torvalds wrote: >> > On Fri, Jun 24, 2016 at 10:51 AM, Linus Torvalds >> > wrote: >> > > >> > > And in particular, the init_task stack initialization initialized it >> > > to the init_thread pointer. Which was definitely deadly. >> > > >> > > Let's see if that was it.. >> > >> > No, it's still broken. But it's *less* broken, so here's a new version >> > of the patch that at least gets some of the stack setup right, in my >> > hope that somebody will bother to look at this, and being less broken >> > might mean that somebody sees what else I missed.. >> >> I found at least one bug. The changing of task->stack from a "void *" to an >> "unsigned long *": >> >> > - void *stack; >> > + unsigned long *stack; >> >> That subtly changes the pointer arithmetic in do_boot_cpu(): >> >> >> idle->thread.sp = (unsigned long) (((struct pt_regs *) >> (THREAD_SIZE + task_stack_page(idle))) - 1); >> >> >> That ends up adding 128k to the stack page bottom instead of 16k. >> >> But fixing that doesn't seem to fix this: >> >> [18446743832.576241] ------------[ cut here ]------------ >> [18446743832.576241] WARNING: CPU: 1 PID: 0 at /home/jpoimboe/git/linux/arch/x86/kernel/cpu/common.c:1434 cpu_init+0x34b/0x440 >> [18446743832.576241] Modules linked in: >> [18446743832.576241] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.7.0-rc4+ #47 >> [18446743832.576241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.1-20150318_183358- 04/01/2014 >> [18446743832.576241] 0000000000000086 574e5e6c6855ace9 ffff88007c553e88 ffffffff8143cb83 >> [18446743832.576241] 0000000000000000 0000000000000000 ffff88007c553ec8 ffffffff810b0e7b >> [18446743832.576241] 0000059a00000000 0000000000000000 0000000000000000 0000000000000000 >> [18446743832.576241] Call Trace: >> [18446743832.576241] [] dump_stack+0x85/0xc2 >> [18446743832.576241] [] __warn+0xcb/0xf0 >> [18446743832.576241] [] warn_slowpath_null+0x1d/0x20 >> [18446743832.576241] [] cpu_init+0x34b/0x440 >> [18446743832.576241] [] start_secondary+0x1c/0x1a0 >> [18446743832.576241] ---[ end trace 924d57afbaca0720 ]--- >> >> So there's at least another bug lurking.. > > Found another bug: > > #define stack_smp_processor_id() \ > ({ \ > struct thread_info *ti; \ > __asm__("andq %%rsp,%0; ":"=r" (ti) : "0" (CURRENT_MASK)); \ > ti->cpu; \ > }) > > That macro is obviously no longer valid. > > That seems to cause the above warning. When trying to boot CPU 1, > cpu_init() calls the above macro which incorrectly returns 0. Fixed in my queue by removing the function: https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/vmap_stack&id=01b1a4b6fd629820625b64ca6e17c987f2ee8c09 -- Andy Lutomirski AMA Capital Management, LLC From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andy Lutomirski Subject: Re: [PATCH v3 00/13] Virtually mapped stacks with guard pages (x86, core) Date: Fri, 24 Jun 2016 13:53:07 -0700 Message-ID: References: <20160623185340.GO30154@twins.programming.kicks-ass.net> <20160624202530.unmidps4kpebo2na@treble> <20160624205116.4hbnurrnan4afq2t@treble> Reply-To: kernel-hardening@lists.openwall.com Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Return-path: List-Post: List-Help: List-Unsubscribe: List-Subscribe: In-Reply-To: <20160624205116.4hbnurrnan4afq2t@treble> To: Josh Poimboeuf Cc: Linus Torvalds , Brian Gerst , Peter Zijlstra , Oleg Nesterov , Andy Lutomirski , the arch/x86 maintainers , Linux Kernel Mailing List , "linux-arch@vger.kernel.org" , Borislav Petkov , Nadav Amit , Kees Cook , "kernel-hardening@lists.openwall.com" , Jann Horn , Heiko Carstens List-Id: linux-arch.vger.kernel.org On Fri, Jun 24, 2016 at 1:51 PM, Josh Poimboeuf wrote: > On Fri, Jun 24, 2016 at 03:25:30PM -0500, Josh Poimboeuf wrote: >> On Fri, Jun 24, 2016 at 11:11:47AM -0700, Linus Torvalds wrote: >> > On Fri, Jun 24, 2016 at 10:51 AM, Linus Torvalds >> > wrote: >> > > >> > > And in particular, the init_task stack initialization initialized it >> > > to the init_thread pointer. Which was definitely deadly. >> > > >> > > Let's see if that was it.. >> > >> > No, it's still broken. But it's *less* broken, so here's a new version >> > of the patch that at least gets some of the stack setup right, in my >> > hope that somebody will bother to look at this, and being less broken >> > might mean that somebody sees what else I missed.. >> >> I found at least one bug. The changing of task->stack from a "void *" to an >> "unsigned long *": >> >> > - void *stack; >> > + unsigned long *stack; >> >> That subtly changes the pointer arithmetic in do_boot_cpu(): >> >> >> idle->thread.sp = (unsigned long) (((struct pt_regs *) >> (THREAD_SIZE + task_stack_page(idle))) - 1); >> >> >> That ends up adding 128k to the stack page bottom instead of 16k. >> >> But fixing that doesn't seem to fix this: >> >> [18446743832.576241] ------------[ cut here ]------------ >> [18446743832.576241] WARNING: CPU: 1 PID: 0 at /home/jpoimboe/git/linux/arch/x86/kernel/cpu/common.c:1434 cpu_init+0x34b/0x440 >> [18446743832.576241] Modules linked in: >> [18446743832.576241] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.7.0-rc4+ #47 >> [18446743832.576241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.1-20150318_183358- 04/01/2014 >> [18446743832.576241] 0000000000000086 574e5e6c6855ace9 ffff88007c553e88 ffffffff8143cb83 >> [18446743832.576241] 0000000000000000 0000000000000000 ffff88007c553ec8 ffffffff810b0e7b >> [18446743832.576241] 0000059a00000000 0000000000000000 0000000000000000 0000000000000000 >> [18446743832.576241] Call Trace: >> [18446743832.576241] [] dump_stack+0x85/0xc2 >> [18446743832.576241] [] __warn+0xcb/0xf0 >> [18446743832.576241] [] warn_slowpath_null+0x1d/0x20 >> [18446743832.576241] [] cpu_init+0x34b/0x440 >> [18446743832.576241] [] start_secondary+0x1c/0x1a0 >> [18446743832.576241] ---[ end trace 924d57afbaca0720 ]--- >> >> So there's at least another bug lurking.. > > Found another bug: > > #define stack_smp_processor_id() \ > ({ \ > struct thread_info *ti; \ > __asm__("andq %%rsp,%0; ":"=r" (ti) : "0" (CURRENT_MASK)); \ > ti->cpu; \ > }) > > That macro is obviously no longer valid. > > That seems to cause the above warning. When trying to boot CPU 1, > cpu_init() calls the above macro which incorrectly returns 0. Fixed in my queue by removing the function: https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/vmap_stack&id=01b1a4b6fd629820625b64ca6e17c987f2ee8c09 -- Andy Lutomirski AMA Capital Management, LLC From mboxrd@z Thu Jan 1 00:00:00 1970 Reply-To: kernel-hardening@lists.openwall.com MIME-Version: 1.0 In-Reply-To: <20160624205116.4hbnurrnan4afq2t@treble> References: <20160623185340.GO30154@twins.programming.kicks-ass.net> <20160624202530.unmidps4kpebo2na@treble> <20160624205116.4hbnurrnan4afq2t@treble> From: Andy Lutomirski Date: Fri, 24 Jun 2016 13:53:07 -0700 Message-ID: Content-Type: text/plain; charset=UTF-8 Subject: [kernel-hardening] Re: [PATCH v3 00/13] Virtually mapped stacks with guard pages (x86, core) To: Josh Poimboeuf Cc: Linus Torvalds , Brian Gerst , Peter Zijlstra , Oleg Nesterov , Andy Lutomirski , the arch/x86 maintainers , Linux Kernel Mailing List , "linux-arch@vger.kernel.org" , Borislav Petkov , Nadav Amit , Kees Cook , "kernel-hardening@lists.openwall.com" , Jann Horn , Heiko Carstens List-ID: On Fri, Jun 24, 2016 at 1:51 PM, Josh Poimboeuf wrote: > On Fri, Jun 24, 2016 at 03:25:30PM -0500, Josh Poimboeuf wrote: >> On Fri, Jun 24, 2016 at 11:11:47AM -0700, Linus Torvalds wrote: >> > On Fri, Jun 24, 2016 at 10:51 AM, Linus Torvalds >> > wrote: >> > > >> > > And in particular, the init_task stack initialization initialized it >> > > to the init_thread pointer. Which was definitely deadly. >> > > >> > > Let's see if that was it.. >> > >> > No, it's still broken. But it's *less* broken, so here's a new version >> > of the patch that at least gets some of the stack setup right, in my >> > hope that somebody will bother to look at this, and being less broken >> > might mean that somebody sees what else I missed.. >> >> I found at least one bug. The changing of task->stack from a "void *" to an >> "unsigned long *": >> >> > - void *stack; >> > + unsigned long *stack; >> >> That subtly changes the pointer arithmetic in do_boot_cpu(): >> >> >> idle->thread.sp = (unsigned long) (((struct pt_regs *) >> (THREAD_SIZE + task_stack_page(idle))) - 1); >> >> >> That ends up adding 128k to the stack page bottom instead of 16k. >> >> But fixing that doesn't seem to fix this: >> >> [18446743832.576241] ------------[ cut here ]------------ >> [18446743832.576241] WARNING: CPU: 1 PID: 0 at /home/jpoimboe/git/linux/arch/x86/kernel/cpu/common.c:1434 cpu_init+0x34b/0x440 >> [18446743832.576241] Modules linked in: >> [18446743832.576241] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.7.0-rc4+ #47 >> [18446743832.576241] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.8.1-20150318_183358- 04/01/2014 >> [18446743832.576241] 0000000000000086 574e5e6c6855ace9 ffff88007c553e88 ffffffff8143cb83 >> [18446743832.576241] 0000000000000000 0000000000000000 ffff88007c553ec8 ffffffff810b0e7b >> [18446743832.576241] 0000059a00000000 0000000000000000 0000000000000000 0000000000000000 >> [18446743832.576241] Call Trace: >> [18446743832.576241] [] dump_stack+0x85/0xc2 >> [18446743832.576241] [] __warn+0xcb/0xf0 >> [18446743832.576241] [] warn_slowpath_null+0x1d/0x20 >> [18446743832.576241] [] cpu_init+0x34b/0x440 >> [18446743832.576241] [] start_secondary+0x1c/0x1a0 >> [18446743832.576241] ---[ end trace 924d57afbaca0720 ]--- >> >> So there's at least another bug lurking.. > > Found another bug: > > #define stack_smp_processor_id() \ > ({ \ > struct thread_info *ti; \ > __asm__("andq %%rsp,%0; ":"=r" (ti) : "0" (CURRENT_MASK)); \ > ti->cpu; \ > }) > > That macro is obviously no longer valid. > > That seems to cause the above warning. When trying to boot CPU 1, > cpu_init() calls the above macro which incorrectly returns 0. Fixed in my queue by removing the function: https://git.kernel.org/cgit/linux/kernel/git/luto/linux.git/commit/?h=x86/vmap_stack&id=01b1a4b6fd629820625b64ca6e17c987f2ee8c09 -- Andy Lutomirski AMA Capital Management, LLC