From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753146AbdGRVyZ (ORCPT ); Tue, 18 Jul 2017 17:54:25 -0400 Received: from mail-qt0-f177.google.com ([209.85.216.177]:34180 "EHLO mail-qt0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752870AbdGRVxo (ORCPT ); Tue, 18 Jul 2017 17:53:44 -0400 Subject: Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP To: Ard Biesheuvel , Mark Rutland Cc: Kernel Hardening , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , Takahiro Akashi , Catalin Marinas , Dave Martin , James Morse , Laura Abbott , Will Deacon , Kees Cook References: <20170713104950.GB26194@leverpostej> <20170713161050.GG26194@leverpostej> <20170713175543.GA32528@leverpostej> <20170714103258.GA16128@leverpostej> <20170714140605.GB16687@leverpostej> <20170714212717.GB1086@leverpostej> From: Laura Abbott Message-ID: <39a5ad84-4124-5b33-146a-cd4e48f3762f@redhat.com> Date: Tue, 18 Jul 2017 14:53:40 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 07/15/2017 05:03 PM, Ard Biesheuvel wrote: > On 14 July 2017 at 22:27, Mark Rutland wrote: >> On Fri, Jul 14, 2017 at 03:06:06PM +0100, Mark Rutland wrote: >>> On Fri, Jul 14, 2017 at 01:27:14PM +0100, Ard Biesheuvel wrote: >>>> On 14 July 2017 at 11:48, Ard Biesheuvel wrote: >>>>> On 14 July 2017 at 11:32, Mark Rutland wrote: >>>>>> On Thu, Jul 13, 2017 at 07:28:48PM +0100, Ard Biesheuvel wrote: >>> >>>>>>> OK, so here's a crazy idea: what if we >>>>>>> a) carve out a dedicated range in the VMALLOC area for stacks >>>>>>> b) for each stack, allocate a naturally aligned window of 2x the stack >>>>>>> size, and map the stack inside it, leaving the remaining space >>>>>>> unmapped >>> >>>>>> The logical ops (TST) and conditional branches (TB(N)Z, CB(N)Z) operate >>>>>> on XZR rather than SP, so to do this we need to get the SP value into a >>>>>> GPR. >>>>>> >>>>>> Previously, I assumed this meant we needed to corrupt a GPR (and hence >>>>>> stash that GPR in a sysreg), so I started writing code to free sysregs. >>>>>> >>>>>> However, I now realise I was being thick, since we can stash the GPR >>>>>> in the SP: >>>>>> >>>>>> sub sp, sp, x0 // sp = orig_sp - x0 >>>>>> add x0, sp, x0 // x0 = x0 - (orig_sp - x0) == orig_sp >>> >>> That comment is off, and should say x0 = x0 + (orig_sp - x0) == orig_sp >>> >>>>>> sub x0, x0, #S_FRAME_SIZE >>>>>> tb(nz) x0, #THREAD_SHIFT, overflow >>>>>> add x0, x0, #S_FRAME_SIZE >>>>>> sub x0, sp, x0 >>>> >>>> You need a neg x0, x0 here I think >>> >>> Oh, whoops. I'd mis-simplified things. >>> >>> We can avoid that by storing orig_sp + orig_x0 in sp: >>> >>> add sp, sp, x0 // sp = orig_sp + orig_x0 >>> sub x0, sp, x0 // x0 = orig_sp >>> < check > >>> sub x0, sp, x0 // x0 = orig_x0 >>> sub sp, sp, x0 // sp = orig_sp >>> >>> ... which works in a locally-built kernel where I've aligned all the >>> stacks. >> >> FWIW, I've pushed out a somewhat cleaned-up (and slightly broken!) >> version of said kernel source to my arm64/vmap-stack-align branch [1]. >> That's still missing the backtrace handling, IRQ stack alignment is >> broken at least on 64K pages, and there's still more cleanup and rework >> to do. >> > > I have spent some time addressing the issues mentioned in the commit > log. Please take a look. > > git://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git vmap-arm64-mark > I used vmap-arm64-mark to compile kernels for a few days. It seemed to work well enough. Thanks, Laura From mboxrd@z Thu Jan 1 00:00:00 1970 From: labbott@redhat.com (Laura Abbott) Date: Tue, 18 Jul 2017 14:53:40 -0700 Subject: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP In-Reply-To: References: <20170713104950.GB26194@leverpostej> <20170713161050.GG26194@leverpostej> <20170713175543.GA32528@leverpostej> <20170714103258.GA16128@leverpostej> <20170714140605.GB16687@leverpostej> <20170714212717.GB1086@leverpostej> Message-ID: <39a5ad84-4124-5b33-146a-cd4e48f3762f@redhat.com> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org On 07/15/2017 05:03 PM, Ard Biesheuvel wrote: > On 14 July 2017 at 22:27, Mark Rutland wrote: >> On Fri, Jul 14, 2017 at 03:06:06PM +0100, Mark Rutland wrote: >>> On Fri, Jul 14, 2017 at 01:27:14PM +0100, Ard Biesheuvel wrote: >>>> On 14 July 2017 at 11:48, Ard Biesheuvel wrote: >>>>> On 14 July 2017 at 11:32, Mark Rutland wrote: >>>>>> On Thu, Jul 13, 2017 at 07:28:48PM +0100, Ard Biesheuvel wrote: >>> >>>>>>> OK, so here's a crazy idea: what if we >>>>>>> a) carve out a dedicated range in the VMALLOC area for stacks >>>>>>> b) for each stack, allocate a naturally aligned window of 2x the stack >>>>>>> size, and map the stack inside it, leaving the remaining space >>>>>>> unmapped >>> >>>>>> The logical ops (TST) and conditional branches (TB(N)Z, CB(N)Z) operate >>>>>> on XZR rather than SP, so to do this we need to get the SP value into a >>>>>> GPR. >>>>>> >>>>>> Previously, I assumed this meant we needed to corrupt a GPR (and hence >>>>>> stash that GPR in a sysreg), so I started writing code to free sysregs. >>>>>> >>>>>> However, I now realise I was being thick, since we can stash the GPR >>>>>> in the SP: >>>>>> >>>>>> sub sp, sp, x0 // sp = orig_sp - x0 >>>>>> add x0, sp, x0 // x0 = x0 - (orig_sp - x0) == orig_sp >>> >>> That comment is off, and should say x0 = x0 + (orig_sp - x0) == orig_sp >>> >>>>>> sub x0, x0, #S_FRAME_SIZE >>>>>> tb(nz) x0, #THREAD_SHIFT, overflow >>>>>> add x0, x0, #S_FRAME_SIZE >>>>>> sub x0, sp, x0 >>>> >>>> You need a neg x0, x0 here I think >>> >>> Oh, whoops. I'd mis-simplified things. >>> >>> We can avoid that by storing orig_sp + orig_x0 in sp: >>> >>> add sp, sp, x0 // sp = orig_sp + orig_x0 >>> sub x0, sp, x0 // x0 = orig_sp >>> < check > >>> sub x0, sp, x0 // x0 = orig_x0 >>> sub sp, sp, x0 // sp = orig_sp >>> >>> ... which works in a locally-built kernel where I've aligned all the >>> stacks. >> >> FWIW, I've pushed out a somewhat cleaned-up (and slightly broken!) >> version of said kernel source to my arm64/vmap-stack-align branch [1]. >> That's still missing the backtrace handling, IRQ stack alignment is >> broken at least on 64K pages, and there's still more cleanup and rework >> to do. >> > > I have spent some time addressing the issues mentioned in the commit > log. Please take a look. > > git://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git vmap-arm64-mark > I used vmap-arm64-mark to compile kernels for a few days. It seemed to work well enough. Thanks, Laura From mboxrd@z Thu Jan 1 00:00:00 1970 References: <20170713104950.GB26194@leverpostej> <20170713161050.GG26194@leverpostej> <20170713175543.GA32528@leverpostej> <20170714103258.GA16128@leverpostej> <20170714140605.GB16687@leverpostej> <20170714212717.GB1086@leverpostej> From: Laura Abbott Message-ID: <39a5ad84-4124-5b33-146a-cd4e48f3762f@redhat.com> Date: Tue, 18 Jul 2017 14:53:40 -0700 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Subject: Re: [kernel-hardening] Re: [RFC PATCH 6/6] arm64: add VMAP_STACK and detect out-of-bounds SP To: Ard Biesheuvel , Mark Rutland Cc: Kernel Hardening , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , Takahiro Akashi , Catalin Marinas , Dave Martin , James Morse , Laura Abbott , Will Deacon , Kees Cook List-ID: On 07/15/2017 05:03 PM, Ard Biesheuvel wrote: > On 14 July 2017 at 22:27, Mark Rutland wrote: >> On Fri, Jul 14, 2017 at 03:06:06PM +0100, Mark Rutland wrote: >>> On Fri, Jul 14, 2017 at 01:27:14PM +0100, Ard Biesheuvel wrote: >>>> On 14 July 2017 at 11:48, Ard Biesheuvel wrote: >>>>> On 14 July 2017 at 11:32, Mark Rutland wrote: >>>>>> On Thu, Jul 13, 2017 at 07:28:48PM +0100, Ard Biesheuvel wrote: >>> >>>>>>> OK, so here's a crazy idea: what if we >>>>>>> a) carve out a dedicated range in the VMALLOC area for stacks >>>>>>> b) for each stack, allocate a naturally aligned window of 2x the stack >>>>>>> size, and map the stack inside it, leaving the remaining space >>>>>>> unmapped >>> >>>>>> The logical ops (TST) and conditional branches (TB(N)Z, CB(N)Z) operate >>>>>> on XZR rather than SP, so to do this we need to get the SP value into a >>>>>> GPR. >>>>>> >>>>>> Previously, I assumed this meant we needed to corrupt a GPR (and hence >>>>>> stash that GPR in a sysreg), so I started writing code to free sysregs. >>>>>> >>>>>> However, I now realise I was being thick, since we can stash the GPR >>>>>> in the SP: >>>>>> >>>>>> sub sp, sp, x0 // sp = orig_sp - x0 >>>>>> add x0, sp, x0 // x0 = x0 - (orig_sp - x0) == orig_sp >>> >>> That comment is off, and should say x0 = x0 + (orig_sp - x0) == orig_sp >>> >>>>>> sub x0, x0, #S_FRAME_SIZE >>>>>> tb(nz) x0, #THREAD_SHIFT, overflow >>>>>> add x0, x0, #S_FRAME_SIZE >>>>>> sub x0, sp, x0 >>>> >>>> You need a neg x0, x0 here I think >>> >>> Oh, whoops. I'd mis-simplified things. >>> >>> We can avoid that by storing orig_sp + orig_x0 in sp: >>> >>> add sp, sp, x0 // sp = orig_sp + orig_x0 >>> sub x0, sp, x0 // x0 = orig_sp >>> < check > >>> sub x0, sp, x0 // x0 = orig_x0 >>> sub sp, sp, x0 // sp = orig_sp >>> >>> ... which works in a locally-built kernel where I've aligned all the >>> stacks. >> >> FWIW, I've pushed out a somewhat cleaned-up (and slightly broken!) >> version of said kernel source to my arm64/vmap-stack-align branch [1]. >> That's still missing the backtrace handling, IRQ stack alignment is >> broken at least on 64K pages, and there's still more cleanup and rework >> to do. >> > > I have spent some time addressing the issues mentioned in the commit > log. Please take a look. > > git://git.kernel.org/pub/scm/linux/kernel/git/ardb/linux.git vmap-arm64-mark > I used vmap-arm64-mark to compile kernels for a few days. It seemed to work well enough. Thanks, Laura