[Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Morris]

I'd like to propose a security topic, "Kernel Hardening" (or "Kernel Self 
Protection"), to discuss how we can better mitigate vulnerabilities 
arising from kernel bugs.

We have some measures in place, although we are really not doing 
everything we can, as demonstrated from time to time when vulnerabilities 
arise which are mitigated by protections in grsecurity (for example), but 
not by mainline.  Much of the necessary work has already been done in that 
project, and as many will know, there have been significant challenges 
involved in past efforts to bring these techniques into mainline.  In some 
cases, the performance hit has been too high for maintainers to accept, 
and I wonder if we can re-visit some of these cases, with new approaches 
or perspectives on cost/benefit.

There are also potentially promising approaches to mitigation with other 
technologies such as KASan and gcc plugins, as well as evolving hardware 
features.

The aim of this session would be to bring relevant core kernel maintainers 
together with representatives of the research community and figure out a 
way to work together to improve hardening and mitigation in the Linux 
kernel.  We'd discuss what gaps we currently have, and what code or 
techniques already exist that can be incorporated into mainline to close 
them.  We'd identify issues that maintainers may have and try and find 
ways to address those issues.  From this, I'd hope that we'd develop an 
overall picture of what needs to be done and a practical idea of how to 
move forward.  We may not necessarily resolve all issues in this session, 
but we can at least characterize them and go away and think more about 
them.

We could also talk to the Core Infrastructure Initiative folk if we 
discover potentially useful tasks with no owners -- they may be able to 
fund developers for them.  It would likely be useful to provide CII with a 
status report after the session in any case.

I'd recommend Kees Cook be involved, due to his existing efforts in kernel 
hardening.  I think it would be good to invite one or two expert security 
researchers in this area -- Kees would know who.  In terms of core kernel 
folk, I'd suggest Ingo and akpm, as a starting point.

Comments?


-- 
James Morris
<jmorris@namei.org>

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Jiri Kosina]

On Mon, 24 Aug 2015, James Morris wrote:

> I'd recommend Kees Cook be involved, due to his existing efforts in 
> kernel hardening.  I think it would be good to invite one or two expert 
> security researchers in this area -- Kees would know who.  In terms of 
> core kernel folk, I'd suggest Ingo and akpm, as a starting point.

I am pretty sure spender will also have a lot to tell us :p

This is partially sarcasm (due to his often vocally expressed opinions 
about our processess (and even code) with respect to security) and 
partially meant seriously (there is always a possibility that at least 
some misunderstandings will be clarified and we'll get closer to 
convergency of the trees).

As a very long time grsecurity user, I'd definitely be interested.

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Jiri Kosina]

[ resending with proper list-member From: sorry for duplicates to those 
  directly CCed ]

On Mon, 24 Aug 2015, James Morris wrote:

> I'd recommend Kees Cook be involved, due to his existing efforts in 
> kernel hardening.  I think it would be good to invite one or two expert 
> security researchers in this area -- Kees would know who.  In terms of 
> core kernel folk, I'd suggest Ingo and akpm, as a starting point.

I am pretty sure spender will also have a lot to tell us :p

This is partially sarcasm (due to his often vocally expressed opinions 
about our processess (and even code) with respect to security) and 
partially meant seriously (there is always a possibility that at least 
some misunderstandings will be clarified and we'll get closer to 
convergency of the trees).

As a very long time grsecurity user, I'd definitely be interested.

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Morris]

On Mon, 24 Aug 2015, Jiri Kosina wrote:

> On Mon, 24 Aug 2015, James Morris wrote:
> 
> > I'd recommend Kees Cook be involved, due to his existing efforts in 
> > kernel hardening.  I think it would be good to invite one or two expert 
> > security researchers in this area -- Kees would know who.  In terms of 
> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.
> 
> I am pretty sure spender will also have a lot to tell us :p

He actually presented at the 2010 security summit:
https://grsecurity.net/spender_summit.pdf

I'm not sure he'd be interested in working with us on this, though.


- James
-- 
James Morris
<jmorris@namei.org>

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Linus Walleij]

On Mon, Aug 24, 2015 at 6:20 AM, James Morris <jmorris@namei.org> wrote:

> There are also potentially promising approaches to mitigation with other
> technologies such as KASan and gcc plugins, as well as evolving hardware
> features.

What I've discovered when running KASan the last few weeks is that
this points back to the question of tests ... I've been using Trinity
to find bugs, but it is more likely to kill itself or cause OOM than
trigger any boundary overrun bugs.

Kselftest may be helpful, but basically any loads that heavily
exercise the kernel internals are helpful to harden the kernel.
Some of these are custom test suites I suspect. Any good hints
for a simple embedded developer like me?

Linus Walleij

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Jason Cooper]

On Mon, Aug 24, 2015 at 02:29:41PM +0200, Linus Walleij wrote:
> Kselftest may be helpful, but basically any loads that heavily
> exercise the kernel internals are helpful to harden the kernel.
> Some of these are custom test suites I suspect. Any good hints
> for a simple embedded developer like me?

A great place to start is this paper:

  http://messenger.jhuapl.edu/the_mission/publications/Hill.2007.pdf

It's a walk-through of how NASA's JHU/APL facility stress-tests software
intended for deep space missions, including Messenger, New Horizons, and
STEREO.  Lot's of good concepts, no specific test harnesses, though.

thx,

Jason.

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Aneesh Kumar K.V]

James Morris <jmorris@namei.org> writes:

> I'd like to propose a security topic, "Kernel Hardening" (or "Kernel Self 
> Protection"), to discuss how we can better mitigate vulnerabilities 
> arising from kernel bugs.
>
> We have some measures in place, although we are really not doing 
> everything we can, as demonstrated from time to time when vulnerabilities 
> arise which are mitigated by protections in grsecurity (for example), but 
> not by mainline.  Much of the necessary work has already been done in that 
> project, and as many will know, there have been significant challenges 
> involved in past efforts to bring these techniques into mainline.  In some 
> cases, the performance hit has been too high for maintainers to accept, 
> and I wonder if we can re-visit some of these cases, with new approaches 
> or perspectives on cost/benefit.
>
> There are also potentially promising approaches to mitigation with other 
> technologies such as KASan and gcc plugins, as well as evolving hardware 
> features.
>

We also have to make sure that the compiler based approach work with
architectures other than x86. Archs like ppc64 have different memory
layout and features like KASan may not really map easily with the
layout. For example we may not be able to implement inline kasan
instrumentation on ppc64. Also we have issues with stack and
global out of bounds access check.

I would be interested in this discussion, if we are scheduling this for
ksummit. I work mostly on ppc64 memory-management subsystem and can
bring in details of challenges faced with KASan implementation on ppc64.

-aneesh

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 5:29 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
> On Mon, Aug 24, 2015 at 6:20 AM, James Morris <jmorris@namei.org> wrote:
>
>> There are also potentially promising approaches to mitigation with other
>> technologies such as KASan and gcc plugins, as well as evolving hardware
>> features.
>
> What I've discovered when running KASan the last few weeks is that
> this points back to the question of tests ... I've been using Trinity
> to find bugs, but it is more likely to kill itself or cause OOM than
> trigger any boundary overrun bugs.
>
> Kselftest may be helpful, but basically any loads that heavily
> exercise the kernel internals are helpful to harden the kernel.
> Some of these are custom test suites I suspect. Any good hints
> for a simple embedded developer like me?

I agree with the sentiment here, but not with the language. Finding
flaws (which is what selftests, KASan, Trinity, etc do) isn't
hardening. Hardening is stopping the exploitation of flaws. The
hardening the kernel needs is about taking away exploitation tools,
not killing bugs. (Though killing bugs is still great.)

As an example, making the kernel code memory read-only means an
attacker cannot just directly change the kernel's execution path when
they use an arbitrary memory-writing flaw. (This feature is mostly
enabled via CONFIG_DEBUG_RODATA, and was very recently added to ARM,
though isn't at 100% coverage for all the physical memory aliases.)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
> On Mon, 24 Aug 2015, Jiri Kosina wrote:
>
>> On Mon, 24 Aug 2015, James Morris wrote:
>>
>> > I'd recommend Kees Cook be involved, due to his existing efforts in
>> > kernel hardening.  I think it would be good to invite one or two expert
>> > security researchers in this area -- Kees would know who.  In terms of

Many of the folks that are good at kernel exploitation don't want to
help us fix the situation. :)

I'd recommend Lee Campbell, he's got a solid bit of experience from
the offense side. I think we should extend an invite to spender and
pageexec as well. They've been on the cutting edge of this for
decades, and it would be silly not to invite them.

>> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.

Perhaps also Linus and rmk? Some of the protections are very central
to the kernel (e.g. constification, "read-mostly", segmentation
through page table swaps or domains, etc). I'd also want Andy
Lutomirski around, as he's got a lot of deep chipset knowledge. :)

>>
>> I am pretty sure spender will also have a lot to tell us :p
>
> He actually presented at the 2010 security summit:
> https://grsecurity.net/spender_summit.pdf

This was his bullet list of things that grsecurity/PaX already does
and that should be in mainline (with my notes in parens). He suggested
it would take us 10 years to catch up. We're 5 years into that, with
only a few things partially off this list:

 Remove infoleaks
 o Symbol information (partial improvement via kptr_restrict)
 o Slabinfo (partial improvement with 0400 root perms)
 o PAX_USERCOPY (even if gcc fixed their sizeof bug, we'd be no where
near this plugin's level of protection)
 Remove RWX from kernel (in good shape on x86, started on arm)
 Protect sensitive data
 o Constify function pointers! (no where close to this plugin)
 o IDT/GDT/syscall table/etc (this is partially done, aliases are
writable still)
 o Vsyscall shadow table, see sgrakkyu's remote
SELinux-disabling exploit
(http://sgrakkyu.antifork.org/sctp_houdini.c, luto fixed this via
vsyscall emulation)
Protect against invalid userland memory
accesses in general (SMEP and SMAP on x86 get us this, but it can be
emulated or done differently e.g. ARM domains)
Make refcount overflows unexploitable (nothing done on this front)
 o Currently equivalent to use-after-free
kmalloc(sizeof(somestruct) * attacker_len) (some partial improvements
here to replace this anti-pattern manually)
 o See recent ethtool get_rxnfc() vulnerability
Basically, secure the kernel! Your super finegrained
security systems will thank you

This is far from a comprehensive list, though. The biggest value, I
think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
for constification and integer overflow.

> I'm not sure he'd be interested in working with us on this, though.

As I understand him, he's quite binary about his methods: either it's
done totally right, or it's not worth doing at all. With that in mind,
I think his position is clear, even if he's rather prickly. :)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 9:20 AM, Aneesh Kumar K.V
<aneesh.kumar@linux.vnet.ibm.com> wrote:
> James Morris <jmorris@namei.org> writes:
>
>> I'd like to propose a security topic, "Kernel Hardening" (or "Kernel Self
>> Protection"), to discuss how we can better mitigate vulnerabilities
>> arising from kernel bugs.
>>
>> We have some measures in place, although we are really not doing
>> everything we can, as demonstrated from time to time when vulnerabilities
>> arise which are mitigated by protections in grsecurity (for example), but
>> not by mainline.  Much of the necessary work has already been done in that
>> project, and as many will know, there have been significant challenges
>> involved in past efforts to bring these techniques into mainline.  In some
>> cases, the performance hit has been too high for maintainers to accept,
>> and I wonder if we can re-visit some of these cases, with new approaches
>> or perspectives on cost/benefit.
>>
>> There are also potentially promising approaches to mitigation with other
>> technologies such as KASan and gcc plugins, as well as evolving hardware
>> features.
>>
>
> We also have to make sure that the compiler based approach work with
> architectures other than x86. Archs like ppc64 have different memory
> layout and features like KASan may not really map easily with the
> layout. For example we may not be able to implement inline kasan
> instrumentation on ppc64. Also we have issues with stack and
> global out of bounds access check.
>
> I would be interested in this discussion, if we are scheduling this for
> ksummit. I work mostly on ppc64 memory-management subsystem and can
> bring in details of challenges faced with KASan implementation on ppc64.

Should we have a separate discussion for bug-hunting? KASan needs to
expand its architecture coverage, Trinity needs a maintainer, and
there's always new things happening with smatch and coccinelle.

I'd just like to keep bug-hunting entirely separate from
self-protection, as I feel they're distinct topics (though related).

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Andy Lutomirski]

On Mon, Aug 24, 2015 at 10:17 AM, Kees Cook <keescook@chromium.org> wrote:
> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>> On Mon, 24 Aug 2015, Jiri Kosina wrote:
>>
>>> On Mon, 24 Aug 2015, James Morris wrote:
>>>
>>> > I'd recommend Kees Cook be involved, due to his existing efforts in
>>> > kernel hardening.  I think it would be good to invite one or two expert
>>> > security researchers in this area -- Kees would know who.  In terms of
>
> Many of the folks that are good at kernel exploitation don't want to
> help us fix the situation. :)
>
> I'd recommend Lee Campbell, he's got a solid bit of experience from
> the offense side. I think we should extend an invite to spender and
> pageexec as well. They've been on the cutting edge of this for
> decades, and it would be silly not to invite them.
>
>>> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.
>
> Perhaps also Linus and rmk? Some of the protections are very central
> to the kernel (e.g. constification, "read-mostly", segmentation
> through page table swaps or domains, etc). I'd also want Andy
> Lutomirski around, as he's got a lot of deep chipset knowledge. :)
>

What is this chipset knowledge you speak of? :)

One thing that grsecurity addresses (partially or fully?  I haven't
looked that closely): we have tons of static, non-const data structure
that contain function pointers, and we can't make them const because
they get filled in when things are initialized.  Grsecurity mitigates
this with some combination of compiler plugins and pax_open_kernel,
but we could probably come up with a more straightforward solution.
We could add an ro_after_init section, or we could even have a section
for things that are const but are writable through a special function.

>>>
>>> I am pretty sure spender will also have a lot to tell us :p
>>
>> He actually presented at the 2010 security summit:
>> https://grsecurity.net/spender_summit.pdf
>
> This was his bullet list of things that grsecurity/PaX already does
> and that should be in mainline (with my notes in parens). He suggested
> it would take us 10 years to catch up. We're 5 years into that, with
> only a few things partially off this list:
>
>  Remove infoleaks
>  o Symbol information (partial improvement via kptr_restrict)
>  o Slabinfo (partial improvement with 0400 root perms)
>  o PAX_USERCOPY (even if gcc fixed their sizeof bug, we'd be no where
> near this plugin's level of protection)
>  Remove RWX from kernel (in good shape on x86, started on arm)

We still need to do something about the direct map, IIRC.  Or did we
already fix that?

>  Protect sensitive data
>  o Constify function pointers! (no where close to this plugin)

As above, this would be nice.  If we did this upstream, we could do
even better if we outright disallowed non-const function pointers (as
opposed to fixing them up semi-transparently as the plugin does).

>  o IDT/GDT/syscall table/etc (this is partially done, aliases are
> writable still)

I don't think the RO GDT patches ever happened.

FWIW, we can't really eliminate a writable GDT alias without killing
compat performance: set_thread_area rather critically depends on
writing to the GDT at context switch time.

>  o Vsyscall shadow table, see sgrakkyu's remote
> SELinux-disabling exploit
> (http://sgrakkyu.antifork.org/sctp_houdini.c, luto fixed this via
> vsyscall emulation)

Not done yet, because even modern binaries still have the thing
mapped.  We could devote two or three minutes of a KS session to
figuring out how to kill it off for real for modern binaries.

>
> This is far from a comprehensive list, though. The biggest value, I
> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
> for constification and integer overflow.

At the risk of poking a big elephant, I think we should do something
about perf, too.  Perf is really useful, but it's also a *huge* attack
surface.

--Andy

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Julia Lawall]

> One thing that grsecurity addresses (partially or fully?  I haven't
> looked that closely): we have tons of static, non-const data structure
> that contain function pointers, and we can't make them const because
> they get filled in when things are initialized.  Grsecurity mitigates
> this with some combination of compiler plugins and pax_open_kernel,
> but we could probably come up with a more straightforward solution.
> We could add an ro_after_init section, or we could even have a section
> for things that are const but are writable through a special function.

Would it be helpful/possible to make a substructure for the constant parts?

julia

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 10:28 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Mon, Aug 24, 2015 at 10:17 AM, Kees Cook <keescook@chromium.org> wrote:
>> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>>> On Mon, 24 Aug 2015, Jiri Kosina wrote:
>>>
>>>> On Mon, 24 Aug 2015, James Morris wrote:
>>>>
>>>> > I'd recommend Kees Cook be involved, due to his existing efforts in
>>>> > kernel hardening.  I think it would be good to invite one or two expert
>>>> > security researchers in this area -- Kees would know who.  In terms of
>>
>> Many of the folks that are good at kernel exploitation don't want to
>> help us fix the situation. :)
>>
>> I'd recommend Lee Campbell, he's got a solid bit of experience from
>> the offense side. I think we should extend an invite to spender and
>> pageexec as well. They've been on the cutting edge of this for
>> decades, and it would be silly not to invite them.
>>
>>>> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.
>>
>> Perhaps also Linus and rmk? Some of the protections are very central
>> to the kernel (e.g. constification, "read-mostly", segmentation
>> through page table swaps or domains, etc). I'd also want Andy
>> Lutomirski around, as he's got a lot of deep chipset knowledge. :)
>>
>
> What is this chipset knowledge you speak of? :)

You appear to enjoy fixing deep x86 madness. :)

> One thing that grsecurity addresses (partially or fully?  I haven't
> looked that closely): we have tons of static, non-const data structure
> that contain function pointers, and we can't make them const because
> they get filled in when things are initialized.  Grsecurity mitigates
> this with some combination of compiler plugins and pax_open_kernel,

Right. That's the constification and KERNEXEC pieces.

> but we could probably come up with a more straightforward solution.
> We could add an ro_after_init section, or we could even have a section
> for things that are const but are writable through a special function.

I don't think it would be that clean: there are many targets that
legitimately need updating at runtime, but should be otherwise
read-only. The idea with solving that is to use inline
open/close_kernel calls to make them writable briefly and in a way
that is ROP-defensive.

>>>>
>>>> I am pretty sure spender will also have a lot to tell us :p
>>>
>>> He actually presented at the 2010 security summit:
>>> https://grsecurity.net/spender_summit.pdf
>>
>> This was his bullet list of things that grsecurity/PaX already does
>> and that should be in mainline (with my notes in parens). He suggested
>> it would take us 10 years to catch up. We're 5 years into that, with
>> only a few things partially off this list:
>>
>>  Remove infoleaks
>>  o Symbol information (partial improvement via kptr_restrict)
>>  o Slabinfo (partial improvement with 0400 root perms)
>>  o PAX_USERCOPY (even if gcc fixed their sizeof bug, we'd be no where
>> near this plugin's level of protection)
>>  Remove RWX from kernel (in good shape on x86, started on arm)
>
> We still need to do something about the direct map, IIRC.  Or did we
> already fix that?

We have a lot of targets still, but we're headed (slowly) in the right
direction.

>>  Protect sensitive data
>>  o Constify function pointers! (no where close to this plugin)
>
> As above, this would be nice.  If we did this upstream, we could do
> even better if we outright disallowed non-const function pointers (as
> opposed to fixing them up semi-transparently as the plugin does).

I would love this. :)

>>  o IDT/GDT/syscall table/etc (this is partially done, aliases are
>> writable still)
>
> I don't think the RO GDT patches ever happened.
>
> FWIW, we can't really eliminate a writable GDT alias without killing
> compat performance: set_thread_area rather critically depends on
> writing to the GDT at context switch time.

If that's what's needed, I am fine killing compat performance. (We
could attempt to make this a CONFIG for those that expect to run
compat loads for their kernels.)

>>  o Vsyscall shadow table, see sgrakkyu's remote
>> SELinux-disabling exploit
>> (http://sgrakkyu.antifork.org/sctp_houdini.c, luto fixed this via
>> vsyscall emulation)
>
> Not done yet, because even modern binaries still have the thing
> mapped.  We could devote two or three minutes of a KS session to
> figuring out how to kill it off for real for modern binaries.

What uses it? I can run with vsyscall=none without problems...

>> This is far from a comprehensive list, though. The biggest value, I
>> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
>> for constification and integer overflow.
>
> At the risk of poking a big elephant, I think we should do something
> about perf, too.  Perf is really useful, but it's also a *huge* attack
> surface.

No disagreement from me.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Andy Lutomirski]

On Mon, Aug 24, 2015 at 11:01 AM, Kees Cook <keescook@chromium.org> wrote:
> On Mon, Aug 24, 2015 at 10:28 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>> On Mon, Aug 24, 2015 at 10:17 AM, Kees Cook <keescook@chromium.org> wrote:
>>> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>>>> On Mon, 24 Aug 2015, Jiri Kosina wrote:
>>>>
>>>>> On Mon, 24 Aug 2015, James Morris wrote:
>>>>>
>>>>> > I'd recommend Kees Cook be involved, due to his existing efforts in
>>>>> > kernel hardening.  I think it would be good to invite one or two expert
>>>>> > security researchers in this area -- Kees would know who.  In terms of
>>>
>>> Many of the folks that are good at kernel exploitation don't want to
>>> help us fix the situation. :)
>>>
>>> I'd recommend Lee Campbell, he's got a solid bit of experience from
>>> the offense side. I think we should extend an invite to spender and
>>> pageexec as well. They've been on the cutting edge of this for
>>> decades, and it would be silly not to invite them.
>>>
>>>>> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.
>>>
>>> Perhaps also Linus and rmk? Some of the protections are very central
>>> to the kernel (e.g. constification, "read-mostly", segmentation
>>> through page table swaps or domains, etc). I'd also want Andy
>>> Lutomirski around, as he's got a lot of deep chipset knowledge. :)
>>>
>>
>> What is this chipset knowledge you speak of? :)
>
> You appear to enjoy fixing deep x86 madness. :)
>
>> One thing that grsecurity addresses (partially or fully?  I haven't
>> looked that closely): we have tons of static, non-const data structure
>> that contain function pointers, and we can't make them const because
>> they get filled in when things are initialized.  Grsecurity mitigates
>> this with some combination of compiler plugins and pax_open_kernel,
>
> Right. That's the constification and KERNEXEC pieces.
>
>> but we could probably come up with a more straightforward solution.
>> We could add an ro_after_init section, or we could even have a section
>> for things that are const but are writable through a special function.
>
> I don't think it would be that clean: there are many targets that
> legitimately need updating at runtime, but should be otherwise
> read-only. The idea with solving that is to use inline
> open/close_kernel calls to make them writable briefly and in a way
> that is ROP-defensive.

I really dislike open_kernel.  It's basically "turn off RO completely
-- please ROP me".  If we had a nice, conservative "poke this value
into this place that is hopefully actually a compile-time constant
address", I'd be much happier with it.

>
>>>  Protect sensitive data
>>>  o Constify function pointers! (no where close to this plugin)
>>
>> As above, this would be nice.  If we did this upstream, we could do
>> even better if we outright disallowed non-const function pointers (as
>> opposed to fixing them up semi-transparently as the plugin does).
>
> I would love this. :)
>
>>>  o IDT/GDT/syscall table/etc (this is partially done, aliases are
>>> writable still)
>>
>> I don't think the RO GDT patches ever happened.
>>
>> FWIW, we can't really eliminate a writable GDT alias without killing
>> compat performance: set_thread_area rather critically depends on
>> writing to the GDT at context switch time.
>
> If that's what's needed, I am fine killing compat performance. (We
> could attempt to make this a CONFIG for those that expect to run
> compat loads for their kernels.)

We would need to change the default behavior or arch_set_fs, too,
since some 64-bit processes use that mechanism as well.  But that's
easy.

>
>>>  o Vsyscall shadow table, see sgrakkyu's remote
>>> SELinux-disabling exploit
>>> (http://sgrakkyu.antifork.org/sctp_houdini.c, luto fixed this via
>>> vsyscall emulation)
>>
>> Not done yet, because even modern binaries still have the thing
>> mapped.  We could devote two or three minutes of a KS session to
>> figuring out how to kill it off for real for modern binaries.
>
> What uses it? I can run with vsyscall=none without problems...

Nothing, but it's still mapped by default.  I want something like the
stack access flags or an arch_prctl so glibc can turn it off by
default.

--Andy

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Morris]

On Mon, 24 Aug 2015, Kees Cook wrote:

> Should we have a separate discussion for bug-hunting? KASan needs to
> expand its architecture coverage, Trinity needs a maintainer, and
> there's always new things happening with smatch and coccinelle.
> 
> I'd just like to keep bug-hunting entirely separate from
> self-protection, as I feel they're distinct topics (though related).

I agree -- this needs to be a separate, related discussion.


-- 
James Morris
<jmorris@namei.org>

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Thomas Gleixner]

On Mon, 24 Aug 2015, Kees Cook wrote:
> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
> This is far from a comprehensive list, though. The biggest value, I
> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
> for constification and integer overflow.

There is another aspect. We need to make developers more aware of the
potential attack issues. I learned my lesson with the futex disaster
and since then I certainly look with a different set of eyes at user
space facing code. I doubt that we want that everyone experiences the
disaster himself (though that's a very enlightening experience), but
we should try to document incidents and the lessons learned from
them. Right now we just rely on those who are deep into the security
realm or the few people who learned it the hard way.

Thanks,

	tglx

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 11:19 AM, Andy Lutomirski <luto@amacapital.net> wrote:
> On Mon, Aug 24, 2015 at 11:01 AM, Kees Cook <keescook@chromium.org> wrote:
>> On Mon, Aug 24, 2015 at 10:28 AM, Andy Lutomirski <luto@amacapital.net> wrote:
>>> On Mon, Aug 24, 2015 at 10:17 AM, Kees Cook <keescook@chromium.org> wrote:
>>>> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>>>>> On Mon, 24 Aug 2015, Jiri Kosina wrote:
>>>>>
>>>>>> On Mon, 24 Aug 2015, James Morris wrote:
>>>>>>
>>>>>> > I'd recommend Kees Cook be involved, due to his existing efforts in
>>>>>> > kernel hardening.  I think it would be good to invite one or two expert
>>>>>> > security researchers in this area -- Kees would know who.  In terms of
>>>>
>>>> Many of the folks that are good at kernel exploitation don't want to
>>>> help us fix the situation. :)
>>>>
>>>> I'd recommend Lee Campbell, he's got a solid bit of experience from
>>>> the offense side. I think we should extend an invite to spender and
>>>> pageexec as well. They've been on the cutting edge of this for
>>>> decades, and it would be silly not to invite them.
>>>>
>>>>>> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.
>>>>
>>>> Perhaps also Linus and rmk? Some of the protections are very central
>>>> to the kernel (e.g. constification, "read-mostly", segmentation
>>>> through page table swaps or domains, etc). I'd also want Andy
>>>> Lutomirski around, as he's got a lot of deep chipset knowledge. :)
>>>>
>>>
>>> What is this chipset knowledge you speak of? :)
>>
>> You appear to enjoy fixing deep x86 madness. :)
>>
>>> One thing that grsecurity addresses (partially or fully?  I haven't
>>> looked that closely): we have tons of static, non-const data structure
>>> that contain function pointers, and we can't make them const because
>>> they get filled in when things are initialized.  Grsecurity mitigates
>>> this with some combination of compiler plugins and pax_open_kernel,
>>
>> Right. That's the constification and KERNEXEC pieces.
>>
>>> but we could probably come up with a more straightforward solution.
>>> We could add an ro_after_init section, or we could even have a section
>>> for things that are const but are writable through a special function.
>>
>> I don't think it would be that clean: there are many targets that
>> legitimately need updating at runtime, but should be otherwise
>> read-only. The idea with solving that is to use inline
>> open/close_kernel calls to make them writable briefly and in a way
>> that is ROP-defensive.
>
> I really dislike open_kernel.  It's basically "turn off RO completely
> -- please ROP me".  If we had a nice, conservative "poke this value
> into this place that is hopefully actually a compile-time constant
> address", I'd be much happier with it.

Well... open_kernel turns off preempt. With the inlining, no-preempt,
and barriers, it should be correct, yes? Used as:

kernel_open();
readonly_thing = whatever;
kernel_close();

This should be way better than what we have already.

>>>>  Protect sensitive data
>>>>  o Constify function pointers! (no where close to this plugin)
>>>
>>> As above, this would be nice.  If we did this upstream, we could do
>>> even better if we outright disallowed non-const function pointers (as
>>> opposed to fixing them up semi-transparently as the plugin does).
>>
>> I would love this. :)
>>
>>>>  o IDT/GDT/syscall table/etc (this is partially done, aliases are
>>>> writable still)
>>>
>>> I don't think the RO GDT patches ever happened.
>>>
>>> FWIW, we can't really eliminate a writable GDT alias without killing
>>> compat performance: set_thread_area rather critically depends on
>>> writing to the GDT at context switch time.
>>
>> If that's what's needed, I am fine killing compat performance. (We
>> could attempt to make this a CONFIG for those that expect to run
>> compat loads for their kernels.)
>
> We would need to change the default behavior or arch_set_fs, too,
> since some 64-bit processes use that mechanism as well.  But that's
> easy.
>
>>
>>>>  o Vsyscall shadow table, see sgrakkyu's remote
>>>> SELinux-disabling exploit
>>>> (http://sgrakkyu.antifork.org/sctp_houdini.c, luto fixed this via
>>>> vsyscall emulation)
>>>
>>> Not done yet, because even modern binaries still have the thing
>>> mapped.  We could devote two or three minutes of a KS session to
>>> figuring out how to kill it off for real for modern binaries.
>>
>> What uses it? I can run with vsyscall=none without problems...
>
> Nothing, but it's still mapped by default.  I want something like the
> stack access flags or an arch_prctl so glibc can turn it off by
> default.

Oh, instead of just 87983c66bc02 plus =none, you want a per-process
runtime config? I think =none should just become the global default.
:)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Thomas Gleixner]

On Mon, 24 Aug 2015, Thomas Gleixner wrote:
> On Mon, 24 Aug 2015, Kees Cook wrote:
> > On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
> > This is far from a comprehensive list, though. The biggest value, I
> > think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
> > for constification and integer overflow.
> 
> There is another aspect. We need to make developers more aware of the
> potential attack issues. I learned my lesson with the futex disaster
> and since then I certainly look with a different set of eyes at user
> space facing code. I doubt that we want that everyone experiences the
> disaster himself (though that's a very enlightening experience), but
> we should try to document incidents and the lessons learned from
> them. Right now we just rely on those who are deep into the security
> realm or the few people who learned it the hard way.

A good way to start would be to actually force developers to document
meticulously the possible states of user space variable(s) which
influence the behaviour of the interface. That can be a mindboggling
exercise depending on the complexity of the interface, but it helps
both the implementer and the reviewer.

Thanks,

	tglx

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 11:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Mon, 24 Aug 2015, Kees Cook wrote:
>> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>> This is far from a comprehensive list, though. The biggest value, I
>> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
>> for constification and integer overflow.
>
> There is another aspect. We need to make developers more aware of the
> potential attack issues. I learned my lesson with the futex disaster
> and since then I certainly look with a different set of eyes at user
> space facing code. I doubt that we want that everyone experiences the
> disaster himself (though that's a very enlightening experience), but
> we should try to document incidents and the lessons learned from
> them. Right now we just rely on those who are deep into the security
> realm or the few people who learned it the hard way.

Yeah, it can be a hard perspective shift to make. And shifting the
thinking about the kernel itself to always operating in a compromised
state makes thinking about how to protect it much easier. User space
is trying to hurt us! :)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Bottomley]

On Mon, 2015-08-24 at 09:35 -0700, Kees Cook wrote:
> On Mon, Aug 24, 2015 at 5:29 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
> > On Mon, Aug 24, 2015 at 6:20 AM, James Morris <jmorris@namei.org> wrote:
> >
> >> There are also potentially promising approaches to mitigation with other
> >> technologies such as KASan and gcc plugins, as well as evolving hardware
> >> features.
> >
> > What I've discovered when running KASan the last few weeks is that
> > this points back to the question of tests ... I've been using Trinity
> > to find bugs, but it is more likely to kill itself or cause OOM than
> > trigger any boundary overrun bugs.
> >
> > Kselftest may be helpful, but basically any loads that heavily
> > exercise the kernel internals are helpful to harden the kernel.
> > Some of these are custom test suites I suspect. Any good hints
> > for a simple embedded developer like me?
> 
> I agree with the sentiment here, but not with the language. Finding
> flaws (which is what selftests, KASan, Trinity, etc do) isn't
> hardening. Hardening is stopping the exploitation of flaws.

Um, forgive me for being dense, but doesn't fixing the flaws stop their
exploitation?  In any event, Hardening means "reducing the attack
surface" and that encompasses both active and passive means (including
actual bug fixing).

>  The
> hardening the kernel needs is about taking away exploitation tools,
> not killing bugs. (Though killing bugs is still great.)

It's both.  One of the old standards for attacking C code was buffer
overruns.  Remove those via detection tools and you reduce the attack
surface.

> As an example, making the kernel code memory read-only means an
> attacker cannot just directly change the kernel's execution path when
> they use an arbitrary memory-writing flaw. (This feature is mostly
> enabled via CONFIG_DEBUG_RODATA, and was very recently added to ARM,
> though isn't at 100% coverage for all the physical memory aliases.)

Yes, I like this one a lot.  Adding mechanisms that don't increase
complexity like this are good active means.  However, I become less
enamoured of things like selinux and grsecurity which add complexity in
the name of active attack surface reduction.  That's not to say never do
it, it's just to say that attack surface is directly related to
complexity.

James

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Morris]

On Mon, 24 Aug 2015, James Bottomley wrote:

> Um, forgive me for being dense, but doesn't fixing the flaws stop their
> exploitation?  In any event, Hardening means "reducing the attack
> surface" and that encompasses both active and passive means (including
> actual bug fixing).

Hardening is mitigating those flaws.  You'll never find every flaw, but 
you can mitigate against entire classes of flaws being exploited.

> >  The
> > hardening the kernel needs is about taking away exploitation tools,
> > not killing bugs. (Though killing bugs is still great.)
> 
> It's both.  One of the old standards for attacking C code was buffer
> overruns.  Remove those via detection tools and you reduce the attack
> surface.

In this case, we're specifically talking about hardening the kernel to 
mitigate exploitation of flaws.  Kernel self-protection may be a better 
term (and recently surfaced in an NSA presentation at LSS: p.23 of

http://kernsec.org/files/lss2015/lss2015_selinuxinandroidlollipopandm_smalley.pdf


-- 
James Morris
<jmorris@namei.org>

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[josh]

On Mon, Aug 24, 2015 at 01:09:01PM -0700, James Bottomley wrote:
> On Mon, 2015-08-24 at 09:35 -0700, Kees Cook wrote:
> > On Mon, Aug 24, 2015 at 5:29 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
> > > On Mon, Aug 24, 2015 at 6:20 AM, James Morris <jmorris@namei.org> wrote:
> > >
> > >> There are also potentially promising approaches to mitigation with other
> > >> technologies such as KASan and gcc plugins, as well as evolving hardware
> > >> features.
> > >
> > > What I've discovered when running KASan the last few weeks is that
> > > this points back to the question of tests ... I've been using Trinity
> > > to find bugs, but it is more likely to kill itself or cause OOM than
> > > trigger any boundary overrun bugs.
> > >
> > > Kselftest may be helpful, but basically any loads that heavily
> > > exercise the kernel internals are helpful to harden the kernel.
> > > Some of these are custom test suites I suspect. Any good hints
> > > for a simple embedded developer like me?
> > 
> > I agree with the sentiment here, but not with the language. Finding
> > flaws (which is what selftests, KASan, Trinity, etc do) isn't
> > hardening. Hardening is stopping the exploitation of flaws.
> 
> Um, forgive me for being dense, but doesn't fixing the flaws stop their
> exploitation?  In any event, Hardening means "reducing the attack
> surface" and that encompasses both active and passive means (including
> actual bug fixing).

There's a difference between fixing an individual vulnerability
discovered by fuzzing and other tools, and fixing (or at least making it
easier to do the right thing for) an entire *class* of
vulnerabilities.  We should fix the former when we find them, but any
time we see more than a couple bugs of the same time, we should be
asking "how can we never have another?".

- Josh Triplett

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Thomas Gleixner]

On Tue, 25 Aug 2015, James Morris wrote:
> On Mon, 24 Aug 2015, James Bottomley wrote:
> 
> > Um, forgive me for being dense, but doesn't fixing the flaws stop their
> > exploitation?  In any event, Hardening means "reducing the attack
> > surface" and that encompasses both active and passive means (including
> > actual bug fixing).
> 
> Hardening is mitigating those flaws.  You'll never find every flaw, but 
> you can mitigate against entire classes of flaws being exploited.

Fair enough, but I agree with James, that we should not make this an
isolated topic.

Let's look at it from a different POV. 10+ years ago locking was
something which was only understood by a minority of the developers
and they had to figure out (or not) the horrible once per year
deadlocks.

Then RT happened to expose a gazillion of those issues and once we got
tired of debugging this hard to understand problems we came up with a
tool which tells Joe Developer where he screwed up.

As a consequence developers got more aware of locking semantics and
the number of hard to figure out lock inversion problems got reduced
significantly.

We added a lot of other mechanisms to detect different classes of
failure (use after free, ....). Both runtime debugging facilities and
static code analysis have helped to improve the code quality and the
awareness of developers.

Security and the pitfalls of user space facing code are still a riddle
wrapped up in an enigma for most developers, so the question is how
can we improve that situation.

While we certainly want to add mechanisms which prevent flaws to be
exploited we surely want to do something about educating people how to
avoid the flaws in the first place.

One can read about interesting ways to subvert a lot of the hardening
techniques every other day, so while we certainly want to add
hardening techniques, it's even more important that these techniques
become the last parachute and not the catch it all mechanism because
we gave up on the underlying issues.

I totally agree that we cannot prevent all flaws, but we certainly can
do better in reducing the quantity. And that means that we need to
educate people. And that education involves traditional training,
documentation and clever usage of tools. If we can use hardening
techniques to slap developers on their fingers, that's certainly a
good thing. But we don't want to decouple the hardening from 'reduce
the flaws' as you might create the impression that it's not that
important to think security aware because the hardering techniques
will prevent the exploits anyway.

Thanks,

	tglx

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Greg KH]

On Mon, Aug 24, 2015 at 12:00:15PM -0700, Kees Cook wrote:
> On Mon, Aug 24, 2015 at 11:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > On Mon, 24 Aug 2015, Kees Cook wrote:
> >> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
> >> This is far from a comprehensive list, though. The biggest value, I
> >> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
> >> for constification and integer overflow.
> >
> > There is another aspect. We need to make developers more aware of the
> > potential attack issues. I learned my lesson with the futex disaster
> > and since then I certainly look with a different set of eyes at user
> > space facing code. I doubt that we want that everyone experiences the
> > disaster himself (though that's a very enlightening experience), but
> > we should try to document incidents and the lessons learned from
> > them. Right now we just rely on those who are deep into the security
> > realm or the few people who learned it the hard way.
> 
> Yeah, it can be a hard perspective shift to make. And shifting the
> thinking about the kernel itself to always operating in a compromised
> state makes thinking about how to protect it much easier. User space
> is trying to hurt us! :)

Microsoft's security team, which was responsible for forcing all of
their developers to undergo some security training every year, has
boiled it all down to these simple 4 words:

	All input is evil.

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Morris]

On Mon, 24 Aug 2015, Thomas Gleixner wrote:

> While we certainly want to add mechanisms which prevent flaws to be
> exploited we surely want to do something about educating people how to
> avoid the flaws in the first place.

What if we had a session where a security researcher presented an overview 
of classes kernel security flaws and how they're typically exploited?  
This would be aimed at core kernel developers, specifically.

> 
> One can read about interesting ways to subvert a lot of the hardening
> techniques every other day, so while we certainly want to add
> hardening techniques, it's even more important that these techniques
> become the last parachute and not the catch it all mechanism because
> we gave up on the underlying issues.

Nobody is giving up on underlying issues, in fact, we are doing pretty 
well there, but not so well with kernel mitigations, which keeps biting us 
in uncomfortable places.

> I totally agree that we cannot prevent all flaws, but we certainly can 
> do better in reducing the quantity. And that means that we need to 
> educate people. And that education involves traditional training, 
> documentation and clever usage of tools. If we can use hardening 
> techniques to slap developers on their fingers, that's certainly a good 
> thing. But we don't want to decouple the hardening from 'reduce the 
> flaws' as you might create the impression that it's not that important 
> to think security aware because the hardering techniques will prevent 
> the exploits anyway.

You're right, but as mentioned, I do believe that we are doing pretty well 
at code quality vs., say, proprietary development, as demonstrated by 
numerous quantitative studies.  The Linux kernel is used heavily and 
across many different types of environments, with a tight user/developer 
feedback loop, and all the code is subject to public review and 
accountability.  We actually pioneered this stuff :)

There is a lot of good testing happening, both in the community and via 
corporate QA, and of course there can be more, but the point of this 
proposal is that we are not leading in kernel self-protection and in fact 
somewhat behind the state of the art.  This is a problem which we 
specifically need to address in consultation with the research community 
and with core kernel folk on board.

I'm not saying the other stuff is not important, it absolutely is, but I 
wanted to specifically highlight kernel self-protection as a significant 
issue requiring collaboration between different groups.

There's no reason we can't have an expanded track, including an education 
session for kernel developers as mentioned above, as well as a testing 
session, and collect these together with kernel self-protection as a 
"kernel security hardening" track.  (This is of course distinct from 
functional security mechanisms, which we covered at LSS last week). I 
didn't want to have such a wide scope given limited availability of core 
developers on any one topic, but if folk are happy to consider expanding 
it, then fine.


- James
-- 
James Morris
<jmorris@namei.org>

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 1:09 PM, James Bottomley
<James.Bottomley@hansenpartnership.com> wrote:
> On Mon, 2015-08-24 at 09:35 -0700, Kees Cook wrote:
>> On Mon, Aug 24, 2015 at 5:29 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>> > On Mon, Aug 24, 2015 at 6:20 AM, James Morris <jmorris@namei.org> wrote:
>> >
>> >> There are also potentially promising approaches to mitigation with other
>> >> technologies such as KASan and gcc plugins, as well as evolving hardware
>> >> features.
>> >
>> > What I've discovered when running KASan the last few weeks is that
>> > this points back to the question of tests ... I've been using Trinity
>> > to find bugs, but it is more likely to kill itself or cause OOM than
>> > trigger any boundary overrun bugs.
>> >
>> > Kselftest may be helpful, but basically any loads that heavily
>> > exercise the kernel internals are helpful to harden the kernel.
>> > Some of these are custom test suites I suspect. Any good hints
>> > for a simple embedded developer like me?
>>
>> I agree with the sentiment here, but not with the language. Finding
>> flaws (which is what selftests, KASan, Trinity, etc do) isn't
>> hardening. Hardening is stopping the exploitation of flaws.
>
> Um, forgive me for being dense, but doesn't fixing the flaws stop their
> exploitation?  In any event, Hardening means "reducing the attack
> surface" and that encompasses both active and passive means (including
> actual bug fixing).

I think "hardening" isn't a very precise word for security work,
actually. The language I try to use around these things (each in the
context of "security"):

flaw - a bug that provides some type of attack primitive (generally
memory reads and writes).
exploit - delivery of executable payload via flaws (this is a
technique to gain control via the flaw primitives)

Many times all we see are the flaws and a theoretical example. If
we're lucky, we see a brief PoC that triggers an Oops or something.
Then we fix the flaw and move on. We are not as exposed to fully
functional exploits. Overwriting syscall tables, IDT entries, function
pointers, or the kernel itself, chaining code snippets with ROP, heap
manipulation, and on and on.

So, elimination of flaws is certainly good, but there will always be
more. And it is worth noting that they exist _now_. There are multiple
full privileges escalation flaws in the kernel you're using right now.
We -- the upstream kernel community -- just don't know where they are.
Even Jon's conservative estimate is that these flaws tend to live at
least 5 years: https://lwn.net/Articles/410606/

We are finding the bugs, and we can do better, but that's not what I
think needs the most attention right now. We need to kill classes of
bugs and classes of exploits. To kill a bug class, we must remove the
possibility that it can ever go wrong in the first place. Many people
stop here when thinking about hardening, but we must move on to
killing classes of exploit. I'll continue to use my W^X kernel code
example, since it was not, from an operational stance, a flaw that
kernel code was writable. But it's a exploitation weakness. If an
attacker just needs to directly target a kernel memory location with
their memory-write primitive, all their work is done, that user loses
control of their kernel, game over.

We need to add the safety nets under the acrobats, since they can fall
at any time.

>>  The
>> hardening the kernel needs is about taking away exploitation tools,
>> not killing bugs. (Though killing bugs is still great.)
>
> It's both.  One of the old standards for attacking C code was buffer
> overruns.  Remove those via detection tools and you reduce the attack
> surface.

Well, attack surface is just what interfaces are exposed to an
attacker. But yes, killing bug classes is important. Making bugs
unexploitable is even better because then security bugs really do just
become bugs.

>> As an example, making the kernel code memory read-only means an
>> attacker cannot just directly change the kernel's execution path when
>> they use an arbitrary memory-writing flaw. (This feature is mostly
>> enabled via CONFIG_DEBUG_RODATA, and was very recently added to ARM,
>> though isn't at 100% coverage for all the physical memory aliases.)
>
> Yes, I like this one a lot.  Adding mechanisms that don't increase
> complexity like this are good active means.  However, I become less
> enamoured of things like selinux and grsecurity which add complexity in
> the name of active attack surface reduction.  That's not to say never do
> it, it's just to say that attack surface is directly related to
> complexity.

FWIW, really only reduces userspace attack surface (all syscalls are
still available to a confined process). seccomp reduces attack
surface. grsecurity has an MAC component, but grsecurity (with PaX) is
much larger. Regardless, some of these nets will increase complexity.
It's the same for anti-lock brakes and airbags[1]. We have to take on
this burden to protect our users from our mistakes.

-Kees

[1] Borrowing mricon's great analogy as presented at the Security
Summit this year:
http://kernsec.org/files/lss2015/giant-bags-of-mostly-water.pdf

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 1:17 PM, James Morris <jmorris@namei.org> wrote:
> On Mon, 24 Aug 2015, James Bottomley wrote:
>
>> Um, forgive me for being dense, but doesn't fixing the flaws stop their
>> exploitation?  In any event, Hardening means "reducing the attack
>> surface" and that encompasses both active and passive means (including
>> actual bug fixing).
>
> Hardening is mitigating those flaws.  You'll never find every flaw, but
> you can mitigate against entire classes of flaws being exploited.
>
>> >  The
>> > hardening the kernel needs is about taking away exploitation tools,
>> > not killing bugs. (Though killing bugs is still great.)
>>
>> It's both.  One of the old standards for attacking C code was buffer
>> overruns.  Remove those via detection tools and you reduce the attack
>> surface.
>
> In this case, we're specifically talking about hardening the kernel to
> mitigate exploitation of flaws.  Kernel self-protection may be a better
> term (and recently surfaced in an NSA presentation at LSS: p.23 of
>
> http://kernsec.org/files/lss2015/lss2015_selinuxinandroidlollipopandm_smalley.pdf

Yup. Using "kernel self-protection" cuts right to the point. I'm in
debt to Stephen for finding the right marketing term for "kernel
memory corruption vulnerability exploit mitigation". :)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 1:46 PM, Thomas Gleixner <tglx@linutronix.de> wrote:
> On Tue, 25 Aug 2015, James Morris wrote:
>> On Mon, 24 Aug 2015, James Bottomley wrote:
>>
>> > Um, forgive me for being dense, but doesn't fixing the flaws stop their
>> > exploitation?  In any event, Hardening means "reducing the attack
>> > surface" and that encompasses both active and passive means (including
>> > actual bug fixing).
>>
>> Hardening is mitigating those flaws.  You'll never find every flaw, but
>> you can mitigate against entire classes of flaws being exploited.
>
> Fair enough, but I agree with James, that we should not make this an
> isolated topic.
>
> Let's look at it from a different POV. 10+ years ago locking was
> something which was only understood by a minority of the developers
> and they had to figure out (or not) the horrible once per year
> deadlocks.
>
> Then RT happened to expose a gazillion of those issues and once we got
> tired of debugging this hard to understand problems we came up with a
> tool which tells Joe Developer where he screwed up.
>
> As a consequence developers got more aware of locking semantics and
> the number of hard to figure out lock inversion problems got reduced
> significantly.
>
> We added a lot of other mechanisms to detect different classes of
> failure (use after free, ....). Both runtime debugging facilities and
> static code analysis have helped to improve the code quality and the
> awareness of developers.
>
> Security and the pitfalls of user space facing code are still a riddle
> wrapped up in an enigma for most developers, so the question is how
> can we improve that situation.
>
> While we certainly want to add mechanisms which prevent flaws to be
> exploited we surely want to do something about educating people how to
> avoid the flaws in the first place.

Absolutely. I just think they are separate discussions. Teaching
someone how to avoid integer overflows is totally different from
teaching them why a fix-position writeable and executable memory
region should never exist.

>
> One can read about interesting ways to subvert a lot of the hardening
> techniques every other day, so while we certainly want to add
> hardening techniques, it's even more important that these techniques
> become the last parachute and not the catch it all mechanism because
> we gave up on the underlying issues.

Yes, it's absolutely an arms race. But there's no real way around
this. What we have to do is actually start running the race. :)

> I totally agree that we cannot prevent all flaws, but we certainly can
> do better in reducing the quantity. And that means that we need to
> educate people. And that education involves traditional training,
> documentation and clever usage of tools. If we can use hardening
> techniques to slap developers on their fingers, that's certainly a
> good thing. But we don't want to decouple the hardening from 'reduce
> the flaws' as you might create the impression that it's not that
> important to think security aware because the hardering techniques
> will prevent the exploits anyway.

Kernel self-protection systems aren't for slapping developers on their
fingers: it's about maintaining kernel integrity for a user's system
that is under active attack. The harder it is to attack, the better
the chance they will have to detect it.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Andy Lutomirski]

On Mon, Aug 24, 2015 at 3:55 PM, Kees Cook <keescook@chromium.org> wrote:
> On Mon, Aug 24, 2015 at 1:09 PM, James Bottomley
> <James.Bottomley@hansenpartnership.com> wrote:
>> On Mon, 2015-08-24 at 09:35 -0700, Kees Cook wrote:
>>>  The
>>> hardening the kernel needs is about taking away exploitation tools,
>>> not killing bugs. (Though killing bugs is still great.)
>>
>> It's both.  One of the old standards for attacking C code was buffer
>> overruns.  Remove those via detection tools and you reduce the attack
>> surface.
>
> Well, attack surface is just what interfaces are exposed to an
> attacker. But yes, killing bug classes is important. Making bugs
> unexploitable is even better because then security bugs really do just
> become bugs.

There are plenty of cases where mitigation of an exploit technique
class changes a family of bugs from privilege escalation to DoS, and
that's a big deal.  For example, min mmap addresses mostly turn
NULL-pointer dereferences into DoS.  SMAP (and, to a lesser extent,
UDEREF) turns "upside-down" gsbase (a lovely x86ism that's involved in
the BadIRET privesc and quite a few others) into just DoS.

These aren't cases where it's a PITA for an attacker to get more than
DoS -- these are cases where the mitigations really do prevent
privilege escalation completely because they kick in before the
attacker gets to control a value that's used for anything.

--Andy

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 3:22 PM, James Morris <jmorris@namei.org> wrote:
> On Mon, 24 Aug 2015, Thomas Gleixner wrote:
>
>> While we certainly want to add mechanisms which prevent flaws to be
>> exploited we surely want to do something about educating people how to
>> avoid the flaws in the first place.
>
> What if we had a session where a security researcher presented an overview
> of classes kernel security flaws and how they're typically exploited?
> This would be aimed at core kernel developers, specifically.

I think this would be great, but I wouldn't want to sacrifice
discussion time on which self-protections to start working on, and the
best ways to approach them.

Could we assign this as homework instead? There are countless examples
of well described kernel exploits already visible on the web. Here's
one that used ARM's historical lack of text protection (from Lee
Campbell):
http://googleprojectzero.blogspot.com/2015/01/exploiting-nvmap-to-escape-chrome.html
or overwriting function pointers (from the attack mentioned in
spender's 2010 LSS slides):
http://kernelbof.blogspot.com/2009/07/even-when-one-byte-matters.html
or Andy's exploit of the NMI issue:
http://www.openwall.com/lists/oss-security/2015/08/04/8

>> One can read about interesting ways to subvert a lot of the hardening
>> techniques every other day, so while we certainly want to add
>> hardening techniques, it's even more important that these techniques
>> become the last parachute and not the catch it all mechanism because
>> we gave up on the underlying issues.
>
> Nobody is giving up on underlying issues, in fact, we are doing pretty
> well there, but not so well with kernel mitigations, which keeps biting us
> in uncomfortable places.
>
>> I totally agree that we cannot prevent all flaws, but we certainly can
>> do better in reducing the quantity. And that means that we need to
>> educate people. And that education involves traditional training,
>> documentation and clever usage of tools. If we can use hardening
>> techniques to slap developers on their fingers, that's certainly a good
>> thing. But we don't want to decouple the hardening from 'reduce the
>> flaws' as you might create the impression that it's not that important
>> to think security aware because the hardering techniques will prevent
>> the exploits anyway.
>
> You're right, but as mentioned, I do believe that we are doing pretty well
> at code quality vs., say, proprietary development, as demonstrated by
> numerous quantitative studies.  The Linux kernel is used heavily and
> across many different types of environments, with a tight user/developer
> feedback loop, and all the code is subject to public review and
> accountability.  We actually pioneered this stuff :)
>
> There is a lot of good testing happening, both in the community and via
> corporate QA, and of course there can be more, but the point of this
> proposal is that we are not leading in kernel self-protection and in fact
> somewhat behind the state of the art.  This is a problem which we
> specifically need to address in consultation with the research community
> and with core kernel folk on board.
>
> I'm not saying the other stuff is not important, it absolutely is, but I
> wanted to specifically highlight kernel self-protection as a significant
> issue requiring collaboration between different groups.

Right. We already know how to do bug-hunting, testing, etc. They can
surely be improved, but right now I think we need to spend time
learning how to build these kinds of defenses.

> There's no reason we can't have an expanded track, including an education
> session for kernel developers as mentioned above, as well as a testing
> session, and collect these together with kernel self-protection as a
> "kernel security hardening" track.  (This is of course distinct from
> functional security mechanisms, which we covered at LSS last week). I
> didn't want to have such a wide scope given limited availability of core
> developers on any one topic, but if folk are happy to consider expanding
> it, then fine.

We could have everyone write their first kernel privilege escalation exploit. :)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 3:57 PM, Kees Cook <keescook@chromium.org> wrote:
> On Mon, Aug 24, 2015 at 1:17 PM, James Morris <jmorris@namei.org> wrote:
>> On Mon, 24 Aug 2015, James Bottomley wrote:
>>
>>> Um, forgive me for being dense, but doesn't fixing the flaws stop their
>>> exploitation?  In any event, Hardening means "reducing the attack
>>> surface" and that encompasses both active and passive means (including
>>> actual bug fixing).
>>
>> Hardening is mitigating those flaws.  You'll never find every flaw, but
>> you can mitigate against entire classes of flaws being exploited.
>>
>>> >  The
>>> > hardening the kernel needs is about taking away exploitation tools,
>>> > not killing bugs. (Though killing bugs is still great.)
>>>
>>> It's both.  One of the old standards for attacking C code was buffer
>>> overruns.  Remove those via detection tools and you reduce the attack
>>> surface.
>>
>> In this case, we're specifically talking about hardening the kernel to
>> mitigate exploitation of flaws.  Kernel self-protection may be a better
>> term (and recently surfaced in an NSA presentation at LSS: p.23 of
>>
>> http://kernsec.org/files/lss2015/lss2015_selinuxinandroidlollipopandm_smalley.pdf
>
> Yup. Using "kernel self-protection" cuts right to the point. I'm in
> debt to Stephen for finding the right marketing term for "kernel
> memory corruption vulnerability exploit mitigation". :)

pageexec informs me he used the term earlier...
https://pax.grsecurity.net/docs/PaXTeam-H2HC12-PaX-kernel-self-protection.pdf

Regardless, you're all awesome.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Theodore Ts'o]

On Mon, Aug 24, 2015 at 04:20:47PM -0700, Kees Cook wrote:
> 
> Could we assign this as homework instead?  There are countless examples
> of well described kernel exploits already visible on the web....

Might I suggest a somewhat higher-level homework?  What are the kernel
self-protection features that would be most useful for us to
implement, and --- this is critically important --- why aren't we
doing them already, and how can we fix that higher-order issue?

Is it because adding a particular feature would incur a huge
performance penalty?

Is it because no company has been willing to fund developers to work
on that particular feature to date?  (BTW, I consider the fact that
various companies collectively wasn't able to find a place for the
trinity maintainer to find a place to land to be somewhat of a failure
of the ecosystem, but maybe the tool wasn't as useful as we think, or
it maybe we failed to make the case to the correct set of
bean-counters.)

If the answer is that it's obvious what needs to be done, but (a) we
can't find anyone to bell the cat, or (b) the patches are going to be
rejected out of hand for one reason or another, the kernel summit is a
great opportunity to see if some face-to-face discussion address the
problem.  OTOH, if the fundamental problem is that we can't get the
headcount funded, then discussion at the kernel summit is probably not
going to be a good use of our time.  :-/

      	      	       	      	     - Ted

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[James Morris]

On Mon, 24 Aug 2015, Theodore Ts'o wrote:

> If the answer is that it's obvious what needs to be done, but (a) we
> can't find anyone to bell the cat, or (b) the patches are going to be
> rejected out of hand for one reason or another, the kernel summit is a
> great opportunity to see if some face-to-face discussion address the
> problem.  OTOH, if the fundamental problem is that we can't get the
> headcount funded, then discussion at the kernel summit is probably not
> going to be a good use of our time.  :-/

With my corp hat on, I find that it's much more difficult to find an 
appropriate developer to hire for this kind of work than it is to find 
funding for it.

Emily seems to be facing the same issue, from her presentation at LSS.



-- 
James Morris
<jmorris@namei.org>

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 4:54 PM, Theodore Ts'o <tytso@mit.edu> wrote:
> On Mon, Aug 24, 2015 at 04:20:47PM -0700, Kees Cook wrote:
>>
>> Could we assign this as homework instead?  There are countless examples
>> of well described kernel exploits already visible on the web....
>
> Might I suggest a somewhat higher-level homework?  What are the kernel
> self-protection features that would be most useful for us to
> implement, and --- this is critically important --- why aren't we

I listed a bunch in my earlier reply. You can see a nice breakdown of
them in pageexec's slides on the topic:
https://pax.grsecurity.net/docs/PaXTeam-H2HC12-PaX-kernel-self-protection.pdf

> doing them already, and how can we fix that higher-order issue?

They are each complex, and no one has the interest, skills, tenacity,
and work-priority available to them yet. I continue to chip away at
pieces here and there, but I'm just one person, and I'm probably
lacking in skill too.

> Is it because adding a particular feature would incur a huge
> performance penalty?

Some do, yes. Most don't. Mostly it's just the will to fight for the
changes. I think the concepts behind grsec/PaX's KERNEXEC would be a
great first target. Many other features depend on being able to trust
various aspects of kernel's resulting memory footprint.

> Is it because no company has been willing to fund developers to work
> on that particular feature to date?  (BTW, I consider the fact that

Yes, that's a big part of it. Also, the political navigation of the
feature is time-consuming. But I've been trying to socialize these
ideas for a while now, so they shouldn't be too much of a shock to
anyone. The concept and benefits of defensive features (as opposed to
bug killing) has been traditionally difficult to get people to see.

> various companies collectively wasn't able to find a place for the
> trinity maintainer to find a place to land to be somewhat of a failure
> of the ecosystem, but maybe the tool wasn't as useful as we think, or
> it maybe we failed to make the case to the correct set of
> bean-counters.)

Agreed. Though AIUI, loss of trinity maintenance had more to do with
people not contributing back to it. He should speak for himself,
though. :)

> If the answer is that it's obvious what needs to be done, but (a) we
> can't find anyone to bell the cat, or (b) the patches are going to be
> rejected out of hand for one reason or another, the kernel summit is a
> great opportunity to see if some face-to-face discussion address the
> problem.  OTOH, if the fundamental problem is that we can't get the
> headcount funded, then discussion at the kernel summit is probably not
> going to be a good use of our time.  :-/

Understanding the value of self-protection technologies will go a long
way to accepting them when someone tries to present them for upstream.
Accepting the limitations they present (e.g. gcc plugins may only work
with certain compiler versions, etc) can be a hard pill to swallow,
but I think it's best for the end users.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Rafael J. Wysocki]

On Monday, August 24, 2015 05:05:25 PM Greg KH wrote:
> On Mon, Aug 24, 2015 at 12:00:15PM -0700, Kees Cook wrote:
> > On Mon, Aug 24, 2015 at 11:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> > > On Mon, 24 Aug 2015, Kees Cook wrote:
> > >> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
> > >> This is far from a comprehensive list, though. The biggest value, I
> > >> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
> > >> for constification and integer overflow.
> > >
> > > There is another aspect. We need to make developers more aware of the
> > > potential attack issues. I learned my lesson with the futex disaster
> > > and since then I certainly look with a different set of eyes at user
> > > space facing code. I doubt that we want that everyone experiences the
> > > disaster himself (though that's a very enlightening experience), but
> > > we should try to document incidents and the lessons learned from
> > > them. Right now we just rely on those who are deep into the security
> > > realm or the few people who learned it the hard way.
> > 
> > Yeah, it can be a hard perspective shift to make. And shifting the
> > thinking about the kernel itself to always operating in a compromised
> > state makes thinking about how to protect it much easier. User space
> > is trying to hurt us! :)
> 
> Microsoft's security team, which was responsible for forcing all of
> their developers to undergo some security training every year, has
> boiled it all down to these simple 4 words:
> 
> 	All input is evil.

Right.

And not just input, but also everything you created and then allowed
someone else to modify.

Thanks,
Rafael

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Shuah Khan]

On Mon, Aug 24, 2015 at 10:35 AM, Kees Cook <keescook@chromium.org> wrote:
> On Mon, Aug 24, 2015 at 5:29 AM, Linus Walleij <linus.walleij@linaro.org> wrote:
>> On Mon, Aug 24, 2015 at 6:20 AM, James Morris <jmorris@namei.org> wrote:
>>
>>> There are also potentially promising approaches to mitigation with other
>>> technologies such as KASan and gcc plugins, as well as evolving hardware
>>> features.
>>
>> What I've discovered when running KASan the last few weeks is that
>> this points back to the question of tests ... I've been using Trinity
>> to find bugs, but it is more likely to kill itself or cause OOM than
>> trigger any boundary overrun bugs.
>>
>> Kselftest may be helpful, but basically any loads that heavily
>> exercise the kernel internals are helpful to harden the kernel.
>> Some of these are custom test suites I suspect. Any good hints
>> for a simple embedded developer like me?
>
> I agree with the sentiment here, but not with the language. Finding
> flaws (which is what selftests, KASan, Trinity, etc do) isn't
> hardening. Hardening is stopping the exploitation of flaws. The
> hardening the kernel needs is about taking away exploitation tools,
> not killing bugs. (Though killing bugs is still great.)

I agree with Kees on this. Kselftest or any other test suites can help
with regression testing and make sure Kernel works the way it should.
Also these tests can tell us if kernel is hardened or not.

Hardening means something different to me. i.e making sure kernel
can protect against attacks and fail gracefully. This is something to
address during design and development process.

>
> As an example, making the kernel code memory read-only means an
> attacker cannot just directly change the kernel's execution path when
> they use an arbitrary memory-writing flaw. (This feature is mostly
> enabled via CONFIG_DEBUG_RODATA, and was very recently added to ARM,
> though isn't at 100% coverage for all the physical memory aliases.)
>

This sounds similar to ExecShield (NX bit) on Intel. Yes this is a good example.

thanks,
-- Shuah

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Tue, Aug 25, 2015 at 8:15 AM, Shuah Khan <shuahkhan@gmail.com> wrote:
> On Mon, Aug 24, 2015 at 10:35 AM, Kees Cook <keescook@chromium.org> wrote:
>> As an example, making the kernel code memory read-only means an
>> attacker cannot just directly change the kernel's execution path when
>> they use an arbitrary memory-writing flaw. (This feature is mostly
>> enabled via CONFIG_DEBUG_RODATA, and was very recently added to ARM,
>> though isn't at 100% coverage for all the physical memory aliases.)
>>
>
> This sounds similar to ExecShield (NX bit) on Intel. Yes this is a good example.

Yup! That's exactly the NX bit (or other architecture equivalent). The
trouble tends to be around correctly setting up the kernel memory maps
to actually split up regions and mark the permissions correctly, and
make sure nothing was depending on the side-effects of the old
permissions.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Mark Brown]

[-- Attachment #1: Type: text/plain, Size: 1086 bytes --]

On Tue, Aug 25, 2015 at 09:15:32AM -0600, Shuah Khan wrote:
> On Mon, Aug 24, 2015 at 10:35 AM, Kees Cook <keescook@chromium.org> wrote:

> > I agree with the sentiment here, but not with the language. Finding
> > flaws (which is what selftests, KASan, Trinity, etc do) isn't
> > hardening. Hardening is stopping the exploitation of flaws. The
> > hardening the kernel needs is about taking away exploitation tools,
> > not killing bugs. (Though killing bugs is still great.)

> I agree with Kees on this. Kselftest or any other test suites can help
> with regression testing and make sure Kernel works the way it should.
> Also these tests can tell us if kernel is hardened or not.

> Hardening means something different to me. i.e making sure kernel
> can protect against attacks and fail gracefully. This is something to
> address during design and development process.

Testsuites can help here if we get into the habit of making sure they
exercise error conditions; they're off to the side a bit but they can
be a useful way of promoting good practice (at least in my experience).

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Tue, Aug 25, 2015 at 9:30 AM, Mark Brown <broonie@kernel.org> wrote:
> On Tue, Aug 25, 2015 at 09:15:32AM -0600, Shuah Khan wrote:
>> On Mon, Aug 24, 2015 at 10:35 AM, Kees Cook <keescook@chromium.org> wrote:
>
>> > I agree with the sentiment here, but not with the language. Finding
>> > flaws (which is what selftests, KASan, Trinity, etc do) isn't
>> > hardening. Hardening is stopping the exploitation of flaws. The
>> > hardening the kernel needs is about taking away exploitation tools,
>> > not killing bugs. (Though killing bugs is still great.)
>
>> I agree with Kees on this. Kselftest or any other test suites can help
>> with regression testing and make sure Kernel works the way it should.
>> Also these tests can tell us if kernel is hardened or not.
>
>> Hardening means something different to me. i.e making sure kernel
>> can protect against attacks and fail gracefully. This is something to
>> address during design and development process.
>
> Testsuites can help here if we get into the habit of making sure they
> exercise error conditions; they're off to the side a bit but they can
> be a useful way of promoting good practice (at least in my experience).

Yeah, this is what I've done with a bunch of the newer tests in the
lkdtm module. They're designed to Oops the machine by performing
actions that should be caught by various mitigations (e.g. writing to
kernel text, executing userspace memory from the kernel, etc).

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Luis R. Rodriguez]

On Mon, Aug 24, 2015 at 10:46:33PM +0200, Thomas Gleixner wrote:
> While we certainly want to add mechanisms which prevent flaws to be
> exploited we surely want to do something about educating people how to
> avoid the flaws in the first place.

<-- snip -->

> I totally agree that we cannot prevent all flaws, but we certainly can
> do better in reducing the quantity. And that means that we need to
> educate people. And that education involve documentation and clever usage of
> tools.

What folks have been discussing mostly after this post are reactive security
solutions, that's fine but I think we need to also be a bit more proactive
about what we let folks design in terms of *new* userspace APIs. For instance
although it would not solve all userspace facing issues, I certainly think
things like struct nla_policy helps move away from loose userspace APIs.
The next evolutionary step was generic netlink but that may be too tied down
to networking.

  Luis

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Shuah Khan]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 08/25/2015 10:30 AM, Mark Brown wrote:
> On Tue, Aug 25, 2015 at 09:15:32AM -0600, Shuah Khan wrote:
>> On Mon, Aug 24, 2015 at 10:35 AM, Kees Cook
>> <keescook@chromium.org> wrote:
> 
>>> I agree with the sentiment here, but not with the language.
>>> Finding flaws (which is what selftests, KASan, Trinity, etc do)
>>> isn't hardening. Hardening is stopping the exploitation of
>>> flaws. The hardening the kernel needs is about taking away
>>> exploitation tools, not killing bugs. (Though killing bugs is
>>> still great.)
> 
>> I agree with Kees on this. Kselftest or any other test suites can
>> help with regression testing and make sure Kernel works the way
>> it should. Also these tests can tell us if kernel is hardened or
>> not.
> 
>> Hardening means something different to me. i.e making sure
>> kernel can protect against attacks and fail gracefully. This is
>> something to address during design and development process.
> 
> Testsuites can help here if we get into the habit of making sure
> they exercise error conditions; they're off to the side a bit but
> they can be a useful way of promoting good practice (at least in my
> experience).
> 

Yes. I agree. We can expand the Kselftest suite to do error checking,
maybe error injection. There are several tests now that are focused
on error checking and make sure syscalls fail correctly when used
incorrectly etc.

We could look into enhancing existing tests and add new tests as needed
to focus on finding kernel hardening related bugs and continue use them
for regression testing.

thanks,
- -- Shuah

- -- 
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBAgAGBQJV3J7LAAoJEAsCRMQNDUMcT1sQANiUpiZuYMjEtUFAkaxjEWGq
wI8xv69csf25ZClnOnjya3ALFksiIFbkdi7n4NWe6okKOgACpe5T0muVnSGbd6bz
ynw8zrBtCn0Kv9x+ee805wZ+VE5LtZe1bLqmhEIV0pDR+9wfn+BEmi2wsdeorW/R
tdE1REiyD9EThkfR6A15pVutD1lSPAxoPN1CfizO4VL/MSqa+/whU9V0Nrsj4CLl
MD9a4TP1TKgDdSFWWO6Gmdlh4cqXcJtXvGlnMxh3CZOuXolKMvye3Zgp9eyCp8HD
lHdJvbeON2z9lugFFfhF/iNvToHAuce3ArdlrlEcOJMDWBQ5ndUgGDBGYHNgWdGo
JoUKMJkI3g46ZsA7jfBJZ/G5wkH6k3b0Rwj+RPI2uPmu43TU80At3GwdiLq+aYGR
yp1p9ZuwrvtzL2sZOlPrj/uV1ZnIBj//fOuJWG0bJbsPYsnXe2vkZ4lQ/MsAe+sA
WKBYx8EO1YWCHbbsf37Vs1q8GRSUYuoOR2x7fUnvQlGO+QRfpb8ZcboMC7L0eC0C
plujBslA8E155TVUel9nreBK0WBWjynWOVlMf/GevCzWpiVirowlbKsOzttVHVCE
FczNYNd+CAzqxDfTrtzrjX3g5pEfjdmUgT4+6gwieG2AnilupV3yRvSW0e1Gqkii
R2LPXaZ0cMKJQ+2GYdD9
=kpAC
-----END PGP SIGNATURE-----

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 24, 2015 at 10:17 AM, Kees Cook <keescook@chromium.org> wrote:
> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>> On Mon, 24 Aug 2015, Jiri Kosina wrote:
>>
>>> On Mon, 24 Aug 2015, James Morris wrote:
>>>
>>> > I'd recommend Kees Cook be involved, due to his existing efforts in
>>> > kernel hardening.  I think it would be good to invite one or two expert
>>> > security researchers in this area -- Kees would know who.  In terms of
>
> Many of the folks that are good at kernel exploitation don't want to
> help us fix the situation. :)
>
> I'd recommend Lee Campbell, he's got a solid bit of experience from
> the offense side. I think we should extend an invite to spender and
> pageexec as well. They've been on the cutting edge of this for
> decades, and it would be silly not to invite them.
>
>>> > core kernel folk, I'd suggest Ingo and akpm, as a starting point.
>
> Perhaps also Linus and rmk? Some of the protections are very central
> to the kernel (e.g. constification, "read-mostly", segmentation
> through page table swaps or domains, etc). I'd also want Andy
> Lutomirski around, as he's got a lot of deep chipset knowledge. :)

I think another valuable developer to invite would be Matthew Garrett.
He's been looking at hardening the line between root and kernel for a
while now.

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Matthew Garrett]

(Resending now this address is subscribed)

On Wed, Aug 26, 2015 at 1:51 PM, Kees Cook <keescook@chromium.org> wrote:
> I think another valuable developer to invite would be Matthew Garrett.
> He's been looking at hardening the line between root and kernel for a
> while now.

Yeah, we can't rely on userspace to avoid or mitigate all privilege
escalation bugs - avoiding userspace escalation inherently turning
into owning the kernel remains valuable. There's various approaches to
validating userspace state that are only achievable if the kernel is
still trustworthy, so the assumption that root = game over is one that
we shouldn't allow the kernel to embody.

[Ksummit-discuss] grsecurity and kernel hardening

[Stephen Hemminger]

Looks like grsecurity has had a turn for the worse
 http://www.theregister.co.uk/2015/08/27/grsecurity/

But can't say that I blame him for being mad about vendors
using without giving back...

Re: [Ksummit-discuss] grsecurity and kernel hardening

[James Bottomley]

On Thu, 2015-08-27 at 15:08 -0700, Stephen Hemminger wrote:
> Looks like grsecurity has had a turn for the worse
>  http://www.theregister.co.uk/2015/08/27/grsecurity/

We could add this to the legal discussion.  The above article reads like
a classic how not to do things right.  Trademarks aren't like copyrights
or patents, they're really tricky things to use successfully.  You can't
register them and only pull them out of a drawer when someone does
something you don't like.  For trademarks, you have to establish your
mark and the norms first.  This usually requires some sort of policy
statement, the most critical element of which is (especially if the mark
is actually part of the distributed open source base or a tag attached
to the patch set) what acceptable and non-acceptable use is.  In
particular, you can't allow people to use it all over the place and then
later turn around and decide X use is a trademark infringement because
you now decide you don't like it or whoever's using it hasn't paid you
any money.

The Iceweasel controversy notwithstanding, Mozilla seems to have about
the most effective Open Source trademark practises.  They certainly
demonstrate that a trademark gives you more control than you'd otherwise
expect over a permissively licensed project.

>  But can't say that I blame him for being mad about vendors
>  using without giving back...

We also have to be a little careful about this.  The strict requirements
are whatever the licence says and nothing more.  We like it when vendors
give back, either by becoming contributing members of the community or
sponsoring conferences, or even just giving developers cash, but we have
to acknowledge that they don't do it because they're required to, or
even because they feel morally obliged to ... they mostly do it because
they have sound business reasons for doing so.

Getting a vendor to see that they have sound business reasons for giving
back is a specialised skill usually practised over long time period and
only successfully by a few people in the industry.  It doesn't happen
automatically and certainly not because you have a dummy spit over
someone taking something you advertised as "for free" and having the
temerity to expect not to have to pay you.

Perhaps it would be useful to run a "how to get your company to regard
open source as an investment" type training course, but it's probable
that the kernel summit isn't the best venue.

James

Re: [Ksummit-discuss] grsecurity and kernel hardening

[Stephen Hemminger]

On Thu, 27 Aug 2015 15:49:52 -0700
James Bottomley <James.Bottomley@HansenPartnership.com> wrote:

> Getting a vendor to see that they have sound business reasons for giving
> back is a specialised skill usually practised over long time period and
> only successfully by a few people in the industry.  It doesn't happen
> automatically and certainly not because you have a dummy spit over
> someone taking something you advertised as "for free" and having the
> temerity to expect not to have to pay you.
> 
> Perhaps it would be useful to run a "how to get your company to regard
> open source as an investment" type training course, but it's probable
> that the kernel summit isn't the best venue.

Agreed, plus the KS is not necessarily going to be the right venue
for this. Kind of hard to preach to the converted.

[Ksummit-discuss] Self nomination

[Matthew Garrett]

Unsure whether Kees suggesting me is sufficient, so:

I'd like to be involved if we're going to have a meaningful discussion 
about more proactively mitigating attacks on the kernel. I've been 
working full-time in the security field for about the past three years, 
and it's certainly resulted in a fairly strong shift in attitude towards 
how well we're doing here. At this point even Microsoft are showing more 
aggressive security development than we are, and when the only person 
who appears to be doing meaningful work in adding mitigation features to 
the kernel is Kees, that's kind of a bad sign.

I've been doing development in this field for some time now - the secure 
boot patchset is an attempt to avoid allowing userspace privilege 
escalations to turn into persistent kernel compromises, for instance. As 
a member of the SFC's kernel copyright enforcement group, I'm also 
interested in being involved in any legal discussions that come up. Even 
if I don't make the invite list, I'd hope that these both end up as high 
priority topics.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Eric W. Biederman]

Greg KH <greg@kroah.com> writes:

> On Mon, Aug 24, 2015 at 12:00:15PM -0700, Kees Cook wrote:
>> On Mon, Aug 24, 2015 at 11:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
>> > On Mon, 24 Aug 2015, Kees Cook wrote:
>> >> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
>> >> This is far from a comprehensive list, though. The biggest value, I
>> >> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
>> >> for constification and integer overflow.
>> >
>> > There is another aspect. We need to make developers more aware of the
>> > potential attack issues. I learned my lesson with the futex disaster
>> > and since then I certainly look with a different set of eyes at user
>> > space facing code. I doubt that we want that everyone experiences the
>> > disaster himself (though that's a very enlightening experience), but
>> > we should try to document incidents and the lessons learned from
>> > them. Right now we just rely on those who are deep into the security
>> > realm or the few people who learned it the hard way.
>> 
>> Yeah, it can be a hard perspective shift to make. And shifting the
>> thinking about the kernel itself to always operating in a compromised
>> state makes thinking about how to protect it much easier. User space
>> is trying to hurt us! :)
>
> Microsoft's security team, which was responsible for forcing all of
> their developers to undergo some security training every year, has
> boiled it all down to these simple 4 words:
>
> 	All input is evil.

I have a linux version of that one.

An unprivileged user can call that.

It is absolutely amazing how much kernel code does not worry
about evil input after it checks that the caller is root.

Eric

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[josh]

On Mon, Aug 31, 2015 at 03:10:01PM -0500, Eric W. Biederman wrote:
> Greg KH <greg@kroah.com> writes:
> 
> > On Mon, Aug 24, 2015 at 12:00:15PM -0700, Kees Cook wrote:
> >> On Mon, Aug 24, 2015 at 11:52 AM, Thomas Gleixner <tglx@linutronix.de> wrote:
> >> > On Mon, 24 Aug 2015, Kees Cook wrote:
> >> >> On Mon, Aug 24, 2015 at 4:56 AM, James Morris <jmorris@namei.org> wrote:
> >> >> This is far from a comprehensive list, though. The biggest value, I
> >> >> think, would be in using KERNEXEC, UDEREF, USERCOPY, and the plugins
> >> >> for constification and integer overflow.
> >> >
> >> > There is another aspect. We need to make developers more aware of the
> >> > potential attack issues. I learned my lesson with the futex disaster
> >> > and since then I certainly look with a different set of eyes at user
> >> > space facing code. I doubt that we want that everyone experiences the
> >> > disaster himself (though that's a very enlightening experience), but
> >> > we should try to document incidents and the lessons learned from
> >> > them. Right now we just rely on those who are deep into the security
> >> > realm or the few people who learned it the hard way.
> >> 
> >> Yeah, it can be a hard perspective shift to make. And shifting the
> >> thinking about the kernel itself to always operating in a compromised
> >> state makes thinking about how to protect it much easier. User space
> >> is trying to hurt us! :)
> >
> > Microsoft's security team, which was responsible for forcing all of
> > their developers to undergo some security training every year, has
> > boiled it all down to these simple 4 words:
> >
> > 	All input is evil.
> 
> I have a linux version of that one.
> 
> An unprivileged user can call that.
> 
> It is absolutely amazing how much kernel code does not worry
> about evil input after it checks that the caller is root.

Another that has only started getting consideration recently: relative to the
kernel, root has less privilege.

- Josh Triplett

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Eric W. Biederman]

Kees Cook <keescook@chromium.org> writes:

> We are finding the bugs, and we can do better, but that's not what I
> think needs the most attention right now. We need to kill classes of
> bugs and classes of exploits. To kill a bug class, we must remove the
> possibility that it can ever go wrong in the first place. Many people
> stop here when thinking about hardening, but we must move on to
> killing classes of exploit. I'll continue to use my W^X kernel code
> example, since it was not, from an operational stance, a flaw that
> kernel code was writable. But it's a exploitation weakness. If an
> attacker just needs to directly target a kernel memory location with
> their memory-write primitive, all their work is done, that user loses
> control of their kernel, game over.
>
> We need to add the safety nets under the acrobats, since they can fall
> at any time.

I think it makes sense to close the classes of vulnerabilities that we
can.

At the same time I think we to serious consider tossing attempts that
fails to close a class of exploits.

The kernel address space randomization on x86_64 I find disturbing.  We
have a 2GB address space for kernel code.  We have pages that are 2MB in
that address space.  So we only have 10 bits that can change.  Only 9
bits that can change if the kernel needs more than one 2MB page.  Which
means that at most we need to brute force 1024 things to exploit any
weakness.

I don't see that attempt at kernel self protection actually
accomplishing anything in the way of protection, and I do see it costing
us debuggability which impacts kernel maintenance.  That is enabling
this protection seems to increase the effort to fix kernel bugs and as
such increases the number of bugs overall.

Is it reasonable to suggest when we have kernel security features that
only make people feel good, but don't actually protect them that we toss
the feature?

>> Yes, I like this one a lot.  Adding mechanisms that don't increase
>> complexity like this are good active means.  However, I become less
>> enamoured of things like selinux and grsecurity which add complexity in
>> the name of active attack surface reduction.  That's not to say never do
>> it, it's just to say that attack surface is directly related to
>> complexity.>

> FWIW, really only reduces userspace attack surface (all syscalls are
> still available to a confined process). seccomp reduces attack
> surface. grsecurity has an MAC component, but grsecurity (with PaX) is
> much larger. Regardless, some of these nets will increase complexity.
> It's the same for anti-lock brakes and airbags[1]. We have to take on
> this burden to protect our users from our mistakes.

Except given your reference[1] what we need to do is protect our users
from their own mistakes.  Which means we need to do more than just tell
our users no it is not ok to do that thing you want to do (they will do
it anyway).  We need to figure out safe ways to allow our users to do
the things they want or need to do.

This means things like not hiding new features behind CAP_SYS_ADMIN so
that we don't have to bother with securing kernel code.

This means things like figuring out how to make it possible for users to
mount that usb key they found in the parking lot and not have their
computer get owned.

All of which says that we need to increase the amount of the kernel
code that we are willing to defend from attacks, and figure out how to
defend that code.

Allowing our users to reduce the kernel attack surface is valid, but I
don't think for us as kernel developers it is valid to rely on users
reducing the kernel attack surface.

> -Kees
>
> [1] Borrowing mricon's great analogy as presented at the Security
> Summit this year:
> http://kernsec.org/files/lss2015/giant-bags-of-mostly-water.pdf

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Jiri Kosina]

On Mon, 31 Aug 2015, Eric W. Biederman wrote:

> At the same time I think we to serious consider tossing attempts that
> fails to close a class of exploits.
> 
> The kernel address space randomization on x86_64 I find disturbing.  We
> have a 2GB address space for kernel code.  We have pages that are 2MB in
> that address space.  So we only have 10 bits that can change.  Only 9
> bits that can change if the kernel needs more than one 2MB page.  Which
> means that at most we need to brute force 1024 things to exploit any
> weakness.

I understand what you are saying. OTOH, assuming that kernel doesn't leak 
addressess when restrict_kptr is set, it's rather likely that bruteforcing 
attempt to exploit the kernel is going to crash it instead of privilege 
escalation happenin (because the address wouldn't match, and either the 
kernel will #GP directly, or do some totally random garbage and crash 
shortly afterwards).

It's still *pretty* bad, but arguably better than being silently 
compromised without anyone noticing.

> I don't see that attempt at kernel self protection actually 
> accomplishing anything in the way of protection, and I do see it costing 
> us debuggability which impacts kernel maintenance.  

When kaslr appeared, my first reaction also was "this will be disaster for 
debugability and post-mortem crashdump analysis", but it turns out to work 
out pretty well.

What particular concerncs do you have in this respect?

-- 
Jiri Kosina
SUSE Labs

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Mon, Aug 31, 2015 at 1:58 PM, Eric W. Biederman
<ebiederm@xmission.com> wrote:
> Kees Cook <keescook@chromium.org> writes:
>
>> We are finding the bugs, and we can do better, but that's not what I
>> think needs the most attention right now. We need to kill classes of
>> bugs and classes of exploits. To kill a bug class, we must remove the
>> possibility that it can ever go wrong in the first place. Many people
>> stop here when thinking about hardening, but we must move on to
>> killing classes of exploit. I'll continue to use my W^X kernel code
>> example, since it was not, from an operational stance, a flaw that
>> kernel code was writable. But it's a exploitation weakness. If an
>> attacker just needs to directly target a kernel memory location with
>> their memory-write primitive, all their work is done, that user loses
>> control of their kernel, game over.
>>
>> We need to add the safety nets under the acrobats, since they can fall
>> at any time.
>
> I think it makes sense to close the classes of vulnerabilities that we
> can.

Absolutely, though I'm also suggesting we add proactive defenses. A
net isn't fixing a class of vulnerabilities in my bad analogy. A net
is proactive defense. Making the trapeze cables out of steel is a
vuln-fix.

> At the same time I think we to serious consider tossing attempts that
> fails to close a class of exploits.

I would agree if no one wanted them.

> The kernel address space randomization on x86_64 I find disturbing.  We
> have a 2GB address space for kernel code.  We have pages that are 2MB in
> that address space.  So we only have 10 bits that can change.  Only 9
> bits that can change if the kernel needs more than one 2MB page.  Which
> means that at most we need to brute force 1024 things to exploit any
> weakness.
>
> I don't see that attempt at kernel self protection actually
> accomplishing anything in the way of protection, and I do see it costing
> us debuggability which impacts kernel maintenance.  That is enabling
> this protection seems to increase the effort to fix kernel bugs and as
> such increases the number of bugs overall.

We'll have to agree to disagree. The kernel knows where its symbols
are, relocations are exported in crash dumps, etc. Everything needed
to debug a randomized base offset kernel is already there. For system
owners that run tight containers or remote services, kASLR provides a
real (if statistical) defense.

> Is it reasonable to suggest when we have kernel security features that
> only make people feel good, but don't actually protect them that we toss
> the feature?

I'm sure such a situation would be met with debate, like all kernel topics. :)

>>> Yes, I like this one a lot.  Adding mechanisms that don't increase
>>> complexity like this are good active means.  However, I become less
>>> enamoured of things like selinux and grsecurity which add complexity in
>>> the name of active attack surface reduction.  That's not to say never do
>>> it, it's just to say that attack surface is directly related to
>>> complexity.>
>
>> FWIW, really only reduces userspace attack surface (all syscalls are
>> still available to a confined process). seccomp reduces attack
>> surface. grsecurity has an MAC component, but grsecurity (with PaX) is
>> much larger. Regardless, some of these nets will increase complexity.
>> It's the same for anti-lock brakes and airbags[1]. We have to take on
>> this burden to protect our users from our mistakes.
>
> Except given your reference[1] what we need to do is protect our users
> from their own mistakes.  Which means we need to do more than just tell
> our users no it is not ok to do that thing you want to do (they will do
> it anyway).  We need to figure out safe ways to allow our users to do
> the things they want or need to do.
>
> This means things like not hiding new features behind CAP_SYS_ADMIN so
> that we don't have to bother with securing kernel code.

The kernel cannot be one-size-fits all. But we all have to play in the
same codebase, which means sometimes a feature is inherently opposed
to another. See all the interfaces that equate root with kernel access
(e.g. x86 msr module). These things are useful to those that need it
and a disaster to those that don't. We're always going to have the
play a balancing game and deal with these kinds of things on a
case-by-case basis. All that said, I agree: I don't like hiding stuff
behind CAP_SYS_ADMIN either.

> This means things like figuring out how to make it possible for users to
> mount that usb key they found in the parking lot and not have their
> computer get owned.

Well, that use-case is a whole different story. But it's a great
example of need to close bug classes, program defensively, add
self-protection features, etc.

> All of which says that we need to increase the amount of the kernel
> code that we are willing to defend from attacks, and figure out how to
> defend that code.

Sure, I totally agree. But in parallel to that is creating systems in
the kernel that stop exploitation methods (that exist regardless of
the flaw used as the primitive). There is a difference between killing
bug classes and killing exploitation methods. Our kernel has way too
few of the latter.

> Allowing our users to reduce the kernel attack surface is valid, but I
> don't think for us as kernel developers it is valid to rely on users
> reducing the kernel attack surface.

Totally agreed. Distro kernels are a good example, since they DO need
a one-size-fits-all config. :(

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Kees Cook]

On Tue, Sep 1, 2015 at 2:03 AM, Jiri Kosina <jikos@kernel.org> wrote:
> On Mon, 31 Aug 2015, Eric W. Biederman wrote:
>
>> At the same time I think we to serious consider tossing attempts that
>> fails to close a class of exploits.
>>
>> The kernel address space randomization on x86_64 I find disturbing.  We
>> have a 2GB address space for kernel code.  We have pages that are 2MB in
>> that address space.  So we only have 10 bits that can change.  Only 9
>> bits that can change if the kernel needs more than one 2MB page.  Which
>> means that at most we need to brute force 1024 things to exploit any
>> weakness.
>
> I understand what you are saying. OTOH, assuming that kernel doesn't leak
> addressess when restrict_kptr is set, it's rather likely that bruteforcing
> attempt to exploit the kernel is going to crash it instead of privilege
> escalation happenin (because the address wouldn't match, and either the
> kernel will #GP directly, or do some totally random garbage and crash
> shortly afterwards).
>
> It's still *pretty* bad, but arguably better than being silently
> compromised without anyone noticing.

This reminds me: we must assume all users on a system (and all devices
being attached) have access to a write-anywhere primitive (since they
DO -- we just haven't found them all yet, and we never will), what can
we do to close the windows of exploitability? (e.g. making memory
read-only, adding CFI to defend against ROP via the stack, etc?)

-Kees

-- 
Kees Cook
Chrome OS Security

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Dan Carpenter]

On Mon, Aug 24, 2015 at 09:35:50AM -0700, Kees Cook wrote:
> As an example, making the kernel code memory read-only means an
> attacker cannot just directly change the kernel's execution path when
> they use an arbitrary memory-writing flaw.

Part of the problem is that C defaults to writable and you have to
explicitly add a const if you want it read only.  Then when you are
reading the code and you see:

static struct uvc_format_desc uvc_fmts[] = {

You can't immediately tell if it is writeable because it needs to be or
because that's just the default.  We should make a new keyword that
means non constant then we could grep for structs which have a function
pointer and complain if they weren't specified as const or __writable.

Maybe there is a way to do this with GCC attributes but I have never
used them before and I couldn't immediately see how to do it.

regards,
dan carpenter

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Yves-Alexis Perez]

[-- Attachment #1: Type: text/plain, Size: 1425 bytes --]

On mar., 2015-09-22 at 15:24 +0300, Dan Carpenter wrote:
> On Mon, Aug 24, 2015 at 09:35:50AM -0700, Kees Cook wrote:
> > As an example, making the kernel code memory read-only means an
> > attacker cannot just directly change the kernel's execution path when
> > they use an arbitrary memory-writing flaw.
> 
> Part of the problem is that C defaults to writable and you have to
> explicitly add a const if you want it read only.  Then when you are
> reading the code and you see:
> 
> static struct uvc_format_desc uvc_fmts[] = {
> 
> You can't immediately tell if it is writeable because it needs to be or
> because that's just the default.  We should make a new keyword that
> means non constant then we could grep for structs which have a function
> pointer and complain if they weren't specified as const or __writable.
> 
> Maybe there is a way to do this with GCC attributes but I have never
> used them before and I couldn't immediately see how to do it.

If you look at the constify plugin (part of PaX), you'll see it uses a
gcc plugin to do something like this for structure containing only
function pointers. Historically it was done manually by adding “const”
where needed.

Now with the gcc plugin it's the opposite. All structure like these are
const by default, and a no_const attribute has been added for some of
them (around a hundred afair).

Regards,
-- 
Yves-Alexis


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Julia Lawall]

On Tue, 22 Sep 2015, Dan Carpenter wrote:

> On Mon, Aug 24, 2015 at 09:35:50AM -0700, Kees Cook wrote:
> > As an example, making the kernel code memory read-only means an
> > attacker cannot just directly change the kernel's execution path when
> > they use an arbitrary memory-writing flaw.
>
> Part of the problem is that C defaults to writable and you have to
> explicitly add a const if you want it read only.  Then when you are
> reading the code and you see:
>
> static struct uvc_format_desc uvc_fmts[] = {
>
> You can't immediately tell if it is writeable because it needs to be or
> because that's just the default.  We should make a new keyword that
> means non constant then we could grep for structs which have a function
> pointer and complain if they weren't specified as const or __writable.
>
> Maybe there is a way to do this with GCC attributes but I have never
> used them before and I couldn't immediately see how to do it.

But the bigger problem is that some of these kinds of structs have a
single writeable field lurking inside fo them.  For example,
platform_driver contains struct device_driver driver; that is initialized
in the platform driver library, and not in the device driver itself.
Would it be beneficial to put const on the individual function-typed
fields, if that is indeed possible?  For example, I don't find any
assignments to a platform_driver remove field.

julia

Re: [Ksummit-discuss] [TECH TOPIC] Kernel Hardening

[Andy Lutomirski]

On Tue, Sep 22, 2015 at 5:59 AM, Julia Lawall <julia.lawall@lip6.fr> wrote:
>
>
> On Tue, 22 Sep 2015, Dan Carpenter wrote:
>
>> On Mon, Aug 24, 2015 at 09:35:50AM -0700, Kees Cook wrote:
>> > As an example, making the kernel code memory read-only means an
>> > attacker cannot just directly change the kernel's execution path when
>> > they use an arbitrary memory-writing flaw.
>>
>> Part of the problem is that C defaults to writable and you have to
>> explicitly add a const if you want it read only.  Then when you are
>> reading the code and you see:
>>
>> static struct uvc_format_desc uvc_fmts[] = {
>>
>> You can't immediately tell if it is writeable because it needs to be or
>> because that's just the default.  We should make a new keyword that
>> means non constant then we could grep for structs which have a function
>> pointer and complain if they weren't specified as const or __writable.
>>
>> Maybe there is a way to do this with GCC attributes but I have never
>> used them before and I couldn't immediately see how to do it.
>
> But the bigger problem is that some of these kinds of structs have a
> single writeable field lurking inside fo them.  For example,
> platform_driver contains struct device_driver driver; that is initialized
> in the platform driver library, and not in the device driver itself.
> Would it be beneficial to put const on the individual function-typed
> fields, if that is indeed possible?  For example, I don't find any
> assignments to a platform_driver remove field.
>

PaX gets around this with special tricks to write to fields in
otherwise const structures in appropriately annotated code.  We could
clean that up and do it upstream.

--Andy