[kernel-hardening] Proposal for kernel self protection features

[kernel-hardening] Proposal for kernel self protection features

[Emese Revfy]

Hi,

My name is Emese Revfy, the author of the size_overflow, initify gcc plugin
and the original creator of the constify gcc plugin in grsecurity/PaX.

I'm very happy about this Kernel Self Protection Project and
I'd like to take part in it. I can do the following tasks:
 * gcc plugin infrastructure
 * size_overflow: Briefly about the plugin, it recomputes expressions
    that are marked by a size_overflow attribute or which are in the hash table
    with double integer precision. The recomputed expression is checked against
    TYPE_MIN/TYPE_MAX in the end and during some casts.
    It needs three hash tables:
    * size_overflow_hash: This hash table stores functions, function pointers,
       structure fields and variable declarations for the size_overflow
       duplication. It must be regenerated for every kernel version.
    * disable_size_overflow_hash: We ignore (don't instrument) the functions
       in this hash table.
    * size_overflow_hash_aux: For out-of-tree modules.
    The non-public version of the plugin also supports LTO so when the kernel
    will support LTO then hash tables will no longer be needed.
    There are three kinds of false positives that are caused by:
       * the developers: some code uses integer overflow for no reason.
          These can be fixed in the source code.
       * the developers intentionally: usually in the net part of the
          kernel (counters etc.). We use an gcc attribute (intentional_overflow)
          to ignore these functions.
       * gcc intentional overflow: gcc computes some expressions by overflow
          when it optimizes. Sadly it is doing this in the front end where
          there is no plugin support. Most of these false positives I handle
          from the plugin or sometimes I patch the kernel source code.
          There are some unsolved issues.
    You can read more about the plugin:
    https://forums.grsecurity.net/viewtopic.php?f=7&t=3043
    This is a complex plugin and it has false positives but it also found
    a lot of bugs (overflows, underflows and trunctions) and
    thanks to the data flow duplication it has a small performance impact only.
    I know that I will have to adapt it to be suitable for vanilla.
 * constify: This plugin constifies all structure types which contain only
    function pointers or are explicitly marked for constification.
    If some code wants to modify a read-only object it will cause
    a compile error.
    There are two gcc attributes:
       * no_const: if we don't want constification
       * do_const: if we want to constify a structure type which has a
          non-function pointer field
 * latent_entropy: This plugin generates some entropy from program state
    throughout the uptime of the kernel. It has small performance loss.
    The plugin uses an attribute which can be
    on a function (to extract entropy beyond init functions) or on a
    variable (to initialize it with a random number generated at compile time)
 * structleak: This plugin forcibly initializes certain local variables
    based on the __user attribute that could otherwise leak kernel stack
    to userland if they aren't properly initialized by later code.
    It has small performance loss.
 * colorize: This plugin is just for fun, it isn't a security related plugin.
    If the gcc version doesn't support colors (before gcc-4.9) then
    it colorizes the compiler output.
 * initify: This plugin isn't security related either.
    It moves string constants (__func__ and function string arguments
    marked by the nocapture attribute) only referenced in
    __init/__exit functions to __initconst/__exitconst sections.
    It reduces memory usage (many kB), I think it may be important for
    embedded systems.

I think the first plugin can be a simpler plugin e.g., constify or latent_entry.
Please let me know if you want one, more or all of them.

--
Emese

[kernel-hardening] Re: Proposal for kernel self protection features

[Kees Cook]

On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
> Hi,
>
> My name is Emese Revfy, the author of the size_overflow, initify gcc plugin
> and the original creator of the constify gcc plugin in grsecurity/PaX.
>
> I'm very happy about this Kernel Self Protection Project and
> I'd like to take part in it.

Welcome! Your participation would be extremely valuable. :)

> I can do the following tasks:
>  * gcc plugin infrastructure
>  * size_overflow: Briefly about the plugin, it recomputes expressions
>     that are marked by a size_overflow attribute or which are in the hash table
>     with double integer precision. The recomputed expression is checked against
>     TYPE_MIN/TYPE_MAX in the end and during some casts.
>     It needs three hash tables:
>     * size_overflow_hash: This hash table stores functions, function pointers,
>        structure fields and variable declarations for the size_overflow
>        duplication. It must be regenerated for every kernel version.
>     * disable_size_overflow_hash: We ignore (don't instrument) the functions
>        in this hash table.
>     * size_overflow_hash_aux: For out-of-tree modules.

By out-of-tree, you mean built-out-of-tree? I assume a future
downstream kernel that used size_overflow would have their
non-upstream code covered during the size_overflow_hash regeneration?
(Though they'd be on the their own for handling the false positives.)

>     The non-public version of the plugin also supports LTO so when the kernel
>     will support LTO then hash tables will no longer be needed.
>     There are three kinds of false positives that are caused by:
>        * the developers: some code uses integer overflow for no reason.
>           These can be fixed in the source code.
>        * the developers intentionally: usually in the net part of the
>           kernel (counters etc.). We use an gcc attribute (intentional_overflow)
>           to ignore these functions.
>        * gcc intentional overflow: gcc computes some expressions by overflow
>           when it optimizes. Sadly it is doing this in the front end where
>           there is no plugin support. Most of these false positives I handle
>           from the plugin or sometimes I patch the kernel source code.
>           There are some unsolved issues.

Has there been any discussion with gcc folks about this problem?

>     You can read more about the plugin:
>     https://forums.grsecurity.net/viewtopic.php?f=7&t=3043
>     This is a complex plugin and it has false positives but it also found
>     a lot of bugs (overflows, underflows and trunctions) and
>     thanks to the data flow duplication it has a small performance impact only.
>     I know that I will have to adapt it to be suitable for vanilla.

That's a great write-up! I would call other people's attention to the
portion where you point out it blocks real-world cases of flaws:
CVE-2013-0914 CVE-2013-0913 CVE-2013-2141

>  * constify: This plugin constifies all structure types which contain only
>     function pointers or are explicitly marked for constification.
>     If some code wants to modify a read-only object it will cause
>     a compile error.
>     There are two gcc attributes:
>        * no_const: if we don't want constification
>        * do_const: if we want to constify a structure type which has a
>           non-function pointer field

I assume some level of no_const marking could go away when KERNEXEC or
at least a similar "temporary writing" ability lands in upstream?

>  * latent_entropy: This plugin generates some entropy from program state
>     throughout the uptime of the kernel. It has small performance loss.
>     The plugin uses an attribute which can be
>     on a function (to extract entropy beyond init functions) or on a
>     variable (to initialize it with a random number generated at compile time)

How large is the entropy extracted during early init? I bet a lot of
architectures would rejoice to have this available. (CCing Ted for
thoughts.)

>  * structleak: This plugin forcibly initializes certain local variables
>     based on the __user attribute that could otherwise leak kernel stack
>     to userland if they aren't properly initialized by later code.
>     It has small performance loss.

Does this operate only on kernel stack structures?

>  * colorize: This plugin is just for fun, it isn't a security related plugin.
>     If the gcc version doesn't support colors (before gcc-4.9) then
>     it colorizes the compiler output.

Hah, nice. :)

>  * initify: This plugin isn't security related either.
>     It moves string constants (__func__ and function string arguments
>     marked by the nocapture attribute) only referenced in
>     __init/__exit functions to __initconst/__exitconst sections.
>     It reduces memory usage (many kB), I think it may be important for
>     embedded systems.

I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
be interested in this! (CCing Josh for thoughts.)

> I think the first plugin can be a simpler plugin e.g., constify or latent_entropy.
> Please let me know if you want one, more or all of them.

Well I would love to see all of them upstream. :) As for first steps,
you've convinced me about the complexity of the size_overflow plugin.
Perhaps the constify plugin would be a good first target? Can you
speak to any known bugs it stopped? Having some mention of the threat
it mitigates would be helpful. (Do I remember correctly that it
constified security_operations, which was a common target in
exploits?)

Thanks very much!

-Kees

-- 
Kees Cook
Chrome OS Security

[kernel-hardening] Re: Proposal for kernel self protection features

[Josh Triplett]

On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
> On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
> >  * initify: This plugin isn't security related either.
> >     It moves string constants (__func__ and function string arguments
> >     marked by the nocapture attribute) only referenced in
> >     __init/__exit functions to __initconst/__exitconst sections.
> >     It reduces memory usage (many kB), I think it may be important for
> >     embedded systems.
> 
> I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
> be interested in this! (CCing Josh for thoughts.)

I'd be quite interested.

Could the plugin operate in a mode where it emits warnings to add such
annotations explicitly in the code, rather than just automatically
moving the data?

- Josh Triplett

[kernel-hardening] Re: Proposal for kernel self protection features

[Greg KH]

On Fri, Nov 06, 2015 at 04:25:08PM -0800, Josh Triplett wrote:
> On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
> > On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
> > >  * initify: This plugin isn't security related either.
> > >     It moves string constants (__func__ and function string arguments
> > >     marked by the nocapture attribute) only referenced in
> > >     __init/__exit functions to __initconst/__exitconst sections.
> > >     It reduces memory usage (many kB), I think it may be important for
> > >     embedded systems.
> > 
> > I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
> > be interested in this! (CCing Josh for thoughts.)
> 
> I'd be quite interested.
> 
> Could the plugin operate in a mode where it emits warnings to add such
> annotations explicitly in the code, rather than just automatically
> moving the data?

That would be nice for the constanfy mode as well, especially as some
people aren't using gcc to build the kernel anymore, so it would be good
to mark these "for real" in the .c code wherever possible to allow other
compilers to take advantage of the plugin indirectly.

thanks,

greg k-h

[kernel-hardening] Re: Proposal for kernel self protection features

[Kees Cook]

On Fri, Nov 6, 2015 at 6:46 PM, Greg KH <gregkh@linuxfoundation.org> wrote:
> On Fri, Nov 06, 2015 at 04:25:08PM -0800, Josh Triplett wrote:
>> On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
>> > On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
>> > >  * initify: This plugin isn't security related either.
>> > >     It moves string constants (__func__ and function string arguments
>> > >     marked by the nocapture attribute) only referenced in
>> > >     __init/__exit functions to __initconst/__exitconst sections.
>> > >     It reduces memory usage (many kB), I think it may be important for
>> > >     embedded systems.
>> >
>> > I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
>> > be interested in this! (CCing Josh for thoughts.)
>>
>> I'd be quite interested.
>>
>> Could the plugin operate in a mode where it emits warnings to add such
>> annotations explicitly in the code, rather than just automatically
>> moving the data?
>
> That would be nice for the constanfy mode as well, especially as some
> people aren't using gcc to build the kernel anymore, so it would be good
> to mark these "for real" in the .c code wherever possible to allow other
> compilers to take advantage of the plugin indirectly.

Yeah, I think both modes have value, but I want to make sure we keep
in mind that gaining these plugins in like using the stack-protector
flag in gcc: the build results will cover non-upstream code too. I
want to be sure that our many many downstream users of the kernel will
still gain the benefits (i.e. it's less useful to Linux everywhere in
the world if only the upstream code has been fixed).

-Kees

-- 
Kees Cook
Chrome OS Security

[kernel-hardening] Re: Proposal for kernel self protection features

[Josh Triplett]

On Fri, Nov 06, 2015 at 08:16:59PM -0800, Kees Cook wrote:
> On Fri, Nov 6, 2015 at 6:46 PM, Greg KH <gregkh@linuxfoundation.org> wrote:
> > On Fri, Nov 06, 2015 at 04:25:08PM -0800, Josh Triplett wrote:
> >> On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
> >> > On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
> >> > >  * initify: This plugin isn't security related either.
> >> > >     It moves string constants (__func__ and function string arguments
> >> > >     marked by the nocapture attribute) only referenced in
> >> > >     __init/__exit functions to __initconst/__exitconst sections.
> >> > >     It reduces memory usage (many kB), I think it may be important for
> >> > >     embedded systems.
> >> >
> >> > I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
> >> > be interested in this! (CCing Josh for thoughts.)
> >>
> >> I'd be quite interested.
> >>
> >> Could the plugin operate in a mode where it emits warnings to add such
> >> annotations explicitly in the code, rather than just automatically
> >> moving the data?
> >
> > That would be nice for the constanfy mode as well, especially as some
> > people aren't using gcc to build the kernel anymore, so it would be good
> > to mark these "for real" in the .c code wherever possible to allow other
> > compilers to take advantage of the plugin indirectly.
> 
> Yeah, I think both modes have value, but I want to make sure we keep
> in mind that gaining these plugins in like using the stack-protector
> flag in gcc: the build results will cover non-upstream code too. I
> want to be sure that our many many downstream users of the kernel will
> still gain the benefits (i.e. it's less useful to Linux everywhere in
> the world if only the upstream code has been fixed).

I agree in both cases: having the plugin usable in "make it so" mode for
the benefit of legacy or out-of-tree code, and having it usable in
"suggest changes to the source" (or outright *edit* the source and
produce a patch) mode to avoid actually mandating the plugin.  Not least
of which because I'd find it surprising if the plugin ever worked across
as broad a range of GCC versions as the kernel typically wants to
support.

(For that matter, as the LLVM Linux project progresses, it might
actually prove easier to provide a clang-based plugin.)

- Josh Triplett

[kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

> I agree in both cases: having the plugin usable in "make it so" mode for
> the benefit of legacy or out-of-tree code, and having it usable in
> "suggest changes to the source" (or outright *edit* the source and
> produce a patch) mode to avoid actually mandating the plugin.  Not least
> of which because I'd find it surprising if the plugin ever worked across
> as broad a range of GCC versions as the kernel typically wants to
> support.

All gcc plugins in PaX support all plugin capable gcc versions (4.5-5). 
This is PaXTeam's requirement if somebody writes a plugin for PaX.
And of course the plugin infrastructure handles gcc versions that don't
support plugins.

-- 
Emese

[kernel-hardening] Re: Proposal for kernel self protection features

[Josh Triplett]

On Sat, Nov 07, 2015 at 11:07:02PM +0100, Emese Revfy wrote:
> 
> > I agree in both cases: having the plugin usable in "make it so" mode for
> > the benefit of legacy or out-of-tree code, and having it usable in
> > "suggest changes to the source" (or outright *edit* the source and
> > produce a patch) mode to avoid actually mandating the plugin.  Not least
> > of which because I'd find it surprising if the plugin ever worked across
> > as broad a range of GCC versions as the kernel typically wants to
> > support.
> 
> All gcc plugins in PaX support all plugin capable gcc versions (4.5-5). 

The kernel supports older GCC than that, though.

> And of course the plugin infrastructure handles gcc versions that don't
> support plugins.

...huh?  How does *that* work?

[kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

On Sun, 8 Nov 2015 00:21:55 -0800
Josh Triplett <josh@joshtriplett.org> wrote:

> On Sat, Nov 07, 2015 at 11:07:02PM +0100, Emese Revfy wrote:
> > 
> > > I agree in both cases: having the plugin usable in "make it so" mode for
> > > the benefit of legacy or out-of-tree code, and having it usable in
> > > "suggest changes to the source" (or outright *edit* the source and
> > > produce a patch) mode to avoid actually mandating the plugin.  Not least
> > > of which because I'd find it surprising if the plugin ever worked across
> > > as broad a range of GCC versions as the kernel typically wants to
> > > support.
> > 
> > All gcc plugins in PaX support all plugin capable gcc versions (4.5-5). 
> 
> The kernel supports older GCC than that, though.
> 
> > And of course the plugin infrastructure handles gcc versions that don't
> > support plugins.
> 
> ...huh?  How does *that* work?

If a gcc plugin kernel option is enabled then the kernel Makefile prints
this out if the gcc version can support plugins but there is a problem:
"warning, your gcc installation does not support plugins, perhaps the necessary headers are missing?"
or this if the version is too old:
"warning, your gcc version does not support plugins, you should upgrade it to gcc 4.5 at least"
The compilation goes on without plugins.

-- 
Emese

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Alyssa Milburn]

On Fri, Nov 06, 2015 at 09:42:17PM -0800, Josh Triplett wrote:
> (For that matter, as the LLVM Linux project progresses, it might
> actually prove easier to provide a clang-based plugin.)

If anyone is interested in LLVM-based plugins (I've been assuming nobody
is, especially since LLVMLinux seems to have been pretty quiet recently),
I've been investigating security improvements via compiler plugins as part
of my MSc project, and I'd love to write LLVM/clang passes/analyses if
anyone has any particular wishes for them. (But trying to replicate this in
gcc plugins is not something I can justify at present, especially since
Emese already has many covered.)

I already partially reimplemented a couple of the plugins from the
grsecurity patchset, but right now they're a mess bound up in all of my
other code (and I have no nice infrastructure, just hacks resting on top of
the last version of LLVMLinux + patches + LTO + LTO-on-LLVM + ...). On the
kernel side I just wanted to see how *this* vulnerability could have been
mitigated with *that* technique for *this* performance penalty, so as long
as I can compile+boot a kernel for benchmarking, I've been happy.

- Alyssa

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[PaX Team]

On 8 Nov 2015 at 9:27, Alyssa Milburn wrote:

> On Fri, Nov 06, 2015 at 09:42:17PM -0800, Josh Triplett wrote:
> > (For that matter, as the LLVM Linux project progresses, it might
> > actually prove easier to provide a clang-based plugin.)
> 
> If anyone is interested in LLVM-based plugins (I've been assuming nobody
> is, especially since LLVMLinux seems to have been pretty quiet recently),

actually i originally wanted to use clang/llvm as the basis for my
compiler based work, way back in 2009. it's not a coincidence that
a bit later i could contribute all that code to make clang able to
compile linux. while for practical reasons my interests had shifted
to gcc afterwards, i still have an interest in revitalizing my work
on the clang/llvm side so if you can/want to, feel free to share your
work and i'll help bring it into shape. i'm also only a humble student
of llvm internals so any comparative work such as ports of existing
gcc plugins would be of tremendous help in getting familiar with how
the clang/llvm side of the world works.

cheers,
 PaX Team

[kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

> > Could the plugin operate in a mode where it emits warnings to add such
> > annotations explicitly in the code, rather than just automatically
> > moving the data?
> 
> That would be nice for the constanfy mode as well, especially as some
> people aren't using gcc to build the kernel anymore, so it would be good
> to mark these "for real" in the .c code wherever possible to allow other
> compilers to take advantage of the plugin indirectly.

Yes, I can do it of course. There can be two kernel config options:
 * warning (dry run) mode: the plugin just prints out the warnings 
 * constify: do the constification automatically

-- 
Emese

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[David Sterba]

On Sat, Nov 07, 2015 at 10:58:10PM +0100, Emese Revfy wrote:
> > > Could the plugin operate in a mode where it emits warnings to add such
> > > annotations explicitly in the code, rather than just automatically
> > > moving the data?
> > 
> > That would be nice for the constanfy mode as well, especially as some
> > people aren't using gcc to build the kernel anymore, so it would be good
> > to mark these "for real" in the .c code wherever possible to allow other
> > compilers to take advantage of the plugin indirectly.
> 
> Yes, I can do it of course. There can be two kernel config options:
>  * warning (dry run) mode: the plugin just prints out the warnings 
>  * constify: do the constification automatically

It would be great if all the plugins can run in a verbose mode and
describe the changes or at least point to the file:line. It should help
to understand and review the resulting assembly code where applicable.

[kernel-hardening] Re: Proposal for kernel self protection features

[Greg KH]

On Sat, Nov 07, 2015 at 10:58:10PM +0100, Emese Revfy wrote:
> > > Could the plugin operate in a mode where it emits warnings to add such
> > > annotations explicitly in the code, rather than just automatically
> > > moving the data?
> > 
> > That would be nice for the constanfy mode as well, especially as some
> > people aren't using gcc to build the kernel anymore, so it would be good
> > to mark these "for real" in the .c code wherever possible to allow other
> > compilers to take advantage of the plugin indirectly.
> 
> Yes, I can do it of course. There can be two kernel config options:
>  * warning (dry run) mode: the plugin just prints out the warnings 
>  * constify: do the constification automatically

That sounds wonderful, I would love to see this happen.

thanks,

greg k-h

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Ard Biesheuvel]

On 7 November 2015 at 01:25, Josh Triplett <josh@joshtriplett.org> wrote:
> On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
>> On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
>> >  * initify: This plugin isn't security related either.
>> >     It moves string constants (__func__ and function string arguments
>> >     marked by the nocapture attribute) only referenced in
>> >     __init/__exit functions to __initconst/__exitconst sections.
>> >     It reduces memory usage (many kB), I think it may be important for
>> >     embedded systems.
>>
>> I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
>> be interested in this! (CCing Josh for thoughts.)
>
> I'd be quite interested.
>
> Could the plugin operate in a mode where it emits warnings to add such
> annotations explicitly in the code, rather than just automatically
> moving the data?
>

Well, I suppose this operates primarily on string literals and
initializers, for which there really isn't a way to annotate them
other than doing something like

#define INITSTR(x) ({ static char const __initconst __str[] = #x; __str; })

which unfortunately breaks the concatenation of string literals (e.g.,
'printk(KERN_ERR INITSTR(foo))' will not work)

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Mathias Krause]

On 7 November 2015 at 13:41, Ard Biesheuvel <ard.biesheuvel@linaro.org> wrote:
> On 7 November 2015 at 01:25, Josh Triplett <josh@joshtriplett.org> wrote:
>> On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
>>> On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
>>> >  * initify: This plugin isn't security related either.
>>> >     It moves string constants (__func__ and function string arguments
>>> >     marked by the nocapture attribute) only referenced in
>>> >     __init/__exit functions to __initconst/__exitconst sections.
>>> >     It reduces memory usage (many kB), I think it may be important for
>>> >     embedded systems.
>>>
>>> I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
>>> be interested in this! (CCing Josh for thoughts.)
>>
>> I'd be quite interested.
>>
>> Could the plugin operate in a mode where it emits warnings to add such
>> annotations explicitly in the code, rather than just automatically
>> moving the data?
>>
>
> Well, I suppose this operates primarily on string literals and
> initializers, for which there really isn't a way to annotate them
> other than doing something like
>
> #define INITSTR(x) ({ static char const __initconst __str[] = #x; __str; })
>
> which unfortunately breaks the concatenation of string literals (e.g.,
> 'printk(KERN_ERR INITSTR(foo))' will not work)

There were multiple attempts in the past to do this kind of annotation
on a source code level. The last one being [1], attempting to provide
pr_<LVL>() style of helpers for this. But it was shot down by Ingo
[2].

So I don't think a mode were the plugin generates patches is of much
use as those won't be accepted anyway. It's nice to get a feeling for
the coverage, but not so much for actual code changes.


Regards,
Mathias

[1] https://lkml.org/lkml/2014/8/21/255
[2] https://lkml.org/lkml/2014/8/21/290

[kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

On Fri, 6 Nov 2015 16:25:08 -0800
Josh Triplett <josh@joshtriplett.org> wrote:

> On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
> > On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
> > >  * initify: This plugin isn't security related either.
> > >     It moves string constants (__func__ and function string arguments
> > >     marked by the nocapture attribute) only referenced in
> > >     __init/__exit functions to __initconst/__exitconst sections.
> > >     It reduces memory usage (many kB), I think it may be important for
> > >     embedded systems.
> > 
> > I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
> > be interested in this! (CCing Josh for thoughts.)
> 
> I'd be quite interested.
> 
> Could the plugin operate in a mode where it emits warnings to add such
> annotations explicitly in the code, rather than just automatically
> moving the data?

I can do it, but as Ard said it also moves e.g., format strings and __func__
that is not possible to do with source code changes.

-- 
Emese

[kernel-hardening] Re: Proposal for kernel self protection features

[Josh Triplett]

On Sat, Nov 07, 2015 at 10:42:35PM +0100, Emese Revfy wrote:
> On Fri, 6 Nov 2015 16:25:08 -0800
> Josh Triplett <josh@joshtriplett.org> wrote:
> 
> > On Fri, Nov 06, 2015 at 03:30:39PM -0800, Kees Cook wrote:
> > > On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
> > > >  * initify: This plugin isn't security related either.
> > > >     It moves string constants (__func__ and function string arguments
> > > >     marked by the nocapture attribute) only referenced in
> > > >     __init/__exit functions to __initconst/__exitconst sections.
> > > >     It reduces memory usage (many kB), I think it may be important for
> > > >     embedded systems.
> > > 
> > > I bet the Tinification project ( https://tiny.wiki.kernel.org/ ) would
> > > be interested in this! (CCing Josh for thoughts.)
> > 
> > I'd be quite interested.
> > 
> > Could the plugin operate in a mode where it emits warnings to add such
> > annotations explicitly in the code, rather than just automatically
> > moving the data?
> 
> I can do it, but as Ard said it also moves e.g., format strings and __func__
> that is not possible to do with source code changes.

Ah.  It'd be nice to have GCC options in default GCC to move those.

[kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

> >     * size_overflow_hash_aux: For out-of-tree modules.
> 
> By out-of-tree, you mean built-out-of-tree? I assume a future

I mean out-of-tree modules e.g., zfs. But the gcc infrastructure
in PaX also supports built-out-of-tree and cross compilation.

> downstream kernel that used size_overflow would have their
> non-upstream code covered during the size_overflow_hash regeneration?
> (Though they'd be on the their own for handling the false positives.)

Yes, everybody can regenerate the hash table (the plugin prints out a message
about missing entries that I parse with a python script). I store functions
from out-of-tree modules in a separate hash table because we don't regenerate it
for every kernel version. 

> >        * gcc intentional overflow: gcc computes some expressions by overflow
> >           when it optimizes. Sadly it is doing this in the front end where
> >           there is no plugin support. Most of these false positives I handle
> >           from the plugin or sometimes I patch the kernel source code.
> >           There are some unsolved issues.
> 
> Has there been any discussion with gcc folks about this problem?

I never tried it. PaXTeam has some open tickets
(e.g., https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61311) for a long time so
because of it I haven't any courage.

> >  * structleak: This plugin forcibly initializes certain local variables
> >     based on the __user attribute that could otherwise leak kernel stack
> >     to userland if they aren't properly initialized by later code.
> >     It has small performance loss.
> 
> Does this operate only on kernel stack structures?

Yes, it does but the coverage can be increased.

> > I think the first plugin can be a simpler plugin e.g., constify or latent_entropy.
> > Please let me know if you want one, more or all of them.
> 
> Well I would love to see all of them upstream. :) As for first steps,
> you've convinced me about the complexity of the size_overflow plugin.
> Perhaps the constify plugin would be a good first target? Can you

Yes, I agree.

> speak to any known bugs it stopped? Having some mention of the threat
> it mitigates would be helpful. (Do I remember correctly that it
> constified security_operations, which was a common target in
> exploits?)

I don't remember any bugs, but I think spender has some exploits that
are stopped by constification :) The constify plugin stops exploits that
want to modify ops structures to control indirect calls through them.

-- 
Emese

[kernel-hardening] Re: Proposal for kernel self protection features

[Kees Cook]

On Sat, Nov 7, 2015 at 1:34 PM, Emese Revfy <re.emese@gmail.com> wrote:
>> speak to any known bugs it stopped? Having some mention of the threat
>> it mitigates would be helpful. (Do I remember correctly that it
>> constified security_operations, which was a common target in
>> exploits?)
>
> I don't remember any bugs, but I think spender has some exploits that
> are stopped by constification :) The constify plugin stops exploits that
> want to modify ops structures to control indirect calls through them.

Yeah, just listing a few somewhere in the future patch or docs would
be cool, or we can add that to the wiki, etc. Mostly I just want to be
able to help people understand what a given mitigation could have
stopped, etc. (And given the frequency of ops structure abuse in just
the exploits I reviewed for the Kernel Summit slides, that's a lot...)
Since many people don't understand the value of exploit-blocking, it's
nice to point specifically to known exploits that would have been
blocked.

-Kees

-- 
Kees Cook
Chrome OS Security

[kernel-hardening] Re: Proposal for kernel self protection features

[Paolo Bonzini]

On 07/11/2015 22:34, Emese Revfy wrote:
>>>        * gcc intentional overflow: gcc computes some expressions by overflow
>>>           when it optimizes. Sadly it is doing this in the front end where
>>>           there is no plugin support. Most of these false positives I handle
>>>           from the plugin or sometimes I patch the kernel source code.
>>>           There are some unsolved issues.
>>
>> Has there been any discussion with gcc folks about this problem?
> 
> I never tried it. PaXTeam has some open tickets
> (e.g., https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61311) for a long time so
> because of it I haven't any courage.

I'm not sure I still count as a gcc guy, having averaged at most 1 patch
a year for some time now.  However, I surely would like to know more
about it, and perhaps can look into fixing some of the easier issues.
Do open tickets and CC me (I'm bonzini@gnu.org on the GCC tracker).

Paolo

[kernel-hardening] Re: Proposal for kernel self protection features

[PaX Team]

On 11 Nov 2015 at 14:32, Paolo Bonzini wrote:

> I'm not sure I still count as a gcc guy, having averaged at most 1 patch
> a year for some time now.  However, I surely would like to know more
> about it, and perhaps can look into fixing some of the easier issues.
> Do open tickets and CC me (I'm bonzini@gnu.org on the GCC tracker).

here's a few existing bugs of interest:

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61311 (the header issue is
also tracked in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61176#c18)

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61313

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41757 (this ship has sailed
already i guess)

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46354

i think fixing the LTO API for plugins (including backports to 4.9/5 at
least) would be the most important and useful act for the kernel as it'd
then help convince kernel developers of the merits of adding LTO build
support to linux itself. IIRC, it was tried before and ran into opposition
due to perceived lack of usefulness, something that IPA/LTO capable plugins
could change.

thanks,
 PaX Team

[kernel-hardening] Re: Proposal for kernel self protection features

[Paolo Bonzini]

On 12/11/2015 13:22, PaX Team wrote:
> here's a few existing bugs of interest:
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61311 (the header issue is
> also tracked in https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61176#c18)
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61313
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=41757 (this ship has sailed
> already i guess)
> 
> https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46354
> 
> i think fixing the LTO API for plugins (including backports to 4.9/5 at
> least) would be the most important and useful act for the kernel as it'd
> then help convince kernel developers of the merits of adding LTO build
> support to linux itself. IIRC, it was tried before and ran into opposition
> due to perceived lack of usefulness, something that IPA/LTO capable plugins
> could change.

Do you have any pointers on the overflows that can be introduced by the
front-end?  I guess I can also look for hints in the plugin source code.

Paolo

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

 
> Do you have any pointers on the overflows that can be introduced by the
> front-end?  I guess I can also look for hints in the plugin source code.

Hi,

You can find most of them in fold-const.c. Here is an example:
https://gcc.gnu.org/ml/gcc-help/2011-10/msg00136.html

-- 
Emese

[kernel-hardening] Re: Proposal for kernel self protection features

[PaX Team]

On 6 Nov 2015 at 15:30, Kees Cook wrote:

> >        * gcc intentional overflow: gcc computes some expressions by overflow
> >           when it optimizes. Sadly it is doing this in the front end where
> >           there is no plugin support. Most of these false positives I handle
> >           from the plugin or sometimes I patch the kernel source code.
> >           There are some unsolved issues.
> 
> Has there been any discussion with gcc folks about this problem?

i seem to recall that there's an effort (gcc branch?) where some work
is being done to move the frontend canonicalization (the prime cause
for most if not all intentional overflows) into a proper GIMPLE pass
which would make life easier for the size overflow plugin. however i
have no idea where that work stands.

as a general note, some gcc (and eventually clang/llvm) folks could
perhaps be involved in kernel/plugin related discussions. this would
not only help heal some of the rift that developed over the years
between these communities but would ideally develop into a mode of
cooperation where features that kernel developers desire from the
compiler (e.g., a kernel specific optimization mode between -Os and
-O2 that comes up every now and then) could be prototyped and tested
as a plugin in the kernel and then submitted to gcc when they're
found good/mature enough. since gcc is developed at a much slower
pace than the kernel (1 release per year vs. 5 of linux), this would
certainly help put features faster into the hands of users (and support
even those using older gcc versions as well) while ensuring longer
term reliable support from the compiler. just some food for thought ;).

> >     You can read more about the plugin:
> >     https://forums.grsecurity.net/viewtopic.php?f=7&t=3043
> >     This is a complex plugin and it has false positives but it also found
> >     a lot of bugs (overflows, underflows and trunctions) and
> >     thanks to the data flow duplication it has a small performance impact only.
> >     I know that I will have to adapt it to be suitable for vanilla.
> 
> That's a great write-up! I would call other people's attention to the
> portion where you point out it blocks real-world cases of flaws:
> CVE-2013-0914 CVE-2013-0913 CVE-2013-2141

note that there're many cases where integer related problems just got
fixed without anyone (ourselves included) trying to figure out the exact
consequences.

finding these integer handling bugs is not just about security but general
correctness as well. e.g., 646200a041203f440fb6fcf9cacd9efeda9de74c started
out as a runtime alert from the overflow plugin and then turned out to be
something entirely different.

> >  * constify: This plugin constifies all structure types which contain only
> >     function pointers or are explicitly marked for constification.
> >     If some code wants to modify a read-only object it will cause
> >     a compile error.
> >     There are two gcc attributes:
> >        * no_const: if we don't want constification
> >        * do_const: if we want to constify a structure type which has a
> >           non-function pointer field
> 
> I assume some level of no_const marking could go away when KERNEXEC or
> at least a similar "temporary writing" ability lands in upstream?

the no_const attribute exists because PaX does automatic ops structure
constification and there're some ops types that cannot be made read-only
this way for one reason or another:

  - some/all objects of the ops type are dynamically allocated on the
    kernel heap as part of a larger object and cannot be easily made
    read-only, it would require sometimes non-trivial source changes.

  - some static objects of the ops type are written to at runtime,
    typically once during initialization. handling this case is
    somewhat messy in the current plugin, it's either done via forcibly
    overriding the constness of the object or by using the no_const
    attribute and placing the object into a section that will be part
    of read-only kernel data after init (in either case open/close
    calls are needed). i have plans to move to the latter approach
    exclusively but that requires rewriting the core of the plugin
    to not rely on existing gcc infrastructure to detect const violations.

> >  * latent_entropy: This plugin generates some entropy from program state
> >     throughout the uptime of the kernel. It has small performance loss.
> >     The plugin uses an attribute which can be
> >     on a function (to extract entropy beyond init functions) or on a
> >     variable (to initialize it with a random number generated at compile time)
> 
> How large is the entropy extracted during early init? I bet a lot of
> architectures would rejoice to have this available. (CCing Ted for
> thoughts.)

that's a good question nobody can answer unfortunately. thing is, this
kind of instrumentation essentially builds a PRNG whose internal structure
reflects the control flow of instrumented kernel functions, i.e., it's not
something anyone can describe and analyze because of its size and dynamic
nature (think loops with variable number of rounds, interrupts, etc).

what one can do is measure the number of states the PRNG can end up in say
right after boot and see how wide and uniform the distribution of states
is. at the time i developed this plugin i ran a few tests on an allnoconfig
amd64 kernel under qemu/kvm and even that environment produced a few bits
of entropy (i.e., more than one state). however this kind of testing can
scale only so far, e.g., to establish that say 30 bits of entropy can be
generated this way by the time init is called, one would have to boot the
kernel billions of times already...

> Perhaps the constify plugin would be a good first target? Can you
> speak to any known bugs it stopped? Having some mention of the threat
> it mitigates would be helpful.

(function) pointer constification is one of two strategies to protect
control flow (the other is code pointer target verification) in a similar
way that constifying code is a strategy to protect executable from
modifications.

since, unlike code, not all function pointers can be made const by design,
this constification effort basically amounts to attack surface reduction
only, albeit a cheap and effective one i think. also it's not just about
security (mitigating exploits, rootkits, etc) per se but also a sort of
enforcement mechanism for kernel policy that exists for some of these ops
types already and forces bad actors (khm, AV, khm) to be overt when they're
circumventing it in their products.

as for the effectiveness of this attack surface reduction method, on a
4.3 allyes-minus-some-debugging-options amd64 kernel the number of static
writable function pointers goes from 62k down to 28k in PaX (the ratio
would be somewhat worse on vanilla since PaX does more forced constification
that require source changes as well). on my own test machine's kernel the
ratio is even better, more than 75% of these function pointers get constified.

> (Do I remember correctly that it constified security_operations, which
> was a common target in exploits?) 

the plugin did not only constify security_operations objects but we also
put the security_ops pointer itself into read-only memory as well (that is,
until 4.2 where the new LSM stacking code made this approach infeasible
without extensive source changes).

cheers,
 PaX Team

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Laura Abbott]

On 11/8/15 5:13 AM, PaX Team wrote:
> On 6 Nov 2015 at 15:30, Kees Cook wrote:
>>>   * latent_entropy: This plugin generates some entropy from program state
>>>      throughout the uptime of the kernel. It has small performance loss.
>>>      The plugin uses an attribute which can be
>>>      on a function (to extract entropy beyond init functions) or on a
>>>      variable (to initialize it with a random number generated at compile time)
>>
>> How large is the entropy extracted during early init? I bet a lot of
>> architectures would rejoice to have this available. (CCing Ted for
>> thoughts.)
>
> that's a good question nobody can answer unfortunately. thing is, this
> kind of instrumentation essentially builds a PRNG whose internal structure
> reflects the control flow of instrumented kernel functions, i.e., it's not
> something anyone can describe and analyze because of its size and dynamic
> nature (think loops with variable number of rounds, interrupts, etc).
>
> what one can do is measure the number of states the PRNG can end up in say
> right after boot and see how wide and uniform the distribution of states
> is. at the time i developed this plugin i ran a few tests on an allnoconfig
> amd64 kernel under qemu/kvm and even that environment produced a few bits
> of entropy (i.e., more than one state). however this kind of testing can
> scale only so far, e.g., to establish that say 30 bits of entropy can be
> generated this way by the time init is called, one would have to boot the
> kernel billions of times already...
>

I took a look at something closely related sometime ago[1] for
ARM DT based targets. The stack canary was always the same because
it was early enough no entropy was being added and there's no standard
RNG. This series proposed allowing entropy to be read out of the DT.
This still relied on reliable entropy being added to the DT somehow.
Ultimately, I never followed up and this still seems to be an issue.

The out of tree solution which was never submitted was to make a call
to the hwrng very early to seed the pool. This was very SoC specific
though. At the time I wrote the patches, I didn't look to see what
Pax/GrSec might have had to address the issue.

Thanks,
Laura

[1]http://thread.gmane.org/gmane.linux.kernel/1646202/focus=61628

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Jason Cooper]

On Mon, Nov 09, 2015 at 10:07:25AM -0800, Laura Abbott wrote:
> On 11/8/15 5:13 AM, PaX Team wrote:
> >On 6 Nov 2015 at 15:30, Kees Cook wrote:
> >>>  * latent_entropy: This plugin generates some entropy from program state
> >>>     throughout the uptime of the kernel. It has small performance loss.
> >>>     The plugin uses an attribute which can be
> >>>     on a function (to extract entropy beyond init functions) or on a
> >>>     variable (to initialize it with a random number generated at compile time)
> >>
> >>How large is the entropy extracted during early init? I bet a lot of
> >>architectures would rejoice to have this available. (CCing Ted for
> >>thoughts.)
> >
> >that's a good question nobody can answer unfortunately. thing is, this
> >kind of instrumentation essentially builds a PRNG whose internal structure
> >reflects the control flow of instrumented kernel functions, i.e., it's not
> >something anyone can describe and analyze because of its size and dynamic
> >nature (think loops with variable number of rounds, interrupts, etc).
> >
> >what one can do is measure the number of states the PRNG can end up in say
> >right after boot and see how wide and uniform the distribution of states
> >is. at the time i developed this plugin i ran a few tests on an allnoconfig
> >amd64 kernel under qemu/kvm and even that environment produced a few bits
> >of entropy (i.e., more than one state). however this kind of testing can
> >scale only so far, e.g., to establish that say 30 bits of entropy can be
> >generated this way by the time init is called, one would have to boot the
> >kernel billions of times already...
> >
> 
> I took a look at something closely related sometime ago[1] for
> ARM DT based targets. The stack canary was always the same because
> it was early enough no entropy was being added and there's no standard
> RNG. This series proposed allowing entropy to be read out of the DT.
> This still relied on reliable entropy being added to the DT somehow.
> Ultimately, I never followed up and this still seems to be an issue.

I had a proposal a while back (can't find atm, sorry) to have the
bootloader load the random-seed into RAM and hand the kernel an address
and size.  The decompressor would then use some of it to init KASLR, and
the rest would be fed into random.c as non-credit entropy during driver
init.  -ETIME

The benefit of this approach is that it would /not/ require a bootloader
upgrade.  You set the seed address and size in the DT once, append it
(legacy boards already in the field), and adjust the bootup scripts to
read the extra file into RAM before jumping to the kernel.

You may need to adjust userspace to write to a flash partition or
/boot/random-seed, depending on how things are laid out.

thx,

Jason.

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Jason Cooper]

On Mon, Nov 09, 2015 at 06:28:32PM +0000, Jason Cooper wrote:
> I had a proposal a while back (can't find atm, sorry) to have the

Found it:

  https://marc.info/?l=linux-kernel&m=139223237824952&w=2

thx,

Jason.

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Valdis.Kletnieks]

[-- Attachment #1: Type: text/plain, Size: 359 bytes --]

On Mon, 09 Nov 2015 18:28:32 +0000, Jason Cooper said:
> I had a proposal a while back (can't find atm, sorry) to have the
> bootloader load the random-seed into RAM ...

It's *easy* to come up with an API to hand the kernel 64 or 128 bits of
random to kick things off.

The *hard* part is finding 64 or so bits of trustable random to hand to
the kernel....


[-- Attachment #2: Type: application/pgp-signature, Size: 848 bytes --]

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Jason Cooper]

Hey Valdis,

On Mon, Nov 09, 2015 at 01:57:57PM -0500, Valdis.Kletnieks@vt.edu wrote:
> On Mon, 09 Nov 2015 18:28:32 +0000, Jason Cooper said:
> > I had a proposal a while back (can't find atm, sorry) to have the
> > bootloader load the random-seed into RAM ...
> 
> It's *easy* to come up with an API to hand the kernel 64 or 128 bits of
> random to kick things off.
> 
> The *hard* part is finding 64 or so bits of trustable random to hand to
> the kernel....

/var/lib/misc/random-seed has served that role for years, I'm only
advocating loading it earlier in the boot process.  It's *much* harder
to guess the state of random-seed than the dtb or mac address(es)...

thx,

Jason.

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Theodore Tso]

[-- Attachment #1: Type: text/plain, Size: 698 bytes --]

On Mon, Nov 9, 2015 at 2:02 PM, Jason Cooper <
kernel-hardening@lakedaemon.net> wrote:

> /var/lib/misc/random-seed has served that role for years, I'm only
> advocating loading it earlier in the boot process.  It's *much* harder
> to guess the state of random-seed than the dtb or mac address(es)...
>

If the bootloader is willing to reach into the file system, which means (a)
having a minimal file system layer, like Grub does, and (b) can find the
block device where the file is found, that's a perfectly *fine*
implementation.    I'm not sure mobile handset vendors will be all that
psyched into either using or replicating all of Grub's functionality so it
could do that, though....

-- Ted

[-- Attachment #2: Type: text/html, Size: 1202 bytes --]

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Josh Triplett]

On Mon, Nov 09, 2015 at 02:11:35PM -0500, Theodore Tso wrote:
> On Mon, Nov 9, 2015 at 2:02 PM, Jason Cooper <
> kernel-hardening@lakedaemon.net> wrote:
> 
> > /var/lib/misc/random-seed has served that role for years, I'm only
> > advocating loading it earlier in the boot process.  It's *much* harder
> > to guess the state of random-seed than the dtb or mac address(es)...
> >
> 
> If the bootloader is willing to reach into the file system, which means (a)
> having a minimal file system layer, like Grub does, and (b) can find the
> block device where the file is found, that's a perfectly *fine*
> implementation.    I'm not sure mobile handset vendors will be all that
> psyched into either using or replicating all of Grub's functionality so it
> could do that, though....

How crazy would it be to append it to the end of the initramfs, as we've
started making possible for critical firmware/microcode/tables?

- Josh Triplett

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Jason Cooper]

On Mon, Nov 09, 2015 at 12:06:23PM -0800, Josh Triplett wrote:
> On Mon, Nov 09, 2015 at 02:11:35PM -0500, Theodore Tso wrote:
> > On Mon, Nov 9, 2015 at 2:02 PM, Jason Cooper <
> > kernel-hardening@lakedaemon.net> wrote:
> > 
> > > /var/lib/misc/random-seed has served that role for years, I'm only
> > > advocating loading it earlier in the boot process.  It's *much* harder
> > > to guess the state of random-seed than the dtb or mac address(es)...
> > >
> > 
> > If the bootloader is willing to reach into the file system, which means (a)
> > having a minimal file system layer, like Grub does, and (b) can find the
> > block device where the file is found, that's a perfectly *fine*
> > implementation.    I'm not sure mobile handset vendors will be all that
> > psyched into either using or replicating all of Grub's functionality so it
> > could do that, though....
> 
> How crazy would it be to append it to the end of the initramfs, as we've
> started making possible for critical firmware/microcode/tables?

Well, my main goal was to strengthen KASLR on ARM.  Which means the code
needs to reside in the decompressor...

thx,

Jason.

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Jason Cooper]

On Mon, Nov 09, 2015 at 02:11:35PM -0500, Theodore Tso wrote:
> On Mon, Nov 9, 2015 at 2:02 PM, Jason Cooper <
> kernel-hardening@lakedaemon.net> wrote:
> 
> > /var/lib/misc/random-seed has served that role for years, I'm only
> > advocating loading it earlier in the boot process.  It's *much* harder
> > to guess the state of random-seed than the dtb or mac address(es)...
> >
> 
> If the bootloader is willing to reach into the file system, which means (a)
> having a minimal file system layer, like Grub does, and (b) can find the
> block device where the file is found, that's a perfectly *fine*
> implementation.    I'm not sure mobile handset vendors will be all that
> psyched into either using or replicating all of Grub's functionality so it
> could do that, though....

Well, That's why I referred to reading from /boot or from a flash
partition.  Existing bootloaders in the field already have that
capability.  That's how they load the kernel.

thx,

Jason.

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Matthew Garrett]

On Mon, Nov 09, 2015 at 09:09:22PM +0000, Jason Cooper wrote:

> Well, That's why I referred to reading from /boot or from a flash
> partition.  Existing bootloaders in the field already have that
> capability.  That's how they load the kernel.

This doesn't really handle cases like network booting. Most SoCs have 
some kind of RNG, recent x86 has hardware RNG, older x86 frequently has 
an RNG in a TPM. Pulling an entropy seed from the filesystem is a 
reasonable fallback, but we should definitely be thinking of it as a 
fallback - someone with physical access to your system while it's turned 
off may be able to infer the ASLR state for your next boot, for 
instance.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Yves-Alexis Perez]

[-- Attachment #1: Type: text/plain, Size: 1326 bytes --]

On lun., 2015-11-09 at 21:13 +0000, Matthew Garrett wrote:
> On Mon, Nov 09, 2015 at 09:09:22PM +0000, Jason Cooper wrote:
> 
> > Well, That's why I referred to reading from /boot or from a flash
> > partition.  Existing bootloaders in the field already have that
> > capability.  That's how they load the kernel.
> 
> This doesn't really handle cases like network booting. Most SoCs have 
> some kind of RNG, recent x86 has hardware RNG, older x86 frequently has 
> an RNG in a TPM. Pulling an entropy seed from the filesystem is a 
> reasonable fallback, but we should definitely be thinking of it as a 
> fallback - someone with physical access to your system while it's turned 
> off may be able to infer the ASLR state for your next boot, for 
> instance.

About the TPM RNG: I was definitely interested in the “get entropy from
TPM” kind of thing for my laptop (even though I think my laptop would
not be the worst place to find entropy). Right now I'm using rng-tools
to read from /dev/hwrng (handled by tpm_rng module), which then feeds
entropy to /dev/random (or maybe directly using the RND ioctls).

Do you think bypassing userland completely in order to be able to feed
that entropy even before userland is completely initialized would be
helpful?

Regards,

-- 
Yves-Alexis


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Marcus Meissner]

On Tue, Nov 10, 2015 at 11:42:31AM +0100, Yves-Alexis Perez wrote:
> On lun., 2015-11-09 at 21:13 +0000, Matthew Garrett wrote:
> > On Mon, Nov 09, 2015 at 09:09:22PM +0000, Jason Cooper wrote:
> > 
> > > Well, That's why I referred to reading from /boot or from a flash
> > > partition.  Existing bootloaders in the field already have that
> > > capability.  That's how they load the kernel.
> > 
> > This doesn't really handle cases like network booting. Most SoCs have 
> > some kind of RNG, recent x86 has hardware RNG, older x86 frequently has 
> > an RNG in a TPM. Pulling an entropy seed from the filesystem is a 
> > reasonable fallback, but we should definitely be thinking of it as a 
> > fallback - someone with physical access to your system while it's turned 
> > off may be able to infer the ASLR state for your next boot, for 
> > instance.
> 
> About the TPM RNG: I was definitely interested in the “get entropy from
> TPM” kind of thing for my laptop (even though I think my laptop would
> not be the worst place to find entropy). Right now I'm using rng-tools
> to read from /dev/hwrng (handled by tpm_rng module), which then feeds
> entropy to /dev/random (or maybe directly using the RND ioctls).
> 
> Do you think bypassing userland completely in order to be able to feed
> that entropy even before userland is completely initialized would be
> helpful?

The kernel has infrastructure for this (feeding hardware random generators
into the random pool) these days.

e.g.
drivers/char/hw_random/tpm-rng.c

How "good" the generator is is determined by  the "quality" member of
the hwrng struct.

(It seems only virtio-rng.c sets one, but more of them could be reviewed
and used.)

Ciao, Marcus

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Yves-Alexis Perez]

[-- Attachment #1: Type: text/plain, Size: 685 bytes --]

On mar., 2015-11-10 at 11:47 +0100, Marcus Meissner wrote:
> The kernel has infrastructure for this (feeding hardware random generators
> into the random pool) these days.
> 
> e.g.
> drivers/char/hw_random/tpm-rng.c

Thanks, I was missing those bits. Actually those are in (on v4.3, starting
from drivers/char/hw_random/core.c#L483):

hwrng_register()
	add_early_randomness()
		rng_get_data()
		add_device_randomness()

but as far as I can tell it only gets called once when registering the hwrng
driver, and only if the RNG driver doesn't define an init function (tpm-rng
doesn't). But that's still better than no randomness at all.

Regards,
-- 
Yves-Alexis


[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 473 bytes --]

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[Theodore Tso]

[-- Attachment #1: Type: text/plain, Size: 1408 bytes --]

On Mon, Nov 9, 2015 at 1:07 PM, Laura Abbott <laura@labbott.name> wrote:
>
>
> I took a look at something closely related sometime ago[1] for
> ARM DT based targets. The stack canary was always the same because
> it was early enough no entropy was being added and there's no standard
> RNG. This series proposed allowing entropy to be read out of the DT.
> This still relied on reliable entropy being added to the DT somehow.
> Ultimately, I never followed up and this still seems to be an issue.
>

Yeah this is something that has been bothering me for at least a year or
two.
I've brought it up with Grant Likely, hoping that maybe this was something
Linaro could perhaps focus on, but both he and I have just not had the time
to follow up.

We really really need a standard way to pass entropy from the bootloader
into the random driver, for each architecture, and then each boot loader
needs to fetch whatever entropy it can, which may be SoC or architecture
specific.   Or someone needs to beat up on the ARM consortium to guarantee
HWRNG with a standard interface.

It could be via the device tree, or via a fixed memory segment, such as how
the
boot command line is passed from the bootloader to the kernel.   I can help
provide a design and consultation/advice.   But I have zero time to do the
implementation, especially on the ARM architecture (where I lack expertise
and
sample hardware).

-- Ted

[-- Attachment #2: Type: text/html, Size: 1961 bytes --]

Re: [kernel-hardening] Re: Proposal for kernel self protection features

[PaX Team]

On 9 Nov 2015 at 10:07, Laura Abbott wrote:

> I took a look at something closely related sometime ago[1] for
> ARM DT based targets. The stack canary was always the same because
> it was early enough no entropy was being added and there's no standard
> RNG. This series proposed allowing entropy to be read out of the DT.
> This still relied on reliable entropy being added to the DT somehow.
> Ultimately, I never followed up and this still seems to be an issue.
> 
> The out of tree solution which was never submitted was to make a call
> to the hwrng very early to seed the pool. This was very SoC specific
> though. At the time I wrote the patches, I didn't look to see what
> Pax/GrSec might have had to address the issue.

to extract entropy via the plugin by the time start_kernel is called
you'd have to use the plugin for code that runs beforehand, i.e., the
firmware, boot loader, etc and also find a way to pass this entropy
to the kernel somehow. another feature of the plugin is to initialize
global variables with compile-time generated random numbers, but if
the kernel images are available to an attacker then it's of no use
of course. last but not least PaX also feeds hashes of every page into
the random pool as it gets taken over by the kernel allocator, this
could be run earlier to extract whatever entropy is left in RAM.

cheers,
 PaX Team

[kernel-hardening] Re: Proposal for kernel self protection features

[Kees Cook]

On Sun, Nov 8, 2015 at 5:13 AM, PaX Team <pageexec@freemail.hu> wrote:
> On 6 Nov 2015 at 15:30, Kees Cook wrote:
>
>> >        * gcc intentional overflow: gcc computes some expressions by overflow
>> >           when it optimizes. Sadly it is doing this in the front end where
>> >           there is no plugin support. Most of these false positives I handle
>> >           from the plugin or sometimes I patch the kernel source code.
>> >           There are some unsolved issues.
>>
>> Has there been any discussion with gcc folks about this problem?
>
> i seem to recall that there's an effort (gcc branch?) where some work
> is being done to move the frontend canonicalization (the prime cause
> for most if not all intentional overflows) into a proper GIMPLE pass
> which would make life easier for the size overflow plugin. however i
> have no idea where that work stands.
>
> as a general note, some gcc (and eventually clang/llvm) folks could
> perhaps be involved in kernel/plugin related discussions. this would
> not only help heal some of the rift that developed over the years

Do you have some people in mind from gcc? I'd love to get them on CC,
but it's not a community I know very well (yet).

> between these communities but would ideally develop into a mode of
> cooperation where features that kernel developers desire from the
> compiler (e.g., a kernel specific optimization mode between -Os and
> -O2 that comes up every now and then) could be prototyped and tested
> as a plugin in the kernel and then submitted to gcc when they're
> found good/mature enough. since gcc is developed at a much slower
> pace than the kernel (1 release per year vs. 5 of linux), this would
> certainly help put features faster into the hands of users (and support
> even those using older gcc versions as well) while ensuring longer
> term reliable support from the compiler. just some food for thought ;).
>
>> >     You can read more about the plugin:
>> >     https://forums.grsecurity.net/viewtopic.php?f=7&t=3043
>> >     This is a complex plugin and it has false positives but it also found
>> >     a lot of bugs (overflows, underflows and trunctions) and
>> >     thanks to the data flow duplication it has a small performance impact only.
>> >     I know that I will have to adapt it to be suitable for vanilla.
>>
>> That's a great write-up! I would call other people's attention to the
>> portion where you point out it blocks real-world cases of flaws:
>> CVE-2013-0914 CVE-2013-0913 CVE-2013-2141
>
> note that there're many cases where integer related problems just got
> fixed without anyone (ourselves included) trying to figure out the exact
> consequences.

Right, absolutely. I just think it's important to have some concrete
context for people who aren't familiar with the protections to see
either flaws or exploits that are known examples that got solved.

> finding these integer handling bugs is not just about security but general
> correctness as well. e.g., 646200a041203f440fb6fcf9cacd9efeda9de74c started
> out as a runtime alert from the overflow plugin and then turned out to be
> something entirely different.

Exactly -- there's much more to it than just security.

>
>> >  * constify: This plugin constifies all structure types which contain only
>> >     function pointers or are explicitly marked for constification.
>> >     If some code wants to modify a read-only object it will cause
>> >     a compile error.
>> >     There are two gcc attributes:
>> >        * no_const: if we don't want constification
>> >        * do_const: if we want to constify a structure type which has a
>> >           non-function pointer field
>>
>> I assume some level of no_const marking could go away when KERNEXEC or
>> at least a similar "temporary writing" ability lands in upstream?
>
> the no_const attribute exists because PaX does automatic ops structure
> constification and there're some ops types that cannot be made read-only
> this way for one reason or another:
>
>   - some/all objects of the ops type are dynamically allocated on the
>     kernel heap as part of a larger object and cannot be easily made
>     read-only, it would require sometimes non-trivial source changes.

Right, AIUI, constify only handles stuff living in .data (by
effectively moving them to .rodata). Once those targets vanish,
heap-allocated function pointers become much more interesting for
exploits. (I remain glad that the INET_DIAG leak was plugged -- if
anyone has the sha for that change handy, I'd love to add it to my
notes.)

>   - some static objects of the ops type are written to at runtime,
>     typically once during initialization. handling this case is
>     somewhat messy in the current plugin, it's either done via forcibly
>     overriding the constness of the object or by using the no_const
>     attribute and placing the object into a section that will be part
>     of read-only kernel data after init (in either case open/close
>     calls are needed). i have plans to move to the latter approach
>     exclusively but that requires rewriting the core of the plugin
>     to not rely on existing gcc infrastructure to detect const violations.

It sounds like we have 3 kinds of (non-heap) const targets:
1) those actually never written to (trivial)
2) those written to during init only (should be easy?)
3) those written to after init (hard)

IIUC, mark_data_ro() is called after init finishes, shouldn't it be
possible to mark those in some way (if they're not already?) For case
3, upstream would need KERNEXEC.

>> >  * latent_entropy: This plugin generates some entropy from program state
>> >     throughout the uptime of the kernel. It has small performance loss.
>> >     The plugin uses an attribute which can be
>> >     on a function (to extract entropy beyond init functions) or on a
>> >     variable (to initialize it with a random number generated at compile time)
>>
>> How large is the entropy extracted during early init? I bet a lot of
>> architectures would rejoice to have this available. (CCing Ted for
>> thoughts.)
>
> that's a good question nobody can answer unfortunately. thing is, this
> kind of instrumentation essentially builds a PRNG whose internal structure
> reflects the control flow of instrumented kernel functions, i.e., it's not
> something anyone can describe and analyze because of its size and dynamic
> nature (think loops with variable number of rounds, interrupts, etc).
>
> what one can do is measure the number of states the PRNG can end up in say
> right after boot and see how wide and uniform the distribution of states
> is. at the time i developed this plugin i ran a few tests on an allnoconfig
> amd64 kernel under qemu/kvm and even that environment produced a few bits
> of entropy (i.e., more than one state). however this kind of testing can
> scale only so far, e.g., to establish that say 30 bits of entropy can be
> generated this way by the time init is called, one would have to boot the
> kernel billions of times already...

Sure, I guess I was curious about the estimate; if any testing had been done.

>> Perhaps the constify plugin would be a good first target? Can you
>> speak to any known bugs it stopped? Having some mention of the threat
>> it mitigates would be helpful.
>
> (function) pointer constification is one of two strategies to protect
> control flow (the other is code pointer target verification) in a similar
> way that constifying code is a strategy to protect executable from
> modifications.
>
> since, unlike code, not all function pointers can be made const by design,
> this constification effort basically amounts to attack surface reduction
> only, albeit a cheap and effective one i think. also it's not just about

Right -- as you measure below, it more than halves the exploit
targets, which is a nice bar to raise at no cost.

> security (mitigating exploits, rootkits, etc) per se but also a sort of
> enforcement mechanism for kernel policy that exists for some of these ops
> types already and forces bad actors (khm, AV, khm) to be overt when they're
> circumventing it in their products.
>
> as for the effectiveness of this attack surface reduction method, on a
> 4.3 allyes-minus-some-debugging-options amd64 kernel the number of static
> writable function pointers goes from 62k down to 28k in PaX (the ratio
> would be somewhat worse on vanilla since PaX does more forced constification
> that require source changes as well). on my own test machine's kernel the
> ratio is even better, more than 75% of these function pointers get constified.
>
>> (Do I remember correctly that it constified security_operations, which
>> was a common target in exploits?)
>
> the plugin did not only constify security_operations objects but we also
> put the security_ops pointer itself into read-only memory as well (that is,
> until 4.2 where the new LSM stacking code made this approach infeasible
> without extensive source changes).

Yeah, so basically all the PoC exploits I wrote in 2010 would have
been blocked by constification. (And UDEREF.)

-Kees

>
> cheers,
>  PaX Team
>

-Kees


-- 
Kees Cook
Chrome OS Security

[kernel-hardening] Re: Proposal for kernel self protection features

[PaX Team]

On 9 Nov 2015 at 13:52, Kees Cook wrote:

> On Sun, Nov 8, 2015 at 5:13 AM, PaX Team <pageexec@freemail.hu> wrote:
> > as a general note, some gcc (and eventually clang/llvm) folks could
> > perhaps be involved in kernel/plugin related discussions. this would
> > not only help heal some of the rift that developed over the years
> 
> Do you have some people in mind from gcc? I'd love to get them on CC,
> but it's not a community I know very well (yet).

sorry, i have no idea which gcc developer would be interested in such
a cross-field cooperation, probably it's worth a shot on their list ;).

> >   - some static objects of the ops type are written to at runtime,
> >     typically once during initialization. handling this case is
> >     somewhat messy in the current plugin, it's either done via forcibly
> >     overriding the constness of the object or by using the no_const
> >     attribute and placing the object into a section that will be part
> >     of read-only kernel data after init (in either case open/close
> >     calls are needed). i have plans to move to the latter approach
> >     exclusively but that requires rewriting the core of the plugin
> >     to not rely on existing gcc infrastructure to detect const violations.
> 
> It sounds like we have 3 kinds of (non-heap) const targets:
> 1) those actually never written to (trivial)
> 2) those written to during init only (should be easy?)
> 3) those written to after init (hard)

yes, that roughly describes the situation. we've been doing work on
all three categories with the first one handled trivially from the
compiler (equivalent to patching in 'const' at the source level).

the other two require more finesse however so as to not 'cheat' too
much (a.k.a. inducing undefined behaviour) as that could result in
incorrect optimization decisions later. e.g., i once forcibly constified
an ops object that was being copied to during init (instead of setting
a ptr to the chosen object) and then the compiler would figure that
all the indirect calls through that object could be optimized into
direct calls since, well, the object was const. so tricking the compiler
has to be done carefully (and the current PaX code for these cases
may not always be the best example you want to follow ;).

> IIUC, mark_data_ro() is called after init finishes, shouldn't it be
> possible to mark those in some way (if they're not already?)

i mentioned two ways to handle this above, the best is not forced
constification but to simply place the object into a special section
that will be linked together with .rodata (in PaX it's called
".data..read_only"). this also works for the third case but then
you'll lose the compiler's already existing mechanism to warn/error
out on writes to const objects so this requires reimplementing that
logic in the plugin (that's the rewrite i alluded to above).

> For case 3, upstream would need KERNEXEC. 

you only need a way to temporarily make it possible to modify otherwise
read-only memory, much like how UDEREF (SMAP) needs temporary permissions
to access userland. the naive approach is simple enough (similar to how
the memory holding module code is made read-only on load then writable
again on unload) but doing it securely (not subject to races) and fast
enough (not having to flush TLB entries) is another matter, that will
require some (arch dependent) infrastructure first.

[kernel-hardening] Re: Proposal for kernel self protection features

[Kees Cook]

Hi Emese,

On Fri, Nov 6, 2015 at 2:55 PM, Emese Revfy <re.emese@gmail.com> wrote:
>  * constify: This plugin constifies all structure types which contain only
>     function pointers or are explicitly marked for constification.
>     If some code wants to modify a read-only object it will cause
>     a compile error.
>     There are two gcc attributes:
>        * no_const: if we don't want constification
>        * do_const: if we want to constify a structure type which has a
>           non-function pointer field

Can you give a status update on the constify plugin work? I'm curious
to hear what your plans are.

Thanks!

-Kees

-- 
Kees Cook
Chrome OS & Brillo Security

[kernel-hardening] Re: Proposal for kernel self protection features

[Emese Revfy]

 
> Can you give a status update on the constify plugin work? I'm curious
> to hear what your plans are.

Hi,

I'll begin with the gcc plugin infrastructure from February on.
When I'm done with it I'll continue to work on the following plugins
(I'm not sure of the order maybe I can work in parallel on them):
1. initify
2. structleak
3. latent_entropy
4. constify

-- 
Emese

[kernel-hardening] Re: Proposal for kernel self protection features

[Kees Cook]

On Tue, Jan 19, 2016 at 1:20 PM, Emese Revfy <re.emese@gmail.com> wrote:
>
>> Can you give a status update on the constify plugin work? I'm curious
>> to hear what your plans are.
>
> Hi,
>
> I'll begin with the gcc plugin infrastructure from February on.
> When I'm done with it I'll continue to work on the following plugins
> (I'm not sure of the order maybe I can work in parallel on them):
> 1. initify
> 2. structleak
> 3. latent_entropy
> 4. constify

Sounds great, thanks!

-Kees

-- 
Kees Cook
Chrome OS & Brillo Security