LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Kristen Carlson Accardi <kristen@linux.intel.com>
To: Ingo Molnar <mingo@kernel.org>
Cc: keescook@chromium.org, tglx@linutronix.de, mingo@redhat.com,
	bp@alien8.de, arjan@linux.intel.com, x86@kernel.org,
	linux-kernel@vger.kernel.org,
	kernel-hardening@lists.openwall.com, rick.p.edgecombe@intel.com
Subject: Re: [PATCH v4 00/10] Function Granular KASLR
Date: Thu, 06 Aug 2020 12:24:58 -0700
Message-ID: <df539dc3a4791d0f873aec9b303f49b64237a6e8.camel@linux.intel.com> (raw)
In-Reply-To: <20200806153258.GB2131635@gmail.com>

Hi Mingo, thanks for taking a look, I am glad you like the idea. Some
replies below:

On Thu, 2020-08-06 at 17:32 +0200, Ingo Molnar wrote:
> * Kristen Carlson Accardi <kristen@linux.intel.com> wrote:
> 
> > Function Granular Kernel Address Space Layout Randomization
> > (fgkaslr)
> > -----------------------------------------------------------------
> > ----
> > 
> > This patch set is an implementation of finer grained kernel address
> > space
> > randomization. It rearranges your kernel code at load time 
> > on a per-function level granularity, with only around a second
> > added to
> > boot time.
> 
> This is a very nice feature IMO, and it should be far more effective 
> at randomizing the kernel, due to the sheer number of randomization 
> bits that kernel function granular randomization presents.
> 
> If this is a good approximation of fg-kaslr randomization depth:
> 
>   thule:~/tip> grep ' [tT] ' /proc/kallsyms  | wc -l
>   88488
> 
> ... then that's 80K bits of randomization instead of the mere
> handful 
> of kaslr bits we have today. Very nice!
> 
> > In order to hide our new layout, symbols reported through 
> > /proc/kallsyms will be displayed in a random order.
> 
> Neat. :-)
> 
> > Performance Impact
> > ------------------
> > * Run time
> > The performance impact at run-time of function reordering varies by
> > workload.
> > Using kcbench, a kernel compilation benchmark, the performance of a
> > kernel
> > build with finer grained KASLR was about 1% slower than a kernel
> > with standard
> > KASLR. Analysis with perf showed a slightly higher percentage of 
> > L1-icache-load-misses. Other workloads were examined as well, with
> > varied
> > results. Some workloads performed significantly worse under
> > FGKASLR, while
> > others stayed the same or were mysteriously better. In general, it
> > will
> > depend on the code flow whether or not finer grained KASLR will
> > impact
> > your workload, and how the underlying code was designed. Because
> > the layout
> > changes per boot, each time a system is rebooted the performance of
> > a workload
> > may change.
> 
> I'd guess that the biggest performance impact comes from tearing
> apart 
> 'groups' of functions that particular workloads are using.
> 
> In that sense it might be worthwile to add a '__kaslr_group'
> function 
> tag to key functions, which would keep certain performance critical 
> functions next to each other.
> 
> This shouldn't really be a problem, as even with generous amount of 
> grouping the number of randomization bits is incredibly large.

So my strategy so far was to try to get a very basic non-performance
optimized fgkaslr mode merged first, then add performance optimized
options as a next step. For example, a user might pass in
fgkaslr="group" to the fgkaslr kernel parameter to select a layout
which groups some things by whatever criteria we want to mitigate some
of the performance impact of full randomization, or they might chose
fgkaslr="full", which just randomizes everything (the current
implementation). If people think it's worth adding the performance
optimizations for the initial merge, I can certainly work on those, but
i thought it might be better to keep it super simple at first.

> 
> > Future work could identify hot areas that may not be randomized and
> > either
> > leave them in the .text section or group them together into a
> > single section
> > that may be randomized. If grouping things together helps, one
> > other thing to
> > consider is that if we could identify text blobs that should be
> > grouped together
> > to benefit a particular code flow, it could be interesting to
> > explore
> > whether this security feature could be also be used as a
> > performance
> > feature if you are interested in optimizing your kernel layout for
> > a
> > particular workload at boot time. Optimizing function layout for a
> > particular
> > workload has been researched and proven effective - for more
> > information
> > read the Facebook paper "Optimizing Function Placement for Large-
> > Scale
> > Data-Center Applications" (see references section below).
> 
> I'm pretty sure the 'grouping' solution would address any real 
> slowdowns.
> 
> I'd also suggest allowing the passing in of a boot-time pseudo-
> random 
> generator seed number, which would allow the creation of a 
> pseudo-randomized but repeatable layout across reboots.

We talked during the RFC stage of porting the chacha20 code to this
early boot stage to use as a prand generator. Ultimately, this means
you now have a secret you have to protect (the seed), and so I've
dropped this for now. I could see maybe having this as a debug option?
I certainly use a prand myself even now when I'm still debugging
functional issues (although the one I use for my own debugging isn't
suitable for merging).

> 
> > Image Size
> > ----------
> > Adding additional section headers as a result of compiling with
> > -ffunction-sections will increase the size of the vmlinux ELF file.
> > With a standard distro config, the resulting vmlinux was increased
> > by
> > about 3%. The compressed image is also increased due to the header
> > files,
> > as well as the extra relocations that must be added. You can expect
> > fgkaslr
> > to increase the size of the compressed image by about 15%.
> 
> What is the increase of the resulting raw kernel image? Additional 
> relocations might increase its size (unless I'm missing something) - 
> it would be nice to measure this effect. I'd expect this to be
> really 
> low.

By raw kernel image, do you mean just what eventually gets copied into
memory after decompression minus the relocation table? If so, this is
almost no difference - the only difference is that there is a little
bit of change in the padding between sections vs what the non-
randomized kernel is because of alignment differences with the new
layout. so you wind up with a few extra bytes give or take.

> 
> vmlinux or compressed kernel size doesn't really matter on x86-64, 
> it's a boot time only expense well within typical system resource 
> limits.
> 
> > Disabling
> > ---------
> > Disabling normal KASLR using the nokaslr command line option also
> > disables
> > fgkaslr. It is also possible to disable fgkaslr separately by
> > booting with
> > fgkaslr=off on the commandline.
> 
> I'd suggest to also add a 'nofgkaslr' boot option if it doesn't yet 
> exist, to keep usage symmetric with kaslr.
> 
> Likewise, there should probably be a 'kaslr=off' option as well.
> 
> The less random our user interfaces are, the better ...
> 
> >  arch/x86/boot/compressed/Makefile             |   9 +-
> >  arch/x86/boot/compressed/fgkaslr.c            | 811
> > ++++++++++++++++++
> >  arch/x86/boot/compressed/kaslr.c              |   4 -
> >  arch/x86/boot/compressed/misc.c               | 157 +++-
> >  arch/x86/boot/compressed/misc.h               |  30 +
> >  arch/x86/boot/compressed/utils.c              |  11 +
> >  arch/x86/boot/compressed/vmlinux.symbols      |  17 +
> >  arch/x86/include/asm/boot.h                   |  15 +-
> >  arch/x86/kernel/vmlinux.lds.S                 |  17 +-
> >  arch/x86/lib/kaslr.c                          |  18 +-
> >  arch/x86/tools/relocs.c                       | 143 ++-
> >  arch/x86/tools/relocs.h                       |   4 +-
> >  arch/x86/tools/relocs_common.c                |  15 +-
> >  include/asm-generic/vmlinux.lds.h             |  18 +-
> >  include/linux/decompress/mm.h                 |  12 +-
> >  include/uapi/linux/elf.h                      |   1 +
> >  init/Kconfig                                  |  26 +
> >  kernel/kallsyms.c                             | 163 +++-
> >  kernel/module.c                               |  81 ++
> >  tools/objtool/elf.c                           |   8 +-
> >  26 files changed, 1670 insertions(+), 85 deletions(-)
> >  create mode 100644 Documentation/security/fgkaslr.rst
> >  create mode 100644 arch/x86/boot/compressed/fgkaslr.c
> >  create mode 100644 arch/x86/boot/compressed/utils.c
> >  create mode 100644 arch/x86/boot/compressed/vmlinux.symbols
> 
> This looks surprisingly lean overall.

Most of the changes outside of fgkaslr.c, module.c, and kallsyms.c were
little tweaks here and there to accommodate using -ffunction-sections
and handling >64K elf sections, otherwise yes, I tried to keep it very
self contained and non-invasive.



  reply index

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-17 16:59 Kristen Carlson Accardi
2020-07-17 16:59 ` [PATCH v4 01/10] objtool: Do not assume order of parent/child functions Kristen Carlson Accardi
2020-07-17 16:59 ` [PATCH v4 02/10] x86: tools/relocs: Support >64K section headers Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 03/10] x86/boot: Allow a "silent" kaslr random byte fetch Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 04/10] x86: Makefile: Add build and config option for CONFIG_FG_KASLR Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 05/10] x86: Make sure _etext includes function sections Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 06/10] x86/tools: Add relative relocs for randomized functions Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 07/10] x86/boot/compressed: Avoid duplicate malloc() implementations Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 08/10] x86: Add support for function granular KASLR Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 09/10] kallsyms: Hide layout Kristen Carlson Accardi
2020-07-20  1:25   ` Kees Cook
2020-07-20 16:59     ` Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 10/10] module: Reorder functions Kristen Carlson Accardi
2020-07-28 17:29   ` Jessica Yu
2020-07-22  9:27 ` [PATCH v4 00/10] Function Granular KASLR Miroslav Benes
2020-07-22 14:39   ` Kees Cook
2020-07-22 14:51     ` Joe Lawrence
2020-07-22 14:56       ` Joe Lawrence
2020-07-22 18:24         ` Kristen Carlson Accardi
2020-07-22 16:07     ` Josh Poimboeuf
2020-07-22 19:42       ` Kees Cook
2020-07-22 19:56         ` Kristen Carlson Accardi
2020-07-22 21:33           ` Josh Poimboeuf
2020-08-21 23:02             ` Kristen Carlson Accardi
2020-08-25 16:16               ` Joe Lawrence
2020-08-28 10:21               ` Miroslav Benes
2020-08-28 19:24                 ` Josh Poimboeuf
2021-01-23 22:59                   ` Fangrui Song
2021-01-25 17:21                     ` Josh Poimboeuf
2020-08-03 11:39   ` Evgenii Shatokhin
2020-08-03 17:45     ` Kees Cook
2020-08-03 18:17       ` Joe Lawrence
2020-08-03 19:38         ` Frank Ch. Eigler
2020-08-03 20:11           ` Kees Cook
2020-08-03 21:12             ` Frank Ch. Eigler
2020-08-03 21:41               ` Kees Cook
2020-08-04  0:48                 ` Frank Ch. Eigler
2020-08-04 17:04         ` Jessica Yu
2020-08-04 18:23 ` Joe Lawrence
2020-08-07 16:38   ` Kristen Carlson Accardi
2020-08-07 17:20     ` Kees Cook
2020-08-10 16:10       ` Kristen Carlson Accardi
2020-08-12 17:18   ` Kristen Carlson Accardi
2020-08-06 15:32 ` Ingo Molnar
2020-08-06 19:24   ` Kristen Carlson Accardi [this message]
2020-08-06 19:27   ` Kees Cook

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=df539dc3a4791d0f873aec9b303f49b64237a6e8.camel@linux.intel.com \
    --to=kristen@linux.intel.com \
    --cc=arjan@linux.intel.com \
    --cc=bp@alien8.de \
    --cc=keescook@chromium.org \
    --cc=kernel-hardening@lists.openwall.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=mingo@redhat.com \
    --cc=rick.p.edgecombe@intel.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git