From: Evgenii Shatokhin <eshatokhin@virtuozzo.com>
To: Kristen Carlson Accardi <kristen@linux.intel.com>
Cc: Miroslav Benes <mbenes@suse.cz>,
keescook@chromium.org, tglx@linutronix.de, mingo@redhat.com,
bp@alien8.de, arjan@linux.intel.com, x86@kernel.org,
linux-kernel@vger.kernel.org,
kernel-hardening@lists.openwall.com, rick.p.edgecombe@intel.com,
live-patching@vger.kernel.org,
Joe Lawrence <joe.lawrence@redhat.com>,
Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v4 00/10] Function Granular KASLR
Date: Mon, 3 Aug 2020 14:39:32 +0300 [thread overview]
Message-ID: <e9c4d88b-86db-47e9-4299-3fac45a7e3fd@virtuozzo.com> (raw)
In-Reply-To: <alpine.LSU.2.21.2007221122110.10163@pobox.suse.cz>
Hi,
> On Fri, 17 Jul 2020, Kristen Carlson Accardi wrote:
>
>> Function Granular Kernel Address Space Layout Randomization (fgkaslr)
>> ---------------------------------------------------------------------
>>
>> This patch set is an implementation of finer grained kernel address space
>> randomization. It rearranges your kernel code at load time
>> on a per-function level granularity, with only around a second added to
>> boot time.
>
> [...]
>> Modules
>> -------
>> Modules are randomized similarly to the rest of the kernel by shuffling
>> the sections at load time prior to moving them into memory. The module must
>> also have been build with the -ffunction-sections compiler option.
It seems, a couple more adjustments are needed in the module loader code.
With function granular KASLR, modules will have lots of ELF sections due
to -ffunction-sections.
On my x86_64 system with kernel 5.8-rc7 with FG KASLR patches, for
example, xfs.ko has 4849 ELF sections total, 2428 of these are loaded
and shown in /sys/module/xfs/sections/.
There are at least 2 places where high-order memory allocations might
happen during module loading. Such allocations may fail if memory is
fragmented, while physically contiguous memory areas are not really
needed there. I suggest to switch to kvmalloc/kvfree there.
1. kernel/module.c, randomize_text():
Elf_Shdr **text_list;
...
int max_sections = info->hdr->e_shnum;
...
text_list = kmalloc_array(max_sections, sizeof(*text_list), GFP_KERNEL);
The size of the allocated memory area is (8 * total_number_of_sections),
if I understand it right, which is 38792 for xfs.ko, a 4th order allocation.
2. kernel/module.c, mod_sysfs_setup() => add_sect_attrs().
This allocation can be larger than the first one.
We found this issue with livepatch modules some time ago (these modules
are already built with -ffunction-sections) [1], but, with FG KASLR, it
affects all kernel modules. Large ones like xfs.ko, btrfs.ko, etc.,
could suffer the most from it.
When a module is loaded sysfs attributes are created for its ELF
sections (visible as /sys/module/<module_name>/sections/*). and contain
the start addresses of these ELF sections. A single memory chunk is
allocated
for all these:
size[0] = ALIGN(sizeof(*sect_attrs)
+ nloaded * sizeof(sect_attrs->attrs[0]),
sizeof(sect_attrs->grp.attrs[0]));
size[1] = (nloaded + 1) * sizeof(sect_attrs->grp.attrs[0]);
sect_attrs = kzalloc(size[0] + size[1], GFP_KERNEL);
'nloaded' is the number of loaded ELF section in the module.
For the kernel 5.8-rc7 on my system, the total size is 56 + 72 *
nloaded, which is 174872 for xfs.ko, 43 pages, 6th order allocation.
I enabled 'mm_page_alloc' tracepoint with filter 'order > 3' to confirm
the issue and, indeed, got these two allocations when modprobe'ing xfs:
----------------------------
/sys/kernel/debug/tracing/trace:
modprobe-1509 <...>: mm_page_alloc: <...> order=4
migratetype=0 gfp_flags=GFP_KERNEL|__GFP_COMP
modprobe-1509 <stack trace>
=> __alloc_pages_nodemask
=> alloc_pages_current
=> kmalloc_order
=> kmalloc_order_trace
=> __kmalloc
=> load_module
modprobe-1509 <...>: mm_page_alloc: <...> order=6
migratetype=0 gfp_flags=GFP_KERNEL|__GFP_COMP|__GFP_ZERO
modprobe-1509 <stack trace>
=> __alloc_pages_nodemask
=> alloc_pages_current
=> kmalloc_order
=> kmalloc_order_trace
=> __kmalloc
=> mod_sysfs_setup
=> load_module
----------------------------
I suppose, something like this can be used as workaround:
* for randomize_text():
-----------
diff --git a/kernel/module.c b/kernel/module.c
index 0f4f4e567a42..a2473db1d0a3 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -2433,7 +2433,7 @@ static void randomize_text(struct module *mod,
struct load_info *info)
if (sec == 0)
return;
- text_list = kmalloc_array(max_sections, sizeof(*text_list), GFP_KERNEL);
+ text_list = kvmalloc_array(max_sections, sizeof(*text_list), GFP_KERNEL);
if (!text_list)
return;
@@ -2466,7 +2466,7 @@ static void randomize_text(struct module *mod,
struct load_info *info)
shdr->sh_entsize = get_offset(mod, &size, shdr, 0);
}
- kfree(text_list);
+ kvfree(text_list);
}
/* Lay out the SHF_ALLOC sections in a way not dissimilar to how ld
-----------
* for add_sect_attrs():
-----------
diff --git a/kernel/module.c b/kernel/module.c
index 0f4f4e567a42..a2473db1d0a3 100644
--- a/kernel/module.c
+++ b/kernel/module.c
@@ -1541,7 +1541,7 @@ static void free_sect_attrs(struct
module_sect_attrs *sect_attrs)
for (section = 0; section < sect_attrs->nsections; section++)
kfree(sect_attrs->attrs[section].battr.attr.name);
- kfree(sect_attrs);
+ kvfree(sect_attrs);
}
static void add_sect_attrs(struct module *mod, const struct load_info
*info)
@@ -1558,7 +1558,7 @@ static void add_sect_attrs(struct module *mod,
const struct load_info *info)
size[0] = ALIGN(struct_size(sect_attrs, attrs, nloaded),
sizeof(sect_attrs->grp.bin_attrs[0]));
size[1] = (nloaded + 1) * sizeof(sect_attrs->grp.bin_attrs[0]);
- sect_attrs = kzalloc(size[0] + size[1], GFP_KERNEL);
+ sect_attrs = kvzalloc(size[0] + size[1], GFP_KERNEL);
if (sect_attrs == NULL)
return;
-----------
[1] https://github.com/dynup/kpatch/pull/1131
Regards,
Evgenii
next prev parent reply other threads:[~2020-08-03 12:03 UTC|newest]
Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-07-17 16:59 [PATCH v4 00/10] Function Granular KASLR Kristen Carlson Accardi
2020-07-17 16:59 ` [PATCH v4 01/10] objtool: Do not assume order of parent/child functions Kristen Carlson Accardi
2020-07-17 16:59 ` [PATCH v4 02/10] x86: tools/relocs: Support >64K section headers Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 03/10] x86/boot: Allow a "silent" kaslr random byte fetch Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 04/10] x86: Makefile: Add build and config option for CONFIG_FG_KASLR Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 05/10] x86: Make sure _etext includes function sections Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 06/10] x86/tools: Add relative relocs for randomized functions Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 07/10] x86/boot/compressed: Avoid duplicate malloc() implementations Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 08/10] x86: Add support for function granular KASLR Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 09/10] kallsyms: Hide layout Kristen Carlson Accardi
2020-07-20 1:25 ` Kees Cook
2020-07-20 16:59 ` Kristen Carlson Accardi
2020-07-17 17:00 ` [PATCH v4 10/10] module: Reorder functions Kristen Carlson Accardi
2020-07-28 17:29 ` Jessica Yu
2020-07-22 9:27 ` [PATCH v4 00/10] Function Granular KASLR Miroslav Benes
2020-07-22 14:39 ` Kees Cook
2020-07-22 14:51 ` Joe Lawrence
2020-07-22 14:56 ` Joe Lawrence
2020-07-22 18:24 ` Kristen Carlson Accardi
2020-07-22 16:07 ` Josh Poimboeuf
2020-07-22 19:42 ` Kees Cook
2020-07-22 19:56 ` Kristen Carlson Accardi
2020-07-22 21:33 ` Josh Poimboeuf
2020-08-21 23:02 ` Kristen Carlson Accardi
2020-08-25 16:16 ` Joe Lawrence
2020-08-28 10:21 ` Miroslav Benes
2020-08-28 19:24 ` Josh Poimboeuf
2021-01-23 22:59 ` Fangrui Song
2021-01-25 17:21 ` Josh Poimboeuf
2020-08-03 11:39 ` Evgenii Shatokhin [this message]
2020-08-03 17:45 ` Kees Cook
2020-08-03 18:17 ` Joe Lawrence
2020-08-03 19:38 ` Frank Ch. Eigler
2020-08-03 20:11 ` Kees Cook
2020-08-03 21:12 ` Frank Ch. Eigler
2020-08-03 21:41 ` Kees Cook
2020-08-04 0:48 ` Frank Ch. Eigler
2020-08-04 17:04 ` Jessica Yu
2020-08-04 18:23 ` Joe Lawrence
2020-08-07 16:38 ` Kristen Carlson Accardi
2020-08-07 17:20 ` Kees Cook
2020-08-10 16:10 ` Kristen Carlson Accardi
2020-08-12 17:18 ` Kristen Carlson Accardi
2020-08-06 15:32 ` Ingo Molnar
2020-08-06 19:24 ` Kristen Carlson Accardi
2020-08-06 19:27 ` Kees Cook
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e9c4d88b-86db-47e9-4299-3fac45a7e3fd@virtuozzo.com \
--to=eshatokhin@virtuozzo.com \
--cc=arjan@linux.intel.com \
--cc=bp@alien8.de \
--cc=joe.lawrence@redhat.com \
--cc=jpoimboe@redhat.com \
--cc=keescook@chromium.org \
--cc=kernel-hardening@lists.openwall.com \
--cc=kristen@linux.intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=live-patching@vger.kernel.org \
--cc=mbenes@suse.cz \
--cc=mingo@redhat.com \
--cc=rick.p.edgecombe@intel.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).