Re: [PATCH v6 1/1] x86_64: fix KASan shadow region page tables

[Andrey Ryabinin <a.ryabinin@samsung.com>]

On 06/18/2015 12:08 PM, Borislav Petkov wrote:
> On Thu, Jun 18, 2015 at 10:36:52AM +0300, Alexander Popov wrote:
>> Physical addresses in KASan shadow region page tables need fixup similarly
>> to the other page tables. Current code doesn't do it which causes
>> kernel halt if phys_base is not zero.
>> So let's initialize KASan shadow region page tables in kasan_early_init()
>> using __pa_nodebug() which considers phys_base.
>>
>> Signed-off-by: Alexander Popov <alpopov@ptsecurity.com>
>> ---
>>
>> Notes:
>>     Changes from v2:
>>     - move KASan shadow region page tables to BSS;
>>     - use __PAGE_KERNEL flags for describing kasan_zero_page in kasan_zero_pte.
>>     
>>     Changes from v3:
>>     - improve commit message.
>>     
>>     Changes from v4:
>>     - add Andrey's patch which removes faulty clear_page(init_level4_pgt);
>>     - call kasan_map_early_shadow() for early_level4_pgt and init_level4_pgt
>>         in kasan_early_init().
>>     
>>     Changes from v5:
>>     - restore clear_page(init_level4_pgt), but put it just after clear_bss()
>>         to consolidate early KASan initialization;
>>     - remove the comment which stopped bringing much profit to the code
>>         readability. Otherwise describing all the new order dependences
>>         would be too verbose.
>>
>>  arch/x86/include/asm/kasan.h |  8 ++------
>>  arch/x86/kernel/head64.c     | 10 ++++------
>>  arch/x86/kernel/head_64.S    | 29 -----------------------------
>>  arch/x86/mm/kasan_init_64.c  | 36 ++++++++++++++++++++++++++++++++++--
>>  4 files changed, 40 insertions(+), 43 deletions(-)
> 
> So I applied it ontop of rc8 and did enable KASAN, see attached .config.
> 
> Build log contains:
> 
> scripts/Makefile.kasan:23: CONFIG_KASAN: compiler does not support all options. Trying minimal configuration
> 
> and the guest booted up to here:
> 
> ...
> [    0.000000] e820: [mem 0x80000000-0xfeffbfff] available for PCI devices
> [    0.000000] clocksource refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
> [    0.000000] setup_percpu: NR_CPUS:8 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1
> [    0.000000] PERCPU: Embedded 490 pages/cpu @ffff88006b400000 s1970136 r8192 d28712 u2097152
> [    0.000000] pcpu-alloc: s1970136 r8192 d28712 u2097152 alloc=1*2097152
> [    0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7 
> [    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 515944
> [    0.000000] Kernel command line: root=/dev/sda1 resume=/dev/sdb1 debug ignore_loglevel log_buf_len=16M earlyprintk=ttyS0,115200 console=ttyS0,115200 console=tty0 
> [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
> [    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
> [    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
> [    0.000000] xsave: enabled xstate_bv 0x7, cntxt size 0x340 using standard form
> [    0.000000] BUG: unable to handle kernel paging request at fffff94000000083
> [    0.000000] IP: [<ffffffff81219c2b>] __asan_store4+0x2b/0xb0
> [    0.000000] PGD 800000000208b161 PUD 208a063 PMD 2089063 PTE 8000000002088163
> [    0.000000] Oops: 0009 [#1] PREEMPT SMP KASAN
> [    0.000000] Modules linked in:
> [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.1.0-rc8+ #3
> [    0.000000] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.7.5-20140531_083030-gandalf 04/01/2014
> [    0.000000] task: ffffffff81cbd580 ti: ffffffff81ca8000 task.ti: ffffffff81ca8000
> [    0.000000] RIP: 0010:[<ffffffff81219c2b>]  [<ffffffff81219c2b>] __asan_store4+0x2b/0xb0
> [    0.000000] RSP: 0000:ffffffff81cafe18  EFLAGS: 00010082
> [    0.000000] RAX: fffff94000000083 RBX: ffffea0000000440 RCX: 0000000000000004
> [    0.000000] RDX: dffffc0000000000 RSI: 0000000000000004 RDI: ffffea000000041c
> [    0.000000] RBP: ffffffff81cafe48 R08: ffffffff82030ce8 R09: ffffffff81cafee8
> [    0.000000] R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000000f
> [    0.000000] R13: ffffea0000000400 R14: 0000000000000010 R15: 0000000000000001
> [    0.000000] FS:  0000000000000000(0000) GS:ffff88006b400000(0000) knlGS:0000000000000000
> [    0.000000] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [    0.000000] CR2: fffff94000000083 CR3: 0000000001cb8000 CR4: 00000000000406b0
> [    0.000000] Stack:
> [    0.000000]  ffffffff81cafeb8 ffffffff81fcacc0 ffffffff81cafee8 ffffffff81cafee0
> [    0.000000]  ffffffff81cafef0 ffffffff82030ce8 ffffffff81cafe98 ffffffff81f942f8
> [    0.000000]  ffffffff82030cd0 0000000482030d40 0000000000099000 0000000000000099
> [    0.000000] Call Trace:
> [    0.000000]  [<ffffffff81fcacc0>] ? __next_mem_range+0x2cf/0x309
> [    0.000000]  [<ffffffff81f942f8>] __free_pages_bootmem+0x4d/0x10a
> [    0.000000]  [<ffffffff81f98fab>] __free_memory_core+0x79/0x8d
> [    0.000000]  [<ffffffff81f991fe>] free_all_bootmem+0x5f/0xe3
> [    0.000000]  [<ffffffff81f892fc>] mem_init+0xe/0x4a
> [    0.000000]  [<ffffffff81f6ff90>] start_kernel+0x2bb/0x500
> [    0.000000]  [<ffffffff81f6f120>] ? early_idt_handler_array+0x120/0x120
> [    0.000000]  [<ffffffff81f6f4a8>] x86_64_start_reservations+0x2a/0x2c
> [    0.000000]  [<ffffffff81f6f5e3>] x86_64_start_kernel+0x139/0x148
> [    0.000000] Code: 55 48 b8 ff ff ff ff ff 7f ff ff 48 89 e5 48 83 ec 30 48 39 c7 76 59 48 89 f8 48 ba 00 00 00 00 00 fc ff df 48 c1 e8 03 48 01 d0 <66> 83 38 00 75 07 c9 c3 0f 1f 44 00 00 48 8d 4f 03 48 89 ce 48 
> [    0.000000] RIP  [<ffffffff81219c2b>] __asan_store4+0x2b/0xb0
> [    0.000000]  RSP <ffffffff81cafe18>
> [    0.000000] CR2: fffff94000000083
> [    0.000000] ---[ end trace 1e56b2b303866485 ]---
> [    0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
> [    0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task!
> 
> Now, if this really is caused by a compiler shortcoming, I think the
> build should be failed in the scripts/Makefile.kasan check instead of
> continuing.
> 

Whatever compiler you use, it shouldn't crash.
I see on problem which should be fixed by patch below.
Could you please try it?

---
From: Andrey Ryabinin <a.ryabinin@samsung.com>
Subject: [PATCH] x86_64: kasan: flush tlbs after switching cr3

load_cr3() doesn't cause tlb_flush if PGE enabled.
This may cause tons of false positive reports or kernel crash.
To fix this __flush_tlb_all() should be called explicitly
after cr3 changed.

Signed-off-by: Andrey Ryabinin <a.ryabinin@samsung.com>
---
 arch/x86/mm/kasan_init_64.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/arch/x86/mm/kasan_init_64.c b/arch/x86/mm/kasan_init_64.c
index 0e4a05f..5d26642 100644
--- a/arch/x86/mm/kasan_init_64.c
+++ b/arch/x86/mm/kasan_init_64.c
@@ -208,6 +208,7 @@ void __init kasan_init(void)

 	memcpy(early_level4_pgt, init_level4_pgt, sizeof(early_level4_pgt));
 	load_cr3(early_level4_pgt);
+	__flush_tlb_all();

 	clear_pgds(KASAN_SHADOW_START, KASAN_SHADOW_END);

@@ -234,5 +235,6 @@ void __init kasan_init(void)
 	memset(kasan_zero_page, 0, PAGE_SIZE);

 	load_cr3(init_level4_pgt);
+	__flush_tlb_all();
 	init_task.kasan_depth = 0;
 }
-- 
2.4.2