From: Andy Lutomirski <luto@kernel.org>
To: "Kirill A. Shutemov" <kirill@shutemov.name>
Cc: Dmitry Vyukov <dvyukov@google.com>,
Andrey Ryabinin <aryabinin@virtuozzo.com>,
Alexander Potapenko <glider@google.com>,
"Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
Linus Torvalds <torvalds@linux-foundation.org>,
Andrew Morton <akpm@linux-foundation.org>,
"x86@kernel.org" <x86@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@redhat.com>, "H. Peter Anvin" <hpa@zytor.com>,
Andi Kleen <ak@linux.intel.com>,
Dave Hansen <dave.hansen@intel.com>,
linux-arch <linux-arch@vger.kernel.org>,
"linux-mm@kvack.org" <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
kasan-dev <kasan-dev@googlegroups.com>
Subject: Re: KASAN vs. boot-time switching between 4- and 5-level paging
Date: Mon, 10 Jul 2017 13:07:13 -0700 [thread overview]
Message-ID: <CALCETrVJQ_u-agPm8fFHAW1UJY=VLowdbM+gXyjFCb586r0V3g@mail.gmail.com> (raw)
In-Reply-To: <20170710184704.realchrhzpblqqlk@node.shutemov.name>
On Mon, Jul 10, 2017 at 11:47 AM, Kirill A. Shutemov
<kirill@shutemov.name> wrote:
> On Mon, Jul 10, 2017 at 08:56:37AM -0700, Andy Lutomirski wrote:
>>
>>
>> > On Jul 10, 2017, at 7:17 AM, Kirill A. Shutemov <kirill@shutemov.name> wrote:
>> >
>> >> On Mon, Jul 10, 2017 at 02:43:17PM +0200, Dmitry Vyukov wrote:
>> >> On Mon, Jul 10, 2017 at 2:33 PM, Kirill A. Shutemov
>> >> <kirill@shutemov.name> wrote:
>> >>> On Thu, Jun 01, 2017 at 05:56:30PM +0300, Andrey Ryabinin wrote:
>> >>>>> On 05/29/2017 03:46 PM, Andrey Ryabinin wrote:
>> >>>>> On 05/29/2017 02:45 PM, Andrey Ryabinin wrote:
>> >>>>>>>>>> Looks like KASAN will be a problem for boot-time paging mode switching.
>> >>>>>>>>>> It wants to know CONFIG_KASAN_SHADOW_OFFSET at compile-time to pass to
>> >>>>>>>>>> gcc -fasan-shadow-offset=. But this value varies between paging modes...
>> >>>>>>>>>>
>> >>>>>>>>>> I don't see how to solve it. Folks, any ideas?
>> >>>>>>>>>
>> >>>>>>>>> +kasan-dev
>> >>>>>>>>>
>> >>>>>>>>> I wonder if we can use the same offset for both modes. If we use
>> >>>>>>>>> 0xFFDFFC0000000000 as start of shadow for 5 levels, then the same
>> >>>>>>>>> offset that we use for 4 levels (0xdffffc0000000000) will also work
>> >>>>>>>>> for 5 levels. Namely, ending of 5 level shadow will overlap with 4
>> >>>>>>>>> level mapping (both end at 0xfffffbffffffffff), but 5 level mapping
>> >>>>>>>>> extends towards lower addresses. The current 5 level start of shadow
>> >>>>>>>>> is actually close -- 0xffd8000000000000 and it seems that the required
>> >>>>>>>>> space after it is unused at the moment (at least looking at mm.txt).
>> >>>>>>>>> So just try to move it to 0xFFDFFC0000000000?
>> >>>>>>>>>
>> >>>>>>>>
>> >>>>>>>> Yeah, this should work, but note that 0xFFDFFC0000000000 is not PGDIR aligned address. Our init code
>> >>>>>>>> assumes that kasan shadow stars and ends on the PGDIR aligned address.
>> >>>>>>>> Fortunately this is fixable, we'd need two more pages for page tables to map unaligned start/end
>> >>>>>>>> of the shadow.
>> >>>>>>>
>> >>>>>>> I think we can extend the shadow backwards (to the current address),
>> >>>>>>> provided that it does not affect shadow offset that we pass to
>> >>>>>>> compiler.
>> >>>>>>
>> >>>>>> I thought about this. We can round down shadow start to 0xffdf000000000000, but we can't
>> >>>>>> round up shadow end, because in that case shadow would end at 0xffffffffffffffff.
>> >>>>>> So we still need at least one more page to cover unaligned end.
>> >>>>>
>> >>>>> Actually, I'm wrong here. I assumed that we would need an additional page to store p4d entries,
>> >>>>> but in fact we don't need it, as such page should already exist. It's the same last pgd where kernel image
>> >>>>> is mapped.
>> >>>>>
>> >>>>
>> >>>>
>> >>>> Something like bellow might work. It's just a proposal to demonstrate the idea, so some code might look ugly.
>> >>>> And it's only build-tested.
>> >>>
>> >>> [Sorry for loong delay.]
>> >>>
>> >>> The patch works for me for legacy boot. But it breaks EFI boot with
>> >>> 5-level paging. And I struggle to understand why.
>> >>>
>> >>> What I see is many page faults at mm/kasan/kasan.c:758 --
>> >>> "DEFINE_ASAN_LOAD_STORE(4)". Handling one of them I get double-fault at
>> >>> arch/x86/kernel/head_64.S:298 -- "pushq %r14", which ends up with triple
>> >>> fault.
>> >>>
>> >>> Any ideas?
>> >>
>> >>
>> >> Just playing the role of the rubber duck:
>> >> - what is the fault address?
>> >> - is it within the shadow range?
>> >> - was the shadow mapped already?
>> >
>> > I misread trace. The initial fault is at arch/x86/kernel/head_64.S:270,
>> > which is ".endr" in definition of early_idt_handler_array.
>> >
>> > The fault address for all three faults is 0xffffffff7ffffff8, which is
>> > outside shadow range. It's just before kernel text mapping.
>> >
>> > Codewise, it happens in load_ucode_bsp() -- after kasan_early_init(), but
>> > before kasan_init().
>>
>> My theory is that, in 5 level mode, the early IDT code isn't all mapped
>> in the page tables. This could sometimes be papered over by lazy page
>> table setup, but lazy setup can't handle faults in the page fault code
>> or data structures.
>>
>> EFI sometimes uses separate page tables, which could contribute.
>
> As far as I can see all involved code is within the same page:
>
> (gdb) p/x &x86_64_start_kernel
> $1 = 0xffffffff84bad2ae
> (gdb) p/x &early_idt_handler_array
> $2 = 0xffffffff84bad000
> (gdb) p/x &early_idt_handler_common
> $3 = 0xffffffff84bad120
> (gdb) p/x &early_make_pgtable
> $4 = 0xffffffff84bad3b4
>
Can you give the disassembly of the backtrace lines? Blaming the
.endr doesn't make much sense to me.
Or maybe Andrey will figure it out quickly.
next prev parent reply other threads:[~2017-07-10 20:07 UTC|newest]
Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-05-25 20:33 [PATCHv1, RFC 0/8] Boot-time switching between 4- and 5-level paging Kirill A. Shutemov
2017-05-25 20:33 ` [PATCHv1, RFC 1/8] x86/boot/compressed/64: Detect and handle 5-level paging at boot-time Kirill A. Shutemov
2017-05-25 20:33 ` [PATCHv1, RFC 2/8] x86/mm: Make virtual memory layout movable for CONFIG_X86_5LEVEL Kirill A. Shutemov
2017-05-25 20:33 ` [PATCHv1, RFC 3/8] x86/mm: Make PGDIR_SHIFT and PTRS_PER_P4D variable Kirill A. Shutemov
2017-05-25 20:33 ` [PATCHv1, RFC 4/8] x86/mm: Handle boot-time paging mode switching at early boot Kirill A. Shutemov
2017-05-25 20:33 ` [PATCHv1, RFC 5/8] x86/mm: Fold p4d page table layer at runtime Kirill A. Shutemov
2017-05-27 15:09 ` Brian Gerst
2017-05-27 22:46 ` Kirill A. Shutemov
2017-05-27 22:56 ` Brian Gerst
2017-05-25 20:33 ` [PATCHv1, RFC 6/8] x86/mm: Replace compile-time checks for 5-level with runtime-time Kirill A. Shutemov
2017-05-25 20:33 ` [PATCHv1, RFC 7/8] x86/mm: Hacks for boot-time switching between 4- and 5-level paging Kirill A. Shutemov
2017-05-26 22:10 ` KASAN vs. " Kirill A. Shutemov
2017-05-29 10:02 ` Dmitry Vyukov
2017-05-29 11:18 ` Andrey Ryabinin
2017-05-29 11:19 ` Dmitry Vyukov
2017-05-29 11:45 ` Andrey Ryabinin
2017-05-29 12:46 ` Andrey Ryabinin
2017-06-01 14:56 ` Andrey Ryabinin
2017-07-10 12:33 ` Kirill A. Shutemov
2017-07-10 12:43 ` Dmitry Vyukov
2017-07-10 14:17 ` Kirill A. Shutemov
2017-07-10 15:56 ` Andy Lutomirski
2017-07-10 18:47 ` Kirill A. Shutemov
2017-07-10 20:07 ` Andy Lutomirski [this message]
2017-07-10 21:24 ` Kirill A. Shutemov
2017-07-11 0:30 ` Andy Lutomirski
2017-07-11 10:35 ` Kirill A. Shutemov
2017-07-11 15:06 ` Andy Lutomirski
2017-07-11 15:15 ` Andrey Ryabinin
2017-07-11 16:45 ` Andrey Ryabinin
2017-07-11 17:03 ` Kirill A. Shutemov
2017-07-11 17:29 ` Andrey Ryabinin
2017-07-11 19:05 ` Kirill A. Shutemov
2017-07-13 12:58 ` Andrey Ryabinin
2017-07-13 13:52 ` Kirill A. Shutemov
2017-07-13 14:15 ` Kirill A. Shutemov
2017-07-13 14:19 ` Andrey Ryabinin
2017-07-24 12:13 ` Kirill A. Shutemov
2017-07-24 14:07 ` Andrey Ryabinin
2017-07-10 16:57 ` Andrey Ryabinin
2017-05-25 20:33 ` [PATCHv1, RFC 8/8] x86/mm: Allow to boot without la57 if CONFIG_X86_5LEVEL=y Kirill A. Shutemov
2017-05-25 23:24 ` [PATCHv1, RFC 0/8] Boot-time switching between 4- and 5-level paging Linus Torvalds
2017-05-26 0:40 ` Andy Lutomirski
2017-05-26 4:18 ` Kevin Easton
2017-05-26 7:21 ` Andy Lutomirski
2017-05-26 13:00 ` Kirill A. Shutemov
2017-05-26 13:35 ` Andi Kleen
2017-05-26 15:51 ` Linus Torvalds
2017-05-26 15:58 ` Kirill A. Shutemov
2017-05-26 16:13 ` Linus Torvalds
2017-05-26 18:24 ` hpa
2017-05-26 19:23 ` Dave Hansen
2017-05-26 19:36 ` hpa
2017-05-26 19:40 ` hpa
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CALCETrVJQ_u-agPm8fFHAW1UJY=VLowdbM+gXyjFCb586r0V3g@mail.gmail.com' \
--to=luto@kernel.org \
--cc=ak@linux.intel.com \
--cc=akpm@linux-foundation.org \
--cc=aryabinin@virtuozzo.com \
--cc=dave.hansen@intel.com \
--cc=dvyukov@google.com \
--cc=glider@google.com \
--cc=hpa@zytor.com \
--cc=kasan-dev@googlegroups.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=kirill@shutemov.name \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mingo@redhat.com \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).