From: Marco Elver <elver@google.com>
To: Jann Horn <jannh@google.com>
Cc: Andrew Morton <akpm@linux-foundation.org>,
Alexander Potapenko <glider@google.com>,
"H . Peter Anvin" <hpa@zytor.com>,
"Paul E . McKenney" <paulmck@kernel.org>,
Andrey Konovalov <andreyknvl@google.com>,
Andrey Ryabinin <aryabinin@virtuozzo.com>,
Andy Lutomirski <luto@kernel.org>, Borislav Petkov <bp@alien8.de>,
Catalin Marinas <catalin.marinas@arm.com>,
Christoph Lameter <cl@linux.com>,
Dave Hansen <dave.hansen@linux.intel.com>,
David Rientjes <rientjes@google.com>,
Dmitry Vyukov <dvyukov@google.com>,
Eric Dumazet <edumazet@google.com>,
Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
Hillf Danton <hdanton@sina.com>, Ingo Molnar <mingo@redhat.com>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Jonathan Corbet <corbet@lwn.net>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Kees Cook <keescook@chromium.org>,
Mark Rutland <mark.rutland@arm.com>,
Pekka Enberg <penberg@kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
SeongJae Park <sjpark@amazon.com>,
Thomas Gleixner <tglx@linutronix.de>,
Vlastimil Babka <vbabka@suse.cz>, Will Deacon <will@kernel.org>,
"the arch/x86 maintainers" <x86@kernel.org>,
"open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
kernel list <linux-kernel@vger.kernel.org>,
kasan-dev <kasan-dev@googlegroups.com>,
Linux ARM <linux-arm-kernel@lists.infradead.org>,
Linux-MM <linux-mm@kvack.org>
Subject: Re: [PATCH v4 02/11] x86, kfence: enable KFENCE for x86
Date: Fri, 9 Oct 2020 19:40:20 +0200 [thread overview]
Message-ID: <20201009174020.GA2263081@elver.google.com> (raw)
In-Reply-To: <CANpmjNOZtkFcyL8FTRTZ6j2yqCOb2Hgsy8eF8n5zgd7mDYezkw@mail.gmail.com>
On Wed, Oct 07, 2020 at 04:41PM +0200, Marco Elver wrote:
> On Wed, 7 Oct 2020 at 16:15, Jann Horn <jannh@google.com> wrote:
[...]
> > > > > + return false;
> > > > > +
> > > > > + if (protect)
> > > > > + set_pte(pte, __pte(pte_val(*pte) & ~_PAGE_PRESENT));
> > > > > + else
> > > > > + set_pte(pte, __pte(pte_val(*pte) | _PAGE_PRESENT));
> > > >
> > > > Hmm... do we have this helper (instead of using the existing helpers
> > > > for modifying memory permissions) to work around the allocation out of
> > > > the data section?
> > >
> > > I just played around with using the set_memory.c functions, to remind
> > > myself why this didn't work. I experimented with using
> > > set_memory_{np,p}() functions; set_memory_p() isn't implemented, but
> > > is easily added (which I did for below experiment). However, this
> > > didn't quite work:
> > [...]
> > > For one, smp_call_function_many_cond() doesn't want to be called with
> > > interrupts disabled, and we may very well get a KFENCE allocation or
> > > page fault with interrupts disabled / within interrupts.
> > >
> > > Therefore, to be safe, we should avoid IPIs.
> >
> > set_direct_map_invalid_noflush() does that, too, I think? And that's
> > already implemented for both arm64 and x86.
>
> Sure, that works.
>
> We still want the flush_tlb_one_kernel(), at least so the local CPU's
> TLB is flushed.
Nope, sorry, set_direct_map_invalid_noflush() does not work -- this
results in potential deadlock.
================================
WARNING: inconsistent lock state
5.9.0-rc4+ #2 Not tainted
--------------------------------
inconsistent {SOFTIRQ-ON-W} -> {IN-SOFTIRQ-W} usage.
ksoftirqd/1/16 [HC0[0]:SC1[1]:HE1:SE0] takes:
ffffffff89fcf9b8 (cpa_lock){+.?.}-{2:2}, at: spin_lock include/linux/spinlock.h:354 [inline]
ffffffff89fcf9b8 (cpa_lock){+.?.}-{2:2}, at: __change_page_attr_set_clr+0x1b0/0x2510 arch/x86/mm/pat/set_memory.c:1658
{SOFTIRQ-ON-W} state was registered at:
lock_acquire+0x1f3/0xae0 kernel/locking/lockdep.c:5006
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:354 [inline]
__change_page_attr_set_clr+0x1b0/0x2510 arch/x86/mm/pat/set_memory.c:1658
change_page_attr_set_clr+0x333/0x500 arch/x86/mm/pat/set_memory.c:1752
change_page_attr_set arch/x86/mm/pat/set_memory.c:1782 [inline]
set_memory_nx+0xb2/0x110 arch/x86/mm/pat/set_memory.c:1930
free_init_pages+0x73/0xc0 arch/x86/mm/init.c:876
alternative_instructions+0x155/0x1a4 arch/x86/kernel/alternative.c:738
check_bugs+0x1bd0/0x1c77 arch/x86/kernel/cpu/bugs.c:140
start_kernel+0x486/0x4b6 init/main.c:1042
secondary_startup_64+0xa4/0xb0 arch/x86/kernel/head_64.S:243
irq event stamp: 14564
hardirqs last enabled at (14564): [<ffffffff8828cadf>] __raw_spin_unlock_irqrestore include/linux/spinlock_api_smp.h:160 [inline]
hardirqs last enabled at (14564): [<ffffffff8828cadf>] _raw_spin_unlock_irqrestore+0x6f/0x90 kernel/locking/spinlock.c:191
hardirqs last disabled at (14563): [<ffffffff8828d239>] __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:108 [inline]
hardirqs last disabled at (14563): [<ffffffff8828d239>] _raw_spin_lock_irqsave+0xa9/0xce kernel/locking/spinlock.c:159
softirqs last enabled at (14486): [<ffffffff8147fcff>] run_ksoftirqd kernel/softirq.c:652 [inline]
softirqs last enabled at (14486): [<ffffffff8147fcff>] run_ksoftirqd+0xcf/0x170 kernel/softirq.c:644
softirqs last disabled at (14491): [<ffffffff8147fcff>] run_ksoftirqd kernel/softirq.c:652 [inline]
softirqs last disabled at (14491): [<ffffffff8147fcff>] run_ksoftirqd+0xcf/0x170 kernel/softirq.c:644
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(cpa_lock);
<Interrupt>
lock(cpa_lock);
*** DEADLOCK ***
1 lock held by ksoftirqd/1/16:
#0: ffffffff8a067e20 (rcu_callback){....}-{0:0}, at: rcu_do_batch kernel/rcu/tree.c:2418 [inline]
#0: ffffffff8a067e20 (rcu_callback){....}-{0:0}, at: rcu_core+0x55d/0x1130 kernel/rcu/tree.c:2656
stack backtrace:
CPU: 1 PID: 16 Comm: ksoftirqd/1 Not tainted 5.9.0-rc4+ #2
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1 04/01/2014
Call Trace:
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x198/0x1fd lib/dump_stack.c:118
print_usage_bug kernel/locking/lockdep.c:3350 [inline]
valid_state kernel/locking/lockdep.c:3361 [inline]
mark_lock_irq kernel/locking/lockdep.c:3575 [inline]
mark_lock.cold+0x12/0x17 kernel/locking/lockdep.c:4006
mark_usage kernel/locking/lockdep.c:3905 [inline]
__lock_acquire+0x1159/0x5780 kernel/locking/lockdep.c:4380
lock_acquire+0x1f3/0xae0 kernel/locking/lockdep.c:5006
__raw_spin_lock include/linux/spinlock_api_smp.h:142 [inline]
_raw_spin_lock+0x2a/0x40 kernel/locking/spinlock.c:151
spin_lock include/linux/spinlock.h:354 [inline]
__change_page_attr_set_clr+0x1b0/0x2510 arch/x86/mm/pat/set_memory.c:1658
__set_pages_np arch/x86/mm/pat/set_memory.c:2184 [inline]
set_direct_map_invalid_noflush+0xd2/0x110 arch/x86/mm/pat/set_memory.c:2189
kfence_protect_page arch/x86/include/asm/kfence.h:62 [inline]
kfence_protect+0x10e/0x120 mm/kfence/core.c:124
kfence_guarded_free+0x380/0x880 mm/kfence/core.c:375
rcu_do_batch kernel/rcu/tree.c:2428 [inline]
rcu_core+0x5ca/0x1130 kernel/rcu/tree.c:2656
__do_softirq+0x1f8/0xb23 kernel/softirq.c:298
run_ksoftirqd kernel/softirq.c:652 [inline]
run_ksoftirqd+0xcf/0x170 kernel/softirq.c:644
smpboot_thread_fn+0x655/0x9e0 kernel/smpboot.c:165
kthread+0x3b5/0x4a0 kernel/kthread.c:292
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:294
next prev parent reply other threads:[~2020-10-09 17:41 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-09-29 13:38 [PATCH v4 00/11] KFENCE: A low-overhead sampling-based memory safety error detector Marco Elver
2020-09-29 13:38 ` [PATCH v4 01/11] mm: add Kernel Electric-Fence infrastructure Marco Elver
2020-10-02 6:33 ` Jann Horn
2020-10-02 7:53 ` Jann Horn
2020-10-02 14:22 ` Dmitry Vyukov
2020-10-02 15:06 ` Mark Rutland
2020-10-02 18:27 ` Jann Horn
2020-10-05 18:59 ` Marco Elver
2020-10-02 17:19 ` Marco Elver
2020-10-02 19:31 ` Jann Horn
2020-10-02 21:12 ` Marco Elver
2020-10-02 21:28 ` Marco Elver
2020-10-02 22:27 ` Jann Horn
2020-10-12 14:20 ` Marco Elver
2020-09-29 13:38 ` [PATCH v4 02/11] x86, kfence: enable KFENCE for x86 Marco Elver
2020-10-02 5:45 ` Jann Horn
2020-10-07 13:08 ` Marco Elver
2020-10-07 14:14 ` Jann Horn
2020-10-07 14:41 ` Marco Elver
2020-10-09 17:40 ` Marco Elver [this message]
2020-10-02 6:08 ` Jann Horn
2020-09-29 13:38 ` [PATCH v4 03/11] arm64, kfence: enable KFENCE for ARM64 Marco Elver
2020-10-02 6:47 ` Jann Horn
2020-10-02 14:18 ` Marco Elver
2020-10-02 16:10 ` Jann Horn
2020-09-29 13:38 ` [PATCH v4 04/11] mm, kfence: insert KFENCE hooks for SLAB Marco Elver
2020-09-29 13:38 ` [PATCH v4 05/11] mm, kfence: insert KFENCE hooks for SLUB Marco Elver
2020-10-02 7:07 ` Jann Horn
2020-10-05 9:29 ` Alexander Potapenko
2020-09-29 13:38 ` [PATCH v4 06/11] kfence, kasan: make KFENCE compatible with KASAN Marco Elver
2020-09-29 13:38 ` [PATCH v4 07/11] kfence, kmemleak: make KFENCE compatible with KMEMLEAK Marco Elver
2020-09-29 13:38 ` [PATCH v4 08/11] kfence, lockdep: make KFENCE compatible with lockdep Marco Elver
2020-09-29 13:38 ` [PATCH v4 09/11] kfence, Documentation: add KFENCE documentation Marco Elver
2020-09-29 13:38 ` [PATCH v4 10/11] kfence: add test suite Marco Elver
2020-09-29 13:38 ` [PATCH v4 11/11] MAINTAINERS: Add entry for KFENCE Marco Elver
2020-09-29 14:21 ` SeongJae Park
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20201009174020.GA2263081@elver.google.com \
--to=elver@google.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=akpm@linux-foundation.org \
--cc=andreyknvl@google.com \
--cc=aryabinin@virtuozzo.com \
--cc=bp@alien8.de \
--cc=catalin.marinas@arm.com \
--cc=cl@linux.com \
--cc=corbet@lwn.net \
--cc=dave.hansen@linux.intel.com \
--cc=dvyukov@google.com \
--cc=edumazet@google.com \
--cc=glider@google.com \
--cc=gregkh@linuxfoundation.org \
--cc=hdanton@sina.com \
--cc=hpa@zytor.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=jannh@google.com \
--cc=kasan-dev@googlegroups.com \
--cc=keescook@chromium.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=luto@kernel.org \
--cc=mark.rutland@arm.com \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=penberg@kernel.org \
--cc=peterz@infradead.org \
--cc=rientjes@google.com \
--cc=sjpark@amazon.com \
--cc=tglx@linutronix.de \
--cc=vbabka@suse.cz \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).