* Re: [PATCH] lockdep: Add debug printk() for downgrade_write() warning. [not found] ` <e1a38e21-d5fe-dee3-7081-bc1a12965a68@i-love.sakura.ne.jp> @ 2019-01-07 4:19 ` Andrew Morton 2019-01-07 5:58 ` Dmitry Vyukov 0 siblings, 1 reply; 6+ messages in thread From: Andrew Morton @ 2019-01-07 4:19 UTC (permalink / raw) To: Tetsuo Handa Cc: linux-mm, Dmitry Vyukov, Ingo Molnar, Peter Zijlstra, Will Deacon On Sun, 6 Jan 2019 19:56:59 +0900 Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote: > syzbot is frequently hitting downgrade_write(&mm->mmap_sem) warning from > munmap() request, but I don't know why it is happening. Since lockdep is > not printing enough information, let's print more. This patch is meant for > linux-next.git only and will be removed after the problem is solved. > > --- a/kernel/locking/lockdep.c > +++ b/kernel/locking/lockdep.c > @@ -50,6 +50,7 @@ > #include <linux/random.h> > #include <linux/jhash.h> > #include <linux/nmi.h> > +#include <linux/rwsem.h> > > #include <asm/sections.h> > > @@ -3550,6 +3551,24 @@ static int __lock_downgrade(struct lockdep_map *lock, unsigned long ip) > curr->lockdep_depth = i; > curr->curr_chain_key = hlock->prev_chain_key; > > +#if defined(CONFIG_RWSEM_XCHGADD_ALGORITHM) && defined(CONFIG_DEBUG_AID_FOR_SYZBOT) > + if (hlock->read && curr->mm) { > + struct rw_semaphore *sem = container_of(lock, > + struct rw_semaphore, > + dep_map); > + > + if (sem == &curr->mm->mmap_sem) { > +#if defined(CONFIG_RWSEM_SPIN_ON_OWNER) > + pr_warn("mmap_sem: hlock->read=%d count=%ld current=%px, owner=%px\n", > + hlock->read, atomic_long_read(&sem->count), > + curr, READ_ONCE(sem->owner)); > +#else > + pr_warn("mmap_sem: hlock->read=%d count=%ld\n", > + hlock->read, atomic_long_read(&sem->count)); > +#endif > + } > + } > +#endif > WARN(hlock->read, "downgrading a read lock"); > hlock->read = 1; > hlock->acquire_ip = ip; I tossed it in there. But I wonder if anyone is actually running this code. Because --- a/lib/Kconfig.debug~info-task-hung-in-generic_file_write_iter +++ a/lib/Kconfig.debug @@ -2069,6 +2069,12 @@ config IO_STRICT_DEVMEM If in doubt, say Y. +config DEBUG_AID_FOR_SYZBOT + bool "Additional debug code for syzbot" + default n + help + This option is intended for testing by syzbot. + ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] lockdep: Add debug printk() for downgrade_write() warning. @ 2019-01-07 5:58 ` Dmitry Vyukov 0 siblings, 0 replies; 6+ messages in thread From: Dmitry Vyukov @ 2019-01-07 5:58 UTC (permalink / raw) To: Andrew Morton Cc: Tetsuo Handa, Linux-MM, Ingo Molnar, Peter Zijlstra, Will Deacon On Mon, Jan 7, 2019 at 5:19 AM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Sun, 6 Jan 2019 19:56:59 +0900 Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote: > > > syzbot is frequently hitting downgrade_write(&mm->mmap_sem) warning from > > munmap() request, but I don't know why it is happening. Since lockdep is > > not printing enough information, let's print more. This patch is meant for > > linux-next.git only and will be removed after the problem is solved. > > > > --- a/kernel/locking/lockdep.c > > +++ b/kernel/locking/lockdep.c > > @@ -50,6 +50,7 @@ > > #include <linux/random.h> > > #include <linux/jhash.h> > > #include <linux/nmi.h> > > +#include <linux/rwsem.h> > > > > #include <asm/sections.h> > > > > @@ -3550,6 +3551,24 @@ static int __lock_downgrade(struct lockdep_map *lock, unsigned long ip) > > curr->lockdep_depth = i; > > curr->curr_chain_key = hlock->prev_chain_key; > > > > +#if defined(CONFIG_RWSEM_XCHGADD_ALGORITHM) && defined(CONFIG_DEBUG_AID_FOR_SYZBOT) > > + if (hlock->read && curr->mm) { > > + struct rw_semaphore *sem = container_of(lock, > > + struct rw_semaphore, > > + dep_map); > > + > > + if (sem == &curr->mm->mmap_sem) { > > +#if defined(CONFIG_RWSEM_SPIN_ON_OWNER) > > + pr_warn("mmap_sem: hlock->read=%d count=%ld current=%px, owner=%px\n", > > + hlock->read, atomic_long_read(&sem->count), > > + curr, READ_ONCE(sem->owner)); > > +#else > > + pr_warn("mmap_sem: hlock->read=%d count=%ld\n", > > + hlock->read, atomic_long_read(&sem->count)); > > +#endif > > + } > > + } > > +#endif > > WARN(hlock->read, "downgrading a read lock"); > > hlock->read = 1; > > hlock->acquire_ip = ip; > > I tossed it in there. > > But I wonder if anyone is actually running this code. Because > > --- a/lib/Kconfig.debug~info-task-hung-in-generic_file_write_iter > +++ a/lib/Kconfig.debug > @@ -2069,6 +2069,12 @@ config IO_STRICT_DEVMEM > > If in doubt, say Y. > > +config DEBUG_AID_FOR_SYZBOT > + bool "Additional debug code for syzbot" > + default n > + help > + This option is intended for testing by syzbot. > + Yes, syzbot always defines this option: https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L14 https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kmsan.config#L9 It's meant specifically for such cases. Tetsuo already got some useful information for past bugs using this feature. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] lockdep: Add debug printk() for downgrade_write() warning. @ 2019-01-07 5:58 ` Dmitry Vyukov 0 siblings, 0 replies; 6+ messages in thread From: Dmitry Vyukov @ 2019-01-07 5:58 UTC (permalink / raw) To: Andrew Morton Cc: Tetsuo Handa, Linux-MM, Ingo Molnar, Peter Zijlstra, Will Deacon On Mon, Jan 7, 2019 at 5:19 AM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Sun, 6 Jan 2019 19:56:59 +0900 Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp> wrote: > > > syzbot is frequently hitting downgrade_write(&mm->mmap_sem) warning from > > munmap() request, but I don't know why it is happening. Since lockdep is > > not printing enough information, let's print more. This patch is meant for > > linux-next.git only and will be removed after the problem is solved. > > > > --- a/kernel/locking/lockdep.c > > +++ b/kernel/locking/lockdep.c > > @@ -50,6 +50,7 @@ > > #include <linux/random.h> > > #include <linux/jhash.h> > > #include <linux/nmi.h> > > +#include <linux/rwsem.h> > > > > #include <asm/sections.h> > > > > @@ -3550,6 +3551,24 @@ static int __lock_downgrade(struct lockdep_map *lock, unsigned long ip) > > curr->lockdep_depth = i; > > curr->curr_chain_key = hlock->prev_chain_key; > > > > +#if defined(CONFIG_RWSEM_XCHGADD_ALGORITHM) && defined(CONFIG_DEBUG_AID_FOR_SYZBOT) > > + if (hlock->read && curr->mm) { > > + struct rw_semaphore *sem = container_of(lock, > > + struct rw_semaphore, > > + dep_map); > > + > > + if (sem == &curr->mm->mmap_sem) { > > +#if defined(CONFIG_RWSEM_SPIN_ON_OWNER) > > + pr_warn("mmap_sem: hlock->read=%d count=%ld current=%px, owner=%px\n", > > + hlock->read, atomic_long_read(&sem->count), > > + curr, READ_ONCE(sem->owner)); > > +#else > > + pr_warn("mmap_sem: hlock->read=%d count=%ld\n", > > + hlock->read, atomic_long_read(&sem->count)); > > +#endif > > + } > > + } > > +#endif > > WARN(hlock->read, "downgrading a read lock"); > > hlock->read = 1; > > hlock->acquire_ip = ip; > > I tossed it in there. > > But I wonder if anyone is actually running this code. Because > > --- a/lib/Kconfig.debug~info-task-hung-in-generic_file_write_iter > +++ a/lib/Kconfig.debug > @@ -2069,6 +2069,12 @@ config IO_STRICT_DEVMEM > > If in doubt, say Y. > > +config DEBUG_AID_FOR_SYZBOT > + bool "Additional debug code for syzbot" > + default n > + help > + This option is intended for testing by syzbot. > + Yes, syzbot always defines this option: https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L14 https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kmsan.config#L9 It's meant specifically for such cases. Tetsuo already got some useful information for past bugs using this feature. ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] lockdep: Add debug printk() for downgrade_write() warning. 2019-01-07 5:58 ` Dmitry Vyukov (?) @ 2019-01-09 11:47 ` Tetsuo Handa 2019-01-09 18:22 ` Waiman Long -1 siblings, 1 reply; 6+ messages in thread From: Tetsuo Handa @ 2019-01-09 11:47 UTC (permalink / raw) To: Peter Zijlstra Cc: Dmitry Vyukov, Andrew Morton, Linux-MM, Ingo Molnar, Will Deacon Hello, Peter. We got two reports. Neither RWSEM_READER_OWNED nor RWSEM_ANONYMOUSLY_OWNED is set, and (presumably) sem->owner == current is true, but count is -1. What does this mean? https://syzkaller.appspot.com/text?tag=CrashLog&x=169dbb9b400000 [ 2580.337550][ T3645] mmap_sem: hlock->read=1 count=-4294967295 current=ffff888050e04140, owner=ffff888050e04140 [ 2580.353526][ T3645] ------------[ cut here ]------------ [ 2580.367859][ T3645] downgrading a read lock [ 2580.367935][ T3645] WARNING: CPU: 1 PID: 3645 at kernel/locking/lockdep.c:3572 lock_downgrade+0x35d/0xbe0 [ 2580.382206][ T3645] Kernel panic - not syncing: panic_on_warn set ... https://syzkaller.appspot.com/text?tag=CrashLog&x=1542da4f400000 [ 386.342585][T16698] mmap_sem: hlock->read=1 count=-4294967295 current=ffff8880512ae180, owner=ffff8880512ae180 [ 386.348586][T16698] ------------[ cut here ]------------ [ 386.357203][T16698] downgrading a read lock [ 386.357294][T16698] WARNING: CPU: 1 PID: 16698 at kernel/locking/lockdep.c:3572 lock_downgrade+0x35d/0xbe0 [ 386.372148][T16698] Kernel panic - not syncing: panic_on_warn set ... ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] lockdep: Add debug printk() for downgrade_write() warning. 2019-01-09 11:47 ` Tetsuo Handa @ 2019-01-09 18:22 ` Waiman Long 0 siblings, 0 replies; 6+ messages in thread From: Waiman Long @ 2019-01-09 18:22 UTC (permalink / raw) To: Tetsuo Handa, Peter Zijlstra Cc: Dmitry Vyukov, Andrew Morton, Linux-MM, Ingo Molnar, Will Deacon On 01/09/2019 06:47 AM, Tetsuo Handa wrote: > Hello, Peter. > > We got two reports. Neither RWSEM_READER_OWNED nor RWSEM_ANONYMOUSLY_OWNED is set, and > (presumably) sem->owner == current is true, but count is -1. What does this mean? > > https://syzkaller.appspot.com/text?tag=CrashLog&x=169dbb9b400000 > > [ 2580.337550][ T3645] mmap_sem: hlock->read=1 count=-4294967295 current=ffff888050e04140, owner=ffff888050e04140 > [ 2580.353526][ T3645] ------------[ cut here ]------------ > [ 2580.367859][ T3645] downgrading a read lock > [ 2580.367935][ T3645] WARNING: CPU: 1 PID: 3645 at kernel/locking/lockdep.c:3572 lock_downgrade+0x35d/0xbe0 > [ 2580.382206][ T3645] Kernel panic - not syncing: panic_on_warn set ... > > https://syzkaller.appspot.com/text?tag=CrashLog&x=1542da4f400000 > > [ 386.342585][T16698] mmap_sem: hlock->read=1 count=-4294967295 current=ffff8880512ae180, owner=ffff8880512ae180 > [ 386.348586][T16698] ------------[ cut here ]------------ > [ 386.357203][T16698] downgrading a read lock > [ 386.357294][T16698] WARNING: CPU: 1 PID: 16698 at kernel/locking/lockdep.c:3572 lock_downgrade+0x35d/0xbe0 > [ 386.372148][T16698] Kernel panic - not syncing: panic_on_warn set ... > > A call to up_read() while the count was previously 0 will cause the count to become -1. I would suggest adding some debug code in up_read() to catch the offending caller. Cheers, Longman ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] lockdep: Add debug printk() for downgrade_write() warning. 2019-01-07 5:58 ` Dmitry Vyukov (?) (?) @ 2019-01-10 10:12 ` Tetsuo Handa -1 siblings, 0 replies; 6+ messages in thread From: Tetsuo Handa @ 2019-01-10 10:12 UTC (permalink / raw) To: Dmitry Vyukov, Andrew Morton, Peter Zijlstra Cc: Linux-MM, Ingo Molnar, Will Deacon On 2019/01/07 14:58, Dmitry Vyukov wrote: > On Mon, Jan 7, 2019 at 5:19 AM Andrew Morton <akpm@linux-foundation.org> wrote: >> I tossed it in there. >> >> But I wonder if anyone is actually running this code. Because >> >> --- a/lib/Kconfig.debug~info-task-hung-in-generic_file_write_iter >> +++ a/lib/Kconfig.debug >> @@ -2069,6 +2069,12 @@ config IO_STRICT_DEVMEM >> >> If in doubt, say Y. >> >> +config DEBUG_AID_FOR_SYZBOT >> + bool "Additional debug code for syzbot" >> + default n >> + help >> + This option is intended for testing by syzbot. >> + > > > Yes, syzbot always defines this option: > > https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kasan.config#L14 > https://github.com/google/syzkaller/blob/master/dashboard/config/upstream-kmsan.config#L9 > > It's meant specifically for such cases. > > Tetsuo already got some useful information for past bugs using this feature. > Andrew, you can drop this patch, for a patch that fixes this problem is already available at https://lkml.kernel.org/r/1547093005-26085-1-git-send-email-longman@redhat.com . Peter, please apply the fix from Waiman Long. ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-01-10 10:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <1546771139-9349-1-git-send-email-penguin-kernel@I-love.SAKURA.ne.jp>
[not found] ` <e1a38e21-d5fe-dee3-7081-bc1a12965a68@i-love.sakura.ne.jp>
2019-01-07 4:19 ` [PATCH] lockdep: Add debug printk() for downgrade_write() warning Andrew Morton
2019-01-07 5:58 ` Dmitry Vyukov
2019-01-07 5:58 ` Dmitry Vyukov
2019-01-09 11:47 ` Tetsuo Handa
2019-01-09 18:22 ` Waiman Long
2019-01-10 10:12 ` Tetsuo Handa
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.