From: Thomas Gleixner <tglx@linutronix.de>
To: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Sebastian Sewior <bigeasy@linutronix.de>,
"Paul E. McKenney" <paulmck@linux.ibm.com>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@kernel.org>,
Martin Schwidefsky <schwidefsky@de.ibm.com>,
LKML <linux-kernel@vger.kernel.org>,
linux-s390@vger.kernel.org, Stefan Liebler <stli@linux.ibm.com>
Subject: Re: WARN_ON_ONCE(!new_owner) within wake_futex_pi() triggerede
Date: Sun, 3 Feb 2019 17:30:39 +0100 (CET) [thread overview]
Message-ID: <alpine.DEB.2.21.1902031718170.8200@nanos.tec.linutronix.de> (raw)
In-Reply-To: <20190202112006.GB3381@osiris>
On Sat, 2 Feb 2019, Heiko Carstens wrote:
> On Sat, Feb 02, 2019 at 11:14:27AM +0100, Thomas Gleixner wrote:
> > On Sat, 2 Feb 2019, Heiko Carstens wrote:
> > So after the unlock @timestamp 337.215675 the kernel does not deal with
> > that futex at all until the failed lock attempt where it rightfully rejects
> > the attempt due to the alleged owner being gone.
> >
> > So this looks more like user space doing something stupid...
> >
> > As we talked about the missing barriers before, I just looked at
> > pthread_mutex_trylock() and that does still:
> >
> > if (robust)
> > {
> > ENQUEUE_MUTEX_PI (mutex);
> > THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
> > }
> >
> > So it's missing the barriers which pthread_mutex_lock() has. Grasping for
> > straws obviously....
Looks more like a solid tree than a straw now. :)
> Excellent! Taking a look into the disassembly of nptl/pthread_mutex_trylock.o
> reveals this part:
>
> 140: a5 1b 00 01 oill %r1,1
> 144: e5 48 a0 f0 00 00 mvghi 240(%r10),0 <--- THREAD_SETMEM (THREAD_SELF, robust_head.list_op_pending, NULL);
> 14a: e3 10 a0 e0 00 24 stg %r1,224(%r10) <--- last THREAD_SETMEM of ENQUEUE_MUTEX_PI
Awesome.
> I added a barrier between those two and now the code looks like this:
>
> 140: a5 1b 00 01 oill %r1,1
> 144: e3 10 a0 e0 00 24 stg %r1,224(%r10)
> 14a: e5 48 a0 f0 00 00 mvghi 240(%r10),0
>
> Looks like this was a one instruction race...
Fun. JFYI, I said that I reversed the stores in glibc and on my x86 test VM
it took more than _3_ days to trigger. But the good news is, that the trace
looks exactly like the ones you provided. So it looks we are on the right
track.
> I'll try to reproduce with the patch below (sprinkling compiler
> barriers just like the other files have).
Looks about right.
Thanks,
tglx
next prev parent reply other threads:[~2019-02-03 16:30 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-27 8:11 WARN_ON_ONCE(!new_owner) within wake_futex_pi() triggered Heiko Carstens
2018-11-28 14:32 ` Thomas Gleixner
2018-11-29 11:23 ` Heiko Carstens
2019-01-21 12:21 ` Heiko Carstens
2019-01-21 13:12 ` Thomas Gleixner
2019-01-22 21:14 ` Thomas Gleixner
2019-01-23 9:24 ` Heiko Carstens
2019-01-23 12:33 ` Thomas Gleixner
2019-01-23 12:40 ` Heiko Carstens
2019-01-28 13:44 ` Peter Zijlstra
2019-01-28 13:58 ` Peter Zijlstra
2019-01-28 15:53 ` Thomas Gleixner
2019-01-29 8:49 ` Peter Zijlstra
2019-01-29 22:15 ` [PATCH] futex: Handle early deadlock return correctly Thomas Gleixner
2019-01-30 12:01 ` Thomas Gleixner
2019-02-08 12:05 ` [tip:locking/urgent] " tip-bot for Thomas Gleixner
2019-01-29 9:01 ` WARN_ON_ONCE(!new_owner) within wake_futex_pi() triggered Heiko Carstens
2019-01-29 9:33 ` Peter Zijlstra
2019-01-29 9:45 ` Thomas Gleixner
2019-01-29 10:24 ` Heiko Carstens
2019-01-29 10:35 ` Peter Zijlstra
2019-01-29 13:03 ` Thomas Gleixner
2019-01-29 13:23 ` Heiko Carstens
[not found] ` <20190129151058.GG26906@osiris>
2019-01-29 17:16 ` Sebastian Sewior
2019-01-29 21:45 ` Thomas Gleixner
[not found] ` <20190130094913.GC5299@osiris>
2019-01-30 12:15 ` Thomas Gleixner
[not found] ` <20190130125955.GD5299@osiris>
2019-01-30 13:24 ` Sebastian Sewior
2019-01-30 13:29 ` Thomas Gleixner
2019-01-30 14:33 ` Thomas Gleixner
2019-01-30 17:56 ` Thomas Gleixner
2019-01-30 21:07 ` Sebastian Sewior
2019-01-30 23:13 ` WARN_ON_ONCE(!new_owner) within wake_futex_pi() triggerede Thomas Gleixner
2019-01-30 23:35 ` Paul E. McKenney
2019-01-30 23:55 ` Thomas Gleixner
2019-01-31 0:27 ` Thomas Gleixner
2019-01-31 1:45 ` Paul E. McKenney
2019-01-31 16:52 ` Heiko Carstens
2019-01-31 17:06 ` Sebastian Sewior
2019-01-31 20:42 ` Heiko Carstens
2019-02-01 16:12 ` Heiko Carstens
2019-02-01 21:59 ` Thomas Gleixner
[not found] ` <20190202091043.GA3381@osiris>
2019-02-02 10:14 ` Thomas Gleixner
2019-02-02 11:20 ` Heiko Carstens
2019-02-03 16:30 ` Thomas Gleixner [this message]
2019-02-04 11:40 ` Heiko Carstens
2019-01-31 1:44 ` Paul E. McKenney
2019-01-30 13:25 ` WARN_ON_ONCE(!new_owner) within wake_futex_pi() triggered Thomas Gleixner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.21.1902031718170.8200@nanos.tec.linutronix.de \
--to=tglx@linutronix.de \
--cc=bigeasy@linutronix.de \
--cc=heiko.carstens@de.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=paulmck@linux.ibm.com \
--cc=peterz@infradead.org \
--cc=schwidefsky@de.ibm.com \
--cc=stli@linux.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).